# Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study

Antti Kallonen, Milla Juutinen, Alpo Varri, Guy Carrault, Patrick Pladys, Alain Beuchée

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Antti Kallonen, Milla Juutinen, Alpo Varri, Guy Carrault, Patrick Pladys, et al.. Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study. International Journal of Medical Informatics, 2024, International Journal of Medical Informatics, 184, pp.105366. 10.1016/j.ijmedinf.2024.105366. hal-04477465

HAL Id: hal-04477465

https://hal.science/hal-04477465

Submitted on 24 May 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study

Antti Kallonen, Milla Juutinen, Alpo Varri, Guy Carrault, Patrick Pladys, Alain Beuchée

► To cite this version:

Antti Kallonen, Milla Juutinen, Alpo Varri, Guy Carrault, Patrick Pladys, et al.. Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study. International Journal of Medical Informatics, 2024, International Journal of Medical Informatics, 184, pp.105366. 10.1016/j.ijmedinf.2024.105366. hal-04477465

HAL Id: hal-04477465

https://hal.science/hal-04477465v1

Submitted on 24 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study

Antti Kallonen<sup>a, *</sup>, Milla Juutinen<sup>a</sup>, Alpo Värri<sup>a</sup>, Guy Carrault<sup>b</sup>, Patrick Pladys<sup>b, c</sup>, Alain Beuchée<sup>b, c</sup>

<figure><img src="image_4.png" /></figure>

## ARTICLE INFO

**Keywords:**

Biosignal processing

Deep learning model

Disease classification

Decision support system

Sepsis

## ABSTRACT

**Background:** Neonatal sepsis is responsible for significant morbidity and mortality worldwide. Its accurate and timely diagnosis is hindered by vague symptoms and the urgent necessity for early antibiotic intervention. The gold standard for diagnosing the condition is the identification of a pathogenic organism from normally sterile sites via laboratory testing. However, this method is resource-intensive and cannot be conducted continuously.

**Objective:** This study aimed to predict the onset of late-onset sepsis (LOS) with good diagnostic value as early as possible using non-invasive biosignal measurements from neonatal intensive care unit (NICU) monitors.

**Methods:** In this prospective multicenter study, we developed a multimodal machine learning algorithm based on a convolutional neural network (CNN) structure that uses the power spectral density (PSD) of recorded biosignals to predict the onset of LOS. This approach aimed to discern LOS-related pathogenic spectral signatures without labor-intensive manual artifact removal.

**Results:** The model achieved an area under the receiver operating characteristic score of 0.810 (95 % CI 0.698–0.922) on the validation dataset. With an optimal operating point, LOS detection had 83 % sensitivity and 73 % specificity. The median early detection was 44 h before clinical suspicion. The results highlighted the additive importance of electrocardiogram and respiratory impedance (RESP) signals in improving predictive accuracy. According to a more detailed analysis, the predictive power arose from the morphology of the electrocardiogram’s R-wave and sudden changes in the RESP signal.

**Conclusion:** Raw biosignals from NICU monitors, in conjunction with PSD transformation, as input to the CNN, can provide state-of-the-art prediction performance for LOS without the need for artifact removal. To the knowledge of the authors, this is the first study to highlight the independent and additive predictive potential of electrocardiogram R-wave morphology and concurrent, sudden changes in the RESP waveform in predicting the onset of LOS using non-invasive biosignals.

## 1. Introduction

Neonatal sepsis is responsible for significant morbidity and mortality worldwide, with an estimated global incidence of 2,202 per 100,000 live births and a global case fatality rate of between 11 % and 19 % [1]. It may also result in significant long-term morbidity for survivors, such as neurodevelopmental disorders, cerebral palsy, and hearing impairments [2]. The condition is characterized by systemic infection in the first

months of life [3]. It can be categorized as either early-onset sepsis (EOS), which occurs within the first 48–72 h of life, or late-onset sepsis (LOS), which occurs after the first 48–72 h of life, and these two conditions are caused by different pathogens [4]. In general, EOS is caused by pathogens transmitted from the maternal genital tract during birth [5]. By contrast, LOS is caused by pathogens acquired from the hospital environment [6]. Both LOS and EOS are treatable with the timely administration of antibiotics [4]. However, the accurate and timely diagnosis of sepsis is currently challenging because the condition

<footnote><sup>a</sup> Faculty of Medicine and Health Technology, Tampere University, FI-33014, Tampere, Finland</footnote>

<footnote><sup>b</sup> Inserm, LTSI - UMR 1099, University of Rennes, F-35000, Rennes, France</footnote>

<footnote><sup>c</sup> Pediatric Department, CHU Rennes, F-35000, Rennes, France</footnote>

<footnote>* Corresponding author at: Faculty of Medicine and Health Technology, Tampere University, Finland.</footnote>

<footnote>E-mail addresses: antti.kallonen@tuni.fi (A. Kallonen), milla.juutinen@tuni.fi (M. Juutinen), alpo.varri@tuni.fi (A. Värri), guy.carrault@univ-rennes1.fr (G. Carrault), patrick.pladys@chu-rennes.fr (P. Pladys), alain.beuchee@chu-rennes.fr (A. Beuchée).</footnote>
<figure><img src="image_2.png" /></figure>

# Early detection of late-onset neonatal sepsis from noninvasive biosignals using deep learning: A multicenter prospective development and validation study

Antti Kallonen<sup>a, *</sup>, Milla Juutinen<sup>a</sup>, Alpo Värri<sup>a</sup>, Guy Carrault<sup>b</sup>, Patrick Pladys<sup>b, c</sup>, Alain Beuchée<sup>b, c</sup>

<figure><img src="image_3.png" /></figure>

## ARTICLE INFO

**Keywords:**

Biosignal processing

Deep learning model

Disease classification

Decision support system

Sepsis

## ABSTRACT

**Background:** Neonatal sepsis is responsible for significant morbidity and mortality worldwide. Its accurate and timely diagnosis is hindered by vague symptoms and the urgent necessity for early antibiotic intervention. The gold standard for diagnosing the condition is the identification of a pathogenic organism from normally sterile sites via laboratory testing. However, this method is resource-intensive and cannot be conducted continuously.

**Objective:** This study aimed to predict the onset of late-onset sepsis (LOS) with good diagnostic value as early as possible using non-invasive biosignal measurements from neonatal intensive care unit (NICU) monitors.

**Methods:** In this prospective multicenter study, we developed a multimodal machine learning algorithm based on a convolutional neural network (CNN) structure that uses the power spectral density (PSD) of recorded biosignals to predict the onset of LOS. This approach aimed to discern LOS-related pathogenic spectral signatures without labor-intensive manual artifact removal.

**Results:** The model achieved an area under the receiver operating characteristic score of 0.810 (95 % CI 0.698–0.922) on the validation dataset. With an optimal operating point, LOS detection had 83 % sensitivity and 73 % specificity. The median early detection was 44 h before clinical suspicion. The results highlighted the additive importance of electrocardiogram and respiratory impedance (RESP) signals in improving predictive accuracy. According to a more detailed analysis, the predictive power arose from the morphology of the electrocardiogram’s R-wave and sudden changes in the RESP signal.

**Conclusion:** Raw biosignals from NICU monitors, in conjunction with PSD transformation, as input to the CNN, can provide state-of-the-art prediction performance for LOS without the need for artifact removal. To the knowledge of the authors, this is the first study to highlight the independent and additive predictive potential of electrocardiogram R-wave morphology and concurrent, sudden changes in the RESP waveform in predicting the onset of LOS using non-invasive biosignals.

## 1. Introduction

Neonatal sepsis is responsible for significant morbidity and mortality worldwide, with an estimated global incidence of 2,202 per 100,000 live births and a global case fatality rate of between 11 % and 19 % [1]. It may also result in significant long-term morbidity for survivors, such as neurodevelopmental disorders, cerebral palsy, and hearing impairments [2]. The condition is characterized by systemic infection in the first

months of life [3]. It can be categorized as either early-onset sepsis (EOS), which occurs within the first 48–72 h of life, or late-onset sepsis (LOS), which occurs after the first 48–72 h of life, and these two conditions are caused by different pathogens [4]. In general, EOS is caused by pathogens transmitted from the maternal genital tract during birth [5]. By contrast, LOS is caused by pathogens acquired from the hospital environment [6]. Both LOS and EOS are treatable with the timely administration of antibiotics [4]. However, the accurate and timely diagnosis of sepsis is currently challenging because the condition

<footnote><sup>a</sup> Faculty of Medicine and Health Technology, Tampere University, FI-33014, Tampere, Finland</footnote>

<footnote><sup>b</sup> Inserm, LTSI - UMR 1099, University of Rennes, F-35000, Rennes, France</footnote>

<footnote><sup>c</sup> Pediatric Department, CHU Rennes, F-35000, Rennes, France</footnote>

<footnote>* Corresponding author at: Faculty of Medicine and Health Technology, Tampere University, Finland.</footnote>

<footnote>E-mail addresses: antti.kallonen@tuni.fi (A. Kallonen), milla.juutinen@tuni.fi (M. Juutinen), alpo.varri@tuni.fi (A. Värri), guy.carrault@univ-rennes1.fr (G. Carrault), patrick.pladys@chu-rennes.fr (P. Pladys), alain.beuchee@chu-rennes.fr (A. Beuchée).</footnote>
Nomenclature

Acronym Description

<table><tr><td>AUROC</td><td>Area Under the Receiver Operating Characteristic</td></tr><tr><td>CI</td><td>Confidence Interval</td></tr><tr><td>CNN</td><td>Convolutional Neural Network</td></tr><tr><td>CPP</td><td>Committee for the Protection of Persons</td></tr><tr><td>CRP</td><td>C-Reactive Protein</td></tr><tr><td>ECG</td><td>Electrocardiogram</td></tr><tr><td>eCRF</td><td>electronic Case Report Form</td></tr><tr><td>EEG</td><td>Electroencephalography</td></tr><tr><td>EHR</td><td>Electronic Health Record</td></tr><tr><td>EOS</td><td>Early-Onset Neonatal Sepsis</td></tr><tr><td>FFT</td><td>Fast Fourier Transform</td></tr></table>

<table><tr><td>HRV</td><td>Heart Rate Variability</td></tr><tr><td>IQR</td><td>Interquartile Range</td></tr><tr><td>LOS</td><td>Late-Onset Neonatal Sepsis</td></tr><tr><td>MLA</td><td>Machine Learning Algorithm</td></tr><tr><td>NEO-KISS</td><td>Neonatal Nosocomial Infection Surveillance System</td></tr><tr><td>NICU</td><td>Neonatal Intensive Care Unit</td></tr><tr><td>PCT</td><td>Procalcitonin</td></tr><tr><td>PLETH</td><td>Photoplethysmography Signal</td></tr><tr><td>PSD</td><td>Power Spectral Density</td></tr><tr><td>QRS</td><td>QRS Complex in the ECG Waveform</td></tr><tr><td>R-Wave</td><td>R-Wave Component in the ECG Waveform</td></tr><tr><td>RESP</td><td>Respiration Impedance Signal</td></tr><tr><td>RGB</td><td>Red Green Blue</td></tr><tr><td>STFT</td><td>Short-Time Fourier Transform</td></tr></table>

presents with non-specific clinical features, such as lethargy and respiratory distress, which can be caused by clinical conditions other than infection [7]. The administration of antibiotics is also a balancing act because unnecessary antibiotic use can complicate the clinical course of the patient. It is currently estimated that 31 % of annual sepsis-related neonatal deaths are attributable to antibiotic resistance [8].

The gold standard in the diagnosis of LOS and EOS caused by a pathogen is the identification of a pathogenic organism from normally sterile sites, such as blood or cerebrospinal fluid [4]. This is, however, very resource intensive because the pathogen must be cultured in a laboratory. Therefore, additional biomarkers, such as procalcitonin (PCT) or C-reactive protein (CRP), have been used to detect the condition as early as possible [9]. Blood-borne biomarkers have the drawback of being invasive, and they cannot be continuously evaluated.

In this study, we explored the predictive potential of raw, non-invasive biosignals recorded directly from a neonatal intensive care unit (NICU) monitor in the context of predicting the onset of LOS with good diagnostic value as early as possible with a convolutional neural network (CNN). CNN is a supervised machine learning algorithm (MLA) comprised of artificial neurons that learn to extract common features presented to them in relation to the target variable the MLA is trained to predict. This MLA is inspired by biological processes because its connectivity pattern resembles the organization of neurons in an animal's visual cortex [10,11]. CNNs are also capable of handling noisy input data, particularly when trained with multiple degraded images [12]. It has also been shown that CNN performance is not affected by artifacts in EEG data [13].

Numerous studies have concentrated on creating MLAs for the identification of LOS through machine learning techniques. The majority of these studies employ MLAs such as naive Bayes, random forests, and logistic regression on structured tabular data [14,15]. The approaches can be divided into two categories: invasive models, which use laboratory measurements such as C-reactive protein, white blood cell count, urinalysis, absolute neutrophil count, and procalcitonin, along with clinical health record data [16–20]; and non-invasive approaches, which utilize vital measurements derived from monitoring units, primarily focusing on heart rate characteristics, such as heart rate variability (HRV) [21–24].

Our method distinguishes itself by utilizing raw noninvasive biosignals with CNN architecture. In contrast, the most comparable non-invasive deep learning methods use models on derived HRV features, rather than processing the raw biosignals directly [22,23]. Our model is also multimodal in that it integrates information from electrocardiogram (ECG), respiratory impedance (RESP), and photoplethysmography (PLETH) signals into one model and can operate without artifact removal. The interplay between biosignal components, such as ECG R-wave and RESP, has been previously explored in the context of predicting LOS using machine learning [21,24]. However, these

multimodal studies also use computed features, such as R–R intervals or respiratory rates. There is a research gap in examining the predictive value of the information content of raw biosignals in the early diagnosis of LOS [25]. Current practices often involve preprocessing steps, such as HRV calculation, which may inadvertently discard predictive information related to the onset of LOS as the ECG waveform morphology contains changes during sepsis that may have additional predictive value [26].

## 2. Materials and methods

### 2.1. Overview of approach

Our MLA uses raw biosignals recorded from NICU monitors. The recorded monitor signals were divided into 30-minute temporally aligned segments by the recording system and stored in corresponding data files. These segmented temporal signals were further processed with a 2D power spectral density (PSD) implemented with a short-time Fourier transform (STFT) function to capture the morphology of each biosignal in a frequency–time matrix [27]. The combined 2D PSD and CNN approach is motivated by CNN performance in the speech recognition domain, where it is proven that CNN is able to interpret 2D spectral domain features related to a learning target [28–30]. We did not perform any artifact removal prior to introducing the signals into the processing pipeline. This approach leverages the proven effectiveness of CNNs in EEG analysis, demonstrating that a CNN is resilient to noise within biosignals. Additionally, this approach removes the need for artifact removal at the biosignal level, which is typically labor-intensive. Such an undertaking can often present a prohibitive cost and obstacle, thereby impeding the utilization of sophisticated MLAs as decision-support tools in NICUs. The rationale is that the FFT transformation moves the signal information content to a frequency domain, where the artifact is presented as a limited, line-like disturbance in the 2D spectral image. We did not incorporate any additional demographic inputs into the MLA for other critical parameters, such as gestational age. This mitigates the risk of the MLA overfitting to specific demographic variables, particularly when multiple biosignal segments originate from a single patient. The concern is that the model could overfit these demographic characteristics instead of detecting subtle signs of LOS within the PSDs. To prevent the overfitting of features specific to a patient or monitoring device, the PSDs are intentionally augmented during the training process. Our hypothesis is that the proposed MLA possesses the intrinsic ability to discern these confounding elements from the spectral characteristics of the input biosignals. Specifically, even though factors such as gestational age can influence neonatal ECG waveforms, we anticipate that the MLA will effectively differentiate pathological spectral features from normal biosignals. This expectation is grounded in the diverse representation in our development data, which encompasses
Nomenclature

Acronym Description

<table><tr><td>AUROC</td><td>Area Under the Receiver Operating Characteristic</td></tr><tr><td>CI</td><td>Confidence Interval</td></tr><tr><td>CNN</td><td>Convolutional Neural Network</td></tr><tr><td>CPP</td><td>Committee for the Protection of Persons</td></tr><tr><td>CRP</td><td>C-Reactive Protein</td></tr><tr><td>ECG</td><td>Electrocardiogram</td></tr><tr><td>eCRF</td><td>electronic Case Report Form</td></tr><tr><td>EEG</td><td>Electroencephalography</td></tr><tr><td>EHR</td><td>Electronic Health Record</td></tr><tr><td>EOS</td><td>Early-Onset Neonatal Sepsis</td></tr><tr><td>FFT</td><td>Fast Fourier Transform</td></tr></table>

<table><tr><td>HRV</td><td>Heart Rate Variability</td></tr><tr><td>IQR</td><td>Interquartile Range</td></tr><tr><td>LOS</td><td>Late-Onset Neonatal Sepsis</td></tr><tr><td>MLA</td><td>Machine Learning Algorithm</td></tr><tr><td>NEO-KISS</td><td>Neonatal Nosocomial Infection Surveillance System</td></tr><tr><td>NICU</td><td>Neonatal Intensive Care Unit</td></tr><tr><td>PCT</td><td>Procalcitonin</td></tr><tr><td>PLETH</td><td>Photoplethysmography Signal</td></tr><tr><td>PSD</td><td>Power Spectral Density</td></tr><tr><td>QRS</td><td>QRS Complex in the ECG Waveform</td></tr><tr><td>R-Wave</td><td>R-Wave Component in the ECG Waveform</td></tr><tr><td>RESP</td><td>Respiration Impedance Signal</td></tr><tr><td>RGB</td><td>Red Green Blue</td></tr><tr><td>STFT</td><td>Short-Time Fourier Transform</td></tr></table>

presents with non-specific clinical features, such as lethargy and respiratory distress, which can be caused by clinical conditions other than infection [7]. The administration of antibiotics is also a balancing act because unnecessary antibiotic use can complicate the clinical course of the patient. It is currently estimated that 31 % of annual sepsis-related neonatal deaths are attributable to antibiotic resistance [8].

The gold standard in the diagnosis of LOS and EOS caused by a pathogen is the identification of a pathogenic organism from normally sterile sites, such as blood or cerebrospinal fluid [4]. This is, however, very resource intensive because the pathogen must be cultured in a laboratory. Therefore, additional biomarkers, such as procalcitonin (PCT) or C-reactive protein (CRP), have been used to detect the condition as early as possible [9]. Blood-borne biomarkers have the drawback of being invasive, and they cannot be continuously evaluated.

In this study, we explored the predictive potential of raw, non-invasive biosignals recorded directly from a neonatal intensive care unit (NICU) monitor in the context of predicting the onset of LOS with good diagnostic value as early as possible with a convolutional neural network (CNN). CNN is a supervised machine learning algorithm (MLA) comprised of artificial neurons that learn to extract common features presented to them in relation to the target variable the MLA is trained to predict. This MLA is inspired by biological processes because its connectivity pattern resembles the organization of neurons in an animal's visual cortex [10,11]. CNNs are also capable of handling noisy input data, particularly when trained with multiple degraded images [12]. It has also been shown that CNN performance is not affected by artifacts in EEG data [13].

Numerous studies have concentrated on creating MLAs for the identification of LOS through machine learning techniques. The majority of these studies employ MLAs such as naive Bayes, random forests, and logistic regression on structured tabular data [14,15]. The approaches can be divided into two categories: invasive models, which use laboratory measurements such as C-reactive protein, white blood cell count, urinalysis, absolute neutrophil count, and procalcitonin, along with clinical health record data [16–20]; and non-invasive approaches, which utilize vital measurements derived from monitoring units, primarily focusing on heart rate characteristics, such as heart rate variability (HRV) [21–24].

Our method distinguishes itself by utilizing raw noninvasive biosignals with CNN architecture. In contrast, the most comparable non-invasive deep learning methods use models on derived HRV features, rather than processing the raw biosignals directly [22,23]. Our model is also multimodal in that it integrates information from electrocardiogram (ECG), respiratory impedance (RESP), and photoplethysmography (PLETH) signals into one model and can operate without artifact removal. The interplay between biosignal components, such as ECG R-wave and RESP, has been previously explored in the context of predicting LOS using machine learning [21,24]. However, these

multimodal studies also use computed features, such as R–R intervals or respiratory rates. There is a research gap in examining the predictive value of the information content of raw biosignals in the early diagnosis of LOS [25]. Current practices often involve preprocessing steps, such as HRV calculation, which may inadvertently discard predictive information related to the onset of LOS as the ECG waveform morphology contains changes during sepsis that may have additional predictive value [26].

## 2. Materials and methods

### 2.1. Overview of approach

Our MLA uses raw biosignals recorded from NICU monitors. The recorded monitor signals were divided into 30-minute temporally aligned segments by the recording system and stored in corresponding data files. These segmented temporal signals were further processed with a 2D power spectral density (PSD) implemented with a short-time Fourier transform (STFT) function to capture the morphology of each biosignal in a frequency–time matrix [27]. The combined 2D PSD and CNN approach is motivated by CNN performance in the speech recognition domain, where it is proven that CNN is able to interpret 2D spectral domain features related to a learning target [28–30]. We did not perform any artifact removal prior to introducing the signals into the processing pipeline. This approach leverages the proven effectiveness of CNNs in EEG analysis, demonstrating that a CNN is resilient to noise within biosignals. Additionally, this approach removes the need for artifact removal at the biosignal level, which is typically labor-intensive. Such an undertaking can often present a prohibitive cost and obstacle, thereby impeding the utilization of sophisticated MLAs as decision-support tools in NICUs. The rationale is that the FFT transformation moves the signal information content to a frequency domain, where the artifact is presented as a limited, line-like disturbance in the 2D spectral image. We did not incorporate any additional demographic inputs into the MLA for other critical parameters, such as gestational age. This mitigates the risk of the MLA overfitting to specific demographic variables, particularly when multiple biosignal segments originate from a single patient. The concern is that the model could overfit these demographic characteristics instead of detecting subtle signs of LOS within the PSDs. To prevent the overfitting of features specific to a patient or monitoring device, the PSDs are intentionally augmented during the training process. Our hypothesis is that the proposed MLA possesses the intrinsic ability to discern these confounding elements from the spectral characteristics of the input biosignals. Specifically, even though factors such as gestational age can influence neonatal ECG waveforms, we anticipate that the MLA will effectively differentiate pathological spectral features from normal biosignals. This expectation is grounded in the diverse representation in our development data, which encompasses
biosignals from neonates across various gestational ages, both with and without potential indications of LOS. To substantiate this assertion, we established a stratified validation cohort in our study, which was controlled for the most significant confounding factors. To increase the signal-to-noise ratio and magnify the biological signal content of the spectra, the biosignals were oversampled in the STFT function. The resulting imaginary and real matrices were then scaled, interpolated to similar dimensions, and fed into a CNN in a concatenated multi-dimensional matrix, similar to the way in which color channels are fed into a CNN when using RGB images as input.

## 2.2. Population

The study employed a prospective cohort design, including preterm infants born before 34 weeks of gestation. Patients were eligible for inclusion if they were hospitalized in one of the NICUs of six university hospitals in the western region of France (university hospitals of Rennes, Angers, Nantes, Brest, Poitiers, and Tours) in 2017–2019. Each of these NICUs is part of a larger university hospital, which ensures a high standard of care. The selection of these hospitals was based on their ability to provide a wide range of neonatal services and their participation in academic research, making them suitable environments for our study. The western region of France is characterized by a diverse population and healthcare needs, which allowed us to capture a broad spectrum of LOS cases. This diversity also contributes to the generalizability of our findings. The NICUs in these hospitals are designed to handle complex cases and provide comprehensive neonatal care, including critical care for preterm and full-term infants with various medical conditions. The study protocol followed the principles of the Declaration of Helsinki, and all experimental protocols were approved by the French national agency for the safety of medicines and health products (Authorization number: 2016062400181) and the CHU hospital district ethics committee (CPP Ouest IV 34/16) with informed parental consent.

The LOS group comprised patients with at least one clinical suspicion of LOS, that is, meeting NEO-KISS criteria for clinical sepsis or laboratory-confirmed bloodstream infection with a temporally locatable antibiotic administration after 72 h of life. The inclusion criteria for this study are broad, reflecting our aim to investigate the feasibility of developing a universally applicable non-invasive infection monitoring system for diverse NICU populations. The ground truth of LOS labels remains elusive, as there is low international and moderate inter-observer agreement on the LOS diagnosis [31]. The NEO-KISS protocol outlines specific criteria for identifying late-onset sepsis in neonates that rely on both clinical observations and laboratory test results. To diagnose clinical sepsis when there is no laboratory confirmation of a pathogen, the protocol requires that the infant must have been treated with appropriate antimicrobial therapy for a minimum of five days, show no growth of pathogens in blood cultures, and exhibit no signs of infection at any other bodily site. Additionally, the infant must present with at least two clinical indicators, such as fever, instability in body temperature, rapid heart rate, or episodes of apnea. In cases where a bloodstream infection is confirmed through laboratory tests, the criteria necessitate the detection of a pathogenic organism in the blood culture that is not a coagulase-negative staphylococcus, coupled with at least two of the aforementioned clinical signs [32]. The comparison group consisted of patients who did not exhibit any suspicion of late-onset sepsis as defined by the NEO-KISS criteria. This criterion was selected to standardize the LOS annotation protocol across the centers.

For validation purposes, a nested case-control design was implemented within this cohort. Using cardinality matching, 30 infants from the LOS group were matched with 30 controls based on gestational age, birth weight, and twin pregnancy status. The primary objective was to account for the influence of maturation on biosignals, acknowledging that extremely premature infants have a higher susceptibility to LOS, and that maturation itself alters the characteristics of biosignal

waveforms, including the ECG. Consequently, we established that the MLA would not mistake the biological variations associated with maturation for indicators of LOS. Unmatched infants were included in the development dataset.

## 2.3. Data sources

Continuous ECG, photoplethysmography (PLETH), and RESP signals were collected from the NICU monitors for each of the study patients during their stays. Data were collected directly from the Philips Intelligence neonatal monitor interface in the NICU. This interface provides raw biosignals for one ECG channel, the PLETH waveform, and the RESP waveform. All of these were used in the model development to give the model all available biosignals to learn specific patterns that indicated the presence of LOS. The ECG channel output is equal to the channel being visualized in the monitor user interface. The ECG waveform was filtered for powerline interference in the monitor device. The sampling frequencies were 500 Hz for ECG, 125 Hz for PLETH, and 160 Hz for RESP. Biosignal data were recorded from the monitor serial port using a Raspberry Pi single-board computer and stored in the device's memory card [33]. As the biosignal data were recorded directly from the NICU monitor, and all the centers had the same monitor model, the data quality should be equal. The recordings were connected to patient-specific clinical information obtained from the hospital EHR database and stored in a research database after anonymization for analysis. Clinical data were recorded using an electronic case report form (eCRF) within the hospital information system, which was accessible to the research team and maintained in compliance with data protection regulations to ensure patient confidentiality and data integrity.

## 2.4. Data pipeline

The data pipeline shown in Fig. 1A outlines the patient cohorts and processing steps utilized in the development and validation of the LOS model. The objective of the model is binary classification, aimed at determining whether each time segment in a patient's NICU stay timeline is indicative of LOS. The development and validation cohorts were divided based on patient identifiers to prevent any data leakage between the datasets. Further, no normalization or standardization of the data was performed before splitting into development and validation datasets.

The training data labeling was aligned with the clinical suspicion timestamp, with biosignal segments from the 24-hour period before clinical suspicion marked as positive, and all others considered negative. An illustration of this labeling process is presented in Supplementary Figure 9. When the model infers data, it analyses individual biosignal segments without associating them with a specific patient. An illustration of the model inference is presented in Supplementary Figure 10. However, performance evaluation takes place at the population level to assess the model's predictive accuracy across the timelines of patients' NICU stays. This approach facilitates the efficient training of a deep learning model because each patient in the NICU, with an extended stay, contributes multiple data points to the LOS model. This means that even with a smaller patient population, the model can be trained effectively due to the abundant data available. Additionally, by analyzing a patient's entire NICU stay timeline, the model's capacity for early detection was accurately determined.

Training of the model was conducted over a predetermined number of epochs, utilizing the Adam optimization algorithm alongside binary cross entropy as the loss function. Before finalizing the model parameters, we conducted limited hyperparameter optimization using a grid search algorithm on a subset of the development data. The list of searched hyperparameters is shown in Supplementary Table 2. The optimization goal was to maximize the area under the receiver operating characteristic (AUROC) for LOS detection on the internal validation dataset. Additional details of the MLA implementation, architecture, and
biosignals from neonates across various gestational ages, both with and without potential indications of LOS. To substantiate this assertion, we established a stratified validation cohort in our study, which was controlled for the most significant confounding factors. To increase the signal-to-noise ratio and magnify the biological signal content of the spectra, the biosignals were oversampled in the STFT function. The resulting imaginary and real matrices were then scaled, interpolated to similar dimensions, and fed into a CNN in a concatenated multi-dimensional matrix, similar to the way in which color channels are fed into a CNN when using RGB images as input.

## 2.2. Population

The study employed a prospective cohort design, including preterm infants born before 34 weeks of gestation. Patients were eligible for inclusion if they were hospitalized in one of the NICUs of six university hospitals in the western region of France (university hospitals of Rennes, Angers, Nantes, Brest, Poitiers, and Tours) in 2017–2019. Each of these NICUs is part of a larger university hospital, which ensures a high standard of care. The selection of these hospitals was based on their ability to provide a wide range of neonatal services and their participation in academic research, making them suitable environments for our study. The western region of France is characterized by a diverse population and healthcare needs, which allowed us to capture a broad spectrum of LOS cases. This diversity also contributes to the generalizability of our findings. The NICUs in these hospitals are designed to handle complex cases and provide comprehensive neonatal care, including critical care for preterm and full-term infants with various medical conditions. The study protocol followed the principles of the Declaration of Helsinki, and all experimental protocols were approved by the French national agency for the safety of medicines and health products (Authorization number: 2016062400181) and the CHU hospital district ethics committee (CPP Ouest IV 34/16) with informed parental consent.

The LOS group comprised patients with at least one clinical suspicion of LOS, that is, meeting NEO-KISS criteria for clinical sepsis or laboratory-confirmed bloodstream infection with a temporally locatable antibiotic administration after 72 h of life. The inclusion criteria for this study are broad, reflecting our aim to investigate the feasibility of developing a universally applicable non-invasive infection monitoring system for diverse NICU populations. The ground truth of LOS labels remains elusive, as there is low international and moderate inter-observer agreement on the LOS diagnosis [31]. The NEO-KISS protocol outlines specific criteria for identifying late-onset sepsis in neonates that rely on both clinical observations and laboratory test results. To diagnose clinical sepsis when there is no laboratory confirmation of a pathogen, the protocol requires that the infant must have been treated with appropriate antimicrobial therapy for a minimum of five days, show no growth of pathogens in blood cultures, and exhibit no signs of infection at any other bodily site. Additionally, the infant must present with at least two clinical indicators, such as fever, instability in body temperature, rapid heart rate, or episodes of apnea. In cases where a bloodstream infection is confirmed through laboratory tests, the criteria necessitate the detection of a pathogenic organism in the blood culture that is not a coagulase-negative staphylococcus, coupled with at least two of the aforementioned clinical signs [32]. The comparison group consisted of patients who did not exhibit any suspicion of late-onset sepsis as defined by the NEO-KISS criteria. This criterion was selected to standardize the LOS annotation protocol across the centers.

For validation purposes, a nested case-control design was implemented within this cohort. Using cardinality matching, 30 infants from the LOS group were matched with 30 controls based on gestational age, birth weight, and twin pregnancy status. The primary objective was to account for the influence of maturation on biosignals, acknowledging that extremely premature infants have a higher susceptibility to LOS, and that maturation itself alters the characteristics of biosignal

waveforms, including the ECG. Consequently, we established that the MLA would not mistake the biological variations associated with maturation for indicators of LOS. Unmatched infants were included in the development dataset.

## 2.3. Data sources

Continuous ECG, photoplethysmography (PLETH), and RESP signals were collected from the NICU monitors for each of the study patients during their stays. Data were collected directly from the Philips Intelligence neonatal monitor interface in the NICU. This interface provides raw biosignals for one ECG channel, the PLETH waveform, and the RESP waveform. All of these were used in the model development to give the model all available biosignals to learn specific patterns that indicated the presence of LOS. The ECG channel output is equal to the channel being visualized in the monitor user interface. The ECG waveform was filtered for powerline interference in the monitor device. The sampling frequencies were 500 Hz for ECG, 125 Hz for PLETH, and 160 Hz for RESP. Biosignal data were recorded from the monitor serial port using a Raspberry Pi single-board computer and stored in the device's memory card [33]. As the biosignal data were recorded directly from the NICU monitor, and all the centers had the same monitor model, the data quality should be equal. The recordings were connected to patient-specific clinical information obtained from the hospital EHR database and stored in a research database after anonymization for analysis. Clinical data were recorded using an electronic case report form (eCRF) within the hospital information system, which was accessible to the research team and maintained in compliance with data protection regulations to ensure patient confidentiality and data integrity.

## 2.4. Data pipeline

The data pipeline shown in Fig. 1A outlines the patient cohorts and processing steps utilized in the development and validation of the LOS model. The objective of the model is binary classification, aimed at determining whether each time segment in a patient's NICU stay timeline is indicative of LOS. The development and validation cohorts were divided based on patient identifiers to prevent any data leakage between the datasets. Further, no normalization or standardization of the data was performed before splitting into development and validation datasets.

The training data labeling was aligned with the clinical suspicion timestamp, with biosignal segments from the 24-hour period before clinical suspicion marked as positive, and all others considered negative. An illustration of this labeling process is presented in Supplementary Figure 9. When the model infers data, it analyses individual biosignal segments without associating them with a specific patient. An illustration of the model inference is presented in Supplementary Figure 10. However, performance evaluation takes place at the population level to assess the model's predictive accuracy across the timelines of patients' NICU stays. This approach facilitates the efficient training of a deep learning model because each patient in the NICU, with an extended stay, contributes multiple data points to the LOS model. This means that even with a smaller patient population, the model can be trained effectively due to the abundant data available. Additionally, by analyzing a patient's entire NICU stay timeline, the model's capacity for early detection was accurately determined.

Training of the model was conducted over a predetermined number of epochs, utilizing the Adam optimization algorithm alongside binary cross entropy as the loss function. Before finalizing the model parameters, we conducted limited hyperparameter optimization using a grid search algorithm on a subset of the development data. The list of searched hyperparameters is shown in Supplementary Table 2. The optimization goal was to maximize the area under the receiver operating characteristic (AUROC) for LOS detection on the internal validation dataset. Additional details of the MLA implementation, architecture, and
A

<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Data pipeline and LOS model deployment to NICU workflow. A, Data pipeline used in the study. The illustration shows how the patient data are divided into development and validation datasets, the operations performed on the different cohorts, and the performance evaluation step where the trained model is evaluated with the validation data. B, LOS model deployment to the NICU workflow. The diagram illustrates the integration of the LOS model into the NICU workflow through a continuous monitoring loop. In this setup, biosignals are streamed in real time from patient monitors to the LOS model, which generates predictions. When a prediction surpasses a predefined threshold, an alert is triggered on the NICU central console, prompting clinicians to undertake appropriate actions to optimize patient care. Simultaneously, a model performance evaluation loop operates in tandem to consistently validate the effectiveness of the LOS model.</figcaption></figure>

PSD preprocessing can be found in the Supplemental Appendix.

## 2.5. Model performance evaluation and deployment to NICU

To be clinically useful in an NICU setting, the LOS model must be able to detect the condition in a population with good diagnostic value. The model must also be able to detect the condition as early as possible to facilitate early intervention with antibiotics. Therefore, the main evaluation metric was defined as taking the highest output value predicted during the entire NICU stay of a patient and then comparing the highest value to a known class label. For an LOS patient, this means being in the positive class, and for a control patient, the negative class. The optimal MLA model would output a negative class during the entire NICU stay of a control patient and a positive class near antibiotic administration for an LOS patient. An illustration of this is provided in Fig. 1A. With this definition, it is possible to calculate a receiver operating characteristic (ROC) curve for use in detecting conditions in the validation dataset. This evaluation scheme reflects the real-world situation where the MLA would be deployed in a monitoring context, and always generates a new prediction value when there is more data available. This approach also highlights the specificity of the model—any high-output prediction during the stay of a control patient would result in a false positive. Moreover, this further explores the MLA's robustness to artifacts—that is, the MLA is trained with artifacts present in the development data, but it should not react to them when

presented with signals, including artifacts in the validation data.

The optimal operating point was extracted for the MLA during hyperparameter optimization on the development dataset by calculating Youden's index [34]. Using this operating point, the temporal relationship with the clinical suspicion of LOS can be calculated. The time difference to clinical suspicion was calculated as the difference between the first crossing of the operating point and the clinical suspicion timestamp. The aggregate statistics of this temporal difference provide an approximation of the early predictive potential of the proposed MLA compared to present clinical practice.

To investigate the interplay between the frequency and temporal domains, we conducted a dominant frequency analysis of a sampled biosignal waveform. This was done to more accurately characterize the temporal location of the identified pathogenic spectral components. The dominant frequency in the temporal waveforms was calculated by applying an STFT in a sliding window to obtain the frequency domain representation [35]. After that, the frequency corresponding to the highest magnitude in the selected frequency range was calculated, corresponding to the dominant frequency within the window.

Fig. 1B illustrates the integration of the proposed LOS detection model within the NICU workflow. According to this diagram, patient biosignals are continuously streamed into the LOS model, which generates real-time predictions. When these predictions surpass the operating point, an alert is triggered on the central console of the NICU. This alert prompts the clinical staff to examine the patient more closely using
A

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Data pipeline and LOS model deployment to NICU workflow. A, Data pipeline used in the study. The illustration shows how the patient data are divided into development and validation datasets, the operations performed on the different cohorts, and the performance evaluation step where the trained model is evaluated with the validation data. B, LOS model deployment to the NICU workflow. The diagram illustrates the integration of the LOS model into the NICU workflow through a continuous monitoring loop. In this setup, biosignals are streamed in real time from patient monitors to the LOS model, which generates predictions. When a prediction surpasses a predefined threshold, an alert is triggered on the NICU central console, prompting clinicians to undertake appropriate actions to optimize patient care. Simultaneously, a model performance evaluation loop operates in tandem to consistently validate the effectiveness of the LOS model.</figcaption></figure>

PSD preprocessing can be found in the Supplemental Appendix.

## 2.5. Model performance evaluation and deployment to NICU

To be clinically useful in an NICU setting, the LOS model must be able to detect the condition in a population with good diagnostic value. The model must also be able to detect the condition as early as possible to facilitate early intervention with antibiotics. Therefore, the main evaluation metric was defined as taking the highest output value predicted during the entire NICU stay of a patient and then comparing the highest value to a known class label. For an LOS patient, this means being in the positive class, and for a control patient, the negative class. The optimal MLA model would output a negative class during the entire NICU stay of a control patient and a positive class near antibiotic administration for an LOS patient. An illustration of this is provided in Fig. 1A. With this definition, it is possible to calculate a receiver operating characteristic (ROC) curve for use in detecting conditions in the validation dataset. This evaluation scheme reflects the real-world situation where the MLA would be deployed in a monitoring context, and always generates a new prediction value when there is more data available. This approach also highlights the specificity of the model—any high-output prediction during the stay of a control patient would result in a false positive. Moreover, this further explores the MLA’s robustness to artifacts—that is, the MLA is trained with artifacts present in the development data, but it should not react to them when

presented with signals, including artifacts in the validation data.

The optimal operating point was extracted for the MLA during hyperparameter optimization on the development dataset by calculating Youden's index [34]. Using this operating point, the temporal relationship with the clinical suspicion of LOS can be calculated. The time difference to clinical suspicion was calculated as the difference between the first crossing of the operating point and the clinical suspicion timestamp. The aggregate statistics of this temporal difference provide an approximation of the early predictive potential of the proposed MLA compared to present clinical practice.

To investigate the interplay between the frequency and temporal domains, we conducted a dominant frequency analysis of a sampled biosignal waveform. This was done to more accurately characterize the temporal location of the identified pathogenic spectral components. The dominant frequency in the temporal waveforms was calculated by applying an STFT in a sliding window to obtain the frequency domain representation [35]. After that, the frequency corresponding to the highest magnitude in the selected frequency range was calculated, corresponding to the dominant frequency within the window.

Fig. 1B illustrates the integration of the proposed LOS detection model within the NICU workflow. According to this diagram, patient biosignals are continuously streamed into the LOS model, which generates real-time predictions. When these predictions surpass the operating point, an alert is triggered on the central console of the NICU. This alert prompts the clinical staff to examine the patient more closely using
B

<figure><img src="image_6.png" /><figcaption>Fig. 1. *(continued)*.</figcaption></figure>

the monitoring software, interpret the model's explanatory information, and conduct further clinical assessments or interventions as needed. Concurrently, there should be an ongoing process of model validation in which the LOS model's performance is regularly assessed using both internal hospital data and potentially external datasets to ensure its accuracy and reliability. If the model's performance degrades, it necessitates retraining and updating the system with the improved model.

No imputation of data or outlier removal was conducted, as this process would involve estimating raw biosignal values, such as those from ECG recordings. The developed LOS model is temporally agnostic; it could predict the label for each signal segment independently; therefore, its performance was not expected to be affected by any gaps in the biosignal data. It is designed to predict labels for discrete biosignal segments without the need for continuous data input. During the training phase, the data are augmented with various perturbations to the computed PSDs to prevent overfitting. Additionally, the imbalance between positive and negative segments in the training data was addressed using a balancing sampler [36]. This technique ensured that each batch of PSDs was balanced, effectively representing both classes proportionately, even when positive signal segments were less frequent in the training dataset. As our MLA operates at the biosignal level, it encounters learning targets in the form of signal segments, which are not directly reflective of the patient population demographics, so the unbalance in the development dataset has minimal effect on the evaluated model results. Thus, a valid performance measure is the MLA's ability to accurately predict across all patient signal segments within the validation dataset. Subsequently, we evaluate the MLA's performance at the

population level on the matched validation dataset to ensure its effectiveness and reliability.

## 2.6. *Development and validation datasets*

The study cohort included in the analysis consisted of 309 patients. The population was divided into development and validation datasets. Patients with no biosignal data were discarded, as this indicates a technical failure of the recording system. The study profile graph is presented in *Supplementary Figure 2*. Using DeLong's method, we estimated that a validation dataset with a population of 30 LOS patients and 30 controls would result in a minimum detectable AUROC of 0.753, with a power of 0.95 and a significance of 0.05. A validation dataset was created by matching 30 LOS and control patients according to gestational age, birth weight, and twin pregnancy status to equalize the effect of the maturation of the cardiovascular system on the recorded biosignals (Table 1). The remainder of the patients were assigned to the development dataset, which consisted of 13 LOS cases and 190 controls (Table 2). In the validation dataset, none of the demographic variables or available comorbidities differed significantly between the LOS and control patients. The development dataset was unbalanced between LOS and control patients for multiple variables. The population imbalance was not mirrored in the model during the training phase. This is because the signal segments and targets were introduced to the LOS model using a balanced data sampling technique, and the training was conducted with individually labeled PSD segments.
B

<figure><img src="image_5.png" /><figcaption>Fig. 1. (continued).</figcaption></figure>

the monitoring software, interpret the model’s explanatory information, and conduct further clinical assessments or interventions as needed. Concurrently, there should be an ongoing process of model validation in which the LOS model’s performance is regularly assessed using both internal hospital data and potentially external datasets to ensure its accuracy and reliability. If the model’s performance degrades, it necessitates retraining and updating the system with the improved model.

No imputation of data or outlier removal was conducted, as this
process would involve estimating raw biosignal values, such as those
from ECG recordings. The developed LOS model is temporally agnostic;
it could predict the label for each signal segment independently;
therefore, its performance was not expected to be affected by any gaps in
the biosignal data. It is designed to predict labels for discrete biosignal
segments without the need for continuous data input. During the
training phase, the data are augmented with various perturbations to the
computed PSDs to prevent overfitting. Additionally, the imbalance be-
tween positive and negative segments in the training data was addressed
using a balancing sampler [36]. This technique ensured that each batch
of PSDs was balanced, effectively representing both classes propor-
tionately, even when positive signal segments were less frequent in the
training dataset. As our MLA operates at the biosignal level, it encoun-
ters learning targets in the form of signal segments, which are not
directly reflective of the patient population demographics, so the un-
balance in the development dataset has minimal effect on the evaluated
model results. Thus, a valid performance measure is the MLA’s ability to
accurately predict across all patient signal segments within the valida-
tion dataset. Subsequently, we evaluate the MLA’s performance at the

population level on the matched validation dataset to ensure its effec-
tiveness and reliability.

## 2.6. *Development and validation datasets*

The study cohort included in the analysis consisted of 309 patients. The population was divided into development and validation datasets. Patients with no biosignal data were discarded, as this indicates a technical failure of the recording system. The study profile graph is presented in *Supplementary Figure 2*. Using DeLong's method, we estimated that a validation dataset with a population of 30 LOS patients and 30 controls would result in a minimum detectable AUROC of 0.753, with a power of 0.95 and a significance of 0.05. A validation dataset was created by matching 30 LOS and control patients according to gestational age, birth weight, and twin pregnancy status to equalize the effect of the maturation of the cardiovascular system on the recorded biosignals (Table 1). The remainder of the patients were assigned to the development dataset, which consisted of 13 LOS cases and 190 controls (Table 2). In the validation dataset, none of the demographic variables or available comorbidities differed significantly between the LOS and control patients. The development dataset was unbalanced between LOS and control patients for multiple variables. The population imbalance was not mirrored in the model during the training phase. This is because the signal segments and targets were introduced to the LOS model using a balanced data sampling technique, and the training was conducted with individually labeled PSD segments.
<table><caption>**Table 1**

Characteristics of the patients in the validation dataset.</caption><thead><tr><th></th><th>LOS (n = 30)</th><th>Control (n = 30)</th><th>P</th><th>SMD</th></tr></thead><tbody><tr><td>Gestational age, median [IQR]; weeks</td><td>27.00 [26.00, 28.00]</td><td>27.00 [26.00, 27.75]</td><td>0.964</td><td>0.09</td></tr><tr><td>Birth weight, median [IQR]; g</td><td>872.50 [752.50, 1,040.00]</td><td>900.00 [782.50, 1,000.75]</td><td>0.953</td><td>0.232</td></tr><tr><td>Apgar score at 1 min, median [IQR]</td><td>6.00 [4.00, 9.00]</td><td>5.00 [2.00, 7.00]</td><td>0.139</td><td>0.415</td></tr><tr><td>Apgar score at 5 min, median [IQR]</td><td>9.00 [7.00, 10.00]</td><td>8.50 [7.25, 9.00]</td><td>0.206</td><td>0.158</td></tr><tr><td>Respiratory resuscitation (n = 56), n (%)</td><td>27 (90.0)</td><td>29 (96.7)</td><td>0.605</td><td>0.27</td></tr><tr><td>Surfactant (n = 36), n (%)</td><td>19 (63.3)</td><td>17 (56.7)</td><td>0.792</td><td>0.136</td></tr><tr><td>Premature rupture of membranes (n = 13), n (%)</td><td>5 (16.7)</td><td>8 (26.7)</td><td>0.531</td><td>0.245</td></tr><tr><td>Male (n = 30), n (%)</td><td>14 (46.7)</td><td>16 (53.3)</td><td>0.796</td><td>0.134</td></tr><tr><td>Natural delivery (n = 21), n (%)</td><td>13 (43.3)</td><td>8 (26.7)</td><td>0.279</td><td>0.355</td></tr><tr><td>Single birth (n = 49), n (%)</td><td>23 (76.7)</td><td>26 (86.7)</td><td>0.505</td><td>0.261</td></tr><tr><td>Recording duration, median [IQR]; h</td><td>719.81 [338.25, 1,495.38]</td><td>902.04 [382.88, 1,571.19]</td><td>0.813</td><td>0.156</td></tr><tr><td>Comorbidities, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>Neurological damage</td><td>5 (16.7)</td><td>7 (23.3)</td><td>0.747</td><td>0.167</td></tr><tr><td>Necrotizing enterocolitis</td><td>4 (13.3)</td><td>0 (0.0)</td><td>0.121</td><td>0.555</td></tr><tr><td>Centers, n (%)</td><td></td><td></td><td>0.002</td><td>1.336</td></tr><tr><td>1</td><td>5 (16.7)</td><td>8 (26.7)</td><td></td><td></td></tr><tr><td>2</td><td>9 (30.0)</td><td>2 (6.7)</td><td></td><td></td></tr><tr><td>3</td><td>1 (3.3)</td><td>10 (33.3)</td><td></td><td></td></tr><tr><td>4</td><td>3 (10.0)</td><td>6 (20.0)</td><td></td><td></td></tr><tr><td>5</td><td>5 (16.7)</td><td>3 (10.0)</td><td></td><td></td></tr><tr><td>6</td><td>7 (23.3)</td><td>1 (3.3)</td><td></td><td></td></tr></tbody></table>

IQR = interquartile range; SMD = standardized mean difference.

### 3. Results

Using the previously defined evaluation scheme, the model achieved an AUROC score of 0.810 (95 % CI 0.698–0.922) on the validation dataset (Fig. 2A). Using hyperparameter optimization, the optimal operating point for this model was calculated to be at a threshold value of 0.127. The final loss for the LOS model in the development dataset was 0.0008 and for the validation dataset was 0.0034. The output of the sigmoid layer of a CNN is not well calibrated; thus, this threshold cannot be directly interpreted as the model’s confidence in the negative or positive class, as in the case of a logistic regression model [37,38]. Therefore, the model output cannot be directly interpreted as the probability of LOS in each time segment. The primary objective of our LOS model was to leverage its powerful feature extraction and pattern recognition capabilities, which are critical for distinguishing between complex spectral signatures associated with LOS. The decision boundary, rather than the estimated probabilities, was the focal point of our analysis. Using the optimal threshold, the model detected LOS in the validation population, with a sensitivity of 83 % and a specificity of 73 % (Fig. 2B). Supplementary Figure 3 presents detailed performance metrics for accuracy and negative and positive predictive values with arbitrary cut-off thresholds. The confusion matrix for the validation dataset using the optimal operating point is presented in Supplementary Figure 8. The early detection of LOS was achieved at a median of 44 (IQR −298 to + 17) hours before clinical suspicion at $t_0$. A histogram of the first detection times for each patient related to clinical suspicion is presented in Supplementary Figure 4. An additional supplemental model performance evaluation with fixed time windows before clinical suspicion, as compared to a randomly located similarly sized window sample from control patients, is presented in Supplementary Table 1. This analysis aimed to demonstrate that the model’s performance remained consistent, regardless of the inclusion of the initial recording

<table><caption>**Table 2**

Characteristics of the patients in the development dataset.</caption><thead><tr><th></th><th>LOS (n = 13)</th><th>Control (n = 190)</th><th>P</th><th>SMD</th></tr></thead><tbody><tr><td>Gestational age, median [IQR]; weeks</td><td>25.00 [25.00, 26.00]</td><td>31.00 [29.00, 33.00]</td><td>&lt;</td><td>2.227</td></tr><tr><td>Birth weight, median [IQR]; g</td><td>700.00 [600.00, 780.00]</td><td>1,445.00 [1,228.75, 1,932.50]</td><td>&lt;</td><td>1.982</td></tr><tr><td>Apgar score at 1 min, median [IQR]</td><td>4.00 [2.00, 6.00]</td><td>7.50 [5.00, 9.00]</td><td>0.01</td><td>0.776</td></tr><tr><td>Apgar score at 5 min, median [IQR]</td><td>6.00 [5.00, 9.00]</td><td>10.00 [8.00, 10.00]</td><td>&lt;</td><td>1.105</td></tr><tr><td>Respiratory resuscitation (n = 167), n (%)</td><td>13 (100.0)</td><td>154 (81.1)</td><td>0.175</td><td>0.684</td></tr><tr><td>Surfactant (n = 56), n (%)</td><td>10 (76.9)</td><td>46 (24.2)</td><td>&lt;</td><td>1.241</td></tr><tr><td>Premature rupture of membranes (n = 39), n (%)</td><td>1 (7.7)</td><td>38 (20.0)</td><td>0.468</td><td>0.362</td></tr><tr><td>Male (n = 111), n (%)</td><td>7 (53.8)</td><td>104 (54.7)</td><td>1</td><td>0.018</td></tr><tr><td>Natural delivery (n = 82), n (%)</td><td>5 (38.5)</td><td>77 (40.5)</td><td>1</td><td>0.042</td></tr><tr><td>Single birth (n = 150), n (%)</td><td>12 (92.3)</td><td>138 (72.6)</td><td>0.216</td><td>0.536</td></tr><tr><td>Recording duration, median [IQR]; h</td><td>1140.36 [427.05, 2037.73]</td><td>280.80 [46.49, 597.40]</td><td>0.001</td><td>1.059</td></tr><tr><td>Comorbidities, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>Neurological damage</td><td>5 (38.5)</td><td>17 (8.9)</td><td>0.004</td><td>0.740</td></tr><tr><td>Necrotizing enterocolitis</td><td>3 (23.1)</td><td>2 (1.1)</td><td>&lt;</td><td>0.718</td></tr><tr><td>Centers, n (%)</td><td></td><td></td><td>0.01</td><td></td></tr><tr><td>1</td><td>4 (30.8)</td><td>53 (27.9)</td><td></td><td></td></tr><tr><td>2</td><td>1 (7.7)</td><td>14 (7.4)</td><td></td><td></td></tr><tr><td>3</td><td>1 (7.7)</td><td>42 (22.1)</td><td></td><td></td></tr><tr><td>4</td><td>3 (23.1)</td><td>21 (11.1)</td><td></td><td></td></tr><tr><td>5</td><td>2 (15.4)</td><td>43 (22.6)</td><td></td><td></td></tr><tr><td>6</td><td>2 (15.4)</td><td>17 (8.9)</td><td></td><td></td></tr></tbody></table>

IQR = interquartile range; SMD = standardized mean difference.

period of the patients’ biosignals.

To inspect the relative importance of each input component, a permutation importance analysis was performed on the validation database (Fig. 2D). Based on this analysis, we found that ECG and RESP signals contributed most of the predictive information, with minimal contribution from PLETH signals.

The behavior of the developed MLA was further observed on the temporal axis for the entire validation cohort (Fig. 3A) and a selected LOS patient to establish how the MLA reacted to the onset of LOS (Fig. 3B). The prediction curves for the whole validation cohort also identified true positives, true negatives, false positives, and false negatives and data gaps larger than one hour in the recordings. Available comorbidities for the patients is displayed as a small rectangle indicator. Detailed plots of the validation cohort predictions can be viewed in Supplementary Figure 11. For the control patients, the MLA seemed to mostly output constant low-level scores, even though both the control and LOS patient NICU stays spanning multiple days. For the selected LOS patient, the MLA output began at a low level, but as the time of antibiotic administration approached, the MLA generated multiple high outputs. Finally, the predictor output for LOS patients decreased after the administration of antibiotics. The recorded ECG channel mirrored the displayed ECG channel; thus, it may vary. Based on the data, we confirmed that there was no correlation between changes in the displayed ECG channel and alterations in the model’s output. This suggests that any variations in the model output were not a consequence of a change in the ECG lead.

Additional introspection was performed to inspect specific early high-risk outputs for an LOS patient (Fig. 3B). Neural network activation patterns seem to indicate positively predictive areas in the ECG and RESP spectra (Fig. 4). For ECGs, the real and imaginary parts of the PSD
<table><caption>**Table 1**

Characteristics of the patients in the validation dataset.</caption><thead><tr><th></th><th>LOS (n = 30)</th><th>Control (n = 30)</th><th>P</th><th>SMD</th></tr></thead><tbody><tr><td>Gestational age, median [IQR]; weeks</td><td>27.00 [26.00, 28.00]</td><td>27.00 [26.00, 27.75]</td><td>0.964</td><td>0.09</td></tr><tr><td>Birth weight, median [IQR]; g</td><td>872.50 [752.50, 1,040.00]</td><td>900.00 [782.50, 1,000.75]</td><td>0.953</td><td>0.232</td></tr><tr><td>Apgar score at 1 min, median [IQR]</td><td>6.00 [4.00, 9.00]</td><td>5.00 [2.00, 7.00]</td><td>0.139</td><td>0.415</td></tr><tr><td>Apgar score at 5 min, median [IQR]</td><td>9.00 [7.00, 10.00]</td><td>8.50 [7.25, 9.00]</td><td>0.206</td><td>0.158</td></tr><tr><td>Respiratory resuscitation (n = 56), n (%)</td><td>27 (90.0)</td><td>29 (96.7)</td><td>0.605</td><td>0.27</td></tr><tr><td>Surfactant (n = 36), n (%)</td><td>19 (63.3)</td><td>17 (56.7)</td><td>0.792</td><td>0.136</td></tr><tr><td>Premature rupture of membranes (n = 13), n (%)</td><td>5 (16.7)</td><td>8 (26.7)</td><td>0.531</td><td>0.245</td></tr><tr><td>Male (n = 30), n (%)</td><td>14 (46.7)</td><td>16 (53.3)</td><td>0.796</td><td>0.134</td></tr><tr><td>Natural delivery (n = 21), n (%)</td><td>13 (43.3)</td><td>8 (26.7)</td><td>0.279</td><td>0.355</td></tr><tr><td>Single birth (n = 49), n (%)</td><td>23 (76.7)</td><td>26 (86.7)</td><td>0.505</td><td>0.261</td></tr><tr><td>Recording duration, median [IQR]; h</td><td>719.81 [338.25, 1,495.38]</td><td>902.04 [382.88, 1,571.19]</td><td>0.813</td><td>0.156</td></tr><tr><td>Comorbidities, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>Neurological damage</td><td>5 (16.7)</td><td>7 (23.3)</td><td>0.747</td><td>0.167</td></tr><tr><td>Necrotizing enterocolitis</td><td>4 (13.3)</td><td>0 (0.0)</td><td>0.121</td><td>0.555</td></tr><tr><td>Centers, n (%)</td><td></td><td></td><td>0.002</td><td>1.336</td></tr><tr><td>1</td><td>5 (16.7)</td><td>8 (26.7)</td><td></td><td></td></tr><tr><td>2</td><td>9 (30.0)</td><td>2 (6.7)</td><td></td><td></td></tr><tr><td>3</td><td>1 (3.3)</td><td>10 (33.3)</td><td></td><td></td></tr><tr><td>4</td><td>3 (10.0)</td><td>6 (20.0)</td><td></td><td></td></tr><tr><td>5</td><td>5 (16.7)</td><td>3 (10.0)</td><td></td><td></td></tr><tr><td>6</td><td>7 (23.3)</td><td>1 (3.3)</td><td></td><td></td></tr></tbody></table>

IQR = interquartile range; SMD = standardized mean difference.

### 3. Results

Using the previously defined evaluation scheme, the model achieved an AUROC score of 0.810 (95 % CI 0.698–0.922) on the validation dataset (Fig. 2A). Using hyperparameter optimization, the optimal operating point for this model was calculated to be at a threshold value of 0.127. The final loss for the LOS model in the development dataset was 0.0008 and for the validation dataset was 0.0034. The output of the sigmoid layer of a CNN is not well calibrated; thus, this threshold cannot be directly interpreted as the model’s confidence in the negative or positive class, as in the case of a logistic regression model [37,38]. Therefore, the model output cannot be directly interpreted as the probability of LOS in each time segment. The primary objective of our LOS model was to leverage its powerful feature extraction and pattern recognition capabilities, which are critical for distinguishing between complex spectral signatures associated with LOS. The decision boundary, rather than the estimated probabilities, was the focal point of our analysis. Using the optimal threshold, the model detected LOS in the validation population, with a sensitivity of 83 % and a specificity of 73 % (Fig. 2B). Supplementary Figure 3 presents detailed performance metrics for accuracy and negative and positive predictive values with arbitrary cut-off thresholds. The confusion matrix for the validation dataset using the optimal operating point is presented in Supplementary Figure 8. The early detection of LOS was achieved at a median of 44 (IQR −298 to + 17) hours before clinical suspicion at $t_0$. A histogram of the first detection times for each patient related to clinical suspicion is presented in Supplementary Figure 4. An additional supplemental model performance evaluation with fixed time windows before clinical suspicion, as compared to a randomly located similarly sized window sample from control patients, is presented in Supplementary Table 1. This analysis aimed to demonstrate that the model’s performance remained consistent, regardless of the inclusion of the initial recording

<table><caption>**Table 2**

Characteristics of the patients in the development dataset.</caption><thead><tr><th></th><th>LOS (n = 13)</th><th>Control (n = 190)</th><th>P</th><th>SMD</th></tr></thead><tbody><tr><td>Gestational age, median [IQR]; weeks</td><td>25.00 [25.00, 26.00]</td><td>31.00 [29.00, 33.00]</td><td>&lt;</td><td>2.227</td></tr><tr><td>Birth weight, median [IQR]; g</td><td>700.00 [600.00, 780.00]</td><td>1,445.00 [1,228.75, 1,932.50]</td><td>&lt;</td><td>1.982</td></tr><tr><td>Apgar score at 1 min, median [IQR]</td><td>4.00 [2.00, 6.00]</td><td>7.50 [5.00, 9.00]</td><td>0.01</td><td>0.776</td></tr><tr><td>Apgar score at 5 min, median [IQR]</td><td>6.00 [5.00, 9.00]</td><td>10.00 [8.00, 10.00]</td><td>&lt;</td><td>1.105</td></tr><tr><td>Respiratory resuscitation (n = 167), n (%)</td><td>13 (100.0)</td><td>154 (81.1)</td><td>0.175</td><td>0.684</td></tr><tr><td>Surfactant (n = 56), n (%)</td><td>10 (76.9)</td><td>46 (24.2)</td><td>&lt;</td><td>1.241</td></tr><tr><td>Premature rupture of membranes (n = 39), n (%)</td><td>1 (7.7)</td><td>38 (20.0)</td><td>0.468</td><td>0.362</td></tr><tr><td>Male (n = 111), n (%)</td><td>7 (53.8)</td><td>104 (54.7)</td><td>1</td><td>0.018</td></tr><tr><td>Natural delivery (n = 82), n (%)</td><td>5 (38.5)</td><td>77 (40.5)</td><td>1</td><td>0.042</td></tr><tr><td>Single birth (n = 150), n (%)</td><td>12 (92.3)</td><td>138 (72.6)</td><td>0.216</td><td>0.536</td></tr><tr><td>Recording duration, median [IQR]; h</td><td>1140.36 [427.05, 2037.73]</td><td>280.80 [46.49, 597.40]</td><td>0.001</td><td>1.059</td></tr><tr><td>Comorbidities, n (%)</td><td></td><td></td><td></td><td></td></tr><tr><td>Neurological damage</td><td>5 (38.5)</td><td>17 (8.9)</td><td>0.004</td><td>0.740</td></tr><tr><td>Necrotizing enterocolitis</td><td>3 (23.1)</td><td>2 (1.1)</td><td>&lt;</td><td>0.718</td></tr><tr><td>Centers, n (%)</td><td></td><td></td><td>0.01</td><td></td></tr><tr><td>1</td><td>4 (30.8)</td><td>53 (27.9)</td><td></td><td></td></tr><tr><td>2</td><td>1 (7.7)</td><td>14 (7.4)</td><td></td><td></td></tr><tr><td>3</td><td>1 (7.7)</td><td>42 (22.1)</td><td></td><td></td></tr><tr><td>4</td><td>3 (23.1)</td><td>21 (11.1)</td><td></td><td></td></tr><tr><td>5</td><td>2 (15.4)</td><td>43 (22.6)</td><td></td><td></td></tr><tr><td>6</td><td>2 (15.4)</td><td>17 (8.9)</td><td></td><td></td></tr></tbody></table>

IQR = interquartile range; SMD = standardized mean difference.

period of the patients’ biosignals.

To inspect the relative importance of each input component, a permutation importance analysis was performed on the validation database (Fig. 2D). Based on this analysis, we found that ECG and RESP signals contributed most of the predictive information, with minimal contribution from PLETH signals.

The behavior of the developed MLA was further observed on the temporal axis for the entire validation cohort (Fig. 3A) and a selected LOS patient to establish how the MLA reacted to the onset of LOS (Fig. 3B). The prediction curves for the whole validation cohort also identified true positives, true negatives, false positives, and false negatives and data gaps larger than one hour in the recordings. Available comorbidities for the patients is displayed as a small rectangle indicator. Detailed plots of the validation cohort predictions can be viewed in Supplementary Figure 11. For the control patients, the MLA seemed to mostly output constant low-level scores, even though both the control and LOS patient NICU stays spanning multiple days. For the selected LOS patient, the MLA output began at a low level, but as the time of antibiotic administration approached, the MLA generated multiple high outputs. Finally, the predictor output for LOS patients decreased after the administration of antibiotics. The recorded ECG channel mirrored the displayed ECG channel; thus, it may vary. Based on the data, we confirmed that there was no correlation between changes in the displayed ECG channel and alterations in the model’s output. This suggests that any variations in the model output were not a consequence of a change in the ECG lead.

Additional introspection was performed to inspect specific early high-risk outputs for an LOS patient (Fig. 3B). Neural network activation patterns seem to indicate positively predictive areas in the ECG and RESP spectra (Fig. 4). For ECGs, the real and imaginary parts of the PSD
A

<figure><img src="image_7.png" /><figcaption>**Fig. 2.** LOS model performance on the validation dataset. A. ROC curve of the LOS model, illustrating its capability to identify cases of LOS within the validation population. B. Sensitivity curve of the LOS model, highlighting how changes in the operating point influence the model's sensitivity. C. Specificity curve of the LOS model, showing the impact of altering the operating point on the model's specificity. D. Permutation importance plot, visualizing the relative importance of each biosignal to the model's performance. This plot provides a ranking of features based on their contribution to the model's predictive accuracy, calculated with permutation importance analysis.</figcaption></figure>

have similar activation patterns in the frequency range of 15–31 Hz. The MLA seemed to focus on a specific rapid increase and decrease in frequency on the part of multiple ECG frequency components but did not react to a presumed movement artifact present in the middle of the viewport in the range of 900–1,350 s, thus validating the approach of including all biosignal data in the training dataset—that is, no specific artifact removal preprocessing step was needed. The activation pattern of the RESP PSD had more intensity on the imaginary component at the frequency range of 5–10 Hz.

These positively predictive frequency bands were further explored by visualizing a raw signal sample near the inspection point in the temporal domain and coloring parts of it according to the dominant frequency in a sliding window (Fig. 5). In the ECG sample, the positively predictive frequencies in the 15–31 Hz interval seemed to be present in the R-wave (Fig. 5A). In the RESP sample, the positively predictive frequencies in the 5–10 Hz interval seemed to be present in a specific transient that indicated a sudden change in the RESP waveform (Fig. 5B). The observed dominant frequency bands within the temporal domain connected the pathogenic spectral features identified in the PSDs to discernible characteristics on the ECG and RESP waveforms. This association enhances the explainability of the model's performance and relates it to the clinically known features of these biosignals.

## 4. Discussion

In this study, we presented a novel non-invasive MLA that can detect LOS using PSD presentations of high-frequency biosignals. We consider this study sufficiently powered to establish the presence of the phenomenon because the AUROC of 0.810 calculated for the proposed MLA exceeds the minimum observable AUROC value of 0.753 in the validation dataset according to DeLong's method. Overall, the results indicate that there may be considerable early detection potential in the case of neonatal infection. The proposed MLA has learned to output a high score correlating with clinical suspicion of sepsis in LOS patients, and it maintains a constant low level score for control patients (Fig. 3A). It should also be noted that the pathogenic spectral features identified are temporally sparse and not consistently present in the recorded biosignals (Fig. 4). The findings presented in Fig. 3A demonstrate the significant early detection capabilities of the developed LOS model. Fig. 5A and 5B suggest that the predictive performance of the model is attributed to the morphology of the R-wave and abrupt shifts in the RESP waveform. This addresses the research gap in examining the morphology of raw biosignals for the prediction of LOS onset.

In the context of LOS, the premise of advanced predictive models is that by exploiting the high-resolution, physiological streams of data that are continuously acquired through routine patient monitoring, subclinical signatures of LOS can be promptly identified before the LOS
A

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** LOS model performance on the validation dataset. A. ROC curve of the LOS model, illustrating its capability to identify cases of LOS within the validation population. B. Sensitivity curve of the LOS model, highlighting how changes in the operating point influence the model's sensitivity. C. Specificity curve of the LOS model, showing the impact of altering the operating point on the model's specificity. D. Permutation importance plot, visualizing the relative importance of each biosignal to the model's performance. This plot provides a ranking of features based on their contribution to the model's predictive accuracy, calculated with permutation importance analysis.</figcaption></figure>

have similar activation patterns in the frequency range of 15–31 Hz. The MLA seemed to focus on a specific rapid increase and decrease in frequency on the part of multiple ECG frequency components but did not react to a presumed movement artifact present in the middle of the viewport in the range of 900–1,350 s, thus validating the approach of including all biosignal data in the training dataset—that is, no specific artifact removal preprocessing step was needed. The activation pattern of the RESP PSD had more intensity on the imaginary component at the frequency range of 5–10 Hz.

These positively predictive frequency bands were further explored by visualizing a raw signal sample near the inspection point in the temporal domain and coloring parts of it according to the dominant frequency in a sliding window (Fig. 5). In the ECG sample, the positively predictive frequencies in the 15–31 Hz interval seemed to be present in the R-wave (Fig. 5A). In the RESP sample, the positively predictive frequencies in the 5–10 Hz interval seemed to be present in a specific transient that indicated a sudden change in the RESP waveform (Fig. 5B). The observed dominant frequency bands within the temporal domain connected the pathogenic spectral features identified in the PSDs to discernible characteristics on the ECG and RESP waveforms. This association enhances the explainability of the model's performance and relates it to the clinically known features of these biosignals.

## 4. Discussion

In this study, we presented a novel non-invasive MLA that can detect LOS using PSD presentations of high-frequency biosignals. We consider this study sufficiently powered to establish the presence of the phenomenon because the AUROC of 0.810 calculated for the proposed MLA exceeds the minimum observable AUROC value of 0.753 in the validation dataset according to DeLong's method. Overall, the results indicate that there may be considerable early detection potential in the case of neonatal infection. The proposed MLA has learned to output a high score correlating with clinical suspicion of sepsis in LOS patients, and it maintains a constant low level score for control patients (Fig. 3A). It should also be noted that the pathogenic spectral features identified are temporally sparse and not consistently present in the recorded biosignals (Fig. 4). The findings presented in Fig. 3A demonstrate the significant early detection capabilities of the developed LOS model. Fig. 5A and 5B suggest that the predictive performance of the model is attributed to the morphology of the R-wave and abrupt shifts in the RESP waveform. This addresses the research gap in examining the morphology of raw biosignals for the prediction of LOS onset.

In the context of LOS, the premise of advanced predictive models is that by exploiting the high-resolution, physiological streams of data that are continuously acquired through routine patient monitoring, subclinical signatures of LOS can be promptly identified before the LOS
B

<figure><img src="image_8.png" /><figcaption>Fig. 2. (continued).</figcaption></figure>

deteriorates to the extent apparent to physicians. In this study, we developed and validated a CNN-based deep learning architecture that is able to extract predictive information from the PSD transforms of ECG, PLETH, and RESP signals regarding the onset of LOS. As this study demonstrates, the labeling of biosignal segments in the 24 h preceding the administration of antibiotics to a positive class and all others to a negative class results in a highly accurate LOS detector.

Further observation of the predictive components learned by the
MLA indicates that the detector reacts to specific spectral signatures
present in the ECG and RESP spectra of septic patients. This result is in
line with the current literature because previous studies have indicated
that the parameters calculated from ECGs and respiratory waveforms
contain information related to the onset of LOS [39]. Furthermore, the
identified predictive parts of the spectra are present only in a limited
temporal window and, thus, are most likely biological in origin and not
the result of an artifact from the monitoring device. In a more detailed
inspection of the ECG and RESP signals in the temporal domain, we
discovered that the MLA reacted to the morphology of the R-wave in the
ECG sample (Fig. 5A) and to a sudden change in the RESP sample
(Fig. 5B). Previous studies have explored the potential of predicting LOS
non-invasively using deep learning, and the possibility of non-invasively
predicting LOS by utilizing deep learning techniques has been investi-
gated. However, these methods depend on feature engineering, such as
computing R-R intervals and HRV parameters from biosignals prior to
the learning stage, which may diminish the amount of biosignal infor-
mation available to the model. To our knowledge, this is the first study
that highlights the value of the raw biosignals and independent, additive
predictive values of R-wave morphology and concurrent, sudden
changes in the RESP waveform in predicting the onset of LOS.

The strengths of this study include its relatively large sample size of
309 prospectively collected patients across six institutions with contin-
uous high-frequency biosignal recordings. In addition, the validation set
was matched between LOS and control patients to stratify the results
according to known confounding factors in this study population.
Another strength is the study design, in which high-frequency biosignals
were constantly recorded during a patient’s entire stay. This method of
data collection accumulates large amounts of detailed physiological
information that can be related to concurrent events recorded in hospital
EHRs. Furthermore, the developed labeling scheme and temporally
agnostic PSD-based approach make it possible to leverage the power of a
deep neural network to deliver state-of-the-art predictive performance
in detecting LOS accurately and early.

This study also has several limitations that must be acknowledged. The proposed MLA demonstrates high precision in predicting LOS within the validation dataset. It shows no evident correlation with comorbidities related to necrotizing enterocolitis and neurological damage (Fig. 3A). However, the potential for cross-reactivity of the MLA with other conditions that affect cardiorespiratory stability cannot be thoroughly assessed within this study. The study also lacks manual inspection of all recorded ECG, PLETH, and RESP signals, so the amount of artifacts cannot be precisely quantified. Inspecting all recorded high-frequency biosignals manually and annotating artifacts would be very time-consuming and costly. Moreover, one of the main points of the proposed approach is that the 2D PSD and CNN combined approach is able to identify specific information related to LOS without artifact removal. Therefore, we cannot rule out that at least some late detections could be due to poor signal quality before antibiotic administration. Some very early detections may indicate an undetected subclinical
B

<figure><img src="image_7.png" /><figcaption>Fig. 2. (continued).</figcaption></figure>

deteriorates to the extent apparent to physicians. In this study, we developed and validated a CNN-based deep learning architecture that is able to extract predictive information from the PSD transforms of ECG, PLETH, and RESP signals regarding the onset of LOS. As this study demonstrates, the labeling of biosignal segments in the 24 h preceding the administration of antibiotics to a positive class and all others to a negative class results in a highly accurate LOS detector.

Further observation of the predictive components learned by the
MLA indicates that the detector reacts to specific spectral signatures
present in the ECG and RESP spectra of septic patients. This result is in
line with the current literature because previous studies have indicated
that the parameters calculated from ECGs and respiratory waveforms
contain information related to the onset of LOS [39]. Furthermore, the
identified predictive parts of the spectra are present only in a limited
temporal window and, thus, are most likely biological in origin and not
the result of an artifact from the monitoring device. In a more detailed
inspection of the ECG and RESP signals in the temporal domain, we
discovered that the MLA reacted to the morphology of the R-wave in the
ECG sample (Fig. 5A) and to a sudden change in the RESP sample
(Fig. 5B). Previous studies have explored the potential of predicting LOS
non-invasively using deep learning, and the possibility of non-invasively
predicting LOS by utilizing deep learning techniques has been investi-
gated. However, these methods depend on feature engineering, such as
computing R-R intervals and HRV parameters from biosignals prior to
the learning stage, which may diminish the amount of biosignal infor-
mation available to the model. To our knowledge, this is the first study
that highlights the value of the raw biosignals and independent, additive
predictive values of R-wave morphology and concurrent, sudden
changes in the RESP waveform in predicting the onset of LOS.

The strengths of this study include its relatively large sample size of
309 prospectively collected patients across six institutions with contin-
uous high-frequency biosignal recordings. In addition, the validation set
was matched between LOS and control patients to stratify the results
according to known confounding factors in this study population.
Another strength is the study design, in which high-frequency biosignals
were constantly recorded during a patient’s entire stay. This method of
data collection accumulates large amounts of detailed physiological
information that can be related to concurrent events recorded in hospital
EHRs. Furthermore, the developed labeling scheme and temporally
agnostic PSD-based approach make it possible to leverage the power of a
deep neural network to deliver state-of-the-art predictive performance
in detecting LOS accurately and early.

This study also has several limitations that must be acknowledged. The proposed MLA demonstrates high precision in predicting LOS within the validation dataset. It shows no evident correlation with comorbidities related to necrotizing enterocolitis and neurological damage (Fig. 3A). However, the potential for cross-reactivity of the MLA with other conditions that affect cardiorespiratory stability cannot be thoroughly assessed within this study. The study also lacks manual inspection of all recorded ECG, PLETH, and RESP signals, so the amount of artifacts cannot be precisely quantified. Inspecting all recorded high-frequency biosignals manually and annotating artifacts would be very time-consuming and costly. Moreover, one of the main points of the proposed approach is that the 2D PSD and CNN combined approach is able to identify specific information related to LOS without artifact removal. Therefore, we cannot rule out that at least some late detections could be due to poor signal quality before antibiotic administration. Some very early detections may indicate an undetected subclinical
C

<figure><img src="image_9.png" /></figure>

D

<figure><img src="image_10.png" /><figcaption>Fig. 2. (continued).</figcaption></figure>
C

<figure><img src="image_8.png" /></figure>

D

<figure><img src="image_9.png" /><figcaption>Fig. 2. (continued).</figcaption></figure>
A

<figure><img src="image_11.png" /><figcaption>**Fig. 3.** Evolution of LOS model scores across the entire validation dataset and for an individual patient with LOS. A, Score evolution of all LOS and control patients on the validation dataset. This set of individual figures depicts the progression of the model scores for each patient. The operating point is shown as a horizontal line, determined through hyperparameter optimization of the development data. Vertical lines mark instances in which data gaps exceed one hour. Additionally, the figures categorize each patient as true negative, true positive, false positive, or false negative with respect to LOS detection. The score evolution plot displays any available comorbidities as rectangular indicators in its upper right corner, if they are present. B, Score evolution on a selected LOS patient. A single LOS patient from the validation dataset was chosen for a more detailed examination. The progression of the model score is annotated with the timing of antibiotic administration and an “inspection point,” which identifies the moment when the input PSDs are sampled on a particularly high-risk prediction. This closer inspection aims to determine which spectral components are predictive of LOS in this high-risk output instance.</figcaption></figure>
A

<figure><img src="image_10.png" /><figcaption>**Fig. 3.** Evolution of LOS model scores across the entire validation dataset and for an individual patient with LOS. A, Score evolution of all LOS and control patients on the validation dataset. This set of individual figures depicts the progression of the model scores for each patient. The operating point is shown as a horizontal line, determined through hyperparameter optimization of the development data. Vertical lines mark instances in which data gaps exceed one hour. Additionally, the figures categorize each patient as true negative, true positive, false positive, or false negative with respect to LOS detection. The score evolution plot displays any available comorbidities as rectangular indicators in its upper right corner, if they are present. B, Score evolution on a selected LOS patient. A single LOS patient from the validation dataset was chosen for a more detailed examination. The progression of the model score is annotated with the timing of antibiotic administration and an “inspection point,” which identifies the moment when the input PSDs are sampled on a particularly high-risk prediction. This closer inspection aims to determine which spectral components are predictive of LOS in this high-risk output instance.</figcaption></figure>
B

<figure><img src="image_12.png" /><figcaption>Fig. 3. (continued).</figcaption></figure>

infection, or the MLA response and baseline level may display individual variability. However, as the temporal distribution of detections in relation to clinical suspicion showed, the first detection timestamps on most of the validation patients were correlated with the clinical suspicion timestamp, thus giving credibility to the notion that the MLA detected an infectious event based on the biosignals (Supplementary Figure 4). The extremely early prediction of LOS for a particular patient in the validation dataset may also be caused by adaptations to extrauterine life at the beginning of the recording. We speculate that the neonates’ physiological adaptations to extrauterine life might mimic an inflammatory response, which consequently causes the MLA to output high LOS scores. This may also explain some of the false positives in Fig. 3A, as the operating point was crossed very early during the monitoring period. A supplemental analysis with fixed windows before clinical suspicion of LOS was performed to rule out that the model performance comes solely on generating high predictions at the beginning of life (Supplementary table 1). Fixed-time window analysis also showed that the MLA retained substantial temporal early predictive capability on increasing window sizes. The MLA also seemed to be very resistant to making false-positive outputs due to noise because the predictions for control patients displayed constant low-level output despite artifacts, gaps in data, care procedures, and ECG lead changes, without any data curation. The output from the MLA could be utilized to refine

existing antibiotic regimens. Specifically, for patients exhibiting consistently low-level MLA output, there may be potential to reduce antibiotic usage. Additionally, the MLA output could aid in detecting antibiotic effectiveness or re-infection, as illustrated in Fig. 3A.

Future research should consider incorporating demographic parameters, such as gestational age, into the model. There is a need to investigate whether the model becomes overfitted to demographic data when applying PSD augmentation and to evaluate the incremental benefits this data brings to model performance. Additionally, the impact of families’ socioeconomic status on model performance requires examination to identify and quantify potential biases across different populations. The model’s performance across different clinical settings with different levels of care should also be explored. Nevertheless, the current LOS model is designed to be resistant to demographic biases, relying solely on measured biosignals for its input, which significantly minimizes the risk of demographic data introducing unfair biases. However, this resistance does not negate the necessity for model validation across a diverse range of populations and demographics, as biosignals could be influenced by genetic or environmental factors. Validation should also extend to biosignals from different ICU monitor vendors to assess any device-specific biases. The permutation analysis presented in Fig. 2D reveals that the PLETH signal neither contributes to nor impairs the performance of the LOS model we have developed. This
B

<figure><img src="image_11.png" /><figcaption>Fig. 3. (continued).</figcaption></figure>

infection, or the MLA response and baseline level may display individual variability. However, as the temporal distribution of detections in relation to clinical suspicion showed, the first detection timestamps on most of the validation patients were correlated with the clinical suspicion timestamp, thus giving credibility to the notion that the MLA detected an infectious event based on the biosignals (Supplementary Figure 4). The extremely early prediction of LOS for a particular patient in the validation dataset may also be caused by adaptations to extrauterine life at the beginning of the recording. We speculate that the neonates' physiological adaptations to extrauterine life might mimic an inflammatory response, which consequently causes the MLA to output high LOS scores. This may also explain some of the false positives in Fig. 3A, as the operating point was crossed very early during the monitoring period. A supplemental analysis with fixed windows before clinical suspicion of LOS was performed to rule out that the model performance comes solely on generating high predictions at the beginning of life (Supplementary table 1). Fixed-time window analysis also showed that the MLA retained substantial temporal early predictive capability on increasing window sizes. The MLA also seemed to be very resistant to making false-positive outputs due to noise because the predictions for control patients displayed constant low-level output despite artifacts, gaps in data, care procedures, and ECG lead changes, without any data curation. The output from the MLA could be utilized to refine

existing antibiotic regimens. Specifically, for patients exhibiting consistently low-level MLA output, there may be potential to reduce antibiotic usage. Additionally, the MLA output could aid in detecting antibiotic effectiveness or re-infection, as illustrated in Fig. 3A.

Future research should consider incorporating demographic parameters, such as gestational age, into the model. There is a need to investigate whether the model becomes overfitted to demographic data when applying PSD augmentation and to evaluate the incremental benefits this data brings to model performance. Additionally, the impact of families' socioeconomic status on model performance requires examination to identify and quantify potential biases across different populations. The model's performance across different clinical settings with different levels of care should also be explored. Nevertheless, the current LOS model is designed to be resistant to demographic biases, relying solely on measured biosignals for its input, which significantly minimizes the risk of demographic data introducing unfair biases. However, this resistance does not negate the necessity for model validation across a diverse range of populations and demographics, as biosignals could be influenced by genetic or environmental factors. Validation should also extend to biosignals from different ICU monitor vendors to assess any device-specific biases. The permutation analysis presented in Fig. 2D reveals that the PLETH signal neither contributes to nor impairs the performance of the LOS model we have developed. This
<figure><img src="image_13.png" /><figcaption>**Fig. 4.** Input attribution for high-risk prediction. This illustration shows the real and imaginary parts of the input PSDs at the “inspection point” defined in Fig. 3B. The positive predictive parts of the PSDs are highlighted, indicating spectral regions that are indicative of pathogenic spectral features and of LOS.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>**Fig. 4.** Input attribution for high-risk prediction. This illustration shows the real and imaginary parts of the input PSDs at the “inspection point” defined in Fig. 3B. The positive predictive parts of the PSDs are highlighted, indicating spectral regions that are indicative of pathogenic spectral features and of LOS.</figcaption></figure>
A

<figure><img src="image_14.png" /></figure>

B

<figure><img src="image_15.png" /><figcaption>**Fig. 5.** Dominant frequencies in the input waveforms near the inspection point. The sliding window size is 170 ms for the ECG sample and one second for the RESP sample. A, Dominant frequencies in the ECG waveform sample near the “inspection point” defined in Fig. 3B. B, Dominant frequencies in the RESP waveform sample near the “inspection point” defined in Fig. 3B.</figcaption></figure>
A

<figure><img src="image_13.png" /></figure>

B

<figure><img src="image_14.png" /><figcaption>**Fig. 5.** Dominant frequencies in the input waveforms near the inspection point. The sliding window size is 170 ms for the ECG sample and one second for the RESP sample. A, Dominant frequencies in the ECG waveform sample near the “inspection point” defined in Fig. 3B. B, Dominant frequencies in the RESP waveform sample near the “inspection point” defined in Fig. 3B.</figcaption></figure>
finding suggests that incorporating a diverse array of biosignals as inputs is a promising avenue for future research, with the potential to identify those that are effective in predicting LOS as well as those that may indicate other life-threatening conditions such as cardiac arrest or sepsis in adult patient populations. The use of multiple ECG channels as input signals should be also explored. Future work should investigate the cross-reactivity of the LOS model with conditions that mimic an inflammatory response, such as those observed in neonates adapting to extrauterine life, in order to improve the LOS model's performance.

Moreover, the potential for automation bias must be considered. The LOS model's ability to derive predictive insights from the spectral analysis of biosignals potentially enables earlier diagnosis than traditional methods, yet this also mandates a thorough exploration of the model's failure modes. It is essential to quantify the model's performance across varying comorbidities, signal measurement conditions, and populations where it may not perform as anticipated. Despite the promise of deep learning-based decision support systems in the early detection of LOS, the ultimate responsibility for care decisions must remain with the treating clinician.

## 5. Summary table

What was already known on the topic

* Late-onset neonatal sepsis (LOS) contributes to substantial global morbidity and mortality.

* The gold standard for diagnosing LOS is the culture-based identification of pathogens from sterile sites, which is resource-intensive.

* Conventional blood-borne biomarkers such as procalcitonin and C-reactive protein have been employed for early detection, but they are invasive and cannot be continuously assessed.

* Noninvasive prediction of LOS has been previously attempted with machine learning algorithms using engineered HRV features from ECGs.

What this study added to our knowledge

* The study used a machine learning algorithm utilizing a CNN combined with 2D power spectral density transformation on multiple raw biosignals from neonatal intensive care monitors to predict the onset of LOS early and with good diagnostic value.

* The method of integrating 2D spectral domain presentation with a multimodal CNN allows the omission of a separate artifact removal step, facilitating the use of noisy biosignals, or those without established automatic artifact removal methods.

* The study revealed that the predictive accuracy of the developed model for LOS onset is significantly enhanced by the synergistic combination of the ECG's R-wave morphology and sudden alterations in the respiratory impedance waveform.

## 6. Code availability

The source code and developed LOS model supporting the findings of this study have been deposited in the Zenodo repository and are available for access [40].

## 7. Author statement

We the undersigned declare that this manuscript is original, has not been published before and is not currently being considered for publication elsewhere. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We understand that the Corresponding Author is the sole contact for the Editorial process. He/she is responsible for

communicating with the other authors about progress, submissions of revisions and final approval of proofs

## CRediT authorship contribution statement

**Antti Kallonen:** Writing – original draft, Visualization, Software, Methodology, Formal analysis, Data curation, Conceptualization. **Milla Juutinen:** Writing – review & editing, Validation. **Alpo Värri:** Writing – review & editing, Supervision, Conceptualization. **Guy Carrault:** Writing – review & editing, Supervision, Resources, Project administration, Investigation, Funding acquisition, Conceptualization. **Patrick Pladys:** Writing – review & editing, Supervision, Resources, Project administration, Investigation, Funding acquisition, Conceptualization. **Alain Beuchée:** Writing – review & editing, Resources, Investigation, Conceptualization.

## Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Antti Kallonen has patents pending on the methodology (EP21801580, US2023-0290511 and FI20205993). Other authors declare no competing interests.

## Data availability

We are unable to make the collected patient data and biosignals publicly available, as the informed parental consent was specific to use within this research project.

## Acknowledgements

JM Roué, C Flamant, A Thollot, G Favrais, G Gascoin and the teams of Angers, Brest, Nantes, Rennes, Tours, Poitiers, and the parents of the newborns for their participation in the data collection and clinical annotation.

This study received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant agreement No. 689260 (Digi-NewB project).

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ijmedinf.2024.105366.

## References

[1] C. Fleischmann-Struzek, D.M. Goldfarb, P. Schlattmann, L.J. Schlapbach, K. Reinhart, N. Kissoon, The global burden of paediatric and neonatal sepsis: a systematic review, Lancet Respir. Med. 6 (2018) 223-230.

[2] A.C. Seale, H. Blencowe, et al., Neonatal severe bacterial infection impairment estimates in South Asia, sub-Saharan Africa, and Latin America for 2010, Pediatr Res 74 (Suppl 1) (2013) 73-85.

[3] L. Saiman, Infectious diseases of the fetus and newborn infant, JAMA 307 (2012) 1865.

[4] A.L. Shane, P.J. Sánchez, B.J. Stoll, Neonatal sepsis, Lancet 390 (2017) 1770-1780.

[5] K.D. Fairchild, R.A. Polin, Early-onset neonatal sepsis, W B Saunders Company, 2010.

[6] M.S. Hammoud, A. Al-Taiar, L. Thalib, N. Al-Sweih, S. Pathan, D. Isaacs, Incidence, aetiology and resistance of late-onset neonatal sepsis: a five-year prospective study, J. Paediatr. Child Health 48 (2012) 604-609.

[7] E.H. Verstraete, K. Blot, L. Mahieu, D. Vogelaers, S. Blot, Prediction models for neonatal health care-associated sepsis: a meta-analysis, Pediatrics 135 (2015) e1002-e1004.

[8] R. Laxminarayan, P. Matsoso, S. Pant, et al., Access to effective antimicrobials: a worldwide challenge, Lancet 387 (2016) 168-175.

[9] S. Eschborn, J.-H. Weitkamp, Procalcitonin versus C-reactive protein: review of kinetics and performance for diagnosis of neonatal sepsis, J Perinatol. 39 (2019) 93-903.

[10] D.H. Hubel, T.N. Wiesel, Receptive fields and functional architecture of monkey striate cortex. 1968.
finding suggests that incorporating a diverse array of biosignals as inputs is a promising avenue for future research, with the potential to identify those that are effective in predicting LOS as well as those that may indicate other life-threatening conditions such as cardiac arrest or sepsis in adult patient populations. The use of multiple ECG channels as input signals should be also explored. Future work should investigate the cross-reactivity of the LOS model with conditions that mimic an inflammatory response, such as those observed in neonates adapting to extrauterine life, in order to improve the LOS model's performance.

Moreover, the potential for automation bias must be considered. The LOS model's ability to derive predictive insights from the spectral analysis of biosignals potentially enables earlier diagnosis than traditional methods, yet this also mandates a thorough exploration of the model's failure modes. It is essential to quantify the model's performance across varying comorbidities, signal measurement conditions, and populations where it may not perform as anticipated. Despite the promise of deep learning-based decision support systems in the early detection of LOS, the ultimate responsibility for care decisions must remain with the treating clinician.

## 5. Summary table

What was already known on the topic

* Late-onset neonatal sepsis (LOS) contributes to substantial global morbidity and mortality.

* The gold standard for diagnosing LOS is the culture-based identification of pathogens from sterile sites, which is resource-intensive.

* Conventional blood-borne biomarkers such as procalcitonin and C-reactive protein have been employed for early detection, but they are invasive and cannot be continuously assessed.

* Noninvasive prediction of LOS has been previously attempted with machine learning algorithms using engineered HRV features from ECGs.

What this study added to our knowledge

* The study used a machine learning algorithm utilizing a CNN combined with 2D power spectral density transformation on multiple raw biosignals from neonatal intensive care monitors to predict the onset of LOS early and with good diagnostic value.

* The method of integrating 2D spectral domain presentation with a multimodal CNN allows the omission of a separate artifact removal step, facilitating the use of noisy biosignals, or those without established automatic artifact removal methods.

* The study revealed that the predictive accuracy of the developed model for LOS onset is significantly enhanced by the synergistic combination of the ECG's R-wave morphology and sudden alterations in the respiratory impedance waveform.

## 6. Code availability

The source code and developed LOS model supporting the findings of this study have been deposited in the Zenodo repository and are available for access [40].

## 7. Author statement

We the undersigned declare that this manuscript is original, has not been published before and is not currently being considered for publication elsewhere. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We understand that the Corresponding Author is the sole contact for the Editorial process. He/she is responsible for

communicating with the other authors about progress, submissions of revisions and final approval of proofs

## CRediT authorship contribution statement

**Antti Kallonen:** Writing – original draft, Visualization, Software, Methodology, Formal analysis, Data curation, Conceptualization. **Milla Juutinen:** Writing – review & editing, Validation. **Alpo Värri:** Writing – review & editing, Supervision, Conceptualization. **Guy Carrault:** Writing – review & editing, Supervision, Resources, Project administration, Investigation, Funding acquisition, Conceptualization. **Patrick Pladys:** Writing – review & editing, Supervision, Resources, Project administration, Investigation, Funding acquisition, Conceptualization. **Alain Beuchée:** Writing – review & editing, Resources, Investigation, Conceptualization.

## Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Antti Kallonen has patents pending on the methodology (EP21801580, US2023-0290511 and FI20205993). Other authors declare no competing interests.

## Data availability

We are unable to make the collected patient data and biosignals publicly available, as the informed parental consent was specific to use within this research project.

## Acknowledgements

JM Roué, C Flamant, A Thollot, G Favrais, G Gascoin and the teams of Angers, Brest, Nantes, Rennes, Tours, Poitiers, and the parents of the newborns for their participation in the data collection and clinical annotation.

This study received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant agreement No. 689260 (Digi-NewB project).

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ijmedinf.2024.105366.

## References

[1] C. Fleischmann-Struzek, D.M. Goldfarb, P. Schlattmann, L.J. Schlapbach, K. Reinhart, N. Kissoon, The global burden of paediatric and neonatal sepsis: a systematic review, Lancet Respir. Med. 6 (2018) 223-230.

[2] A.C. Seale, H. Blencowe, et al., Neonatal severe bacterial infection impairment estimates in South Asia, sub-Saharan Africa, and Latin America for 2010, Pediatr Res 74 (Suppl 1) (2013) 73-85.

[3] L. Saiman, Infectious diseases of the fetus and newborn infant, JAMA 307 (2012) 1865.

[4] A.L. Shane, P.J. Sánchez, B.J. Stoll, Neonatal sepsis, Lancet 390 (2017) 1770-1780.

[5] K.D. Fairchild, R.A. Polin, Early-onset neonatal sepsis, W B Saunders Company, 2010.

[6] M.S. Hammoud, A. Al-Taiar, L. Thalib, N. Al-Sweih, S. Pathan, D. Isaacs, Incidence, aetiology and resistance of late-onset neonatal sepsis: a five-year prospective study, J. Paediatr. Child Health 48 (2012) 604-609.

[7] E.H. Verstraete, K. Blot, L. Mahieu, D. Vogelaers, S. Blot, Prediction models for neonatal health care-associated sepsis: a meta-analysis, Pediatrics 135 (2015) e1002-e1004.

[8] R. Laxminarayan, P. Matsoso, S. Pant, et al., Access to effective antimicrobials: a worldwide challenge, Lancet 387 (2016) 168-175.

[9] S. Eschborn, J.-H. Weitkamp, Procalcitonin versus C-reactive protein: review of kinetics and performance for diagnosis of neonatal sepsis, J Perinatol. 39 (2019) 93-903.

[10] D.H. Hubel, T.N. Wiesel, Receptive fields and functional architecture of monkey striate cortex. 1968.
[11] K. Fukushima, Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biol. Cybern. 36 (1980) 193-202.

[12] J. Yim, K.-A. Sohn, Enhancing the performance of convolutional neural networks on quality degraded datasets. In: 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE. (2017) 1-8.

[13] M.N. van Stigt, C.R. Camps, J.M. Coutinho, H.A. Marquering, B.S. Doelkahar, W. V. Potters, The effect of artifact rejection on the performance of a convolutional neural network based algorithm for binary EEG data classification, Biomed Signal Process Control. 85 (2023) 105032.

[14] E. Persad, K. Jost, A. Honoré, et al., Neonatal sepsis prediction through clinical decision support algorithms: a systematic review, Acta Paediatr. 110 (2012) 3201-3226.

[15] C. O'Sullivan, D.-H.-T. Tsai, et al., Machine learning applications on neonatal sepsis treatment: a scoping review, BMC Infect Dis. 23 (2023) 441.

[16] S. Ramgopal, C.M. Horvat, N. Yanamala, E.R. Alpern, Machine learning to predict serious bacterial infections in young febrile infants, Pediatrics 146 (2020), https://doi.org/10.1542/peds.2019-4096.

[17] B. Huang, R. Wang, A.J. Masino, A.E. Obstfeld, Aiding clinical assessment of neonatal sepsis using hematological analyzer data with machine learning techniques, Int. J. Lab. Hematol. 43 (2021) 1341-1356.

[18] S. Mani, A. Ozdas, C. Aliferis, H.A. Varol, Q. Chen, R. Carnevale, Y. Chen, J. Romano-Keeler, H. Nian, J.-H. Weitkamp, Medical decision support using machine learning for early detection of late-onset neonatal sepsis, J. Am. Med. Inform. Assoc. 21 (2014) 326-336.

[19] A.C. Helguera-Repetto, M.D. Soto-Ramírez, O. Villavicencio-Carrisoza, S. Yong-Mendoza, A. Yong-Mendoza, M. León-Juárez, J.A. González-Y-Merchand, V. Zaga-Clavellina, C. Irles, Neonatal sepsis diagnosis decision-making based on artificial neural networks, Front Pediatr. 8 (2020) 525.

[20] D. Husada, P. Chanthavanich, U. Chotigeat, P. Sunttarattiwong, C. Sirivichayakul, K. Pengsa, W. Chokejindchai, J. Kaewkungwal, Predictive model for bacterial late-onset neonatal sepsis in a tertiary care hospital in Thailand, BMC Infect. Dis. 20 (2020) 151.

[21] L. Cabrera-Quiros, D. Kommers, M.K. Wolvers, L. Oosterwijk, N. Arents, J. van der Sluijs-Bens, E.J.E. Cottaar, P. Andriessen, C. van Pul, Prediction of late-onset sepsis in preterm infants using monitoring signals and machine learning, Crit. Care Explor. 3 (2021) e0302.

[22] Z. Peng, G. Varisco, R.-H. Liang, et al., DeepLOS: Deep learning for late-onset sepsis prediction in preterm infants using heart rate variability, Smart Health. 26 (2022) 100335.

[23] C. León, P. Pladys, A. Beuchée, G. Carrault, Recurrent neural networks for early detection of late onset sepsis in premature infants using heart rate variability. In: 2021 Computing in Cardiology (CinC). IEEE. (2021) 1-4.

[24] Z. Peng Z, Varisco G, Long X, et al. A continuous late-onset sepsis prediction algorithm for preterm infants using multi-channel physiological signals from a patient monitor. IEEE J Biomed Health Inform. 27 (2023) 550-61.

[25] S. Kurul, K. Fiebig, R.B. Flint, I.K.M. Reiss, H. Küster, S.H.P. Simons, S. Voller, H. R. Taal, Knowledge gaps in late-onset neonatal sepsis in preterm neonates: a roadmap for future research, Pediatr. Res. 91 (2022) 368-379.

[26] R. Ozdemir, R. Isguder, M. Kucuk, C. Karadeniz, G. Ceylan, N. Katipoglu, M. M. Yilmazer, Y. Yozgat, T. Mese, H. Agin, A valuable tool in predicting poor outcome due to sepsis in pediatric intensive care unit: Tp-e/QT ratio, J. Trop. Pediatr. 62 (2016) 377-384.

[27] P. Stoica, R.L. Moses, Introduction to Spectral Analysis, Pearson Education, 1997.

[28] A. Alsobhani, Speech recognition using convolution deep neural networks, J. Phys. Conf. Ser. (1973) 012166.

[29] L. Rupasinghe, Alahendra A.M.A.T. Ranathunge R A D, Perera P.S. D, Y.N. Kulathunge, Robust Speech Analysis Framework Using CNN, in: 2021 3rd International Conference on Advancements in Computing (ICAC), IEEE, 2021: pp. 485-490.

[30] D. Guiming, W. Xia, W. Guangyan, Z. Yan, L. Dan, Speech recognition based on convolutional neural networks, in: 2016 IEEE International Conference on Signal and Image Processing (ICSIP), IEEE, 2016: pp. 708-711.

[31] G. Bury, S. Leroux, C. Leon Borrego, et al., diagnosis of neonatal late-onset infection in very preterm infant: inter-observer agreement and international classifications, Int. J. Environ. Res. Public Health. 18 (2021), https://doi.org/10.3390/ijerph18030882.

[32] M. Madrid-Aguilar, M.C. López-Herrera, J. Pérez-López, et al., Implementation of NeoKissEs in Spain: a validated surveillance system for nosocomial sepsis in very low birth weight infants, Anales De Pediatría (english Edition). 91 (2019) 3-12.

[33] Pi R. Raspberry pi 3 model b. [online] (https://www.raspberrypi.org 2015. https://www.raspberrypi.com/products/raspberry-pi-3-model-b/).

[34] W.J. Youden, Index for rating diagnostic tests, Cancer 3 (1950) 32-35.

[35] A. Downey, Think DSP: Digital Signal Processing in Python, O'Reilly Media, Inc., 2016.

[36] C. Hughes, Demystifying PyTorch's WeightedRandom Sampler by example. Towards Data, published online Aug 30. (accessed Nov 10, 2023, Science (2022), https://towardsdatascience.com/demystifying-pytorchs-weightedrandomsampl er-by-example-a68aceebb452.

[37] C. Guo, G. Pleiss, Y. Sun, K.Q. Weinberger, On calibration of modern neural networks. In: Precup D, Teh YW, eds. Proceedings of the 34th International Conference on Machine Learning. PMLR, (06-11 Aug 2017) 1321-30.

[38] B. Zadrozny, C. Elkan, Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. New York, NY, USA: Association for Computing Machinery. (2002) 694-9.

[39] R. Joshi, D. Kommers, L. Oosterwijk, L. Feijs, C. van Pul, P. Andriessen, Predicting neonatal sepsis using features of heart rate variability, respiratory characteristics, and ECG-derived estimates of infant motion, IEEE J Biomed Health Inform. 24 (2020) 681-692.

[40] K. Antti, Noninvasive late-onset neonatal sepsis detection model, Anttikallonen (2023.), https://doi.org/10.5281/zenodo.10135339.
[11] K. Fukushima, Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biol. Cybern. 36 (1980) 193-202.

[12] J. Yim, K.-A. Sohn, Enhancing the performance of convolutional neural networks on quality degraded datasets. In: 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE. (2017) 1-8.

[13] M.N. van Stigt, C.R. Camps, J.M. Coutinho, H.A. Marquering, B.S. Doelkahar, W. V. Potters, The effect of artifact rejection on the performance of a convolutional neural network based algorithm for binary EEG data classification, Biomed Signal Process Control. 85 (2023) 105032.

[14] E. Persad, K. Jost, A. Honoré, et al., Neonatal sepsis prediction through clinical decision support algorithms: a systematic review, Acta Paediatr. 110 (2012) 3201-3226.

[15] C. O'Sullivan, D.-H.-T. Tsai, et al., Machine learning applications on neonatal sepsis treatment: a scoping review, BMC Infect Dis. 23 (2023) 441.

[16] S. Ramgopal, C.M. Horvat, N. Yanamala, E.R. Alpern, Machine learning to predict serious bacterial infections in young febrile infants, Pediatrics 146 (2020), https://doi.org/10.1542/peds.2019-4096.

[17] B. Huang, R. Wang, A.J. Masino, A.E. Obstfeld, Aiding clinical assessment of neonatal sepsis using hematological analyzer data with machine learning techniques, Int. J. Lab. Hematol. 43 (2021) 1341-1356.

[18] S. Mani, A. Ozdas, C. Aliferis, H.A. Varol, Q. Chen, R. Carnevale, Y. Chen, J. Romano-Keeler, H. Nian, J.-H. Weitkamp, Medical decision support using machine learning for early detection of late-onset neonatal sepsis, J. Am. Med. Inform. Assoc. 21 (2014) 326-336.

[19] A.C. Helguera-Repetto, M.D. Soto-Ramírez, O. Villavicencio-Carrisoa, S. Yong-Mendoza, A. Yong-Mendoza, M. León-Juárez, J.A. González-Y-Merchand, V. Zaga-Clavellina, C. Irles, Neonatal sepsis diagnosis decision-making based on artificial neural networks, Front Pediatr. 8 (2020) 525.

[20] D. Husada, P. Chanthavanich, U. Chotigeat, P. Sunttarattiwong, C. Sirivichayakul, K. Pengsa, W. Chokejindchai, J. Kaewkungwal, Predictive model for bacterial late-onset neonatal sepsis in a tertiary care hospital in Thailand, BMC Infect. Dis. 20 (2020) 151.

[21] L. Cabrera-Quiros, D. Kommers, M.K. Wolvers, L. Oosterwijk, N. Arents, J. van der Sluijs-Bens, E.J.E. Cottaar, P. Andriessen, C. van Pul, Prediction of late-onset sepsis in preterm infants using monitoring signals and machine learning, Crit. Care Explor. 3 (2021) e0302.

[22] Z. Peng, G. Varisco, R.-H. Liang, et al., DeepLOS: Deep learning for late-onset sepsis prediction in preterm infants using heart rate variability, Smart Health. 26 (2022) 100335.

[23] C. León, P. Pladys, A. Beuchée, G. Carrault, Recurrent neural networks for early detection of late onset sepsis in premature infants using heart rate variability. In: 2021 Computing in Cardiology (CinC). IEEE. (2021) 1-4.

[24] Z. Peng Z, Varisco G, Long X, et al. A continuous late-onset sepsis prediction algorithm for preterm infants using multi-channel physiological signals from a patient monitor. IEEE J Biomed Health Inform. 27 (2023) 550-61.

[25] S. Kurul, K. Fiebig, R.B. Flint, I.K.M. Reiss, H. Küster, S.H.P. Simons, S. Voller, H. R. Taal, Knowledge gaps in late-onset neonatal sepsis in preterm neonates: a roadmap for future research, Pediatr. Res. 91 (2022) 368-379.

[26] R. Ozdemir, R. Isguder, M. Kucuk, C. Karadeniz, G. Ceylan, N. Katipoglu, M. M. Yilmazer, Y. Yozgat, T. Mese, H. Agin, A valuable tool in predicting poor outcome due to sepsis in pediatric intensive care unit: Tp-e/QT ratio, J. Trop. Pediatr. 62 (2016) 377-384.

[27] P. Stoica, R.L. Moses, Introduction to Spectral Analysis, Pearson Education, 1997.

[28] A. Alsobhani, Speech recognition using convolution deep neural networks, J. Phys. Conf. Ser. (1973) 012166.

[29] L. Rupasinghe, Alahendra A.M.A.T. Ranathunge R A D, Perera P.S. D, Y.N. Kulathunge, Robust Speech Analysis Framework Using CNN, in: 2021 3rd International Conference on Advancements in Computing (ICAC), IEEE, 2021: pp. 485-490.

[30] D. Guiming, W. Xia, W. Guangyan, Z. Yan, L. Dan, Speech recognition based on convolutional neural networks, in: 2016 IEEE International Conference on Signal and Image Processing (ICSIP), IEEE, 2016: pp. 708-711.

[31] G. Bury, S. Leroux, C. Leon Borrego, et al., diagnosis of neonatal late-onset infection in very preterm infant: inter-observer agreement and international classifications, Int. J. Environ. Res. Public Health. 18 (2021), https://doi.org/10.3390/ijerph18030882.

[32] M. Madrid-Aguilar, M.C. López-Herrera, J. Pérez-López, et al., Implementation of NeoKissEs in Spain: a validated surveillance system for nosocomial sepsis in very low birth weight infants, Anales De Pediatría (english Edition). 91 (2019) 3-12.

[33] Pi R. Raspberry pi 3 model b. [online] (https://www.raspberrypi.org 2015. https://www.raspberrypi.com/products/raspberry-pi-3-model-b/).

[34] W.J. Youden, Index for rating diagnostic tests, Cancer 3 (1950) 32-35.

[35] A. Downey, Think DSP: Digital Signal Processing in Python, O'Reilly Media, Inc., 2016.

[36] C. Hughes, Demystifying PyTorch's WeightedRandom Sampler by example. Towards Data, published online Aug 30. (accessed Nov 10, 2023, Science (2022), https://towardsdatascience.com/demystifying-pytorchs-weightedrandomsampl er-by-example-a68aceebb452.

[37] C. Guo, G. Pleiss, Y. Sun, K.Q. Weinberger, On calibration of modern neural networks. In: Precup D, Teh YW, eds. Proceedings of the 34th International Conference on Machine Learning. PMLR, (06-11 Aug 2017) 1321-30.

[38] B. Zadrozny, C. Elkan, Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. New York, NY, USA: Association for Computing Machinery. (2002) 694-9.

[39] R. Joshi, D. Kommers, L. Oosterwijk, L. Feijs, C. van Pul, P. Andriessen, Predicting neonatal sepsis using features of heart rate variability, respiratory characteristics, and ECG-derived estimates of infant motion, IEEE J Biomed Health Inform. 24 (2020) 681-692.

[40] K. Antti, Noninvasive late-onset neonatal sepsis detection model, Anttikallonen (2023.), https://doi.org/10.5281/zenodo.10135339.