Boosting pathology detection in infants by deep transfer learning from adult speech
Abstract
Can knowledge extracted from adult speech help improve the performance modelsdeveloped for infant cry?
This work investigates this question in the context ofpathology detection in newborns.
The analysis of infant crying patterns to detectpathology is of interest as it opens the possibility of more accessible diagnostictools in resource-constrained settings.
Classical machine learning approachesleveraging features extracted as Mel frequency cepstral coefﬁcients, have supportedthe viability of the infant cry as a diagnostic input, but performance is not yetat a level of clinical utility.
The application of deep learning models has beenlimited due to the unavailability of large infant cry databases which are costly toacquire.
This work argues that the transfer of useful knowledge from adult speechis possible because it is driven by the same underlying physiologic process as thatof infants.
Our experiments demonstrate that on the task of predicting perinatalasphyxia from infant cry, such transfer learning provides an overall improvementof 13.5% in F1 score over a model trained from random initialization.
Introduction
This work develops a deep convolutional neural network model for cry-based pathology detectionin infants using transfer learning from adult speech.
The accurate diagnosis of conditions affectingnewborns based solely on their cry is important for many practical reasons.
It could give rise tolow-cost, early and accessible diagnostic tools in resource-constrained settings.
We tackle the speciﬁccase of predicting perinatal asphyxia, the inability of a newborn to establish regular breathing in theperiod after birth.
Clinical researchers have shown that respiratory conditions like asphyxia altersinfant crying patterns since both speech and breathing are coordinated by the same regions of thebrain, and newborns do not have voluntary control of either function (1).
Transfer learning (or knowledge transfer) (2) (3) enables the use of knowledge learned from a taskin one domain (source) to solve a task in a different domain (target).
This is especially useful whenthe cost of acquiring labelled data in the target domain is very expensive or if the data distribution isconstantly changing.
Transfer assumption independently and identically distributed especially
Infant cry databases are scarce, and the acquisition/annotation of cries is costly, as it typically must be done in the context of a clinical study. The use of adult speech for transfer learning is thus interesting since large corpuses are freely available.
It may not seem intuitive upon initial consideration but we believe there is common ground for knowledge transfer from adult to newborn speech.
In contrast to newborns, adults have voluntary control over their speech. Furthermore, the way they exercise this control, in terms of accent and speaker mannerisms, has been inﬂuenced over several years by the environment.
Despite this, the same underlying physiologic mechanism for sound productionremains between the brain, vocal chords and respiratory system in both adults and infants.
One canargue then that there exists a latent representation of adult speech which captures this mechanism.
Ifour models can learn this, then it is reasonable to expect that knowledge can be transferred.
To test this hypothesis on our target task of predicting perinatal asphyxia based on infant cry, we selectthe problem of keyword spotting as source task.
Keyword spotting (4) involves the identiﬁcationof commands such as "go", "come", "up", "down", etc from spoken audio.
Models for keywordspotting depend on the fact that each word is characterized by a different time-frequency patterns(or signature).
Incidentally, this is also the fundamental assumption of how the cries of asphyxiatingbabies differ from normal ones (1) (5).
Furthermore, requirements for keyword spotting models alignwell with those of cry-based pathology detection models, namely: small computational footprint,dependence on short audio segments, and speaker independence (4).
In this work, we take a state-of-the-art residual network for keyword spotting.
We train it for thatsource task and attempt to transfer the knowledge gained to serve as priors for our target task ofdetecting perinatal asphyxia.
We ﬁnd that our knowledge transfer model performs signiﬁcantly better- 18.9% higher sensitivity (or recall), 1.9% higher speciﬁcity and 10% higher precision - than thesame residual network trained without transfer learning.
The overall improvement in F1 score was13.5%.
Tasks2.1 Source task
The domain of adult speech is large with many corpuses tailored to different tasks.
We set 3 mainconstraints for a desirable source task: 1) it has to be analogous in some form to our ultimate task ofpathology detection, 2) there should be freely available corpuses for the task to facilitate pre-training,and 3) there should be openly veriﬁable benchmarks to enable contextual evaluation pre-trained ofmodels.
We select the problem of keyword spotting as our source task.
This task is analogous to ourtask of predicting the presence or not of asphyxia in the sense that it relies on learning the differencesin the time-frequency signature of each word of interest.
One would thus expect a neural model tofocus on extracting similar kinds of high level features.
There also exists a reasonably large datasetfor this task - Google speech commands (6) for which strong benchmarks (6) (7) have been set.
2.1.1 Corpus
Google speech commands dataset (6) is an audio corpus collected to aid the training and evaluation ofkeyword spotting models.
It contains approximately 65,000 recordings of utterances of 30 keywordsfrom thousands of persons.
The recordings are 1-second long, 16-bit PCM-encoded WAV audioﬁles at 16kHz sampling frequency.
Typically, work with this dataset has focussed on classifying 10keywords that are "useful as commands in IoT or robotics applications" (6) (7): "Yes", "No", "Up","Down", "Left", "Right", "On", "Off", "Stop", and "Go".
The remaining words are lumped togetherto make an unknown words category.
2.1.2 Model
We select a CNN architecture which has state of the art performance on the speech commands dataset.
To this end, we adopt the ’res8’ model from (7).
The model takes as input MFCC representation ofan input audio, transforms it through a collection of 6 residual blocks, ﬂanked on either side by aconvolutional layer, and outputs a 12-way softmax of either of the 10 keywords or unknown or silence(Figure 1).
In the work of (7), ’res8’ achieves an accuracy of 94%, which is 1% less than the bestmodel ’res15’, but uses only half the number of parameters and 30 times fewer multiplies.
2.2 Target task
Our target task is the detection of perinatal asphyxia from newborn cry.
We develop and evaluateour models using the Chillanto infant cry database (5).
The database contains 1089 1-second longaudio recordings of normal and asphyxiated infants at sampling frequencies of between 8kHz to16kHz.
(5) experimented with audio representations as linear predictive coefﬁcients (LPC) and mel frequency cepstral coefﬁcients (MFCC).
Training a time delay neural network as classiﬁer, theyachieved best precision and recall in predicting asphyxia of 72.7% and 68% using MFCCs.
Buildingon this work, (8) later showed that precision and recall could be improved to 73.4% and 85.3% byusing support vector machines (SVM) which have more stable convergence properties on small buthigh-dimensional datasets.
Methods3.1 Audio preprocessing
Audio samples are ﬁrst downsampled to 8kHz for consistency in the input dimension.
After downsampling, similar to (7), the audio is converted from 1D to 2D through a sequence of steps: spectrogramswere computed for overlapping window sizes of 30ms with a 10ms shift, and across 40 Mel bands.
Only frequency components between 20 and 4000 Hz are considered.
The discrete cosine transformis then applied to the spectrogram output to compute the MFCCs.
The resulting values in each frameis stacked in time to form a spatial, 2D representation of the MFCCs.
3.2 Model pre-training and transfer learning
In order to obtain a pre-trained model for transfer, we train res8 on the Google speech commandsdataset to classify the 10 typical keywords, unknown and silence, using the MFCC "images" as input.
As the aim was to ﬁrst obtain a model with equivalent accuracy as reported in (7), we built upon theiropen-source implementation (9) and employed the same set of training hyperparameters.
We further create a new instance of the res8 model, res8-transfer, for our task of classifying thepresence or not of asphyxia, replacing only output layer to be a binary classiﬁer.
To transfer thelearned model for our target task, we initialise res8-transfer using the weights from all but thefully-connected output layer of the pre-trained model.
We then train the model from this start point.
Considering that we had a high class imbalance of 3:1 (normal:asphyxia), we used a weightedsampling procedure to ensure that each batch of training data passed to the model contained roughlythe same proportion of both classes.
As we had just under a thousand training examples, we alsoapplied data augmentation via time-shifting before transforming an audio into MFCCs.
3.3 Baselines
We implement and compare the performance of our transfer learning model with 2 baseline modelsfor predicting asphyxia.
One is a model which trains a radial basis function SVM on MFCCrepresentations of the chillanto database, similar to (8).
The other is a res8 model trained on MFCCrepresentations of the chillanto database, but using random initialization of weight (res-no-transfer).
[Table]
Experiments
[Table]
4.1 Setup
There were a total of 1389 infant cry samples (1049 normal and 340 asphyxia) in the chillanto dataset.
The samples were split proportionally into training, validation and test set in the ratio 60:20:20,ensuring that samples from the same patients were placed in the same set.
We trained our modelsto select the point at which accuracy on the validation set was highest.
In addition to accuracy,we tracked other relevant metrics: recall or sensitivity , the fraction of asphyxia samples correctlyidentiﬁed; speciﬁcity , the fraction of normal samples correctly identiﬁed; precision , the fraction ofasphyxia predictions that were correct; and F1 score , the harmonic mean of precision and recall.
4.2 Results
We trained the 2 baseline models: an SVM trained on ﬂattened MFCC features and a res8 modeltrained from scratch without any pre-training (res8-no-transfer).
Both models were trained to identifyasphyxia and normal samples from chillanto database.
Results are shown in Table 1.
The SVM 1attains a sensitivity in detecting asphyxia of 81.1% which is 7.5% higher than that of res8-no-transfer.
Though res8-no-transfer performs slightly better on the other metrics of speciﬁcity and precision, the
SVM is the overall better model based on the F1 score of 66.7%.
To carry out our transfer learning experiment, we ﬁrst pre-train a res8 model on the (downsampled)
Google speech commands dataset.
Our model achieved the same test set accuracy of 94.4%.
Usingthe learned weights from this model, we initialized a new res8 instance and further trained it toclassify normal and asphyxiated babies from the chillanto dataset.
We tuned hyperparameters usingour validation set.
This model was trained for 30 epochs using a stochastic gradient descent (SGD)optimizer with start learning rate of 0.001, a schedule to a rate of 0.0001 after 15 epochs, a ﬁxedmomentum of 0.9, and batch size of 32.
Hyperparameters were primarily derived from (6) and (7),but modiﬁed where necessary to suit chillanto dataset.
We minimized the hinge loss as we foundthat it resulted to better accuracy on the validation set than cross-entropy loss.
The performance ofthis model (res8-transfer) is summarized in table 1.
It performs better than the others on all metrics,achieving sensitivity and speciﬁcity of 92.5% and 91.5%, respectively at a precision of 69%.
Discussion
We proposed and empirically tested the hypothesis that adult speech and infant cry are driven bythe same physiologic mechanism whose elements can be captured in model.
Such transfer learning,when compared to an equivalent deep model trained without, led to improvements of almost 20% insensitivity, 2% in speciﬁcity and 10% in precision of detecting perinatal asphyxia.
The precision of 69%, despite the high sensitivity, indicates that there remains a fair number of falsepositives.
While future models should aim to address this, the false identiﬁcation of a baby as havingasphyxia is not clinically dangerous since it implies that the infant may receive additional care suchas increase oxygen support and neuro-protective strategies.