Boosting pathology detection in infants by deep transfer learning from adult speech
Abstract
Can knowledge extracted from adult speech help improve the performance modelsdeveloped for infant cry?
This work investigates this question in the context ofpathology detection from infant cry.
The analysis of infant crying patterns to detectpathology is of interest as it opens the possibility for more accessible diagnostictools in resource-constrained settings.
Classical machine learning approaches,depending on features extracted as Mel frequency cepstral coefﬁcients, have supported the viability of the infant cry as a diagnostic input, but still fall short ofclinically acceptable performance.
The application of deep learning models hasbeen limited due to the unavailability of large infant cry corpus which are costly toacquire.
This work argues that the transfer of useful knowledge from adult speechis possible because it is driven by the same underlying physiologic process as thatof infants.
Our experiments demonstrate that on the task of predicting perinatalasphyxia from infant cry, such transfer learning provides an overall improvementof 13.5% in F1 score over a model trained from random initialization.
Introduction
This work develops a deep convolutional neural network model for cry-based pathology detectionin infants using transfer learning from adult speech.
The accurate diagnosis of conditions affectingnewborns based solely on their cry is important for many practical reasons.
It could result to low-cost,fast, accessible diagnostic tools in resource-limited regions of the world.
We tackle the speciﬁc caseof predicting perinatal asphyxia, the inability of a newborn to establish regular breathing in the periodafter birth.
Clinical researchers have shown that respiratory conditions like asphyxia alters infantcrying patterns since both speech and breathing are coordinated by the same regions of the brain, andnewborns have not developed voluntary control of either function (1).
Transfer learning (2) (3) enables the use of knowledge learned from a task in one domain (source) tosolve a task in a different domain (target).
This is especially useful where the cost of acquiring labelleddata in the target domain is very expensive or where the distribution is constantly changing. Transferlearning relaxes the assumption of many machine learning algorithms that the data distributionin the training and test data must be independently and identically distributed.
Transfer learning,especially based on deep learning models, has seen prominent success in image, and natural languageprocessing.
The use of adult speech is interesting for our problem since large corpuses are freely available.
Whereas infant cry databases are signiﬁcantly fewer, and annotating of cries is a costly procedurethat typically must be done in the context of a clinical study.
Though it might not seem intuitiveupon initial consideration, we believe that there is common ground for knowledge transfer from adultto newborn speech.
In general, adults have voluntary control over their speech and the way theyexercise it, in terms of accent and speaker mannerisms, has also been inﬂuenced by the environment.
However, the same underlying physiologic mechanism for sound production exists between the brain,vocal chords and respiratory system in both adults and infants.
One can then argue that there exists alatent representation of adult speech which captures this mechanism.
If our models can learn this,then it is reasonable to expect that knowledge can be transferred.
To test this hypothesis on our target task of predicting perinatal asphyxia based on infant cry, we selectthe problem of keyword spotting as source task (4).
Keyword spotting involves the identiﬁcationof commands such as "go", "come", "up", "down", etc from spoken audio.
Keyword spottingmodels depend on the fact that each word is characterized by a different time-frequency signature.
Incidentally, this is also the fundamental assumption of how the cries of asphyxiating babies differfrom normal ones (1) (5).
Furthermore, requirements for keyword spotting models align well withthose of cry-based pathology detection models, namely: small computational footprint, dependenceon short audio segments, and speaker independence (4).
In this work, we take a state-of-the-art residual network for keyword spotting.
We train it for thatsource task and attempt to transfer the knowledge gained to serve as priors for our target task ofdetecting perinatal asphyxia.
We ﬁnd that our knowledge transfer model performs signiﬁcantly better- 18.9% higher sensitivity (or recall), 1.9% higher speciﬁcity and 10% higher precision - than thesame residual network trained without transfer learning.
The overall improvement in F1 score was13.5%.
Tasks2.1 Source task
The domain of adult speech is large with many corpuses tailored to different tasks.
We set 3 mainconstraints for a desirable source task: 1) it has to be analogous in some form to our ultimate task ofpathology detection, 2) there should be freely available corpuses for the task to facilitate pre-training,and 3) there should be openly veriﬁable benchmarks to enable validation pre-trained models.
Weselect the problem of keyword spotting as our source task.
This task is analogous to our task ofpredicting the presence or not of asphyxia in the sense that it relies on learning the differences in thetime-frequency signature of each word of interest.
One would expect the model to focus on extractingsimilar kinds of high level features.
There also exists a reasonably large dataset for this task - Googlespeech commands (6) for which strong benchmarks (6) (7) have been set.
2.1.1 Corpus
Google speech commands dataset (6) is an audio corpus collected to aid the training and evaluation ofkeyword spotting models.
It contains approximately 65,000 recordings of utterances of 30 keywordsfrom thousands of persons.
The recordings are 1-second long, 16-bit PCM-encoded WAV audioﬁles at 16kHz sampling frequency.
Typically, work with this dataset has focussed on classifying 10keywords that would be "useful as commands in IoT or robotics applications" (6) (7): "Yes", "No","Up", "Down", "Left", "Right", "On", "Off", "Stop", and "Go".
The remaining words are lumpedtogether to make an unknown words category.
2.1.2 Model
We select a CNN architecture which has achieved state of the art performance on the speech commands dataset.
To this end, we adopt the ’res8’ model from (7).
The model takes as input MFCCrepresentation of an input audio, transforms it through a collection of 6 residual blocks, ﬂanked oneither side by a convolutional layer, and outputs a 12-way softmax of either of the 10 keywords or unknown or silence .
As seen in Figure 2.1.2, each residual block consists of 2 alternating convolutionaland batch normalisation layers.
In the work of (7), ’res8’ achieves an accuracy of 94%, 1% less thanthe best model ’res15’, but using only half the no of parameters and 30 times fewer multiplies.
2.2 Target task
Our target task is the detection of perinatal asphyxia from the newborn cry.
We develop and evaluateour models using the Chillanto infant cry database (5).
The database contains 2268 audio recordingsof normal, asphyxiated and deaf infants.
Recordings are 1 second long and at sampling frequencies of between 8kHz to 16kHz.
The work of (5) experimented with audio representations as linear predictivecoefﬁcients (LPC) and mel frequency cepstral coefﬁcients (MFCC).
Training a time delay neuralnetwork as classiﬁer, they achieved best precision and recall in predicting asphyxia of 72.7% and68% using MFCC representation.
Building on this work, (8) later showed that precision and recallcould be improved to 73.4% and 85.3% by using support vector machines (SVM) which have morestable convergence properties on small but high-dimensional datasets.
Methods
3.1 Audio preprocessing
Audio samples are ﬁrst downsampled to 8kHz for consistency in the input dimension.
After downsampling, similar to (7), the audio is converted from 1D to 2D through a sequence of steps: spectrogramswere computed for overlapping window sizes of 30ms with a 10ms shift, and across 40 Mel bands.
Only frequency components between 20 and 4000 Hz were considered.
The discrete cosine transformis then applied to the spectrogram output to compute the MFCCs.
The resulting values in each frameis stacked in time to form a spatial, 2D representation of the MFCCs.
3.2 Model pre-training and transfer learning
In order to obtain a pre-trained model for transfer, we train res8 on the Google speech commandsdataset to classify the 10 typical keywords, unknown and silence, using as input the MFCC "images".
As aim was to obtain a model with equivalent accuracy as reported in (7), we used their opensource implementation (9) available under an MIT license, and employed the same set of traininghyperparameters.
We create a new instance of the res8 model for our task of classifying the presence or not of asphyxia,replacing only output layer to be a binary classiﬁer.
To transfer the learned model for our target task,we initialise res8 using the weights from all but the fully-connected output layer.
We then train themodel from this start point.
Considering that we had a high class imbalance of 3:1 (normal:asphyxia),we used a weighted sampling procedure to ensure that each batch of training data passed to the modelcontained roughly the same proportion of both classes.
As we had just under a thousand trainingexamples, we also applied data augmentation via time-shifting before transforming an audio into MFCCs.
3.3 Baselines
We implement and compare the performance of our transfer learning model with 2 baseline modelsfor predicting asphyxia.
One is a model which trains a radial basis function SVM on MFCC inputs, [Table] based on the work of (8).
As our second baseline, we train the res8 model using random initialisationof weights, i.e., without transfer learning (res-no-transfer).
This model is trained in the same manneras res8-transfer model described in section 3.2.
The only difference being that no pre-trained weightswere used.
[Table]
Experiments
4.1 Setup
There were a total of 1389 infant cry samples (1049 normal and 340 asphyxia) in the chillanto dataset.
The samples were split proportionally into training, validation and test set by a ratio of 60:20:20,ensuring that samples from the same patients were placed in the same set.
We trained our modelsto select the point at which accuracy on the validation set was highest.
In addition to accuracy,we tracked other relevant metrics including: recall or sensitivity , the fraction of asphyxia samplescorrectly identiﬁed; speciﬁcity , the fraction of normal samples correctly identiﬁed; precision , thefraction of asphyxia predictions that were correct; and F1 score , the harmonic mean of precision andrecall.
4.2 Results
We trained the 2 baseline models: an SVM trained on ﬂattened MFCC features and a res8 modeltrained from scratch without any pre-training (res8-no-transfer).
Both models were trained to identifyasphyxia and normal samples from chillanto database.
Results are shown in Table 1.
The SVMattains a sensitivity in detecting asphyxia of 81.1% which is 7.5% higher than that of res8-no-transfer.
Though res8-no-transfer performs slightly better on the other metrics of speciﬁcity and precision, the
SVM is the overall better model based on the F1 score of 66.7%.
To carry out our transfer learning experiment, we ﬁrst pre-train a res8 model on the (downsampled)
Google speech commands dataset.
Our model achieved the same test set accuracy of 94.4%.
Usingthe learned weights from this model, we initialized a new res8 instance and further trained it toclassify normal and asphyxiated babies from the chillanto dataset.
We tuned hyperparameters usingour validation set.
This model was trained for 30 epochs using a stochastic gradient descent (SGD)optimizer with start learning rate of 0.001, a schedule to a rate of 0.0001 after 15 epochs, a ﬁxedmomentum of 0.9, and batch size of 32.
Hyperparameters were primarily derived from (6) and (7),but modiﬁed where necessary to suit chillanto dataset.
We minimized the hinge loss as we found thatit resulted to better accuracy on the validation set than cross-entropy loss.
The results of this model(res8-transfer) are summarised in table 1.
It performs better than the others on all metrics, achievingsensitivity and speciﬁcity of 92.5% and 91.5%, respectively at a precision of 69%.
Discussion
We proposed and empirically tested the hypothesis that adult speech and infant cry are driven bythe same physiologic mechanism whose elements can be captured in model.
Such transfer learning,when compared to an equivalent deep model trained without, led to improvements of almost 20% insensitivity, 2% in speciﬁcity and 10% in precision of detecting perinatal asphyxia.
The precision of 69%, despite the high sensitivity, indicates that there remains a fair number of falsepositives.
While future models should aim to address this, the false identiﬁcation of a baby as havingasphyxia is not clinically dangerous since it implies that the infant may receive additional care suchas increase oxygen support and neuro-protective strategies.