Automatic Speech Recognition for Small Data and Its Application on Cognitive Assessment
Creators: name: Chen, Liu
Publisher: name: Oregon Health and Science University
Year: 2023
Subjects: subject: automatic speech recognition, small data, mild cognitive impairment

description: Automatic speech recognition (ASR) is an essential component for building automatic cognitive assessment systems designed to monitor older adults' cognitive status. While, in the ASR field, remarkable achievements have been reported on publicly available academic datasets, two under-explored problems are important to building automatic cognitive assessment systems: ASRs' performance on aging voice and accuracy in transcribing keywords. Both problems are important to deliver high-quality transcriptions for assessment purposes. 

In this dissertation, we focus on developing transfer learning techniques/methods to build ASRs that perform well on older adults with possible cognitive impairment. Firstly, we present a transfer learning technique to improve an open-source ASR's performance on older adults (80+ years old) with limited data (i.e., about 10 hours of audio recordings). We demonstrate that the aging voice dramatically impacts an ASR's performance and that adapting the ASR with older adults' recording data through fine-tuning can improve the performance. We propose a transfer learning technique that utilizes intermediate outputs to increase the fine-tuning efficiency with limited training data. This technique achieves better performance than the standard fine-tuning. 

Secondly, we refine the DNN architecture of Wav2Vec 2.0 in order to improve the training efficiency on a small training dataset (i.e., about 100 hours of recorded audio), and a small DNN architecture (i.e., having a small number of parameters). We refine the DNN architecture with the local attention mechanism and parameter sharing. As a preliminary study, we compare the training efficiency difference between our refined DNN architecture and the original architecture through comparing their models that are trained on a small academic dataset.
Our model performs 16% better than the other.

Thirdly, we propose a transfer learning strategy that can effectively resolve the not-transcribe-filler problem while causing minor negative side effects to Whipser ASRs which are the state-of-the-art DNN-based ASRs.
Evaluating models on English and Chinese testing datasets, we show that both large training datasets and scaling the model size of ASRs do not guarantee these models transcribe the keyword accurately. Analyzing the Whisper-large-v1, we show that the acoustic encoder cannot generate general hidden representations for fillers. Moreover, we show that tuning the encoder not only increases the transcription accuracy of fillers on in-domain and out-of-domain testing sets but also moderately impacts the model's native ability of multilingual transcription.

In addition to improving ASRs when there is training data available, we also encountered a project that did not have any training data available. We utilize ASR as a tool to extract handcrafted time-based features from the animal fluency test to improve the accuracy of cognitive assessment. Combining count-based features and our proposed time-based features achieves the best performance in distinguishing older people with mild cognitive impairment from those with normal cognition.