男1俳氏 4

1 (課程博士(英文))Date of submission (montl) day, year)

Jan 11,2019.

Department ofComputer science andSludcn11D Numbcr

En芸ineerlng
Pplical】1's namc

123346

TolnoyosiNdba
Supe"'isolsHitoshilsal)ara

Hiroshiseki

Abst"act (Doctor)
THle oflhesis

Rapid adaptation ofdeep neuralnetwoTk for speech Yec0曾nition(深層学習に基づく音声認識システムの高速適応)

Approx.80o words
In recent yeω'S, automatic speech recognition (ASR) has become a populaTized techn010gy for us
The ASR system models conversion from speech to text by usin宮 trainin号 data. During operation,the ASR system recognizes input speech uttered by various speakers which are recotded il) various

Cond北ions. However, since the ASR system cannotidenti6,speaker characta'istics and back容roundnoise inforn)ation in advance, there is an acoustic mismatch between speech data used for thedevelopment ofthe ASR system (trainin宮 data) and the il)put speech (evaluation data) which leads
to performance degradation. Therefore, recover/improvement of rec0宮nition performance isrequired to accurately rec0宮nize the speech ofvarious speakers uttered in various conditionsAdaptation is one approach to suppress this acoustic mismatch. This metbod fb'st c011ects speech

data (adaptation data) under the same condition as the evaluation data, and then the ASR system
is tuned to match the evaluation data based on the adaptation data. However, the data c0Ⅱection

dependin牙 on each speaker and each condition takes tedious time and leads to the poor user
experience.1n other words, users have to utter additional words before the recognition ofintended
Words which leads to undeSⅡ'ed hi曾h latency. TheNfの'e,北 is imPの,tant to adapt the ASR system
rapidly and robustly usin客 a sma11 amount of adaptation data.1n this research, we focus on therapid adaptation of the ASR system based on a sn)a11 Size adaptation data and propose two
adaptation methods
Auxiliary'feature based adaptation method extracts information which degrades the recognition

Performance, e.g., speaker characteristic and background noise, and uses it as an auxiliary feature.
However, earlier works assume the availability of speech from several ten seconds to several
minutes. A widely used speaker representation ca11ed i・vector requires more than 5.o seconds forrobust estimation ofspeaker characteristic. Therefの'e, the usa今e ofi'vector is not applicable to therec0牙nition of a shoTt time utterance such aS 1ζeyword and key'phrase.1n this thesis, we propose a
Speaker representation, speaker・dass information, which is defined as a set ofsimilarities between

Pre、defined speech clusters and the input speech. The avera宮e word error rate (WER) ofthe model
Without the input of speaker class information waS 11.2%. The ＼入rER of our proposed model was
decreased t0 10.4% and the relative error reduction rate of 7.0% was obtail〕ed. These resultsdemonstrated that speaker class, which was estimated from only the o.5 Second in the utterance,

Provided important information to suppress the diverS北y of speakers, and it is applicable to therec0宮nition of short time utterances, i.e., speech retrieval, speech assistance, and speech command
Input.

Model adaptation, another commonly used adaptation method, re'estimates parameters of theASR system usin晉 the 倉iven adaptation data for the suppression of acoustic mislnatch. Earlier
Works on model adaptation update sub'modules of neural network arch北ecture and introduce
Various restriction methods and regU1雛'ization methods for prevention of an oV飢'fittin曾 Problem.
However, there is a tTade、offbetween complexity and expressiveness.1n other words, the update ofa large number ofparameters causes the ova'fitting problem or requirement of much adaptation

data and the eX訂'eme restriction (10w expressiveness) n)akes it difficult to adapt to the 套iven
adaptation data. Therefore, it is important to maximize expressiveness while lninimizing thenun〕ber of free parameters to robustly update the ASR system usin牙 the sma11 Size adaptation
data,1n this thesis, we proposed a neural network based filterbank ]ayer which minimi乞es thenumber offree parameters while maximizing the expressiveness by focusing on a human auditory
System and physicalcharacteristics ofspeakers.＼入le conducted experiments for speaker adaptationand noise adaptation, and showed that the proposed adaptation method obtained better
recognition P飢'formance than the conventional adaptation methods under the scenario that theavailable adaptation data is limited. By adaptin宮 the filt母,bank layer using 15 Utterances, WER

Was improved from 12.1% t0 11.2%, and a word error reduction rate (WERR) of 7.4% was obtained.
TI)is wERR was better than tl)e unadapted model at a si牙nificance levelofo.005 Under a statistical
Sign test. These results showed that tl)e adjustment ofthe filter shapes can handle the diversity ofSpeakers. T}〕e rec0宮nition performance ofour proposed n)odel was better than the other adaptation
methods, although other adaptation methods also showed performance improvements.
Through the above research, we developed the easy"to'use robust adaptation methods for theASR system by reducin宮 the required speech duration and the cost ofthe data c011ection procedure.