INTERSPEECH 2016
September 8–12, 2016, San Francisco, USA

Unsupervised Joint Estimation of Grapheme-to-phoneme Conversion Systems
and Acoustic Model Adaptation for Non-native Speech Recognition
Satoshi Tsujioka, Sakriani Sakti, Koichiro Yoshino, Graham Neubig, Satoshi Nakamura
Graduate School of Information Science, Nara Institute of Science and Technology (NAIST), Japan{tsujioka.satoshi.tl4, ssakti}@is.naist.jp

Abstract

ation dictionary, which we focus on here, early work proposed
knowledge-based approaches, which try to find the best pronunciation transformation rules using phonological and linguistic knowledge [9–16]. Data-driven approaches to non-native
speech recognition further build upon rule-based approaches
by using rule-based methods to generate multiple pronunciation candidates, then using databases of real acoustic evidence
to verify which of the pronunciation variations are most appropriate [17–25]. This allows for more accurate estimation of theprobability of each pronunciation variation, allowing for more
effective use of the pronunciation dictionary at recognition time.

Non-native speech differs significantly from native speech, often resulting in a degradation of the performance of automatic
speech recognition (ASR). Hand-crafted pronunciation lexicons
used in standard ASR systems generally fail to cover non-native
pronunciations, and design of new ones by linguistic experts istime consuming and costly. In this work, we propose acoustic data-driven iterative pronunciation learning for non-native
speech recognition, which automatically learns non-native pronunciations directly from speech using an iterative estimation
procedure. Grapheme-to-Phoneme (G2P) conversion is used to
predict multiple candidate pronunciations for each word, occurrence frequency of pronunciation variations is estimated fromthe acoustic data of non-native speakers, and these automatically estimated pronunciation variations are used to perform
acoustic model adaptation. We investigate various cases suchas learning (1) without knowledge of non-native pronunciation,and (2) when we adapt to the speaker’s proficiency level. In
experiments on speech from non-native speakers of various levels, the proposed method was able to achieve an 8.9% average
improvement in accuracy.
Index Terms: Non-native Speech Recognition, Iterative Pronunciation Learning, Lexical Modeling, Probabilistic Pronunciation ModelingIn this work, we focus on two weaknesses of the current
approaches to data-driven non-native pronunciation modeling.
First, previous work has mainly focused on using non-native
data as a way to adjust probabilities of candidates built using either rule-based methods, or estimated from hand-crafted native
or non-native pronunciation dictionaries. Rule-based systems
and non-native dictionaries require significant effort to create,
and data from native speakers is not guaranteed to cover pronunciation phenomena witnessed in non-native speakers. Second, previous work on non-native pronunciation estimation has
not covered iterative adaptation of both the dictionary and theacoustic model.
In this paper, we propose a method for unsupervised datadriven iterative pronunciation learning that has the potential
to resolve both of these problems. Specifically, we adapt amethod [26] that uses a trainable G2P converter [27] to generate multiple latent pronunciation candidates and estimate theiroccurrence frequency from acoustic data to update the pronunciation lexicon. In order to solve the problem of requiring resources specific to the non-native accent in question, our model
creates a seed lexicon directly from acoustic evidence by performing phoneme recognition on non-native speech, and taking
phoneme-to-word alignment using Levenshtein distance. In addition, our method is able to perform iterative training to update
both the pronunciation dictionary and we adopt SAT includedin the iterative training for the acoustic model training, leading
to further improvements. The results reveal that the proposed
method is able to achieve about 8.9% improvement on averagein accuracy for various non-native speakers.

1. Introduction
Due to globalization, it is more and more common for speakers
to communicate in languages that are not their mother tongue.
This need for speakers to obtain non-native language skills has
led to development of CALL (Computer Assisted Language
Learning) systems that evaluate learners’ pronunciation andgrammar by using ASR techniques [1]. In these systems, it isnecessary to recognize non-native speech accurately.
However, non-native ASR is difficult because the featuresof non-native speech differ from those of native speech in several ways. Perhaps the most prominent is pronunciation, whichis affected by the accent of the speaker’s mother tongue or dialect. In addition, non-native speech corpora that can be used
for supervised training are limited because it is very expensive
and time consuming to collect a large amount non-native speechwith the corresponding transcriptions [2–4].
Given this background, there are a number of works on
methods to adapt ASR models to non-native speech, which generally focus on adaptation of the acoustic model or the pronunciation dictionary. Methods for acoustic modeling range fromstandard methods for speaker adaptive training (SAT) such as
maximum a posteriori (MAP) [5] and maximum likelihood linear regression (MLLR) [6] specifically tailored for the variancefound in non-native speech [7, 8]. In models of the pronunci-

Copyright © 2016 ISCA

2. Probabilistic Pronunciation Model [26]2.1. Formulation
A conventional ASR system obtains the optimal word sequenceŴ given the acoustic observations X.
Ŵ = argmax P(X|W)P(W).
(1)
W

3091

http://dx.doi.org/10.21437/Interspeech.2016-919

Table 1: Example of pronunciation learning using the probabilistic pronunciation model (threshold: 0.2).

In the probabilistic pronunciation model, Equation (1) can beconverted to:P(X|B)P(B|W),(2)
Ŵ = argmax P(W)W

WordbathroomB∈ΨW

where B = {b1 , ..., bn } denotes a valid pronunciation sequence
for word sequence W = {w1 , ..., wn }, P(B|W) denotes its
probability. bi is the pronunciation of word wi ɽΨW denotes
the set of all the possible pronunciation sequences of word sequence W. This is decomposed as the product of pronunciation
probabilities of each word:(3)
P(B|W) = P(b1 |w1 ) · · · P(bn |wn ).
When each word has multiple latent pronunciation candidates,
they are given weights:
P(bi = pj |wi ) = θij , j = 1, . . . , Ji
(4)
Ji


θij = 1,academic

troubleInitialb aa th r uw mb ae th r uw mb et dh r uh mb ey dh r uw mb ey th r uw mae k ah d ah m ih k
ae k ah d eh m ih k
ae k ah d eh m iy k
ah k ae d ah m iy k
ah k ah d eh m ih kt r ah b ah lt r ah b ah l iy
t r ah b ah l n
t r aw b ah lt r aw b ah l n

θ
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2

Updatedb ae th r uw mθ1.0

ae k ah d eh m iy k
ah k ah d eh m ih k

0.58
0.42

t r ah b ah lt r aw b ah l

0.63
0.37

(5)
word. When updating the pronunciation weight, prune
those pronunciations whose weight is below a threshold.
These pronunciation candidates are defined as Updated,also shown in Table 1.

j=1

where Ji is the number of alternate pronunciations of wi , and pj
denotes one of those pronunciations with a weight θij .
2.2. Updating Pronunciation Probabilities

6. We also examine the following optional steps:- Go to step 3 to re-train the acoustic model with theupdated lexicon and re-recognize.
- Go to step 7 to re-train the G2P converter using the
updated lexicon.

Generally, a G2P converter is used to estimate pronunciation
candidates for unseen words. However, G2P converters are not
perfect, sometimes predicting incorrect candidates, which candegrade recognition accuracy. In order to solve this problem,
Lu et al. proposed a method to obtain correct pronunciation
candidates using real acoustic evidence by acoustic data-driven
iterative pronunciation learning [26]. This method is called “Supervised G2P” in this paper. The overview of this iterative pronunciation learning method is as follows (Fig. 1):

3. Iterative Pronunciation Learning
for Non-Native Speakers
3.1. Unsupervised Data-driven G2P

1. Train the G2P converter on pronunciation candidates inan expert seed lexicon.

As mentioned in the previous section, the supervised datadriven method requires a seed lexicon. This lexicon will either be a lexicon containing native pronunciations, which may
not have the appropriate coverage for non-native speakers, or arule-based or manually created non-native lexicon that needs tobe hand-crafted using the knowledge of linguistic experts.
In this work, we propose a method that is both effective
and flexible: using phoneme recognition results of non-native
speech to train the G2P converter (Shown in Fig. 2) anduse of SAT of acoustic models in the iterative training process. This method has the benefit that it can generate multiple latent pronunciation candidates for the G2P converter thatare not constrained by a native seed lexicon, and without explicit knowledge of non-native pronunciation used in rule-based
methods. Phoneme recognition can be performed with any
acoustic model, but once results are created it is necessary toalign the recognized phonemes to words in the transcript. Thisis done by selecting an alignment with the smallest Levenshtein

2. Use the G2P converter generates multiple latent pronunciation candidates (5 pronunciations for each word)and give equal pronunciation weight to each candidate.
These pronunciation candidates are defined as Initial, an
example of which is shown in Table 1.
3. Train the acoustic model using the Initial lexicon.
4. Recognize training speech and obtain the pronunciation
used for each word.
5. Update the pronunciation weights by calculating thenumber of appearances of each pronunciation the recognized word divided by each appearance of the recognized

Word - phonealignment

Non-native speech
PhonemeRecognition

Training
ACADEMIC

G2P
converter

ACADEMIC
AE K AH D EH M IY K
AH K AH D EH M IH K

ACADEMICAH K AH D EH M IY K
AE K AH D IY M IY K
EH K AH D EH M IY K
AH K AH D IY K IY K UW
EY K AH D EH M IY K UW

Figure 2: Overview of unsupervised G2P.

Figure 1: Overview of iterative pronunciation learning [26].

3092

LOW
WER(%)

Phonemerecognition
Beginner

Intermediate

Advancedacademic
AH K AH D EH M IH K UW

academic
AH K AH D EH M IH K

academic
AE K AH D EH M IY K

Training

G2P

Training

Training

G2P

G2P

WER(%)

distance between the recognized phonemes and the estimatedphonemes for each word in the transcript, then assigning therecognized phonemes to the corresponding word of the transcript phoneme to which they are aligned. This method is called“Unsupervised G2P” in this paper.
3.2. Unsupervised Data-driven G2P on Matched Proficiency Level/Language

HIWIRE
Test data
ERJ
HIWIRE

LOW
MID
HIGH

MID

UpdatedLexiconRe-trainAM

Re-trainUpdated
Re-train
G2P
LexiconAM
Supervised Native G2P + SAT
MID Un-sup G2P + SAT

WER(%)

60
55
50
45
40
35
30
25
20
15
Train G2P

HIGH

UpdatedLexiconRe-trainAM

Re-trainUpdated
Re-train
G2P
LexiconAM
Supervised Native G2P + SAT
HIGH Un-sup G2P + SAT

15,275 utterances for training and 7,952 utterances for testing.
Moreover, this speech data is scored for fluency from 1.0 to 5.0
by five native English speakers. The criteria for score evaluationare (1) phoneme generation, (2) rhythm generation, and (3) intonation generation. We take the average of three criteria of score
evaluation for each speaker and we divided three English proficiency levels as follows: LOW (scored from 1.0 to 2.5), MID(scored from 2.5 to 3.5) and HIGH (scored from 3.5 to 5.0). For
our multi-lingual database, we use the HIWIRE (Human Input
that Works In Real Environments) database. This is a database
designed to be used as a tool for development and test of speechprocessing and recognition techniques dealing with robust nonnative speech recognition, and contains data from the aforementioned languages. Specifically, it contains 900-3000 utterancesfrom French, Greek, Italian, and Spanish.
We use the Kaldi speech recognition toolkit [30] for training and testing of the ASR system. For the acoustic model training data, we use WSJ and part of ERJ corpus (not including thetest data). We use a context-dependent GMM-HMM acoustic
model with 3-states per phoneme. The acoustic features used
39 dimensional MFCC+Δ + ΔΔ, and we also perform feature transformation using LDA (Linear Discriminative Analysis) and MLLT (Maximum Linear Likelihood Transformation)
to reduce feature dimensionally. In addition, we evaluate systems that perform SAT using fMLLR (feature-space Maximum
Linear Likelihood Regression) [31] to adapt the acoustic model

Table 2 shows the details of training and test data. In this work,
we use a part of the English Read by Japanese (ERJ) corpus [29]for training and evaluation. This corpus contains read English
speech data of Japanese college students (190 speakers), with
Table 2: Experimental data.

LOW
MID
HIGH

Re-trainUpdated
Re-train
G2P
LexiconAM
Supervised Native G2P + SAT
LOW Un-sup G2P + SAT

Figure 4: Result of unsupervised G2P on matched proficiency
level + SAT.

4.1. Experimental Conditions

ERJ

60
55
50
45
40
35
30
25
20
15
Train G2P

Baseline
Unsupervised G2P + SAT
Baseline + SAT

4. Experimental Evaluation# Utterances
37318
736
11364
3175
4560
# Utterances
610
4889
2453
480

Re-trainAM

Baseline
Unsupervised G2P + SAT
Baseline + SAT“Unsupervised G2P” used phoneme recognition results of nonnative speech to train the G2P converter. However, it has been
noted that the pronunciation characteristics of non-native speakers vary depending on their proficiency level or native language[28]. “Unsupervised G2P” does not consider English proficiency or native language, so it cannot learn individual models
for speakers with these varying traits.
To solve this problem, we examine a method “Unsupervised G2P on Matched Proficiency Level” that uses phoneme
recognition results of non-native speech for speakers of eachEnglish proficiency to train the G2P converter. This method
also can generate pronunciations that depend on the speaker’s
proficiency level without explicit knowledge or rules tailored tothe specific level. Specifically, we test on low, middle, and highproficiency speakers, and each method is called “LOW Un-sup
G2P,” “MID Un-sup G2P,” and “HIGH Un-sup G2P.” An example of this setup is shown in Fig. 3.
Moreover, we also apply a similar approach to non-native
English ASR for various languages. We test on English spoken
by native speakers of French, Italian, Greek, and Spanish, which
are respectively denoted as “FR Un-sup G2P,” “IT Un-sup G2P,”“GR Un-sup G2P,” and “SP Un-sup G2P.”# Speakers
282
6
93
26
57
# Speakers
5
40
20
24

UpdatedLexiconBaseline
Unsupervised G2P + SAT
Baseline + SAT

Figure 3: Overview of unsupervised G2P on matched proficiency level.

Training data
WSJ

60
55
50
45
40
35
30
25
20
15
Train G2P

# Hour
82.9
1.0
14.3
4.2
6.0
# Hour
0.8
6.6
3.3
0.6

3093

FR

85

75
WER(%)

WER(%)

75
65
55
45

65
55
45

35
Train G2P

UpdatedLexiconRe-trainAM

Re-train
G2P

Baseline
Unsupervised G2P

UpdatedLexicon35
Train G2P

Re-trainAM

UpdatedLexiconRe-trainAM

Re-train
G2P

Baseline
Unsupervised G2PSupervised Native G2P
FR Un-sup G2P
GR

85
75
65
55

UpdatedLexiconRe-train
AM

Supervised Native G2P
IT Un-sup G2P
SP85
WER(%)

WER(%)

IT

85

75
65
55
45

45
35
Train G2P

UpdatedLexiconRe-trainAM

Baseline
Unsupervised G2P

Re-train
G2P

UpdatedLexicon35
Train G2P

Re-trainAM

UpdatedLexiconRe-trainAM

Baseline
Unsupervised G2PSupervised Native G2P
GR Un-sup G2P

Re-train
G2P

UpdatedLexiconRe-train
AM

Supervised Native G2P
SP Un-sup G2PFigure 5: Result of French and Greek.

Figure 6: Result of Italian and Spanish.

to non-native speech. For the language model training data, we
also use WSJ and part of the ERJ corpus (not including the testdata). These language models: 3-gram with Kneser-Ney and
Witten-Bell discounting, are generated by the SRILM toolkit[32]. As a native expert lexicon, we use the CMU lexicon,
which has about 130K words and a 39 phoneme set. For G2P,
we use Sequitur [27], a trainable data-driven G2P tool basedon joint n-gram models. The baseline uses a G2P trained on
the CMU lexicon for conventional ASR without iterative pronunciation learning. In addition, we compare the results whenusing our unsupervised seed lexicon with the results of iterative
learning using the supervised lexicon as a seed.

and 45.2% for Spanish. Moreover, we attempted to apply SAT,
and the WERs degraded to 66.4%, 52.4%, 62.5%, and 50.0%
respectively, likely because the multi-lingual non-native speech
data is too small to properly adapt the acoustic model, as the
HIWIRE corpus is much smaller than the ERJ corpus. Accordingly, we perform the evaluation and analysis of each method
using the Baseline method as a reference.
The results of each language are shown in Fig. 5 and Fig. 6.
For the native French Speakers, “FR Un-sup G2P” is the mosteffective method, likely because the amount of training data islarger (3100 utterances) than that of the corpora in the otherlanguages (a maximum of 2000). However, results in other languages are less conclusive, suggesting that this method will depend on the amount of training data.

4.2. Experimental Results
4.2.1. Results for Speakers of Different Proficiencies

5. Conclusions

First, we show results for Japanese speakers of differing proficiency levels in Fig. 4. The WERs are 52.8% in LOW, 36.5% in
MID, and 23.8% in HIGH for the baseline model (Baseline). In
addition, when we apply SAT to the baseline, we achieve WERsof 45.6% in LOW, 33.0% in MID, and 21.3% in HIGH (Baseline + SAT). We perform the evaluation and analysis of each
iterative pronunciation learning method using these results as areference.
First, comparing the results of “Supervised Native
G2P+SAT” with the baselines, we can confirm previous work infinding iterative pronunciation learning useful. Next, comparing these results with “Unsupervised G2P+SAT,” we can further
see that the proposed method outperforms the supervised native lexicon, particularly for speakers of level LOW. Finally, allof the systems trained on speakers of similar proficiency levels
improved 6.1% in LOW, 1.7% in MID, and 2.7% in the HIGH
level compared to “Unsupervised G2P.” This indicates the usefulness of training different pronunciation lexicons for speakers
of different levels of proficiency.

In this work, we proposed a method to solve the problems
caused in non-native speech recognition due to mismatch between native and non-native pronunciation. Specifically, we
proposed an acoustic data-driven iterative pronunciation learning method that can be learned directly from phoneme transcripts of non-native speech. This unsupervised method proved
beneficial, improving accuracy over a baseline system using supervised G2P, and in several cases improving over an iterative
pronunciation estimation method using the supervised dictionary as a seed lexicon. Finally, we found that matching the English proficiency level can help to improve non-native speech
recognition accuracy significantly, and that the amount of improvement may be dependent on the data size used in training.
One interesting feature of the proposed method is that it ismore effective for non-native speakers of lower proficiency levels. It is conceivable that by combining supervised and unsupervised seed lexicons, we could create a method that works more
effectively for all proficiency levels. This is one interesting avenue for future work.

4.2.2. Results for Speakers of Various Languages

6. Acknowledgements

Next, we show results for speakers of various European languages. In this case, the baseline WER of various native languages is 56.1% for French, 47.6% for Greek, 53.8% for Italian,Part of this work was supported by JSPS KAKENHI Grant
Number 24240032 and 26870371.

3094

7. References[20] I.Amdal, F.Korkmazskiy, and A.C.Surendran. “Joint pronunciation modelling of non-native speakers using data-driven methods,”
In Proc. INTERSPEECH 2000, pp.622–625, Oct. 2000.

[1] C.Chapelle, and J.Jamieson. “Computerʖ Assisted Language Learning as a Predictor of Success in Acquiring English as a Second
Language.” TESOL Quarterly, vol.20, No.1, pp.27–46, 1986.
[2] L.Besacier, et al. “Automatic speech recognition for underresourced languages: A survey,” Speech Communication vol. 56
pp. 85–100, 2014.
[3] Tanja Schultz and Tim Schlippe, ʠ GlobalPhone: Pronunciation
Dictionaries in 20 Languages, ʡ in The 9th edition of the Language Resources and Evaluation Conference (LREC 2014), Reykjavik,Iceland, pp. 26–31, May. 2014.
[4] Tim Schlippe, Sebastian Ochs, and Tanja Schultz,ʠ Wiktionary as a
Source for Automatic Pronunciation Extraction, ʡin The 11th Annual Conference of the International Speech Communication Association (Interspeech), Makuhari, Japan, pp. 26–30, September.
2010.
[5] JL.Gauvain and CH.Lee. “Maximum a posteriori estimation for
multivariate Gaussian mixture observations of Markov chains,”
Speech and audio processing, IEEE transactions, vol.2, No.2
pp.291-298, 1994.
[6] C.J.Leggetter and P.C.Woodland. “Maximum likelihood linear regression for speaker adaptation of continuous density hidden
Markov models,” Computer Speech and Language, vol.9, No.2, pp.
171–185, 1995.
[7] Z.Wang, T.Schultz, and A.Waibel. “Comparison of acoustic model
adaptation techniques on non-native speech,” in Proc. ICASSP
2003, vol.1, pp.540–543, 2003.
[8] D.Imseng, R.Rasipuram, M.Magimai.-Doss. “Fast and Flexible
Kullback-Leibler Divergence Based Acoustic Modeling for Nonnative Speech Recognition,”, in Proc. ASRU, pp. 348–353, Dec.
2011.
[9] M.Wester. “Pronunciation modeling for ASR-knowledge-based
and data-derived methods,” in Proc. Computer Speech and Language vol.17, pp.69–85, 2003.
[10] M.Lehr, K.Gorman, I.Shafran. “Discriminative Pronunciation
Modeling for Dialectal Speech Recognition,”, in Proc. INTERSPEECH 2014, pp. 1458–1562, Sept. 2014.
[11] M.Finke and A.Waibel. “Speaking mode dependent pronunciation
modelling in large vocabulary conventional speech recognition,” inProc. EUROSPEECH 1997, pp. 2379–2382, 1997.
[12] W.Byrne, et al. “Pronunciation modelling using a hand-labelledcorpus for conversational speech recognition,” in Proc. ICASSP
1998, vol.1, pp.313–316, 1998.
[13] M.Riley,
W.Byrne,M.Finke,S.Khudanpur,
A.Ljolje,
J.McDonough, and G.Zavaliagkos. “Stochastic pronunciation
modelling from hand-labelled phonetic corpora,”. Speech Communication, vol.29, pp.209–224, 1999.
[14] J.M.Kessens, M.Wester, and H.Strik. “Improving the performanceof a Dutch CSR by modeling within-word and cross-word pronunciation variation,” Speech Communication, vol.29, pp.193–207,
1999.
[15] M.Wester, J.M.Kessens, and H.Strik. “Pronunciation variation in
ASR: Which variation to model?,” in Proc. ICSLP 2000, pp. 488–
491, 2000.
[16] S.Schaden. “Generating non-Native pronunciation lexicons byphonological rule,” in Proc. ICSLP. 2004.
[17] T.Holter and T.Svendsen. “Incorporating linguistic knowledge
and automatic baseform generation in acoustic subword unit based
speech recognition,” in Proc. EUROSPEECH 1997, pp.1159–1162,
1997.
[18] T.Holter and T.Svendsen. “Maximum likelihood modelling ofpronunciation variation,” Speech Communication, vol.29, pp.177–
191, 1999.
[19] I.Amdal, F.Korkmazskiy, and A.C.Surendran. “Data-driven pronunciation modelling for non-native speakers using association
strength between phones.” ASR2000-Automatic Speech Recognition: Challenges for the new Millenium ISCA Tutorial and Research Workshop (ITRW). 2000.

[21] J.M.Kessens, C.Cucchiarini, and H.Strik. “A data-driven method
for modeling pronunciation variation,” Speech Communication,
vol.40, pp.517–534, 2003.
[22] S.Matsunaga, A.Ogawa, Y.Yamaguchi and A.Imamura. “Nonnative English speech recognition using bilingual English lexicon
and acoustic models.” in Proc. ICASSP 2003 IEEE International
Conference vol.1, pp.625–628, 2003.
[23] M.Kim, YR.Oh, and HK.Kim. “Non-native pronunciation variation modelling using an indirect data driven method,” in Proc.
ASRU 2007, pp.231–236, 2007.
[24] R.Rasipuram, M.Razavi, M.Magimai-Doss, “Integrated Pronunciation Learning for Automatic Speech Recognition Using Probabilistic Lexical Modeling,”, in Proc. ICASSP 2015, 2015.
[25] Tim Schlippe, Wolf Quaschningk, and Tanja Schultz, ʠ Combining Grapheme-to-Phoneme Converter Outputs for Enhanced Pronunciation Generation in Low-Resource Scenarios, ʡ in The 4thWorkshop on Spoken Language Technologies for Under-resourced
Languages (SLTU 2014), St. Petersburg, Russia, pp. 14–16, May.
2014.
[26] L.LuɼA.Ghosal, S.Renals, “Acoustic Data-driven Pronunciation
Lexicon for Large Vocabulary Speech Recognition,”ɼin Proc.
ASRU 2013, pp.374–379, 2013.
[27] M. BisaniɼH. Ney, “Joint-sequence Models for Grapheme-tophoneme Conversion,” Speech Communication, vol. 50, no.5, pp.
434–451, 2008.
[28] X.Wang, and S.Yamamoto. “Second Language Speech Recognition Using Multiple-Pass Decoding with Lexicon Represented byMultiple Reduced Phoneme Sets.” in Proc. INTERSPEECH 2015,
pp.1265–1269, 2015.
[29] N.Minematsu, et, al., “English Speech Database Read by Japanese
Learners for CALL System Development,”, in Proc. LREC 2002,
pp.896–903, 2002
[30] D.Povey, A.Ghoshal, G.Boulianne, L.Burget, O.Glembek,
N.Goel, M.Hannemann, P.Motlicek, Y.Qian, P.Schwarz, J.Silovskʅ
y, G.Semmer, K.Veselʅ y, “The Kaldi Speech Recognition Tool kit,”in Proc. ASRU 2011, 2011.
[31] D.Povey, and S.George. “Feature and model space speaker adaptation with full covariance Gaussians.” in Proc. INTERSPEECH
2006, 2006.
[32] A.Stolcke. “SRILM-an extensible language modeling toolkit,” inProc. INTERSPEECH 2002, 2002.

3095