A SAMPLING-BASED ENVIRONMENT POPULATION PROJECTION APPROACH FOR
RAPID ACOUSTIC MODEL ADAPTATION
Yu Tsao, Shigeki Matsuda, Shinsuke Sakai, Ryosuke Isotani, Hisashi Kawai, and Satoshi Nakamura
Spoken Language Communication Group
National Institute of Information and Communications Technology, Kyoto, 619-0288, Japan
ABSTRACT
We propose an environment population projection (EPP) approach for
rapid acoustic model adaptation to reduce environment mismatcheswith limited amounts of adaptation data. This approach consists of
two stages: population construction and projection. In the population
construction stage, we apply a sampling scheme on the adaptation
data to construct an environment population based on acoustic models
prepared in the training phase. With this sampling procedure, theenvironment samples in the population characterize diverse acoustic
information embedded in the adaptation data. Next, the projection
stage estimates a function to map the environment population into one
set of acoustic models that matches the testing condition. With a wellconstructed environment population, a simple projection function canenable the EPP approach to accurately characterize the testing
environment even with a small amount of adaptation data. To
examine the rapid adaptation ability of EPP, we used only one
adaptation utterance and tested performance in both supervised andunsupervised adaptation modes on Aurora-2 and Aurora-2J tasks. It isfound that EPP achieves satisfactory performance under both modes
for both tasks. On the Aurora-2J task, for example, EPP gives a clear
improvement of a 13.87% (8.58% to 7.39%) word error rate (WER)
reduction over our baseline in the unsupervised adaptation mode.
Index Terms—Stochastic matching, acoustic model adaptation,ensemble classification, environment population projection
1. INTRODUCTION
For automatic speech recognition (ASR), enhancing performancerobustness under training and testing mismatched conditions is a
crucial task. Maximum likelihood (ML)-based model adaptation
approaches have been proposed and shown strong ability to reducesuch mismatch by adjusting acoustic model parameters to match thetesting conditions. For these approaches, a mapping function isusually defined to characterize the mismatch, and the parameters in
the function are estimated according to the available adaptation data
based on the ML criterion. Successful methods include maximum
likelihood-based stochastic matching algorithm [1] and maximum
likelihood linear regression (MLLR) [2]. These ML-based methods
can effectively characterize the environment mismatches whensufficient adaptation data with accurate transcription information areavailable. However, real-world ASR applications usually favor rapid
model adaptation with small amounts of adaptation data with or even
without correct transcription information. The insufficient adaptation
samples along with possible imperfect transcriptions may deteriorate
the ASR performance. Therefore, identifying ways to efficiently takeadvantage of the available adaptation data and to improve thetranscription correctness is vital to the model adaptation capability.
More recently, a cross-validation (CV) based approach has been
proposed to estimate multiple acoustic model sets with a CV scheme
to improve the decoding hypothesis and accordingly enhance theperformance for unsupervised adaptation [3]. Some other approaches
incorporate N-best list to obtain better decoding hypothesis for theunsupervised adaptation mode [4] and to increase the discriminative
power for supervised adaptation [5]. Another class of approaches
develops a confidence measure and performs adaptation using only
samples with high confidence scores [6]. In this study, we propose anenvironment population projection (EPP) approach that takes a

978-1-4577-0539-7/11/$26.00 ©2011 IEEE

5504

sampling procedure to effectively utilize the available adaptation
data. As will be presented later, the proposed EPP approach provides
satisfactory performance in both supervised and unsupervised modes.
The EPP approach extends the ML-based stochastic matchingalgorithm by incorporating the ensemble classification concept [7].
In the implementation, EPP first performs a sampling procedure on
the available adaptation data set to generate several adaptation datasubsets. Each subset carries specific acoustic information embeddedin the entire set of adaptation data. With these adaptation subsets andwith acoustic models from the training phase, EPP calculates several
environment-specific acoustic model samples. The ensemble samples
then form an environment population. Finally, a projection function
is estimated to map the environment population into one set of
acoustic models that matches the testing condition.
We further develop two schemes to improve the accuracy anddiversity of the environment population. First, a specially designed
sampling and resampling scheme is used as the sampling procedure
to enhance the confidence level of each adaptation subset. Therefore,
each environment sample can be estimated more accurately toward a
specific acoustic condition. Second, to further increase the coverageof environment population, we prepare multiple acoustic model setsfrom the training phase instead of a single set. From ourexperimental results in both supervised and unsupervised modes,
EPP gives better recognition performance not only than the baseline
but also than the conventional bias compensation and MLLR with afurther confirmation by significance testing. The improvementssuggest that EPP possesses better model adaptation ability by taking
advantage of important diverse information from the adaptation datathat is usually averaged out in the conventional direct estimation
approaches. The rest of this paper is organized as follows. Section 2
introduces the EPP framework. Section 3 presents theimplementation of EPP, and then section 4 shows the experimental
setup and results. Finally, we provide our conclusions in section 5.
2. ENVIRONMENT POPULATION PROJECTION (EPP)
In this section, we first review the ML-based stochastic matchingalgorithm and then introduce the proposed EPP approach.

2.1. ML-based Stochastic Matching
The ML-based stochastic matching algorithm [1] characterizes an
unknown combination of speaker variability and environment
distortions by a mapping function, G߮ . By this mapping function, theoriginal acoustic model, Λ , is transformed into a new acoustic
ഥ  , that matches the testing condition:model,Λഥ  ൌ G߮ ሺΛ ሻǤሺͳሻ
ΛThe parameters of the mapping function,߮, are estimated based on the
speech utterances, , in an ML manner:߮ො ൌ  ܽ‫ܲ ݔܽ݉݃ݎ‬ሺ ȁ߮ǡ Λ ሻǤሺʹሻ߮

2.2. EPP Framework
Figure 1 shows the EPP framework, which consists of two stages:
Stage-1 prepares an environment population, and Stage-2 estimates a
set of acoustic models that matches the testing condition.

ICASSP 2011

2.2.1 Environment Population Construction
For Stage-1, we first take a sampling and resampling scheme togenerate S adaptation subsets. The scheme performs three steps on
the available adaptation data of T speech frames. Step1 decodes theadaptation utterances to acquire state alignment and posterior
probability for each frame. Step2 pools these T frame samples andrandomly draws T’ (T’<T) frames without replacement from the T
frames. Step3 resamples and draws (T−T’) frames from the T’ framesthat are obtained in Step2. Finally, we collect T frames (same amountas the original adaptation set) for one adaptation subset. We performthe three steps S times and obtain S adaptation subsets. Moreover in
Step3, we devise an additional procedure that draws more frames
from the sample pool for those frames having higher posterior
probabilities; frames with posterior probabilities lower than a certain
threshold are removed from the subset. This procedure enhances
confidence level of each adaptation subset. With the collection of S
adaptation subsets, we calculate S sets of acoustic models by:ഥ ‫ ݏ‬ൌ  ߮ ‫ ݏ‬ሺΛ ሻǡ ‫ ݏ‬ൌ ͳǡʹ ǥ Ǥ ǡ ܵǡሺ͵ሻΛഥ ‫ ݏ‬are, respectively, the acoustic models from thewhere Λ and Λtraining phase and transformed acoustic models using the s-th
adaptation subset. We call  ߮ ‫ ݏ‬population construction function, andthe parameters,߮ ‫ ݏ‬, in ߮ ‫ ݏ‬are calculated by the ML criterion:߮ො ‫ ݏ‬ൌ ܽ‫ܲݔܽ݉݃ݎ‬ሺ‫ ݏ‬ȁ߮ ‫ ݏ‬ǡ Λ ሻǡሺͶሻ߮‫ݏ‬where ‫ ݏ‬is the s-th adaptation subset. The ensemble S environmentഥͳǡ Λഥʹ ǥ ǡ Λഥ ܵ ሽ. We
samples form an environment population, Θ ൌ ሼΛcall this population constructed using information from adaptation
data adaptation-phase environment (AE) population.
Recently, a class of studies indicates that the diverse informationfrom the training data provide crucial prior knowledge for model
adaptation [8-10]. Here, we further include the training informationby using multiple anchor model sets to increase the coverage ofenvironment population. For the p-th anchor model, Λ‫ ݌‬, with the s-thഥ ‫ ݏ݌‬, by a function, ߮ ‫ ݏ‬:
adaptation subset, we obtain a new model, Λ‫݌‬‫ݏ‬ഥ ‫ ݌‬ൌ  ߮ ‫ ݏ‬൫Λ‫ ݌‬൯ǤሺͷሻΛ‫݌‬

Similar to Eq-(2), we estimate the parameters, ߮‫ ݏ݌‬, in߮ ‫ ݏ݌‬by:߮ො‫ ݏ݌‬ൌ ܽ‫ܲݔܽ݉݃ݎ‬൫ܱ‫ ݏ‬ห߮‫ ݏ݌‬ǡ Λ‫ ݌‬൯Ǥሺ͸ሻ߮ ‫ݏ݌‬

With P anchor models and S adaptation subsets, we get P×S samplesഥ ͳͳ ǡ Λഥ ͳʹ ǥ ǡ Λഥ ͳ‫ ݌‬ǡ ǥ ǡ Λഥ ܲܵ ሽ .
and form an environment population, Θ ൌ ሼΛ
We name it training-adaptation-phase environment (TAE) population
because both training and adaptation information are incorporated.

Estimate

SpeechData

OY

Sampling

żφ11 , φ12 ..., φ1Pž
Estimateżφ12 , φ22 ..., φP2 ž

Environment
Population

Estimate

λ

Θ

New
Model

ΛY

Estimate

2.2.2. Environment Population Projection
For Stage-2, we calculate a projection function, Gߣ , to map theഥ  , by:environment population, Θ, to one set of acoustic models, Λഥ  ൌ Gߣ ሺΘሻǤሺ͹ሻΛ
We estimate the parameter set,ߣ, of the projection function, Gߣ , basedon the ML criterion:ߣመ ൌ ܽ‫ܲݔܽ݉݃ݎ‬ሺ ȁߣǡ ΘሻǤሺͺሻߣ

3. IMPLEMENTATION OF THE EPP APPROACH
In this section, we introduce our implementation steps of the EPP
approach. An affine transformation is used as the construction
function, and a linear combination is adopted to be the projection
function. We only consider adaptation on mean vectors in this study.
To enhance adaptation accuracy, we design a hierarchical tree
structure to cluster mean vectors in our EPP implementation. The
leaf nodes are individual means in Λ , and each intermediate nodecontains a group of means. The top node includes the entire set ofmeans. Before adaptation, we perform a searching procedurethrough the tree to locate a node (the c-th node in the following) that
contains sufficient adaptation data for each mean vector.
3.1. Construction Function Estimation
When using an affine transformation as the construction function in
Eq-(5), the estimation of parameters is equivalent to the MLLR
transformation solution [2]. With the m-th mean ߤ‫݌‬ǡ݉ in the p-th‫ݏ‬anchor model, Λ‫ ݌‬, we intend to calculate ߤҧ‫݌‬ǡ݉based on the s-th
adaptation subset. If we know the m-th mean vector belonging to the‫ݏ‬c-th node, EPP estimates an affine transformation, Г‫݌‬ǡܿ, to perform:‫ݏ‬‫ݏ‬ߤҧ‫݌‬ǡ݉ൌ Г‫݌‬ǡܿߦ‫݌‬ǡ݉ ǡሺͻሻ
ሺͳሻ

ሺ‫ܦ‬ሻ

where ߦ‫݌‬ǡ݉ is the augmented vector: ሾͳǡ ߤ‫݌‬ǡ݉ ǡ ǥ ǡ ߤ‫݌‬ǡ݉ ሿ′ (each vector‫ݏ‬by the ML criterion:
has D coefficients). We can calculate Г‫݌‬ǡܿ
ܶ
‫ݏ‬
‫ݏ‬
෡
ߦ‫݌‬ǡ݇ ሻ′ΣെͳГ‫݌‬ǡܿ ൌ ܽ‫ ݊݅݉݃ݎ‬෍ ෍ ‫ ݏܮ‬ሺ‫ݐ‬ሻ‫ ݇ݎ‬ሺ‫ݐ‬ሻሾሺ‫ ݐݕ‬െ Г‫݌‬ǡܿ݇ ሺ‫ ݐݕ‬െГ‫ ݌ݏ‬ǡܿ‫ݐ‬ൌͳ ݇‫ܿא‬‫ݏ‬Г‫݌‬ǡܿ
ߦ‫݌‬ǡ݇ ሻሿǡሺͳͲሻ
where ‫ ݐݕ‬is the t-th observation vector.Σ݇ is the covariance matrix,and ‫ ݇ݎ‬ሺ‫ݐ‬ሻ is the posterior probability of the k-th Gaussian thatbelongs to the c-th node.‫ ݏܮ‬ሺ‫ݐ‬ሻ is the sampling unit that generates thes-th adaptation subset. We set T’=0.7×T, and by the resampling step
each adaptation subset has the same amount T frames as the originalഥ݉ ൌ
adaptation set. By collecting P×S means, we get a matrix, ܵͳʹሼߤҧͳǡ݉ǡ ߤҧͳǡ݉ǥ ǡഥߤͳ‫݌‬ǡ݉ ǡ Ǥ Ǥ ǡ ߤҧ ܲǡ݉ሽ, as the population for the m-th Gaussian.

3.2. Projection Function Estimation
With the prepared environment population, we estimate a projection
function to find the final acoustic models. From a previous study, we
can have several projection function candidates [4]. In this paper, wechoose a linear combination function as the projection function. For
the m-th Gaussian, we first follow the searching process to determine
a node (the c-th node) and then calculate the final mean vector, ߤҧ ǡ݉ ,that matches the testing condition by:ഥ ݉ ‫ ܿݓ‬ǡሺͳͳሻ
ߤҧǡ݉ ൌ 
where ‫ ܿݓ‬is the coefficient vector of the linear combination function
for the c-th node. Similarly, we calculate ‫ ܿݓ‬using the ML criterion:
ܶżφ1S , φ2S ..., φPS žഥ ݇ ‫ ܿݓ‬ሻ′Σെͳഥ‫ݓ‬ෝܿ ൌ ܽ‫ ݊݅݉݃ݎ‬෍ ෍ ‫ ݇ݎ‬ሺ‫ݐ‬ሻሾሺ‫ ݐݕ‬െ ݇ ሺ‫ ݐݕ‬െ ݇ ‫ ܿݓ‬ሻሿ Ǥ ሺͳʹሻ‫ܿݓ‬AnchorModels‫ݐ‬ൌͳ ݇‫ܿא‬The presented EPP implementation using affine transformation as
the construction function and linear combination as the projection
function is only an example. Other types of construction andprojection functions can be implemented in a similar manner.

{ Λ1, Λ2 ,…,ΛP }Figure 1: Environment population projection framework5505

4. EXPERIMENTS
In this section, we first briefly introduce the experimental setup.
Then, we present and discuss our experimental results.
4.1. Experimental Setupwhereഥ ‫ ݌‬ൌ  ߮ ൫Λ‫ ݌‬൯ǡ ‫ ݌‬ൌ ͳǡʹ ǥ Ǥ ǡ ܲǡሺͳ͵ሻΛ‫݌‬߮ො‫ ݌‬ൌ ܽ‫ܲݔܽ݉݃ݎ‬ሺ ห߮‫ ݌‬ǡ Λ‫ ݌‬൯ǤሺͳͶሻ߮‫݌‬

We evaluated the EPP approach on two speech databases, Aurora-2[11] and Aurora-2J [12]. Aurora-2 is a well-known English
connected digit recognition task that is often used to evaluate ASR’snoise robustness. Aurora-2J is a Japanese version digit recognition
task and is designed to have the same structure as Aurora-2. Bothtasks have two training sets, multi-condition and clean condition, and
70 different testing conditions (ten noise types at seven SNR levels).
In this paper, we use the multi-condition training sets for both
the Aurora-2 and Aurora-2J tasks. For both tasks, the multi-condition
training set includes 17 different speaking environments that arefrom the same four types of noise as in test SetA, at different SNRlevels: 5dB, 10dB, 15dB, 20dB, and clean condition. In the following
discussions, we report performance using 50 testing conditions (tennoise types at five SNR levels, 0dB, 5dB, 10dB, 15dB, 20dB). Eachcondition has 1001 utterances collected from 104 testing speakers(52 male and 52 female). Each speaker pronounced nine or ten
utterances. The testing speakers did not participate in the training
phase. Here, we report our results in average word error rate (WER).
For both the Aurora-2 and Aurora-2J tasks, we used a modified
ETSI advanced front-end (AFE) for feature extraction [13]. Everyfeature vector comprised 13 static plus their first and second ordertime derivatives. Meanwhile, we followed a complex back-end
topology presented in [13] to train baseline hidden Markov models(HMMs). Each digit was modeled with 20 mixtures per state, and thesilence and short pause were modeled with 36 mixtures per state.
Because we focused on rapid model adaptation, only one
utterance was used for adaptation. We tested EPP performance inboth supervised and unsupervised adaptation modes on both Aurora2 and Aurora-2J tasks. Similar results were obtained for all theevaluations. Due to the limited space, we only report supervised
experiments on Aurora-2 and unsupervised on Aurora-2J. For thesupervised mode, we used the first single utterance to adapt model
parameters for each speaker. The adapted model was then used totest recognition on the remaining eight or nine utterances from thatsame speaker. For each of the 50 testing conditions, we performed
the adaptation and testing procedures 104 times and finally reported
the average WER over 897 (1001-104) testing utterances. For theunsupervised mode, we conducted a per-utterance self-adaptation
scheme: each testing utterance was first decoded into N-best lists; theN-best hypotheses were then used for adaptation. Finally, the adapted
models were used to recognize the same testing utterance.
Accordingly, each condition contained 1001 testing utterances.
4.2. Experimental Results
As mentioned in Section 3, we built a tree structure to facilitatemodel adaptation. For the Aurora-2 task, we prepared a three-layered
tree structure (including root, intermediate, and leaf nodes) based on
the distances between mean vectors in the Aurora-2 baseline HMM
set. For all the experiments on Aurora-2 in the following discussions,
we used this tree structure to perform model adaptation. In the sameway, we built another three-layered tree for Aurora-2J and used thetree to conduct all the model adaptation experiments on Aurora-2J.
4.2.1. Supervised Experiments
First, we compare the EPP performance using different environment
populations. In addition to AE and TAE, we designed anotherenvironment population by taking P anchor model sets in the training
phase, ሼΛͳ ǡ Λʹ ǥ ǡ Λܲ ሽ, and the original adaptation data set (withoutഥ ‫ ݌‬, by:doing the sampling procedure) to calculate the new model set, Λ

5506

Then, the ensemble P sets of transformed acoustic models form aഥͳ ǡ Λഥʹ ǥ ǡ Λഥ ܲ ሽ. We call this population the trainingpopulation:Θ ൌ ሼΛphase environment (TE) population. Here, we set P=5 by using fiveanchor HMM sets. In addition to the baseline HMM set, we preparedother four anchor HMM sets that represented four different acoustic
characteristics in the training set, including anchor HMM sets forhigh SNR, low SNR, male speakers, and female speakers.
Table I presents the EPP testing results using TE, AE, and TAE
populations in a supervised adaptation mode on Aurora-2. To get theenvironment populations, EPP-AE used one anchor HMM set (thebaseline HMMs) and 30 adaptation subsets (P=1,S=30); EPP-TE
used five anchor HMM sets and the original adaptation utterance
without sampling (P=5,S=1); EPP-TAE used five anchor HMM setswith six adaptation sampling subsets (P=5,S=6). In this set ofexperiments, we used the affine transformation as the construction
function in Eq-(5) and the linear combination function as the
projection function in Eq-(7). SetA, SetB, and SetC in Table I show
the average WERs over 20, 20, and 10 conditions. We also list
Baseline and MLLR results in Table I for comparison. For Baseline,
we directly used the baseline HMM set to test recognition without
performing adaptation. For MLLR, we performed MLLR adaptation
to calculate new HMMs. Then, the testing utterances were tested
recognition with the adapted HMM set. It is noted that we canconsider MLLR as EPP with (P=1,S=1) in this condition.
From the results, we first observe that MLLR and EPP with anyof the three populations can provide clear improvements overBaseline for all sets, especially SetC. Next, we observe that EPP-AE
and EPP-TE give improvements over MLLR. The results indicatethat either by using multiple adaptation subsets (S=1 to S=30) or byincorporating multiple training anchor models (P=1 to P=5), we canachieve better adaptation performance. When comparing EPP-AE
with EPP-TAE, we see that by using a same number of environment
samples (both are 30), EPP-TAE achieves better performance.
Finally by comparing EPP-TE with EPP-TAE, we find that EPPTAE outperforms EPP-TE by using more adaptations subsets (S=1 to
S=6). The results suggest that by including both training andadaptation information in the environment population, EPP canmodel the testing conditions more accurately. Compared to Baseline,
EPP-TAE gives a 6.05% (6.45% to 6.06%) average WER reduction.
TABLE I. SUPERVISED MODE AVERAGE WER (%) ON AURORA-2
Test Condition
Baseline
MLLR(P=1,S=1)
EPP-AE(P=1,S=30)
EPP-TE(P=5,S=1)
EPP-TAE(P=5,S=6)

SetA
5.88
5.74
5.66
5.57
5.50

SetB
6.70
6.57
6.46
6.56
6.49

SetC
7.08
6.53
6.35
6.26
6.33

Overall
6.45
6.23
6.12
6.10
6.06

From Table I, the overall performance of EPP-TAE (S=6,P=5) seems
only marginally better than that of MLLR(S=1,P=1) with an average
WER reduction of 2.73% (6.23% to 6.06%). In this study, we further
took a matched pair t-Test significance test [14] to verify theperformance improvement. Because each SNR condition has ten
results in the Aurora-2 test set, we conducted t-Test by ten pair-wise
results. Instead of using a fixed threshold to determine the t-Test
results, we directly present P-values of the matched pair t-Test. A
small P-value indicates consistent performance improvements overten results. We list the MLLR and EPP-TAE WER results along with
P-values at different SNR condition in Table II. Each block in thefirst three columns lists an average WER over ten different noise
types. Each block in the fourth column lists a P-value of EPP-TAE
versus MLLR. From Table II, small P-values are observed for almost
every condition. This observation indicates that EPP-TAE isconsistently better than MLLR for each particular SNR condition.
TABLE II. SUPERVISED MODE WER (%) AND P-VALUES ON AURORA-2
dB
20
15
10
5
0
All

Baseline
0.69
1.19
2.71
6.79
20.87
6.45

MLLR
0.69
1.17
2.59
6.55
20.15
6.23

EPP-TAE
0.62
1.10
2.49
6.41
19.69
6.06

P-value
0.015
0.070
0.061
0.047
0.001

4.2.2. Unsupervised Experiments
In this set of experiments, we intentionally used another function,
bias compensation, as the construction function to test the EPP
performance. The linear combination is still used as the projection‫ ݏ‬, Eq-(5) becomes:
function. When using a compensation bias, ܾ‫݌‬ǡܿ‫ݏ‬Ǥሺͳͷሻߤҧ‫ ݏ݌‬ǡ݉ ൌ ߤ‫݌‬ǡ݉ ൅ ܾ‫݌‬ǡܿ
Similar to Eq-(10), we estimate the bias compensation for the p-th
environment with given the s-th adaptation subset by :
ܶ
‫ݏ‬
‫ݏ‬
ൌ ܽ‫ ݊݅݉݃ݎ‬෍ ෍ ‫ ݏܮ‬ሺ‫ݐ‬ሻ‫ ݇ݎ‬ሺ‫ݐ‬ሻሾሺ‫ ݐݕ‬െ ߤ‫݌‬ǡ݇ െ ܾ‫݌‬ǡܿሻ′Σെͳ
ܾ෠‫݌‬ǡܿ݇ ሺ‫ ݐݕ‬െܾ‫ ݏ݌‬ǡܿ‫ݐ‬ൌͳ ݇‫ܿא‬‫ݏ‬ߤ‫݌‬ǡ݇ െ ܾ‫݌‬ǡܿሻሿǤሺͳ͸ሻ
Table III reports the EPP results with three different environment
populations in the unsupervised mode on Aurora-2J. We also list
Baseline and Bias results for comparison. Similar to the supervisedmode, for Baseline, we directly used the baseline HMMs to test
recognition; for Bias, we used a bias compensation function to adapt
HMMs and then decoded the same testing utterance using theadapted HMMs. From Table III, we observe similar phenomena tothat from Table I. First, both Bias (P=1,S=1) and the three EPP
setups give clear improvements over Baseline. Next, EPP-TAE
achieves the best performance comparing to Bias and the other two
EPP setups. For the overall testing conditions, EPP-TAE provides a13.87% (8.58% to 7.39%) average WER reduction over Baseline.

TABLE III. UNSUPERVISED MODE AVERAGE WER (%) ON AURORA-2J
Test Set
Baseline
Bias(P=1,S=1)
EPP-AE (P=1,S=30)
EPP-TE (P=5,S=1)
EPP-TAE(P=5,S=6)

SetA
6.80
6.43
6.27
6.24
6.20

SetB
10.67
9.23
8.87
8.85
8.67

SetC
7.95
7.32
7.32
7.34
7.22

Overall
8.58
7.73
7.52
7.50
7.39

Similarly, we verify the performance improvements of EPP-TAE
over Bias by using the matched pair t-Test. Table IV shows WERsand P-values at different SNR levels. We find similar observations tothat from the supervised experiments: though the overall average
improvement of EPP-ATE over Bias is 4.40% (7.73% to 7.39%), thesmall P-values listed in Table IV indicate that the improvements are
consistent among ten results for each of the five SNR levels.
TABLE IV. UNSUPERVISED MODE WER (%) AND P-VALUES ON AURORA-2J
dB
20
15
10
5
0
All

Baseline
0.49
1.01
3.12
9.87
28.39
8.58

Bias
0.41
0.88
2.56
8.51
26.28
7.73

EPP-TAE
0.36
0.72
2.25
7.90
25.73
7.39

P-value
0.035
0.001
0.019
0.001
0.017

5507

5. CONCLUSION
We propose an EPP approach to perform rapid model adaptation for
reducing environment mismatches. EPP uses a sampling scheme toprepare an environment population and then estimates a projection
function to map the population to one set of acoustic models thatmatches the testing condition. We evaluated the EPP approach inboth supervised and unsupervised modes on Aurora-2 and Aurora-2J
tasks. To investigate the rapid adaptation capability of EPP, we used
only one adaptation utterance for all the evaluations. We observed
similar results from all the evaluations and reached three major
conclusions:(1) EPP provides significant improvements over thebaseline under both supervised and unsupervised modes; (2) EPP
using affine transformation or bias compensation as the construction
function can give better performance than their direct estimation
counterparts; (3) Incorporating additional training information byusing multiple anchor model sets in the training phase can enable
EPP to achieve even better performance. In the future, we will
explore other sampling schemes and the correlation between numberof samples and achievable performance. Moreover, other types of
construction and projection functions will be further studied.
6. REFERENCES
A. Sankar and C.-H. Lee, “A maximum-likelihood approach tostochastic matching for robust speech recognition,” IEEE Trans.
on Speech Audio Proc., vol. 4, pp.190-202, 1996.
[2] C. J. Leggetter and P. C. Woodland, “Maximum likelihood
linear regression for speaker adaptation of continuous density
hidden Markov models,” Computer Speech and Language, vol.
9, pp.171-185, 1995.
[3] T. Shinozaki, Y. Kubota, and S. Furui, “Unsupervised crossvalidation algorithm for improved adaptation performance,” inProc. ICASSP, pp. 4377-4380, 2009.
[4] Y. Tsao, J. Li, and C.-H. Lee, “Ensemble speaker and speaking
environment modeling approach with advanced online
estimation process,” in Proc. ICASSP, pp. 3833-3836, 2009.
[5] P. Nguyen, P. Gelin, J.-C. Junqua, and J.-T. Chien, “N-best based
supervised and unsupervised adaptation for native and nonnative speakers in cars,” in Proc. ICASSP, pp. 173-176, 1999.
[6] W. K. Lo and F. K. Soong, “Generalized posterior probability
for minimum error verification of recognized sentences,” inProc. ICASSP, pp.85-88, 2005.
[7] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification,
New York, Wiley, 2001.
[8] R. Kuhn, J.-C. Junqua, P. Nguyen, and N. Niedzielski, “Rapid
speaker adaptation in Eigenvoice space,” IEEE Trans. Speech
Audio Processing, vol. 8, pp. 695-707, 2000.
[9] M. J. F. Gales, “Cluster adaptive training of hidden Markov models,”IEEE Trans. Speech Audio Proc., vol. 8, pp. 417-428, 2000.
[10] Y. Tsao and C.-H. Lee, “An ensemble speaker and speaking
environment modeling approach to robust speech recognition,”
IEEE Trans. on Audio, Speech, and Language Proc., vol. 17,
pp.1025-1037, 2009.
[11] D. Macho, L. Mauuary, B. Noe, Y. M. Cheng, D. Ealey, D.
Jouver, H. Kelleher, D. Pearce, and F. Saadoun, “Evaluation ofa noise-robust DSR front-end on Aurora databases,” in Proc.
ICSLP, pp. 17-20, 2002.
[12] S. Nakamura, K. Takeda, K. Yamamoto, T. Yamada, S.
Kuroiwa, N. Kitaoka, T. Nishiura, A. Sasou, M. Mizumachi, C.
Miyajima, M. Fujimoto, and T. Endo, “AURORA-2J: An
evaluation framework for Japanese noisy speech recognition,”
IEICE Trans. Inf. Syst., vol. E88-D, pp. 535-544, 2005.
[13] J. Wu and Q. Huo, “Several HKU approaches for robust speech
recognition and their evaluation on Aurora connected digit
recognition tasks,” in Proc. Eurospeech, pp. 21-24, 2003.
[14] A. J. Hayter, Probability and Statistics for Engineers andScientists, Duxbury Press; 3rd edition, 2006.
[1]