## 5 Experimental Setup
### Applying Domain Adaptation Using Shallow Fusion  
LM domain adaptation is a process of shallowly fusing the base LM with a domain corpus. The base LM was trained based on recurrent neural network (RNN) LM using Korean and English text corpora. The dev. Set consisted of 168 Economy sentences and 1895 lecture sentences. Other parameters and hardware settings are described in Table 4. These are based on the hyperparameters of Librispeech, with some values adjusted.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Character Accuracy (\%)** & **Economy** & **AILec1** & **AILec2** \\ \hline KR Baseline & 71.0 & 76.4 & 77.4 \\ KR–EN 1  & 72.6 & 76.3 & 76.7 \\ Konglish Native-like EN 2  & 74.3 & 77.8 & 78.5 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Character accuracy of KR baseline, KR–EN, and Konglish native-like EN.  
The CS sentences for the domain adaption were extracted from 9.6 million sentences in the general domain text corpus. Among the English words sorted by frequency in the dev. Set, the CS sentences of each domain were extracted from the general domain corpus, including words with frequencies of 2000 or less. Figure 8 shows the frequency and the cumulative rate per English word, which consists of 380 ranks in the dev. set. This implies that the cumulative rate of the words remained almost above 95% in English word rank. The proportion of very rare English words in the word list was determined to be 95%, according to empirical experiments. Since very rare English words appear only in certain domains, they have the advantage of contributing to domain adaptation. Hence, it is regarded that a word frequency of 2000 or less is very rare for English words in Korean. Finally, both 13,708 and 23,395 CS sentences from each domain were extracted as the domain DBs.  
The threshold of cosine similarity was set to above 0.6. In this case, 21,218 and 86,786 sentences were found for each domain after removing duplications. For instance, Figure 9 describes the cosine similarity in ascending order of the dev. set of the lecture domain.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Configurations** & \multicolumn{2}{c}{**Hyperparameters**} \\ \hline \multirow{8}{*}{Model 1  & Encoder type & VGGBLSTM \\  & Encoder layers & 5 \\  & Decoder layers & 2 \\  & Encoder units & 1024 \\ Model 1  & Projection units & 1024 \\  & Attention type & Location \\  & Attention dimension & 1024 \\  & Number of channels & 10 \\  & Filter size & 100 \\ \hline \multirow{4}{*}{Training} & Optimizer & AdaDelta \\  & CTC weight & 0.5 \\  & Epochs & 20 \\  & Batch size & 16 \\ \hline \multirow{2}{*}{RNN LM 2 Training} & Optimizer & SGD \\  & Epochs & 185–660 3 \\ \hline \multirow{4}{*}{Decoding} & Beam size & 10 \\  & CTC weight & 0.5 \\ \cline{1-1}  & LM weight & 0.7 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The configuration of the experimental setup for the model.  
Figure 8: Finding very rare English words (domain adaptation 1); the frequency on a log scale for the left y-axis (dotted line); the cumulative rate (%) for the right y-axis (solid line); English word rank for the horizontal x-axis.  
Through steps 1-3 in Section 4, the domain DBs for adaptation can be summarized as shown in Table 5, where domain adaptation 1 is DA 1, domain adaptation 2 is DA 2, and domain adaptation 3 is DA 3. The base LM was trained with RNN LM using 2.1 million sentences in Korean and English. DA 1 + 2 + 3 means that all of the domain DBs were used for adaptation.  
Table 6 shows the results of the LM adaptation. With the base LM only, the CA improved with Economy and AILec2, but not with AILec2. After adapting LM at each step (i.e., steps 1-3 in Section 4), the results show that the performance improved steadily when compared with the K-Base LM. When all of the domain DBs were combined (i.e., K-Base LM + DA 1 + 2 + 3), we obtained the highest performance among all of the combinations. The LM adaptation seemed to work well, as expected. Thus, the proposed method for selecting CS sentences is effective for ASR problems.