SlowGuess/ACL_2020_OCR huggingface dataset:
We also attempt to visualize the learned phone representations by projecting down to 2D. For a specific phone, we use the precomputed alignments to compute averaged layer-wise representations across the frames within each phone alignment. Figure 7 shows t-SNE based (Maaten and Hinton, 2008) 2D visualizations of representations for the 10 most frequent phones in our data,  $\{ah', ih', iy', dh', d', l', n', r', s', t'\}$ . Each subplot corresponds to a layer in the network. The plots for phones from the US-accented samples appear to have slightly more well-formed clusters, compared to the Indian-accented samples. These kinds of visualizations of representations are, however, limiting and thus motivates the need for analysis like the MI computation presented earlier.
5 Classifier-driven Analysis
5.1 Accent Classifiers
We train an accent classifier for each layer that takes the corresponding representations from the layer as its input. We implemented a classifier with two convolutional layers of kernel size, stride and padding set to (31,21), (3,2), (15,10) and (11,5), (2,1) and (5,2), respectively. We used batch nor
(a) US accent
(b) Indian accent
malization (Ioffe and Szegedy, 2015) followed by ReLU activations for each unit. The network also contained two max-pooling layers of size (5,3) and (3,2), respectively, and a final linear layer with hidden dimensionality of 500 (with a dropout rate of 0.4). Table 1 lists the number of utterances we used for each accent for training and evaluation. The accent classifiers were trained for 25 epochs using Adam optimizer (Kingma and Ba, 2015) and a learning rate of 0.001.
Figure 8 shows the accent accuracies obtained by the accent classifier specific to each layer (along with error bars computed over five different runs).  $\mathrm{RNN}_0$  is most accurate with an accuracy of about  $33\%$  and  $\mathrm{RNN}_4$  is least accurate. It is interesting that  $\mathrm{RNN}_0$  representations are most discriminative across accents; this is also consistent with what we observe in the MI plots in Figure 5.
5.2 Phone Classifiers
Akin to accent classifiers, we build a phone classifier for each layer whose input representations are labeled using phone alignments. We train a simple multi-layer perceptron for each DS2 layer (500-dimensional, dropout rate of 0.4) for 10 epochs
Figure 8: Accuracy  $(\%)$  of accent probes trained on hidden representations at different layers.
Figure 9: Trends in accuracy  $(\%)$  of phone probes for frame-level (dotted) and averaged representations (solid) at different layers.
using the Adam optimizer. We train both frame-level classifiers, as well as phone-level classifiers that use averaged representations for each phone as input. The accuracies of both types of phone classifiers are shown in Figure 9. As expected, the phone accuracies improve going from SPEC to  $\mathrm{RNN}_4$  and the accuracies of US/Canadian samples are much higher than that of Indian samples. Classifiers using the averaged representations consistently perform much better than their frame-level counterparts. We note that Belinkov and Glass (2017) report a dip in phone accuracies for the last RNN layers, which we do not observe in our experiments. To resolve this inconsistency, we ran phone classifiers on TIMIT (which was used in Belinkov and Glass (2017)) using representations from our DS2 model and the dip in  $\mathrm{RNN}_4$  accuracies surfaced (as shown in Figure 9). This points to differences between the TIMIT and Mozilla Common Voice datasets. (An additional experiment examining how phone classifiers behave on different datasets is detailed in Appendix D.)
This is the first detailed investigation of how accent information is reflected in the internal representations of an end-to-end ASR system. In devising analysis techniques for ASR, while we do follow the broad approaches in the literature, the details are often different. Most notably, the use of EMD for attribution analysis is novel, and could be of interest to others working with speech and other temporal data. Similarly, the phone focus mea
sures in the information mixing analysis are new. We also highlight that this is the first instance of analysis of ASR consisting of multiple analysis techniques. On the one hand, this has uncovered robust trends that manifest in more than one analysis. On the other hand, it also shows how some trends are influenced more by the neural-network architecture more than the data. This provides a platform for future work in speech neural-network analysis, across architectures, data-sets and tasks.
In our results, we encountered some unexpected details. For instance, while the  $\mathrm{RNN}_0$  layer is seen to reduce the phone focus the most, uniformly across all accents (as shown in Figure 3, it is also seen to segregate accent information the most, recovering accent information "lost" in the convolution layer (as shown in Figure 5). We also see this trend surfacing in Figure 8 where the accent classifier gives the highest accuracy for  $\mathrm{RNN}_0$  and the accuracies quickly taper off for subsequent layers. This suggests that the first RNN layer is most discriminative of accents. Models that use an adversarial objective to force the representations to be accent invariant (e.g., Sun et al., 2018)) might benefit from defining the adversarial loss as a function of the representations in the first RNN layer.
7.1 Accented Speech Recognition
Huang et al. (2001) show that accents are the primary source of speaker variability. This poses a real-world challenge to ASR models which are primarily trained on native accented datasets. The effect of accents is not limited to the English language, but also abundant in other languages such as Mandarin, Spanish, etc.
An interesting line of work exploits the ability to identify accents in order to improve performance. Zheng et al. (2005) combine accent detection, accent discriminative acoustic features, acoustic model adaptation using MAP/MLLR and model selection to achieve improvements over accented Mandarin speech. Vergyri et al. (2010) investigate the effect of multiple accents on the performance of an English broadcast news recognition system using a multiple accented English dataset. They report improvements by including data from all accents for an accent-independent acoustic model training.
Sun et al. (2018) propose the use of domain adversarial training (DAT) with a Time Delay Neu
ral Network (TDNN)-based acoustic model. They use native speech as the source domain and accented speech as the target domain, with the goal of generating accent-invariant features which can be used for recognition. Jain et al. (2018) also use an accent classifier in conjunction with a multi-accent TDNN based acoustic model in a multitask learning (MTL) framework. Further, Viglino et al. (2019) extended the MTL framework to use an end-to-end model based on the DS2 architecture and added a secondary accent classifier that uses representations from intermediate recurrent layers as input. Chen et al. (2020) propose an alternate approach using generative adversarial networks (GANs) to disentangle accent-specific and accent-invariant components from the acoustic features.
7.2 Analysis of ASR Models
Nagamine et al. (2015, 2016) were the first to examine representations of a DNN-based acoustic model trained to predict phones. They computed selectivity metrics for each phoneme and found better selectivity and more significance in deeper layers. This analysis was, however, restricted to the acoustic model. Belinkov and Glass (2017) were the first to analyze a Deep Speech 2 model by training phone classifiers that used representations at each layer as its input. These ideas were further extended in Belinkov et al. (2019) with classifiers used to predict phonemes, graphemes and articulatory features such as place and manner of articulation. Belinkov and Glass (2019) present a comparison of different analysis methods that have been used in prior work for speech and language. The methods include recording activations of pretrained networks on linguistically annotated datasets, using probing classifiers, analyzing attention weights and ABX discrimination tasks (Schatz et al., 2013).
Other related work includes the analysis of an audio-visual model for recognition in Alishahi et al. (2017), where the authors analyzed the activations of hidden layers for phonological information and observed a hierarchical clustering of the activations. Elloumi et al. (2018) use auxiliary classifiers to predict the underlying style of speech as being spontaneous or non-spontaneous and as having a native or non-native accent; their main task was to predict the performance of an ASR system on unseen broadcast programs. Analogous to saliency maps
used to analyze images, Li et al. (2020) propose reconstructing speech from the hidden representations at each layer using highway networks. Apart from ASR, analysis techniques have also been used with speaker embeddings for the task of speaker recognition (Wang et al., 2017).
The predominant tool of choice for analyzing ASR models in prior work has been classifiers that are trained to predict various phonological attributes using quantities extracted from the model as its input. We propose a number of alternatives other than just the use of classifiers to probe for information within an end-to-end ASR model. We hope this spurs more analysis-driven investigations into end-to-end ASR models.
This work presents a thorough analysis of how accent information manifests within an end-to-end ASR system. The insights we gleaned from this investigation provide hints on how we could potentially adapt such end-to-end ASR models, using auxiliary losses, to be robust to variations across accents. We will investigate this direction in future work.
The authors thank the anonymous reviewers for their constructive feedback and comments. The second author gratefully acknowledges support from a Google Faculty Research Award and IBM Research, India (specifically the IBM AI Horizon Networks-IIT Bombay initiative).
Afra Alishahi, Marie Barking, and Grzegorz Chrupał. 2017. Encoding of phonology in a recurrent neural model of grounded speech. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 368-378.
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. 2016. Deep Speech 2: End-to-End Speech Recognition in English and Mandarin. In Proceedings of the 33rd International Conference on Machine Learning, pages 173-182.
Yonatan Belinkov, Ahmed Ali, and James Glass. 2019. Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition. In Proc. Interspeech 2019, pages 81-85.
Yonatan Belinkov and James Glass. 2017. Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems. In Advances in Neural Information Processing Systems, pages 2441-2451.
Yonatan Belinkov and James Glass. 2019. Analysis Methods in Neural Language Processing: A Survey. Transactions of the Association for Computational Linguistics, pages 49-72.
Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. 2020. On Identifiability in Transformers. In International Conference on Learning Representations.
Yi-Chen Chen, Zhaojun Yang, Ching-Feng Yeh, Mahaveer Jain, and Michael L Seltzer. 2020. AIP-Net: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6979-6983.
Chung-Cheng Chiu, Tara N Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J Weiss, Kanishka Rao, Ekaterina Gonina, et al. 2018. State-of-the-Art Speech Recognition with Sequence-to-Sequence Models. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4774-4778.
Misha Denil, Alban Demiraj, and Nando De Freitas. 2014. Extraction of Salient Sentences from Labelled Documents. arXiv preprint arXiv:1412.6815.
Zied Elloumi, Laurent Besacier, Olivier Galibert, and Benjamin Lecouteux. 2018. Analyzing Learned Representations of a Deep ASR Performance Prediction Model. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 9-15.
John S Garofolo. 1993. Timit acoustic phonetic continuous speech corpus. Linguistic Data Consortium, 1993.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. In Proceedings of the 23rd International Conference on Machine Learning, pages 369-376.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural computation, 9(8):1735-1780.
Takaaki Hori, Shinji Watanabe, Yu Zhang, and William Chan. 2017. Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM. In Proc. Interspeech 2017, pages 949–953.
Chao Huang, Tao Chen, Stan Li, Eric Chang, and Jianlai Zhou. 2001. Analysis of speaker variability. In Seventh European Conference on Speech Communication and Technology, pages 1377-1380.
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, pages 448-456.
Abhinav Jain, Minali Upreti, and Preethi Jyothi. 2018. Improved Accented Speech Recognition Using Accent Embeddings and Multi-task Learning. In Proc. Interspeech 2018, pages 2454-2458.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).
Chung-Yi Li, Pei-Chieh Yuan, and Hung-Yi Lee. 2020. What Does a Network Layer Hear? Analyzing Hidden Representations of End-to-End ASR Through Speech Synthesis. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6434-6438.
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9:2579-2605.
Mozilla. Mozilla common voice dataset. https://voice.mozilla.org/en/datasets.
Tasha Nagamine, Michael L Seltzer, and Nima Mesgarani. 2015. Exploring How Deep Neural Networks Form Phonemic Categories. In Proc. Interspeech 2015, pages 1912-1916.
Tasha Nagamine, Michael L Seltzer, and Nima Mesgarani. 2016. On the Role of Nonlinear Transformations in Deep Neural Network Acoustic Models. In Proc. Interspeech 2016, pages 803-807.
Sean Naren. 2016. https://github.com/SeanNaren/deepspeech.pytorch.git.
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206-5210.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. 2018. Assessing Generative Models via Precision and Recall. In Advances in Neural Information Processing Systems, pages 5228-5237.
Thomas Schatz, Vijayaditya Peddinti, Francis Bach, Aren Jansen, Hynek Hermansky, and Emmanuel Dupoux. 2013. Evaluating speech features with the Minimal-Pair ABX task: Analysis of the classical MFC/PLP pipeline. In Proc. Interspeech 2013, pages 1-5.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In International Conference on Learning Representations, ICLR, Workshop Track Proceedings.
Sining Sun, Ching-Feng Yeh, Mei-Yuh Hwang, Mari Ostendorf, and Lei Xie. 2018. Domain adversarial training for accented speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4854-4858.
Dimitra Vergyri, Lori Lamel, and Jean-Luc Gauvain. 2010. Automatic Speech Recognition of Multiple Accented English Data. In Proc. Interspeech 2010, pages 1652-1655.
Thibault Viglino, Petr Motlicek, and Milos Cernak. 2019. End-to-end Accented Speech Recognition. Proc. Interspeech 2019, pages 2140-2144.
Elena Voita, Rico Sennrich, and Ivan Titov. 2019. The bottom-up evolution of representations in the transformer: A study with machine translation and language modeling objectives. In _Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4396-4406_.
Voxforge.org. Free and open source speech recognition (linux, windows and mac) - voxforge.org. http:// www.voxforge.org/. Accessed 06/25/2014.
Shuai Wang, Yanmin Qian, and Kai Yu. 2017. What does the speaker embedding encode? In Proc. Interspeech 2017, pages 1497-1501.
Yanli Zheng, Richard Sproat, Liang Gu, Izhak Shafran, Haolang Zhou, Yi Su, Daniel Jurafsky, Rebecca Starr, and Su-Youn Yoon. 2005. Accent detection and speech recognition for shanghai-accented mandarin. In Ninth European Conference on Speech Communication and Technology, pages 217-220.
A Dataset Information across Accents
Figure 10(a) shows the frequency of each phone across all the accents used in our dataset. Figure 10(b) shows the average duration in seconds of each phone across all the accents in our dataset. The error bars for each phone denote the variance in coverage and duration across all the accents. We observe that the variance is very small, thus indicating that the difference in phone coverage and duration across accents is minimal.
(b) Phonetic duration histogram
Figure 10: Histograms showing phonetic coverage and duration for all accents with labels on the X-axis showing phones.
B Information Mixing: Neighbourhood Analysis
We explore the variation in the phone focus measure described in Section 3.2 for the aligned phone and its neighbors that precede and succeed it, across different layers of the model. Figure 11 shows that the focus of the actual (input) phone is maximum for the CONV layer and shows the fastest decrease across neighbors. This is expected due to the localized nature of convolutions. From  $\mathrm{RNN}_0$  to  $\mathrm{RNN}_4$  the focus of the actual phone decreases and is increasingly comparable to the first (and other) neighbors. We see an increase in neighbors  $12^{th}$  and onwards because of its cumulative nature. Figure 11 shows this analysis for the TIMIT
Figure 11: Phone focus of the aligned (actual) phone as compared to its preceding and succeeding neighbors on the TIMIT dataset.
dataset. We found the trends from such a comparison of phone focus across neighbors on our accented datasets to be very similar to the trends exhibited by the different layers on TIMIT.
C Experiments on Attribution Analysis
Common gradient-based explainability techniques include examining the gradient, as well as the dot product of the gradient and the input. We analyze both these techniques in this section. We also compare grapheme-level attributions with word-level attributions.
In Figure 12, we visualize grapheme-level attributions for the text "I'm going to them". The grapheme-level attribution is shown for the first letter in the transcription. The blue heatmaps correspond to computing the absolute value of the dot product of the gradient and the input (referred to as INP-GRAD) and the green heatmaps correspond to computing the L2 norm of the gradient (referred to as GRAD). On comparing the two, we observe that the former is more diffuse and discontinuous than the latter. In general, we observe that the grapheme-level attributions are distributed non-uniformly across the frames of the underlying phone. For some accents, the attribution of the frames of the nearby phones is also comparable.
Figure 13 shows the word-level attributions for the word "FIRE" using INP-GRAD. This can be contrasted with the word-level attributions for the same word shown in Figure 2 in Section 3.1. There is more discontinuity in INP-GRAD compared to GRAD; this could be attributed to the underlying speech containing sparse spectral distributions near fricatives or stop onsets, thus making alignments from the former technique less reliable for further downstream processing.
Unlike tasks like sentiment analysis where the positive and negative signs of the dot product carry meaningful information, in our setting we make use of the absolute value of the dot product.
Figure 12: Grapheme-level attributions for the first letter in the text transcribed by the model.
D Phone Classifiers
D.1 Effect of Changing Distribution of Phones
We investigate the influence of changing the distribution of phones on phone classifier accuracies. We sample phones from the Mozilla Common Voice dataset so as to mimic the phone distribution of the TIMIT dataset. Figure 14 shows no significant difference in changing the phone distribution. The plot also shows the accuracy on the TIMIT dataset which is higher than the phone accuracies for the
speech samples from Mozilla Common Voice for all layers (except  $\mathrm{RNN}_3$  and  $\mathrm{RNN}_4$ ). This reflects the differences in both datasets; TIMIT comprises clean broadband recordings while the speech samples from Common Voice are much noisier.
D.2 Effect of Changing Sample Size
In Section 3.2, we subsample 250 utterances from 1000 in the original test set. Even with only 250 utterances, we find the phone accuracy trends to be preserved as shown in Figure 15.
Figure 13: Word-level attributions for the word corresponding to "FIRE" in the transcription by the model.
Figure 14: Trends in phone accuracy on Common Voice accented speech samples using TIMIT's phone distribution (dotted) and the original phone distribution (solid). The line in black shows performance on the TIMIT dataset.
D.3 Confusion in Phones
We analyze the confusion in the phones for each accent<sup>7</sup>. As expected, phone confusions for each accent are more prevalent in initial layers compared to the later layers. A comparison between the confusion matrices at layer  $\mathrm{RNN}_4$  for all accents shows a prominent difference between native accents, like US and Canada, and non-native accents, like Indian
Figure 15: Trends in phone accuracy for 250 utterances (solid) randomly sampled from our test set consisting of 1000 utterances (dashed) for each accent.
and Scotland, indicating that for the latter even at the final layers there is some residual confusion remaining. Instead of showing all the confusion matrices for each accent, we resort to an aggregate entropy-based analysis. In Figure 16, we compute the entropy of the phone distributions, averaged over all phones, across layers for different accents. We observe that in the beginning the accents are clustered together and they diverge gradually as we move to higher layers. As we approach the last 3 recurrent layers  $(\mathrm{RNN}_2, \mathrm{RNN}_3$  and  $\mathrm{RNN}_4)$ , we find a clear separation, with Indian followed by Scotland accent having the highest entropy while
7Available at https://github.com/archiki/ ASR-Accent-Analysis/tree/master/ PhoneProbes
Accent	Top-5 Confusions
1st	2nd	3rd	4th	5th
US	aa-ao (0.134)	zh-jh (0.125)	z-s (0.112)	aa-ah (0.109)	g-k (0.092)
Canada	zh-ah (0.118)	zh-v (0.088)	g-k (0.074)	th-t (0.072)	y-uw (0.070)
African	z-s (0.102)	ae-ah (0.098)	er-ah (0.098)	aa-ah (0.089)	g-k (0.086)
Australia	zh-sh (0.462)	th-t (0.115)	aa-ah (0.105)	g-k (0.097)	z-s (0.92)
England	zh-sh (0.222)	z-s (0.126)	aa-ah (0.121)	ae-ah (0.119)	th-t (0.114)
Scotland	jh-d (0.148)	ch-t (0.146)	zh-sh (0.125)	zh-ey (0.125)	ae-ah (0.123)
Indian	zh-ah (0.38)	th-t (0.262)	z-s (0.175)	uh-uw (0.154)	er-ah (0.153)
Table 3: Five highest confusion pairs for all accents. Notation: phone_i - phone_j(x) means that phone_i is confused with phone_j with likelihood x.
US and Canada samples have the lowest entropy.
Figure 16: Entropy of the distribution of the outputs of the phone probes averaged across phones and plotted for each accent across layers.
Table 3 shows the five highest confusion pairs for each accent. We observe that each accent displays a different trend and most confusions make phonetic sense; for example,  $z$  getting confused as  $s$  ,or  $g$  getting confused as  $k$  .