Analyzing analytical methods: The case of phonology in neural models of spoken language (2020)

Abstract: Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method. As a step in this direction we study the case of representations of phonology in neural network models of spoken language. We use two commonly applied analytical techniques, diagnostic classifiers and representational similarity analysis, to quantify to what extent neural activation patterns encode phonemes and phoneme sequences. We manipulate two factors that can affect the outcome of analysis. First, we investigate the role of learning by comparing neural activations extracted from trained versus randomly-initialized models. Second, we examine the temporal scope of the activations by probing both local activations corresponding to a few milliseconds of the speech signal, and global activations pooled over the whole utterance. We conclude that reporting analysis results with randomly initialized models is crucial, and that global-scope methods tend to yield more consistent and interpretable results and we recommend their use as a complement to local-scope diagnostic methods.

## introduction as end-to-end architectures based on neural networks became the tool of choice for processing speech and language, there has been increased interest in techniques for analyzing and interpreting the representations emerging in these models. a large array of analytical techniques have been proposed and applied to diverse tasks and architectures @xcite . given the fast development of analysis techniques for nlp and speech processing systems, relatively few systematic studies have been conducted to compare the strengths and weaknesses of each methodology and to assess the reliability and explanatory power of their outcomes in controlled settings. this paper reports a step in this direction: as a case study, we examine the representation of phonology in neural network models of spoken language. we choose three different models that process speech signal as input, and analyze their learned neural representations. we use two commonly applied analytical techniques: (i) diagnostic models and (ii) representational similarity analysis to quantify to what extent neural activation patterns encode phonemes and phoneme sequences. in our experiments, we manipulate two important factors that can affect the outcome of analysis. one pitfall not always successfully avoided in work on neural representation analysis is the role of learning. previous work has shown that sometimes non-trivial representations can be found in the activation patterns of randomly initialized, untrained neural networks @xcite . here we investigate the representations of phonology in neural models of spoken language in light of this fact, as extant studies have not properly controlled for role of learning in these representations. the second manipulated factor in our experiments is the scope of the extracted neural activations. we control for the temporal scope, probing both local activations corresponding to a few milliseconds of the speech signal, as well as global activations pooled over the whole utterance. when applied to global-scope representations, both analysis methods detect a robust difference between the trained and randomly initialized target models. however we find that in our setting, rsa applied to local representations shows low correlations between phonemes and neural activation patterns for both trained and randomly initialized target models, and for one of the target models the local diagnostic classifier only shows a minor difference in the decodability of phonemes from randomly initialized versus trained network. this highlights the importance of reporting analysis results with randomly initialized models as a baseline. this paper comes with a repository which contains instructions and code to reproduce our experiments. foot_0 2 related work ## methods in this section we first describe the speech models which are the targets of our analyses, followed by a discussion of the methods used here to carry out these analyses. ## conclusion in this systematic study of analysis methods for neural models of spoken language we offered some suggestions on best practices in this endeavor. nevertheless our work is only a first step, and several limitations remain. the main challenge is that it is often difficult to completely control for the many factors of variation in the target models, due to the fact that a particular objective function, or even a dataset, may require relatively important architectural modifications. in future we will sample target models with a larger number of plausible combinations of factors. likewise, a choice of an analytical method may often entail changes in other aspects of the analysis: for example, unlike a global diagnostic classifier, global rsa captures the sequential order of phonemes. in future work we hope to further disentangle these differences.