Understanding Adaptive, Multiscale Temporal Integration In Deep Speech Recognition Systems
Authors: Menoua Keshishian
Samuel Victor Norman-Haignere
Nima Mesgarani
Venue: NeurIPS 2021 Poster (2021)
Keywords: temporal integration
receptive field
speech recognition
timescale
deep learning

Abstract: Natural signals such as speech are hierarchically structured across many different timescales, spanning tens (e.g., phonemes) to hundreds (e.g., words) of milliseconds, each of which is highly variable and context-dependent. While deep neural networks (DNNs) excel at recognizing complex patterns from natural signals, relatively little is known about how DNNs flexibly integrate across multiple timescales. Here, we show how a recently developed method for studying temporal integration in biological neural systems – the temporal context invariance (TCI) paradigm – can be used to understand temporal integration in DNNs. The method is simple: we measure responses to a large number of stimulus segments presented in two different contexts and estimate the smallest segment duration needed to achieve a context invariant response. We applied our method to understand how the popular DeepSpeech2 model learns to integrate across time in speech. We find that nearly all of the model units, even in recurrent layers, have a compact integration window within which stimuli substantially alter the response and outside of which stimuli have little effect. We show that training causes these integration windows to shrink at early layers and expand at higher layers, creating a hierarchy of integration windows across the network. Moreover, by measuring integration windows for time-stretched/compressed speech, we reveal a transition point, midway through the trained network, where integration windows become yoked to the duration of stimulus structures (e.g., phonemes or words) rather than absolute time. Similar phenomena were observed in a purely recurrent and purely convolutional network although structure-yoked integration was more prominent in the recurrent network. These findings suggest that deep speech recognition systems use a common motif to encode the hierarchical structure of speech: integrating across short, time-yoked windows at early layers and long, structure-yoked windows at later layers. Our method provides a straightforward and general-purpose toolkit for understanding temporal integration in black-box machine learning models.

Understanding Adaptive, Multiscale Temporal Integration In Deep Speech Recognition Systems
Menoua Keshishian"
Sam V. Norman-Haignere
Department of Electrical Engineering Zuckerman Mind Brain Behavior Institute
Department of Electrical Engineering
Zuckerman Mind Brain Behavior Institute
Columbia University
Columbia University
New York, NY 10027 mk401 1ete0111columba..omm edu
New York, NY 10027
sn2776@columbia edu
Nima Mesgarani
Department of Electrical Engineering Zuckerman Mind Brain Behavior Institute
Columbia University
New York, NY 10027 nimaβee. columbia edu
Abstract
Natural signals such as speech are hierarchically structured across many different timescales, spanning tens (e.g.. phonemes) to hundreds (e.g., words) of millisec- onds, each of which is highly variable and context-dependent While deep neural networks (DNNs) excel at recognizing complex patterns from natural signals, rela- tively little known about how DNNs flexibly integrate across multiple timescales. Here, show how recently developed method for studying temporal integration in biological neural systems the temporal context invariance (TCT) paradigm be used understand temporal integration in DNNs. The method is simple: we measure responses to a large number of stimulus segments presented in two different contexts and estimate the smallest segment duration needed achieve a context invariant response. We applied our method to understand how the popular DeepSpeech2 model learns to integrate across time in speech. We find that nearly all of the model units, even in recurrent layers, have compact integration window within which stimuli substantially alter the response and outside of which stimuli have little effect. We show that training causes these integration windows to shrink at early layers and expand higher layers, creating hierarchy of integration windows across the network. Moreover, by measuring integration windows for simesstrecchhed/coppssee speech, we reveal transition point, midway through the trained network, where integration windows become yoked to the duration of stimulus structures (e.g., phonemes or words) rather than absolute time. Similar phenomena were observed in a purely recurrent and purely convolutional network although cruuturr-yokke integration was more prominent in the recurrent network. These findings suggest that deep speech recognition systems to encode the hierarchical structure of speech: integrating across short, time-yoked windows at carly layers and long, structure-yoked windows at later layers. Our method provides a straightforward and general-purpose toolkif for understanding temporal integration in black-box machine learning models.
common motif
These authors contributed equally.
Code available
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Introduction
A central challenge of representing natural signals, such as speech and music, is that they are structured across many different timescales (Chomsky and Halle, 1968; Lerdahl and Jackendoff 1985, Hickok and Poeppel 2007). Speech, for instance, is composed of phonemes, syllables, and words spanning tens to hundreds of milliseconds, and the duration of these structures varies widely due to a myriad of factors (e.g., talker, speaking rate, prosody, etc.) (Figure [T. Intelligent systems, whether biological or artificial, must learn to flexibly integrate across these different timescales to derive meaning from natural signals. DNNs have proven remarkably effective at learning complex temporal structures from natural signals, such as speech, and the representations learned by these networks show non-trivial similarities to those present in the brain (Kell et al., 2018). However, relatively little is known about how deep speech recognition systems flexibly integrate across multiple timescales. Here, we address this gap by using a recently developed method for studying temporal integration in biological neural systems the temporal context invariance (TCI) paradigm (Norman- Haignere et al. 2020) to probe how a widely used deep speech recognition system (DeepSpeech2) learns to integrate across multiple timescales.
We study temporal integration by measuring how training alters the integration window of DNN units. Integration windows are defined as the time window when stimuli alter the neural response and outside of which stimuli have little effect (Theunissen and Miller 1995] Hasson et al.) 2008). Because this definition is simple and general, integration windows can be used to measure and compare the analysis timescale of responses from different models, stimuli, and modalities. While convolutional neural networks (CNNs) have a well-defined upper bound on their integration window, determined by the size of the temporal kernels, the effective integration window, when stimuli empirically alter the response, can differ substantially from this upper bound (Luo et al. 2016) and change substantially with training, as we show. Recurrent neural networks (R can in principle integrate over arbitrarily long time windows (Hochreiter and Schmidhuber 1997; Sussillo and Barak 2013). but in practice, RNNs also show have a much greater impact on the measured response (Mahto et al. 2020 Chien and Honey 2020).
effective integration windows within which stimuli
We have recently designed a method to estimate the effective integration window of an arbitrary response by presenting segments of natural stimuli (here speech) in two different random orders such that each segment occurs in two different contexts (i.e., is surrounded by different stimuli) (Figure et al. 2020). If the integration window is less than the segment duration, there will be a moment when the response is the same across the two contexts. We can thus vary the segment duration in order to find the smallest segment duration outside of which stimuli have little effect on the response. The random ordering of segments makes it possible to use a simple correlation measure quantify at each moment time the degree to which the response context invariant (Figure ())gureep..com as detailed below. The goal of this paper is to introduce this paradigm to the machine learning community and to use this paradigm to understand how deep speech recognition systems leam to flexibly integrate across multiple timescales in speech.
2h) (Norman-Haignen
2 Relationship with prior work
Understanding latent representations of DNNs is highly active area of research. A variety of feature visualization techniques have been developed to understand mrage-trained networks (Zeiler and Fergus, 2014: Mahendran and Vedaldi 2015 Yosinski et al., 2015) with less work focused on understanding speech-trained networks (Huang et al. 2015 Krug and Stober 2019 Begus and Zhou 2021 Keshishian et al.) 2020). Many visualization methods are based upon ca calculating the gradient of the network's ' response with respect to the input (Springenberg ct al. | 2014 Bach et al. 2015,Seelarre et al. 2016 Shrikumar et al., 2017 [Sundararajan et al. 20 17 Samek et al. 2021)) While for a linear system, the temporal extent of the gradient is equivalent to the integration window, for a nonlinear system, the link between the local gradient and the overall computation is unclear (Balduzzi et al. 2017). and as a consequence, there is a need for empirical methods that can estimate the effective integration window of highly nonlinear systems (Norman- Haignere et al. 2020 Chien
and Honey 2020).
Probing tasks have been used to examine the representation of particular structures, such as phonemes, across different layers of speech recognition networks (Belinkov and Glass! 2017] Nagamine et al.)
2015 Nagamine and Mesgarani 2017). and recent study examined the geometry of how invariant

---

Structure durations
b. Duration "BEFORE"
Word duration variability
Phoneme
Syllable (=1 phonel
syllable)
Density
Density
Density
Center 90%
200
0
150 Duration (ms)
900
400
800
1.41
2.83 Durational variability
Duration (ms)
Figure Duration statistics of speech structures in the LibriSpeech corpus. (a) Histogram of
durations for all phonemes, multi- mulli-phonemee syllables, and multi-syllable words. (b) Histogram of
durations for all utterances the word "before" in the corpus. The central 90% interval is marked in gray with the 5th and 95th percentile indicated. (c) Histogram of durational variability across all words in the corpus. Durational variability measured as the ratio between the 95th and 5th percentile duration for each word (as illustrated in panel b).
feature representations evolve time (Stephenson et al. 2020p. However, to knowledge, little known about how speech-traines networks learn to integrate across time, despite the centrality of temporal integration the task of speech recognition.
Neuroscience has produced a variety of experimental methods for studying sensory timescales. Scrambling methods have been used to examine selectivity for intact temporal structure at different timescales (Hasson et al. 2008 Overath et al. 2015). Chien and Honey (220)) recently introduced experimental method that estimates the delay needed for response become context- iivarrttt after a stimulus change, and they used this method to analyze neural language models (Chien and Honey 2020). However, this method does not directly measure the time window in the stimulus that is needed to achieve an invariant response, which requires measuring responses to segments of differing durations. Moreover, speech recognition presents many unique challenges that present in language modeling. For example, a key contribution of our paper is showing that there transition from fixed, time-yoked integration to adaptive, structure-yoked integration in deep speech recognition models, distinction that is not meaningful in language models where the input is inherently structure-based (i.e., words).
3
Temporal context invariance (TCI) paradigm
3.1 Stimuli
The input to our analysis is set of sound segments of varying duration. We tested durations ranging from 20 milliseconds and 2.48 seconds useudo-logarithmi. steps (20, 40, 60, 80, 100, 120, 140, 160, 200, 240, 280, 340, 400, 480, 580, 700, 840, 1000, 1200, 1440, 1720, 2060, 2480 ms).
Sound segments were excerpted from the LibriSpeech corpus' dev-clean and test-clean sets. We attempted to choose segments that were composed of words spoken at a moderate rate and which also had moderate amount of natural variation in their duration, such that stretching compressing the word would not place the word outside of its typical duration (stretching and compression analyses described below). To this end, we first selected all words spoken at least 100 times in the corpus. For each word, we computed histogram of durations, and we selected all words where the difference between the 5th and 95th percentile duration was at least factor of this durational variability metric across words). For each word, we then randomly selected up to 100 of its utterances that fell within the central 20% of duration distribution. This resulted in pool of 6,184 utterances across 1,095 unique words. Finally, for each utterance, we excerpted 23 segments, one per duration, with the middle of the word at the center of the segment (if the segment duration was shorter than the word, we excerpted just the middle portion of the word). This selection procedure resulted in 6, 184 unique segments per segment duration. We note, that while we used speech-specific recipe to extract segments, having in mind the types of experiments would
(see Figure [e for a histogram

---

Temporal context invariance (TCI) paradigm
b. Crose-context correlation
Correlation
Correlation
Segment duration Integration window
Integration
magnitude
Resporce
Same response
Integration Window
Segment duration Integration window
M Coreuaae
Different response
Same segment surrounded by different context segments
Time lag from shared segment
Same segments in different contexts
Cross-context correlation example
correlation vs. segment duration
Segment duration
40 ms
160 ms
700
2060
Correlation Correlation from LSTMS from LSTM1
Peak correlation
1.00
0.75
0.00
Peak correlation
1.00
0.75
0.00
0
2.48
2.48
2.48 0
2.48
40
160
700
2060
Time lag relative shared segment (gray)
Segment duration
Figure 2: Temporal context invariance (TCI) paradigm. (a) Schematic of the paradigm used to measure integration windows. Segments of natural stimuli (here speech) are presented using two different random sequences. As consequence, each segment is surrounded by different context in each sequence. the segment duration longer than the integration window, there will be moment when the integration window is fully contained within the segment, yielding similar response across the two contexts. (b) Schematic of the analysis used to assess context invariance for a single unit and segment duration. We organize the responses of unit all segments of given duration as a [segment time] matrix (the segment-aligned response or SAR matrix). Each row contains the unit's response to single segment, aligned to segment onset. separate SAR matrix is calculated for each sequence, such that corresponding rows contain the response to the same central segment, surrounded by different context segments. We correlate corresponding columns across the two SAR matrices (the cross-context correlation), schematized by linked columnar boxes. When the integration window overlaps the context segments (first box pair), we expect correlation of since the context is random. If the integration is less than the segment duration, there will be a moment/lag when fully contained within the shared central segments (gray region), yielding a correlation of at that moment (second box pair). (c) The cross-context correlation for two example units from different layers DeepSpeech2 (LSTMI and LSTM5). Each plot shows different segment duration. The gray region shows the time interval when the shared segment was present. Black dashed lines show the peak of the cross-contens correlation across all lags. (d) The peak correlation for all tested segment durations. Black dashed lines highlight the segment durations shown in panel c. The red dashed line indicates the segment duration needed to achieve peak correlation value 0.75, which how integration windows were defined.

---

conduct, this is not necessary or intrinsic to the TCI method. The segment extraction recipe can change depending on the nature of the stimuli and the intended experiments. Time aligmments for phonemes and words of the L.brriSpeech dataset were extracted from (Lugosch et al.] (019). which used the Montreal Forced Aligner to align the transcripts to sounds (McAuliffe et al.] (017). Syllables were identified based on the extracted phonemes using the syllabijy python library (Gorman 2013).
All of the segments from a given duration were randomly ordered and then concatenated with cross- fading at the boundary to minimize transition artifacts (20ms half-Hanning window). We used two different random orderings of the segments such that each segment occurred in two different contexts (Fig 2b). Due to limited memory, we divided the randomly ordered segments into 48-second stimuli with 16-seconds of overlap between consecutive stimuli. The responses to the first and last 8 seconds of each stimulus were discarded to avoid padding artifacts due to batching.
3.2 DeepSpeech2 model
We applied our TCI paradigm measure integration windows from DeepSpeech2 model (Amodei) et al. (016) (trained for 20 epochs). We used a publicly available implementation of this model (Naren 2019). Our DeepSpeech2 model consists of two 2D convolutional layers (Conv1/2), five bi-LSTM layers (LSTM1/2/3/4/5), and a softmax readout that is trained to predict graphemes using CTC loss (Graves et al., 2006). All layers except the readout included batch normalization. The model reached a word error rate (WER) of 7.6% on the LibriSpeech test-clean set after training for 20 epochs. We analyzed logarithmically transformed probabílities for the softmax readout (to avoid the high sparsity of probabilities). We compared results from our trained model with a randomly initialized model to assess effects of training.
All models were implemented in PyTorch (Paszke et al. 2019) and trained using Py Lightning (Falcon 2019) on the training set of the LibriSpeech corpus (Panayotov et al.] 2015). We used the CTC (Graves et al. (0066), the Adam optimizer (learning rate: 1.5e-4, weight decay: 1e-5) (Kingma and Ba 2014), and batch size of 64. To improve model generalization, we augmented the stimuli by adding background noise, reverberation, and frequency masking. Augmentations were performed using the Sound eXchange (SoX) backend of the audio library for PyTorch (torchaudio).
Training and inference of all models were performed on NVIDIA A40 GPUs (one per train- ing/inference) at the internal cluster at the Zuckerman Institute of Columbia University. Each epoch of training took approximately 2.5 hours for each model, for a total of 110 epochs across all models. For each experiment, we inference across three time-stretches and 23 segment durations, for an average of 22 hours per experiment. Inference was substantially faster for the convolutional model.
3.3 Cross-context correlation
Our analysis involves correlating the response of unit to different stimulus segments across two different contexts, analysis we refer to as the "cross-context correlation" (Figure 2p). The analysis is applied separately to different segment durations. For each segment duration, we have two stimulus sequences (r and It)( The stimulus sequences could be vector-valued as in spectrogram or scalar as in a waveform. Each sequence contains all the segments from that duration, randomly ordered and concatenated. For simplicity, we will ignore the fact that we divided our sequences into 48-second stimuli due to memory limitations and assume that there are just two long sequences, each of which creates unique context for each segment. Each unit in our model produces an output sequence in response to these two stimuli (we do not specify the index of the unit to simplify notation):
T [t] = f (d) r [1] = f (x
(2)
where f()) is scalar function specifying the ttimulus-responee mapping, here deep network. From these two response sequences we create two segment-by-time matrices, which we refer to as the segment- response (SAR) matrices (see Figure (2p for a schematic):
SAR [s, 1 = FA [s]
(3)
SAR [s, = rB [g]
(4)
5

---

where 0A[s] is the onset in samples of segment in sequence A, and og(s) is the onset of that same segment sequence B. Thus, each row of the SAR matrix contains the response sequence that surrounds a particular stimulus segment. Corresponding rows contain the response sequence to the same segment but surrounded by different context segments. The cross-context correlation (Pece (t)
is computed by correlating corresponding columns across the two SAR matrices:
Pece [t]
(5)
where coir indicates the Pearson correlation (though we see reason why another measure of dependence could not be used). The set of time lags (t) relative to segment onset should be larger than the integration windows being measured. Here used time lags from -1 seconds to T + seconds, where T is the segment duration. The sampling rate was 100 Hz.
The goal of our analysis is to assess if there is a lag when the response is the same across contexts. Before segment onset (lag<0), the cross-context correlation should be near zero for a causal response, since the integration window must overlap the preceding segments, which are random across contexts. As time progresses, the integration window will start to overlap the shared segment, and the cross- context correlation should increase. Critically, if the integration window is less than the segment duration, there will be a lag where the integration window is fully contained within the shared segment, and the response will thus be the same across contexts, yielding a correlation of 1. For non-causal response, such as a bidirectional LSTM, we expect the same pattern, but the build-up of the cross-contes correlation can begin before the onset of the shared segment.
We plot the cross-context correlation for segments of increasing duration for two example units from different layers of the DeepSpeech2 network (LSTM1 and LSTM5) (Figure[p). For both units, the cross-context correlation started at zero, rose near the onset of the shared segment, and fell back to zero after the offset of the shared segment. Critically, the peak of the cross-context correlation varied across segment durations and units. For the unit from the LSTM1 layer (top panel), the peak of the cross-context correlation was close to 1 for segment durations short as 160 milliseconds. For longer segment durations, the cross-contex. correlation remained high for an extended period, indicating that the unit's integration window was short relative to the segment duration and thus remained within the shared segments for an extended period. By comparison, the unit from the LSTM5 layer required much longer segment durations for the cross-context correlation to approach 1, and the offll-u of the cross-context correlation with lag was slower, implying a substantially longer integration window. These general trends were observed in virtually all units (Appendix Figure 1 shows several more units)
We quantified the integration window of each unit by measuring the peak of the cross-context correlation across lag as a function of the segment duration (Figure (t). The units showed clear pattern where the peak correlation rose steeply as the segment duration was increased, and then gradually approached an asymptotic value. This asymptotic value was high for nearly all units (Appendix Figure 2; 98.9% of units had an asymptotic correlation greater than 0.9), indicating mostly context-invariant response. We defined the integration window of each unit as the segment duration needed to achieve a correlation value of 0.75 (estimated by linearly interpolating the correlation vs. segment duration curves shown in Figure Results were robust to the choice of threshold, though higher thresholds inevitably result in longer integration windows.
4 Effect of training on network integration windows
Figure Plots the distribution of integration windows for the units in each layer for both trained and randomly initialized models. The integration windows from the random model varied little with layer, particularly amongst the five LSTM layers (median integration windows varied between 167 and 204 ms across the LSTM layers). We found that training caused the integration windows of the early layers (Convl LSTM2) to shrink and the integration windows of later layers (LSTM3- softmax) to expand (Wilcoxon rank-sum test comparing trained and random models across units, p < 1c-5). For the trained model, there was an approximately 10-fold increase between Convl (median window: 58 ms) and LSTM5 (median window: 592 ms). This pattern appears to qualitatively mimic the hierarchical nature of temporal integration in biological auditory systems (Khatami and Escabi 2020) Norman- Haignere et al. 2020).

---

1280
We found that time-stretchingccoppesss an untrained model, yielding adaptation indices near O. contrast, for the trained model, we found there was transition point between layers LSTM1 and LSTM3, where integration windows Random init.
Integration window (ms)
Trained
320
80
20
Convi
Conv2
LSTM1
LSTM2
LSTW3
LSTM
LSTM5
Softmox
Model layer
Figure 3: Effect of training on network integration windows. Violin plots show the distribution of integration windows across all units from each layer for trained (orange) and randomly initialized model (blue). The violin plots show the center 99% the distribution, discarding extreme outliers. Horizontal lines indicate distribution medians.
Time- vs. structure-yoked integration
The duration of speech structures such phonemes and words is highly variable (Figure D. Thus, natural question whether the time windows measured from the network reflect a static time window or instead reflect the duration of the structures features to which a unit responds. A simple way to address this question is to time-compress or time-stretch the stimulus such that all stimulus structures get mapped to new timescale. If the integration window is time-based. then the integration window should remain fixed, while if the integration window is structure-drivens should appear to shift in proportion to the degree of stretching/oompresioon..
Stretching and compression were implemented using the SoX backend of torchaudio, which uses the WSOLA algorithm to modify the rate of speech without changing the pitch. Stimuli were stretched or compressed by 20%. Results were robust to the amount of stretching/oompresionn (see Appendix Figure 3) and the algorithm used to perform stretching/comprrssionn (see Appendix Figure 4). WER on LibriSpeech test-clean was best for natural speech (7.6%) and worst for compressed speech (10.4%), with stretched speech (8.3%) only slightly worse than natural speech.
Figures J-- show seatter plots of integration windows measured for stretched vs. natural speech for different model layers (each dot corresponds to one unit; note that for this analysis only we used stretch factor of 60% so that it was visually easier to see the differences between stretched vs. natural speech). For early layers (e.g.. Convl), we found that integration windows were very similar for natural vs. stretched speech. windows increased by factor that was nearly the same as the amount of stretching in the stimulus,
contrast, for late layers (e.g., LSTM4), we found that integration
suggesting integration windows were determined nearly fully by the duration of stimulus structures.
To quantify this change, we computed the following index, reflecting the degree to which the integration window adapts the duration of stimulus structures:
Tmod/ Tref Dmod/ Dref -
Adaptation index
(6)
where mod is the measured integration window for stretched/compressed speech and Trey is the integration window for natural speech. Dmod and Dref are the relative durations of the stimulus structures for the corresponding conditions (i.c., Dmod/ Dref would be 1.2 for a stretch factor of 20%). An index of 0 indicates fixed, time-yoked window, while value of I indicates fully adaptive, structure-yoker window. Figures 4g plot this adaptation index across all layers for stretched (top panel) and compressed speech (bottom panel) from both trained (orange violins) and randomly initialized (blue violins) networks.
had little impact on measured integration windows from
shifted from mostly time-yoked integration (median adaptation index of 0. 19/0.21 in LSTM1 for stretched/compressedd speech) to mostly dtructureyyoken integration (median adaptation index of

---

Layer
Layer
Layer
g.Timescale adaptation time-stretching
Random init.
Adaptation index
Trained
Integration window for time-sttie time-stretched speech (ms)
Conv
15TM1
1STM2
20
1280
1280
1280
Layer
Summary plot
1280
1280
Cond2
Conv2
LSTM1
LSTM2
LSTM3
LSTM4
LSTMS
Sofmaa
Model layer
h. Timescaae adaptation to tomp-coop
LSTM4
320
1280
Random Init.
Conv1
trained
e. Readout
1280
80
LSTM1
Adaptation
LSTM2
LSTM
LSTMS
Softmax
20
Softmax
20
20
1280
20
60
320
1280
Conv1
Comv
LSTM1
LSTM2
LSTM3
LSTM
LSTMS
Softmas
Integration window for natural speech (ms)
Model layer
Figure 4: Integration windows for time- www.ttchhed/ooppeseeeeeeee speech. (a-e) For each layer, scatter plot shows the integration windows measured for stretched vs. natural speech. Each dot corresponds single unit. The black dashed line is the line of unity, showing what integration windows were unaffected by stretching. The red dashed line shows what one would expect if integration windows expanded in direct proportion to the magnitude of stretching. (f) Integration windows from all layers are jointly plotted on the same graph (colored by layer), illustrating the progression from short, time-yoked windows to long, structure-yoked windows. (g-h) For each unit, we computed an adaptation index, reflecting the degree to which its integration window shifted with the magnitude of tretchinggcompprssion. A value of 0 indicates a fixed, time-yoked window (i.e., near the black dashed line in panels a-f), while a value of 1 indicates fully adaptive, structure-yoker window (i.e., near the red dashed line panels a-f). Violin plots show the distribution of adaptation indices across all units from each layer. Results are plotted for trained (orange) and randomly initialized models (blue), measured with either stretching (g) or compression (h). Violin plots show the center 99% of the distribution, discarding extreme outliers. Horizontal lines indicate distribution medians.
would expect if
0.84/0.63 in LSTM3 for stretched/comprrsseed speech). The magnitude of adaptation was slightly larger for stretched vs. compressed speech in the later layers (Wilcoxon rank-sum test across units: p < 1e-5 for LSTM2 through LSTM5). This observation suggests that the network was better able to adapt its integration window for stretched speech, which might be related to the network's better performance for stretched (8.3% WER) vs. compressed speech (10.4% WER). Collectively, these findings suggest that the network learns to integrate across short, fixed time windows early layers and long, structure-yoked windows at later layers.
6 Architecture comparison
Theoretical arguments suggest that gated recurrent architectures, such as LSTMs, may be particularly well-suited to adaptive integration (Tallec and Ollivier 2018). Thus, natural question is whether our findings depend upon the presence of gated RNNs. To address this question, we analyzed two additional models: a fully-recurrent model with 7 bi-L.TMM layers; and cully-coovollttional model based on QuartzNet, which uses separable convolutions and residual connections (Kriman et al.) 2020) (simply repeating the convolutional layers DeepSpeech2 produced an architecture with very poor performance). We simplified the QuartzNet model by taking the 5x5 model (see (Kriman al. 20200), removing the fully-connected C3 layer, changing the number of output channels (C) of all hidden layers to 1024, changing the kemel size (K) of all hidden layers to 41, and using

---

a. Integration windows
b. Adaptation to time-stretching
c. Adaptation to time-compression
1280
Integration window (ms)
DS2
RNN
Adaptation index
RNN CNN
Adaptation index
RNN
CNN
CNN
320
0.5
0.5
20
Layer-
Layer-2
Layer-3
Layor-d
Laver-S
Layar-6
Layer-
Saltmax
Layar-1
Laystr-2
Layer:g
Laytr-4
Layer-
Layer Layer-
Softmax
Layer-3
Layer-2
Layer-3
Layor-
Layer-S
Layar-6
Layer
Soltmax
Model layer
Model
laye-
Model layer
Figure 5: Effect of model architecture. (a-c) Integration window and adaptation index distributions for DeepSpeech2 (blue), fully-recurrent (orange), and fully-convolutional (green) architectures. The violin plots show the center 99% of the distribution, discarding extreme outliers. Horizontal lines indicate distribution medians.
symmetrical padding such that the model would be non-causal, similar to the bi-LSTM modules. For the CNN model, we do not show results from the intermediate layers within each residual block to make the results easier to compare with the other models (see Appendix Figure for all layers). We approximately matched the performance of the three models (DeepSpeech, RNN, CNN) by selecting the training epoch that led to WER closest to 10% on the LibriSpeech test-clean set (10 epochs for DeepSpeech2 and RNN, 50 epochs for the CNN).
Both the convolutional and recurrent networks learned similar hierarchy of integration windows for natural speech, with substantially longer integration windows in later network layers (Figure (5). Moreover, both convolutional and recurrent models showed greater adaptation at later layers, similar to DeepSpeech2. However, the magnitude of adaptation was greater for both DeepSpeech2 and the fully recurrent model, compared with the convolutional model (Figure 5D-)). These results indicate that core findings do not depend recurrence, but also suggest that RNNs may show a greater predisposition for adaptive integration.
7 Robustness to task, stimuli, and context
We performed several additional experiments to test the robustmess of our findings. We found similar results for a model directly trained to recognize words, as opposed to graphemes (Appendix Figure 6). We also observed similar results for frequent and infrequent words, demonstrating that temporal adaptation is not limited to frequent words. Our paradigm also allows us to consider "natural contexts", when a segment is subset of longer segment and thus surrounded by its actual context in natural speech, addition to the case described so far when segment surrounded by random other segments. While one of the two contexts being compared has to be random that the contexts differ, the other context can be random or natural (see Appendix Figure 7 for a schematic). In practice, we observed similar results using mandom-randon and random-natura contexts (Appendix Figure 7), suggesting that our findings are robust to the particular type of context used to assess invariance.
8 Conclusion
We have described an approach for understanding how DNNs learn to integrate across time in complex natural signals, inspired by recent work in understanding temporal integration biological systems (Norman-Haiil Haignere et al. 2020). Our method involves presenting segments of natural stimuli in two different contexts and estimating the smallest segment duration outside of which responses are largely invariant to the surrounding context stimuli. Using this approach, we have shown that deep speech recognition models integrate across time using short, fixed windows at early layers and long, adaptive windows at later layers. These findings were robust across the architecture (Figure task (Appendix Figure 6), speech stimuli (Appendix Figure 8), and the type of context used to assess context invariance (Appendix Figure 7).

---

We emphasize that our core method is simple and general. Nothing about our approach is specific to speech or the particular models tested. We thus believe that the approach described this paper should be widely applicable to understanding how intelligent systems integrate across the complex temporal structures that compose natural signals.
Limitations
The uniform stretchingccompresiionn used to assess adaptation is unnatural and thus it possible that we might observe greater adaptation with more natural forms of stretching/copprssion. Uniform strecching/compressonn is experimentally useful because it shifts the timescale of all stimulus struc- tures, and thus should produce a shift in integration windows regardless of the particular structures unit represents.
Due to time limitations, we were only able to explore limited number of architectures, tasks, and stimuli, though we observed similar results across all of these manipulations. Future work could thus expand upon our findings by exploring wider range of architectures (e.g., attention-base architectures), tasks (e.g., speaker recognition), and stimuli (e.g., music).
10 Broader impacts
Our findings raise questions about the extent to which biological and artificial systems use similar mechanisms to integrate across time. The hierarchy of temporal integration windows learned by DeepSpeech2 qualitatively mimics the hierarchy of integration windows observed in biological auditory systems (Khatami and Escabi [2020] Norman-Haignerr et al. (020)). A key long-term goal of this work to build computational models that can mimic the mechanisms of temporal integration in the brain. Building accurate computational models of brain responses has many potential benefits, including better understanding and treating sensory deficits. We note that neural network training consumes large amounts of energy which could have a negative impact on the environment.
Acknowledgments and Disclosure of Funding
This work was funded by NIH LKK99CC08001100AA to NM) and a grant from Marie-Josée and Henry R. Kravis.
8051-01A1 to SNH, ROIDC014279 and IROIEB028155
References
Amodei, D., R. Anubhai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, J. Chen, M. Chrzanowski,
A. Coates, G. Diamos, E. Elsen, Engel, L. Fan, C. Fougner, T. Han, A. Hannun, B. Jun, P. LeGresley, L. Lin, S. Narang, A. Ng, S. Ozair, R. Prenger, J. Raiman, S. Satheesh, D. Seetapun, S. Sengupta, Y. Wang, Z. Wang, C. Wang, B. Xiao, D. Yogatama, J. Zhan, and Z. Zhu (2016).
Deep speech 2: End-to-end speech recognition in english and mandarin. International conference on machine learning, 173-182.
Bach, S., A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS One.
Balduzzi, D., M. Frean, L. Leary, J. Lewis, K. W.DD Ma, and B. McWilliams (2017). The shattered gradients problem: If resnets are the answer, then what is the question? International Conference on Machine Learning, 342-350.
Beguš, G. and A. Zhou (2021). Interpreting intermediate convolutional layers of cnns trained on raw speech. arXiv: /0::9489
Belinkov, Y. and Glass (2017). Analyzing hidden representations in end-to-end automatic speech recognition systems. arXiv: /Tfo:::700448
Chien, H.-Y. S. and C. J. Honey (2020). Constructing and forgetting temporal context in the human cerebral cortex. Neuron.
10

---

Chomsky, N. and M. Halle (1968). The sound pattern of English. Harper & Row.
Falcon, W. (2019). Pytorch lightning python module. [Github repository) Saaprchliggaainngppoooooo---leee
httpa://github.cco
Gorman, K. (2013). Syllabify python module for syllabifying arpabet transcriptions.
[Github
Graves, A., S. Fernández, F. Gomez, and J. Schmidhuber (2006). Connectionist temporal classifi- cation: labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd international conference Machine learning, 369-376.
Hasson, U., E. Yang, I. Vallines, D. J. Heeger, and N. Rubin (2008). A hierarchy of temporal receptive windows in human cortex. Journal of Neuroscience, 28:2539-255 2550..
Hickok, G. and D. Poeppel (2007). The cortical organization of speech processing. Nature Reviews Neuroscience, 8:393-402.
Hochreiter, S. and J. Schmidhuber (1997). Long short-term memory. Neural computation, 9:1735-1780.
Huang, J.-T., Li, and Y. Gong (2015). An analysis of convolutional neural networks for speech recognition. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4989 4993.
Kell, A. J. E., D. K. Yamins, E. N. Shook, S. V. Norman-Haignere, and J. H. McDermott (2018). A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy. Neuron.
Keshishian, M.. H. Akbari, Khalighinejad, J. L. Herrero, A. D. Mehta, and N. Mesgarani (2020). Estimating and interpreting nonlinear receptive field of sensory neural responses with deep neural network models. eLife 2020:9:e53445.
Khatami, F. and M. A. Escabí (2020). Spiking network optimized for word recognition in noise predicts auditory system hierarchy. PLOS Computational Biology.
Kingma, D. and J. Ba (2014). Adam: method for stochastic optimization. arXiv:1412.660
Kriman, S., S. Beliaev, B. Ginsburg, J. Huang, O. Kuchaiev, V. Lavrukhin, R. Leary, J. Li, and Y. Zhang (2020). Quartznet: Deep automatic speech recognition with Id time-channel separable convolutions. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6124-6128.
Krug, A. and S. Stober (2019). Visualizing deep neural networks for speech recognition with learned topographic filter maps. arXiv:191204067, 6124-6128.
Lerdahl, F. and R. S. Jackendoff (1985). A generative theory of tonal music. MIT press.
Lugosch, L., M. Ravanelli, Ignoto, V. S. Tomar, and Y. Bengio (2019), Speech model pre-training for end-to-end spoken language understanding. INTERSPEECH--2119
Luo, W., Y. Li, R. Urtasun, and R. Zemel (2016). Understanding the effective receptive field in deep convolutional neural networks. 30th Conference on Neural Information Processing Systems (NIPS).
Mahendran, A. and A. Vedaldi (2015). Understanding deep image representations by inverting them. Proceedings of the IEEE conference computer vision and pattern recognition, 5188-5196.
Mahto, S., V. A. Vo, S. Turck, and A. Huth (2020). language models. arXiv:200112227.
representation learning in Istm
McAuliffe, M., M. Socolof, Mihue, M. Wagner, and M. Sonderegger (2017). Montreal forced aligner: trainable text-speech alignment using kaldi. INTERSPEECH-2017.

---

Nagamine, T. and N. Mesgarani (2017). Understanding the representation and computation of multilayer perceptrons: case study speech recognition. Proceedings of the 34th International Conference on Machine Learning 70, 2564-2573.
Nagamine, T., M. L. Seltzer, and N. Mesgarani (2015). Exploring how deep neural networks form phonemic categories. INTERSPEECH-2015, 1912-1916.
Naren, S. (2019). Deepspeech2 pytorch implementation. [Github repository) https://githi: Com http:/Sannaarenddeepppeee pytorch
Norman-Haignere, S. V.. Long, O. Devinsky, W. Doyle, I. Irobunda, E. M. Merricks, N. A. Feldstein, G. M. McKhann, C. A. Schevon, A. Flinker, and N. Mesgarani (2020). Multiscale integration organizes hierarchical computation human auditory cortex. bioRxiv.
Overath, T., J. H. McDermott, J. M. Zarate, and Poeppel (2015). The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nature Neuroscience.
Panayotov, V., G. Chen, D. Povey, and S. Khudanpur (2015). Librispeech: An corpus based on public domain audio books. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
Paszke, A., S. Gross, F. Massa, A. Lerer, Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L.. Fang, J. Bai, and S. Chintala (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32, pp. 8024-8035.
Samek, W., G. Montavon, S. Lapuschkin, C. J. Anders, and K.-.. Müller (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE.
Selvaraju, R. R., A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra (2016). Grad-cam: Why did you say that?
Shrikumar, A., P. Greenside, and A. Kundaje (2017). Learning important features through propagating activation differences. International Conference on Machine Learning, 3145-3153.
Springenberg, J. T., A. Dosovitskiy, T. Brox, and M. Riedmiller (2014). Striving for simplicity: The all convolutional net. arXiv: 14/:::04/2666
Stephenson, C., J. Feather, S. Padhy, O. Elibol, H. Tang. J. McDermott, and S. Chung (2020). Untangling in invariant speech recognition. arXiv:200301788.
Sundararajan, M., A. Taly, and Q. Yan (2017). Axiomatic attribution for deep networks. International Conference on Machine Learning, 3319-3328.
Sussillo, D. and O. Barak (2013). Opening the black box: low-dimensional dynamics in high- dimensional recurrent neural networks. Neural computation, 25:626-6644 649.
Tallec, C. and Y. Ollivier (2018). Can recurrent neural networks warp time? arXiv: 1188.
Theunissen, and J. P. Miller (1995). Temporal encoding in nervous systems: rigorous definition. Journal of Computational Neuroscience, 2:149-166 162..
Yosinski, J., J. Clune, A. Nguyen, T. Fuchs, and H. Lipson (2015). Understanding neural networks through deep visualization. arXiv:150606579
Zeiler, M. D. and R. Fergus (2014). Visualizing and understanding convolutional networks. European conference on computer vision, 318-833.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Described in the limitations
section.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as URL)? [Yes] The speech dataset and model implementation are publicly available and cited. Code implementing cross- coss-context correlation analysis is available GitHub.
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A] We plot the distribution of values using violin plots.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you're rou're using/curating? [No]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "bWN-A16iMoh", "reviewer": "Reviewer_Dxi4", "summary": "The paper proposes an method of analyzing individual neurons in a neural network based on monitoring the output of a particular neuron while changing the size of input segments at the center and their context. The concept of integration window is introduced and defined as the size of the center segment that achieves a maximum correlation of 0.75 while changing the context. The paper uses DeepSpeech2 as a model to demonstrate the approach, and finds that the integration window significantly increases as we move deeper into the network and further away from the input. Further experiments are done to study the impact of stretching and compression the input spectrogram on the integration windows. The results show that the integration windows \"adapt\" better at the higher layers, meaning that stretching the input also stretches the integration windows, while the lower layers adapt less.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I am giving the paper a score 7, because the approach is novel and the results of the analyzing DeepSpeech2 themselves are a significant contribution.\n\n# Novelty\n\nThe approach is novel, though there are a few closely related concepts not mentioned in the prior work. Changing the context while measuring the output in the limit is in fact measuring the gradient. A few gradient-based approach are listed below.\n\nDeep inside convolutional networks: visualising image classification models\nKaren Simonyan, Andrea Vedaldi, Andrew Zisserman\nWorkshop at ICLR, 2014\n\nStriving for simplicity: the all convolution\nJost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller\narXiv:1412.6806\n\nGrad-CAM: Why did you say that?\nRamprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra\nICCV, 2017\n\n\n# Approach\n\nThe major assumption of the approach is that analyzing the response of segments with arbitrary context concatenated is meaningful. The network is trained on continuous speech without the artifacts of concatenation, so in principle the concatenated input segments are considered out of domain and the results could really be anything. However, from the experiments, it seems that we have enough evidence to believe that the results are reliable.\n\nThe same applies to the stretching and compression experiments. I believe the input to the network is in the log magnitude domain, and a simple interpolation might introduce artifacts that are simply not natural and can be considered out of domain. Again, the experiments seem to suggest that the results are reliable.\n\nAnother relative strict constraint not considered in the approach is that fact that first two convolution layers have finite receptive fields. In other words, the integration windows are capped by the hard constraints.\n\nAs mentioned before, the approach is reminiscent to the gradient-based approaches. It would be great to use another set of analysis approaches to at least see if the findings are consistent.\n\n\n# Presentation\n\nThe paper is a nice read.\n\nI find the caption in Figure 2 a bit too long, and some of them are actually repeated in the main text.\n\nI also find the text description in 3.3 extremely hard to follow. This is a good place to use mathematical notations to precisely define what is being computed to improve understanding.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed the limitation of the approach and its potential social impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "NM8kYSRmU2x", "reviewer": "Reviewer_mPjw", "summary": "In this work, the authors conduct analyses to understand how ASR models -- specifically, DeepSpeech2 -- integrates temporal information in various layers of the model. The analysis is based on the temporal context invariance (TCI) paradigm proposed for biological neural systems which is based on studying the output from a neuron from segments of different lengths in two different contexts; correlated outputs indicate sensitivity to the particular input and the length of the input segments indicate how long the segments need to be to produce relatively invariant outputs. The analyses indicate that the networks become increasingly sensitive to longer time scales and that earlier layers in the network are more responsive to static time windows whereas deeper layers are sensitive to the duration of the structures to which it responds.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I found this paper to be really interesting and intriguing. Generally speaking, the paper is excellently presented. The work is extremely well motivated, with detailed discussion of related work. The experiment in Section 4 was particularly interesting. However, I do have a number of comments that I would like the authors to address:\n\n- **Dataset creation and SAR Matrix Description**: While I generally understood the gist of the argument, one aspect of the paper that I found hard to follow, was the fact that the paper presents all ideas in prose rather than through mathematical notation which makes the work somewhat imprecise and ambiguous. In particular, Section 3.3 which describes the creation of the SAR matrix is somewhat vague (e.g., “. First, we organize the response timecourses to all segments of a given duration as a matrix, which we refer to as the segment-aligned response (SAR) matrix (Figure 2b). Each row of the SAR matrix contains the response timecourse surrounding a single segment, aligned to segment onset. Different rows thus correspond to different segments and different columns correspond to different lags relative to segment onset.”) I wasn’t sure what the authors meant by “timecourses”. I think this section would be much clearer and easier to understand if this was described with precise mathematical notation. Currently, this is described in Section 3.3 and the caption to Figure 2 which together take up almost 2 pages of the paper. \n\n- **Specific choice of stimuli**: I think that Section 3.1 which describes how the dataset used to build the SAR matrix was constructed could be re-written to be a bit more precise. In particular, something I was curious about was what fraction of the words chosen for the analysis involve common versus uncommon words. For example, I could imagine that the behavior is very different for common words such as “a”, “an”, “the”, etc. versus more uncommon words. I would be curious to see some analysis along these lines.\n\n- **Bidirectional Recurrent Layers**: In the caption to figure 2, the authors state: “If the segment duration is longer than the integration window (top panel), there will be a moment when the window is fully contained within each segment. As a consequence, the response at that moment will by definition be unaffected by the surrounding context segments.” Given that the model uses bidirectional recurrent layers, is it not still possible for the model to be affected by surrounding context segments? Could the authors please clarify if this is an assumption in the TCI paradigm, or if this still holds in some way.\n\n- **Limitations**: A couple of limitations of this work is that the analyses are all limited to a particular model architecture. It would be interesting to conduct similar analyses on a variety of different architectures (e.g., only LSTMs, only Feedforward layers, etc.), and to vary the output units (e.g., sentence-pieces/words instead of graphemes) to see if the observations are similar different in these cases.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Apart from the limitations mentioned above, I do not see any negative societal impacts of this work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "MAaPJeITAkO", "reviewer": "Reviewer_XEMX", "summary": "This paper addressed the topic of identifying the integration window of hidden layers within ASRs. Specifically, the work applies temporal context invariance proposed in “Multiscale integration organizes hierarchical computation in human auditory cortex“ to study the integration window of each layer in ASRs.\nThe integration window is defined as the latency for which neurons provide saturated responses to a certain signal. In other words, the time a unit needs for identifying the underlying temporal signals is its integration window.\nThe approach infers integration window through comparing the correlation between the response to the same signal segment within different utterances, where the window length is the time when the correlation exceeds a predefined threshold (0.75 in the paper).\nThe paper also studies the effect of stretching and shrinking the audio, and showed that upper layers’ integration windows are more proportional to time scaling while lower layers are more invariant.\nThe time-variance study provides an insight on whether the approach is effective on inferring the integration window of each layer: if the approach or the underlying unit simply reflect the time scale of the signal then it should scale proportionally with the input, while if the approach or the unit only reflect a fixed time response then they should be completely invariance. \nThe time-variance study shows that all layers within an untrained ASR are invariant to the time scaling, where the trained ASR exhibits different variance at different layers.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper provides a mechanism to help understand the temporal window of underlying layers of ASRs, which allows gaining insights about ASRs’ behaviors and are quite meaningful to the speech community. The time-variance study further reveals each layer’s temporal behavior and proves the effectiveness of the approach. One small drawback is that the criterion for preparing the input signals makes it less flexible and harder to scale. On the other hand, given the access to the semantic of the input signal, a further analysis on how semantics affect integration windows can provide deeper understanding on the ASR behavior.\nRelated works for identifying the width of the receptive field in artificial neural networks should be discussed, for example like “Understanding the Effective Receptive Field in Deep Convolutional Neural Networks”.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors have reasonably discussed their limitations.\nAuthors mentioned exploring how architecture can affect the analysis as a future work; it will be interesting to include self-attention-based components like Transformer/Conformer and compare how they differ from RNNs, in particular that Transformer/Conformer allows limiting the self-attention window and thus can exhibit distinct behaviors than RNNs.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "37sEZyOvZp_", "reviewer": "Reviewer_GCtL", "summary": "The authors investigate the DeepSpeech2 speech recognition model using temporal context invariance (TCI), which was recently introduced and used to study temporal integration in biological systems. The analysis shows that the \"integration windows\" for which the intermediate outputs of DeepSpeech2 are approximately independent of additional context are organized hierarchically, with lower layers integrating across shorter, fixed, time windows, and progressively higher layers integrating over longer timescales, which they show are progressively more governed by the structural evolution of speech rather than absolute time.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- Solid analysis of the temporal response properties of a well known speech recognition model.\n\nLimitations:\n- While interesting, the results are as expected, making them perhaps less significant. Also, DeepSpeech2 is an older speech model, and the findings fall short of identifying shortcomings or limitations of the model, or deep learning more broadly, to inform future research.\n- The TCI analysis is applied only to one model, leaving contrastive analysis across architectures and applications for future work. \n- I would have really liked to see the authors open source a TCI analysis framework for ML researchers to analyze their models with...\n\nCurrent Assessment:\n\n- A solid analysis of the DeepSpeech2 model with the TCI framework. Lower significance. \n\nPost-rebuttal:\n\nThank you to the authors for their detailed responses. Based on the responses to the reviewers' questions regarding the analysis, and the clarification regarding the release of a TCI analysis tool, I have increased my score by 1 to 6. The rebuttal includes several promissory updates to the paper, including clearer algorithmic descriptions, and a contrastive analysis across architectures, which I expect will further strengthen the paper (the expected result being a \"good\" paper (7)).", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper the authors use temporal context invariance (TCI) to understand temporal integration in black-box deep neural networks. The study is carried out using the DeepSpeech 2 model. The authors find that integratoin windows are specialized to different time scales in different layers,  the higher the layers the more sensitive to longer time scales.  In addition, lower layers are more sensitive to static windows while higher layers are more adaptive to duration of structures.  All reviewers find the work interesting and novel in methodology.  The hierarchical structure and pattern of response revealed in the experiments, despite in line with some previously observations/speculations, are still quite interesting and intriguing.  The method investigated in the paper also can potentially provide a new tool to the community to gain insights into the black-box machine learning models.  I would recommend accept.   The authors should address the questions raised by the reviewers in the revision. If possible, please also include new experimental results on other model architectures to make the paper stronger. 
Decision: Accept (Poster)