Variable-rate hierarchical CPC leads to acoustic unit discovery in speech
Authors: Santiago Cuervo
Adrian Lancucki
Ricard Marxer
Paweł Rychlikowski
Jan K Chorowski
Venue: NeurIPS 2022 Accept (2022)
Keywords: representation learning
self-supervised learning
acoustic unit discovery
speech
spoken language
contrastive predictive coding
hierarchical learning
(application) signal and speech processing

Abstract: The success of deep learning comes from its ability to capture the hierarchical structure of data by learning high-level representations defined in terms of low-level ones. In this paper we explore self-supervised learning of hierarchical representations of speech by applying multiple levels of Contrastive Predictive Coding (CPC). We observe that simply stacking two CPC models does not yield significant improvements over single-level architectures. Inspired by the fact that speech is often described as a sequence of discrete units unevenly distributed in time, we propose a model in which the output of a low-level CPC module is non-uniformly downsampled to directly minimize the loss of a high-level CPC module. The latter is designed to also enforce a prior of separability and discreteness in its representations by enforcing dissimilarity of successive high-level representations through focused negative sampling, and by quantization of the prediction targets. Accounting for the structure of the speech signal improves upon single-level CPC features and enhances the disentanglement of the learned representations, as measured by downstream speech recognition tasks, while resulting in a meaningful segmentation of the signal that closely resembles phone boundaries.

Variable-rate hierarchical CPC
leads to acoustic unit discovery in speech
Santiago Cuervo 1.2
Adrian Lańcucki
Ricard Marxer
Pawel Rychlikowski-
Jan Chorowski!
University of Wroclaw, Poland
Université de Toulon, Aix Marseille Univ, CNRS, LIS, France 3 NVIDIA, Poland Pathway, France
Abstract
The success deep learning comes from its ability to capture the hierarchical structure of data learning high-level representations defined in terms of low-level ones. this paper we explore self-supervised learning of hierarchical represen- tations of speech by applying multiple levels of Contrastive Predictive Coding (CPC). We observe that simply stacking two CPC models does not yield significant improvements
single-level architectures. Inspired by the fact that speech is often described as a sequence of discrete units unevenly distributed in time, we propose model in which the output of low-level CPC module is non-uniformly downsampled to directly minimize the loss of high-level CPC module. The latter is designed also enforce prior of separability and discreteness in its representations by enforcing dissimilarity of successive high-level representations through focused negative sampling, and by quantization of the prediction targets. Accounting for the structure of the speech signal improves upon single-level CPC features and enhances the disentanglement of the learned representations, as mea- sured by downstream speech recognition tasks, while resulting in meaningful segmentation of the signal that closely resembles phone boundaries.
Introduction
Perceptual signals, such as speech, images, or natural language are hierarchical, and exhibit non- uniform information density. We believe that by caring for these two aspects of the data can learn improved representations of such signals. Modern self-supervised representation learners do so only partially: while they are implemented using deep neural networks which internally learn hierarchy of concepts, the usual training criteria are only applied at the top layer and do not fully benefit from the hierarchical nature of the signals. Moreover, the models often assume uniform information density and process the data at fixed input-driven rates (e.g.. by striding or uniform pooling over time or pixels on speech and images, respectively). We demonstrate that the quality of learned representations can be improved by explicitly modeling the data at two different, non-uniform sampling rates, and with different training criteria applied each level of data modeling.
We demonstrate our results on speech, which is a hierarchical signal with a well understood structure. We extend frame-wise feature extraction model with boundary predictor, followed by another feature transformation applied to variable length contiguous segments of frames. We show that through careful design of the second level training criterion we can improve on the quality of
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

the learned representations and perform unit discovery by detecting boundaries which overlap with ground truth segmentation of the data. Therefore, we show the benefits of coupling variable-rate data representations with several levels of contrastive training.
1.1
Background: unsupervised representation learning
The deep learning revolution was started by layerwise stacking of pre-trained shallow models such as Restricted Boltzman Machines [Hinton and Salakhutdinov 2006 or denoising autoencoders (Vincent ett al. 2008]. However, it was soon observed that deep models can be trained from scratch without the need for unsupervised pretraining (Krizhevsky et al. [2012). Consequently, unsupervised feature modeling was limited to learning shallow word representations Miioooov et al., 2013). Recently, novel self-supervised training criteria such as unmasking (Devlin al. 2019]. contrastive predictions
van den Oord et al. 2018 Baevski et al. 2020 Chen et al. 2020 have facilitated pre-training deep
networks on unlabeled datasets.
Despite these advances, improving the quality of self-supervised representations of data by modeling the signal several levels of hierarchy remains an unsolved problem. [öwe al. [2019) have stacked several Contrastive Predictive Coding (CPC, van den Oord et al. (2018]) modules, however their goal not to improve the representations, but to avoid global backpropagation training. In the speech domain, their pre-training regime has failed to match the quality attained by training of a single deep CPC network. Bhati al. [2021] introduced a hierarchical model which stacked two CPC modules separated by differentiable boundary detector. They demonstrated that adding the second level of CPC modeling leads improved phone boundary detection, but the expense of the predictive capability of the learned features. This tension between boundary detection and feature informativeness was further confirmed by Cuervo et al. (2022).
We demonstrate that is possible to improve the predictive quality of features by introducing a second CPC level which is trained on variable length segments extracted to directly optimize the performance of the second-level CPC criterion. This result has several implications. First, it demonstrates the benefits of more closely matching the variable information rate of speech. Second, it paves the way for unsupervised discovery of structure in data, for instance acoustic unit discovery in speech.
1.2 Unit discovery in speech
Speech a well-suited benehmark modality for structure discovery using predictive coding methods. Patterns at the level of acoustic frames change in predictable way due to the physical constraints of the vocal apparatus. Yet these evolve depending on higher level latent variables, such as phones or prosody. Due to the structure of language, speech is also predictable at the level of phones: in English we would expect the phone "IH" to be followed by the phone "T" to form the word "it". At word level, same rationale can be applied to other language-related signals such as handwritten text.
predict that the pronoun "it" will be followed by a verb like "is" or "does". The
Furthermore, acoustic units have some easy to characterise properties. For instance, phone durations are limited by the rate at which they can be articulated and perceived, averaging around 100 ms in English. Useful units are also contrastive, for easier distinguishing between subsequent phones by a listener, and inherently redundant, which makes them robust to external noise, but also more predictable. Existing approaches to acoustic unit discovery impose those properties on the features extracted with a frame-level, self-supervised model. The main idea behind our method to enforce such priors of average duration, contrastiveness and predictability in differentiable, self-supervised objective function, which a two-level formulation permits. By optimizing this function we learn to extract information bearing units from unlabeled continuous sequential data.
2
Model
We describe the operation of our model the speech domain, noting that the design considerations involved could apply to other kinds of sequential data with inherent discrete structure, such as handwriting encoded as long lines of written characters. The major parts of the model (Figure are two CPC modules, separated by trained variable-rate downsampling segment extractor.

---

Reward
52
84
LH
CPC
Segment encoder
F6
By
Segment houndary
K0.7
Fy
Rg
Stare
K0.8
Quuniication
Action
Low--fevel Jenture extractor
Downsampling module
High--eveel CPC feature cxtractor
The model is composed of three parts. The low-level feature extractor is a CPC model which processes the signal at the frame level. The downsampling module groups and averages segments of adjacent representations. It is trained by reinforcement learning to compresses the features depending on their information content. Finally, the bigh-level CPC feature extractor models slowly changing features. The loss Lh is used as reward to guide training of the policy. Low-level CPC is depicted as black box, and high-level CPC detail to show quantization and adjacent negative sampling.
Figure 1:
Low-level feature extractor The feature extractor a typical CPC network for audio data van den Oord et 2018 Rivière al.) 2020] that models the regularities of the signal at the acoustic level. takes as input the raw waveform I and maps it to sequences of encodings z E R xd and context vectors R both d-dimensional vectors calculated for all T time steps. The encodings produced by a strided convolutional encoder Jene and contain local information on the receptive field of the encoder time 2t (xnc( The context vectors are the output of recurrent neural network for which summarizes observations to time t: c. gar (Z<e).
Downsampling module This module aims to downsample the frame representations depending on their information content
improve the performance of the higher level feature extractor. It contains a boundary predictor and an average pooling layer. The boundary predictor a transformer network with bidirectional attention [Vaswani et al.) 2017) that predicts sequence of unit boundary indicators b E {0, 1) conditioned on z. For two consecutive non-zero elements in by and bi+1, the pooling layer compresses the sub-sequence of encodings between boundaries 2t into single segment representation
Zt+i-
High-level feature extractor After downsampling, the resulting sequence of segment representations is processed by a second CPC network aiming to model the signal at the level of discrete units. It maps the sequence s E RT'xx xd' to sequences of pseudo-unit representations E R xd' and unit-level context representations h E Ig xd' The pseudo-unit representations are produced by fully- connected network Senci Ug (s,). The context vectors are the output of recurrent neural network Sar which summarizes the history of aseudo-units up to the k-th (high-level) time-step: h4 = (ucku) For the CPC task at the segment level we quantize the prediction targets using an online k-means vector quantizer [van den Oord et al.) 2017 g, which transforms the sequence of candidate units into a sequence of quantized targets wi..... u..:: u q(u)). Quantizing the targets has been shown to improve the quality of speech representations in contrastive prediction-bass self-supervised learning Baevski et al.) 2020) Nguyen et al. 2022], perhaps by contributing to the prior of discreteness wish to impose the high-level features.

---

2.1 Training objective
The objective function capturing our strategy modeling the signal simultaneously at the level of high resolution low-level continuous features and variable-rate high-level discrete umits can be simply defined as the sum of CPC losses L1 and Cи operating at both levels. Because the hard decisions of the boundary predictor in the downsampling module are not differentiable, we model the boundary detector as stochastic reinforcement learning policy. loss L* minimizes the expected value the high-level loss L.. whose gradient can be estimated through policy gradient algorithms [Williams/ 1992. Sutton et al. [999). Finally, the model also learns to optimize the loss Lo of the k-means quantizer in the high-level module, and regularization term Lj for the boundary predictor, which reduces the search space of the policy by imposing a prior on the expected average sampling rate of discrete units. The total loss is the sum of all described terms L L1 LH + LQ L, Lr.
Low-level CPC The low-level feature extractor is trained to minimize the standard CPC loss
exp (xp(pT 2t+x) E2,EN exp(
CL --=-E
where Pi.....PP PN are the output of N prediction heads that predict the N upcoming encodings conditioned on ca. and / is the set of negative samples drawn randomly from the set of encodings of the prediction window.
High-level CPC In contrast to the frame-level features in the low-level CPC loss, where the acoustic frames change smoothly over time, at the level of discrete units we expect successive elements be distinguishable from each other. Therefore, at the high-level we modify the standard CPC loss to incorporate this prior knowledge of contiguous heterogeneity. We do so by sampling the distractors in the contrastive task from representations adjacent to the prediction target
exp (
-==
I Lie(ktm-LAtt exp(pT.u))
where P...... Pas are the output of M prediction heads that predict the M upcoming pseudo-unit representations conditioned on h4-
Quantizer To quantize pseudo-und unit representation uı we choose the i-th embedding e; from finite size learnable codebook that minimizes the distance to the candidate unit representation
"
e min || - csll-
The codebook trained through the online k-means loss from van den Oord et al. [0177)
Lg Σ [|sg(us) (e))
ufll A Allu - sg(ug))l
(4)
where sg is the stop gradient operator and À is an hyperparameter.
Boundary predictor is modeled as a 0 o--paramterized stochastic policy me that takes as input the sequence of encodings and defines for each time-step the probability of unit boundary conditioned on 2: ne ( (b.(2) 22) = pl 2z)2.. The policy is trained to minimize the expected value of C
C E8 [E (b)l2,
Σ No (o(b)=)(H(())
(5)
We use REINFORCE Williams, 1992]
to estimate the gradient of C,, with respect VoC, Es (b) log(To(b))))))
(6)
where the expectation can be approximated through Monte-Carks methods from experience rollouts.
Average sampling rate regularization The model of the boundary detector as a stochastic policy allows us promote average pscudo-uni length (or sampling rate) in differentiable way. Let l denote the desired average pseudo-unit length, we define an additional regularization term as
Σ b
(E ne((t))
C = Eb, wisT
=

---

with the right hand side resulting from assuming b, as a Bernoulli variable. During training we compute equation on randomly sampled segments from all the possible segments of lenghh in a batch. This prior reduces the difficulty of the policy optimization as it restricts the search space to policies that produce segmentations matching the average expected unit length.
Experimental setup
The experiments were performed the train-clean-100 subset of LibriSpeech (Panayotov al. (2015). CC BY 4.0) using the aligned phone labels provided by van den Oord et al. [2018]. To evaluate low-level frame representations, we measured phone prediction accuracy of a linear classifier trained on frozen features. Additionally, we report phone accuracy of the transcriptions obtained from the sequence of representations. It was computed with 1 I-layer bidirectional LSTM network with 256 hidden units, followed by a 1D convolution with 256 filters. kernel width 8 and stride of and a CTC loss [Graves al.) 2006). We modified the loss to forbid emitting the blank token, since as we evaluate on English data only we assume no repetition in successive phones. High-level representations have a lower variable resolution, therefore we evaluated them only using the CTC transcription task. To prevent the case when the predicted transcribed sequence is shorter than the ground truth sequence, we upsample high-level representations according to the length of the segments before compression.
The model trained to extract segments which on average match the duration of phones, therefore we also assess unit discovery capabilities. We evaluate frame-level representations on the ABX task from the ZeroSpeech 2021 competition (Nguyen et al. 2020]. in which the objective is to discriminate between samples that differ on single unit. We compute the ABX score on the dev-clean set. Furthermore, we report the agreement between segment boundaries learned by the downsampling strategy and the boundaries of human-defined information-bearing units, such phones. We evaluate phone segmentation performance of the predicted pseudo-unit boundaries through precision, recall, F1 F1-score and over-segmentation robust R-value [Räsänen et al. 2009) with a 20 ms tolerance against phone boundaries obtained from the aligned frame labels. We also evaluate boundary prediction on a processed version of the TIMIT dataset [Garofolo et al. [993] in which non-speech events have been trimmed to a maximum of 20 ms. We evaluate The setup of our boundary detection evaluation was chosen for comparability with recent studies on unsupervised phone boundary detection (Kreuk et al.
2020 Bhati et al. 2021 Cuervo et al. 2022
Kamper 2022]
3.1 Models
Our model reads single channel raw waveforms sampled at 16kHz, chunked into sequences of 20480 samples (1.28 sec). At the low-level model. the encoder Jene applies five ID convolutions with
internal dimension 256 and filter widths (10: 8; 4; 4; 4). Convolutions are followed by channel-wise magnitude normalization and ReLU activations. Convolutions are strided by (5; 4; 2: 2; 2). resulting in 160-fold rate reduction, yielding 256-dimensional frame representations extracted every 10ms. The context-buille with 256 units. At the high-level, Senc is a network with two fully connected hidden layers of 256 units and ReLU activations. Sar is a single layer LSTM network with 256 units. We use N=12 and 2 predictions, and 128 and negative samples, for the CPC losses at low and high level models respectively. Each prediction head accesses all past contexts through a single auto-regressive transformer layer (Vaswani et al.] (017)] with 8 scaled dot- attention heads with internal dimension 2048 and dropout [Srivastava et al., 2014) with p = 0.1. The target quantizer q uses a codebook of 512 embeddings each of dimension 256, and we use the standard literature value of 0.25 in equation 4 The boundary predictor is a single layer bidirectional transformer network with 8 scaled dot-pro attention heads with internal dimension 2048, We set the expected average unit duration = 8 in equation 7 to roughly match the average phone duration measured on the 100 hour subset of the LibriSpeech dataset (7.58 frames).
model Jar is a two-layer LSTM network [Hochreiter and Schmidhuber, 1997]
We compare our model to several baselines. Regarding models with single-level feature extraction: the CPC implementation from [Rivière et al. 20200] which we use our frame-level model, ACPC [Chorowski et al. (02T] which improved on vanilla CPC by promoting temporally-stable features, and the CPC optimized for phone boundary detection by Kreuk et al. 22020]
5

---

Table 1: Frame-level representations evaluation. Linear frame-wise accuracy and CTC phone accuracy on the test split of LibriSpeech train clean 100, and ABX within and across speakers on the ZeroSpeech 2021 dev-clean set. Our model results in frame representations with better linear separability of phones than the ones obtained from model trained with supervised downsampling. Overall the model improves phone discriminability.
Frame accuracy
Phone accuracy t
ABX within +
ABX across
Architecture
Model
+
CPC Rivière et al.
2020
67.50
83.20
6.68
8.39
Single level
ACPC [Chorowski al.
2021
68.60
83.33
5.37
7.09
Two-level CPC no downsampling
67.49
83.38
6.66
8.34
SCPC
[Bhaii et al. 2021
43.79
68.38
20.18
16.26
Multi-level
Two-leveI w. downsampling
67.92
83.39
6.66
8.32
mACPC
[Cuervo et al. 2022
70.25
83.35
5.13
6.84
Ours
72.57
83.95
5.08
6.72
Downsampling (supervised)
71.01
84.70
5.07
6.68
For multi-level modeling we compare: a stack two CPC modules both using the architecture from Rivière et al. [2020] in which the output vectors of the low-level model are used as input for the high-level model (two-level CPC) with no downsampling; a stack of two CPC modules in which the high-level model downsamples the outputs of the low-level model
fixed rate (two-level CPC
with downsampling) using a ID conv with dim 256, filter width 9 and stride of 9. resulting in a sampling rate close to the average phone rate of LibriSpeech clean 100; SCPC [Bhati et al., 2021 using a boundary predictor based on detecting peaks feature pairwise dissimilarities: MACPC which improves upon SCPC representation learning by adding long-term prediction both levels Cuervo et al. 2022]. Finally, we add a supervised topline which the
is forced two-level CPC model.
return ground-truth boundary locations and otherwise is equal
For the model of Kreuk et al. [2200] we base our implementation the authors' implementa- tion obtaining consistent results with the ones reported. For SCPC and mACPC we use the implementations provided in Cuervo et al. [2022]. We make our code available at https
//github hww.://bwo.oooccoooiii-oii-----e lab/hCPCC
For single-level models we train until convergence for 50 epochs on the train split of LibriSpeech clean 100. Multi-level models are trained for 50 additional epochs after appending to the pre-trained frame-level modules the high-level network and downsampler, where applicable. We obtained similar results using a training schedule with a weighted loss in which during the first epochs the frame-level loss outweighs the high-level one. However, this method required careful tuning, so we defaulted to the more stable frame-level pre-training.
All models trained using batch size of 64, the Adam optimizer [Kinmma and Ba 2015 learning rate of 0,0002, and an initial warm-up phase where the learning rate is increased linearly during the first 10 epochs. For the quantization module we follow the setup from Lańcucki al. [020]. Training our model takes around 13 hours on 2x Nvidia RTX 3080 GPUs. Evaluation models are trained for 10 epochs using the same hyperparameters, with the exception of the warm-up learning rate schedule.
with
4 Results
Low-level representations In the frame-level evaluation (Table D.. the models which perform content- centent-dependent downsampling have the highest phone classification accuracies and ABX scores. Conversely, two-level CPC baselines with constant or no downsampling show little to no improve- ments over single-level models. The proposed model achieves higher frame accuracy than the model with downsampling according to phone labels, hinting at a possible misalignment between the objec- tives of matching human-defined phone boundaries and disentanglement of representations. Finally, we observe poor accuracy of SCPC despite having two levels. This is consistent with (Bhati et al.) 2022 Cuervo al. 2022]. and caused by the absence of context modeling in the frame-level module

---

Table 2: High-level representations evaluation. Average sampling rate and CTC phone accuracy on the test split of LibriSpeech train clean 100. Our model gives the best results in phone accuracy and has the lowest average sampling rate among unsupervised methods with variable downsampling.
Avg. sampling
Downsampling
Model
Phone accuracy t
rate (Hz)
None
Two-level CPC no downsampling
100
83.41
Constant
Two-level CPC with downsampling
10.94
67.75
SCPC BBhti et al. 2021
15.91
55.49
MACPC Cuerrv et al 2022
14.47
69.66
Variable
Ours
12.32
78.93
Downsampling (supervised)
10.87
85.74
of SCPC, which optimizes for boundary prediction (see below), but cripples the ability to capture phone class information.
Transcription performance measured by phone accuracy shows a smaller difference between un- supervised models, likely due to preservation of similar amount of phonetic information in the representation, which non-linear classifier network is able to access, Incorporating top-down infor- mation about the structure of the signal might promote disentanglement of low-level representations, therefore making them more amenable to linear discrimination.
High-level representations Table shows the results of the evaluations on high-level represen- tations. Our model performs best among unsupervised models with variable downsampling, both in terms of compression rate and phone transcription accuracy. The model with no downsampling obtains the best performance phone transcription among unsupervised models, showing that all downsampling strategies result in lossy compression. On the other hand, the model which down- samples the signal according phone labels aligned with supervision exceeds the performance of not downsampling. proving that a better compression is possible and can be beneficial in such unsupervised transcription task (c.f. discussion in sec. 5.2
Matching phone boundaries Boundary detection results are presented in Table The boundaries of units predicted by our model often match with human annotated phone boundaries. On the LibriSpeech dataset our models attain the highest precision among tested models, at the expense of lower recall. Through a visual inspection of the boundarics made by other models we learned that the remaining models tend to predict false positive boundaries inside non-speech segments, which explains their poor precision. This caused by segmentation criteria of detecting peaks of dissimilarity between adjacent representations. Non-speech segments of audio show little variation, and noise can easily trigger the peak detector. We confirm this by performing boundary detection on TIMIT without non-speech events, in which dissimilarity- based segmentation models give the best overall performance. We conclude that due to a different training objective, our model might be more robust the quality of recordings.
Discussion
5.1 On the relative importance of the priors
Our model is based on several priors about the high-level structure of the speech signal, implemented through architectural choices. This begs question the relative impact of them on the results. Table and Table already show the importance of the priors implied by multi-level modeling and variable-rate downsampling. To complement these results we provide in Table ablation study on the priors imposed to the high-level features, such as average downsampling rate through policy regularization, and the prior of discreteness using adjacent negative sampling and target quantization. We present the results of the ablation in terms of linear frame-wise phone classification accuracy, in which our model improved the most with respect the baselines, and phone- houndbby detection performance, illustrate the effect of the priors the resulting downsampling strategies.
7

---

Table 3: Phone boundary detection results on the test split of LibriSpeech train clean 100 (top) and TIMIT test split (bottom). For comparability with Kreuk et al. 2020 Bhati al. [202] Cuervo et al. [202 non speech events were removed from TIMIT. Our model produces segmentations competitive with the state-of-the-art, while being robust to non-speech events.
Dataset
Architecture
Model
Precision
Recall
F1
R-val
Single level
Kreuk et al. 2020
61.12
82.53
70.23
61.87
LibriSpeech clean 100
MACPC [Cuervo et al. SCPC Ours
2022
59.15
83.17
69.13
57.71
Multi-level
BBhat al. 2021
64.05
83.11
72.35
66.40
79.94
77.92
78.91
81.98
Single level
Kreuk et al.
2020)
84.80
85.77
85.27
87.35
TIMIT (non-speech removed)
mACPC Cuervo et al. 2022
84.63
84.79
84.70
86.86
Multi-level
SCPC
Bhati et al. 2021
85.31
85.36
85.31
87.38
Ours
80.08
81.40
80.73
83.50
Table 4: Ablation the priors of average sampling rate (policy regularization) and discreteness (adjacent negative sampling and target quantization) terms of frame-wise linear phone classification accuracy and phone boundary detection R-score on the test split of LibriSpeech train clean 100.
Policy regularization
Adjacent negative sampling
Target quantization
Frame accuracy
R-val
X AXXX
64.
21.10
66.87
0.0
72.52
81.09
72.57
81.98
Adjacent negative sampling, through which we implement the prior of unit distinguishability, appears essential. Without it, the boundary predictor converged to predict a roughly constant probability, sampling the boundaries during training roughly every four frames. In early experiments, which we add to the supplemental material, we observed that using the standard strategy for negative sampling at the high level to train the boundary detector resulted in rewarding turned the high-level task into the easier problem of short term prediction the level of smooth acoustic features. When coupled with the average sampling rate constraint, this seemed to result in random production of segments, and frame accuracy plunged even below single-level models.
segeenegenttionn The model
Policy regularization in terms of the expected average segment length is another crucial element. Without frame accuracy dropped close to that of a single-level model. Segmentations produced the early stages of training were unreasonable, therefore not producing any informative experience rollout for the policy gradient estimator. Eventually, the policy would collapse local minimum of not predicting any boundaries at all. This could be addressed by promoting exploration or through different parametrization of the actions space. Note that the average segment length prior can be seen both as a feature and a limitation. It allows to control the time scale at which the signal is modeled. However it implies some degree of supervision in an ideally unsupervised method, particularly in our experimental setup we used the training data to estimate it. In other applications and domains defining characteristic sampling rate might be difficult.
Finally, target quantization seems to contribute the least to the accuracy. However, we observed that it tended accelerate training convergence. Early experiments available in the supplemental material also show that target quantization provides a stronger supervision signal for the reinforcement learning policy by more strongly penalizing unreasonable segmentations.
5.2 On human defined units vs. discovered pseudo-units for representation quality
Results at both low and high level display non-intuitive misalignment between matching human- defined unit (phone) boundaries and improving representations. frame-level, seems that the objective of maximizing for discrete predictable structure at the high-level, which not necessarily means pushing towards phonetic structure, results in more useful top-down feedback than down- sampling match phonetic information content. At high-level however, optimization objective

---

results in phonetic information loss, as we were not able to match the phone accuracy of the model with downsampling according to phonetic units. It could be possible that there are patterns between n-grams of phones, which map to single high-level unit, and facilitate the predictive task more than at the level of individual phones. Yet, by being many-to-one mappings, they could make it impossible to retrieve the original constituents after compression. We consider that an interesting objective for future work is characterizing the patterns of the discovered pseudo-units sequences, and perhaps investigate their correlation with models of human speech perception.
Related Work
Perhaps the most widely used variable-length feature discovery approach in deep learning is the extraction of subword units from text (Schuster and Nakajima 2012 Sennrich et al. 2016 Kudo and Richardson 2018). These approaches rely on exact symbol matches to detect segments. They serve inspiration but not directly applicable to learning non-uniform segmentations of dense representations. Segmental unit-discovery in Kamper et al. [2017) chunked the training corpus into segments that were then clustered into small vocabulary. We use similar assumptions in our system: every frame belongs to segment and the high-level representations are trained to match categories.
The discovery of acoustic units in unlabeled data has been approached with non- non-parametric Bayesian inference on engineered features (c.g. MFCCs). Lee and Glass [2012) proposed a Gibbs sampling inference Dirichlet Process HMM while Ondel et al. [2016] derived a Variational Bayes scheme. similar segmental HMM model has been coupled with a deep-learning based autoencoder in the HMM Variational Autoencoder Ebbers et al. [017]. Variable-length unit discovery in an end-to-end deep learning setting was recently proposed by Chooowski et al. [019) who modified the VQ-VAE quantization objective promote token stability between neighboring frames and Dieleman al. [202T) who used run-length encoding of VQ-VAE tokens. These approaches are similar in spirit to the dissimilarity-basee peak detector used in SCPC (Bhati al. 2021]: they detect segments by merging frames with similar representations. In contrast, we learn a generic boundary detection which is trained using top-down supervision from the upper-level CPC loss operating on segments.
Hicrarchical modeling of data is sometimes performed in the context of generative models. VQ- VAE 2 Razavi et al.) 2019] trained language model on discrete tokens and generated output images in two steps: it first sampled sequence of discrete tokens form the language model, then sampled the image conditioned on them. Similarly Lakhotia al. [2021) trained two-level language models SimilarlyLakhotta on speech. However, in these approaches the second-level models did not influence the learning of the low-level features. We have demonstrated the benefits of using the high-level loss tune the low-level representation.
Finally, recently proposed systems that train speech recognizers umpaired speech and transcriptions (Ych et al. 2018 Baevski et al.) [222]] typically require a step which approximately detects segments of similar frames and tries to pair transcriptions with speech using these noisy segments. These methods could benefit from using our proposed segmentation and high-level features.
7 Conclusions
We have successfully demonstrated the benefits of training two-level CPC system which extracts frame-synchronous low level features using a first-level CPC loss, and then discovers a variable-rate segmentation using supervision from second high level CPC criterion. A key component of our system is a segment boundary predictor trained using REINFORCE to directly optimize the second- level CPC objective. This leads to the discovery of discernible and predictable second-level segments which coincide with ground-truth phonetic segmentation. Moreover, the extra supervision from the second level yields improvements to the quality of the frame-level representations, nearly matching that of supervised topline using ground-truth phonetic boundaries.
Work contribution of authors
The idea of training two-layer CPC systems was discussed between the authors while Jan Chorowski was still with the University of Wroctaw. All experimental work was done by Santiago Cuervo

---

under supervision of Adrian Lańcucki. Ricard Marxer and Pawel Rychlikowski. The manuscript was written by Santiago Cuervo, Adrian Lañcucki, Ricard Marxer and Pawel Rychlikowski.
Acknowledgements
The authors thank the Polish National Science Center for funding under the OPUS-18 20 /TT/////B/ST6//43799 grant and the PIGrid consortium for computational resources. We also thank the French National Research Agency for their support through the ANR-20--CE23-012-00 (MIM) grant.
References
A. Baevski, Y. Zhou, A. Mohamed, and M. Auli wav2vec 2.0: A framework for self-supervised learning speech representations. Advances in Neural Information Processing Systems, pages 12449-12460, 2020.
A. Baevski, W.-N. Hsu, A. Conneau, and M. Auli. Unsupervised speech recognition. In Advances in Neural Information Processing Systems, pages
2021.
S. Bhati, J. Villalba, P. Zelasko, L. Moro- Velázquzz, and N. Dehak. Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation. In Proc. Interspeech 2021, pages 366-370, 2021.
S. Bhati, J. Villalba, P. Zelasko, L. Moro- VelroVelzzzuzz, and N. Dehak. Unsupervised speech segmen- tation and variable rate representation learning using segmental contrastive predictive coding. IEEE/ACM Transactions on Audio, Speech, and Language Processing, pages 2002-2014, 2022.
Chen, Kornblith, M. Norouzi, and G. Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607, 2020.
Chorowski, N. Chen, R. Marxer, H. Dolfing, A. Lańcucki, G. Sanchez, T. Alumäe, and A. Laurent. Unsupervised neural segmentation and clustering for unit discovery in sequential data. In NeurlPS 2019 Pereceptoooooootioonn as generative reasoning- reasoning-Structure Structure, Causality, Probability, 2019.
J. Chorowski, G. Ciesielski, J. Dzikowski, A. Lancucki, R. Marxer, M. Opala, P. Pusz, P. Rychlikowski, and M. Stypulkowski. Aligned Contrastive Predictive Coding, In Proc. Interspeech 2021, pages 976-980, 2021.
S. Cuervo, M. Grabias, J. Chorowski, G. Ciesielski, A. Lańeucki, P. Rychlikowski, and R. Marxer. Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words. In IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pages 3189-3193, 2022.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171 1171-4186, 2019.
Dieleman, C. Nash, J. Engel, and K. Simonyan. Variable-rate discrete representation learning. arXiv preprint arXiv:2103.08 060699 2021.
Ebbers, J. Heymann, L. Drude, Glarner, Haeb-Umbach, and B. Raj. Hidden markov model variational autoencoder for acoustic unit discovery. In InterSpeech, pages 488 492, 2017.
J. Garofolo, I.. Lamel, W. Fisher, J. Fiscus, D. Pallett, and N. Dahlgren. Darpa timit acoustic- continuous speech corpus cd-rom TIMIT, 1993.
A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference Machine Learning, page 369-376, 2006.
G. F. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313 2006.
10

---

S. Hochreiter and J. Schmidhuber. Long Short-Tern Term Memory. Neural Computation, 100-11735-1888 1997.
H. Kamper. Word segmentation on discovered phone units with dynamic programming and self- supervised scoring, arXiv preprint arXi::2202.11929, 2022.
H. Kamper, A. Jansen, and S. Goldwater. A segmental framework for uulyy--nsueervised large- vocabulary speech recognition. Computer Speech & Language, 46: 16:154-174, 2017.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
Kreuk, J. Keshet, and Y. Adi. Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation. In Proc. Interspeech 2020, pages 3700-3704, 2020.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
T. Kudo and J. Richardson. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 66-71, 2018.
K. Lakhotia, E. Kharitonov, W.-N. Hsu, Y. Adi, A. Polyak, B. Bolte, T.A. Nguyen, J. Copet, A. Baevski, A. Mohamed, et al. On generative spoken language modeling from raw audio. Transactions of the Association for Computational Linguistics, 111336-1354 2021.
A. Lańcucki, J. Chorowski, G. Sanchez, R. Marxer, N. Chen, J. Dolfing, Khurana, T. Alumäe, and A. Laurent. Robust training of vector quantized bottleneck models. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-7, 2020.
C.yy Lee and Glass. nonparametric bayesian approach to acoustic model discovery. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 40-49, 2012.
S. Löwe, P. O' O'Connor and B. Veeling. Putting an end to end-to-end: Gradient-isolated learning of representations. Advances neural information processing systems, 32, 2019.
Mikolov, W.-. Yih, and Zweig. Linguistic regularities continuous space word representations. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 746-751, 2013.
T. A. Nguyen, M. de Seyssel, P. Rozé, M. Rivière, E. Kharitonov, A. Baevski, E. Dunbar, and E. Dupoux. The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling, 2020.
T. A. Nguyen, B. Sagot, and E. Dupoux. Are discrete units necessary for Spoken Language Modeling? arXiv e-prints, art. arXiv:2203.05936 Mar. 2022.
L. Ondel, L. Burget, and J. Černocký. Variational inference for acoustic unit discovery. Procedia Computer Science, 81:80-86, 2016.
V. Panayotov, G. Chen, Povey, and S. Khudanpur. Librispeech: An asr corpus based on public domain audio books. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206-5210, 2015.
A. Razavi, A. Van den Oord, and O. Vinyals. Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019.
M. Rivière, A. Joulin, P.-E. Mazaré, and Dupoux. Unsupervised pretraining transfers well across languages. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7414-7418, 2020.
O. J. Räsänen, U. K. Laine, and T. Altosaar. improved speech segmentation quality measure: the value In Proc. Interspeech 2009, pages 1851-1854, 2009,

---

M. Schuster and K. Nakajima. Japanese and korean voice search. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1149-5152, 2012.
R. Sennrich, B. Haddow, and A. Birch. Neural machine translation rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1715 1715-1725, 2016.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56): 1929-1958, 2014.
R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, volume 12, 1999.
A. van den Oord, O. Vinyals, and k. kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems, volume 30, 2017.
A. van den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint ar arXiv: 1007:038073334 2018.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, Jones, A. N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need, In Advances in Neural Information Processing Systems, volume 30, 2017.
Vincent, Larochelle, Y. Bengio, and P.-A. Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1103, 2008
R. Williams. Simple statistical cradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
C.-K. Ych, J. Chen, C. Yu, and D. Yu. Unsupervised speech recognition via segmental empirical output distribution matching. International Conference on Learning Representations, 2018.
12


Reviews:
[{"review_id": "SVfXys6X5yS", "reviewer": "Reviewer_wUr5", "summary": "This paper proposes a new method to segment pseudo-phoneme boundaries by combining multi-level CPC and boundary predictor agent. The reward for the boundary predictor agent is the high-level CPC objetive, thereby producing sparse boundary flag. The learned representations show nice performance on phoneme prediction and boundary detection tasks.", "questions": "- line 92, conditioned on **c** → doesn’t make sense judging from Figure 1. I guess it’s **z** not **c**?\n- line 92, and and ?\n- line 132, again, c → z\n- what are **p**s in equation (1) and (2)? what nn architecture was used to produce this?\n- why does the policy trained to minimize the expected value of L_H makes the agent predict the phoneme boundaries? Is there any guarantee for this?", "limitations": "It is written in the disccussion part.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "strengths\n\n- The idea of training boundary predictor using reinforcemnent learning is a clever idea.\n\nweakness\n\n- The performance of the training algorithm significantly reduces when there is no average frame sample rate regularization (Eq. (7)). This implicitly demonstrates that the agent fail to find the boundaries without the help of regulaizration and show that the reward function for the agent may not be an appropriate one.\n- Maybe the proposed training scheme cannot really gurantee to find the optimal boundary if there is no endeavour to reconstuct the original signal.\n- It'd have been nicer if there were some qualitative analysis.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "SJOPcwUZJE", "reviewer": "Reviewer_EQcC", "summary": "This paper studies unsupervised acoustic unit discovery using a hierarchical CPC training criterion. In particular, the CPC loss is computed at two levels: frame-level and segment-level. Segmentations are produced by a jointly trained boundary detector, which can be understood as a policy network that minimizes the top-level CPC loss optimized using the REINFORCE algorithm. \n\nThe authors further introduce two crucial designs, without which the secondary CPC does not improve the performance from single-level CPC. The first design is how distractors of the top-level CPC are selected. The authors show in ablation studies that it is important to select distractors from the adjacent frames of the target one to encourage contrastiveness. The second is to inject prior knowledge of average phone duration in the boundary prediction module. \n\nThe proposed model achieves better frame-level phone accuracy, segment-level phone accuracy, and phone segmentation compared to several baselines. \n", "questions": "1. Why do the authors modify CTC to not predict blank symbols for evaluation if phone accuracy is computed at the sequence level?\n2. Following the previous question, do phone accuracy in Table 1 and Table 2 both represent the same metric and are both computed at the sequence level? If they are computed at the frame level / segment level instead, how are the segment level ground truth phones determined if a segment contains frames from more than one phones?\n\nSee weaknesses for other questions\n", "limitations": "Yes, the authors have addressed limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is easy to follow, with all the design choices well motivated and clearly described.\n2. The experiment section also contains a good set of ablation studies and baselines to justify the importance of design choices. \n3. Strong results compared to the baselines\n\nWeaknesses\n1. The authors could have included ABX results on the frame-level feature, segment-level features, and quantized frame/segment features. This would be more directly related to “unit discovery.” The evaluation included in the paper only concerns boundary detection.\n2. More ablation studies can be added. \n    - It is shown that selecting adjacent representations of the target one is beneficial for high-level CPC. Would it also be helpful for training low-level CPC?\n    - In line 202, the author states “Multi-level models are trained for 50 additional epochs after appending to the pre-trained frame-level modules the high-level network and downsampler, where applicable.” How important is it to pre-train the frame-level modules? It would be nice to also show the results of training everything from scratch.\n    - Add boundary detection results to Table 4\n3. (minor) The following statement is wrong: “In a similar spirit Lakhotia et al. [2021] trained language models on raw speech. However, in these approaches the second-level models still operated at a frequency synchronous with the input one.” In fact, Lakhotia et al. [2021] and Kharitonov et al. [2021] both use deduplicated unit sequences ([1, 1, 2, 3, 3, 3] -> [1, 2, 3]) for training LMs, so the LM operate on the segment level instead of on the frame level.\n\nKharitonov, Eugene, et al. \"Text-free prosody-aware generative spoken language modeling.\" arXiv preprint arXiv:2109.03264 (2021).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Lz1IdMS-GXP", "reviewer": "Reviewer_fYLw", "summary": "This paper proposed to achieve unsupervised acoustic unit discovery in speech using a specially designed variable-rate hierarchical CPC method. The low-level CPC is applied at a frame-level while the high-level CPC is applied by non-uniformly downsampling the low-level CPC outputs. The non-uniformly downsampling is performed based on high-level unit segmentations estimated by a boundary predictor, which is trained using REINFORCE. ", "questions": "1. \"We consider that an interesting objective for future work is characterizing the patterns of the discovered pseudo-units sequences, and perhaps investigate their correlation with models of human speech perception.\"\n\nThis seems to be a very interesting direction, but it's very possible that human and machines have different solutions. I wonder how do the authors think to make the findings useful rather than \"inventing\" a different phonology. \n\nIn particular, there exists very-well performed algorithms to achieve automatic frame-to-subword-unit alignment without requiring any phone labels. This could easily achieve much better phone accuracy and possibly a better averaged sampling rate. \n\n2. What if some collar (e.g. 1-2 frames) is allowed in the phone boundary detection experiments? ", "limitations": "1. This research direction is certainly very interesting from a scientific perspective, but I would still hope the authors to discuss more about the potential use of such automatically-discovered acoustic units. \n\n2. Hope to see more statistics and analysis about the automatically discovered high-level acoustic units. It's unclear to me how many of them were found, how well they correspond to the phones, and how about their durations etc.?\n\n3. Acoustic unit generation, clustering and discovery have been long-standing research problems in the speech community. Although the related work tried to cover some recent influential works related in the NLP community, the pioneering and more directly related works from the speech communities are mostly ignored. Actually it would be quite interesting to digging into those works and find the connections and relations. At least, even if the current framework of related work is kept, this paper should be cited as it is the original paper for the widely used wordpiece models:\nhttps://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37842.pdf\n  ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe paper is easy to follow. The proposed method is novel and does technically sound from the speech perspective.  \n\nWeaknesses:\n1. Figure 1 should be redesigned to be more descriptive and better looking. The resolution should be considerably improved and better to use a vector image. \n2. The formats of the references are not consistent. \n3. The results may not be strong enough to demonstrate the superiority of the proposed method. For instance:\n(1) In Table 1, the improvement in phone accuracy is small over existing method. I am not sure how sensible it is to evaluate the frame accuracy of phonemes here (as the phoneme boundaries are known to be noisy).\n(2) In Table 2, although the proposed method achieved the best phone accuracy among the downsampling-based methods, the result is still quite a bit worse than the no-downsampling method. \n(3) In Table 3, the phone boundary detection results are quite a lot behind previous numbers. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept