REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR
Authors: Liang-Hsuan Tseng
En-Pei Hu
Cheng-Han Chiang
Yuan Tseng
Hung-yi Lee
Lin-shan Lee
Shao-Hua Sun
Venue: NeurIPS 2024 poster (2024)
Keywords: Speech processing
unsupervised learning
reinforcement learning
adversarial learning
Primary area: speech_and_audio
TL;DR: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR

Abstract: Unsupervised automatic speech recognition (ASR) aims to learn the mapping between the speech signal and its corresponding textual transcription without the supervision of paired speech-text data. A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data. In this paper, we propose REBORN, Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR. REBORN alternates between (1) training a segmentation model that predicts the boundaries of the segmental structures in speech signals and (2) training the phoneme prediction model, whose input is a segmental structure segmented by the segmentation model, to predict a phoneme transcription. Since supervised data for training the segmentation model is not available, we use reinforcement learning to train the segmentation model to favor segmentations that yield phoneme sequence predictions with a lower perplexity. We conduct extensive experiments and find that under the same setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech, TIMIT, and five non-English languages in Multilingual LibriSpeech. We comprehensively analyze why the boundaries learned by REBORN improve the unsupervised ASR performance.

REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR
Liang-Hsual Tseng"
En-Pei Hu"
Cheng-Han Chiang Shao-Hua Sun
Yuan Tseng
Hung-yi Lee
Lin-shan Lee
National Taiwan University
Abstract
Unsupervised automatic speech recognition (ASR) aims to learn the mapping between the speech signal and its corresponding textual transcription without the supervision of paired speech-text data. A word/phoneme in the speech signal is represented by segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data. In this paper, we propose REBORN, Reinforcement-Leuue for Unsupervised ASR. REBORN alternates between (1) training a segmentation model that predicts the boundaries of the segmental structures in speech signals and (2) training the phoneme prediction model, whose input the speech feature segmented by the segmentation model, to predict a phoneme transcription. Since supervised data for training the segmentation model is not available, we use re- inforcement learning to train the segmentation model to favor segmentations that yield phoneme sequence predictions with lower perplexity. We conduct extensive experiments and find that under the same setting, REBORN outperforms all prior unsupervised ASR models on LibriSpeech, TIMIT, and five non-English languages in Multilingual LibriSpeech. We comprehensively analyze why the boundaries learned by REBORN improve the unsupervised ASR performance.
Boundary Segmentation with Iterative Training
Introduction
Automatic speech recognition (ASR) systems convert speech signals into their transcription texts. Most state-of-the-an ASR systems trained with dense supervision using large amount of labeled speech-text paired data [10] [2]. However, more than 80% of languages in the world have limited access to speech-text paired data [51) and collecting such paired data requires intensive labor and costs, fundamentally limiting the applicability of supervised ASR systems to these languages. This leads to significant efforts devoted developing unsupervised ASR (UASR) systems, which aim learn the mapping between speech signals and textual transcriptions (words or phonemes) without any speech-text pairs
30 33
UASR models learn to align the distribution of input speech signal and output text without paired data. Learning to match two distributions of sequences unsupervisedly has been extensively studied in unsupervised neural machine translation (NMT) [ where the aim is to learn a neural network that translate text in source language to text in target language without paired data 26 27). Prior works show that adversarial training 20) can be used to learn such mappings [26 27], which employs generator learning to translate, and discriminator learning distinguish generated text from the real text in the target language.
Equal contribution.
Correspondence to: Shao-Hua Sun <shachuasCntu. edu.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Can we adopt such an adversarial training scheme for UASR? In unsupervised NMT, text can be easily tokenized into sub-word tokens, so unsupervised NMT only needs learn the mapping between the source and target language's 's token embeddings. However, in UASR, phoneme or word represented by a variable-length segment in the speech signal whose boundaries are unknown. Moreover, the length of a speech signal ís much longer than the length of its textual transcription. The above characteristics of speech make learning the mapping between the segmental structures in speech and textual transcription challenging. Existing works in unsupervised ASR rely on handcrafted rules or separately learned modules to obtain the boundaries of the segmental structures 4 33 64]
Yet, such handcrafted boundaries are often sub- sub-optimal, bottlenecking the performance of UASR.
This paper focuses on learning better segmental boundaries improve the mapping between seg- mented speech and textual transcription. We propose REBORN (Reinforeement-Lennee Boundary Segmentation with Iterative Training), an unsupervised ASR framework with a segmentation model and phoneme prediction model. The segmentation model determines segmental structure boundaries in speech signals, while the phoneme prediction model assigns a phoneme to each segmental structure. After properly initializing the phoneme prediction model, we use iterative algorithm that alternates between two stages to train the segmentation model and refine the phoneme prediction model. The first stage trains the segmentation model. Since we do not have the ground truth segmental bound- aries, we use reinforcement learning to train the segmentation model to favor segmentations that yield phoneme sequence predictions with a lower perplexity. We experiment with various learning objectives and implement them as reward functions for learning the segmentation model. In the second stage, based on the boundaries predicted by the segmentation model learned in the previous stage, we use an adversarial loss similar to generative adversarial networks (GANs) 20 to train the phoneme prediction model.
We conduct extensive experiments on LibriSpeech [45. TIMIT [9], and Multilingual LibriSpeech (MLS) [50] to compare the phoneme/phone error rate (PER) and word error rate (WER) with prior works. We show that our method outperforms all prior UASR methods on LibriSpeech, TIMIT, and five languages in MLS when using the same amount of training data. We perform thorough ablation studies to show that iterative training and the rewards we design are critical to the performance of REBORN. By analyzing the segmental structure obtained by our segmentation model, we find that the segmental structures are acoustic units smaller than phonemes, which helps the phoneme prediction model predict more accurate transcriptions. To facilitate further research and replication, our code and models are available at
conm//andybb7776//ebrmn-uu
2 Related Work
An ASR model takes speech signals as the input and predicts the textual transcriptions. Unsupervised ASR (UASR) aims to train ASR model without access to paired speech-text data. Instead, the only available training data is unlabeled speech data and unlabeled text data, while the correspondence between speech and text is not available. In this paper, follow prior UASR works 4 33 to predict phoneme transcriptions from speech signals. To achieve this, a lexicon is required to transform the unlabeled text into phoneme sequences. Learning the mapping between speech signals and phoneme sequences can be formulated as a distribution matching problem, where we want to learn an ASR model whose output phoneme sequences match the distribution of real phoneme sequences.
Yeh et al. 64 address this distribution matching problem using an unsupervised loss function based on empirical output distribution matching R4. which guides the ASR model to produce phoneme sequences with statistical distributions close to real phoneme sequences. Wang et al. [60] further extend this approach by matching the N N-skipgram and positional unigram distributions.
Liu et al. 333 propose use the generative adversarial network (GAN) 20 for UASR. The GAN solves the distribution matching problem by training generator whose output distribution resembles a target distribution, and a discriminator trained to distinguish the outputs of the generator and the samples from the target distribution. When using GAN to train unsupervised ASR, the generator the phoneme prediction model. The phoneme prediction model takes in speech features of the segmental structures in the speech signal, and outputs a phoneme transcription. Prior works rely on hand-crafted or separately trained unsupervised phoneme segmentation models to find the boundary of the segmental structures [I]] (U133,6) 61).
2

---

Language model $
Ree
Discriminator 6
Reei
Merged boundary
Phoneme Predictor (Generator) +
Phoneme Predictor (Generator] A
Rerative Training
Segrnent-wibe mean pooling
Segment-wise mean pooling
Phoneme Predictor (Gen erator)
Segmentation Model O
Segmentation Model
Original boundary
Segmentation Model
(a) Stage S:gggennittion model training
(b) 2: Phoneme prediction model training
(c) Boundary merging
Figure 1: (a) and (b): REBORN iterates between using RL to train the segmentation model and using adversarial training to train the phoneme prediction model. (c): An illustration of the seg- mentation/boundaryy merging. 1 means the start of segment while 0 is not. Given the original segmentation and the predicted phoneme sequence, we merge the segments that result in the same phoneme prediction into the same segment, yielding the merged boundary.
With the emergence of foundation models, wav2vec-U [4] shows that UASR performance can greatly benefit from using speech foundation models as the feature extractor. However, wav2vec-U still relies on the two-stage feature preprocessing step based k-means to find the boundaries of the segmental structure in the speech signal. In our paper, we call the segmentation boundaries obtained by the feature preprocessing in wav2vec-U as "k-means-basee segmentation" EURO (18] changes the decoding algorithm used to decode the phoneme prediction sequences and explores other speech foundation models for feature extraction, including HuBERT (23) and WavLM (12).
In these prior methods, the segmentation is either non-trainable or learned separately from the phoneme prediction model. REBORN differs from prior works by introducing a trainable segmenta- tion model tailored for the phoneme prediction model, and the segmentation model and phoneme prediction model can be iteratively polished. wav2vec-U 2.0 [30 simplifies the feature preprocessing step in wav2vec-U and does not explicitly consider the segmentation structure in their model. RE- BORN has performance better than way2vec-U 2.0 on almost all datasets, showing that learning the segmentation boundary for feature preprocessing is important for the performance of UASR.
Although recent UASR is originally to be cross-modality distribution matching problem, it is also related representation learning. For example, the speech features might extracted from self- supervised speech foundation models; and the segmentation problem is also commonly investigated in tasks like acoustic unit discovery. We discuss these two related topics further in Appendix
3 Method: REBORN
The difficulty of mapping speech signals to their corresponding phoneme transcriptions lies in the segmental structures speech whose boundaries are unknown. To tackle this challenge, we propose REBORN, which trains a UASR system with unpaired speech and text data. REBORN contains segmentation model and phoneme prediction model. The segmentation model takes the speech feature input and determines the boundaries of the segmental structures in the speech signal, and the phoneme prediction model predicts a phoneme for each segmental structure. We will use the terms segment and segmenial structure interchangeably. In our paper, we do not use the term segment in the exact sense as in linguistics, where segment refers to discrete units that can be identified in the stream of speech [15]. We use segment to broadly refer meaningful unit (e.g., a phone or word) or span in the speech identified by the segmentation model.
span in the speech, which may be
The overall training process of REBORN is outlined in Figure I First, we initialize the phoneme prediction model using wav2vec-U U (Section 3.3). Next, the training of REBORN iterates through two stages: Stage 1 (Figure a Section 3.1 Training the segmentation model to learn better segmentation until the segmentation model converges while fixing the phoneme prediction model. Stage 2 (Figure [b), Section 3.2 Training the phoneme predictor based on the segment boundaries predicted by the segmentation model until the phoneme prediction model converges. The iterative process ceases when the UASR performance does not improve over the previous iteration. In UASR.
3

---

we cannot use PER as a validation metric to select the best model or determine when to stop training. We use an unsupervised evaluation metric achieve it instead, detailed Appendix [.4 and
3.1 Stage Training the Segmentation Model
3.1.1
Segmentation Model
Given an input speech signal, we use self-supervised feature extractor model to extract the speech features X Ii..... .TT] from the waveform. The segmentation model takes the speech features and predicts the boundary of the segmental structure in the speech features. For each feature 2;, the segmentation model assigns binary value d,. indicating whether 2; is the first frame of segment. We split X into segments based on i b1:7 and mean-pool the features within the same segment. The mean-pooled features [s), sT are forwarded through the phoneme prediction model to obtain the phoneme prediction [y)..... yTT using greedy decoding. The resulting phoneme prediction will be de-duplicated into W,. .. by removing the same consecutive phoneme.
The segmentation model is one- mneddmensiooal CNN, denoted by The and parametrized by A. Given [x1,-... .r), To predicts probability ma [i], Vi E [1..... 77. Let B, Bernoulli [i]) be Berouulii random variable representing whether x, is the beginning of a segment. B, means 2; is the beginning of a segment; B, 0 means x; not the beginning of segment. During training, we sample b; from Bernoulli [i]]] during inference, we take b, = if Toli] 2 0.5, otherwise b, = 0.
3.1.2 Training the Segmentation Model with RL
To help the phoneme prediction model predict phonemes, want to train the segmentation model to capture the segmental structure in speech. However, the optimal segmentation boundary is not available for training. Recognizing that the segmentation quality directly affects the phoneme prediction of the phoneme prediction model, we estimate the quality of the phoneme prediction to guide the segmentation model training. The phoneme prediction model is from the previous REBORN iteration or initialized from wav2vec-U in the first REBORN iteration, and it is fixed when training the segmentation model. Given that the feature segmentation based boundary decision is inherently non-differentiable, we leverage RL to train the segmentation model. While related works may employ techniques like soft monotonic aligmment or straight tthhough estimator to approximate gradients [7 60). we find that RL provides natural and suitable approach for our scenario, where the estimated quality of the phoneme sequence across different segmentations serves as the RL reward.
We train the segmentation model using the policy gradient method (57) hased on the REINFORCE algorithm 633. For each utterance, we calculate utterance-wiss reward R (defined in Eq. 4 in the next subsection) from the de-duplicated phoneme prediction to train the segmentation model. Based on the policy gradient method, the segmentation model τy is optimized using the following Vo log (o(6) where E is taken over i sampled from xo and approximated with the mean of a batch of training sequences.
gradient: Bi
3.1.3 Reward Designs
Given utterance in the training set, the utterance- wise reward R is the weighted sum of perplexity difference reward Rppl. edit-distance reward Redn. and length difference reward Rien-
We introduce some notations: Given an utterance, use the currently trained segmentation model Ty for segmentation and the phoneme prediction model trained in the last REBORN iteration to obtain the de-duplicated phoneme prediction sequence Y/. For the same utterance, we use the segmentation model from the previous REBORN iteration T--1 for features segmentation and the same phoneme prediction model to obtain another de- de-duplicated phoneme prediction sequence, denoted as Y%_1 In the first REBORN iteration, Y is the de-duplicated phoneme prediction from way2vec-U.
Perplexity Difffrrncce Reward. The perplexity difference reward is designed favor phoneme segmentation better than the segmentation learned in the previous iteration. Intuitively, a better phoneme segmentation prediction by.7 should yield a more reasonable phoneme prediction V'-M- We use perplexity (PPI.), the negative likelihood of phoneme sequence scored by a phoneme language model (LM), to evaluate how reasonable phoneme sequence is. Perplexity measures how likely

---

phoneme sequence can be observed in the real world; phoneme sequence with lower perplexity means it more likely to be observed in real-world phoneme datasets. We use a 4-gram phoneme LM trained on the phonemicized unlabeled text corpora. To guide the segmentation model to generate segmentation with a better phoneme prediction than the phoneme predictions obtained in the previous iteration, we define perplexity difference reward as follows:
Rppt P PPLg,
where P 1 is the perplexity of Y/_1 and PP is the perplexity of Yg. The perplexity difference reward guides the segmentation model to produce segmentation that results phoneme sequence with lower perplexity comparing with the previous iteration.
When only using Hypl. the segmentation model learns some segmentation method that leads to phoneme predictions with lower perplexity but does not correspond to better phoneme prediction results. To prevent such undesirable behaviors, we design two regularization rewards, the edit distance reward and length difference reward, to ensure that the policy learned by the segmentation model does not drastically alter the phoneme predictions between iterations. The two regularization rewards punish the segmentation model if Yg is too different from Y/_1 Crucially, regularization works only if the 1 in the first iteration is good enough. As previously mentioned, we ensure this by using the predictions from a trained wav2vec-U as Y in the first iteration.
Edit distance reward. We use Levenshtein distance diev as the edit distance to calculate the difference between Y and Yg We take the normalized negative edit distance as the reward, which is defined in Eq.
Length difference reward. We use the length difference reward Rien guide the segmentation model to predict segmentation such that the length of Yg does not differ significantly from the length of Y The length difference reward for an utterance defined Eq. 3 where [Y'| the length of Y'
di (g Yo) 19-1|
| 18
Redi
(2)
Rien
(3)
18/_11
The final utterance -iree reward R is the weighted sum of Rypls Redit and Ryen
R Cppl Rppl + Cedit Redi + Gen Rien
(4)
where Cppl, Cedir and Ceen are the weighting coefficients. During training, Rppl. Redit and Rien are normalized within each batch. Appendix C.5 discusses how the coefficients are determined.
3.1.4
Initializing my with Behavior Cloning
Before training the segmentation model ma with RL, we initialize it with behavior cloning (BC) 44 BC uses the supervised objective to train the segmentation model to predict the boundaries in speech features. Given speech features T1:7, the segmentation model is trained to predict the 0/1 boundary by by using some boundaries b; as the target. In REBORN, the target boundary for BC the merged boundary (Section 3.1.5) obtained using T--1, the segmentation model learned in the previous iteration. We then frame BC as a binary classification task and optimize it with the cross-entropy loss. Note that in the first REBORN iteration, use the k-means-baat baaed boundary from wav2vec-U as the BC target.
48].
3.1.5 Boundary Merging
After training the segmentation model until convergence, the boundaries of the whole dataset and perform boundary merging. The process uses the phoneme prediction model trained in the previous iteration to refine the boundary predictions b, Even if some consecutive speech features are split into segments by the segmentation model, the mean-pooled features of the two segments may yield the same phoneme prediction. In this case, the two segments will be merged into one segment. An illustration of boundary merging is shown in Figure Ic). Boundary merging differs from phoneme sequence de-duplication: boundary merging modifies the segmentation prediction b, while de-duplication modifies the phoneme sequence.
the segmentation model predict
5

---

3.2 Stage 2: Training the Phoneme Prediction Model
The phoneme predictor takes the mean-pooled features of segmental structures $ = {8,, 82, 8T and predicts a phoneme sequence Y [y) y2, yrr S are obtained by mean-pooling the features X in the same segmental structure based on [ b find that it is effective to perform boundary merging stabilize the training in this stage.
where bj is the merged boundaries. We
After obtaining the merged phoneme boundaries, we train the phoneme predictor model using GAN training. The phoneme prediction model is the generator in GAN training. The generator aims to output phoneme predictions that look like real phoneme sequences fool the discriminator. The discriminator takes in a phoneme sequence, which can be the output of the generator or a phoneme sequence from the unpaired text corpora, and the goal of the discriminator is to distinguish whether the input phoneme sequences are outputs of the generator. The generator and discriminator are updated using the loss in GAN training. In this stage, the parameters of the segmentation model are not updated, and the generator is initialized from the previous iteration. We discuss the effect of applying boundary merging and parameter initialization Appendix A.
3.3 Initialization of REBORN
In Stage 1, the segmentation model depends the phoneme prediction when calculating the rewards. As result, REBORN cannot work without properly initializing the phoneme prediction model. We use wav2vec-U to train phoneme prediction model and use it as the initialization for the phoneme prediction model REBORN. We briefly introduce wav2vec-U in Appendix
Experiment Setup
4.1 Dataset
We use three datasets commonly used in ASR to evaluate the performance of REBORN.
LibriSpeech 45 is an English speech recognition corpora that contains 960 hours of training data. Following EURO 18), we use 100 hours of audio from the train-clean- set as the unlabeled speech data. The unlabeled text data is derived from the remaining 860 hours, which does not overlap with the transcription of the unlabeled speech data.
TIMIT 19 is another English speech recognition with the human-labeled phone boundary. We follow the matched setting, which is more broadly used 4 18 33 where the speech and text data come from the same set of utterances.
Multilingual LibriSpeech (MLS) [50 is ASR dataset including German (de), Dutch (nl), French (fr), Spanish (es), Italian (it), and Portuguese (pt). Following [4, we randomly sample and use 100 hours of speech data for each language and use the LM data provided by the dataset unpaired text.
4.2 Evaluation Metrics
We use phoneme error rate (PER) and word error rate (WER) to evaluate the ASR performance. If not specified, use greedy decoding to obtain phoneme-level results. For decoding word-level outputs, we perform WFST decoding 33 39 using PyKaldi [9]. We leave the details Appendix CC7
4.3 Implementation Details
For the English datasets, we use wav2vec 2.0 extract speech features from speech signals; for
MLS, we use XLSR-53 [14] as the feature extractor. Both of them can be found in fairseq 44). The 4- gram phoneme LM derived using KenLM (22]. We describe more details about the data preparation
and LM formulation in Appendix C.I which basically follows wav2vec-U []] All the trainable models, including the segmentation model, the phoneme prediction model, and the discriminator
in GAN training, are composed of one-dimensional CNN layers, as detailed in Appendix C.3] For
LibriSpeech and TIMIT, we train REBORN for two iterations. For MI.S, we only train one iteration since we do not find the performance to improve in the second iteration.

---

Table 1: PER/WER LibriSpeech using 100 hours speech data. t: Our reproduction. (wav2vec-U U and wav2vec-U 2.0 only report results of using 960 hours of unlabeled speech). HMM ST indicates HMM self- training (Appendix
Table 2: PER results on TIMIT. The cross-
mark (X) in the greedy-decoding column indi- cates that an additional LM (4-gram) used during decoding. REBORN reaches the best performance with no LM used for decoding. showing that REBORN can benefit from the external LM via RL.
PER/WER (%)
Appresich
dov-elcan dov-other test-clean test-other
WITH ORACLE BOUNDARY
Greedy decoddng core-dev core-test all-test
PER (%)
Approach
Train from oracle
6.3/12.8
9.7/16.3
6.4/12.7
10.0/16.8
BASELINE
(a) Train from oraele
9.1
10.3
9.2
EURO (TuBERT)
15.2/23.1
20.7/29.3
15.1/22.8
21,1/29.8
wav2vee-U?
19.3/20.4
22.9/25.6
19.3/210
23.2/25.8
wav2vceUU (seproduced)
20.2
22.2
20.3
wav2vec-Ur 2.0l
12.2/17.2
16.3/21.7
126/11.7
16.3/22.2
(c) wav2vec-Ur U-WPST
17.8
16.8
wav2vec- 2.0 HM'
10.0/1.4
13.1/19.0
10.3/16.0
13.1/19.6
(d) EURO (wav2vec 2.0)
X
18.5
19.8
METHOD
(e) EURO (WavLM)
x
14.3
14.6
REBORN
8.3/2.55
11.9/17.6
8.9/1311
12.5/18.7
REBORN HMM ST
5.2/9.3
8,5/13,5
5.49.6
8,5/13.7
REBORN
12.4
13.5
12.4
5 Results
5.1
Main Results
We show the PER and WER of LibriSpeech, TIMIT, and MLS in Table [ Table 2 and Table 3
We compare REBORN with several prior UASR works: wav2vec-U [4] wav2vec- 2.0 (30 and EURO [8]. We have the following observations:
REBORN significantly outperforms all prior methods on LibriSpeech. Our experiment on LibriSpeech follows Gao et [18] to use the 100-hour training split of LibriSpeech as the unlabeled speech data. Compared with all the baselines in Table T. REBORN achieves the lowest PER and WER. wav2vec-U and EURO both use hand-crafted rules obtain the segmental boundaries, while wav2vec- U 2.0 removes the feature segmentation steps. The superior performance of REBORN illustrates the importance of learning the segmental boundaries tailored for the phoneme prediction model. Notably, without using HMM self-training, REBORN already has PER/WER lower than wav2vec-U 2.0 with HMM self-training, which is the prior state-of-the-arr (SoTA) method on LibriSpeech. In Appendix A.4 we further apply the REBORN pipeline with speech foundation models other than wav2vee 2.0. We find that REBORN yields notable performance improvement when using HuBERT (33 WavLM [2] the feature extractor, illustrating the generalizability of our method.
Self-training further improves PER/WER of REBORN. Self-training uses the phoneme predictor's prediction as the pscudo-label and trains new phoneme prediction model using the pscudo-label as the training data. is commonly used in UASR to boost the performance 4 30 In Table I we show that REBORN can be integrated with Hidden Markov Models (HMM) self-training (Appendix (C2) to further lower the PER/WER. We reach a new SOTA on LibriSpeech under the setting of TOO- hour speech data, outperforming the prior SoTA in PER and WER by 5% and 6%, respectively. Surprisingly, REBORN with HMM self-training outperforms training wav2vec-U with the oracle boundary. This shows that REBORN can be combined with self-training to improve performance effectively. Due to limited computation resources, we only conduct self- self-training on LibriSpeech.
REBORN outperforms all prior UASR methods on TIMIT. Table 2 shows the PER of TIMIT, and we again find that REBORN achieves the lowest PER compared with all prior UASR methods. This indicates that REBORN not only works on large datasets like LibriSpeech but also works on small datasets like TIMIT, which only contains about hours of audio data for training. Even using greedy decoding only, REBORN outperforms the prior best fest-performing UASR model (row (e)), which relies on prefix decoding. EURO shows that replacing the feature extractor with WavLM [[2] (row (e)) outperforms using wav2vec 2.0 (row (d)) as the speech feature extractor. Since REBORN using wav2vec 2.0 already outperforms EURO with WavLM, we leave changing the feature extractor in REBORN TIMIT as future work.
7

---

Table 3: WER MLS. f: Results from Baevski et al. [4] *: Our reproduction of wav2vec-U, U. used as the initialization of the phoneme prediction model in REBORN. I: Results from Liu et al.
Table 4: Boundary evaluation results of different segmentation methods on LibriSpeech test-clean split. The second-last column (Freq.) the num- ber of segments per second. All the methods share the same phoneme prediction model trained with the k-means- based segmentation of wav2vec- U.
30
WER (%) 4
Approach
de
es
it
pt
Avg.
Boundary method
Prec.
Rec.
Fl
Freq.
PER%
wav2vec-U1
32.5
40.2
39.8
33.3
58.1
59.8
44.0
(a) Oracle
1.0
1.0
1.0
11.89
13.8
wav2vec-U"
33.9
38.1
37.7
33.1
51.8
59.4
42.3
(b) k-means-bused
0.64
0.77
0.70
14.27
19.3
wav2vec-UU..0
23.5
35.1
35.7
25.8
46.9
48.5
35.9
(c) Strgar and Harwath 56
0.75
0.75
0.75
12.26
23.2
REBORN
20.9
26.9
28.2
24.7
39,9
51.5
32.0
(d) REBORN
0.57
0.78
0.65
16.29
12.9
REBORN performs well on languages other than English. The results on MLS are presented in T We find out that a single iteration is enough for performance convergence MLS. Recall that REBORN is initialized from wav2vec-U, but REBORN outperforms wav2vec-U by a large margin in all six languages. This large performance gap underlines the importance of learning the boundaries of the segmental boundaries. Moreover, REBORN outperforms wav2vec-U U 2.0 except for Portuguese, and the average WER on the six languages is 3.9% lower than wav2vec-U 2.0.
5.2 Boundary Analysis
The core of REBORN is the segmentation model that learns the segmental structure's boundaries. In this section, we look at how different segmental boundaries affect the performance of phoneme prediction. We compare the boundaries obtained by four methods: (a) the oracle phoneme boundaries obtained by forced alignment [36). which requires paired speech-text data to learn the alignment; (b) the k-means-based segmentation boundary wav2vec-U: (c) the phoneme boundary obtained by the
SoTA unsupervised phoneme segmentation method [56] on LibriSpeech; (d) the boundaries learned by the segmentation model of REBORN in the first iteration before boundary merging.
First, we focus on the PER in Table when using different segmentation methods. The PER is obtained by taking the four different segmented features to the identical phoneme prediction model: the phoneme prediction model trained using the k-means-based segmentation of wav2vec-U. Note that the phoneme prediction model here is unsupervised and non-ideal. We find that simply replacing the k-means-basee based boundary with the oracle phoneme boundary reduces the PER by 5.5%, showing the imperfect hand-crafted 6-means-based segmentation is the bottleneck of UASR. Next, even when using the boundary predicted by the SoTA unsupervised phoneme segmentation (56). the PER does not reduce. Conversely, the boundary learned by REBORN achieves the lowest PER. This is because the boundary learned by REBORN is directly tailored for the phoneme prediction model.
Next, we discuss the phoneme boundary results in Table 4 The phoneme boundary results is evaluated with boundary precision, recall, and F1 with a 20ms tolerance window, following the harsh scheme from Strgar and Harwath [56]. We leave the discussion about the different schemes for boundary
evaluation in Appendix E We also report the average number of segments per second (Freq. in
Table 4.. Interestingly, the boundary Fls of the oracle boundary (row (a)) and Strgar and Harwath [56] (row (c)) are much higher than the boundary Fl of REBORN (row (d)), but the PER of REBORN much better. This is because REBORN'S segmentation model predicts more segments than the number of phonemes in the utterance (before boundary merging), indicating that it learns some segmental structures smaller than the phones. This can be seen from the lower precision (0.57) and higher frequency (16.29) compared to oracle boundaries. However, segmenting the speech feature into smaller units not problematic because even if the consecutive speech features of the same phonemes are split into two segments by the segmentation model, as long as they have the same phoneme prediction, the duplicated phoneme prediction will be removed in the de-duplication step, and thus not affect the PER.
8

---

Table 5: We compare different reward functions and the effect of BC initialization with first itera- tion results LibriSpeech test-clean. Row (c) is REBORN.
Figure 2: PER across training epochs the test-clean split of LibriSpeech. BC pretraining speeds up convergence and raises performance.
35
BC pretrained
from scratch
PER (%)
LM PPL
30
Ablation
Stage
Stage 2
(Stage
(a). Repil
14.7
12.9
10.0
(b). Rppt Redn
13.8
12.1
10.8
20
(c). Repi + Remn + Rign
12.9
11.9
11.2
15
(d). REBORN w/o BC
14.9
14.2
13.8
Epoch
5.3 Ablation Study
5.3.1 Reward Design and Behavior Cloning
We verify the effectiveness our three proposed reward functions and behavior cloning initialization in T While the perplexity difference reward alone is sufficient to guide the segmentation model, adding edit distance and length difference rewards improves performance further. As previously mentioned, using only the perplexity difference reward will make the segmentation model predict segments overly focusing on lowering PPL. And phoneme sequence with a lower perplexity may not always be better transcription. To deal with such overfitting problem, design the two regularization rewards, namely, Redit and Rien- to make the model learn to segment for lower perplexity while grounded Table further evidence the effectiveness of the two rewards. Additionally, we observe that removing BC leads to decline in PER compared to using BC (see row (d) row (c)). To deepen this analysis, we present the PER curve on the LibriSpeech test-clean set with and without BC. The results in Figure 2 indicate that BC helps enhance performance and accelerate convergence.
reasonable transcription from the previous iteration. Our results in
5.3.2 Iterative Training
In Figure show the PER LibriSpeech test-clean split during the iterative training process of REBORN. We observe that after training the segmentation model in the first iteration, the PER drastically drops by 6.4% compared with wav2vec-U, which is used to initialize the phoneme prediction model in iteration one. This shows that the quality of the segmental structure's boundaries important the performance of UASR. Afterward, the PER of REBORN decreases steadily across iterative training, showing that training the segmentation model and the phoneme prediction model is critical improving the performance of REBORN, We find that the PER does not drop significantly after the fifth iteration. Last but not least, we provide more evidence in Appendix A.3 to showcase that the phoneme predictors are iteratively polished.
Figure 3: PER of each stage during REBORN'S two- stage iterative training the test-clean split of Lib- riSpeech. St.: stage; w2vu: wav2vec-U.
Table 6: The boundary evaluation results on TIMIT. REBORN achieves better bound- ary evaluation scores than the original means-based method, and is comparable with Strgar and Harwath [56]'s 's after bound- ary merging. from literature. All the metrics are calculated on the rest-all split.
18
Rebom
wav2vec-U
EURO
wav2vec-U 2.0
16
PER
12
Boundary method
Prec. Roc. Fl R-val. PER%
k-means-based
0.62 0.75 0.68 0.68
20.3
Strgar and Harwath 56
0.85 0.79 0.82 0.84
0.61
0.71 0.62 12.4
Init. (wZvd)
Har-1 (St.
Her-1 (St.22
(St.1
(it.2)
(St
(St.2)
(St.
(st.2
(St.
Bor-S (St.
boundary merging
0.78 0.79
0.82
Han-
Har-d
Har-4
B

---

5.3.3 Boundary Evaluation on TIMIT
In Section 5.2 we demonstrate that REBORN segmentation delivers fine-grained segments, achieving the highest performance gain among segmentation methods given the same non-ideal phoneme prediction model. Here, we extend our boundary evaluation results to TIMIT, a smaller dataset with phone boundaries, to provide more comprehensive boundary insights. Table 6 shows that the initial boundaries learned by REBORN, optimized explicitly for the phoneme prediction model, already achieve high recall. With boundary merging, i.e., consecutive segments with the same phoneme prediction are merged as illustrated in Figure[((c) REBORN achieves results close to Surgar and Harwath [56] s. In line with the previous context, emphasize that REBORN achieves substantial performance improvement in PER by tailoring segmentation to the non-ideal unsupervised phoneme prediction model, rather than solely aiming for higher boundary evaluation scores. This represents a critical and intriguing finding in our work.
6 Discussion
In unsupervised ASR, learning the mapping between the segmental structures in speech and their corresponding transcription is challenging, especially without paired data. To tackle this, pro- pose REBORN, an iterative training method for unsupervised ASR. REBORN iteratively trains the segmentation model and the phoneme prediction model to improve the ASR performance. The segmentation model is trained using RL with carefully designed rewards to guide the segmentation model to find the boundaries of the segmental structures in speech signals. Extensive experiments on three datasets spanning seven languages show that REBORN outperforms prior best-performing unsupervised ASR models in all datasets except one language in MLS. We conduct comprehensive ablation studies to show that each component in REBORN is critical to the performance. We also explain the effectiveness of REBORN by analyzing its segmentation patterns and find that REBORN tends to produce segmental structures smaller than phones, which helps the generators predict the phoneme transcription better.
Limitations. Recent advancements in unsupervised ASR are still in the developmental stages, and we have not yet implemented REBORN in more realistic scenarios, such as low-resource languages or noisy environments. Additionally, the iterative training paradigm may amplify any existing biases in the dataset, an aspect that remains unexplored in this study. Furthermore, if the phoneme prediction model is poorly initialized, REBORN may not be able to provide huge performance improvements. In this work, lexicons are required to perform phonemicization on the unpaired text. Consequently, learning the unsupervised ASR system directly from speech to word-level tokens remains an open problem, where recent works aim tackle 30
Broader impacts. Unsupervised automatic speech recognition (UASR) predicting the textual transcriptions for the given speech signals using only unpaired speech and text data is highly attractive. The existence of thousands of low-resourced languages spoken globally is the major reason. This paper presents REBORN, a novel unsupervised ASR training algorithm. We foresee that REBORN has great potential to make low-resource languages more accessible to train ASR systems, making speech technology more accessible and boundaryless.
Acknowledgment
We thank the National Center for High-performance Computing (NCHC) of the National Applied Research Laboratories (NARLabs) in Taiwan for providing computational and storage resources. Shao-Hua Sun was supported by the Yushan Fellow Program by the Ministry of Education, Taiwan. We also extend our gratitude to Alexei Baevski, Wei-Ning Hsu, and Alexander H. Liu, authors of wav2vec-U and wav2vec-U 2.0, for providing valuable insights into their work.
References
[1] Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. In International Conference on Learning Representations, 2018.
10

---

[2] Alexei Baevski, Steffen Schneider, and Michael Auli. vq-wav2vec: Self self-suue learning of discrete speech representations. Proceedings of Inter national Conference on Learning Representations, 2020.
[3] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, 2020.
[4] Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. Unsupervised speech recognition. Advances in Neural Information Processing Systems, 2021.
[5] Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. Data2vec: general framework for self-superrvsed learning in speech, vision and language. In International Conference on Machine Learning. 2022.
[6] Mathieu Bernard and Hadrien Titeux. Phonemizer: Text to phones transcription for multiple languages in python. Journal of Open Source Software, 2021.
[7] Saurabhchand Bhati, Jesús Villalba, Piotr Zelasko, Laureano Horo-Velázquez, and Najim Dehak. Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation. In Interspeech, 2021.
[8] LFM ten Bosch, X Wang, and LCW Pols. Duration modeling with hidden markov models. The Journal of the Acoustical Society of America, 1993.
[9] Dogan Can, Victor Martinez, Pavlos Papadopoulos, and Shrikanth S Narayanan. Pykaldi: A python wrapper for kaldi. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2018
[10] Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du. Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang. Zhao You, and Zhiyong Yan. Gigaspeech: An evolving, multi-domain ASR corpus with 10, 000 hours of transcribed audio. In Interspeech, 2021.
[11] Kuan-Ya Yu Chen, Che-Ping Tsai, Da-Rong Liu, Hung-Yi Yi Lee, and Lin-Shan Lee. Completely Unsupervised Phoneme Recognition by a Generative Adversarial Network Harmonized with Iteratively Refined Hidden Markov Models. In Interspeech, 2019.
[12] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre- training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 2022.
[13] Chung-Cheng Cheng Chiu, James Qin, Yu Zhang, Jiahui Yu, and Yonghui Wu. Self-supervised learning with mandom-projection quantizer for speech recognition. In International Conference on Machine Learning. 2022.
[14] Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. Unsupervised Cross-Lingual Representation Learning for Speech Recognition. In Interspeech, 2021.
[15] David Crystal. A dictionary of linguistics and phonetics. John Wiley & Sons, 2011.
[16] Santiago Cuervo, Adrian Lancucki, Ricard Marxer, Pawel Rychlikowski, and Jan K Chorowski. Variable-rate hierarchical cpc leads to acoustic unit discovery in speech. Advances in Neural Information Processing Systems, 35:34995-35006. 2022.
[17] Siyuan Feng, Piotr Želasko, Laureano Moro-Velázquue and Odette Scharenborg. Unsupervised Acoustic Unit Discovery by Leveraging a Language-Independenn Subword Discriminative Feature Representation. In Proc. Interspeech 2021, 2021.
[18] Dongji Gao, Jiatong Shi, Shun-Po Chuang, Leibny Paola Garcia, Hung-yi Lee, Shinji Watan- abe, and Sanjeev Khudanpur. Euro: Espnet unsupervised asr open-source toolkit. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2023.
11

---

[19] John S Garofolo. Timit acoustic phonetic continuous speech corpus. Linguistic Data Consortium, 1993, 1993.
[20] Ian Goodfellow, Jean Pouget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances Neural Information Processing Systems, 2014.
[21] Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. Advances in Neural Information Processing Systems, 2017.
[22] Kenneth Heafield. Kenlm: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, 2011.
[23] Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021.
[24] J. Kahn, M. Rivière W. Zheng, F. Kharitonov, Q. Xu, P.E. Mazaré, J. Karadayi, V. Liptchin- sky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux. Libri-light: A benchmark for asr with limited or no supervision. In IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing, 2020.
[25] Jacob Kahn, Ann Lee, and Awni Hannun. telf-training for end-to-end speech recognition, In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
[26] Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Mare' Aurelio Ranzato. Unsuper- vised machine translation using monolingual corpora only. In International Conference on Learning Representations, 2018
[27] Guillaume Lample, Alexis Conneau, Mare' Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data, In International Conference on Learning Representations, 2018.
[28] Chia-ying Lee and James Glass. A nonparametric bayesian approach to acoustic model discovery. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, 2012.
[29] Shaoshi Ling, Yuzong Liu, Julian Salazar, and Katrin Kirchhoff. Deep contextualized acoustic representations for semi-supervised speech recognition. In ICASSP 2020-2020 IEEE Interna- lional Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
[30] Alexander H. Liu, Wei-Ning Hsu, Michael Auli, and Alexei Baevski. Towards end-to-end unsupervised speech recognition. In IEEE Spoken Language Technology Workshop, 2022.
[31] Andy T Liu, Shu-wen Yang, Po-Han Chi, Po-chun Hsu, and Hung-yi Lee. Mockingjay: Unsupervised speech representation learning with deep bidirectional transformer encoders. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
[32] Andy Liu, Shang-Wea Li, and Hung-yi Lee. Tera: Self-supervised learning of transformer encoder representation for speech. IEEF/ACM Transactions on Audio, Speech, and Language Processing, 2021.
[33] Da-Rong Liu, Kuan-Yu Chen, Hung-Yi Yi Lee, and Lin-Shan Lee. Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embed- dings. In Interspecch, 2018.
[34] Yu Liu, Jianshu Chen, and Li Deng. Unsupervised sequence classification using sequential output statistics. Advances in Neural Information Processing Systems, 2017.
12

---

[35] Vimal Manohar, Hossein Hadian, Daniel Povey, and Sanjeev Khudanpur. Semi-supervised training of acoustic models using lattice-free mmi. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). 2018.
[36] Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger. Montreal forced aligner: Trainable text-speech alignment using kaldi. Interspeech, 2017.
[37] AbdeIrahman Mohamed, Hung-yi Lee, Lasse Borgholt, Jakob D Havtorn, Joakim Edin, Christian Igel, Katrin Kirchhoff, Shang-Wen Li, Karen Livescu, Lars Maaløe, et al. Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal Processing, 2022.
[38] Mchryar Mohri. Finite-state transducers in language and speech processing. Computational Linguistics, 1997.
[39] Mehryar Mohri, Fernando Pereira, and Michael Riley. Weighted finite-state transducers in speech recognition. Computer Speech Language, 2002.
[40] Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé, Morgane Rivière, Evgeny Kharitonov, Alexei Bacvski, Ewan Dunbar, and Emmanuel Dupoux. The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling. arXiv preprint arXiv:2011.11588 2020.
[41] Junrui Ni, Liming Wang. Yang Zhang, Kaizhi Qian, Heting Gao, Mark Hasegawa-Johnson, and Chang D Yoo. Towards unsupervised speech recognition without pronunciation models. arXiv preprint arXiv:2406. 283800800 2024.
[42] Lucas Ondel, Lukáš Burget, and Jan Černocky. Variational inference for acoustic unit discovery. Procedia Computer Science, 2016.
[43] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03740 2018
[44] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: fast, extensible toolkit for sequence modeling. In Proceedings of Demonstrations, 2019.
NAACL-IILT 2019:
[45] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2015.
[46] Jongseok Park, Kyubyong & Kim. g2pe. https://github. http:/thhuu.com///yynnnnn////// /gpp 2019.
[47] Dean A. Pomerleau. Alvinn: An autonomous land vehicle in neural network. In Advances in Neural Information Processing Systems, 1988.
[48] Dean A. Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 1991.
[49] Daniel Povey, Amnb Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. The kaldi speech recognition toolkit, IEFE 201 workshop on automatic speech recognition and understanding. 2011.
[50] Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. ML.::
A Large-Scale Multilingual Dataset for Speech Research. In Interspeech, 2020.
[51] Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi,, al. Scaling speech technology to 1,000+ languages. arXiv preprint, 2023.
[52] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and llya Sutskever. Robust speech recognition via large-scale weak supervision. International Conference Machine Learning. 2023,
13

---

[53] Thomas Schatz, Vijayaditya Peddinti, Francis Bach, Aren Jansen, Hynek Hermansky, and Emmanuel Dupoux. Evaluating speech features with the minimal-paii abx task: Analysis of the classical mfc/plp pipeline. In INTERSPEECH 2013: 14th Annual Conference of the International Speech Communication Association, 2013.
[54] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Proceedings of the 32nd International Conference Machine Learning, 2015.
[55] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimization algorithms. CoRR, 2017.
[56] Luke Strgar and David Harwath. Phoneme segmentation using self-supervised speech models. In IEEE Spoken Language Technology Workshop, 2022.
[57] Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 1999.
[58] Zheng-Hur Tan, Najim Dehak, et al. rvad: An unsupervised segment-based robust voice activity detection method. Computer speech & language, 2020.
[59] Karel Vesely, Lukáš Burget, and Jan Černocky. Semi- Suni--iueeeieed DNN Training with Word Selection for ASR. In Proc. Interspeech 2017, 2017.
[60] Liming Wang, Mark Hasegawa-Johnson, and Chang D Yoo. Unsupervised speech recognition with n-skipgram and positional unigram matching. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.
[61] Yu-Hsuan Wang, Hung-yi Lee, and Lin-shan Lee. Segmental audio word2vec: Representing utterances as sequences of vectors with applications spoken term detection. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018.
[62] Shu wen Yang. Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdel- rahman Mohamed, and Hung yi Lee. SUPERB: Speech Processing Universal PERformance Benchmark. In Proc. Interspeech 2021, 2021.
[63] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 1992.
[64] Chih-Kuan Ych, Jianshu Chen, Chengzhu Yu, and Dong Yu. Unsupervised speech recognition via segmental empirical output distribution matching. In International Conference on Learning Representations, 2018.
14

---

Appendix
A Additional Results
A.1 Stability Analysis
In unsupervised ASR, stability has always been one of the major concerns since the unlabeled scenario makes the problem difficult and the training unstable. In Table 7 we demonstrate that REBORN is stable by calculating the mean PER and the corresponding standard deviations, well as the converged rate. We introduce that a converged experimental run is categorized by its result yielding PER < 40%, following Baevski et al. [4) As represented in the table, REBORN generates significantly better performance while being stable with a 100% converged rate both stages.
Next, we focus the ablations of REBORN stage 2 GAN training in Table 7 When the boundary merging is not applied, the GAN training on the original REBORN--eemeen than the initial stage. However, the average PER and the standard deviation are larger. The observation indicates that the training is slightly unstable, but does not diverge too much since still fully converged. On the other hand, when train the next-iteration GAN with completely random initialized parameters, the performance degrades even if we train the model for more steps. These two ablations directly demonstrate the effectiveness of adopting boundary merging and parameter initialization from the previous iteration, while the GAN training can still work on the original boundary with randomly initialized parameters.
features still better
Furthermore, we find out that the instability might highly correlated to the boundary frequency (the last column in Table 7). and this observation also corresponds with the prior works 30 Just as we performed in Table we show the topline result of training GAN from the oracle boundary in the last row. We contribute the good result to the disappearance of the length mismatch issue between the segmented feature and its textual transcription. only the k-means for segmentation instead of the two-stage k-means adjacent segmentation (or "k-means- based segmentation" in the main text) in wav2vec-U, U, the length mismatch issue arises and can not obtain reasonable results.
the INITAIL STAGE, when we use
As for REBORN, the boundary without merging has higher frequency compared with the wav2vec-U segmentation. However, using the original boundary for GAN training can still yield fully converged results with lower PER. We contribute this result to the improved quality of REBORN boundary, showing the importance and effectiveness of our segmentation learning via RL.
Table 7: The table for stability analysis. Each method is trained for 5 runs on LibriSpeech and evaluated on the test-clean split. REBORN performs steadily well while being fully converged in both of the stages. The ablations show the effectiveness of applying boundary merging and parameter initialization from the previous iteration for the stage 2 GAN training. The abbreviation "adj."
indicates adjacent pooling, which is the 2nd stage of pooling in wav2vec-U (sce Appendix B The
"Freq." " in the last column is the number of boundaries per second.
Ablation
Type
mean PER + std (%)
&-converged (1)
Freq. (Hz)
INITIAL STAGE
wav2vec-U U (b-means-only)
GAN
> 100
0% 100%
28.5
wav2vec-U U (k-means + adj.)
GAN
20.1 + 0.6
14.3
ITERATION
REBORN stage
RL
12.9 + 0.7
100%
16.3
REBORN stage
GAN
12.0 + 0.1
100%
11.6
w/o boundary merging w/ random initialization
GAN
13.9 + 2.1
100%
16.3
GAN
14.1 + 0.4
100%
11.6
TOPLINE
Train from oracle boundary
GAN
6.64 + 0.2
100%
11.9
15

---

Table Ablation study results of various configurations on the unsupervised metric assessed using validation set and phoneme error rates evaluated on the LibriSpeech dataset.
Stage
Stage 2
Configuration
Unsupervised Metric (1)
Dev-clean PER (%)
Test-clean PER (%)
Unsupervised Metric (1)
Dev-clean PER (%)
Test-clean
PER (%)
Repl Rypl Redit
256621.6
14.0
14.7
206312.79
12.2
12.9
251476.6
13.3
13.8
198447.93
11.7
12.1
Rypt + Redit + Rign
243419.4
12.7
12.9
191043.37
11.7
11.9
REBORN w/o BC
252302.3
14.8
14.9
237658.23
14.1
14.2
A.2 Reward Design Details
When training reinforcement learning, various methods have demonstrated that incorporating regu- larization terms can enhance model training stability [54 [55] In response this insight, have integrated both edit-distance and length rewards as forms of regularization our training regime. To identify the effective reward configuration, employed an unsupervised metric (Eq. [0] for selection. Detailed unsupervised metric scores and further evaluation results of our reward configurations are presented in Table
Our findings, as shown in Table 3 indicate that using perplexity (PPL) alone as a reward yields the lowest perplexity scores upon model convergence. However, this approach also results in higher scores on the unsupervised metric, implying a decline in model performance. Further analysis reveals that the PER increases when the PPL. reward the sole metric. This suggests that, without regularization, the model tends to minimize perplexity by producing more lengthy and less accurate transcriptions, which do not align well with the expected outputs.
A.3 Iteratively Polished Phoneme Predictors
In REBORN, we introduce iterative paradigm that can gradually polish the segmentation model and the phoneme predictor in turn. In Table g we wish to provide more evidence directly demonstrate that the phoneme predictors are actually improved through the iterations. We carry this out by feeding the identical oracle-segmented features to the phoneme predictors different stages. Given the same ideally-segmented features, a better phoneme predictor should yield better performance as well. In we show that the phoneme predictor the initial stage gives 13.8% PER when evaluating on the oracle-segmented features. REBORN'S 's iterative learning paradigm gradually improves the phoneme predictor, and the performance finally achieves the best in the 4th iteration by 7.4% PER. The relative performance gain compared with the initial stage is over 45%, presented in the table. The result even 1% close to the topline listed in the bottom row, indicating the effectiveness of REBORN'S 's iterative training.
Table 9
Table 9: We demonstrate that the REBORN phoneme predictors gradually improved though iterative training. Fach the phoneme predictors takes the same oracle-segmented features as input.
PER (%)
Relative
Phoneme Predictor
(evaluate oracle) Gain (t)
INITIAL STAGE
wav2vec-U
13.8
0%
REBORN
Iteration
10.9
21%
Iteration
9.9
Iteration
7.6
45%
Iteration 4
7.4
46% %
Iteration
8.7
37%
TOPLINE
Train from oracle
6.4
54%
16

---

Table 10: We implement REBORN across different speech foundation models on LibriSpeech, The
results are evaluated on test-clean. REBORN has exhibited strong generalizability by providing substantial performance improvements across different speech foundation models. We extract the 15th layer representations from HuBERT and W following EURO [8]
PER on test-clean (4)
PER diff.
Feature extractor
Initial stage
Iter. 1-stagel
Iter. 1-stage2
after Iter.
wav2vec 2.0 3
19.3%
12.9%
11.9%
-7.4%
HuBERT 23
20.1%
12.8%
10.4%
-9.7%
WavLM 12
18.7%
14.7%
12.7%
-6.0%
A.4 Generalizability across Different Speech Foundation Models
We evaluate REBORN'S generalizability across different speech foundation models in Table TO
Specifically, in Table T0 replace the feature extractor from wav2vee 2.0 with HuBERT [23
WavLM [12]. As presented in the table, REBORN can yield performance improvements regardless of which speech foundation model is used and how well the initial stage performed. The results further
strengthen the robustness and the generalizability of our method. Due to the computational resource limitation, we only conduct the ablation for one iteration.
B wav2vec-U
wav2vec-U 4 is popular UASR framework whose variants
18
30 have achieved SoTA perfor-
mance on multiple UASR benchmarks. Following prior works
33 wav2vec-U also uses adversarial
training to train the phoneme prediction model. The wav2vec-U framework takes the raw waveform as the input and predicts phoneme transcription via three stages: (1) the feature extraction stage, (2) the feature preprocessing stage, and (3) the phoneme prediction stage.
Stage (1) extracts speech features using self-supervised speech foundation model, wav2vec 2.0 3 For a spoken utterance, we denote the extracted feature as = [ T2,...,..) Frl] E TR 7x where T is the time dimension and d is the feature dimension. Stage (2) is the feature preprocessing stage. The dimension of the extracted speech features is first reduced using PCA from d' = 512. Next, k-means clustering and additional heuristics, namely, adjacent pooling, are adopted sequentially to define the boundary of the segmental structures in the speech features. The features in the same segmental structure are mean-pooled over the two stages into one feature embedding to reduce sequence length from T to T" to match phoneme sequences better. In the main text. we call this process "k-means-based segmentation". We denote the output from the feature preprocessing stage as [s1, $2,..., ST I.T'' x In Stage (3), the phoneme prediction model takes the pooled speech feature as input and predicts the phoneme sequence Y [V1. 1/..... vr Last, de-duplication step removes the consecutive duplicate phoneme predictions from Y. resulting in shortened phoneme sequence Y [v' 1/2 ..., V's) without consecutive duplicate phonemes.
1024 to
During training, the generator takes the preprocessed speech feature as input and generates the logits of Y. We can shorten the logits of Y based on its argmax prediction, and the shortened logits (denoted as G G(S) are then taken input by the discriminator D for GAN training. The adversarial learning loss is defined as follows:
LGAN min max E [log (log(D(Z))) Z~2
+ E [log(1 D(G(S))) S
(5)
9
where Z is tokenized phoneme sequence sampled from the phonemicized text corpora Z and is the preprocessed speech feature sampled from the preprocessed speech corpora S. Besides the min-max objective in the adversarial training, wav2vec-U 4 also adopts the three regularized objectives for better convergence and performance. First, the gradient penalty Lyp ( is applied on the discriminator D to stabilize training, where the input of D is the linear combination of generated
17

---

sample G and a real sample Z.
Eg
[IID (1-a)2)112 1
(6)
Next, the smoothness penalty Cup computed across consecutive segments of a generated logit sequence, promoting the generation of closely aligned neighboring outputs by the model.
sp
Σ
l1 lell
Furthermore, the phoneme diversity loss Lpa is applied to encourage higher vocabulary usage of the generated sequences. As shown in Eq. [8] the entropy of the averaged softmax distribution of the generator outputs (HG(G(S))) is maximized over the phoneme vocabulary across batch B.
Spu
Σ - Hg(G(S))
(8)
B
SEB
Finally, we describe the overall training objectives & by weighting with coefficients X, 7 and n. respectively.
C = LGAN - Xgug + 7 + 1/Cp
In this work, we utilize the same training objective from wav2vec-U perform GAN training ($ 3.2
C Implementation Details
C.1 Data Preparation and Phoneme LM Formulation
Our data preparation process mainly follows wav2vec-UP for fair comparisons. Specifically, we use wav2vec 2.0 large pretrained on LibriLight [24] as the feature extractor in English and the multilingual version of wav2vec 2.0 (XLSR-53) for the other langauges Before the feature extraction stage, we attempt to remove most of the silence through an unsupervised voice activity detection (VAD) method [58 However, the method is not completely perfect. To stabilize the GAN training, special <silence> token introduced to the text data. The <silence> token is inserted the front and the end of each sentence, and also between words by probability of 0.25 according wav2vec-U. Then, we phonemicize all the words in the text data along with the previously inserted <silence> tokens for GAN training. The two off-the- phonemizers For LibriSpeech, we use the G2P phonemeizer [46), and the numerical stress makers are removed reduce the phoneme set, which is shown be beneficial for the performance 4 [8 30)
used to perform phonemicization.
As for Multilingual LibriSpeech, we use Phonemizer 6 toolkit that supports a wide variety of languages other than English. The language-switching labels are removed well as the phonemes appear less than 1000 times in the text data. For the Italian, we isolate the double consonant symbol in the lexicon, which is crucial for the initial stage converge. Note that for TIMIT we do not adopt any of the above data preparations since it originally contains its phone inventories including silence. We use the 39-phone inventory which can easily found Kaldi 49 for training and evaluation? following prior UASR works [8]
6
After the data preparation, we directly use the phonemicized text with <silence> token to train the phoneme L.M. More specifically, we utilize KenLM 22 to build the 4-gram LM mentioned
unsupervised
http://hhh..coaaa.oa/llllllllala 48--8-3396
map
https://gtthubb.c///p///////
18

---

Since in the GAN training stage, we are actually matching the two distributions between the segmented speech features and phonemicized text with <silence>, we naturally build our external LM for RI. segmentation learning from the same preprocessed text. The LM is also utilized for the unsupervised rross-validation described in C.5
in 3.1.3
C.2 Self-Training with Hidden Markov Models
Originated from semi semi- supervised learning. self- toffitraining aims at providing a trivial approach to utilize the unpaired or unlabeled data effectively [25] [5] [59]. More specifically, given supervised ASR system MASR learned from the labeled speech-texx text data Dibbeled, we can use MASR to generate pseudo labels of a significant but unlabeled speech data Dunlaheldd- Now, we can utilize the speech data along with the pseudo labels to perform supervised learning or re-tuning [59 to obtain a new ASR system M' ASR- Generally, given that the pseudo label good enough, we may expect that M' ASR performs better than MASR.
In the unsupervised setup, the core concept of self-training is suited well. By simply using an unsupervised ASR system MUASK to perform pseudo labeling, we can train a new ASR model M' UASk by the supervised objective. Since no labeled data is used (we are just using the pseudo labels), the new ASR system does not violate the unsupervised setup. More specifically, we use Hidden Markov Models (HMMs; Bosch et Das the backbone of the new ASR system. This can be dated back to [1] and is also adopted by recent UASR 4 30 Furthermore, to give fair comparisons and increase reproducibility. we follow the publicly available code provided by wav2vec-U for HMM self-training Our results in Table indicate that with the high-qualin pseudo labels generated by REBORN, the new ASR model based on HMMs performs better than the original UASR system and is the best among all the other methods.
C.3 Model Architecture
We parametrize the segmentation model To using a two-layer I-dimensional convolutional neural network (CNN). This choice of architecture allows us to efficiently extract relevant information from adjacent speech features, making it particularly well-suited for our segmentation task. The two con- volutional layers in our CNN have different kernel sizes, namely 7 and 3. This is lightweight model, making it computationally efficient for applications. The phoneme prediction model (generator) and the discriminator in GAN training are parametrized by single-layer 1-dimensional CNN and two-layer 1-dimensional CNN, respectively, following wav2vec-U. Note that the phoneme prediction model generates the phoneme sequences non-autoorereessivll due to its architecture.
C.4 Hyperparameter Search
We search the optimal hyperparameters of our phoneme predictor and segmentation models with unsupervised validation metrics. For the phoneme predictor, we directly adopt the search space of the hyperparameters and the unsupervised validation metric from wav2vec-U [4.Wr We search the hyperparameters indicated as X, and v in Appendix B only during the initialization stage of the phoneme predictor ($ 3.3). We directly adopt the same hyperparameters the following iterations of GAN training. As for the segmentation model, we take the idea from Baevski et al. [4] and derive similar unsupervised metric.
C.5 Unsupervised Validation Metric for Segmentation Model Selection
Each segmentation model, denoted ne, undergoes unsupervised performance evaluation by pre- dicting the transcriptions the validation set. The phoneme predictions generated by the model are processed to form a de-duplicated set, denoted as Y [vi ... V.M, where consecutive duplicate phonemes are removed.
The unsupervised metric places significant emphasis two key aspects: Language Model (LM) negative log-likeliban likelihoodd (NLL) and vocabulary usage. Vocabulary usage is calculated as U(Y)) = Evev I(v E Yj), where V denotes the entire vocabulary set. The validation metric seeks to
http://ssstttoooooeecee///////es masuprvisedd/alldissl1....
19

---

minimize the NLL while simultaneously maximizing vocabulary usage. The NLL metric reflects the model's proficiency in generating sensible sentences consistent with the dataset distribution. while vocabulary usage quantifies the diversity of phonemes employed by the model. This combined approach ensures that our unsupervised ASR system's s predictions align with the expected LM distribution and exhibit a wide linguistic range, thus mitigating the risk of favoring repetitive, high- probability sentences.
Additionally, the lengths of transcriptions are taken into account. We employ NLL without length normalization, thereby favoring predictions that score high under the language model but without an undue preference for overly lengthy transcriptions. The optimization objective is formally expressed as:
Σ Ei=1
M log PLM (v(,))
0- = arg min
(10)
U(Y))
C.6 Training Details
For the training of our segmentation model, we initially employ behavioral cloning, followed by updates using policy gradient methods. In the BC phase, configure our model with batch size of 128 and a learning rate of 0.0005, training for 20 epochs. We also assign cross-entropy weights of [1. 5] for boundary classification, address the imbalance where the class labeled significantly outnumbers the class labeled 1.
During the policy gradient update phase, the model is trained with the same batch size of 128, but with a reduced learning rate of 0.0001 and weight decay of 0.0001. We utilize the Adam W optimizer in conjunction with the CosineAnnealing proceeds for 40 epochs, while for the TIMIT dataset, owing to its smaller size, we extend the training to 500 epochs ensure convergence.
scheduler. For the LS and MLS datasets, the training
To refine the performance of our reinforcement learning framework, we have tuned the reward coefficients, leveraging the validation metric (Eq. TO). to secure optimal outcomes across varied datasets. Specifically, the coefficient for the PPL difference reward, Cppl. is fixed 1.0 to prioritize the minimization of perplexity in generated sequences. The coefficient for the edit-distance reward, Cedit. set at 0.2. And the coefficient for the length difference reward, Gjen is selected from a predefined set of values within the range [0.0, 0.2, 0.4, 0.6, 0.8). These values carefully chosen to balance the importance of length consistency with other factors, with the optimal reward configuration for each dataset detailed in TI1
noteworthy that, as the iterations progress, the length constraints become less critical. This observation suggests that the model gradually learns to generate sequences of appropriate length, reducing the need for explicit length-based regularizations.
As for the Stage 2 GAN training, we inherit the three weighted coefficients in Eq. P from the previous iteration. The initial training of wav2vec-U takes 150000 steps following the original paper. After the initialization stage, we find out that optimizing with 20000 steps for each iteration is enough for converging, taking advantage of initializing the parameters from the previous stage.
All the experiments can be done using a single NVIDIA V100 GPU. The initial stage of wav2vec-1 GAN training and the REBORN stage reinforcement learning takes about 12 hours of training
Table 11: Best reward configurations obtained through hyperparameter searches on each dataset.
LibriSpeech
TIMIT
MLS
Dataset
iter.
iter: 2-5
iter.
de
fr
es
it
pl
Cypl
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Cedit
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
Ceea
0.2
0.0
0.0
0.4
0.2
0.8
0.4
0.6
0.4
20

---

an NVIDIA V100 GPU for each run. As for REBORN stage 2, since the model is not randomly initialized, it only takes about 1.5 hours of training time.
C.7 WFST Decoding for Obtaining Word-Level Outputs
In REBORN, phoneme-level outputs can be generated directly with greedy decoding. However, obtaining word-level outputs requires additional modules such WFST or external language mod- els 18 B4 In this work, we incorporate REBORN with WFST to generate word-level transcriptions. Our decoding procedure mainly follows Baevski al. 4], which builds the WFST using PyKaldi (9), and additional self-loops are introduced to mimic the CTC behavior. Moreover, two hyperparameters: the acoustic scale a, and the additive blank weight are added during the WFST decoding process.
In wav2vec-U, the search interval of a is [0, 8] and is tuned within [-3, 8). As for REBORN, we find that using much smaller search space for both of the hyperparameters enough for obtaining reasonable outputs. More specifically, We tune the acoustic scale in (0.4, 1.2] with step 0.1 and the blank weight in [-5, 2] with step 1.
It is worth mentioning that we utilize the original boundary generated by the segmentation model instead of the merged one ($ 3.1.5) for WFST decoding. We find that the increased amount of segments directly leads to longer generator outputs, which is beneficial for WFST to generate higher- quality outputs. The phenomenon is also discovered by the prior works 4 30) In wav2vee-U U [4], the segmental feature used for WFST decoding is the one before adjacent pooling; while in wav2vec-UU 2.0 (30), they simply lower the decoding stride to generate more lengthy outputs for WFST decoding.
D Speech Self-supervised Learning and Acoustic Unit Discovery
Sulf-supe learning (SSL) aims to explore effective ways for representation learning without labeled data. Recent speech SSL models can be categorized into the three classes based on the pretraining objective (37], including generative 29
31 32]. contrastive 2 3 43 and predictive [5]
1 23
The SSL representtaio can be utilized for many different downstream tasks, such as phoneme
recognition, speaker identification, emotion recognition [62]. Recent UASR also takes advantage
from speech representations from SSL models [8].
Extended from self-supervised learning, acoustic unit discovery mainly focuses on deriving phoneme or word-like units or learning speech representations that retain only linguistically relevant infor- mation in an unsupervised manner [16] 17 28 40) 42 The task is highly related to unsupervised
phoneme/word segmentation, and the standard evaluation protocols include the phoneme/word boundary F1, frame-level ABX score (53) ZeroSpeech Challenge [40]. or phoneme accuracy with supervised learned linear prediction head [6]. Although both acoustic unit discovery and unsupervised ASR are learned without supervision, they differ lot in their learning objectives. As an extended topic of representation learning, acoustic unit discovery targets on learning setter-segmmnted representations; while unsupervised ASR directly aims solving the distribution matching problem, and may utilize high-quality speech representations to reach the goal.
E Different Schemes for Boundary Evaluation
According Strgar and Harwath [56), the original protocol for evaluating phoneme boundaries contains some ambiguity. The issue arises from double counting when both the ground truth boundaries and the predicted boundaries fall within the same tolerance window. To address this issue. they propose harsh (or strict) evaluation protocol that prevents double counting. The original protocol referred to as the leniens boundary evaluation protocol. can be assumed that studies conducted before Strgar and Harwath [56) used the lenient evaluation method, which often overestimated the quality the predicted boundaries. We encourage interested readers refer the original paper for more detailed explanations.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims make accurately reflect the paper's contributions and scope, and they are supported by our experimental results in Section
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
We do not propose theoretical results in this work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Appendix C we try our best to provide details of our experiments and implementations.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification: Our code and models are released along with the publication. Please refer to for more details.
https://gitthb com/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e-g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.cc/ppp wwwppbblccgguiicuceccccccooooopp /eeggudddsseCeebbbinninn Stt.summmisioooiiciir..oooi for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental setting/details can be found in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We include the stability discussion of our method in Appendix A.I
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mention the details of computing resources in Appendix C.6
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We ensure our research does not violate the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts of our work in Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We consider that our work does not pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We ensure that the assets used this work are properly cited or mentioned.
Most of them can be found in Appendix C We include the URLs if possible.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets./paperswtt package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We release all models and the code base along with the publication. Please refer to lattps://g/t gighub...... com/ wttm//ayyyy776//bbrrrrr/o uasr for more details.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "qs5j78hIDB", "reviewer": "Reviewer_TC8W", "summary": "This paper proposes a new UASR approach which explicitly learns both segmentation (phoneme boundary) prediction and phoneme class prediction. To do segmentation, the paper proposes some reinforcement-learning objectives. The phoneme class prediction follows an existing approach. It turns out the proposed approach is effective, achieving the new SOTA for UASR on public datasets.", "questions": "- Besides Table 4, it would be better to add a table for TIMIT which comes with word/phone-level timestamps. Another dataset Buckeye can be considered.\n- Typo or presentation suggestions:\n  - Figure 1: probably add \"1 means the start of a segment\" to the caption to make it self-contained", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper's presentation is clear and is easy to follow.\n- The proposed technique (RL for learning segmentation model) is novel and non-trivial\n- The proposed technique is effective. It achieves SOTA by a big margin across several datasets and languages.\n  - There is analysis why the proposed technique works so effectively\n  - See my concern later", "weaknesses": "- In fact, ASR don't need accurate segmentation, e.g. CTC model is peaky -- the peak of a phoneme can appear at any frame for this phoneme. For this reason, it is not so clear to me why improving segmentation/phoneme granularity can help improve UASR so much.\n  - Please justify.\n  - For this reason, I put the rating to \"border line\" instead of \"weak accept\"\n\n- Unfortunately, the predicted segments are not more accurate than other approaches and probably are not good to be used to obtain accurate timestamps for ASR (or not? -- you may use TIMIT or Buckeye's ground-truth timestamp to check this)\n\n- L324:  \"some segmental structures smaller than the phonemes\": does it probably mean that -- if we use sub-phoneme structures in wav2vec-U 2.0, it may be as performant as the proposed approach, as we are able to learn a smaller granularity of signals. In this case, we won't need the RL learning.\n  - Would be better to give a real example", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thorough and constructive comments. Please find the response to your questions below.\n\n\n> ASR don't need accurate segmentation, e.g. CTC model is peaky -- the peak of a phoneme can appear at any frame for this phoneme.\n\nWhile recent **supervised ASR** does not need explicit segmentation, under the **unsupervised scenario**, learning the mapping from very long speech features to short text transcriptions has been found to be very challenging. To mitigate this difficulty, **segmentation is critical to ensure good unsupervised ASR performance and stability**, which is evident from early works [1-3] to recent studies [4-5]. \n\nTo investigate the importance of segmentation in UASR, we compare the training stability of UASR algorithms: wav2vec-U, wav2vec-U 2.0, and REBORN. wav2vec-U 2.0 is the only UASR algorithm that does not segment the speech features. wav2vec-U uses k-means segmentation (see Appendix D), while REBORN uses a segmentation model learned using RL. Following the EURO setup described in Section 4.1, we use each algorithm to train UASR models on LibriSpeech. Since each algorithm has its own configuration and search space, including the random seeds, we iterate through these configurations, with each resulting in one model. For each algorithm, we report the *\"percentage of models yielding PER < 40%\"* among all training configurations and random seeds. The results are shown in Table 1 below. \n\n*Table 1: The importance of segmentation in current UASR. Results are obtained from LibriSpeech following the EURO setup. As the table indicates, **wav2vec-U and REBORN are fully converged in PER with proper segmentation**.*\n\n| Method | &nbsp; Segmentation | &nbsp; Number of models we trained &nbsp;&nbsp;&nbsp; | Percentage of models with PER < 40% ↑ |\n|-|:-:|:-:|:-:|\n| wav2vec-U (k-means-based) | ✓ | 40 | 100% |\n| REBORN | ✓ | 50 | 100% |\n| wav2vec-U (no-segmentation) | x | 40 | 0% |\n| wav2vec-U 2.0 | x | 64 | 19% |\n\nFrom Table 1, we observe that wav2vec-U 2.0, which does not have segmentation, is highly unstable. Conversely, REBORN and wav2vec-U (with k-means segmentation) always result in a UASR model with PER lower than 40%. Additionally, removing the segmentation from wav2vec-U makes the algorithm unstable and never yields a model with PER lower than 40%.\n\n> For this reason, it is not so clear to me why improving segmentation/phoneme granularity can help improve UASR so much.\n\nThe results in Table 1 have justified the importance of segmentation in UASR. Moreover, we found that the quality of the segmentation strongly affects the performance of UASR (Table 4 in the paper): **using hand-crafted rules or separately learned segmentation boundaries yields suboptimal UASR performance while using the oracle phoneme boundary greatly improves the UASR performance**. Motivated by the above observation, we thus propose to use RL to learn segmentation that is **tailored for the phoneme prediction model**. The segmentation boundary in REBORN is learned with feedback from the phoneme prediction model, which ensures the boundary is useful for improving the UASR performance and **attains state-of-the-art results on many widely used UASR datasets**.\n\n> The predicted segments are not more accurate than other approaches and probably are not good to be used to obtain accurate timestamps.\n\nAs suggested by the reviewer, we use TIMIT’s human-annotated phone-level timestamps for evaluation and report the results in Table 2 below.\n\n*Table 2: Boundary evaluation results on TIMIT. REBORN with boundary merging (Figure 1-(c\\)) is close to the existing SoTA method.*\n\n|Method|Precision|Recall|F1 Score|R-Value|\n|-|-|-|-|-|\n|k-means-based|0.62|0.75|0.68|0.68|\n|Strgar and Harwath [6]|0.85|0.79|0.82|0.84|\n|REBORN|0.61|0.83|0.71|0.62|\n|REBORN (w/ boundary merging)|0.80|0.78|0.79|0.82|\n\nThe results show that the initial boundaries learned by REBORN (before boundary merging in Figure 1-(c)), which are specifically optimized for the phoneme prediction model, already achieve a high recall. With boundary merging, i.e., consecutive segments with the same phoneme prediction are merged as illustrated in Figure 1-(c\\), **REBORN achieves better alignment, bringing our results close to existing SoTA unsupervised segmentation methods**.\n\n> It would be better to add a table for TIMIT which comes with word/phone-level timestamps.\n\nWe thank the reviewer for suggesting this experiment, which strengthens the contribution of our method. We will revise the paper to include this result. \n\n> if we use sub-phoneme structures in wav2vec-U 2.0, it may be as performant as the proposed approach\n\nSince wav2vec-U 2.0 uses a fixed stride 1D-CNN to downsample the speech features and does not segment the speech features, we are unsure how to incorporate sub-phoneme structures in wav2vec-U 2.0. We would appreciate it if the reviewer could provide more details. We are willing to implement, analyze, and discuss it if time allows. \n\n> Figure 1 presentation suggestions\n\nWe thank the reviewer for the suggestion. We will revise Figure 1 based on the suggestion.\n\n### References\n\n[1] Liu et al. “Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings.” *Interspeech, 2018*.\n\n[2] Yeh et al. \"Unsupervised speech recognition via segmental empirical output distribution matching.” *ICLR, 2019*.\n\n[3] Chen et al. “Completely Unsupervised Phoneme Recognition by a Generative Adversarial Network Harmonized with Iteratively Refined Hidden Markov Models.” *Interspeech, 2019*.\n\n[4] Baevski et al. \"Unsupervised speech recognition.\" *NeurIPS, 2021*.\n\n[5] Gao et al. \"Euro: Espnet unsupervised asr open-source toolkit.\" *ICASSP, 2023*. \n\n[6] Strgar et al. “Phoneme segmentation using self-supervised speech models.” *SLT, 2022*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ehQ5DNUi3", "reviewer": "Reviewer_vhRd", "summary": "The paper proposes an improvement over wav2vec-U for unsupervised speech recognition, specifically for the task of predicting phonemes. For the final WER performance, a given lexicon is used to get from the phoneme sequence to words.\n\nSo the paper focuses on improving the segmentation/boundaries of the phonemes.\n\nTwo model parts:\n\nThe segmenter, trained via RL and various different reward functions.\n\nGAN-style training for the phoneme prediction model based on the segments from the segmenter.\n\nStage 1 trains only the segmenter, and assumes a given frozen phoneme prediction model. In the first iteration, the phoneme prediction model is initialized from  wav2vec-U.\n\nStage 2 trains only the phoneme prediction model, taking the segments from the segmenter.\n\nIn GAN-style training of the phoneme prediction model: Phoneme prediction model is the generator, and discriminator tries to distinguish a generated/predicted phoneme sequence vs a real one, using the unpaired text data.", "questions": "So you train a phoneme generator model here. But it's a bit unclear, how do you get from phonemes to words? It refers to the appendix, and WFST decoding is mentioned. It mentions that it needs a lexicon. So to make this clear: the lexicon provides a given mapping from phonemes to words? If this is given, is it fair to call this unsupervised ASR then? It seems like you cheat here by giving it a crucial part for the unsupervised ASR task. While this is still an interesting problem to study then, I think this should be made much more clear. This is not really unsupervised ASR to me. Or if you really want to stick to the term, at least make it very clear (not just in the appendix, and only for readers who know what a \"lexicon\" is) that this is what you take as a given here. This also has further implications: The phoneme inventory is then also given.\n\nThe exact definition of the phoneme predictor/generator is a bit unclear. Is this an auto-regressive model? Or non-autoregressive?\n\nHow do you sample from the phoneme predictor for policy gradient? Does this involve beam search?", "limitations": "-", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Good results.\n\nCode will be published.", "weaknesses": "Some parts could be made more clear. (See below.)\n\nNeeds Wav2vec-U for initialization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thorough and constructive comments. Please find the response to your questions below.\n\n> How do you get from phonemes to words? ... it needs a lexicon.  If lexicon is used, is it fair to call this unsupervised ASR? I think this should be made much more clear.\n\nWe thank the reviewer for raising this concern. **Our work does utilize an additional phonemizer (L668-L670) to obtain lexicon, following the standard setup in UASR [1-5]**. We acknowledge that using a lexicon is a limitation of recent UASR works, including ours, which is discussed in Appendix A (L540).  As suggested by the reviewer, **we will revise the paper to clearly discuss this in Section 2 and Section 4.3**. \n\n> The definition of the phoneme predictor is unclear. Is it autoregressive or non-autoregressive?\n\nIt is non-autoregressive. Precisely, it is a one-layer CNN following wav2vec-U. We will revise the paper to include this information in Section 4.3 to make it clear.\n\n> How do you sample from the phoneme predictor for policy gradient? Does this involve beam search?\n\nWhen training the segmentation model using policy gradient, we obtain the phoneme predictor by taking the argmax prediction from the phoneme predictor.  Since our phoneme prediction model is non-autoregressive, we do not perform a beam search. We will revise the paper to include this detail in Section 3. \n\n> Needs wav2vec-U for initialization\n\nWe acknowledge that relying on a reasonably effective phoneme prediction model for initialization may present a limitation, as noted in L539. However, we would like to clarify that our method is not limited to the wav2vec-U for initialization; instead, we have experimented with using different feature extractors, such as HuBERT [6] and WavLM [7] following EURO [4], and starting from different phoneme predictors on LibriSpeech (see Appendix C.5). The results show that **REBORN achieves comparable performance improvements even with a completely different backbone model and a less effective phoneme predictor**.\n\n\n### References\n\n[1] Liu et al. “Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings.” *Interspeech, 2018*.\n\n[2] Yeh et al. \"Unsupervised speech recognition via segmental empirical output distribution matching.” *ICLR, 2019*.\n\n[3] Baevski et al. \"Unsupervised speech recognition.\" *NeurIPS, 2021*.\n\n[4] Gao et al. \"EURO: Espnet unsupervised asr open-source toolkit.\" *ICASSP, 2023*.\n\n[5] Liu et al. “Towards end-to-end unsupervised speech recognition.” *SLT, 2022*.\n\n[6] Hsu et al. \"Hubert: Self-supervised speech representation learning by masked prediction of hidden units.\" *TASLP, 2021*.\n\n[7] Chen et al. \"Wavlm: Large-scale self-supervised pre-training for full stack speech processing.\" *JSTSP, 2022*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PFrEX8101x", "reviewer": "Reviewer_AM1A", "summary": "This paper addresses the challenging problem of learning speech recognition without parallel recordings and transcriptions. They build upon state-of-the-art approaches of Wav2Vec-U(2) in this area and propose to refine a pretrained model using a reinforcement learning approach. The main idea is to split the problem into two parts: segmentation and segment classification into “phonemes”. Since neither segmentation nor labels are known, they use policy gradients and leverage “phoneme” classifier rewards to learn the segmentation model. The training is stabilized by splitting it into iterations where only one of the two models is being trained while the other is frozen, and additional rewards (penalties?) that act as regularizers for the main perplexity difference reward. In essence, the model is overall encouraged to improve the perplexity vs the previous model checkpoint, but not change either the segment length or phone edit distance too much. The evaluation is performed on 100h subset of LibriSpeech and full TIMIT for English, and 100h subsets of MLS for German, Dutch, French, Spanish, Italian and Portuguese. The authors demonstrate significant improvements in WER and PER vs state-of-the-art baseline models.", "questions": "* Could REBORN be leveraged in a semi-supervised setup, e.g., starting from an ASR pretrained on a low amount of supervised data, to leverage a larger amount of non-parallel data for fine-tuning?", "limitations": "* The proposed method depends strongly on the availability of an initial pretrained ASR model (presumably unsupervised). My understanding is that it is more of a “refinement” stage for existing models rather than stand-alone method. I think it should be presented as such - the title and the abstract don’t mention that.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "* Their main idea of framing UASR as an RL problem is original and interesting. Overall I think it is a solid contribution to UASR literature.\n* The proposed RL training is able to significantly improve upon Wav2Vec-U ASR results. \n* The evaluation setup is comprehensive enough to believe the method may be generally applicable for other languages.\n* The method, architecture, and experimental setup are described comprehensively.\n* The ablations related to reward function designs are interesting and demonstrate that the proposed edit-distance and length-difference rewards are effective regularizers for the main perplexity difference reward.", "weaknesses": "* The author’s (mis)use of the word “phoneme” is problematic. E.g., p.2 l.64 mentions “segmental structures (that) are acoustic units smaller than phonemes”. Phoneme is a perceptual construct that represents all possible (sequences of) sounds (phones) for which there is no lexical distinction. Such sequence may very well be empty. Therefore it is problematic to talk about phonemic segmentation; what the authors may have had in mind is phonetic segmentation instead. This problem is discussed more thoroughly in: Moore, R. K., and L. Skidmore. \"On the use/misuse of the term 'phoneme'.\" Proceedings of Interspeech 2019.\n* The authors claim that “the effect of feature segmentation on phoneme prediction is non-differentiable”, however, it can be. The authors may consult e.g. the segmental CPC approach with its differentiable boundary detection in: Bhati, S., Villalba, J., Żelasko, P., Moro-Velázquez, L., Dehak, N. (2021) Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation. Proc. Interspeech 2021, 366-370, doi: 10.21437/Interspeech.2021-1874", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thorough and constructive comments. Please find the response to your questions below.\n\n> The author’s (mis)use of the word “phoneme” is problematic. What the authors may have had in mind is phonetic segmentation instead. \n\nWe thank the reviewer for pointing this out. **We will thoroughly revise the paper to ensure the correct use of these terms**. \n\n> The authors claim that “the effect of feature segmentation on phoneme prediction is non-differentiable”, however, it can be.\n\nWe would like to clarify that in our paper, we meant that the process of predicting phonemes based on predicted segments is naturally non-differentiable. As the reviewer pointed out, some techniques could allow for approximating the backpropagation gradients. **We will revise our claim in the paper to avoid confusion and discuss the work [Bhati et al., 2021] referred by the reviewer**.\n\n> Could REBORN be leveraged in a semi-supervised setup?\n\nWe thank the reviewer for this interesting idea. We believe that REBORN could benefit especially when the labels of the limited supervised data are noisy. In this way, the phoneme predictor might be non-ideal and could possibly get improved through REBORN’s iterative paradigm. We would love to investigate this idea in the future thoroughly.\n\n> It should be presented as a “refinement” stage for existsing models, but the title and the abstract don’t mention that. \n\nWe will rethink the paper's title and abstract to clarify this. We appreciate the reviewer’s suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pW5nRWzrnP", "reviewer": "Reviewer_q9jx", "summary": "This paper proposes a training framework for unsupervised speech recognition models, building on top of wav2vec-u. The authors note that the performance in wav2vec-u is hindered by the quality of segment boundaries, as they show that the phoneme-error-rate can be improved significantly with ground truth segmentation boundaries. The proposed method introduces a segment boundary prediction mode and a phoneme prediction model. First, the segment boundary prediction model is trained with a policy gradient method. The reward function is based on the perplexity of the phoneme sequence. This sequence is predicted from the computed segmentation boundaries, and is compared to earlier versions of the boundary prediction model, i.e, the boundary segmentation model from a previous iteration, or wav2vec-u at iteration 0. The perplexity should be as low as possible and can be computed by using a language model from unpaired text data. Secondly, the phoneme prediction model is trained with an adversarial loss, where the phoneme prediction model acts as a generator of a phoneme sequence, and the discriminator has to distinguish between these and real phoneme sequences from the unpaired text data. These steps as then repeated until no performance improvements are observed.", "questions": "Why do you state in 4.3 to only use 1 to 2 iterations, while figure 2 shows results with improvements until 5 iterations? \n\nWhy do you think that your model performs better on 5 languages but worse on Portuguese compared to wav2vec-u 2.0? \n\nWhy did you decide to mean-pool the segment before predicting the phoneme? I would venture that this throws away information and is not necessary?", "limitations": "The ultimate goal of UASR is to enable speech technology for digitally rare languages without good labeled data. Although this work studies UASR on 6 languages, they are all from rich European countries. It remains to be seen whether this method works for languages from a different continent, especially because this method relies heavily on high quality textual data, to compute the perplexity in the RL loss, and for the discriminator.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors clearly show that the wav2vec-u model suffers from the quality of the segmentation boundaries, and propose a well-described methodology to improve on it. The authors report results on the expected datasets, and perform a small ablation study to analyze the behavior of the proposed method.", "weaknesses": "The authors propose a *very* complex methodology, with a lot of moving and finicky components, as reinforcement learning and GANs are both known to be fickle and hard to reproduce. I worry therefore that it will be hard to reproduce these results, but the authors promised in the checklist to release the code.\n \nThe authors use many tricks to make the method more stable and performant:\n1. Initialize the phoneme prediction model with the pre-trained wav2vec-u model\n2. Introduce 2 auxiliary reward functions, which heavily punish predicted sequences which are longer or shorter than the previous iteration\n3. Use behavior cloning (supervised loss on boundary pseudolabels) before the RL loss \n4. boundary merging (segments with the same phoneme prediction are merged)\n5. self-training of the phoneme prediction model (supervised loss on phoneme pseudolabels)\n\nThese are not a weakness per se, but point towards the overall complex picture of the proposed method. I hope future work can find a simpler and more elegant formulation of the UASR problem :)\n\nThe authors claim their method is iterative, but in paragraph 4.3 state they only use 1 to 2 iterations. I then wonder why the authors try to sell this as an iterative method, instead of follow-up stages to (significantly) improve on wav2vec-u.\n\nMinor comments on writing:\n* line 21: supervisedly trained with a ... -> trained with self-supervision using a ... \n* line 33: translate and -> translate, and \n* line 50: model: -> model. \n* line 74: remove \"the\" \n* line 77: GAN -> The GAN\n* line 95: The wav2vec-U network \n* line 98: to -> for\n* line 99: I cannot really parse this sentence, origanated to be -> originally (?) \n* line 102: We discuss these two related topics further in Appendix F.\n* line 149: The REINFORCE algorithm", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thorough and constructive comments. Please find the response to your questions below.\n\n> The authors propose a very complex methodology. **These are not a weakness per se**, but point towards the overall complex picture of the proposed method. \n\nTo ensure the reproducibility of our work, **we will release the codes, training and evaluation scripts, and model weights**. \n\n> Why the authors try to sell this as an iterative method, instead of follow-up stages to (significantly) improve on wav2vec-u. \n\n> Why do you state in 4.3 to only use 1 to 2 iterations, while figure 2 shows results with improvements until 5 iterations?\n\nWhile the performance gain is significant in the early iterations, Figure 2 shows that further improvements can still be achieved with more iterations. **In Table 1, we use only two iterations to emphasize the effective integration of REBORN and self-training,** as well as to demonstrate the efficiency of REBORN, i.e., achieving improved performance only requires few iterations. \n\n> Minor comments on writing:\n\nWe thank the reviewer for the detailed suggestions. We will revise the paper to fix them.\n\n> Why do you think that your model performs better on 5 languages but worse on Portuguese?\n\nWe hypothesize that our method performs slightly worse in Portuguese because the cross-lingual feature extractor (XLSR-53) uses the least amount of Portuguese data compared to other languages during self-supervised pre-training.\n\n> Why did you decide to mean-pool the segment before predicting the phoneme?\n\nOur phoneme prediction model is initialized from wav2vec-U, which uses mean-pooled features. To ensure consistency with wav2vec-U, we also use mean-pooling in REBORN. Exploring other pooling methods is interesting and potentially beneficial, which is left for future work. \n\n> The ultimate goal of UASR is to enable speech technology for digitally rare languages without good labeled data. Although this work studies UASR on 6 languages, they are all from rich European countries.\n\nWe thank the reviewer for pointing this out. Indeed, we have not yet tested REBORN on a digitally rare language. Referencing prior works [1-3], we tested REBORN on the most commonly used datasets in UASR. We will revise the paper to include this limitation in Appendix A.\n\n### References\n\n[1] Chen et al. “Completely Unsupervised Phoneme Recognition by a Generative Adversarial Network Harmonized with Iteratively Refined Hidden Markov Models.” *Interspeech, 2019*.\n\n[2] Baevski et al. \"Unsupervised speech recognition.\" *NeurIPS, 2021*.\n\n[3] Gao et al. \"Euro: Espnet unsupervised asr open-source toolkit.\" *ICASSP, 2023*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: I personally am not convinced after prompting the discussion among authors and reviewers, but I'm trusting the reviewers' judgment and recommend acceptance.

I find the numbers reported in the paper sloppy, if not fishy. There is a clear intention to hide the weaker results. It's understandable, but overly practiced in this paper. We, as a community, have lost the opportunity to actually know where the improvement comes from. The paper could have been a lot better if it were open about the weaknesses and where it performs worse. It's not only for better science, but opens up opportunity for others to improve.

I recommend revising the paper to make the experiments controlled, i.e., changing one thing and only one thing at the time (especially between wav2vec-U and iteration 1). In addition, reviewers provide valuable suggestions that lead to proposed changes in the rebuttal. All of these will make this a great paper, and I sincerely hope that the authors make their best effort to include these suggestions.
Decision: Accept (poster)