GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement

## Abstract

Abstract The evolution of speech technology has been spurred by the rapid increase in dataset sizes. Traditional speech models generally depend on a large amount of labeled training data, which is scarce for low-resource languages. This paper presents GigaSpeech 2, a large-scale, multi-domain, multilingual speech recognition corpus. It is designed for low-resource languages and does not rely on paired speech and text data. GigaSpeech 2 comprises about 30,000 hours of automatically transcribed speech, including Thai, Indonesian, and Vietnamese, gathered from unlabeled YouTube videos. We also introduce an automated pipeline for data crawling, transcription, and label refinement. Specifically, this pipeline uses Whisper for initial transcription and TorchAudio for forced alignment, combined with multi-dimensional filtering for data quality assurance. A modified Noisy Student Training is developed to further refine flawed pseudo labels iteratively, thus enhancing model performance. Experimental results on our manually transcribed evaluation set and two public test sets from Common Voice and FLEURS confirm our corpus’s high quality and broad applicability. Notably, ASR models trained on GigaSpeech 2 can reduce the word error rate for Thai, Indonesian, and Vietnamese on our challenging and realistic YouTube test set by 25% to 40% compared to the Whisper large-v3 model, with merely 10% model parameters. Furthermore, our ASR models trained on Gigaspeech 2 yield superior performance compared to commercial services. We believe that our newly introduced corpus and pipeline will open a new avenue for low-resource speech recognition and significantly facilitate research in this area.

## 1 Introduction

In recent years, the scaling of model parameters and data size has prevailed and proven effective in a range of areas, including language [, ], vision [, ], as well as speech processing [, , ].
Consequently, pursuing superior AI models is now closely associated with expanding model size and leveraging larger, high-quality datasets. In the realm of Automatic Speech Recognition (ASR), several large-scale open-source labeled speech datasets [, , , , , ] have been proposed.
However, these extensive datasets are only available for several mainstream languages, such as English and Mandarin, hindering speech recognition development for low-resource languages.
Moreover, traditional ASR corpus [, , , ] construction relies heavily on human-labeled speech data, making it time-consuming and a major bottleneck in the fast-paced AI industry. Reducing dependence on vast labeled data is crucial when expanding to new languages and domains [].
YODAS [] attempts to address this issue by building multilingual datasets via scraping audio and transcriptions from YouTube. However, neither manual nor automatic subtitles accurately reflect the speech content, resulting in unguaranteed quality.

With this perspective in mind, we propose a new paradigm for constructing large-scale ASR datasets, focusing exclusively on audio content irrespective of the existence or quality of corresponding text pairs. This approach leverages the gigantic amount of unlabeled audio data, thereby bypassing the constraints of scarce paired data.
We introduce GigaSpeech 2
(^1^11[https://huggingface.co/datasets/speechcolab/gigaspeech2](https://huggingface.co/datasets/speechcolab/gigaspeech2))
, an evolving, large-scale, multi-domain, multilingual ASR corpus for low-resource Southeast Asian languages. GigaSpeech 2 raw comprises about 30,000 hours of automatically transcribed speech, including Thai, Indonesian, and Vietnamese. GigaSpeech 2 refined consists of 10,000 hours of Thai, 6,000 hours each for Indonesian and Vietnamese.
To achieve this, an automated pipeline is developed for data crawling, transcription, and filtering.
Furthermore, a modified Noisy Student Training (NST) [] method is proposed to refine labels from flawed data iteratively.
Through comprehensive evaluations, ASR models trained on GigaSpeech 2 refine can reduce the word error rate for Thai, Indonesian, and Vietnamese on our YouTube test set by 25% to 40% compared to the powerful Whisper large-v3 model, with merely 10% model parameters.

Our contributions can be summarized as follows:

- •
We release GigaSpeech 2, an evolving, large-scale, multi-domain, and multilingual ASR corpus focusing on low-resource languages. GigaSpeech 2 raw comprises about 30,000 hours of automatically transcribed speech across Thai, Indonesian, and Vietnamese. GigaSpeech 2 refine consists of 10,000 hours of Thai, 6,000 hours each for Indonesian and Vietnamese.
- •
We develop an automated pipeline for data crawling, transcription, and label refinement, enabling the creation of large-scale speech datasets without reliance on labeled data.
- •
We propose a modified NST method to refine flawed pseudo labels iteratively. Our modified NST considers scaling, relabeling, and filtering data within each iteration, significantly improving data quality.
- •
We release a series of challenging and realistic speech recognition test sets, including Thai, Indonesian, and Vietnamese. Compared to previous public test sets, GigaSpeech 2 test sets more realistically reflect speech recognition scenarios and mirror the real performance of an ASR system for low-resource languages.
- •
Experimental results on our challenging GigaSpeech 2 test sets, as well as other competitive public test sets including Common Voice and FLEURS, demonstrate the superiority of the ASR models trained on GigaSpeech 2 over several competitive baselines, including Whisper large-v3 and commercial services.

## 2 Related Work

**Table 1: Comparison of data size between GigaSpeech 2 and other common public multilingual speech datasets on Thai (“th”), Indonesian (“id”), and Vietnamese (“vi”).**
| Dataset | Language | Total Duration<br>(h) | Domain | Speech Type | Labeled | Label Type |
| --- | --- | --- | --- | --- | --- | --- |
| Common Voice [] | th | 172.0 | Open domain | Read | Yes | Manual |
| id | 28.0 |  |  |  |  |  |
| vi | 6.0 |  |  |  |  |  |
| FLEURS [] | th | 13.3 | Wikipedia | Read | Yes | Manual |
| id | 12.6 |  |  |  |  |  |
| vi | 13.3 |  |  |  |  |  |
| VoxLingua107 [] | th | 61.0 | YouTube | Spontaneous | No | - |
| id | 40.0 |  |  |  |  |  |
| vi | 64.0 |  |  |  |  |  |
| CMU Wilderness [] | th | 15.6 | Religion | Read | Yes | Manual |
| id | 70.9 |  |  |  |  |  |
| vi | 9.2 |  |  |  |  |  |
| BABEL [] | vi | 87.1 | Conversation | Spontaneous | Yes | Manual |
| VietMed [] | vi | 16.0 | Medical | Spontaneous | Yes | Manual |
| Thai Dialect Corpus [] | th | 840.0 | Open domain | Read | Yes | Manual |
| TITML-IDN [] | id | 14.5 | News | Read | Yes | Manual |
| MEDISCO [] | id | 10.0 | Medical | Read | Yes | Manual |
| YODAS manual [] | th | 497.1 | YouTube | Spontaneous | Yes | Manual |
| id | 1420.1 |  |  |  |  |  |
| vi | 779.9 |  |  |  |  |  |
| YODAS automatic [] | th | 1.9 | YouTube | Spontaneous | Yes | Pseudo |
| id | 8463.6 |  |  |  |  |  |
| vi | 9203.1 |  |  |  |  |  |
| GigaSpeech 2 raw | th | 12901.8 | YouTube | Spontaneous | Yes | Pseudo |
| id | 8112.9 |  |  |  |  |  |
| vi | 7324.0 |  |  |  |  |  |
| GigaSpeech 2 refined | th | 10262.0 | YouTube | Spontaneous | Yes | Pseudo |
| id | 5714.0 |  |  |  |  |  |
| vi | 6039.0 |  |  |  |  |  |

Multilingual Low-Resource Speech Datasets Several public multilingual speech datasets have emerged for low-resource languages. BABEL [], a pioneering dataset, includes conversational telephone data in 17 African and Asian languages. Common Voice [] offers 19,000 hours of validated recordings in over 100 languages. FLEURS [] covers 102 languages with 12 hours of supervised data per language. CMU Wilderness [] provides 20 hours of New Testament data for over 700 languages. VoxLingua107 [] contains 6,628 hours of unlabeled YouTube data across 107 languages.
However, most public multilingual speech datasets focus on high-resource languages, leaving low-resource languages with limited annotated speech data. For example, the available open-source data for Thai, Indonesian, and Vietnamese is scarce, as detailed in Table 1. In contrast, industry-utilized speech models like Whisper [], MMS [], Google USM [], and Universal-1 [] are trained on massive industrial-grade datasets, the details of which remain undisclosed.
To resolve the problem, YODAS [] attempts to crawl audio from YouTube, but neither manual nor automatic subtitles accurately reflect the speech content, resulting in unguaranteed quality.
Moreover, widely used evaluation benchmarks for low-resource languages [, ] only consist of read speech, which is relatively clean and mismatched with real-world speech data.

Multilingual Automatic Speech Recognition As the demand for communication between people worldwide grows, many works [, , , , , , , , , , ] have shifted attention to multilingual speech recognition.
Whisper [], built on 680,000 hours of web data, supports 99 languages. Google USM [], trained on YouTube audio, extends to 100+ languages. Massively Multilingual Speech (MMS) [], trained on religion data, further scales to 1,107 languages.

Noisy Student Training (NST) NST [, , , ] is a self-training technique that leverages unlabeled data to enhance performance. Traditional NST methods start with training a teacher model on high-quality labeled data. Each student model then trains on both noisy-augmented labeled data and pseudo-labeled data generated by its teacher from the unlabeled data.

Figure: Figure 1: Automated construction pipeline of GigaSpeech 2, comprising (1) audio collection, (2) dataset partitioning, (3) automated transcription with Whisper, (4) forced alignment with TorchAudio, (5) transcription normalization, (6) data filtering, and (7) label refinement.
Refer to caption: x1.png

## 3 Dataset Construction

Our proposed automated construction pipeline is illustrated in Fig. 1. Sec. 3.1 covers the stages involved in building GigaSpeech 2 raw and Sec. 3.2 further construct GigaSpeech 2 refined.

### 3.1 GigaSpeech 2 raw: Automated Crawling and Transcription

Audio Collection Due to the scarcity of human-labeled data in low-resource languages, our dataset is collected with a focus solely on the audio content, irrespective of the existence or quality of corresponding text pairs. This strategy allows for a broader range of audio data.
Given the scarcity and uneven distribution of resources for low-resource languages, we strategically focus on crawling videos from YouTube channels based on two key assumptions. First, prioritizing popular channels ensures consistent domain characteristics and audio quality. Second, different channels have no speaker overlap, simplifying the subsequent data partitioning.
The data collection process starts by manually defining categories of interest. The selected topics include Agriculture, Art, Business, Climate, Culture, Economics, Education, Entertainment, Health, History, Literature, Music, Politics, Relationships, Shopping, Society, Sport, Technology, and Travel. Alongside multiple topics, various content formats are also considered, including Audiobook, Commentary, Lecture, Monologue, Movie, News, Talk, and Vlog. This broad selection ensures the comprehensiveness of the dataset across multiple domains for research and analysis. Once the list of YouTube channels is prepared, we use yt-dlp
(^2^22[https://github.com/yt-dlp/yt-dlp](https://github.com/yt-dlp/yt-dlp))
toolkit to download all audio files in WebM format. These files are then converted to WAV format with a single channel and resampled at a 16 kHz sampling rate.

Creating TRAIN/DEV/TEST Splits To ensure no speaker overlap between the splits, we manually verify that there is no speaker overlap between different channels and partition the data by allocating different YouTube channels to each subset. The dataset is divided into three distinct subsets: training (TRAIN), development (DEV), and TETS. The DEV and TEST sets each contain 10 hours and are manually transcribed by professionals, while the remainder is allocated to the TRAIN set. Table 1 shows the amount of data across these three languages. Detailed analysis of GigaSpeech2 is illustrated in Appendix B.

Transcription with Whisper Whisper large-v3 model
(^3^33[https://huggingface.co/openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3))
from OpenAI is employed to transcribe audio files automatically. For each audio recording, a 30-second segment is selected from the middle to perform language detection using Whisper. Only recordings that match the target languages are transcribed.

Forced Alignment with TorchAudio Although Whisper can generate timestamps, inspection reveals they are not precise enough. We resort to the model
(^4^44[https://dl.fbaipublicfiles.com/mms/torchaudio/ctc_alignment_mling_uroman/model.pt](https://dl.fbaipublicfiles.com/mms/torchaudio/ctc_alignment_mling_uroman/model.pt))
from TorchAudio [] for forced alignment, which provides reliable alignment for noisy transcriptions, supports efficient processing on GPUs, and handles longer sequences more effectively [].

Text Normalization Text normalization on transcripts involves applying Normalization Form Compatibility Composition (NFKC), converting all characters to uppercase, removing punctuation, and mapping Arabic numerals to corresponding words in the respective languages.

Multi-dimensional Filtering A series of heuristic filtering rules across text and audio modalities are implemented to exclude relatively poor-quality samples.

- •
Charset Filtering: Segments are retained if they only contain characters permitted by the charset of the respective language.
- •
Language Confidence Filtering: The language identification (LID) model
(^5^55[https://dl.fbaipublicfiles.com/fasttext/supervised-models/lid.176.bin](https://dl.fbaipublicfiles.com/fasttext/supervised-models/lid.176.bin))
from fastText [] is used to filter based on the estimated language confidence score, retaining only segments with confidence scores above a predetermined threshold. This method effectively eliminates meaningless and repetitive segments. Note that language identification based on audio has already been performed before transcription.
- •
Audio Duration Filtering: Segments are filtered based on duration, with only those retained within the predetermined minimum and maximum duration thresholds.
- •
Balancing: We carefully control the duplication of transcripts caused by channel-specific content while preserving natural linguistic patterns.

### 3.2 GigaSpeech 2 refined: Iterative Label Refinement

Some samples remain low quality due to inaccuracies in Whisper transcriptions and imprecise forced alignment boundaries. To address this, we develop a modified NST method.
As illustrated in the bottom right corner of Fig. 1, it starts by training a teacher model on a subset of flawed pseudo labels and iteratively expands the training set, generates new pseudo labels, and filters them. A student model, equal to or larger than the teacher, is trained on refined pseudo labels and assigned as the new teacher. SpecAugment [], Bypass [], and feature mask [] introduce noise during each NST step. Bypass, a type of stochastic depth, learns channel-wise scalar weights to combine the module input and output. Feature mask performs dropout in the hidden dimension of the feedforward and convolution layer but shares over the time dimension. This deliberate noising allows the student model to learn consistency with the teacher model, which remains unaffected by noise when generating pseudo labels []. This iterative process gradually enhances data quality. Detailed algorithm steps are shown in Appendix A Algo. 1.

## 4 Experiments

### 4.1 ASR Model Training on GigaSpeech 2

Our ASR systems are constructed by Zipformer Transducer []. Two Zipformer [] variants, namely Zipformer-M and Zipformer-L, are employed for each NST iteration. Specific configurations are listed in Appendix C.1. During Noisy Student Training,
SpecAugment [] is used as input noise, and Bypass [] and feature mask [] are used as model noise.

**Table 2: Comparison of ASR performance with different NST iterations on various evaluation sets, including GigaSpeech 2 DEV and TEST, Common Voice 17.0 TEST, and FLEURS TEST. Detailing training set size (# Hours), model size (# Params), Character Error Rate (CER) for Thai, and Word Error Rate (WER) for Indonesian and Vietnamese.**
| NST<br>Iter | # Hours<br>(h) | # Vocab | # Params<br>(M) | CER / WER |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| GigaSpeech 2<br>DEV   TEST | Common Voice<br>TEST | FLEURS<br>TEST |  |  |  |  |  |
| Thai |  |  |  |  |  |  |  |
| 1 | 4378 | 500 | 65.5 | 12.14 | 15.10 | 8.88 | 14.33 |
| 2 | 3497 | 500 | 65.5 | 10.97-9.6% | 13.15-12.9% | 6.99-21.3% | 11.93-16.7% |
| 3 | 7219 | 2000 | 68.6 | 10.50-4.3% | 12.46-5.2% | 4.61-34.0% | 10.94-8.3% |
| 4 | 10262 | 2000 | 151.9 | 10.45-0.5% | 12.46-0.0% | 4.15-10.0% | 10.54-3.7% |
| Indonesian |  |  |  |  |  |  |  |
| 1 | 5765 | 2000 | 68.6 | 16.68 | 15.99 | 19.82 | 16.29 |
| 2 | 4534 | 2000 | 68.6 | 15.60-6.5% | 15.23-4.8% | 15.83-20.1% | 14.30-12.2% |
| 3 | 5714 | 2000 | 151.9 | 14.58-6.5% | 14.92-2.0% | 13.83-12.6% | 13.77-3.7% |
| Vietnamese |  |  |  |  |  |  |  |
| 1 | 2351 | 2000 | 68.6 | 16.08 | 16.95 | 24.63 | 17.86 |
| 2 | 1764 | 2000 | 68.6 | 15.08-6.2% | 14.72-13.2% | 18.81-23.6% | 13.50-24.4% |
| 3 | 6039 | 2000 | 151.9 | 14.09-6.6% | 12.83-12.8% | 14.43-23.3% | 11.59-14.1% |

Table 2 presents the ASR results across different NST iterations on three evaluation sets, including the development and test sets from GigaSpeech 2 and the Common Voice 17.0 and FLEURS test set.
Each iteration involves distinct modifications aimed at refining high-quality transcriptions.
A subset of automatic transcriptions generated by Whisper large-v3 is used to train the initial teacher model (Iteration 1). The teacher model then filters the training utterances by applying a CER/WER threshold, using the original labels as references and the new labels generated by the teacher as the hypothesis. The student model is trained on this filtered set with noise injected (Iteration 2). The student model is then used as the teacher to generate new labels on a larger subset of raw automatic transcriptions, applying the same filter to refine the training data. This refined data is used to train the student model with noise injected (Iteration 3). The process repeats in subsequent iterations, and the model size is scaled up to a larger version in the final iteration (Iteration 3 of Indonesian and Vietnamese, Iteration 4 of Thai).

According to the results shown in Table 2, several notable trends can be observed:

1) Across all three languages (Thai, Indonesian, and Vietnamese), iteratively scaling the training data size, adding noise, and filtering labels lead to consistent improvements in the WER performance on the evaluation sets until the final iteration. This indicates that the iterative approach of refining and scaling the training data is effective in enhancing the accuracy of the raw transcriptions.

2) The Thai language achieves the absolute lowest error rates consistently across iterations from Iteration 1 to 4, indicating the effectiveness of the NST approach for this particular language. The best NST model outperforms the standard transcription model data by WER reductions of 1.69%, 2.64%, 4.73%, and 3.79% absolute (13.92%, 17.48%, 53.27%, and 26.45% relative) respectively (Iteration 4 vs. Iteration 1).

Additional ablation studies on our modified NST in Appendix D Table 8 demonstrate the effectiveness of relabeling and discuss the detriment of enlarging noise when scaling the training data.

### 4.2 Comparison to Existing ASR Systems

To demonstrate the efficacy of our ASR models trained on Gigaspeech 2, several mainstream and competitive ASR systems, including Whisper [] from OpenAI, MMS [] from Meta, and commercial services from Azure and Google, are used as benchmarks.

Whisper:
Our work builds upon Whisper [], a suite of large-scale, multitask, and multilingual speech models developed by OpenAI. It leverages the encoder-decoder Transformer architecture [], with model sizes ranging from 39 million parameters (tiny) to 1.55 billion parameters (large). Additionally, Whisper offers variants spanning from an English-only version to a multilingual model capable of handling 99 languages. To conduct a comprehensive evaluation, we test three configurations: Whisper base, Whisper large-v2, and Whisper large-v3 models.

MMS:
The Massively Multilingual Speech (MMS) [] project leverages self-supervised learning (SSL) techniques and a novel dataset to expand the language coverage of speech technology significantly. The core components include pre-trained wav2vec 2.0 [] models for 1,406 languages, a single multilingual ASR model supporting 1,107 languages, speech synthesis models for the same set of languages, and a language identification model capable of recognizing 4,017 languages. In this study, we employ the MMS L1107 configuration.

Azure AI Speech:
Azure Speech CLI offers a convenient way to leverage Microsoft’s speech recognition capabilities directly from the command line. It not only supports a wide range of audio file formats but also possesses the ability to handle various streaming audio inputs.
We utilize the Azure Speech CLI version 1.37 in this paper, which is the latest version available.

Google USM:
The Universal Speech Model (USM) [] is introduced as a single, large-scale model that excels in ASR across over 100 languages. This achievement is made possible by pre-training the model’s encoder on a vast, unlabeled multilingual dataset of 12 million hours, covering more than 300 languages, followed by fine-tuning on a smaller labeled dataset. To conduct a thorough comparison, we utilize their Chirp Speech-to-Text v2 model for performance evaluation.

**Table 3: Comparison of ASR results for models trained on GigaSpeech 2 with open-source multilingual ASR models and commercial ASR services, evaluated on test sets from GigaSpeech 2, Common Voice 17.0, and FLEURS. The evaluation metrics are Character Error Rate (CER) for Thai and Word Error Rate (WER) for both Indonesian and Vietnamese. “$\dagger$" denotes commercial services.**
| Model | # Params<br>(M) | CER / WER |  |  |
| --- | --- | --- | --- | --- |
| GigaSpeech 2 | Common Voice | FLEURS |  |  |
| Thai |  |  |  |  |
| Whisper large-v3 | 1542 | 20.44 | 6.02 | 11.55 |
| Whisper large-v2 | 1541 | 22.47 | 8.79 | 15.50 |
| Whisper base | 72 | 46.47 | 32.59 | 42.28 |
| MMS L1107 | 964 | 31.75 | 14.49 | 23.07 |
| Azure Speech CLI $1.37.0^{\dagger}$ | - | 17.25 | 10.20 | 13.35 |
| Google USM Chirp v$2^{\dagger}$ | - | 49.70 | 14.75 | 63.35 |
| Gigaspeech 2 (proposed) | 151.9 | 12.46 | 4.15 | 10.54 |
| Indonesian |  |  |  |  |
| Whisper large-v3 | 1542 | 20.03 | 7.43 | 7.85 |
| Whisper large-v2 | 1541 | 21.44 | 8.93 | 8.95 |
| Whisper base | 72 | 39.37 | 34.70 | 33.76 |
| MMS L1107 | 964 | 35.27 | 20.72 | 24.49 |
| Azure Speech CLI $1.37.0^{\dagger}$ | - | 18.07 | 10.33 | 11.18 |
| Google USM Chirp v$2^{\dagger}$ | - | 19.63 | 9.70 | 7.23 |
| Gigaspeech 2 (proposed) | 151.9 | 14.92 | 13.83 | 13.77 |
| + Common Voice + FLEURS | 151.9 | 14.95 | 7.33 | 12.74 |
| Vietnamese |  |  |  |  |
| Whisper large-v3 | 1542 | 17.94 | 13.74 | 8.59 |
| Whisper large-v2 | 1541 | 18.74 | 18.00 | 10.26 |
| Whisper base | 72 | 39.88 | 44.07 | 40.41 |
| MMS L1107 | 964 | 46.62 | 43.88 | 55.35 |
| Azure Speech CLI $1.37.0^{\dagger}$ | - | 11.86 | 10.21 | 11.88 |
| Google USM Chirp v$2^{\dagger}$ | - | 13.28 | 12.46 | 11.75 |
| Gigaspeech 2 (proposed) | 151.9 | 12.83 | 14.43 | 11.59 |
| + Common Voice + FLEURS | 151.9 | 12.39 | 11.47 | 9.94 |

We compare the performance of our proposed approach trained on Gigaspeech 2 against these above-mentioned ASR models, including Whisper (base, large-v2, and large-v3), MMS L1107, Azure Speech CLI 1.37.0 and Google USM Chirp v2
(^6^66Abnormal high deletion rates with Google USM in Thai are observed in our repeated testing.)
, across three languages: Thai, Indonesian, and Vietnamese. The ASR performance is evaluated regarding character error rate (CER) or word error rate (WER) on three distinct test sets from GigaSpeech 2, Common Voice 17.0, and FLEURS. According to the results shown in Table 3, there are several intriguing findings:

1) For the Thai language, our ASR model trained on Gigaspeech 2 (Table 3, Thai, Row 7) outperforms all competitors, including commercial services from Azure and Google USM, securing the top rank across all three test sets among the seven models.
It outperforms Whisper large-v3 by WER reductions of 7.98%, 1.87%, and 1.01% absolute (39.04%, 31.06%, and 8.74% relative) (Table 3, Thai, Row 7 vs. Row 1). Remarkably, our model achieves such impressive performance with nearly one-tenth of the parameters compared to Whisper large-v3 (151.9 M vs. 1542.0 M).

2) For the Indonesian and Vietnamese languages, our system demonstrates competitive performance compared to existing baseline models. This highlights the efficacy of our pipeline in delivering high-quality results with a lightweight model.
Specifically, on the GigaSpeech 2 test set in the Indonesian language, our system (Table 3, Indonesian, Row 7) outperforms all baseline models, attaining the best performance.
Compared to Whisper large-v3, the model trained on Indonesian achieves an absolute WER reduction of 5.11%, corresponding to a relative reduction of 25.51% (Table 3, Indonesian, Row 7 vs. Row 1).
Similarly, the model trained on Vietnamese achieves an absolute WER reduction of 5.11% compared to the Whisper large-v3 model. (Table 3, Vietnamese, Row 7 vs. Row 1).

3) Our model exhibits degraded performance compared to commercial ASR systems on the Common Voice and FLEURS test sets in Indonesian and Vietnamese, which can be attributed to the domain mismatch. Contrastively, we observe a performance leap after adding Common Voice and FLEURS training data into GigaSpeech 2 (Table 3, Indonesian & Vietnamese, Row 7 vs. Row 8).

Despite this substantial disparity in training data size, our method achieves the best performance for the Thai language domain and delivers comparable results to commercial models for Indonesian and Vietnamese. This remarkable accomplishment highlights the efficacy of our approach in leveraging limited, free, open-source, unlabeled data to train highly competitive speech recognition models. It showcases a promising path towards developing high-quality speech recognition systems without the need for extensive, proprietary datasets, thereby reducing the barrier to entry and enabling wider accessibility.

### 4.3 Comparison to the YODAS corpus

**Table 4: Comparison of ASR results for models trained on YODAS and GigaSpeech 2, evaluated on test sets from GigaSpeech 2, Common Voice 17.0, and FLEURS. The evaluation metrics are Character Error Rate (CER) for Thai and Word Error Rate (WER) for both Indonesian and Vietnamese.**
| Training Set | # Params<br>(M) | CER / WER |  |  |
| --- | --- | --- | --- | --- |
| GigaSpeech 2 | Common Voice | FLEURS |  |  |
| Thai |  |  |  |  |
| YODAS manual | 68.6 | 27.34 | 10.71 | 14.19 |
| YODAS manual | 151.9 | 28.76 | 10.96 | 16.11 |
| GigaSpeech 2 refined | 151.9 | 12.46 | 4.15 | 10.54 |
| Indonesian |  |  |  |  |
| YODAS manual | 68.6 | 25.77 | 10.82 | 14.63 |
| YODAS manual + automatic | 68.8 | 41.11 | 15.41 | 47.26 |
| YODAS manual | 151.9 | 25.11 | 11.05 | 12.67 |
| GigaSpeech 2 refined | 151.9 | 14.92 | 13.83 | 13.77 |
| Vietnamese |  |  |  |  |
| YODAS manual | 68.6 | 40.35 | 31.07 | 25.68 |
| YODAS manual + automatic | 68.6 | 71.91 | 25.73 | 61.38 |
| YODAS manual | 151.9 | 40.71 | 32.58 | 29.32 |
| GigaSpeech 2 refined | 151.9 | 12.83 | 14.43 | 11.59 |

Table 4 compares ASR performance across different models trained on YODAS [] and GigaSpeech 2 datasets evaluated on various test sets. Note that YODAS Thai automatic is not included because of insufficient data (only 1 hour). Analyzing the trends, the following observations can be concluded:

1) The models trained on GigaSpeech 2 refined yield significantly better results than models trained on the YODAS datasets in the GigaSpeech 2 test set for all three languages.

2) For Thai and Vietnamese, the models trained on GigaSpeech 2 refined consistently outperform those trained on the YODAS manual datasets across all evaluation sets.
For Thai, the best model outperforms the YODAS manual with relative WER reductions of 56.68%, 62.13%, and 34.57% (Table 4, Thai, Row 3 vs. Row 2).
For Vietnamese, it achieves relative WER reductions of 68.48%, 55.71%, and 60.47% relative (Table 4, Vietnamese, Row 4 vs. Row 3).

3) The YODAS manual may suffer from overfitting or noisy data issues due to simplistic filtering rules, leading to inconsistent performance in Indonesian.

4) Purely automatic generation of YODAS tends to degrade performance, as observed for Vietnamese and Indonesian, likely due to the inherent noise and errors in the automatically generated subtitles.

### 4.4 Training ASR Models within ESPNet and Icefall on GigaSpeech 2

Icefall:
The neural Transducer [] architecture is adopted, with Zipformer-L as the encoder and the pruned RNN-T loss [] as the object function. 2000-class Byte Pair Encoding (BPE) [] word pieces are used. More details are provided in Appendix C.1.

ESPNet:
The Conformer [] CTC/AED [] system is adopted from ESPNet [], with Conformer-L as the encoder and a combination of the localized sensitivity of convolutional neural networks and the long-range modeling capabilities of Transformers []. 2000-class BPE word pieces are used. More details can be found in Appendix C.2.

**Table 5: Comparison of ASR models trained on GigaSpeech 2 with Icefall and ESPNet toolkits, evaluated on GigaSpeech 2 TEST set. The evaluation metrics are Character Error Rate (CER) for Thai (th) and Word Error Rate (WER) for both Indonesian (id) and Vietnamese (vi).**
| Toolkit | Model | # Params<br>(M) | CER / WER |  |  |
| --- | --- | --- | --- | --- | --- |
| th | id | vi |  |  |  |
| Icefall | Zipformer/Stateless Pruned RNN-T | 151.9 | 12.46 | 14.92 | 12.83 |
| ESPNet | Conformer/Transformer CTC/AED | 111.8 | 13.70 | 15.50 | 15.60 |

Table 5 shows the results of ASR models trained with Icefall and ESPNet. The models trained with ESPNet are slightly worse than Icefall in all three languages, which is as expected and can be explained by the discrepancy in the number of model parameters (112M vs. 152M). It is worth noting that the results in Table 5 are intended to provide baseline systems for these two popular toolkits to demonstrate the universality of Gigaspeech 2 instead of pursuing state-of-the-art performance.

## 5 Limitation and Future Work

Due to time constraints, we only tested 3-4 iterations of the proposed NST model. We are optimistic that more iterations will yield even better results. The Indonesian and Vietnamese datasets are not as extensive as the Thai datasets, so we plan to add more data for these two languages. We will also expand our low-resource language family in our future investigation. In addition, we did not perform language model fusion to further boost performance since there is a lack of high-quality and in-domain text data for low-resource languages. To resolve potential legal risks, our dataset follows the same license as GigaSpeech [], allowing only non-commercial research and educational use.

## 6 Conclusion

This paper introduces a new multilingual speech dataset, GigaSpeech 2, and a novel automated pipeline to boost speech recognition performance using in-the-wild audio-only data. GigaSpeech 2 aims to address the scarcity of labeled training data on low-resource languages by developing this large-scale, multi-domain, and multilingual corpus. Extensive experiments are conducted to validate the efficacy of our newly introduced corpus. The ASR models trained in three languages, which are Thai, Indonesian, and Vietnamese within GigaSpeech 2, demonstrate superior and impressive performance compared to various powerful ASR models, including Whisper large v2/v3 from OpenAI, MMS from Meta, and even commercial services from Google and Azure. The related resources, including the training corpus, curated test sets, automated pipeline, and recipes, will be released
(^7^77https://github.com/SpeechColab/GigaSpeech2)
later to facilitate research in this direction. In the future, we are eager to extend our paradigm to more low-resource languages and devoted to breaking down the language barrier.

## Acknowledgments and Disclosure of Funding

We thank DataOcean AI for their manual annotation of the evaluation set.

## Appendix A Algorithm of Iterative Label Refinement

Algo. 1 illustrates the workflow of our proposed iterative label refinement.

Figure: Algorithm 1 Iterative Label Refinement

## Appendix B Detailed Analysis of GigaSpeech 2

Detailed analysis of the GigaSpeech 2 dataset is presented through the following visualizations.

### B.1 Domain Distribution of Manual Evaluation Sets

The domain distribution of the manual evaluation sets is shown in Fig. 2.

Figure: (a) Thai
Refer to caption: x2.png

### B.2 Duration Distribution of Training Sets

The utterance-level duration distribution of the training sets is illustrated in Fig. 3.

Figure: (a) Thai raw
Refer to caption: x5.png

## Appendix C Model Configurations

### C.1 Configuration of Zipformer

Two Zipformer-based models are used, following official configurations reported in icefall(^8^88[https://github.com/k2-fsa/icefall](https://github.com/k2-fsa/icefall)). In each Zipformer stack, the hidden dimensions of the first and last feedforward modules are 3/4 and 5/4 of the middle one, respectively. Ahead of the encoder, a convolution subsampling module with a stride of 2 reduces the frame rate to 50 Hz. The input consists of 80-channel FBank features extracted over windows of 25ms, strided by 10ms. The label decoder utilizes a stateless decoder []. 8 V100 32G GPUs are used for training.
Detailed configurations are provided in Table 6.

**Table 6: Configuration of Zipformer at two different scales**
|  | Zipformer-M | Zipformer-L |
| --- | --- | --- |
| Encoder |  |  |
| number of stacks | 6 |  |
| numbers of layers | 2,2,3,4,3,2 | 2,2,4,5,4,2 |
| downsampling factors | 1,2,4,8,4,2 |  |
| output downsampling factor | 2 |  |
| embedding dimensions | 192,256,384,512,384,256 | 192,256,512,768,512,256 |
| embedding unmasked dimensions | 192,192,256,256,256,192 | 192,192,256,320,256,192 |
| feedforward dimensions | 512,768,1024,1536,1024,768 | 512,768,1536,2048,1536,768 |
| convolution kernel sizes | 31,31,15,15,15,31 |  |
| attention heads | 4,4,4,8,4,4 |  |
| attention query dimension | 32 |  |
| attention value dimension | 12 |  |
| positional encoding embedding dimension | 48 |  |
| projected positional encoding dimension per head | 4 |  |
| Decoder |  |  |
| embedding dimensions | 512 |  |
| context size | 2 |  |
| Joiner |  |  |
| embedding dimensions | 512 |  |
| Criterion |  |  |
| use ctc head | false |  |
| use transducer head | true |  |
| pruned range | 5 |  |
| loss smoothing lm scale | 0.25 |  |
| loss smoothing am scale | 0.0 |  |
| simple loss scale | 0.5 |  |
| simple loss scale warmup steps | 2000 |  |
| Frontend |  |  |
| n fft | 512 |  |
| hop length | 256 |  |
| feature dimension | 80 |  |
| Training |  |  |
| use amp | true |  |
| max epochs | 30 |  |
| max duration per batch | 1000 |  |
| ref duration | 600 |  |
| seed | 42 |  |
| Optimization |  |  |
| optimizer | scaledadam |  |
| base learning rate | 0.045 |  |
| seed | 42 |  |
| Scheduler |  |  |
| scheduler | eden |  |
| lr batches | 7500 |  |
| lr epochs | 10000 / training set hours |  |
| warmup batches | 500 |  |
| warmup starting lr | 0.5 |  |
| SpecAugment |  |  |
| time warping factor | 80 |  |
| number of time masks | 10 |  |
| time mask maximum width | 100 |  |
| number of frequency masks | 2 |  |
| frequency mask width range | 0 - 27 |  |

### C.2 Configuration of Conformer

A Conformer-based model is developed adhering to the official configurations outlined in ESPNet(^9^99[https://github.com/espnet/espnet](https://github.com/espnet/espnet)). The model comprises an encoder that employs the Conformer architecture and a decoder that leverages the Transformer architecture. Moreover, the parameters for both the encoder and decoder components, the optimization process, the scheduling mechanism, and SpecAugment settings are carefully designed to ensure a comprehensive and efficient model setup. 4 A100 80G GPUs are used for training.
The specifics of these configurations are detailed in Table 7.

**Table 7: Configuration of Conformer at the large scale.**
| Conformer-L |  |  |  |
| --- | --- | --- | --- |
| Encoder |  | Criterion |  |
| attention head | 8 | ctc weight | 0.3 |
| numbers of blocks | 12 | label smoothing | 0.1 |
| linear unit | 2048 | length normalized | false |
| dropout rate | 0.1 | Frontend |  |
| positional dropout rate | 0.1 | n fft | 512 |
| attention dropout rate | 0.1 | hop length | 256 |
| input layer | conv2d | Training |  |
| normalize before | true | use amp | true |
| macaron style | true | gradient accumulation | 4 |
| relative position type | latest | max epochs | 20 |
| position encoding layer | rel_pos | Optimization |  |
| self-attention layer | rel_selfattn | optimizer | adam |
| activation type | swish | learning rate | 0.0025 |
| use cnn module | true | weight decay | 0.000001 |
| cnn module kernel | 31 | Scheduler |  |
| Decoder |  | scheduler | warmuplr |
| attention heads | 8 | warmup steps | 40000 |
| linear units | 2048 | SpecAugment |  |
| number of blocks | 6 | time warp window | 5 |
| dropout rate | 0.1 | frequency mask width range | 0 - 27 |
| positional dropout rate | 0.1 | number of frequency masks | 2 |
| self-attention dropout rate | 0.1 | time mask width ratio range | 0.0 - 0.05 |
| source attention dropout rate | 0.1 | number of time masks | 10 |

## Appendix D Ablation Study on Noisy Student Training

**Table 8: Ablation study of NST on GigaSpeech 2 Thai, evaluated across various evaluation sets: GigaSpeech 2 DEV and TEST, Common Voice 17.0 TEST, and FLEURS TEST.**
| NST<br>method | # Hours<br>(h) | CER |  |  |  |
| --- | --- | --- | --- | --- | --- |
| GigaSpeech 2<br>DEV   TEST | Common Voice<br>TEST | FLEURS<br>TEST |  |  |  |
| Sys. 1 (Tab. 2, iter 2 $\rightarrow$ iter 3) | 7219 | 10.47 | 12.38 | 4.63 | 10.96 |
| Sys. 2 (Tab. 2, iter 2 $\rightarrow$ iter3, without relabeling) | 7219 | 10.77+2.9% | 12.90+4.2% | 5.23+13.0% | 10.72-2.2% |
| Sys. 3 (Tab. 2, iter 2 $\rightarrow$ iter 3, larger augmentation) | 7219 | 10.65+1.7% | 12.81+3.5% | 5.36+15.8% | 10.86-0.9% |

Based on the ablation study of our proposed NST on the evaluation sets in Table 8, we can analyze the effects of different iterations and their impact on performance:
1) Relabeling the data during the transition from iteration 2 to 3 is crucial for improving performance (Sys.1 vs. Sys.2).
2) Larger augmentation applied in our NST process may have a negative impact on the performance (Sys.1 vs. Sys.3).
These findings suggest that careful consideration of the relabeling and augmentation strategies is crucial for optimizing the performance of the NST model across different evaluation sets and domains.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 1
presentation: 3
questions: For a revised submission it would be nice to see results on more languages .
soundness: 3
strengths: The language resource  will be of value to the ASR community. The methodology for constructing the data set is technically sound, though more advanced methods for e.g. forced alignment exist (only off-the-shelf tools were used here). Results demonstrate the value of the training data.
summary: The paper presents a 30k hr multidomain, multilingual ASR corpus for low-resource Southeast Asian languages, termed GigaSpeech 2.
It describes the assembly of the data set - data crawling, transcription with Whisper ASR, forced alignment with TorchAudio, text normalization, and a modified Noisy Student Training scheme to improve the labels. It presents results on test sets from Thai, Indonesian, and Vietnamese compared to open-source and commercial models.
weaknesses: The paper describes construction of a language resource and empirical ASR results demonstrating essentially that training on more relevant data helps lower WERs. Evaluation was on 3 languages only. Most importantly, the paper does not present a scientific hypothesis or any advancement in methodologies; it is purely a resource and evaluation paper and as such would be better place in a speech conference or language resources conference.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 4
questions: The iteration seems to still have a lot of room even after 2,3 iterations, how much would authors expect the result to improve through a few more iterations?
soundness: 4
strengths: The speech dataset from Thai, Indonesian, and Vietnamese are usually limited, this Gigaspeech2 would be beneficial to the speech community which target those languages.

While the target languages is limited, the experiments comparing with other commercial models, open-source models and datasets are convincing. They demonstrate the proposed pipeline and collected dataset achieves good quality (for example compared with YODAS dataset)
summary: This work presents Gigaspeech2, which contains 30k hours of automatically transcribed speech of Thai, Indonesian, and Vietnamese. 

Authors introduce a pipeline to download and process the dataset by

- collect dataset from Youtube with proper filtering
- force align with torchaudio
- transcribe with Whisper
- iteratively improve the transcription quality by noisy student training

Authors demonstrate through a few experiments that it achieved great ASR performance on target languages (i.e. Thai, Indonesian and Vietnamese) when compared with other commercial models and open-source models
weaknesses: Despite the usefulness of the dataset itself, the main weakness is that the general novelty is limited in this work. The pipeline itself does not have many components that are significantly different from the existing dataset collection procedures. For example, one of the main contribution authors claim is to apply Noisy Student Training iteratively to refine collected dataset, however, this is a quite standard approach and is already applied in one of the original Noisy student training paper (e.g. https://arxiv.org/pdf/1911.04252). It would be great that authors can provide more novelty how their refinement can be distinguished from the existing approach.

Another minor concern is this dataset only contains Thai, Indonesian, and Vietnamese, and their target audience might be too limited in the context of ICLR. It would be nice if authors can extend their approach to much more languages and demonstrate they are useful.

---

confidence: 5
contribution: 2
initial_rating: 6
presentation: 3
questions: - In Table 2, it is not clear if the improvements in WER are coming because of the increased number of refined hours or due to the added model parameters (it would have been best to keep the model parameters the same and just make use of additional refined data).
- In Table 4, can the authors please share some insights on the disparities between the performance of the 68.6M model and the 151.9M model? In some cases, the larger model is performing better, and the differences are quite significant (which ideally should not be the case).
soundness: 3
strengths: - Gigaspeech 2 test sets: a domain-rich human labelled benchmark enabling realistic ASR performance evaluation for Thai, Indonesian, and Vietnamese.
- A comprehensive speech dataset featuring 30K hours of raw audio across three low-resource Southeast Asian languages: Thai, Indonesian, and Vietnamese.
- High-performance ASR systems for the three languages, on par with commercially available solutions.
summary: This paper presents GigaSpeech 2, a large-scale, multi-domain, and multilingual ASR corpus targeting low-resource Southeast Asian languages, specifically Thai, Indonesian, and Vietnamese. It introduces a data collection, and refinement pipeline focused on automated transcription and label refinement using Noisy Student Training (NST), aiming to improve ASR systems for low-resource languages with minimal reliance on labelled data.
weaknesses: - Although GigaSpeech 2 spans multiple domains, the performance drops on out-of-domain test sets (Common Voice, FLEURS) in Indonesian and Vietnamese suggest limitations in cross-domain generalizability (Table 3).
- Even though the authors mention zero reliance on the labelled data, the approach is strictly dependent on having an initial good seed model which can be used for transcription and timestamp prediction; hence, this approach does not completely preclude the need for labelled data.
- The quality of the training data released as a part of this work depends on the robustness of the Whisper model

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 4
questions: -« To ensure no speaker overlap between the splits, we manually verify no speaker overlap between different channels and partition the data by allocating different YouTube channels to each subset. » => How can you be sure ?
Same speaker might speak in multiple channels (especially celebrities). Do you handle this specifically ?

-Corpus made available to the community ? Which license do you plan to have ?

-What is the impact of the seed (1st iteration) ASR system’s (Whipser3's) performance in your case ? I mean, it probably has different performance on the 3 languages targeted, how does this impact the final quality of the dataset and final ASR performance for a given language ?

-text normalization: i know the practice in ASR is to remove punct and case but since Whisper 3 is providing it, why not keeping it as a second tier of labels ? COuld be interesting for NLP researchers who need case/punct ! 
 
-« Moreover, the collected audio must be accompanied by a Creative Commons license. »  but which one, there are several CC licenses with different rules !
soundness: 3
strengths: -strong ASR resource for 3 languages: indo, vn and thai => strong impact for those 3 languages

-methodology (multiple iteration approach) is applicable to many other languages
summary: Authors introduce a dataset (Gigaspeech 2) made up of 30,000h of multilingual speech for low resource languages (such as Thai, Indonesian, Vietnamese) crawled on youtube and automatically transcribed with Whisper large model followed by data quality filtering iterative process.
A modified version of the Noisy Student Training (NST) method by Xie et al. (2020) is notably proposed to iteratively refine labels from collected (noisy) data.
ASR models trained using this corpus improve upon Whisper 3 large baseline (and other commercial systems) for Thai and have similar results for Vietnamese and Indonesian, demonstrating quality of dataset (for languages such as Thai, Indonesian, Vietnamese).
weaknesses: -this is overall a good recipe for collecting large and good quality speech resources, but it applies well-known techniques and eventually we get a resource with 3 languages covered only (extending language coverage is mentioned as future work)

-for readers not familar with NST method, section 3.2 is not easy to understand and would benefit from a bit more context on base NFT method before describing how it was modified by the authors

-the comparison to existing ASR systems is convincing for Thai but less convincing for Vietnamese and Indonesian (although it can be that commercial systems that outperform the Gigaspeech 2 based ASR system might have been trained on commonvoice and fleurs datasets)

-multidimensional filtering and its impact could have been more detailed (how much data is rejected for each specific filter, etc.)

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: Please answer the weaknesses. Based on your resources, I recommend answering the weakness in the following order: W5, W4, W6, W7, W3, W2, W1. I will reconsider the score if the authors promptly address W5 and W4.
soundness: 2
strengths: - Dataset for speech is very important, and the speech community is thriving to see a new massive dataset.
- The proposed data creation flow is clear, and code is provided for reproducibility.
- The provided leaderboard gives the community a better sense of data difficulty. The reported WERs make me more convinced that the test set is realistic and applicable.
summary: This paper presents GigaSpeech 2, an evolving large-scale (30000 hours) multilingual ASR corpus for Thai, Indonesian, and Vietnamese. The dataset is constructed in the following steps: crawled video/audio from YouTube, automatically transcribed using Whisper, forced-aligned with Torchaudio, policy-based filtering, and finally label-refinement with noisy-student-training (NST). A modified NST approach is used to obtain higher-quality labels through iterative refinement, where an ASR model is trained on a subset of noisy data, generates labels for a larger subset, and that generated data is used to train a larger ASR model. Results of some representative ASR models are also reported. It is shown that the model trained with iteratively refined datasets has the SOTA performance on the test datasets.
weaknesses: ## Writing
- W1: Please use the citation format correctly. It should be \citep and \citet, depending on the usage. It seems like the current version is a direct transfer from NeurIPS, which uses \cite.
- W2: The organization of the paper is a little messy. There are too many bold texts with multiple hierarchies (for example Sec 3.1). At the same time, there are also numbered points in Sec 4. When finish reading the paper, I already get lost on what the contributions of the paper are. I suggest making the paper more organized. One simple improvement is to reference the section numbers in your contribution in the Introduction.
 - W3: The discussion of related works is a little weak. You should mention other works that also automatically create audio datasets. For example, [1],[2], [3]. There should be at least one subsection in related works that discusses this topic.

## Novelty
 - W4: While I appreciate the release of a new dataset, I don't find the dataset curation approach much novel. In particular, everything besides the NST seems very trivial and purely engineering work. The proposed NST is similar to the approach taken by [3]. I would like the authors to provide a detailed explanation of how their approach differs from previous approaches.

## Evaluation
 - W5: $\textbf{This is the biggest issue: }$ How good exactly is the training dataset? There are only evaluation results of fine-tuned models on the test dataset, but to claim a usable dataset, you $\textbf{must}$ report the accuracy of the dataset (at least on a random subset). Currently, such disclosure on the training set is missing.
- W6: There have been concerns raised by previous works on training with synthetic datasets. For example, I would like to quote Whisper's paper [4]: "Recent research has shown that training on datasets of mixed human and machine-generated data can significantly impair the performance of translation systems (Ghorbani et al., 2021)." I would like the authors to provide a more detailed analysis of whether their NST approach would solve that problem. It will also be a significant finding if self-training will completely solve that problem, which means synthetical datasets could be used in the future.
- W7: If I understand correctly, Table 3 is comparing pre-trained models with fine-tuned GigaSpeech2 model. This seems a little unfair. If possible, I would like to see fine-tuned public models as well.

## Reference

[1] Sun, Luoyi, et al. "Auto-ACD: A large-scale dataset for audio-language representation learning." ACM Multimedia 2024. 2024.

[2] Liu, Houjun, et al. "Automation of language sample analysis." Journal of Speech, Language, and Hearing Research 66.7 (2023): 2421-2433.

[3] Yeroyan, Ara, and Nikolay Karpov. "Enabling ASR for Low-Resource Languages: A Comprehensive Dataset Creation Approach." arXiv preprint arXiv:2406.01446 (2024).

[4] Radford, Alec, et al. "Robust speech recognition via large-scale weak supervision." International conference on machine learning. PMLR, 2023.