The Common Voice project is a response to the current state of affairs in speech technology, in which training data is either prohibitively expensive or unavailable for most languages BIBREF0. We believe that speech technology (like all technology) should be decentralized and open, and the Common Voice project achieves this goal via a mix of community building, open source tooling, and a permissive licensing scheme. The corpus is designed to organically scale to new languages as community members use the provided tools to translate the interface, submit text sentences, and finally record and validate voices in their new language . The project was started with an initial focus on English in July 2017 and then in June 2018 was made available for any language.The remainder of the paper is laid out as follows: In Section (SECREF2) we motivate Common Voice and review previous multilingual corpora. Next, in Section (SECREF3) we describe the recording and validation process used to create the corpus. Next, in Section (SECREF4) we describe the current contents of Common Voice, and lastly in Section (SECREF5) we show multilingual Automatic Speech Recognition experiments using the corpus.Some notable multilingual speech corpora include VoxForge BIBREF1, Babel BIBREF2, and M-AILABS BIBREF3. Even though the Babel corpus contains high-quality data from 22 minority languages, it is not released under an open license. VoxForge is most similar to Common Voice in that it is community-driven, multilingual (17 languages), and released under an open license (GNU General Public License). However, the VoxForge does not have a sustainable data collection pipeline compared to Common Voice, and there is no data validation step in place. M-AILABS data contains 9 language varieties with a modified BSD 3-Clause License, however there is no community-driven aspect. Common Voice is a sustainable, open alternative to these projects which allows for collection of minority and majority languages alike.The data presented in this paper was collected and validated via Mozilla's Common Voice initiative. Using either the Common Voice website or iPhone app, contributors record their voice by reading sentences displayed on the screen (see Figure (FIGREF5)). The recordings are later verified by other contributors using a simple voting system. Shown in Figure (FIGREF6), this validation interface has contributors mark $<$audio,transcript$>$ pairs as being either correct (up-vote) or incorrect (down-vote).A maximum of three contributors will listen to any audio clip. If an $<$audio,transcript$>$ pair first receives two up-votes, then the clip is marked as valid. If instead the clip first receives two down-votes, then it is marked as invalid. A contributor may switch between recording and validation as they wish.Only clips marked as valid are included in the official training, development, and testing sets for each language. Clips which did not recieve enough votes to be validated or invalidated by the time of release are released as “other”. The train, test, and development sets are bucketed such that any given speaker may appear in only one. This ensures that contributors seen at train time are not seen at test time, which would skew results. Additionally, repetitions of text sentences are removed from the train, test, and development sets of the corpus.The number of clips is divided among the three datasets according to statistical power analyses. Given the total number of validated clips in a language, the number of clips in the test set is equal to the number needed to acheive a confidence level of 99% with a margin of error of 1% relative to the number of clips in the training set. The same is true of the development set.The audio clips are released as mono-channel, 16bit MPEG-3 files with a 48kHz sampling rate. The choice to collect and release MPEG-3 as opposed to a lossless audio format (e.g. WAV) is largely due to the web-based nature of the Common Voice collection platform. MPEG-3 is the most universally supported audio format for the web, and as such is the most reliable recording/playback technique for various devices and browsers. Also practically speaking, the audio quality is appropriate for speech applications.The data presented in Table (TABREF12) shows the currently available data. Each of the released languages is available for individual download as a compressed directory from the Mozilla Common Voice website. The directory contains six files with Tab-Separated Values (i.e. TSV files), and a single clips subdirectory which contains all of the audio data. Each of the six TSV files represents a different segment of the voice data, with all six having the following column headers: [client_id, path, sentence, up_votes, down_votes, age, gender, accent]. The first three columns refer to an anonymized ID for the speaker, the location of the audio file, and the text that was read. The next two columns contain information on how listeners judged the $<$audio,transcript$>$ pair. The last three columns represent demographic data which was optionally self-reported by the speaker of the audio.In order to add a new language to the Common Voice project, two steps must be completed. First, the web-app user interface must be translated into the target language. For example, the text shown in Figure (FIGREF7) must be translated. Secondly, text prompts must be gathered in order to be read aloud. These texts are not translated, but gathered from scratch for each language – translation would be very slow and not scaleable.Translation of the interface is managed through the Pontoon platform. Pontoon allows community members to propose translations, and then the moderators for that language approve or decline the proposals. At the time of writing this paper there are 610 text strings used in the Common Voice interface, where each string can range in length from an isolated word to a paragraph of text.Collecting text for reading aloud is the second step of adding a new language to Common Voice. For languages with more than 500,000 Wikipedia articles, text sentences are extracted from Wikipedia using community provided rule-sets per language. These sentences make up the initial text prompts for the languages in question.Any language community can gather additional sentences through the Sentence Collector taking advantage of automatic validation mechanisms such as checks for sentence length, foreign alphabets, and numbers. Every sentence submitted through the Sentence Collector needs to be approved by two out of three reviewers, leading to a weekly export of new sentences into the Common Voice database. Once the website is translated and at least 5,000 sentences have been added, the language is enabled for voice recordings.The following experiments demonstrate the potential to use the Common Voice corpus for multilingual speech research. These results represent work on an internal version of Common Voice from February 2019. The current corpus contains more languages and more data per language.These experiments use an End-to-End Transfer Learning approach which bypasses the need for linguistic resources or domain expertise BIBREF4. Certain layers are copied from a pre-trained English source model, new layers are initialized for a target language, the old and new layers are stitched together, and all layers are fine-tuned via gradient descent.We made dataset splits (c.f. Table (TABREF19)) such that one speaker's recordings are only present in one data split. This allows us to make a fair evaluation of speaker generalization, but as a result some training sets have very few speakers, making this an even more challenging scenario. The splits per language were made as close as possible to 80% train, 10% development, and 10% test.Results from this dataset are interesting because the text and audio are challenging, the range of languages is wider than any openly available speech corpus, and the amount of data per language ranges from very small (less than 1,000 clips for Slovenian) to relatively large (over 65,000 clips for German).All reported results were obtained with Mozilla's DeepSpeech v0.3.0 — an open-source implementation of a variation of Baidu's first DeepSpeech paper BIBREF5. This architecture is an end-to-end Automatic Speech Recognition (ASR) model trained via stochastic gradient descent with a Connectionist Temporal Classification (CTC) loss function BIBREF6. The model is six layers deep: three fully connected layers followed by a unidirectional LSTM layer followed by two more fully connected layers (c.f. Figure (FIGREF21)). All hidden layers have a dimensionality of 2,048 and a clipped ReLU activation. The output layer has as many dimensions as characters in the alphabet of the target language (including any desired punctuation as well as the blank symbol used for CTC). The input layer accepts a vector of 19 spliced frames (9 past frames + 1 present frame + 9 future frames) with 26 MFCC features each (i.e. a single, 494-dimensional vector).All models were trained with the following hyperparameters on a single GPU. We use a batch-size of 24 for train and 48 for development, a dropout rate of 20%, and a learning rate of $0.0001$ with the ADAM optimizer. The new, target-language layers were initialized via Xavier initialization BIBREF7. After every epoch of backpropagation over the training set, the loss over the entire development set is calculated. This development loss is used to trigger early stopping. Early stopping is triggered when the loss on the held-out development set either (1) increases over a window of five sequential epochs, or (2) the most recent loss over the development set has not improved in a window of five epochs more than a mean loss threshold of $0.5$ and the window of losses shows a standard deviation of less than $0.5$.The results from all experiments can be found in Table (TABREF23). Each cell in the table contains the Character Error Rate of the resulting model on the test set, defined as the Levenshtein distance BIBREF8 of the characters between the ground-truth transcript and the decoding result. The results in Table (TABREF23) show how the number of layers transfered (columns) influence the performance on individual target languages (rows). Shaded cells indicate relative performance per language, where a darker cell represents a more accurate model. From this table we observe a trend in which four layers copied from pre-trained English DeepSpeech result in the best final model. This trend becomes more obvious in Figure (FIGREF24), where we average the improvement over all languages relative to the number of layers transfered from a source model.We have presented Common Voice: a crowd-sourced, multilingual speech corpus which can scale to any language via community effort. All of the speech data is released under a Creative Commons CC0 license, making Common Voice the largest public domain corpus designed for Automatic Speech Recognition. In Section (SECREF3) we described the recording and validation process used to create the corpus. In Section (SECREF4) we presented the current contents of Common Voice, and lastly in Section (SECREF5) we show multilingual Automatic Speech Recognition experiments using the corpus. There are currently 38 language communities collecting data via Common Voice, and we welcome more languages and more volunteers.Common Voice is a living project, and would not be possible without the thousands of hours given by volunteers. We thank all volunteers for their time, and especially the minority language activists who translate, find new texts, and organize Common Voice donation events. We thank George Roter, Gheorghe Railean, Rubén Martín, and Jane Scowcroft for their work on Common Voice, and all members of the Common Voice team, past and present.This material is based upon work when Josh Meyer was supported by the National Science Foundation under Grant No. (DGE-1746060). Opinions, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of the NSF.

Q: What crowdsourcing platform is used for data collection and data validation?

Q: How is validation of the data performed?

Q: Is audio data per language balanced in dataset?