## 5 Validation of the Basque Parliament Database through ASR  
In this section, by using the training set to estimate the acoustic models of an state-of-the-art ASR system and by checking its performance on the test set (using part of it for tuning the system and part of it for properly measuring performance), we aim to evaluate the consistency of the Basque Parliament database and, at the same time, to provide a baseline for other authors who may use the database in the future. The ASR framework consists of an acoustic front end based on a pre-trained Wav2 Vec 2.0 speech encoder [45] which produces a sequence of frame-level acoustic representations (speech embeddings), followed by a classification backend consisting of a neural network trained with Connectionist Temporal Classification (CTC) [46] which outputs a vector of grapheme posteriors for each input embedding. Finally, maybe constrained by the phonological and syntactic restrictions introduced by lexical and language models, a search is performed on the sequence of posteriors to output the sequence of graphemes (including blanks) that maximizes the joint acoustic and syntactic likelihood. The framework on which this work is based is available at [[https://huggingface.co/docs/transformers/model_doc/wav2](https://huggingface.co/docs/transformers/model_doc/wav2) vec2]([https://huggingface.co/docs/transformers/model_doc/wav2](https://huggingface.co/docs/transformers/model_doc/wav2) vec2) (accessed on 24 February 2024). ASR experiments have been carried out using the train-clean dataset to finetune the classification backend and three different approaches to obtain the output sequence of words from the sequence of frame-level grapheme posteriors. In the first approach, no language model is used so the search for the optimal sequence of words takes into account just the grapheme posteriors. In the second approach, an out-of-domain vocabulary (including 2,159,919 words) and an out-of-domain 3-gram language model (including 33.7 million n-grams) are used, both extracted from Wikipedia pages in Spanish and Basque (downloaded on 1 March 2023), the Spanish part being about 10 times larger than the Basque part. In the third approach, an in-domain vocabulary (including 174,168 words) and an in-domain 4-gram language model (including 22.1 million n-grams) are used, both extracted from transcriptions of the train set. The dev set has been used to tune the hyperparameters of the ASR framework and then the optimal hyperparameter values have been used to obtain the output sequences for both the dev and eval sets. ASR performance is given in terms of both WER and Character Error Rate (CER). CER is defined the same way as WER but for individual characters (graphemes, including blanks) instead of full words. CER and WER performance figures on the dev and eval sets, disaggregated per language, are shown in Table 6. Figure 4 shows the global CER and WER performance on the dev and eval sets for different language models and Figure 5 shows CER and WER performance, disaggregated per language, on the dev and eval sets when using the Basque Parliament language model, which is the one yielding the best results. \begin{table}
\begin{tabular}{l r r r r r r r r} \hline  & \multicolumn{4}{c}{**Duration (h)**} & \multicolumn{4}{c}{**\#Segments**} \\ \hline
**Set** & **Total** & **es** & **eu** & **bi** & **Total** & **es** & **eu** & **bi** \\ \hline train & 1445.1 & 1018.6 & 409.5 & 17.0 & 749,945 & 524,942 & 216,201 & 8802 \\ train-clean & 1315.5 & 937.7 & 363.6 & 14.2 & 661,871 & 469,937 & 184,950 & 6984 \\ development & 7.6 & 4.7 & 2.6 & 0.3 & 4095 & 2567 & 1397 & 131 \\ evaluation & 9.6 & 6.4 & 2.8 & 0.4 & 5152 & 3450 & 1521 & 181 \\ \hline \end{tabular}
\end{table} 
Table 5: Duration (in hours) and number of segments in the train, train-clean, dev and eval sets of the Basque Parliament database, disaggregated per language (es: Spanish, eu: Basque, bi: Bilingual). Figure 4: Global CER and WER performance on the dev and eval sets for different language models.