# Under-Representated Speech Dataset from Open Data: Case Study on the Romanian Language
## 3 Methodology
As, most of the time, the exact age at the time of recording cannot be identified, the annotators are instructed to include the person in the most appropriate age group (for example, 14-19 is typically high school, while 19-29 corresponds to faculty students, PhD students, or young working individuals). The age and gender annotations are given for each speaking individual. At this stage, as the annotators process the files, recordings containing large amounts of non-Romanian speech are rejected, and suitable alternatives are downloaded. If a recording includes small amounts of non-Romanian speech, only information about Romanian speakers is extracted. Later at the transcription stage, non-Romanian speech is ignored. The information about the speech being read or spontaneous is either inferred from the video (the speaker is seen reading from some written material or is clearly being interviewed on the street or in some other place, thus being spontaneous speech) or by common sense (very young kindergarten children cannot read; TV news presenters usually read from a teleprompter device). The type of speech annotation is available at the level of the entire video file. Annotators are instructed to avoid clips with mixed speech types. For assessing the quality of speech, the annotators use the Mean Opinion Score (MOS) method [23]. Speech is rated on a scale from 1 to 5, where 1 is considered bad speech, with a high level of distortion, and 5 is excellent speech, with an imperceptible level of distortion. Annotators are instructed to discard any recordings that they would rate below a MOS value of 3. This is considered fair speech quality but with a level of distortion that is perceptible and slightly annoying. Thus, recordings with levels 1 or 2 are not present in the dataset. Finding spontaneous MOS 5 recordings is difficult due to the video blogging trend of adding soundtrack to the recording. The actual transcription of the audio tracks is performed manually by the human annotators using the Subtitle Edit application, and transcriptions are saved in CSV format files with the same ID as the transcribed video file. In order to ensure the high quality of the transcripts, no automated technologies are used at any point of the work. In the transcription process, manual subtitle segmentation is carried out at the sentence level, \begin{table}
\begin{tabular}{c c} \hline \hline
**Keywords** & **Target Group** \\ \hline elevi te invata (“the students teach you”) & Young people \\ probleme adolescent (“problems teenagers”) & Generic \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Proposed search words and phrases. using the rendering of the waveform that Subtitle Edit tool makes available as an indication for the segmentation locations (while the annotators are free to segment at any point they considered relevant). From the resulting segments, 5% of them are transcribed by a second annotator and the corresponding video files are re-annotated. No discrepancies are found in the annotations. We consider this to be due to the clear annotation instructions as well as the experience of the annotators who previously worked on creating other audio corpora.