SlowGuess/ACL_2020_OCR huggingface dataset:
D Corpus Representation: It is difficult to draw conclusions about 'average behavior' across languages. Some language families are better represented in the corpus than others, with more languages, more Bible readings per language, more hours of speech per reading, or more examples of a given phoneme of interest. Additionally, the recordings by language are largely single-speaker (and predominantly male). This means that we can often draw conclusions only about a particular speaker's idiolect, rather than the population of speakers of the language. Metadata giving the exact number of different speakers per recording do not exist.
4 Phonetic Case Studies
We present two case studies to illustrate the utility of our resource for exploration of cross-linguistic typology. Phoneticians have posited several typological principles that may structure phonetic systems. Though previous research has provided some indication as to the direction and magnitude of expected effects, many instances of the principles have not yet been explored at scale. Our case studies investigate how well they account for cross-linguistic variation and systematicity for our phonetic measures from vowels and sibilants. Below we present the data filtering methods for our case studies, followed by an introduction to and evaluation of phonetic dispersion and uniformity.
4.1 Data filtering
For quality, we use only the tokens extracted using high-resource pronunciations (Epitran and WikiPron) and only in languages with mean
See our corpus website for exact numbers of utterances and our phonetic measures per each language.
MCD lower than 8.0. $^{9}$  Furthermore, we only use those utterances with MCD lower than 6.0. The vowel analyses focus on F1 and F2 in ERB taken at the vowel midpoint (Zwicker and Terhardt, 1980; Glasberg and Moore, 1990). $^{10}$  The sibilant analyses focus on mid-frequency peak of /s/ and /z/, also in ERB. Vowel tokens with F1 or F2 measures beyond two standard deviations from the label- and reading-specific mean were excluded, as were tokens for which Praat failed to find a measurable F1 or F2, or whose duration exceeded 300 ms. Sibilant tokens with mid-frequency peak or duration measures beyond two standard deviations from the label- and reading-specific mean were also excluded. When comparing realizations of two labels such as /i/–/u/ or /s/–/z/, we excluded readings that did not contain at least 50 tokens of each label. We show data representation with different filtering methods in Appendix D.
After filtering, the vowel analyses included 48 readings covering 38 languages and 11 language families. The distribution of language families was 21 Indo-European, 11 Austronesian, 3 Creole/Pidgin, 3 Turkic, 2 Afro-Asiatic, 2 Tai-Kadai, 2 Uto-Aztecan, 1 Austro-Asiatic, 1 Dravidian, 1 Hmong-Mien, and 1 Uralic. Approximately 8.2 million vowel tokens remained, with a minimum of  $\approx 31,000$  vowel tokens per reading. The sibilant analysis included 22 readings covering 18 languages and 6 language families. The distribution of language families was 10 Indo-European, 6 Austronesian, 3 Turkic, 1 Afro-Asiatic, 1 Austro-Asiatic, and 1 Creole/Pidgin. The decrease in total number of readings relative to the vowel analysis primarily reflects the infrequency of /z/ cross-linguistically. Approximately 385,000 /s/ and 83,000 /z/ tokens remained, with a minimum of  $\approx 5,200$  tokens per reading.
4.2 Phonetic dispersion
Phonetic dispersion refers to the principle that contrasting speech sounds should be distinct from one another in phonetic space (Martinet, 1955; Jakobson, 1968; Flemming, 1995, 2004). Most studies investigating this principle have focused on its va
lidity within vowel systems, as we do here. While languages tend to have seemingly well-dispersed vowel inventories such as  $\{/i/, /a/, /u/\}$  (Joos, 1948; Stevens and Keyser, 2010), the actual phonetic realization of each vowel can vary substantially (Lindau and Wood, 1977; Disner, 1983). One prediction of dispersion is that the number of vowel categories in a language should be inversely related to the degree of per-category acoustic variation (Lindblom, 1986). Subsequent findings have cast doubt on this (Livijn, 2000; Recasens and Espinosa, 2009; Vaux and Samuels, 2015), but these studies have been limited by the number and diversity of languages investigated.
To investigate this, we measured the correlation between the number of vowel categories in a language and the degree of per-category variation, as measured by the joint entropy of (F1, F2) conditioned on the vowel category. We model  $p(\mathrm{F1}, \mathrm{F2} \mid V)$  using a bivariate Gaussian for each vowel type  $v$ . We can then compute the joint conditional entropy under this model as  $\mathrm{H(F1,F2|V)} = \sum_v p(v) \mathrm{H(F1,F2|V = v)} = \sum_v p(v)\frac{1}{2}\ln \det(2\pi e\Sigma_v)$ , where  $\Sigma_v$  is the covariance matrix for the model of vowel  $v$ .
Vowel inventory sizes per reading ranged from 4 to 20 vowels, with a median of 8. Both Spearman and Pearson correlations between entropy estimate and vowel inventory size across analyzed languages were small and not significant (Spearman  $\rho = 0.11$ ,  $p = 0.44$ ; Pearson  $r = 0.11$ ,  $p = 0.46$ ), corroborating previous accounts of the relationship described in Livijn (2000) and Vaux and Samuels (2015) with a larger number of languages—a larger vowel inventory does not necessarily imply more precision in vowel category production.[11]
4.3 Phonetic uniformity
Previous work suggests that F1 is fairly uniform with respect to phonological height. Within a single language, the mean F1s of /e/ and /o/—which share a height—have been found to be correlated across speakers (Yorkshire English: Watt, 2000; French: Menard et al., 2008; Brazilian Portuguese: Oushiro, 2019; Dutch, English, French, Japanese, Portuguese, Spanish: Schwartz and Menard, 2019). Though it is physically possible for these vowels
<sup>11</sup> Since differential entropy is sensitive to parameterization, we also measured this correlation using formants in hertz, instead of in ERB, as ERB is on a logarithmic scale. This change did not the influence the pattern of results (Spearman  $\rho = 0.12$ ,  $p = 0.41$ ; Pearson  $r = 0.13$ ,  $p = 0.39$ ).
$^{9}$ In the high-MCD languages, even the low-MCD utterances seem to be untrustworthy.  
 $^{10}$ The Equivalent Rectangular Bandwidth (ERB) scale is a psychoacoustic scale that better approximates human perception, which may serve as auditory feedback for the phonetic realization (Fletcher, 1923; Nearey, 1977; Zwicker and Terhardt, 1980; Glasberg and Moore, 1990). The precise equation comes from Glasberg and Moore (1990, Eq. 4).
(a) F1 of /i/-/u/ in ERB
(b) Mid-frequency peak of  $/s/-/z/$  in ERB
Figure 4: Correlations of mean F1 (ERB) between /i/ and /u/ and of mean mid-frequency peak (ERB) between /s/ and /z/. The paired segments share a relevant phonological feature specification that is approximated by the acoustic-phonetic measurement: vowel height by F1 and sibilant place by mid-frequency peak. Each reading is represented by an ellipsoid, centered on the paired means and shaped by  $\frac{1}{10}$  of their respective standard deviations. The solid line reflects the best-fit linear regression line with standard error in gray shading; the dashed line shows the line of equality. Marginal histograms show the range of variation in the segment-specific means.
to differ in F1 realization, the correlations indicate a strong tendency for languages and individual speakers to yoke these two representations together.
Systematicity in the realization of sibilant place of articulation has also been observed across speakers of American English and Czech (Chodroff, 2017). Phonetic correlates of sibilant place strongly covary between /s/ and /z/, which share a [+anterior] place of articulation and are produced the alveolar ridge, and between /ʃ/ and /ʒ/, which share a [-anterior] place of articulation and are produced behind the alveolar ridge.
A principle of uniformity may account for these above findings. Uniformity here refers to a principle in which a distinctive phonological feature should have a consistent phonetic realization, within a language or speaker, across different segments with that feature (Keating, 2003; Chodroff et al., 2019). Similar principles posited in the literature include Maximal Use of Available Controls, in which a control refers to an integrated perceptual and motor phonetic target (Ménard et al., 2008), as well as a principle of gestural economy (Maddieson, 1995). Phonetic realization refers to the mapping from the abstract distinctive feature to an abstract phonetic target. We approximate this phonetic target via an acoustic-phonetic measurement, but we emphasize that the acoustic measurement is not necessarily a direct reflection of an underlying phonetic target (which could be an articulatory gesture, auditory goal, or perceptuo-motor repre
sentation of the sound). We make the simplifying assumption that the acoustic-phonetic formants (F1, F2) directly correspond to phonetic targets linked to the vowel features of height and backness.
More precisely, uniformity of a phonetic measure with respect to a phonological feature means that any two segments sharing that feature will tend to have approximately equal measurements in a given language, even when that value varies across languages. We can observe whether this is true by plotting the measures of the two segments against each other by language (e.g., Figure 4).
Vowels. As shown in Figure 4 and Table 3, the strongest correlations in mean F1 frequently reflected uniformity of height (e.g., high vowels /i/ -u:/  $r = 0.79$ ,  $p < 0.001$ , mid vowels /e/ -o:/  $r = 0.62$ ,  $p < 0.01$ ).<sup>12</sup> Nevertheless, some vowel pairs that differed in height were also moderately correlated in mean F1 (e.g., /o/ -a:/  $r = 0.66$ ,  $p < 0.001$ ). Correlations of mean F1 were overall moderate in strength, regardless of the vowels' phonological specifications.
Correlations of mean F2 were also strongest among vowels with a uniform backness specification (e.g., back vowels /u/–/o:/  $r = 0.69$ ,  $p < 0.001$ ; front vowels /i/–/ε:/  $r = 0.69$ ,  $p < 0.05$ ; Table 4). The correlation between front tense vowels /i/ and /e/ was significant and in the ex
${}^{12}p$  -values are corrected for multiple comparisons using the Benjamini-Hochberg correction and a false discovery rate of 0.25 (Benjamini and Hochberg, 1995).
pected direction, but also slightly weaker than the homologous back vowel pair ( $r = 0.41$ ,  $p < 0.05$ ). Vowels differing in backness frequently had negative correlations, which could reflect influences of category crowding or language-/speaker-specific differences in peripheralization. We leave further exploration of those relationships to future study.
The moderate to strong F1 correlations among vowels with a shared height specification are consistent with expectations based on previous studies, and also with predictions of uniformity. Similarly, we find an expected correlation of F2 means for vowels with a shared height specification. The correlations of vowel pairs that were predicted to have significant correlations, but did not, tended to have small sample sizes (< 14 readings).
Nevertheless, the correlations are not perfect; nor are the patterns. For instance, the back vowel correlations of F2 are stronger than the front vowel correlations. While speculative, the apparent peripheralization of /i/ (as revealed in the negative F2 correlations) could have weakened the expected uniformity relation of /i/ with other front vowels. Future research should take into account additional influences of the vowel inventory composition, as well as articulatory or auditory factors for a more complete understanding of the structural forces in the phonetic realization of vowels.
Sibilants. The mean mid-frequency peak values for  $/s/$  and  $/z/$  each varied substantially across readings, and were also strongly correlated with one another  $(r = 0.87, p < 0.001$ ; Figure 4). This finding suggests a further influence of uniformity on the realization of place for  $/s/$  and  $/z/$ , and the magnitude is comparable to previous correlations observed across American English and Czech speakers, in which  $r$  was  $\approx 0.90$  (Chodroff, 2017).
5 Directions for Future Work
We hope our corpus may serve as a touchstone for further improvements in phonetic typology research and methodology. Here we suggest potential steps forward for known areas (§3.4) where this corpus could be improved:
A Sentence alignments were generated using Unitran, and could be improved with higher-quality G2P and verse-level text segmentation to standardize utterances across languages.
B Consistent and comparable phoneme labels are the ultimate goal. Concurrent work on universal phone recognition (Li et al., 2020) addresses this issue through a universal phone inventory constrained by language-specific PHOIBLE inventories (Moran and McCloy, 2019). However, free-decoding phones from speech alone is challenging. One exciting possibility is to use the orthography and audio jointly to guide semi-supervised learning of per-language pronunciation lexicons (Lu et al., 2013; Zhang et al., 2017).
C Reliable quality assessment for current methods remains an outstanding research question for many languages. For covered languages, using a universal label set to map additional high quality lexicons (e.g., hand-annotated lexicons) to the same label space as ours would enable direct label and alignment assessment through precision, recall, and PER.
D Curating additional resources beyond this corpus would improve coverage and balance, such as contributing additional Epitran modules. Additional readings exist for many languages on the original bible.is site and elsewhere. Annotations with speaker information are not available, but improved unsupervised speaker clustering may also support better analysis.
VoxClamantis v1.0 is the first large-scale corpus for phonetic typology, with extracted phonetic features for 635 typologically diverse languages. We present two case studies illustrating both the research potential and limitations of this corpus for investigation of phonetic typology at a large scale. We discuss several caveats for the use of this corpus and areas for substantial improvement. Nonetheless, we hope that directly releasing our alignments and token-level features enables greater research accessibility in this area. We hope this corpus will motivate and enable further developments in both phonetic typology and methodology for working with cross-linguistic speech corpora.
The authors gratefully acknowledge Colin Wilson for his guidance and discussion on the topic, Florian Metze for resources, and Carlos Aguirre for helpful feedback.
13The magnitude of this correlation did not change when using hertz  $(r = 0.86, p < 0.001)$ .
Gopala Krishna Anumanchipalli, Kishore Prahallad, and Alan W. Black. 2011. Festvox: Tools for creation and analyses of large speech corpora. In Workshop on Very Large Scale Phonetics Research, UPenn, Philadelphia.
Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common Voice: A massively-multilingual speech corpus. In Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020).
Roy Becker-Kristal. 2010. Acoustic typology of vowel inventories and Dispersion Theory: Insights from a large cross-linguistic corpus. Ph.D. thesis, University of California, Los Angeles.
Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1):289-300.
Alan W. Black. 2006. CLUSTERGEN: A statistical parametric synthesizer using trajectory modeling. In Proceedings of INTERSPEECH.
Alan W. Black. 2019. CMU Wilderness Multilingual Speech Dataset. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5971-5975, Brighton, UK. IEEE.
Oliver Blacklock. 2004. Characteristics of Variation in Production of Normal and Disordered Fricatives, Using Reduced-Variance Spectral Methods. Ph.D. thesis, University of Southampton.
Paul Boersma and David Weenink. 2019. Praat: Doing phonetics by computer [computer program]. version 6.0.45.
Taehong Cho and Peter Ladefoged. 1999. Variation and universals in VOT: Evidence from 18 languages. Journal of Phonetics, 27(2):207-229.
Eleanor Chodroff. 2017. Structured Variation in Obstruent Production and Perception. Ph.D. thesis, Johns Hopkins University.
Eleanor Chodroff, Alessandra Golden, and Colin Wilson. 2019. Covariation of stop voice onset time across languages: Evidence for a universal constraint on phonetic realization. The Journal of the Acoustical Society of America, 145(1):EL109-EL115.
Sandra Ferrari Disner. 1983. Vowel Quality: The Relation between Universal and Language-specific Factors. Ph.D. thesis, UCLA.
Gary F. Simons Eberhard, David M. and Charles D. Fennig, editors. 2020. *Ethnologue: Languages of the world*, 23 edition. SIL international. Online version: http://www.ethnologue.com.
Arvo Eek and Einar Meister. 1994. Acoustics and perception of Estonian vowel types. *Phonetic Experimental Research*, XVIII:146-158.
Olle Engstrand and Una Cunningham-Andersson. 1988. Iris - a data base for cross-linguistic phonetic research.
Edward S. Flemming. 1995. Auditory Representations in Phonology. Ph.D. thesis, UCLA.
Edward S. Flemming. 2004. Contrast and perceptual distinctiveness. In Bruce Hayes, R. Kirchner, and Donca Steriade, editors, The Phonetic Bases of Phonological Markedness, 1968, pages 232-276. University Press, Cambridge, MA.
Harvey Fletcher. 1923. Physical measurements of audition and their bearing on the theory of hearing. Journal of the Franklin Institute, 196(3):289-326.
Karen Forrest, Gary Weisman, Paul Milenkovic, and Ronald N. Dougall. 1988. Statistical analysis of word-initial voiceless obstruents: Preliminary data. The Journal of the Acoustical Society of America, 84(1):115-123.
Brian R. Glasberg and Brian C.J. Moore. 1990. Derivation of auditory filter shapes from notched-noise data. Hearing Research, 47(1-2):103-138.
Matthew Gordon and Timo Roettger. 2017. Acoustic correlates of word stress: A cross-linguistic survey. Linguistics Vanguard, 3(1).
Kyle Gorman, Lucas F.E. Ashby, Aaron Goyzueta, Arya D. McCarthy, Shijie Wu, and Daniel You. 2020. The SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion. In Proceedings of the SIGMORPHON Workshop.
Mary Harper. 2011. The IARPA Babel multilingual speech database. Accessed: 2020-05-01.
Arthur S. House and Kenneth N. Stevens. 1956. Analog studies of the nasalization of vowels. The Journal of Speech and Hearing Disorders, 21(2):218-232.
Roman Jakobson. 1968. Child Language, Aphasia and Phonological Universals. Mouton Publishers.
Allard Jongman, Ratree Wayland, and Serena Wong. 2000. Acoustic characteristics of English fricatives. The Journal of the Acoustical Society of America, 108(3):1252-1263.
Martin Joos. 1948. Acoustic phonetics. Language, 24(2):5-136.
Patricia A. Keating. 2003. Phonetic and other influences on voicing contrasts. In Proceedings of the 15th International Congress of Phonetic Sciences, pages 20-23, Barcelona, Spain.
Laura Koenig, Christine H. Shadle, Jonathan L. Preston, and Christine R. Mooshammer. 2013. Toward improved spectral measures of /s:/ Results from adolescents. Journal of Speech, Language, and Hearing Research, 56(4):1175-1189.
John Kominek, Tanja Schultz, and Alan W. Black. 2008. Synthesizer voice quality of new languages calibrated with mean mel cepstral distortion. In Spoken Languages Technologies for Under-Resourceful Languages.
Peter Ladefoged, Richard Harshman, Louis Goldstein, and Lloyd Rice. 1978. Generating vocal tract shapes from formant frequencies. The Journal of the Acoustical Society of America, 64(4):1027-1035.
Peter Ladefoged and Keith Johnson. 2014. A Course in Phonetics. Nelson Education.
Peter Ladefoged and Ian Maddieson. 2007. The UCLA phonetics lab archive.
Jackson L. Lee, Lucas F.E. Ashby, M. Elizabeth Garza, Yeonju Lee-Sikka, Sean Miller, Alan Wong, Arya D. McCarthy, and Kyle Gorman. 2020. Massively multilingual pronunciation mining with WikiPron. In Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020). European Language Resources Association (ELRA). Resources downloadable from https://github.com/kylebgorman.wikipron.
Xinjian Li, Siddharth Dalmia, Juncheng Li, Matthew Lee, Patrick Littell, Jiali Yao, Antonios Anastasopoulos, David R Mortensen, Graham Neubig, Alan W. Black, et al. 2020. Universal phone recognition with a multilingual allophone system. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8249-8253. IEEE.
Mona Lindau and Patricia Wood. 1977. Acoustic vowel spaces. UCLA Working Papers in Phonetics, 38:41-48.
Björn Lindblom. 1986. Phonetic universals in vowel systems. In John J. Ohala and Jeri Jaeger, editors, Experimental Phonology, pages 13-44. Academic Press, Orlando.
Björn Lindblom and Johan Sundberg. 1971. Acoustical consequences of lip, tongue, jaw, and larynx movement. The Journal of the Acoustical Society of America, 50(4B):1166-1179.
Peder Livijn. 2000. Acoustic distribution of vowels in differently sized inventories-hot spots or adaptive dispersion. *Phonetic Experimental Research*, Institute of Linguistics, University of Stockholm (PER-ILUS), 11.
Liang Lu, Arnab Ghoshal, and Steve Renals. 2013. Acoustic data-driven pronunciation lexicon for large vocabulary speech recognition. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pages 374-379. IEEE.
Ian Maddieson. 1995. Gestural economy. In Proceedings of the 13th International Congress of Phonetic Sciences, Stockholm, Sweden.
André Martinet. 1955. Économie Des Changements Phonétiques: Traité de Phonologie Diachronique, volume 10. Bibliotheca Romanica.
Lucie Ménard, Jean-Luc Schwartz, and Jérôme Aubin. 2008. Invariance and variability in the production of the height feature in French vowels. Speech Communication, 50:14-28.
Steven Moran and Daniel McCloy, editors. 2019. PHOIBLE 2.0. Max Planck Institute for the Science of Human History, Jena.
David R. Mortensen, Siddharth Dalmia, and Patrick Littell. 2018. Epitran: Precision G2P for many languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA).
Terrance M. Nearey. 1977. *Phonetic Feature Systems for Vowels*. Ph.D. thesis, University of Alberta. Reprinted 1978 by Indiana University Linguistics Club.
Josef Robert Novak, Nobuaki Minematsu, and Keikichi Hirose. 2016. Phonetisaurus: Exploring graphemeto-phoneme conversion with joint n-gram models in the WFST framework. Natural Language Engineering, 22(6):907-938.
Livia Oushiro. 2019. Linguistic uniformity in the speech of Brazilian internal migrants in a dialect contact situation. In Proceedings of the 19th International Congress of Phonetic Sciences, Melbourne, Australia 2019, pages 686-690, Melbourne, Australia. Canberra, Australia: Australasian Speech Science and Technology Association Inc.
Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, Jan Silovsky, Georg Stemmer, and Karel Vesely. 2011. The Kaldi speech recognition toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society. IEEE Catalog No.: CFP11SRW-USB.
Kishore Prahallad, Alan W. Black, and Ravishankhar Mosur. 2006. Sub-phonetic modeling for capturing pronunciation variations for conversational speech synthesis. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), volume 1. IEEE.