Where are you from? Geolocating Speech and Applications to Language Identification (2024)

Abstract: We train models to answer the question, Where are you from? and show how such models can be repurposed for language identification (LID). To our knowledge, this paper is the first to introduce data sources, methods and models to tackle the task of geolocation of speech at a global scale, and the first to explore using geolocation as a proxy-task for LID. Specifically, we explore whether radio broadcasts with known origin can be used to train regression and classification-based models for geolocating speech. We build models on top of self-supervised pretrained models, using attention pooling to qualitatively verify that the model geolocates the speech itself, and not other channel artifacts.The best geolocation models localize speaker origin to around 650km. We confirm the value of speech geolocation as a proxy task by using speech geolocation models for zero-shot LID. Finally, we show that fine-tuning geolocation models for LID outperforms fine-tuning pretrained Wav2Vec2.0 models, and achieves state-of-the-art performance on the FLEURS benchmark.

## introduction language identification (lid) is a critical component in many modern multilingual speech technologies @xcite . as a result, tasks aimed at producing these class labels have been extensively explored @xcite @xcite and stateof-the-art systems perform remarkably well on common benchmarks, including on the fleurs @xcite and voxlingua @xcite corpora. while these corpora cover ∼100 languages, there are orders of magnitudes more accents and dialects and annotating all of them is intractable. to address this problem @xcite , proposed * equal contribution to scale speech technologies to thousands of languages and dialects by relying primarily on recordings of religious texts. the primary challenge of that effort was to see whether models trained on clean, single speaker recordings with known language labels would generalize to out-of-domain data. in contrast, we demonstrate that models can be trained on widely available heterogenous data collected from radio with soft language labels in the form of geolocations. to our knowledge, we are the first to demonstrate geolocation of speech on a global scale and the first to apply it to lid. geolocation of speech may also be preferable to lid in many circumstances. for instance, a codeswitched hindi-english utterance or a conversation between receptive english-spanish bilinguals in the united states may cause problems for lid systems, but they still likely occurred, respectively, in india and in the united states. these phenomena * not only challenge lid systems, but also the notion of using categorical labels for a phenomenon that occurs on a continuum @xcite . furthermore, audio can be approximately geolocated for free: the current location of a cell phone can be passed as metadata along with any audio recordings on that device; ip addresses can be resolved to 100-200 kilometers @xcite . we investigate whether these data can serve as soft labels for language, dialect, and accent. in this work, we use audio from fm radio broadcasts, that are simultaneous streamed on the web. the key assumption † used in this work is that because fm radio travels only up to about 70 km (fcc), on average, the speech heard on fm stations is at least understood by most people within a 70 km radius of the station, and possibly even representative of the local vernacular. see section 2 for more details. x is the input utterance. e loc is a trained embedding representing the geolocation task. h is the sequence of extracted embeddings. in the attention block, e loc is the query, and outputs of the pretrained model are the keys, k, and values, v. linear∠, transforms pooled representations into cartesian coordinates (in red), which are projected onto the surface of the earth (gray). error from the groundtruth (green), is measured by θ. alternatively, linear xent, produces scores, {s 1 , . . . , s l }, for l locations. ## conclusions we have demonstrated ¶ that radio stations with geolocation can be harvested and used to geolocate speaker traits such as language or dialect at a global scale. furthermore, because geolocation and language are so correlated, training models using geolocations can be used to initialize language id models, and work especially well in few-shot settings. future work should examine their application to accent recognition, and integration with multilingual asr systems.