Automated Tone Transcription and Clustering with T one2 V ec (2024)

Abstract: Lexical tones play a crucial role in Sino-Tibetan languages. However, current phonetic fieldwork relies on manual effort, resulting in substantial time and financial costs. This is especially challenging for the numerous endangered languages that are rapidly disappearing, often compounded by limited funding. In this paper, we introduce pitch-based similarity representations for tone transcription, named Tone2Vec. Experiments on dialect clustering and variance show that Tone2Vec effectively captures fine-grained tone variation. Utilizing Tone2Vec, we develop the first automatic approach for tone transcription and clustering by presenting a novel representation transformation for transcriptions. Additionally, these algorithms are systematically integrated into an open-sourced and easy-to-use package, ToneLab, which facilitates automated fieldwork and cross-regional, cross-lexical analysis for tonal languages. Extensive experiments were conducted to demonstrate the effectiveness of our methods.

## introduction as the second-largest language family in the world, the sino-tibetan languages comprise over 400 languages, nurturing the cultural and communicative bonds of 1.4 billion speakers (wikipedia). given the prevalence of lexical tones in most sino-tibetan languages @xcite , phonetic fieldwork typically involves conducting tone transcription for each word in the survey lexicon across unexplored regions, followed by categorizing these transcriptions into the respective tone categories of the region. exploring lexical tones enriches both linguistic and historical research, including migration patterns @xcite , contact between languages (lapolla, 2010), and their evolution over time @xcite @xcite . however, existing methodologies face two primary obstacles that hinder further investigation, research, and documentation of sino-tibetan languages. ## tone2vec: from tones to vectors in this section, we propose pitch-based similarity representations by quantifying the differences in pitch variations inherent in tones, which we call tone2vec. tone2vec is an easy-to-use, simple, and effective method for measuring similarity distance. tone2vec not only enables the comparison of tonal variations across dialects but also provides a straightforward loss function for training automatic tone transcription and clustering models. ## automatic tone clustering 7.1 clustering on transcription features many studies @xcite @xcite have shown that well-trained machine learning models not only perform well on targeted tasks but also provide hierarchical embeddings. therefore, by extracting intermediate layer features, the automatic tone transcription model m, has already assigned tonal representations for each speech instance. hence, the task of tone clustering can be regarded as a clustering task on transcription features. we then employ the clustering algorithm dbscan @xcite on these representations to determine the number of tone categories automatically, selecting the most probable predicted label in each cluster as a tone category. ## tonelab: a user-friendly platform for tonal languages we have developed an easy-to-use package, tonelab. we aim for tonelab to be a user-friendly platform for lightweight documentation and quantitative analysis in sino-tibetan tonal languages. to sum up, two main modules are introduced. ## limitations as an early exploratory work, this paper focuses solely on single syllables with cnn models. further studies involving continuous speech recognition on powerful transformer-based speech models like wav2vec2.0 @xcite ) could be conducted. and, our models are currently built using only a few thousand labeled speech data points due to the limited open-sourced data. additionally, we found that embeddings from the intermediate layers of trained transcription models effectively reflect tonal representations, though further considerations are needed to enhance phonological theories and phonetic analysis.