# Phoneme Recognition in Korean Singing Voices Using Self-Supervised English Speech Representations
## 5 Conclusions and Further Research  
In the field of ASR and ALT, there exists a significant disparity in the availability of data and the pace of development between English and many other languages. This discrepancy poses a considerable challenge for advancing ASR and ALT capabilities in languages where development is lagging. A potential solution lies in leveraging the wealth of research and resources available in the English language ecosystem by transferring learned priors to other languages. However, to ensure that these priors are broadly applicable and not constrained by language-specific characteristics, it is crucial to adopt approaches that capture general linguistic patterns. Self-supervised learning (SSL) has emerged as a promising strategy in this regard. This study demonstrates the effectiveness of self-supervised methods, such as HuBERT, for cross-lingual adaptation in phoneme recognition. By leveraging an English corpus for pre-training, we successfully adapted the model to recognize Korean phonemes. This showcases the impressive generalization capabilities of self-supervised models, enabling language priors to be utilized, even across different languages and domains. Additionally, we validated the effectiveness of melodic supervision for phoneme recognition of singing voices. The transfer learning improved the phoneme-level error rate of Korean speech by as much as 31.19%. Also, the melodic supervision improved the phoneme-level error rate in singing voices by 0.55%. There are several directions for further research. In general, the melodic variations should not significantly impact phoneme recognition or ALT. Depending on the fact, a method known as feature representation alignment [42] has suggested that aligning the feature distributions of two domains could help to mitigate irrelevant domain shifts. The concept of alignment could serve as a promising direction of research for comparison with the results in the paper. Furthermore, it is evident that melodic information can significantly aid in this alignment despite the notable differences between traditional Korean folk songs and Western songs. Due to the scarcity of available Korean singing voice datasets, however, we were unable to test our configurations on a wide range of datasets except CSD. Furthermore, despite considerable pronunciation differences between Korean and English, as discussed in Section 3.2, and the distant linguistic relationship between the two languages, we achieved the results outlined. We anticipate that similar adaptations, following the three steps described, could be applied to other languages. The ease or difficulty of this process will largely depend on the pronunciation similarities and linguistic relationships between the target language and English. Therefore, an additional step of analyzing these similarities and relationships may be necessary. It should be emphasized that there might be substantial potential for further exploration of diverse domain adaptation methods in the field of audio processing. Strictly speaking, our findings only underscore the potential of SSL in tackling complex and dynamic speech-processing tasks across languages and domains. Future research could focus on further refining these techniques and exploring their applicability in various real-world scenarios, including the development of a general model incorporating human voice representations to enhance cross-lingual phoneme recognition with limited target language data. The primary aim of our research is to preserve and document the traditional Korean folk songs, which have been passed down orally without written text or musical notation. Thus, the recognized phonemes in the paper should go through the word recognition stage to obtain the proper lyrics to be preserved. Another direction of our work involves aligning lyrics with traditional Korean folk songs to support music education. For those applications, the approach in this paper would be helpful where only small-scale datasets are available. Conceptualization, W.W. and J.L.