# JLMS25 and Jiao-Liao Mandarin Speech Recognition Based on Multi-Dialect Knowledge Transfer
## 2 Related Work  
Our model is closely related to the fields of machine learning, ensemble learning, and intelligent computing [5; 6; 7], and the literature in these fields is reviewed in this section.  
The conventional approach to adapting a pre-trained model to a specific target dataset typically involves full parameter fine-tuning [1; 8]. [PERSON] et al. [9] integrated language categories into the output, while [PERSON] et al. [10] incorporated speaker codes into the model's input. [PERSON] et al. [1] developed a technique for addressing multi-dialect speech recognition using a single sequence-to-sequence model. Each method relies on full fine-tuning. However, full fine-tuning presents significant limitations [11; 12; 13], including inefficiency and an increased risk of overfitting.  
Various strategies have been developed to address this issue and minimize the number of parameters involved in fine-tuning through parameter sharing. [PERSON] et al. [14] proposed the shared-hidden-layer multilingual DNN (SHLMDNN), which utilizes distinct output layers for individual languages. [PERSON] et al. [15] proposed the integration of adapters within the model. [PERSON] et al. [16] explored the use of a small number of adapters for efficient fine-tuning in Arabic dialect recognition tasks. [PERSON] et al. [17] used adapters to capture the target speaker's vocal characteristics for personalized recognition while preserving the generalization capability of the ASR model. In contrast, the effectiveness of adapters, particularly in neural machine translation (NMT) tasks, was confirmed by [PERSON] et al. [12]. However, the number of parameters increases rapidly with expansion of target tasks or languages [18; 19].  
[PERSON] et al. [18; 19] employed weight factorization, a multilingual algorithm, to enable distinct parameters for each language while mitigating the parameter growth associated with adding new target languages. This approach enhances the linear transformation functions of the neural network by decomposing each weight matrix into shared and language-dependent components. Let \(X\) represent the input and \(Y\) the output of weight factorization. The formula for weight factorization is presented as follows:  
\[Y=\left(\mathbf{W_{s}}\cdot\left(\mathbf{r_{m}}\cdot\mathbf{s_{m}}^{\top}\right)\right)^{ \top}\cdot X+\left(\mathbf{r_{a}}\cdot\mathbf{s_{a}}^{\top}\right)^{\top}\cdot X, \tag{1}\]  
where \(\mathbf{W_{s}}\) represents the shared weights, while \(\mathbf{r_{m}}\), \(\mathbf{s_{m}}\), \(\mathbf{r_{a}}\), and \(\mathbf{s_{a}}\) denote the language-dependent weights. [PERSON] et al. [20] demonstrated the effectiveness of this approach in an ASR task.  
However, integrating this algorithm into existing pre-trained models presents challenges, and its performance under low-resource conditions still needs to improve. We propose MDKT, a speech recognition framework specifically designed to overcome the challenges of Jiao-Liao Mandarin. MDKT improves the backbone model by introducing two key modules, WFAdapter and AttAdapter, which utilize dialect-dependent parameters to enhance adaptability across dialects and facilitate effective multi-dialect knowledge transfer. WFAdapter reduces the number of fine-tuning parameters by decomposing weights into shared and dialect-specific components. Meanwhile, AttAdapter, positioned before the classifier, integrates phonetic and linguistic knowledge from multiple dialects, enabling robust knowledge transfer for Jiao-Liao Mandarin speech recognition tasks. The MDKT framework employs a three-phase training strategy, beginning with a primary training phase to establish the model's foundation, followed by two fine-tuning phases that optimize performance across different dialects. This structured approach ensures adaptability and effectiveness even under low-resource conditions.