# Multimodal Seed Data Augmentation for Low-Resource Audio Latin Cuengh Language
## 4 Experimental Results and Analyses
### Text Classification  
Our inaugural experiment aimed to benchmark the model's performance on a public dataset, according to the methodology described by [PERSON] [3]. The experiments were bifurcated into two classifications--twenty-class and binary-class--with results tabulated in Table 1.  
Our model demonstrated superior performance in the text-only modality compared to the conventional Text-Classifier. The incorporation of seed words further bolstered the model's performance in this single-modality scenario. We attribute this enhancement to the supplementary information provided by seed words, which, being in the same vector space as the text during feature extraction, mitigate potential negative effects of multimodal feature fusion. In scenarios involving \"text + audio\", our model outperformed others under the same modality. However, other models that were tested under a \"text + audio + video\" modality yielded superior results in some instances. This discrepancy can be attributed to the limitations of our model's design, which currently only incorporates \"text + audio\" and does not extend to video modules. As mentioned earlier, the model's cross-attention includes three modalities: audio and speech modality (A), Latin Cuengh language modality (L), and Chinese text modality (C). LA represents the fusion of the Latin Cuengh language and audio information, CA represents the fusion of Chinese and audio information, and LAC represents the fusion of the Latin Cuengh language, audio, and Chinese information. The three modalities are cross-integrated through a three-layer network. To explore the influence of features between different layers, we conducted an ablation study on the attention weights of several modalities. The results of the experiment are shown in Table 2.  
Table 2 illustrates the outcomes of an ablation study, revealing that the model achieves peak performance when all three feature representation layers are active. Individually, layer 3's fusion method achieves the optimal performance, and when combining adjacent \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multirow{2}{*}{**Modalities**} & \multicolumn{5}{c}{**Twenty-Class**} & \multicolumn{5}{c}{**Binary-Class**} \\ \cline{3-11}  & & **ACC (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **ACC (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** \\ \hline Text-Classifier & Text & 70.88 & 67.40 & 68.07 & 67.44 & 88.09 & 87.96 & 87.95 & 88.09 \\ \hline \multirow{2}{*}{MAG-BERT} & Text + Audio & 72.16 & 68.28 & 68.88 & 68.88 & 88.83 & 88.71 & 88.67 & 88.85 \\  & Text + Audio + Video & 72.65 & 68.64 & 69.08 & 69.28 & 89.24 & 89.10 & 89.10 & 89.13 \\ \hline \multirow{2}{*}{MulT} & Text + Audio & 71.80 & 67.95 & 69.18 & 67.96 & 88.74 & 88.61 & 88.59 & 88.68 \\  & Text + Audio + Video & 72.52 & 69.25 & 70.25 & 69.24 & 89.19 & 89.07 & 89.02 & 89.18 \\ \hline \multirow{2}{*}{MISA} & Text + Audio & 71.60 & 68.37 & 69.57 & 68.30 & 88.45 & 88.31 & 88.32 & 88.35 \\  & Text + Audio + Video & 72.29 & 69.32 & 70.85 & 69.24 & 89.21 & 89.06 & 89.12 & 89.06 \\ \hline \multirow{2}{*}{Ours} & Text & 71.12 & 69.24 & 70.15 & 68.35 & 89.20 & 88.80 & 88.25 & 89.36 \\  & Audio & 71.03 & 68.44 & 69.56 & 67.35 & 88.96 & 89.06 & 88.36 & 89.78 \\  & Text + Audio & 72.38 & 69.58 & 70.26 & 68.91 & 89.66 & 89.80 & 89.47 & 90.14 \\ \hline Human & - & 85.51 & 85.07 & 86.37 & 85.74 & 94.72 & 94.67 & 94.74 & 94.82 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparative multimodal results on MIntRec dataset (“Twenty-class” and “Binary-class” denote the multi-class and binary classification).