## 5 Results
### Results of Statistical Testing  
As shown in Table 5, the MulDIC\({}_{TIC}\) model exhibited the highest performance across all projects in this experiment. This indicates the potential to generalize and apply this model to various open-source projects on GitHub. In this section, we conducted a Mann-Whitney U-test [43] to verify the statistical significance of the results by comparing the classification accuracy levels between the MulDIC\({}_{TIC}\) model and the baseline model for all projects. Since the data in the experiment results did not follow a normal distribution, we adopted the non-parametric statistical test, Mann-Whitney U-test. The hypotheses set for this test are as follows:  
**H0 (null hypothesis).** The averaged performance of a unimodal model is equal to that of a multimodal model.  
**H1 (alternative hypothesis).** The averaged performance of a unimodal model is not equal to that of a multimodal model.  
Table 6 presents the test results for the individual and overall projects. The \(p\)-values for the individual projects (VS Code, Kubernetes, Flutter) and the total project (Total) were very close to 0.00, indicating significantly lower values than the significance level criterion of 0.05 for the hypothesis test. Therefore, we rejected the null hypothesis (H0) and accepted the alternative hypothesis (H1). In other words, there is a significant difference between the baseline and proposed models, and the proposed approach is statistically significant.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Project** & **Model** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{4}{*}{VS Code} & Text Only & 68.45 & 68.27 & 68.36 \\  & MulDIC\({}_{TI}\) & 79.60 & 77.92 & 78.75 \\  & MulDIC\({}_{TC}\) & 68.11 & 67.77 & 67.94 \\  & MulDIC\({}_{TIC}\) & **80.40** & **79.70** & **80.05** \\ \hline \multirow{4}{*}{Kubernetes} & Text Only & 66.91 & 66.10 & 66.50 \\  & MulDIC\({}_{TI}\) & 68.35 & 68.33 & 68.34 \\  & MulDIC\({}_{TC}\) & 79.97 & 78.33 & 79.14 \\  & MulDIC\({}_{TIC}\) & **81.25** & **80.00** & **80.62** \\ \hline \multirow{4}{*}{Flutter} & Text Only & 67.70 & 67.66 & 67.68 \\  & MulDIC\({}_{TI}\) & 69.79 & 69.41 & 69.60 \\  & MulDIC\({}_{TC}\) & 68.68 & 68.42 & 68.55 \\  & MulDIC\({}_{TC}\) & **76.85** & **75.66** & **76.25** \\ \hline \multirow{4}{*}{Roslyn} & Text Only & 62.92 & 62.86 & 62.89 \\  & MulDIC\({}_{TI}\) & 66.13 & 66.04 & 66.08 \\ \cline{1-1}  & MulDIC\({}_{TC}\) & 66.06 & 66.04 & 66.05 \\ \cline{1-1}  & MulDIC\({}_{TIC}\) & **67.95** & **67.93** & **67.94** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Experimental results that compared MulDIC\({}_{TIC}\) with other models.  
Furthermore, we calculated this test's effect size [43] to determine practical significance. The effect sizes ranged from 0.2 to 0.5 in most projects, generally considered moderate. This indicates that the difference between the proposed model and the baseline model is statistically significant and practically significant.  
The test results for Roslyn showed a marginal difference, with the acceptance of the null hypothesis (H0) and a small effect size. This is understandable, considering the degree of performance improvement of the model. Roslyn had the slightest improvement among the four projects, with a 5.07% increase in the F1-score. However, in deep learning models, such a level of performance improvement and difference between models is significant. The test results for Roslyn showed a marginal difference according to the criteria of the statistical test, but in terms of performance experiments, it achieved a practically significant improvement. Additionally, the total project's validation results and effect size indicate that the proposed model is statistically and practically significant. Therefore, we expect our proposed model to perform well when applied to other open-source projects.