## 5 Empirical Studies
### Results and Discussion  
Evaluating the performance of unsupervised learning methods can be challenging as there is no standard way to confirm their accuracy on independent datasets. Therefore, the evaluation methods used in this study are based on the specific problem and dataset we used. Tables 7 and 8 show an average score of 10-fold with the 95% confidence intervals of our experiments with various sample sizes, to assess the model's sensitivity to changes in data size. The results of trial I in Table 7 were based on the default hyperparameters of the skip-gram with a negative sampling training algorithm, but this did not perform as well as the optimized setting in trial II of Table 7. As expected, the performance of a language model dataset can vary depending on the specific task and language it is used for, and the default hyperparameters may not be suitable for other datasets in different domains. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Utility Information**} & \multicolumn{2}{c}{**Vuze**} & \multicolumn{2}{c}{**Spring Framework**} & \multicolumn{2}{c}{**Elasticsearch**} \\ \cline{2-7}  & **File** & **Package** & **File** & **Package** & **File** & **Package** \\  & **UCB I UCA** & **UCB I UCA** & **UCB I UCA** & **UCB I UCA** & **UCB I UCA** & **UCB I UCA** \\ \hline Package & - & 386 1 385 & - & 525 512 & - & 1911 1884 \\ Files & 4778 1 4261 & - & 12,670 10,786 & - & 41,778 137,077 & - \\ Total developers & 41 128 & 41 125 & 912 1441 & 882 1277 & 2047 1897 & 1968 1735 \\ Maximum & 1173 150 & 1173 150 & 6624 150 & 6623 150 & 14,908 150 & 14,881 150 \\ session length & 11 2 & 112 & 112 & 112 & 112 & 112 \\ Total sessions & 19,124 17298 & 19,124 1 6094 & 24,467 113,941 & 22,065 18721 & 63,641 136,039 & 60,027 128,535 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Utility information about development changelog datasets. This table provides the unique counts before (UCB) and after (UCA) processing data for each dataset. We optimized the hyperparameters of the Word2 vec model based on the characteristics of our changelog dataset. As seen in Table 7, for trial II, there is a significant difference in performance between the default and optimized hyperparameters for our dataset. We also conducted an analysis of the model's sensitivity to different sample sizes and found that the 85:15 split ratio yielded the best results for the Vuze dataset at the file level. Table 8 shows the optimized results for each year in the Vuze dataset. We trained the model on one year at a time, starting with 2003, and then added each subsequent year to the training set, until all years from 2003 to 2014 were included. This allowed us to learn the patterns of the data for each year individually. Table 5 summarizes the optimized hyperparameters shown in Table 7 for trial II. Figure 7 shows the ability of the proposed method to visually depict the changes made to different elements in a vector space. The elements are placed close to each other if they have undergone similar changes. To create this visualization, a dimensionality reduction technique was used, which aims to keep as much information as possible while reducing the number of dimensions or features in the dataset. This technique can be useful for visualizing high-dimensional data, as it can be challenging to visualize data in more than three dimensions. By reducing the dimensionality of the data to two or three dimensions, it becomes much easier to plot and visualize the data, which can aid in understanding patterns and relationships within the data. A common method for dimensionality reduction is t-distributed stochastic neighbor embedding (t-SNE) [76], which is a non-linear method that is often used for visualizing high-dimensional data. As shown in Figures 7 and 8,