# Natural Language Processing Application on Commit Messages: A Case Study on HEP Software
## 7 Clustering
### Clustering Evaluation  
Only when there are patterns of clusters, the \(k\)-means clustering labels are valid and can be used in classification models. Results show that for the majority of projects, patterns appear in \(k\)-means clusters and each cluster is mainly dominated by one combination of the _type of change_ and _type of correction_. This is a promising result, which means \(k\)-means labels can be considered as commits labels and applied to the following classification models. In the following, considerations are given for the AliDPG project in the ALISW group. The number of commits assigned to each cluster has been evaluated. Table 5 shows the distribution of clusters for the AliDPG project: the cluster 0 has the most commits with 36.05% commits and cluster 7 has the least commits with 3.35% commits. There are 15.29% commits in cluster 1, 6.24% commits in cluster 2, 8.25% commits in cluster 3, 11.10% commits in cluster 4, 7.45% commits in cluster 5, and 12.02% commits in cluster 6. Before observing the patterns of clusters, the distribution of type of change and type of correction variables of AliDPG have been analysed. Combing the two variables, Table 6 shows types with the percentage of commits above 1%. According to the identification of types (detailed in Section 4), 23.35% of commits have _other_ value for _type of change_ and _type of correction_, which means their types have not been identified by the defined dictionary. \(K\)-means clustering can help assign these undefined commits into different clusters and assign the label for each commit. The second most common _type of change_ is _merge_ without any correction type, accounting for 14.9% of commits. The third most common _type of change_ is _feature addition_ without any correction type, accounting for 14.1% of commits. The following common _type of change_ is _corrective_, which means _type of correction_ has been identified (_fix_), having 6.84% of commits. During this process, the idea that \(k\)-means clustering can help to assign those undefined commits into different clusters and help to explore more comprehensive patterns among commits, has been confirmed. For the majority of projects, each cluster is usually dominated by one combination of type of change and type of correction. As shown in Table 7, the majority of undefined commits have been assigned to cluster 0 where 227 commits have other type. Cluster 1 is dominated by merge without some specific correction, having 186 commits in it. Cluster 2 is dominated by update and update with configuration. Cluster 3 is dominated by the undefined type of change and type of correction but some commits also involve in feature addition (with 16 commits) or fix (with 9 commits). Cluster 4 is dominated by fix, \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Type of Change** & **Type of Correction** & **Number of Commits** & **Percentage of Commits** \\ \hline other & other & 307 & 23.35 & \\ merge & other & 196 & 14.90 & \\ feature addition & other & 186 & 14.14 & 6.84 \\ corrective & fix & 90 & 4.79 & 4.64 \\ configuration & other & 63 & 4.79 \\ update & other & 61 & 4.64 \\ branch-merge & other & 45 & 3.42 \\ cancellation & other & 37 & 2.81 \\ corrective & bug-fix & 24 & 1.80 \\ configuration - feature addition & other & 21 & 1.60 \\ internal or ThirdParty dependency & other & 17 & 1.20 \\ corrective & fault & 14 & 1.00 \\ versioning & other & 14 & 1.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: AliDPG—examples of distribution of types. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Cluster Label** & **Number of Commits** & **Percentage of Commits** \\ \hline
0 & \(\mathcal{A}\)4 & 36.05 & \\
1 & 201 & 15.29 & \\
2 & 82 & 6.24 \\
3 & 112 & 8.52 \\
4 & 146 & 11.10 \\
5 & 98 & 7.45 \\
6 & 158 & 12.02 \\
7 & 44 & 3.35 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: AliDPG—distribution of clusters. bug, and fix, and this cluster is mainly related to correction. Cluster 5 is dominated by feature addition (with 36 commits) and undefined commits (with 35 commits).