# Natural Language Processing Application on Commit Messages: A Case Study on HEP Software
## 4 Database Construction  
We have extracted commit messages from 167 projects belonging to HEP experiment repositories: ALISW [54] contributed with 102 projects, LHCB [55] with 107 projects, CMS-SW [56] provided 41 projects, and, finally, ROOT [57] provided 1 project. At the end, we have kept 65 software projects (detailed in Table 1) with a number of commit entries higher than 100. The commit message collection was conducted till March 2021 and have been obtained by the _git log_ command. Figure 2 shows examples of original unstructured commit information. Each unstructured commit has been processed by extracting its meaningful features, such as the name of author, date, the body of the message, the number of changed files, the number of inserted lines, and the number of deleted lines. Table 2 shows how the unstructured commit data are turned into structured data for each commit. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Experiment Name** & \multicolumn{3}{c}{**Project Names**} \\ \hline ROOT & & root & \\ \hline \multirow{3}{*}{LHCb} & Condorcet & DevelopKit & analysis-essentials \\  & bender-tutorials & developkit-lessons & first-analysis-steps \\  & opendata-project & second-analysis-steps & starterkit-lessons \\ \hline \multirow{3}{*}{CMS-SW} & cmssw & cms-sw.github.io & cmssw-config \\  & cms-bot & cms-docker & genproductions \\  & SCRAM & root & cmssdt-web \\  & cmssdt-ib & pkgtools & hlt-confdb \\  & cms-git-tools & jenkins-backup & Stitched \\  & web-confldb & cmssw-framework & apt-rpm \\  & apt-rpm & int-build & RecoLuminosity-LumiDB \\  & DQM-Integration & cmspkg & \\ \hline \multirow{3}{*}{ALISW} & AliDPG & AliPhysics & AliRoot \\  & CMake & FairRoot & KFParticle \\  & RootUnfold & Vc & ali-bot \\  & alibuild & alidist & arrow \\  & aurora & cctools & clhep \\  & cypthon & create-pull-request & delphes \\  & docks & gcc & geant3 \\  & geant3-oldfork & geant4\_vmc & grpc \\  & gsl & liblzma & libpng \\  & libpng-old & mesos-plugin & rapidjson \\  & release-validation & root & vault-gatekeeper-mesos \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Projects in HEP experiments, i.e., ALISW, LHCb, CMS-SW, and ROOT. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Experiment Name** & \multicolumn{3}{c}{**Project Names**} \\ \hline ROOT & & root & \\ \hline \multirow{3}{*}{LHCb} & Condorcet & DevelopKit & analysis-essentials \\  & bender-tutorials & developkit-lessons & first-analysis-steps \\  & opendata-project & second-analysis-steps & starterkit-lessons \\ \hline \multirow{3}{*}{CMS-SW} & cmssw & cms-sw.github.io & cmssw-config \\  & cms-bot & cms-docker & genproductions \\  & SCRAM & root & cmssdt-web \\  & cmssdt-ib & pkgtools & hlt-confdb \\  & cms-git-tools & jenkins-backup & Stitched \\  & web-confldb & cmssw-framework & apt-rpm \\  & apt-rpm & int-build & RecoLuminosity-LumiDB \\  & DQM-Integration & cmspkg & \\ \hline \multirow{3}{*}{ALISW} & AliDPG & AliPhysics & AliRoot \\  & CMake & FairRoot & KFParticle \\  & RootUnfold & Vc & ali-bot \\  & alibuild & alidist & arrow \\  & aurora & cctools & clhep \\  & cypthon & create-pull-request & delphes \\  & docks & gcc & geant3 \\  & geant3-oldfork & geant4\_vmc & grpc \\  & gsl & liblzma & libpng \\  & libpng-old & mesos-plugin & rapidjson \\  & release-validation & root & vault-gatekeeper-mesos \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Structured dataset example. Figure 2: Original commit data. Furthermore, once obtained the structured data, we have gathered some statistics concerning the commit messages, their projects, and project groups, such as the total number of days in which the commit messages have been pushed, the numbers of developers involved, the total number of commits, the names and the total number of the modified files, the number of authors per commit, and the range of project time period [start date, end date]. Figure 3 shows the number of commits per project: 13 projects have more than 10,000 commits, 12 projects have commits between 2000 to 10,000, 40 projects have commits between 100 to 2000.