# Complex Cases of Source Code Authorship Identification Using a Hybrid Deep Neural Network
## 7 Experiment Setup and Results
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Author** & **Method** & **Complex Cases** & **Dataset** & **Programming Language** & **Average Accuracy** \\ \hline \multirow{8}{*}{**Ours**} & \multirow{8}{*}{HNN} & & C++ & 92\% \\  & & & Java & 97\% \\  & & & JS & 92\% \\  & & & Python & 95\% \\  & & & C & 96\% \\  & & Encoding & GitHub & C\# & 96\% \\  & & standards, & (ours) & Ruby & 95\% \\  & & Mixed data, & & PHP & 92\% \\  & & Artificially & & Swift & 98\% \\  & & generated data, & & Go & 93\% \\  & & Commit-based & & Groovy & 99\% \\  & & & data & Kotlin & 91\% \\  & & & Perl & 96\% \\ \cline{3-6}  & & & GCJ & C++ & 98\% \\  & & & & Java & 99\% \\  & & & Python & 98\% \\ \hline [PERSON], & & & & C++ & 97\% \\ [PERSON], [PERSON] & & & & Java & 100\% \\ [PERSON] [3] & & & & Python & 100\% \\ \hline \multirow{3}{*}{[PERSON], [PERSON], [PERSON], [PERSON], [PERSON]. [4]} & \multirow{3}{*}{RoPGen} & \multirow{3}{*}{-} & GCJ & C++ & 92\% \\  & & & GittHub & Java & 98\% \\ \cline{1-1}  & & & & GitHub & C & 84\% \\ \cline{1-1}  & & & & & Java & 90\% \\ \hline \multirow{3}{*}{[PERSON], [PERSON], } & \multirow{3}{*}{GNN} & \multirow{3}{*}{-} & GCJ & C++ & 60\% \\  & & & & Java & (avg.) \\ \hline \multirow{3}{*}{[PERSON], } & \multirow{3}{*}{XAI} & \multirow{3}{*}{-} & GCJ & C++ & 74\% \\  & & [PERSON]. [7; 8] & & & Python & 72\% \\ \hline \multirow{3}{*}{[PERSON], [PERSON], [PERSON], [PERSON], [PERSON]. [9]} & \multirow{3}{*}{PbRF} & \multirow{3}{*}{-} & GCJ & Java & 98\% \\ \cline{1-1}  & & & & C & 93\% \\ \cline{1-1}  & & & & C++ & 98\% \\ \cline{1-1}  & & & & Python & 88\% \\ \hline \multirow{3}{*}{[PERSON], [PERSON], [PERSON], [PERSON]. [10]} & \multirow{3}{*}{PDGDL} & \multirow{3}{*}{-} & GCJ & C\# & 99\% \\  & & & & C++ & JavaScript & (avg.) \\ \hline \multirow{3}{*}{[PERSON], [PERSON]. [11]} & \multirow{3}{*}{RF, } & RF, & \multirow{3}{*}{-} & GitHub & C++ & 75\% \\  & & & & C++ & 71\% \\ \hline \multirow{3}{*}{Caldeira R.S. [12]} & \multirow{3}{*}{LSTM} & \multirow{3}{*}{-} & GCJ & C++ & 75\% \\  & & & Codeforces & C++ & 71\% \\ \cline{1-1}  & & & & C++ & 71\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of source code authorship identification approaches. The experimental results were obtained for the dataset collected from GitHub and for the open GCJ dataset. The first dataset contains codes of developers programming in several languages. The provided samples can be noisy without the author's obvious writing style features (this is due to following coding standards) and transformed by third-party software. The source codes of the same programmer may contain solutions of completely different problems, which can also confuse the classifying models. The second dataset contains cleaned samples. The conduction of the author's features analysis is not complicated by external factors. Samples contain solutions of the same type of problems, so the model can focus only on the search for informative features and avoid noise. Therefore, the results for these datasets significantly differ. The difference can reach 10% for the same programming language. The high accuracy obtained by [15] for the dataset from GitHub is due to careful preliminary cleaning and data preparation. Authors use one repository per author, that is, they do not take into account style changes over time. In addition, their dataset only presents projects written by the same author in the same programming language, which guarantees data homogeneity. The accuracy obtained by the authors is only valid under ideal conditions, which are impossible in real-life development processes. In the approaches of other authors where GitHub data are used without careful processing [4, 10], the achieved results are inferior in accuracy to our HNN for all programming languages. Since most modern approaches do not consider complex cases besides obfuscation, we decided to compare the three methods with the example of identifying the author of the source code that was obfuscated with the Tigress tool.