# Detection of Software Security Weakness Using Cross-Language Source Code Representation (CLaSCoRe)
## 4 Experimental Results
### Learning Scenario \"Other to C#\"  
The \"Darwinian\" evolution of programming languages resulted in lexical and semantical similarities between them. As an example, the C# programming language has been developed starting from C++ and Java languages. Before, C++ was influenced by C and Algol, the last one being influenced by Fortran [37]. We derive from here that C# is directly influenced by C, C++, and Java, and indirectly influenced by Algol and its predecessor, Fortran. We intend to make use of C# inherited characteristics from C, C++, and Java, in detecting security weaknesses in C# source code by learning their patterns in code written in other languages, like C, C++, and Java. As a result, we trained the same machine learning algorithms used for the \"C# to C#\" scenario to detect CWEs on NIST code written in C, C++, and Java, and tested them on NIST code written in C#. As seen in Table 2, the code snippets of the NIST dataset have some similarities in the order of operations and in the overall business logic, as they are created by the same community of programmers. Despite some differences between the lexical and semantic structures, they are also visible. Our intention is to observe how these two properties of the code, the influence of their ancestor languages and the similar behavior of writing code within a community of programmers, may help security analysts detect weaknesses when they lack static analysis tools, skills, or datasets for the downstream or exotic languages used by the developers in their organizations. According to the results displayed in Table 7, the Support Vector Machines (SVM) algorithm is able to detect CWE patterns in source code written in C#, when no C# code has been used in training if we use code written in C, C++, or Java during the training phase. Other machine learning algorithms did not perform well enough. \begin{table}
\begin{tabular}{c c} \hline \hline
**Programming Language** & **“Not Vulnerable” Code Samples** \\ \hline C\# & 14,323 \\ C++ & 13,341 \\ C & 19,457 \\ Java & 15,171 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The number of “not vulnerable” code samples in the NIST suite, per programming language. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**F1 Score Average**} \\ \hline
**Machine Learning** & **K-NN** & **Decision Tree** & **Random** & **Support Vector** \\
**Algorithm** & **(DT)** & **Forests (RF)** & **Machines (SVM)** \\ \hline Scenario “C\# to C\#” & 0.97 & 0.98 & 0.99 & 0.90 \\ Scenario “Other to C\#” & 0.63 & 0.45 & 0.43 & 0.72 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: F1 Score Average per machine learning algorithm, for the 8 CWEs in scope, in the “C# to C#” and, respectively, “Other to C#” learning scenarios. The results should be positioned and valued in the following context:  
* Not all possible C# code tokens were included in the vocabulary during the experiments; for example, the Windows API tokens (e.g., the \"GetEnvironmentVariable\" token observed in the C# code section of Table 2) or the.NET tokens. * The word embeddings were applied to the NIST corpus with C, C++, Java, and C# synthetic code, with word similarities being identified in this limited context. In the next stages, the word similarities will be generated on a much larger corpus of code from public sources, which will lead to new and more relevant clusters per each subset of tokens. The detailed results for the Support Vector Machines classifier applied to the 8 CWEs in scope are presented in Table 8.  
We can conclude that our method allows security analysts to identify security weaknesses in C# source code with acceptable accuracy when they lack a C# dataset to train the machine learning algorithms, but they have access to labeled code written in C, C++, and Java, from the same community of programmers. The accuracy averages at 0.89 in the \"Other to C#\" scenario with a standard deviation averaging at 0.054 for all CWEs in scope.