## 5 Experimental Studies
### Experimental Setup  
The experiments were conducted on Windows 10 with Python 3.10 installed. The dataset [19] containing Python programs solving tasks of various types, and the task detection problem formulation, are described in Section 3. The results of the conducted experiments do not depend on hardware characteristics as the estimation of the performance of the algorithms is not the subject of the current study.  
In the word2 vec-based approach to program embedding into vector space (see Section 4.1), we used the CBOW neural network [39] implementation available in the Gensim package [40]. Python programs were preliminary transformed into ordered multisets of tokens using the **tokenize** module [38]. The code2 vec-based approach to program embedding (see Section 4.2) involved the use of PathMiner 0.9.0 [27] available as a Docker image tagged **voudy/atminer** on DockerHub. The code2 vec neural network implementation was borrowed from [9] and adopted for the use in Windows-based environments.  
The Markov-chain-based approach to program transformation into vector-based representations (see Section 4.3) and the histogram-based algorithms (see Sections 4.4 and 4.5) were implemented from scratch in pure Python due to their simplicity, using only modules from the standard library, such as **ast**[41] and **dis**[46].  
The used classifier implementations based on KNN [29; 30], SVM [24], RF [31], and MLP [22] were taken from the sklearn library [49]. During the conducted experiments, we apply \(k\)-fold cross-validation during every test run with the aim of verifying classifier quality on different parts of the dataset. Figure 13 describes the \(k\)-fold cross-validation scheme.  
Figure 11: Histograms of AST node types obtained using Algorithm 4 for programs from the first row of Table 2. The programs use different approaches to implementing a recurrent formula [19]; the dark blue color highlights overlapping histograms.  
Figure 12: Program embedding into vector space based on AST node type histograms.  
According to Figure 13, in \(k\)-fold cross-validation, the quality of a classifier is measured by splitting the original dataset into \(k\) folds \(k\) times. \(k-1\) blocks are used during classifier training, and \(1\) block is used during classifier testing using a specialized metric. The blocks that are used during classifier training are highlighted in grey in Figure 13, and the blocks used for classifier testing are highlighted in yellow. Each of the \(k\) runs outputs \(q_{i}\), the \(i\)-th value of a specialized quality metric obtained during classifier testing, \(i=1,\ldots,k\). The result of \(k\)-fold cross-validation is represented by a tuple \(Q=(q_{1},q_{2},\ldots,q_{k})\) containing the obtained specialized metric values. The mean value \(\mu\) for \(Q\) and the standard deviation value \(\sigma\) for \(Q\) are given by the following equations:  
\[\mu(Q)=\frac{1}{k}{\sum_{i=1}^{k}q_{i}}, \tag{7}\]  
\[\sigma(Q)=\sqrt{\frac{1}{k}{\sum_{i=1}^{k}(q_{i}-\mu(Q))}^{2}},\text{ where }Q=(q_{1},q_{2},\ldots,q_{k}). \tag{8}\]  
In our experimental setup, we assume \(k=5\) for cross-validation. Figure 14 describes the implemented source code classification framework used in our study.