## 5 Experiment Setup and Results  
About 45 groups of different features of text were used to train the SVM classifier [1]. Vectors ranging in size from 33 to 5000 features were used, including characteristics of different levels of text analysis:  
* Lexical (punctuation, special symbols, lexicon, slang words, dialectic, archasisms);
* Morphological (lemmas, morphemes, grammar classes);
* Syntactic (complexity, position of words, completeness, sentiments);
* Structural (headings, fragmentation, citation, links, design, mention of location);
* Content-specific (keywords, emoticons, acronyms and abbreviations, foreign words);
* Idiosyncratic stylistic features (spelling and grammatical errors, anomalies);
* Document metadata (steganography, data structures). Even a carefully selected feature space does not guarantee high model efficiency, but equally important are the training parameters of the SVM model. In an early study [1], the following parameters were identified as the most appropriate:  
* Learning algorithm--sequential optimization method;
* Kernel--sigmoid;
* Regularization parameter C = 1;
* Acceptable error level--0.00001;
* Normalization--included;
* Compression heuristic--included. As stated earlier, deep NNs do not need a predetermined set of informative text features, as they are able to search for them on their own. However, these models are also extremely sensitive to learning parameters. These parameters have been selected based on the results of model experiments for related tasks [32; 33]:  
* Optimization algorithm--adaptive moment estimation (Adam);
* Regularization procedure--dropout (0.2);
* Loss function--cross-entropy;
* Hidden layer activation function--rectified linear unit (ReLU);
* Output layer activation function--logistic function for multi-dimensional case (Softmax). A large number of data are required to train models. For this purpose, the corpus was collected from the Moshkov library [34]. The corpus includes 2086 texts written by 500 Russian authors. The minimum size of each text was 100,000 symbols. As part of experiments with models, the number of training examples varied with needs in solving real-life authorship identification tasks (including when the training data are limited). Therefore, the texts were divided into fragments ranging from 1000 to 100,000 characters (- 200-20,000 words). We used three training examples for each author and one for testing. Table 1 shows the accuracy of the SVM model for datasets of 2, 5, 10, and 50 candidate authors. Table 2 shows the results of applying SVM trained on statistical features and extracted aspects. Cross-validation for 10-folds was used as a procedure for evaluating the effectiveness of the models. It should be noted that the results presented in Tables 1 and 2 were obtained by joint application of SVM and the Laplace smoothing method, which gives a slight increase in accuracy (from 0.01 to 0.07) on small sample sizes. Experiments have also shown that the Good-Turing and Katz smoothing methods negatively affect the quality of identification, with an average accuracy 0.04-0.11 lower when using them. Table 3 shows the accuracy of determining the author using the LSTM for datasets of similar size and obtained by 10-fold cross-validation, while Table 4 shows the CNN with Attention and Table 5, the Transformer. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**The Length of Text, Symbols** & **2 Authors** & **5 Authors** & **10 Authors** & **50 Authors** \\ \hline
1000 & 0.9 & 0.71 & 0.64 & 0.49 \\
5000 & 0.91 & 0.79 & 0.77 & 0.54 \\
10,000 & 0.93 & 0.85 & 0.81 & 0.59 \\
20,000 & 0.98 & 0.97 & 0.94 & 0.78 \\
40,000 & 0.99 & 0.99 & 0.97 & 0.82 \\
60,000 & 0.99 & 0.97 & 0.97 & 0.89 \\
80,000 & 0.99 & 0.98 & 0.98 & 0.93 \\
100,000 & 1 & 0.99 & 0.99 & 0.95 \\ Average accuracy & 0.96 & 0.91 & 0.88 & 0.75 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Average accuracy of author identification using SVM.