## 5 Experiment Design  
To compare selected methods, one pipeline of training and evaluation for all models was designed. The first dataset is loaded into program memory. The second step is the preprocessing of texts. Then, a dataset is split into five parts following a K-fold cross-validation scheme. The model is trained on a training set of cleaned documents. The final evaluation is performed after the selected model generates a vector representation of the documents, which are passed into the classifier. The used classifier is a multi-class SVM algorithm [40], taking into account the prevalence of unbalanced training sets. After the procedure is finished, results are averaged and saved. As an evaluation metric, average overall accuracy was used, which is the percent of correct predictions provided by the model with standard deviation across all runs. The code for the experiment is publicly available [41]. The importance of text preprocessing was addressed in the paper [42], where the authors presented how much preprocessing affects the model output. In conclusion, they presented general findings: usually, tokenization achieves the same or better results than more complex processing, such as coreference. The exception here was the Ohsumed collection, where there are many medical words, where the most efficient technique was to treat expressions or groups of words as a single token. In our evaluation of all datasets following preprocessing, steps were applied: lowercasing (except Longformer, which was pre-trained on uncased texts), filtering email addresses and decorations. The increase in the maximum length of the processed sequence is due to the increased VRAM requirements of the GPU cards on which the calculations were performed. The occurrence of outliers from the dataset that were many times longer than the mean would have unnecessarily increased the network size and increased the training time. Due to the limited hardware resources, it was necessary to limit the size of the neural networks in order to be able to perform the training. For models with a defined fixed length of the processed sequence of words and sentences resulting from the architecture of neural networks and their implementation details, it was decided to conduct the experiment in two variants. The first is to limit the length of the text to 30 sentences, each with a maximum of 100 words. The second is to limit the length based on the frequency distribution of sentences and words in the training datasets and, on this basis, to select the 99 th percentile of the distribution. The results of the training process for selected training datasets are presented below. The average accuracy and average training time of the models were measured. The parameters with which the models were trained are as follows: For BoW and TFIDF, the dictionary size was not limited; min_df = 1 and max_df = 0.95 were selected as the number of minimum occurrences of a word and the maximum frequency ratio of a word in the set. Identical min_df and max_df parameters were also used in the LSA and LDA models. LSA was run with parameters svd_features = 100, which specify the number of dimensions after SVD reduction, the number of iterations n_iter = 10. Similarly, LDA, in which n_components = 100 specifies the number of implicit topics to which words were assigned, and the number of iterations epochs = 10. The P-SIF model is parameterised by the number of clusters num_clusters = 40 and the size of the resulting representation embedding_size = 100. Doc2 Vec, DM and DBoW models were parameterized identically: the number of negative examples negative = 10, projection size vector_size = 100, context window window = 5 and minimum word frequency min_count = 1. For the HAN model, the maximum number of sentences and words in sentences is configuration dependent.