# Authorship Attribution in Less-Resourced Languages: A Hybrid Transformer Approach for Romanian
## 2 Related Work on Authorship Attribution
### Deep Learning Architectures
Four standalone embedding methods were leveraged, namely, word based, PoS based, lemma based, and PoS mask based, and a combined embedding method (composition based) was introduced. To form the combined embedding method, the standalone models were combined by various operations at the matrix level, such as the mean, product of norm of the matrices. The authors compared the results with multilingual BERT embeddings. The composition-based embedding method demonstrated improved performance compared to the baseline standalone methods or mBERT. [PERSON] et al. [31] proposed a method formulated as an authorship attribution task designed to identify the automatically generated texts and conduct performance evaluation between human- and machine-generated texts via the Turing Test. The authors leveraged linguistic features alongside NN architectures like RNN and CNN, and the analysis argued that the automatically generated texts presented significant or non-significant differences compared to human texts, depending on the generation method. [PERSON] et al. [32] investigated the identification of authors of Russian texts using SVM and deep NN architectures, including LSTM, CNN with attention, and Transformer. The findings suggested that while all methods were effective, SVM achieved the highest accuracy at 96%, attributed to the optimal parametrization and feature spaces. Deep NNs tended to be less effective, with 93% accuracy. Experiments revealed the susceptibility of SVM to deliberate text anonymization, resulting in a higher accuracy loss compared to deep NN, which experienced up to a 20% decrease in accuracy. The Transformer architecture proved the most effective for anonymized texts, achieving an accuracy of 81%. Another study [33] in the same publication series targeting Russian authorship attribution for literary texts leveraged FastText and SVM for feature extraction and selection, and introduced regularization methods. Their approach achieved an average of 83% in terms of accuracy, while FastText slightly outperformed the previous results, scoring an 84% accuracy. Moreover, a different study [34] introduced a technique for author identification that combined chaos game representation with deep learning methodologies. Working at the character level, their method employed chaos game representation to transform documents into visual representations. Following this, a CNN algorithm was used for classification, reaching an average accuracy of 85.52%. In their recent research, [PERSON] et al. [35] provided a comprehensive examination of the methods, models, datasets, feature types, and evaluation metrics employed in author attribution studies conducted for both source code and English text. The survey included two deep learning studies [36; 37] focused on source code author attribution. These studies introduce interpretable models and introduce the concept of saliency maps to enhance model interpretability. The initial model [36] achieved a 42% accuracy, utilizing an NN for embedding projection, alongside tSNE for visualization and the KNN classification algorithm for predictions. Similarly, the second study [37] utilized a CNN for embedding generation and, like its predecessor, employed a KNN classification method, resulting in a 70% accuracy rate. For authorship identification on text, various approaches including multiheaded RNNs [38] have shown promising results. Multilevel RNNs [38] were leveraged to process multiple authors simultaneously and were optimized for small datasets to prevent overfitting. This method achieved an AUC average score of 80%. Other CNN-based approaches demonstrated elevated performance on short texts [39; 40; 41; 42; 43; 44]. The survey [35] also outlined several challenges and constraints. Such limitations for deep learning techniques include considerations of the number of authors, i.e., the higher the variety of authors, the lower the accuracy. This tendency was observed for both ML and DL models, with DL scoring above ML in most of the cases.