# Authorship Attribution in Less-Resourced Languages: A Hybrid Transformer Approach for Romanian
## 2 Related Work on Authorship Attribution
### Deep Learning Architectures  
Deep learning (DL)-based authorship attribution considers leveraging NNs to discern patterns in written texts. This approach explores the hierarchical and complex relationships within language, allowing for a better understanding of authorial styles. DL architectures such as Recurrent Neural Network (RNN) and Transformer-based models excel at capturing long-range dependencies and contextual information, which are essential for authorship identification. By ingesting large amounts of textual data, these models autonomously learn the features that characterize an author's writing style, mitigating the need for manual feature engineering. Research has shown the effectiveness of NN models in achieving precise author identification. [PERSON] et al. [23] reached a high performance using a Gated Recurrent Unit (GRU), a type of RNN, on a dataset sourced from the Gutenberg Project ([[https://www.gutenberg.org/](https://www.gutenberg.org/)]([https://www.gutenberg.org/](https://www.gutenberg.org/)), accessed 12 February 2024). Their model sequentially represented words using GloVe pre-trained embeddings [24], employing average pooling and incorporating another GRU layer to capture the sequence of sentences within an article. Among the tested models, the article-level GRU achieved the highest accuracy, reaching 69.1% on the C50 dataset and 89.2% on the Gutenberg dataset. The study also explored authorship verification, where a Siamese network-based model achieved exceptional performance, achieving 99.8% accuracy on both the C50 and Gutenberg datasets. In a related exploration, [PERSON] [25] explored textual recommendations based on a limited set of stylometric features. Additionally, [PERSON] [26] considered the author's writing style, yet the model learned from the perspective of reviewers rather than automatically extracting information from the content of each book; however, relying on user reviews may introduce bias, requiring widely read and reviewed books in the systems. Extending the scope, [PERSON] et al. [27] leveraged the entire e-book's content to organize the authors hierarchically. The study leveraged a multilayer self-organizing map (MLSOM), where authors were represented using a hierarchical tree structure, encompassing features such as biography, books, pages, and paragraphs. The benefit of employing the MLSOM algorithm stood in its capacity to efficiently handle complex representations, thus clustering e-books and authors. [PERSON] et al. [28] employed two RNNs, namely Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The study evaluated the performance of index-based word embeddings against pre-trained embeddings on two separate datasets. When combined with word-index embeddings, the GRU model achieved a high accuracy in one of the datasets. Authorship attribution for short online text snippets, such as emails, blogs, or forum posts, involves identifying the original author using stylometric traits as discussed in the research of [PERSON] et al. [29]. Nonetheless, accurately capturing an author's unique writing style presents challenges due to the complexity of the text. The authors proposed a model architecture that combines a convolutional neural network (CNN) with a bidirectional Long Short-Term Memory (LSTM) encoder. The framework extracted lexical stylometric characteristics, which were then processed through a bidirectional encoder to generate the features' vector representation. The feature array underwent normalization using distributed high networks to improve generalization and reduce errors. Subsequently, the bidirectional decoder analyzed the feature vector to extract the distinct writing styles of individual authors. The final step involved predicting the author of a text snippet using a classification layer. Results showed that their approach outperformed previous strategies. Another study by [PERSON] et al. [30] investigated the use of different stylometric feature embeddings for finding the authors of multilingual literary texts.