# Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries
## 1 Introduction
### Current Study Objective  
This paper examines existing techniques for identifying machine-generated text, explores their limitations, and proposes novel approaches that leverage advanced linguistic analysis, contextual understanding, and pattern recognition to enhance the accuracy and reliability of detection systems. By addressing this pressing issue, the research aims to contribute to the development of more resilient and effective tools for safeguarding against the proliferation of machine-generated text with malicious intent in digital contexts.  
In this work, we propose and evaluate two detection methods to distinguish the automated generated texts in the context of digital libraries, leveraging two models' architectures, namely Transformer-based and classic ML-based. As part of this study, starting from a human-authored set of texts (\(\mathcal{T}\) k), we developed an extensive corpus of artificially generated texts of around 52 k documents using six different LLMs and three different generation methods, comprising texts across five domains. Among the text generation techniques, we mention text completion, backtranslation, or paraphrasing. Text completion is an NLP technique used to generate text by predicting the next word or sequence of words in a given context. This method uses statistical language models that have been trained on a large amount of textual data to generate the most probable words that would complete a sentence or a paragraph. The language models used for text completion in this experiment are based on GPT architecture [13] and are pre-trained on Romanian corpus. Backtranslation is a text generation method that involves iterative translations from one language to another and then translating it back to the original language. This method is used to generate variations of the original text that may have different sentence structures, words, or meanings. Paraphrasing algorithms generate a semantically similar text and a grammatically correct output, preserving the input's original meaning. A new text is generated with the same sense while using different words and sentence structures. Paraphrasing can also be used as a simplification instrument for rephrasing complex texts, facilitating text comprehension by decreasing the reading difficulty.  
The main contributions of our work are as follows:  
* Contribution to the development of resources for a less-resourced language, such as Romanian;
* The development of an extensive Romanian dataset, containing both human-authored and machine-generated texts, of around 60 k documents across five domains, generated via seven distinct methods, using several LLMs and generation techniques like text completion, backtranslation, and paraphrasing;
* Two detection models based on different architectures, namely Transformer-based and classic ML-based, designed to distinguish automatically generated texts for the Romanian language. The ML model slightly outperformed the Transformer model, reaching a macro F1 of 0.91, while the Transformer exhibited a macro F1 of 0.90;
* An analysis of textual similarities between human-authored and machine-generated texts for Romanian via similarity measures (cosine, BLEU, ROUGE) and non-parametric statistical tests;
* An exploration of linguistic differences between human and artificially generated texts via textual complexity indices.  
We release as open-source the dataset ([[https://huggingface.co/datasets/readerbench/ro-human-machine-60k](https://huggingface.co/datasets/readerbench/ro-human-machine-60k)]([https://huggingface.co/datasets/readerbench/ro-human-machine-60k](https://huggingface.co/datasets/readerbench/ro-human-machine-60k)), accessed on 8 January 2024), and the codebase ([[https://github.com/readerbench/ro-mgt-detection](https://github.com/readerbench/ro-mgt-detection)]([https://github.com/readerbench/ro-mgt-detection](https://github.com/readerbench/ro-mgt-detection)), accessed on 8 January 2024).