Text Graph Neural Networks for Detecting AI -Generated Content (2025)

Abstract: The widespread availability of Large Language Models (LLMs) such as GPT-4 and Llama-3, among others, has led to a surge in machine-generated content across various platforms, including social media, educational tools, and academic settings. While these models demonstrate remarkable capabilities in generating coherent text, their misuse raises significant concerns. For this reason, detecting machine-generated text has become a pressing need to mitigate these risks. This research proposed a novel classification method combining text-graph representations with Graph Neural Networks (GNNs) and different node feature initialization strategies to distinguish between human-written and machine-generated content. Experimental results demonstrate that the proposed approach outperforms traditional machine learning classifiers, highlighting the effectiveness of integrating structural and semantic relationships in text.

## introduction large language models (llms) are now widely available and easily accessible, resulting in increased machine-generated content across various platforms, including q&a forums, social media, educational resources, and academic contexts. recent advancements in llm technology, such as llama-3 and gpt-4, have enabled these models to generate coherent responses to most user inquiries, making them increasingly attractive for replacing human labor in multiple fields. moreover, this increased accessibility has led to concerns about misuse, including the creation of fake news, financial and legal issues, and education disruptions. given the difficulty people have distinguishing between machine-generated and human-written text, there is a growing need for automated systems that can detect machine-generated content to mitigate and address the risks associated with its misuse @xcite . viewing this problem as a classical text classification task, there are plenty of approaches to tackle it nowadays, from traditional methods such as training machine learning or deep learning models using bag of words or stylometric patterns as features to more advanced approaches based on the transformer architecture, such as bert @xcite or roberta @xcite . however, in recent years, a new area of research known as graph neural networks (gnns), or graph embeddings, has gained significant interest @xcite . these networks have proven to be highly effective in text classification tasks involving complex relational structures as they retain the global structure of a graph within their embeddings. a text can be appropriately represented as a graph using the words/documents as nodes and the edge representing the significant relationship between the nodes. also, it is possible to assign different attributes or weights to the graph's edges/nodes to add more significant information. moreover, graph-based methods capture complex connections and dependencies that traditional methods might miss. understanding these relationships between words/nodes is crucial for exploiting the text's best features. it could also help capture semantic and syntactic nuances in the text, distinguishing between human and machine-generated texts. the contributions of our paper are summarized as follows: ## background several approaches to identifying automatically generated text have been proposed in recent years. many of these works have been published in the framework of a series of shared tasks that focus on this specific problem, such as pan-clef 2024 @xcite or iberautextification 2023 @xcite . in @xcite , the authors proposed an ensemble neural model combining probabilities from pre-trained llms as features and then applying a traditional machine learning classifier. another relevant work assessed text predictability by leveraging features like grammatical accuracy, word frequency, linguistic patterns, and fine-tuned llm representations (duran-silva, 2023). as we will see in the next sections, this last system ranks 1st in the binary classification task (for english text) on the iber-autextification shared task on the other hand, regarding the usage of gnns, many research works implement this approach to solve different types of text classification tasks @xcite . in @xcite , the authors explored the use of gnns with convolutional layers (called textgcn) for text classification, building a single text graph for a corpus based on word co-occurrence and document-word relationships, then used the graph embeddings as input for a final classification layer. experimental results show that the textgcn, even without external word embeddings, outperforms state-of-the-art methods and is particularly robust when training data is reduced. another research work combines bert and gcn for text classification tasks (called bertgcn) @xcite . bertgcn constructs a heterogeneous graph where documents are represented as nodes and uses bert embeddings as feature representation. their experiments demonstrate that bertgcn achieves state-of-the-art performance across multiple text classification datasets. moreover, regarding using text graph and gnns to solve the machine-generated content identification task, in @xcite it is proposed an ensemble architecture that combines text graph and gnns, llms embeddings, and stylometric features achieving better performance than baselines. ## method overview this section describes our classification methodology for detecting machine-generated text, combining text-graph and gnns. section 3.1 describes the text-to-graph construction process, and section 3.2 explains all the pipeline architecture. ## experiments this section shows all the experiment settings, datasets used, and the performance and results obtained for the proposed method. ## conclusions and future work this paper addresses the increasing prevalence of machine-generated content due to advancements in llms. with their rising accessibility, concerns about their misuse have grown. to tackle this, we proposed a model architecture that combines textgraph representations and gnns to detect machinegenerated text; specifically, we implemented a cooccurrence graph where each word is represented as a node, and if two words co-occur within the same text document, it is linked with an edge. then, this graph is fed into a gnn (gat), generating the graph document embeddings as output, which are used to train a final classification model to distinguish between human and machine text documents. based on the experiments in the autextification 2023 engish dataset, our approach demonstrated superior performance compared to baselines and traditional approaches, highlighting the effectiveness of integrating structural and semantic features in identifying machine-generated content. moreover, future studies could enhance these approaches further and investigate their applicability across various languages and datasets with different domains. also, different text graph representations (e.g. heterogeneous graphs) and gnn architectures should be tried using a combination of different node feature initialization strategies.