N ewbie ML at S em E val-2024 Task 8: Ensemble Approach for Multidomain Machine-Generated Text Detection (2024)

Abstract: Large Language Models (LLMs) are becoming popular and easily accessible, leading to a large growth of machine-generated content over various channels. Along with this popularity, the potential misuse is also a challenge for us. In this paper, we use SemEval 2024 task A monolingual dataset with comparative study between some machine learning model with feature extraction and develop an ensemble method for our system. Our system achieved 84.31% accuracy score in the test set, ranked 36th of 137 participants. Our code is available at: https://github.com/baoivy/SemEval-Task8

## introduction in our current time, large language models (llms), such as chatgpt @xcite , gpt-4 1 , llama @xcite and bloomz @xcite ) can be easily observed to be becoming increasingly prevalent from diverse forms ranging of news, multimedia to education. how outstandingly llms answer to user's problems makes them appealing for automatic missions as well as diminishing human labor in many scenarios. nevertheless, this also unexpectedly leads to problems with regard to human's misuses, spreading misinformation and causing disruptions in the education system in particular. therefore, it is necessary to develop systems that can automatically distinguish ai contexts from human-written ones. recently, with the exponential growth of llms, many researchers have attempted to distinguish human-written texts from machine-generated ones. @xcite @xcite have shown us about machine-generated and human writing data from various source. @xcite @xcite used zero-shot classification method to calculate probabilities from perturbated input text. @xcite 1 https://openai.com/ leveraged prompt-base method to utilize llms as detector. gehrmann et al., 2019 used statistical method to detect machine-generated paragraph with language model to compute conditional probability. for fine-tuning language model method, @xcite had a comparative study among pre-trained language model, feature-base and character level classification on deepfake dataset and showed that pre-trained language model has a best result than others. @xcite used feature-base classification with roberta as embedding and lstm + self-attention in classification head. bhattacharjee et al., 2023 used unsupervised and self-supervised learning by leveraging domain adaptation on unlabeled dataset and contrastive learning belonging with pre-trained language model to learn domain representations. @xcite used a novel approach with watermark embedding to detect llms text that employed by llms or neural network. being inspired by feature-based classification technique, we propose to have a comparative study for simple and lightweight machine learning method beside the trend of llm. our system compares various machine learning models among with ensemble method for multiple machine learning method to find the best combination for our system. the rest of the paper is as follows. the section 2 generalizes task description and dataset for our experiment. the section 3 shows the description of our system. the experimental setup and results are presented in section 4. finally, section 5 is the conclusion and discussion about our work. ## system description we will describe our developed model in this section. on section 3.1, we will discuss how we embedded sentences in essays using a pre-trained language model. then, for the crucial section, we would like to present the detail of our model on 3.2. we perform our architecture based on figure 1. first step, the essays need to be embedded through pre-trained language model. next, we use ensemble method with base model and then give a final prediction by using meta-model. ## conclusion in subtask a monolingual of semeval task 8, we have represented our system for machine-generated detection. we proposed to develop our system based on ensemble of multiple traditional machine learning method with hyperparameter tuning. we found that xgboost, svm and knn as base model and logistic regression in meta model would give the highest result.