Overview of the D ag P ap24 Shared Task on Detecting Automatically Generated Scientific Paper (2024)

Abstract: This paper provides an overview of the 2024 ACL Scholarly Document Processing workshop shared task on the detection of automatically generated scientific papers. Unlike our previous task, which focused on the binary classification of whether scientific passages were machine-generated or not, one likely use case for text generation technology in scientific writing is to intersperse human-written text with passages of machine-generated text. We frame the detection problem as a multiclass span classification task: given an expert of text, label token spans in the text as human-written or machine-generated We shared a dataset containing excerpts from human-written papers as well as artificially generated content collected by Elsevier publishing and editorial teams. As a test set, the participants were provided with a corpus of openly accessible human-written as well as generated papers from the same scientific domains of documents. The shared task saw 457 submissions across 28 participating teams and resulted in three published technical reports. We discuss our findings from the shared task in this overview paper.

## introduction a big problem with the ubiquity of generative ai is that it has now become very easy to generate fake scientific papers. this can erode public trust in science and attack the foundations of science: are we standing on the shoulders of robots? one notorious example is how rafael luque massively used chatgpt to "polish" papers that were later found 1 in authorship for sale advertisements. according to a recent study @xcite , chatgpt "contamination" is seen in at least 60,000 published papers (slightly over 1% of all articles). for 1 https://tinyurl.com/rafaelluque the publishing year 2023, it is found that several specific words like "commendable", "intricate" or "meticulously" show a distinctive and disproportionate increase in their prevalence, which might be an indication of llm assistance in writing. the detecting automatically generated papers (dagpap foot_0 ) competition aims to encourage the development of robust, reliable ai-generated scientific text detection systems, utilizing a diverse dataset and varied machine learning models in a number of scientific domains. building on the dagpap22 competition kashnitsky et al. @xcite , this year's dataset consisted of 30,000 scientific articles sourced from sciencedirect foot_1 that were processed to integrate various alteration methods within the human-written content. as for the previous challenge, the 2024 dagpap challenge is a collaboration between a publisher (elsevier) and the research community to attempt a resolution through technical means. ## related work since the dagpap22 competition, there have been several more efforts on addressing the same problem. semeval 2024 hosted a task foot_2 on multigenerator, multi-domain, and multilingual blackbox machine-generated text detection @xcite (to appear at naacl 2024). the shared task offered 4 subtasks (monolingual and multilingual text classification, multi-way classification and human-machine text boundary detection) and attracted a couple hundred teams. the organizers found that all classification tasks turned out to be relatively easy to solve, with many teams beating baselines. the boundary detection problem, however, turned out to be harder to solve. clin33 hosted a similar shared task @xcite aimed at automatic detection of aigenerated texts in english and dutch, spanning multiple genres: medium-length news articles, tweets from the social media platform x (previously known as twitter), product reviews, shortform poetry, and journal columns. thus, the task focused on a cross-domain multilingual binary classification setup that included entirely new held-out test genres. the results were close to perfect for some genres (e.g., news and reviews) yet unsatisfactory for others (e.g., poetry). kaggle, a popular platform for machine learning competitions, also hosted a similar contest foot_3 "llm -detect ai-generated text" @xcite . the competition attracted over 5,000 participants from 4,300 teams and over 110,000 submissions. although kaggle is well-known for establishing state-of-the-art in applied machine learning (e.g., the usage of deberta @xcite for most nlp tasks), the focus of many competitors of the mentioned contest were on reverse-engineering the prompts used to generate the test set. this may limit the generalizability and applicability of these findings to real-world scenarios. some more similar competitions include au-textification (automated text identification) @xcite and mlmac @xcite (machine learning model attribution challenge). interestingly, mlmac organizers note that the most successful approaches were manual, as participants observed similarities between model outputs and developed attribution heuristics based on public documentation of the base models. some notable leaderboards for human-vs. machine-text detection and author attribution are: while most of the competitions and leaderboards address the problem of full-text classification into different classes (human/machine or several llms as origins; and only semeval subtask 3 is an exception); for dagpap24, we focus on a token-level text classification, for a scenario when human writing is interspersed with passages of machine-generated text. ## corpus creation the dataset comprised 30,000 scientific articles sourced from sciencedirect foot_4 . these articles' full texts were processed to integrate various alteration methods within the human-written content. for each alteration, the text segment contained between 2,500 to 4,000 characters. approximately 25% of the entire text was modified using one of the following techniques: column name data index 0 text across the world... before applying any alterations, the texts were segmented into individual sentences. this segmentation was crucial to maintain proper readability and to better simulate scenarios where machine learning and ai are utilized as writing assistants. a significant aspect of this dataset creation is the incorporation of randomness. both the text chunks where alterations would be made and the selection of the alteration method were determined at pseudo-random intervals. after processing, the text was tokenized into whole words. these tokens, along with their respective annotation spans, constitute the label columns. an example row of the final dataset can be found in ## results in @xcite , they address the challenge by using two tokenization methods, finetuning various language models, introducing an anomalous label smoothing (als) method and employing a majority voting method for ensembling model predictions. als is similar to a conditional random fields (crf) in filtering out unreasonable labels. it ensures that the distribution of predicted labels aligns with that of the training set. the team finished at the 2nd place on the final leaderboard with 0.9948 and 0.9944 f1 scores during the development and testing phases respectively. we are interested to see how similar techniques can be applied to the problem in general, when test data distribution does not always match the training data distribution. the authors of @xcite ) achieved an f1 score of 89.83 during the competition (landing at the 6th place on the leaderboard) and 99.46 afterward by fixing a tokenization issue. their solution was also to ensemble several models, here they ensembled several deberta models with varying the number of layers frozen, and input token lengths. finally, @xcite provided a multitask setting where one linear layer was trained for the binary prediction task (human-or machinewritten), and the other was multi-class. they used the binary classifier to guide whether they used the multi-class classifier. ## discussion during the previous iteration of the dagpap challenge @xcite , the participants were able to @xmath0. our hypothesis was that one of the reasons for this could be that the techniques used for generating "fake" examples tended to generate content that often included specific patterns. for example, the model used for summarization tended to open with phrases like "this paper is focused on ..." or "in this paper, the authors ...". to alleviate this issue, for this round of the competition, we leveraged more robust techniques for generating the competition corpus, specifically, gpt-3.5-turbo models, t5-small, and nltk synonym replacement. the task was also reformulated from sequence classification in the previous round to token classification in this round. despite these changes, some participants were still able to achieve over 99% f1 scores. we believe this demonstrates the difficulty in creating representative training corpora for the detection of machine-generated scientific content -in the experience of our teams at elsevier and as reported in @xcite @xcite , detection of machine-generated scientific content is still a challenging task despite these encouraging results. however, we believe that the dagpap24 shared task did offer a step forward to explore this challenging problem, and we hope to work together with the community on resolving this pernicious issue. for future work, we plan to design a competition that focuses on real-world scenarios where human-written and machine-generated scientific texts are mixed. we also plan to investigate the robustness of detection systems against adversarial attacks in the scientific domain, where the generated text is intentionally modified to evade detection.