How You Prompt Matters! E ven Task-Oriented Constraints in Instructions Affect LLM -Generated Text Detection (2024)

Abstract: To combat the misuse of Large Language Models (LLMs), many recent studies have presented LLM-generated-text detectors with promising performance. When users instruct LLMs to generate texts, the instruction can include different constraints depending on the user’s need. However, most recent studies do not cover such diverse instruction patterns when creating datasets for LLM detection. In this paper, we reveal that even task-oriented constraints — constraints that would naturally be included in an instruction and are not related to detection-evasion — cause existing powerful detectors to have a large variance in detection performance. We focus on student essay writing as a realistic domain and manually create task-oriented constraints based on several factors for essay quality. Our experiments show that the standard deviation (SD) of current detector performance on texts generated by an instruction with such a constraint is significantly larger (up to an SD of 14.4 F1-score) than that by generating texts multiple times or paraphrasing the instruction. We also observe an overall trend where the constraints can make LLM detection more challenging than without them. Finally, our analysis indicates that the high instruction-following ability of LLMs fosters the large impact of such constraints on detection performance.

## introduction llms have exhibited human-level generative capabilities in response to various textual instructions @xcite . with such remarkable generative ability, malicious users might exploit llms for cheating on student homework or fabricating misinformation @xcite . to mitigate such potential misuse of llms, many recent works have presented llm-generated-text detectors with highly promising detection performance @xcite @xcite @xcite . when users instruct llms to generate texts, the instruction potentially includes various constraints (e.g., output format and style) @xcite . here, we call such constraints that would naturally be included in instruction and are not related to detection-evasion as task-oriented constraints. despite being very natural, such differences in the instruction can have a large impact on the quality of the generated texts or on the downstream performance of various nlp tasks @xcite @xcite . most studies in llm detection focus on the target llmgenerated text itself, analyzing its linguistic features @xcite @xcite and not how the target texts are generated. moreover, most previous works do not include such a variety of instructions to create their benchmarking datasets for llm detection @xcite @xcite . this paper sheds light on the following question: do the task-oriented constraints in generation instruction affect the llm detection? motivated by this question, this paper first demonstrates that even task-oriented constraints in instruction can lead to inconsistent detection performance of current significant detectors. in particular, as depicted on the right in figure 1 , we explore the standard deviation (sd) of detection performance on various datasets generated via instructions with each different constraint. we focus on student essay writing as one of the generation tasks to consider the constraints, and there is a recognized demand for its detection @xcite . to generate essays via llms, we utilize essay questions created by @xcite . then, as listed in factor task-oriented constraint grammatically (grm.) your essay must be free of grammatical errors. usage (usg.) your essay must utilize a professional-level vocabulary. mechanics (mec.) your essay must be free of spelling and capitalization errors. style (sty.) your essay must include diverse word choices and sentence structures. relevance (rel.) your essay must follow the prompt. organization (org.) your essay must be logically organized. development (dev.) your essay must include concrete evidence that supports your opinion. cohesion (chs.) your essay must have a valid connection between paragraphs. coherence (chr.) your essay must have an effective transition throughout all paragraphs. thesis clarity (tc.) your essay must have a clear position through your essay. persuasiveness (per.) your essay must be persuasive to readers. ## methodology this section describes our strategy for identifying and evaluating the impact of task-oriented constraints in the instruction on the performance of llm-generated text detection. ## experiments and results our experiment investigates the answer to the following question: can current detectors consistently capture llm-generated text variations caused by even task-oriented constraints in the instruction? ## high instruction-following ability leads to inconsistent detection our experiments demonstrate that even taskoriented constraints in instruction induce notably inconsistent detection performance of the generated texts. in this section, to further examine how the constraint causes such an effect, we verify our hypothesis: the high instruction-following ability of llms as a generator causes the large impact of constraints in the instruction on llm detection. ## conclusion this study investigates how much impact even taskoriented constraints in instruction can have on the current detector's performance to the generated texts. our experiments in the domain of student essay writing demonstrate that even task-oriented constraints in instruction have a more significant effect on the detection performance than the effect of sampling texts and paraphrasing the instruction. furthermore, there is an overall trend where the constraints can make llm detection more challenging than without them. our analysis suggests that the high instruction-following ability of an llm as a generator leads to a noteworthy effect of the constraint. taking into account the remarkable speed of recent llm development, the instruction-following ability of llms would be much better, amplifying the effects of the constraint. therefore, in an era of evolving llms, our finding more strongly calls for further development of robust llm detectors against such distribution shifts caused by a constraint in instruction.