How Does the Disclosure of AI Assistance Affect the Perceptions of Writing? (2024)

Abstract: Recent advances in generative AI technologies like large language models have boosted the incorporation of AI assistance in writing workflows, leading to the rise of a new paradigm of human-AI co-creation in writing. To understand how people perceive writings that are produced under this paradigm, in this paper, we conduct an experimental study to understand whether and how the disclosure of the level and type of AI assistance in the writing process would affect people’s perceptions of the writing on various aspects, including their evaluation on the quality of the writing, and their ranking of different writings. Our results suggest that disclosing the AI assistance in the writing process, especially if AI has provided assistance in generating new content, decreases the average quality ratings for both argumentative essays and creative stories. This decrease in the average quality ratings often comes with an increased level of variations in different individuals’ quality evaluations of the same writing. Indeed, factors such as an individual’s writing confidence and familiarity with AI writing assistants are shown to moderate the impact of AI assistance disclosure on their writing quality evaluations. We also find that disclosing the use of AI assistance may significantly reduce the proportion of writings produced with AI’s content generation assistance among the top-ranked writings.

## introduction recent advances in generative artificial intelligence (ai) technologies like large language models (llms) have significantly expanded the scope and depth of tasks where ai tools can collaborate with humans. for instance, @xcite have demonstrated remarkable abilities in language understanding and generation, highlighting the potential of incorporating llm-powered assistants in human writing workflows to enhance their productivity and creativity @xcite . unlike traditional writing support systems that only offer basic grammar and spelling checks, llm-powered writing assistants can provide a diverse range of support to human writers, ranging from polishing human-written texts to drafting written content from scratch, all tailored to human instructions. as such, a new paradigm of human-ai co-creation in writing has emerged, and we expect to see a growing amount of writings that are generated with some degree of ai assistance in the next many years to come. a critical question that arises with the emergence of the human-ai co-writing paradigm is how to present writings appropriately to readers in the future. while the disclosure of ai assistance in the writing process has been advocated for transparency and accountability considerations, empirical understandings on the implications of ai assistance disclosure on people's perceptions of writings are still lacking. for example, does the disclosure of ai assistance during the writing process change people's evaluation on the quality of the writings? in other words, in an era of human-ai collaboration, will people evaluate the quality of writings solely based on the final written content, or also consider the generation process of the writings? if disclosing the involvement of ai assistance does affect people's perceptions of writing quality, does the effect vary with the type of ai assistance and writing task? how will the disclosure of ai assistance change people's ranking among writings that are produced with different levels of ai assistance? to answer these questions, in this paper, we present an experimental study to understand how the disclosure of ai assistance influences people's perceptions of writings on various aspects. our study was divided into two phases. in phase 1, we collected writing samples on two types of tasks (i.e., argumentative essay or creative story), and during their writing process, the human writers used different levels or types of ai assistance pow-ered by chatgpt, a state-of-the-art llm. specifically, participants were recruited to complete their writing tasks under one of the three writing modes: (a) independent, where participants completed the writing task on their own without receiving any assistance from chatgpt; (b) ai editing, where participants took the primary responsibility of drafting the article while they received only text editing and polishing assistance from chatgpt; and (c) ai (content) generation, where chatgpt took the primary responsibility of drafting the initial version of the article while participants provided feedback and directed the subsequent revisions of the article through conversational interactions with chatgpt. after collecting writing samples, in phase 2, we designed a randomized experiment in which participants were recruited to review writing samples collected from phase 1 on various aspects (e.g., overall quality, organization, creativity). depending on the treatment that participants were assigned, the level and type of ai assistance during the writing process was either disclosed or not disclosed to them when they reviewed these writings. our experimental results indicate that disclosing ai assistance in the writing process, especially when ai provides content generation assistance as that in the "ai generation" writing mode, significantly decreases people's evaluation on the overall quality of the writing for both argumentative essays and creative stories. similar patterns have been observed on people's willingness to shortlist a piece of writing for performance-based bonus, and people's detailed evaluations on the writing on aspects like organization and originality. the disclosure of ai assistance often increases the variations in quality evaluations provided by different individuals for the same writing as well. this indicates that with the knowledge of the ai involvement in the writing process, the quality evaluations of writings become unpredictable and highly susceptible to variability depending on who evaluates them. further examinations reveal that factors such as an individual's own writing confidence and their familiarity with chatgpt might play a role in moderating the effects of ai assistance disclosure on people's evaluation of the writing quality. for instance, individuals with higher confidence in their writing skills are more likely to lower their quality ratings when ai use is disclosed, compared to those with lower writing confidence. moreover, we find that disclosing ai assistance would also significantly reduce the proportion of top-ranked writings produced with ai's content generation assistance, but this is only true for argumentative essays and not for creative stories. together, our study offers important experimental evidence regarding the impact of the disclosure of ai use on human perceptions of the human-ai co-created content. we conclude by discussing the design implications of our findings, and outline limitations and future work. human-centered evaluations of texts generated or edited by large language models. with their unparalleled generative capabilities, llms can rapidly generate a large volume of texts. recent research has started to evaluate how humans perceive llm-generated texts. for example, it was found that distinguishing llm-generated texts from human-generated texts is quite challenging, in terms of both objective textual patterns such as text fluency and perplexity @xcite , and subjective human judgment @xcite . on online platforms like airbnb, it was found that ai-generated profiles may be perceived as less trustworthy by users @xcite . interestingly, recent research has also showed that when llms are used to assist humans in writing, the llm-generated texts can impact the human writer, consciously or unconsciously. for instance, one study found that text suggestions provided by llms can change the sentiment of humangenerated texts @xcite . in addition, writing assistants powered by opinionated llms can not only alter the opinions expressed in the human-generated text but also subtly influence the perspectives and beliefs of the human writers themselves @xcite . different from these prior works, in this paper, we focus on examining whether and how the disclosure of varying types of assistance from llms would affect people's perceptions of writings. ## study design to understand how the disclosure of ai assistance would influence people's perceptions of writings, we divide our study into two phases. in the first phase, we collect writing samples on different types of writing tasks, while the human writers use different levels or types of ai assistance during their writing processes. the second phase is our focal experiment, in which we recruit participants to evaluate the writing samples collected. they will be randomly assigned to groups informed or uninformed about ai assistance used in the writing process and the specific type of assistance used. this study was approved by the institutional review board of the authors' institution. ## results in total, 786 workers from prolific took our phase 2 study and passed the attention check (non-disclose: 380, disclose: 406; see appendix a for participants' demographic backgrounds). the average amount of time participants spent on the phase 2 study was 20 minutes, resulting in an average hourly wage of $7.5. on average, each article received 5.6 evaluations from participants in the "non-disclose" treatment and 5.9 evaluations from participants in the "disclose" treatment. below, we analyze the impact of disclosing ai assistance in the writing process on people's assessment of writings based on the data we collected in phase 2. due to the space limit, we focus on sharing the results on the impact of ai disclosure on the perceived quality and ranking of articles in the main text. see appendix g for the impact of ai disclosure on the perceived authorship of articles. ## discussions possible explanations for our findings. in our study, we made a few interesting observations. for example, we find that disclosing the usage of ai assistance may affect people's evaluations of the writing quality to a larger extent when ai provides assistance in generating new content than when ai provides editing assistance. this observation may be caused by raters' perceived human effort contributing to the final writings. in particular, when ai provides content generation assistance (i.e., as in the "ai generation" writing mode), ai directly generates sentences or paragraphs, reducing the writer's effort significantly (we found that writers under the ai generation writing mode spend significantly less time than writers under other modes). as such, when this type of ai assistance is disclosed, raters might be resistant to giving high scores to the quality of the writings, as they may give less credit to the writers who use ai generation assistance compared to those who do not. raters may also significantly raise their expectation for the writing quality when knowing that the writers can utilize ai assistance to generate content on behalf of them. moreover, we also notice that disclosing the usage of ai assistance appears to have a greater impact on people's perceived ranking of argumentative writing than on creative writing. we attribute this to the distinct nature and objectives of these two writing tasks. argumentative writing requires constructing a logical structure and identifying appropriate evidence, tasks for which llms can easily provide support and generate logical structures. when ai usage is not disclosed, this can lead to higher ratings, as raters attribute the quality solely to the writer. however, when ai usage is disclosed, raters may partially credit the quality to the ai as-sistance, resulting in noticeably lower ratings, as reflected in the significant shift in rankings. in contrast, creative writing relies heavily on imagination to create compelling characters and plots, an area where llms still struggle to produce truly novel content. as a result, even when ai's content generation assistance is not disclosed, the perceived quality of the best ai-assisted articles in creative writing do not substantially outperform the best ones written in other modes, especially those written with ai's editing assistance. as such, after ai's content assistance is disclosed, which causes the perceived quality for both articles written with ai's content generation assistance and editing assistance to decrease, we do not see dramatic changes in the the rankings of the creative writings. design implications. our findings have important design implications. for example, for online platforms that often determine the ordering of posts based on historical users' evaluation of posts (e.g., ratings and upvotes), one challenge they face is that the prevalence of ai-assisted writings may "crowd out" and disengage writers who continue to create content without the use of ai assistance. results of our study suggest that these platforms may benefit from introducing content labels to mark those content that are partly generated by ai, as such labels may change users' evaluations of these content in relative to other content that are generated independently by humans, especially when these content have an argumentative flavor. in addition, the fact that disclosing ai assistance tends to decrease the quality perceptions of writings implies a tension that writers face between being transparent and maintaining the perceived quality of their writing. from platforms' point of view, this means that they need to design appropriate incentives for writers to be willing to disclose the ai assistance usage, and ensure no unintended disadvantage or "discrimination" for those who do. from the writers' point of view, they should still recognize their ethical responsibility to inform readers about how the writing is generated to maintain readers' trust, as misleading readers about ai involvement in the writing may erode trust in the long run. limitations. our study has a few limitations. for example, the type of ai assistance and writing tasks we considered do not capture the full range of ai assistance and writing tasks in the real world. both the collection and evaluation of writing sam-ples in our study are conducted on an online platform. participants, primarily motivated by financial payments, may not accurately reflect professional writers and evaluators in the real-world. we also limited our study to u.s. workers to control for the potential variability in english language habits or preferences that differ across countries, limiting the generalizability of findings to non-english speaking populations. future research should look into whether our findings generalize to a broader spectrum of ai writing assistance, a wider range of writing tasks, people with more diverse demographic and cultural backgrounds, and in contexts that more closely mimic real-world writing and evaluation scenarios. it is also worthwhile to note that in our study, we observed that articles completed in the ai-assisted modes, particularly those under the "ai generation" mode, received significantly higher quality ratings from raters compared to articles completed independently by participants. thus, even after the ai assistance is disclosed, participants who used the ai's content generation assistance may still receive a higher average rating of their writing quality than participants who wrote independently. however, as we noted in footnote 3, due to the design of the phase 1 study (i.e., participants self-selected into their preferred writing mode), a comparison across the three writing modes does not allow for a causal interpretation. future work should conduct a rigorously designed study to see whether this observation still holds true. ## conclusions in this paper, we ask the question of whether and how the disclosure of ai assistance would influence people's perceptions of writings. our experimental results suggest that when people are informed of the use of ai content generation assistance in the writing process, there is a significant decrease in the quality evaluation of the writing across different types of writing tasks. furthermore, the disclosure of ai assistance often leads to an increased level of variation in the perceived writing quality. we identify potential factors, such as the rater's confidence in writing and their familiarity with chat-gpt, that might moderate the effect of disclosing ai assistance on the evaluation of writing quality. additionally, our findings suggest that disclosing the use of ai assistance would also significantly reduce the proportion of ai-assisted writings among top-ranked argumentative essays.