SlowGuess/NAACL_2022_OCR huggingface dataset:
For example, P7 felt AI-generated summaries were an "authority that has given you this thing", saying that "for most people, if presented with something, they're going to go with it." As a result, users could lose confidence when they disagree with the AI. P8 shared their hesitation to dramatically edit AI-generated summaries: "it's almost feeling like you're pivoting against the AI...should I question what the AI thinks is important?" This apprehension might increase when participants are summarizing for unfamiliar or difficult documents. Specifically, some anticipated a lack of trust when summarizing challenging articles because they could not reasonably assess the AI's output: "I probably wouldn't use it for a lengthier subject that I wasn't familiar with...just because I wouldn't know if the AI was writing something I wanted to write" (P6).
To foster trust, many wanted information about how the AI generated the summaries or suggestions—why the certain information is included and whether there were any hidden presuppositions by the model. For example, P8 said, “knowing, in a very basic sense, how the AI is generating these summaries, [will] give me a good idea of essentially how much I can trust it.” As the prototypes did not include explanation features, participants noted that they did not trust the AI since they did not understand the mechanism, as P12 put: “there’s too many variables that you don’t know. Too many unknowns for me.”
3.3.2 Interaction-Specific Experience & Needs
We report participants' needs and expectations on efficiency, control, and trust when interacting with each of the five interfaces and present a conceptual comparison between the five interfaces in Figure 2.
Guiding Model Output Participants felt this prototype streamlined the summarization process, as they did not need to compose the summary themselves, but only give their preferences: as P6 said, "I don't have to do quite as much thinking...I don't have to type which takes time."
Most appreciated the control over the summary generation by adjusting parameters. For example, P8 liked the text highlighting feature, as "it gives you the amount of control in terms of being able to choose the parts that you think are important." Many envisioned using the interface to customize the summary for their target audiences. For instance, P6 imagined using it to tailor summary styles for different colleagues: "with my staff, I
Figure 2: Illustration of participants' perception on level of efficiency, control, and trust with each interaction. These conceptual level charts show a qualitative, rather than precise, comparison between interactions.
would use the short style...for my boss, I might use a longer formal summary to look a little more professional." On the other hand, some were concerned about the lack of editing control: "it doesn't have as much control as it seems. When you get to this [final] stage, you're stuck with it" (P7).
Participants felt they had a reasonable understanding of the AI mechanism in this prototype and thus could trust it. Since they could change parameters (e.g., length, style) to experiment with different aspects of the summary, they better understood the process: "we could [trust it more] maybe because I can play around with it. The long and short allows me to kind of have control" (P12).
Selecting or Rating on Model Output Most participants thought this interface could make summarization much more efficient, as they only needed to choose the best summary out of a list rather than to write or edit. Participants valued this low workload: "[it was] gonna summarize the article for me and do all the work...whatever I can use to buy out my part of the labor I am all for." (P6)
Despite the efficiency advantage, many complained about the lack of control, specifically the inability to influence or edit the AI-generated summaries. For example, some felt that choosing the best might not ensure quality: "what if three of these are presented, and none of them are really good enough. Then it's just a matter of picking the least bad one" (P7). Further, since comparing and selecting were simpler tasks than writing or editing, participants paid less attention and thought less critically: "evaluating already written summaries and trying to decide which one is the best is different from just writing your own summary...I am not like super mentally invested in it, if I were writing my own, I'd very careful with word choices" (P5).
Many struggled in selection as they did not know how the candidates were generated: "how do you determine, from an AI standpoint, what information
to keep and what information to get rid of? How do you determine the priority as what stays and what goes? Is it biased in any way?" (P14).
Post-editing Participants were mixed on how much efficiency post-editing would bring. Some thought it could make summarization faster: "I could just run it through this, and then edit it and change the things that I needed to change. It would save me a lot of time and energy" (P1). For others, editing unfamiliar text was an equally time consuming task. For instance, P8 would always ensure the summary aligns with their personal writing habits and style, and, therefore, they would spend a lot of effort in editing and customization: "I feel like it would be just as much work to just write it from scratch...if I'm trying to make it original, I have one less way of being able to word it" (P8).
Participants were satisfied with the editing control granted by this interface. For instance, P10 shared why they liked it more than the Selecting or Rating on Model Output interaction "You can edit it to however you'd like. I think the freedom to edit appeals to me a little bit more."
Similar to other prototypes, participants hoped to see more information on how the AI generated the summary. For example, P16 said it would be helpful to visually see which parts of the article were emphasized in the AI-generated summary, so that they could decide what to focus on. Similarly, P7 imagined a quantifiable way to indicate how much of the content in the summary was matched with the original article. They hoped for "some advanced algorithms checking to make sure that it did it right" to decide how much to trust it.
Interactive Editing Participants were more suspicious about this prototype's efficiency compared to the others. They worried that the dynamic updates to the summary might disrupt their existing edits, from P9: "what if I rewrite the first sentence again, and then it changes everything else. I feel
like it can really start to ripple." Real-time model updates also generate new text, creating more editing work: "with every choice that I have written here, I'm going to have another choice down here to consider whether I want to use or not. It's gonna cause me way too much work" (P12).
For control, many valued that in addition to editing, they could also experiment with different versions due to the dynamic updates. However, some viewed it pointless to iterate with the AI and would rather complete editing in a single turn: "it's giving me a choice that I don't necessarily want...I want it to be as close to a final draft as possible, because then my editorial choices are final and have the feeling of finality" (P3). Participants also worried about unpredictable AI actions that might impact their edits: "I don't have any idea what the second paragraph is going to be until I make a choice with the wording of the first paragraph" (P11); "it is kinda stressful because if you use just one different word, it's going to change the entire thing" (P15).
To ease this uncertainty, many wanted to understand how the AI updates based on their edits. P5 shared that they tended to discuss with coworkers on how certain choices were made—"every word is intentional." And, they hoped to have similar interaction with the AI, "to know the reasoning behind the changes, the kind of logic flow," so that they could make better decisions on what to edit.
Writing with Model Assistance Participants felt that writing and iteratively making improvements based on new model suggestions and comments could be tedious: "I have to go back and read what it suggests and see if it makes sense for them and for me. I just think it takes you more time to do this" (P6). Also, from P14, it "puts more back on the person writing the summary," which could introduce writer's block and stress.
Despite of the high control over the final output and whether to take AI's suggestions, participants wanted to control when they received assistance during summarization. Many viewed the autocompletion and suggestion intrusive and distracting, especially when they were not ready to receive help: "it's harder to write when you have constant changes being thrown your way" (P9). Comparing it with the Interactive Editing interface, P7 found the latter allowed more control over when AI helps: "since you're pressing a button, you still feel like you have some control. And you have control the timing too, which is important, because, what if you want
to think about your first sentence?"
Similar to other interfaces, participants also wished to know why the AI made certain suggestions, so that they could decide whether and how to follow: "I am a why person and I like to understand what I am doing. So if you're telling me to change something, you need to give me the reason" (P6). In addition, some thought auto-completion might amplify human mistakes as it was learning from their writing: "when I wrote my first sentence, I wasn't confident... And then for the bot to come in with that... it's not going to be a good summary, because I didn't know what I was writing."
4 Discussion & Design Implications
In the taxonomy, we synthesized five human-AI interactions in AI-assisted text generation. Through interviews, we surfaced user experience and needs with each interaction technique to inform future research and design. Although our user studies focused on summarization specifically, we believe our insights can be used for designing broader text generation systems. We discuss both general and interaction-specific implications as follows.
Offer the post-editing option regardless of interaction type. In general, humans like to have the "final say" on AI-generated text. Even when participants' role was choosing model output, they still wanted the option to edit to ensure quality. As such, future human-AI text generation systems should provide editing options for the final output.
Ensure customizable timing of AI-assistance in writing. We found unsolicited auto-completion and suggestions could disrupt users' writing experience. Future AI-assisted writing systems should allow users to easily turn off or adjust any automatic functions to decide when to receive help from AI. For example, users can press a button on-demand, instead of getting automatic suggestions.
Align interactive editing with user expectation. While humans in general like editing support such as dictionary or substitution suggestion, participants were more skeptical about dynamic updates in the Interactive Editing case, as AI may make big changes when they intended to make minor edits. Therefore, they desired to adjust the extent of AI-predicted updates based on their intention. Echoing literature on predictable AI systems (Daronnat et al., 2021), future interactive editing systems should consider user expectations and empower
465 8
users to preview AI actions. Systems could also model human editing intention, perhaps via action history like number and location of edits, and adapt AI actions accordingly to better serve user goals.
Support tailoring text to different audiences.
Another use case for AI-assisted writing is to tailor generated text to different audiences. For example, Systems with Guiding Model Output interaction allow humans to specify desired style, audience and use cases and generate customized text. Post editing and interactive editing systems can incorporate scaffolds that provide wording and format suggestions tailored for different scenarios.
Address trade-offs between efficiency and control. Guiding models or selecting model outputs are efficient actions, but humans have concerns about the lack of editing freedom and ownership on AI-generated text. Systems that leverage human editing power or support human writing with AI grant more flexibility, but require more effort. Users' needs in efficiency and control vary based on their goals and context. For summarization, participants felt more responsibility and thus desired more control on editing and generation when working on professional tasks or on texts that were perceived to be important (e.g., Government Bills). Future systems should consider these differences and assign different levels of controls to users accordingly. For instance, systems can classify tasks based on importance and automate more when generating texts that are less important, while inviting humans to participate more in the generation and editing process for more important tasks.
Foster appropriate trust on AI. Our findings echo literature that humans can both over- and under-rely on AI systems (Bussone et al., 2015; Buçinca et al., 2021). For example, consistent with Bhat et al. (2021), users may view AI-generated text as an authority and be conservative on making edits. Others were uncertain if the AI-generated text or suggestion is reliable, especially when working with important text. Our findings point to the need for appropriate trust on AI text generation in general. First, systems should support users to understand how the model generates text, so that they can decide whether to rely on it or not. One technique is to allow humans to participate in the model decision process. Systems can refer to the Guiding Model Output interaction and allow users to specify preferences and experi
iment with different outputs. Systems can also offer explanations to model mechanism, perhaps through visual representations (Zhang et al., 2011; Gehrmann et al., 2019). Second, systems should provide context support. Interview participants, regardless of interaction case, had issues working with AI when summarizing Government Bills, as they were unfamiliar with the format and jargon. To this end, systems should equip users with sufficient context, so that they can effectively evaluate AI suggestions and make decisions accordingly. For example, systems can offer embedded dictionaries, resource search, or user Q&A support.
Limitations. We note a few important limitations in our study. First, we scoped our paper specifically to human-AI interactions for the goal of generating text, while we did not study the needs of humans who develop models, annotate data, or consume final outputs from AI. Second, while we performed a formal systematic literature review in Study 1, we may have missed some important papers due to our sampling strategies. Therefore, our taxonomy of human-AI interaction for text generation might not cover all possible interaction types. Third, our interviews were limited in that participants performed only short-term interaction with a hypothetical AI model in predefined scenarios. Users may report different experiences when interacting with real AI models in real-world settings for longer periods of time. That said, our study is a formative first step that aims to ground future research. Future researchers and developers could design human-AI text summarization systems based on our findings and further evaluate the systems in more realistic settings, with realistic model outputs, and through large-scale experiments.
While humans are commonly asked to generate training data or evaluate final model output in text summarization, we draw attention to the potential of collaborative human interaction when working with AI. Our study first contributes a taxonomy of five types of human-AI interactions for text generation tasks. We provide insights on user experience and needs around efficiency, control, and trust for the five interactions and design implications, outlining a variety of considerations for researchers, developers, and designers working toward incorporating human users in text generation systems.
466 9
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1-13.
Sandeep Avula. 2018. Wizard of Oz: Protocols and Challenges in Studying Searchbots to Support Collaborative Search. In SIGIR 2nd International Workshop on Conversational Approaches to Information Retrieval (CAIR'18).
Sergio Barrachina, Oliver Bender, Francisco Casacuberta, Jorge Civera, Elsa Cubel, Shahram Khadivi, Antonio Lagarda, Hermann Ney, Jesus Tomás, Enrique Vidal, et al. 2009. Statistical approaches to computer-assisted translation. Computational Linguistics 35, 1 (2009), 3-28.
Advait Bhat, Saaket Agashe, and Anirudha Joshi. 2021. How do people interact with biased text prediction models while writing?. In Proceedings of the First Workshop on Bridging Human-Computer Interaction and Natural Language Processing. 116–121.
Tanner Bohn and Charles X Ling. 2021. Hone as You Read: A Practical Type of Interactive Summarization. arXiv preprint arXiv:2105.02923 (2021).
Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-21.
Daniel Buschek, Martin Zurn, and Malin Eiband. 2021. The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3411764.3445372
Adrian Bussone, Simone Stumpf, and Dympna O'Sullivan. 2015. The role of explanations on trust and reliance in clinical decision support systems. In 2015 international conference on healthcare informatics. IEEE, 160-169.
Alex Calderwood, Vivian Qiu, K. Gero, and Lydia B. Chilton. 2020. How Novelists Use Generative Language Models: An Exploratory User Study. In HAI-GEN+user2agent@IUI.
Haw-Shiuan Chang, Jiaming Yuan, Mohit Iyyer, and Andrew McCallum. 2021. Changing the Mind of Transformers for Topically-Controllable Language Generation. arXiv preprint arXiv:2103.15335 (2021).
Peng Chu and Anita Komlodi. 2017. TranSearch: A Multilingual Search User Interface Accommodating User Interaction and Preference. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2466-2472. https://doi.org/10.1145/3027063.3053262
Elizabeth Clark, Anne Spencer Ross, Chenhao Tan,
Yangfeng Ji, and Noah A Smith. 2018. Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces. 329-340.
Elizabeth Clark and Noah A. Smith. 2021. Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, 3566-3575. https://doi.org/10.18653/v1/2021.naacl-main.279
Sylvain Daronnat, Leif Azzopardi, Martin Halvey, and Mateusz Dubiel. 2021. Inferring Trust From Users' Behaviours; Agents' Predictability Positively Affects Trust, Task Performance and Cognitive Load in Human-Agent Real-Time Collaboration. Frontiers in Robotics and AI 8 (2021), 194. https://doi.org/10.3389/frobt.2021.642201
Michael Denkowski and Alon Lavie. 2012. Challenges in Predicting Machine Translation Utility for Human Post-Editors. In Proceedings of the 10th Conference of the Association for Machine Translation in the Americas: Research Papers. Association for Machine Translation in the Americas, San Diego, California, USA. https://aclanthology.org/2012.amta-papers.6
Chenhe Dong, Yinghui Li, Haifan Gong, Miaoxin Chen, Junxin Li, Ying Shen, and Min Yang. 2021. A Survey of Natural Language Generation. arXiv preprint arXiv:2112.11739 (2021).
Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016. Learning-based single-document summarization with compression and anaphoricity constraints. arXiv preprint arXiv:1603.08887 (2016).
Angela Fan, Mike Lewis, and Yann N. Dauphin. 2018. Hierarchical Neural Story Generation. CoRR abs/1805.04833 (2018). arXiv:1805.04833 http://arxiv.org/abs/1805.04833
Jonas Freiknecht and Wolfgang Effelsberg. 2020. Procedural Generation of Interactive Stories Using Language Models. In International Conference on the Foundations of Digital Games (Bugibba, Malta) (FDG '20). Association for Computing Machinery, New York, NY, USA, Article 97, 8 pages. https://doi.org/10.1145/3402942.3409599
Sebastian Gehrmann, Hendrik Strobelt, Robert Krüger, Hanspeter Pfister, and Alexander M Rush. 2019. Visual interaction with deep learning models through collaborative semantic inference. IEEE transactions on visualization and computer graphics 26, 1 (2019), 884-894.
Katy Ilonka Gero and Lydia B. Chilton. 2019. Metaphoria: An Algorithmic Companion for Metaphor Creation. Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3290605.3300526
Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight. 2017. Hafez: an Interactive Poetry Generation System. In Proceedings of ACL 2017, System Demonstrations. Association for Computational