Human- AI Interaction in the Age of LLM s (2024)

Abstract: Recently, the development of Large Language Models (LLMs) has revolutionized the capabilities of AI systems. These models possess the ability to comprehend and generate human-like text, enabling them to engage in sophisticated conversations, generate content, and even perform tasks that once seemed beyond the reach of machines. As a result, the way we interact with technology and each other — an established field called “Human-AI Interaction” and have been studied for over a decade — is undergoing a profound transformation. This tutorial will provide an overview of the interaction between humans and LLMs, exploring the challenges, opportunities, and ethical considerations that arise in this dynamic landscape. It will start with a review of the types of AI models we interact with, and a walkthrough of the core concepts in Human-AI Interaction. We will then emphasize the emerging topics shared between HCI and NLP communities in light of LLMs.

## introduction recently, the development of large language models (llms) has revolutionized the capabilities of ai systems. these models possess the ability to comprehend and generate human-like text, enabling them to engage in sophisticated conversations, generate content, and even perform tasks that once seemed beyond the reach of machines. as a result, the way we interact with technology and each other -an established field called "human-ai interaction" and have been studied for over a decade -is undergoing a profound transformation. this tutorial will provide an overview of the interaction between humans and llms, exploring the challenges, opportunities, and ethical considerations that arise in this dynamic landscape. it will start with a review of the types of ai models we interact with, and a walkthrough of the core concepts in human-ai interaction. we will then emphasize the emerging topics shared between hci and nlp communities in light of llms. ## tutorial outline this will be a three-hour tutorial devoted to the cutting-edge topic of human-ai interaction in the age of llms. each theme will take 35 minutes, followed by 10 minutes for q&a and 10 minutes for a break. each part includes an overview of the corresponding topics, and a deep dive into a set of representative studies. we will conclude our tutorial by highlighting challenges and research opportunities in the field. ## tutorial presenters diyi yang is an assistant professor in the computer science department at stanford university. her research focuses on human-centered natural language processing and computational social science. diyi has organized four workshops at nlp conferences: widening nlp workshops at naacl 2018 and acl 2019, casual inference workshop at emnlp 2021, nlg evaluation workshop at emnlp 2021, and shared stories and lessons learned workshop at emnlp 2022. she also gave a tutorial at acl 2022 on learning with limited data, and a tutorial at eacl 2023 on summarizing conversations at scale. diyi and sherry have co-developed a new course on human-centered nlp that has been offered at both stanford and cmu. sherry tongshuang wu is an assistant professor at the human-computer interaction institute, carnegie mellon university. her primary research investigates how humans (ai experts, lay users, domain experts) interact with (debug, audit, and collaborate) ai systems. sherry has organized two workshops at nlp and hci conferences: shared stories and lessons learned workshop at emnlp 2022 and trust and reliance in ai-human teams at chi 2022 and 2023. she will give a tutorial at emnlp 2023 on designing, learning from, and evaluating human-ai interactions. marti a. hearst is a professor and the interim dean for the uc berkeley school of information. she is both an acl fellow and a sigchi academy member, and former acl president. her research has long combined hci and nlp; recent projects include adding interactivity to scholarly documents and creating interactive newspods. she recently gave invited keynote talks at the eacl nlp + hci workshop, the kdd workshop on data science with a human in the loop, and she advised the 2022 naacl program chairs on the human-centered natural language processing special theme. she has taught courses in nlp, hci, and information visualization for 25 years. ## diversity considerations the topic of human ai interaction will be inclusive to both nlp and hci communities. we will make our tutorial materials digitally accessible to all participants. during the tutorial sessions, we will work with student volunteers to encourage open dialogue and promote active listening, allowing participants to share their thoughts and experiences without fear of judgment. after the tutorial, we will actively collect feedback to identify areas for improvement related to diversity and inclusion and share it with future tutorial presenters. our presenter team will share our tutorial with a worldwide audience by promoting it on social media, and to diverse research communities. our presenters include both junior and senior researchers. thus, we have diversified instructors which will also help encourage diverse audience. diyi has experience co-organizing widening nlp workshops at both naacl and acl, and actively works on inviting undergraduate students to research and promoting diversity such as by speaking at ai4all and local high-schools at atlanta. we will work with acl/naacl d&i teams, and consult resources such as the big directory to diversify our audience participation. ## reading list and prerequisite the tutorial is targeted toward nlp researchers and practitioners working with humans. the prerequisite includes familiarity with basic knowledge of nlp and language systems. knowledge of system deployment is a plus. we will also provide a more paced introduction to some materials. here are a few papers that lay a foundation for this area: • re-examining whether, why, and how human-ai interaction is uniquely difficult to design @xcite ; • does the whole exceed its parts? the effect of ai explanations on complementary team performance @xcite ; • principles of mixed-initiative user interfaces @xcite ); • guidelines for human-ai interaction @xcite ; • supporting peer counselors via ai-empowered practice and feedback @xcite • evaluating human-language model interaction @xcite breadth while we will give pointers to dozens of relevant papers over the course of the tutorial, we plan to cover around 7-8 research papers in close detail. only 1-2 of the "deep dive" papers will come from the presenter team. ## ethics statement given its strong emphasise on human ai interactions, our tutorial provides insights into the intricate relationship between humans and ais (e.g., llms). in our tutorial, we will provide discussions regarding the capabilities and limitations of llms, as well as potential ethical challenges that they might pose, such as around bias, harm and fairness. our conclusion session will also discuss responsible research design in the space of human-ai interaction, and best practices that can encourage ethical and inclusive uses.