Evaluating How Users Game and Display Conversation with Human-Like Agents (2022)

Abstract: Recently, with the advent of high-performance generative language models, artificial agents that communicate directly with the users have become more human-like. This development allows users to perform a diverse range of trials with the agents, and the responses are sometimes displayed online by users who share or show-off their experiences. In this study, we explore dialogues with a social chatbot uploaded to an online community, with the aim of understanding how users game human-like agents and display their conversations. Having done this, we assert that user postings can be investigated from two aspects, namely conversation topic and purpose of testing, and suggest a categorization scheme for the analysis. We analyze 639 dialogues to develop an annotation protocol for the evaluation, and measure the agreement to demonstrate the validity. We find that the dialogue content does not necessarily reflect the purpose of testing, and also that users come up with creative strategies to game the agent without being penalized.

## introduction open-domain dialogue (odd) with conversational agents has been considered as the essence of artificial intelligence (ai). it's a topic of great interest in both academic and industry circles, directly linking the technology and end users. in addition, due to its interactive properties, odd users often provide product feedbacks voluntarily through a range of channels, which are crucial for the further development of services. with the recent emergence of high-performing language models, conversation with chatbots has become increasingly popular. accordingly, various metrics have been introduced to evaluate whether the dialogue has been performed successfully @xcite . however, there has been a paucity of studies to evaluate how users perceive and react to such ai. @xcite quantitatively scrutinize how users perceive humanlike ai devices, but does not reveal the detail on the aspect of human-ai interactions, such as what they talked about and how the users responded. @xcite tackle the offensiveness users show towards human-like agents, but the analysis is based on a questionnaire, which may not fully cover the user dialogue in-the-wild. observing users' responses to a chatbot is critical for creating human-centered chatbots. designers and developers will be able to build safer and more responsible ai models and agents by predicting users' behavior in advance @xcite . recently in korea, a highly human-like agent called 'luda' caught attention with its highquality dialogue generation, and a conversation with the agent created a sensation among general users @xcite ). luda's persona is a female college student in her twenties, and is designed to generate real-like, messenger-styled responses. as a result, soon after being launched, an online space was created to share and enjoy the agent's responses with other users. though the service was prematurely shut down due to several unexpected ethical issues @xcite , we decided to see how the end-users reacted to humanlike ai responses. thus, we started by crawling the posts containing screenshots of dialogue with luda uploaded to the online community, which exhibited a wide range of curiosity towards the human-like agent. while there were cases in which users communicated with luda as if she were a lover or a friend and show their affection even when posting, we also frequently observed various verbal attacks on the agent (3.1) and hostile gaming attempts (3.2). our contribution to dialogue analysis and user behavior study is as follows: ## thematic coding we proceed with the following two annotation processes using the corpus constructed in section 2. four research scientists from linguistics and human-computer interaction (hci) backgrounds participated in the annotation. three researchers proceeded with annotation following the draft guideline foot_2 , and after discussion including the other researcher, the final guideline was confirmed through four times of iterations accompanying relabeling and guideline updates. in this process, the categories and labels were subdivided and augmented if necessary. ## conclusion in this paper, we suggested an annotation protocol for categorizing aspects of users gaming and exhibiting conversations with human-like agents, and calculate the agreements for the proposed attributes. from the results, we observed that it is slightly more challenging to discern the latent user intention compared to analyzing the content of the conversation, although both help studying social chatbots. investigating user content quantitatively shows the type of dialogue that actually takes place and helps data analysis post-mortem. however, by discerning user intention, we obtain an index that can be used more promisingly than merely observing the content, which can also be adopted in future chatbot design from the perspective of product serving and user study. we believe that the proposed protocol, which allows to categorize and quantify user content and intention, can play a significant role in analyzing user feedback and behavior for human-like agents.