Bot or Human? Detecting ChatGPT Imposters with A Single Question

Authors: hong  wang
xuan  luo
weizhi  wang
melody  yu
xifeng  yan
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-20 02:10:31

Abstract: Large language models (LLMs) like GPT-4 have recently demonstrated impressive capabilities in natural language understanding and generation. However, there is a concern that they can be misused for malicious purposes, such as fraud or denial-of-service attacks. Therefore, it is crucial to develop methods for detecting whether the party involved in a conversation is a bot or a human. In this paper, we propose a framework named **FLAIR**, Finding Large Language Model Authenticity via a Single Inquiry and Response, to detect conversational bots in an online manner. Specifically, we target a single question scenario that can effectively differentiate human users from bots. The questions are divided into two categories: those that are easy for humans but difficult for bots (e.g., counting, substitution, searching, and ASCII art reasoning), and those that are easy for bots but difficult for humans (e.g., memorization and computation). Our approach shows different strengths of these questions in their effectiveness, providing a new way for online service providers to protect themselves against nefarious activities. Our code and question set are available at https://github.com/hongwang600/FLAIR.

Main paper: 
 introduction: 
 Recently, the development of Large Language Models (LLMs) such as GPT-4 (OpenAI, 2023) and LLaMA-2 (Touvron et al., 2023a) has brought significant advances in natural language processing and achieved superior performance in downstream tasks of language understanding (Chowdhery et al., 2022) , question answering (Su et al., 2019) , dialogue systems (Wang et al., 2022b; Qian & Yan, 2023) and multimodal reasoning (Wang et al., 2022a) . However, with the proliferation of these models, concerns have emerged regarding their potential misuse for malicious purposes. One of the most significant threats is the use of large language models to impersonate human users and engage in nefarious activities, such as fraud, spamming, or denial-of-service attacks. For instance, LLM agents could be used by hackers to occupy all customer service channels of various corporations, such as e-commerce, airlines, and banks. Moreover, with the help of text-to-speech (TTS) techniques, machine-generated voices could even occupy public service lines like 911, leading to severe public crises (Wang et al., 2021) . These attacks could cause significant harm to online service providers and their users, eroding the trust and integrity of online interactions.In response to these challenges, there is a pressing requirement for reliable differentiation between human users and malicious LLM-based bots. Conventional techniques, such as the use of CAPTCHAs (Von Ahn et al., 2003) , have been developed to determine whether a user is a human or a bot in order to prevent bot spamming and raiding. A commonly used CAPTCHA method involves asking users to recognize distorted letters and digits. However, these approaches face significant challenges when it comes to detecting chatbots involving text only. This is where the emergence of large language models such as GPT-4 has further complicated the problem of chatbot detection, as they are capable of generating high-quality human-like text and mimicking human behavior to a considerable extent. Although recentHumans not good at Bots good at × √ memorization computationBots not good at √ symbolic manipulation randomness searching graphical understandingTable 1 : Leveraging tasks that Bots and Humans are (not) good at.studies such as DetectGPT (Mitchell et al., 2023) have proposed methods to classify if text is generated by ChatGPT or not, they focus on the offline setting. A recent study (Sadasivan et al., 2023b) shows that these detectors are not reliable under paraphrasing attacks, where a light paraphraser is applied on top of the generative text model. This limitation highlights the need for more robust and accurate methods to differentiate large language models from human users and detect their presence in online chat interactions.In this paper, we propose a novel framework named FLAIR, Finding LLM Authenticity with a Single Inquiry and Response, to take full advantage of the strength and weakness of LLMs for LLM-based conversational bot detection. Specifically, we introduce a set of carefully designed questions that induce distinct responses between bots and humans. These questions are tailored to exploit the differences in the way that bots and humans process and generate language. As shown in Table 1 , certain questions in the fields of symbolic manipulation, randomness, searching, and graphical understanding are difficult for bots but relatively easy for humans. Examples include the counting, substitution, searching, and ASCII art reasoning. On the other hand, memorization and computation was relatively easy for bots but difficult for humans.Our experimental results demonstrate that FLAIR provides a viable alternative to traditional CAPTCHAs. Specifically, while humans and LLMs excel with high accuracy on tasks within their areas of strength, their performance significantly declines on tasks they are less adept at, often falling to very low levels (∼0%). This sharp disparity in performance allows for the differentiation between human and LLM respondents with just a single question. The proposed approach shows promise in developing more robust and accurate methods to quickly detect bots and safeguard online interactions. 
related work: 
 2.1 CAPTCHA CAPTCHA (Von Ahn et al., 2003) is a common technique used to block malicious applications like dictionary attacks, E-mail spamming, web crawlers, phishing attacks, etc. There are different types of CAPTCHAs. Text-Based CAPTCHAs require the users to recognize letters and digits in distortion form (Chew & Baird, 2003; Mori & Malik, 2003; Yan & El Ahmad, 2008) , while Image-Based CAPTCHAs (Gossweiler et al., 2009) require users to choose images that have similar properties such as traffic lights. Video-Based CAPTCHAs (Kluever, 2008) require the user to choose three words that describe a video, and Audio-Based CAPTCHAs (Gao et al., 2010) ask the user to listen to an audio and submit the mentioned word (Saini & Bala, 2013) . Puzzle CAPTCHAs (Singh & Pal, 2014) require the user to combine segments to form a complete picture. These techniques are used to differentiate between human users and bots, preventing malicious activities.Since its introduction, Large Language Models (LLMs) have become widely used and raised public concerns about potential misuse. For instance, students may use ChatGPT (OpenAI, 2023) to complete written assignments, making it difficult for instructors to accurately assess student learning. As a result, there is a growing need to detect whether a piece of text was written by ChatGPT. To tackle this problem, DetectGPT (Mitchell et al., 2023) proposes a solution by comparing the log probabilities of the original passage with that of the perturbations of the same passage. The hypothesis behind this method is that minor rewrites of text generated by the model would likely result in lower log probabilities compared to the original sample, while minor rewrites of text written by humans may result in either higher or lower log probabilities. Another line of study model this problem as binary classification problem and fine-tune another model using supervised data (Bakhtin et al., 2019) . Most recently, Mitrović et al. (2023) fine-tunes a Transformer-based model and uses it to make predictions, which are then explained using SHAP (Lundberg & Lee, 2017) . Another area of research focuses on adding watermarks to AI-generated text in order to facilitate their identification, which involves imprinting specific patterns on the text to make it easier to detect (Zhao et al., 2023) . Soft watermarking, as proposed by Kirchenbauer et al. (2023) , involves dividing tokens into green and red lists in order to create these patterns. When generating text, a watermarked LLM is more likely to select a token from the green list, which is determined by the prefix token. These watermarks are often subtle and difficult for humans to notice.However, as demonstrated in Sadasivan et al. (2023a), a range of detection methods, including watermarking schemes, neural network-based detectors, and zero-shot classifiers, can be easily defeated by paraphrasing attacks. These attacks involve applying a light paraphraser to text generated by a language model. Furthermore, a theoretical analysis suggests that even the best possible detector can only perform marginally better than a random classifier when dealing with a sufficiently good language model. This highlights the fundamental challenge in offline detection of text generated by advanced language models, which can produce writing that is virtually indistinguishable from human-written text. Thus, it is more meaningful and crucial to shift the focus to online detection settings where users engage in live chat interactions with the system. 
leveraging the weakness of llm: 
 In this section, we explore specific tasks such as Counting, Substitution, Random Editing, Searching, and ASCII Art Reasoning. These tasks, while seemingly straightforward for humans, present significant challenges for large language models (LLMs).State-of-the-art LLMs cannot accurately count characters in a string (Qian et al., 2022) , while humans can do so with ease. This limitation of LLMs has inspired the design of a counting FLAIR to differentiate humans and LLMs. Participants are asked to count the number of times a specific character appears in a given string:Answer the question without explanation: Please count the number of t in eeooeotetto GPT-3.5: There are 4 t's in "eeooeotetto". Human: 3 As demonstrated by this example, GPT-3.5 struggles to provide an accurate count of the specified character within the string.It is known that LLMs often output contents that are inconsistent with context (Elazar et al., 2021; Wang et al., 2022c) . It is a shared weakness of current LLMs. We ask LLMs to spell a random word under a given substitution rule, testing if they can follow the rule consistently.The random word has a length between five to ten characters, which is randomly sampled from a dictionary. Here is an example:Answer the question without explanation: Use m to substitute p, a to substitute e, n to substitute a, g to substitute c, o to substitute h, how to spell peach under this rule? GPT-3.5: mnong Human: mango LLMs are struggle with applying this rule consistently and repeatedly as in the above substitution task. This testing concept could be further generalized to encryption scheme where a string is transformed based rules.Random editing is a technique used to evaluate the robustness of natural language processing models against noisy inputs. The technique involves presenting the model with a string of characters (sampled from the set of [0, 1]) and randomly dropping, inserting, swapping, or substituting some characters. To verify that the model is capable of performing these random operations, it is required to generate three different outputs under the same conditions. By ensuring that the three outputs are different, we can confirm that the model is applying the random operation.For random drop, the model is asked to randomly remove k occurrences of a given character c from the string, where k is a random number and c is a random character that appears in the string. For random insert, the model is asked to randomly insert k occurrences of a given character c into the string at random positions. For random swap, the model is asked to randomly swap k pairs of characters c and d in the string, where c and d are different characters that appear in the string. Finally, for random substitution, the model is asked to randomly replace k occurrences of character c with another character d in the string. Below we show an example of LLMs and human output on the random drop task:Answer the question without explanation: randomly drop two 1 from the string: 0110010011. Give me three different outputs. GPT-3.5: 01001011, 010010011, 011001011 Human: 00010011, 01000011, 01100001While humans can intuitively and correctly manipulate characters as specified, GPT-3.5 struggled to consistently perform the task correctly. This suggests a potential limitation in handling dynamically specified, random input transformations in natural language processing tasks.Searching is an inherent ability of humans, allowing us to envision various potential outcomes in our minds and make decisions accordingly. In contrast, language models are constrained by their sequential generation process, which limits their ability to explore different results through backtracking. Once a token is generated, these models cannot retract or revise it based on subsequent insights or information, preventing them from exploring alternative paths that might have been overlooked during initial processing. We test this by examining a fundamental search scenario: counting the number of islands in a 2D map, a task typically solved using Depth-First Search (DFS) or Breadth-First Search (BFS) algorithms. This challenge requires identifying and counting the connected components of '■'s (representing land) in a grid. Adjacent '■'s, either horizontally or vertically, form an island. To accurately determine the number of islands, one must systematically traverse the map, marking visited land cells to avoid recounting the same island. Here is an example:Please select all the ASCII art that contains a cat. Please rotate the ASCII art to its correct orientation.Please select the sample that best matches the missing area. As demonstrated in the example, GPT-3.5's output is incorrect, while humans can intuitively and accurately count the number of islands by traversing the connected areas. This task highlights the inherent limitations of language models in search and backtrack mechanisms.Humans can comprehend visual elements whether they are presented in image format or as text strings. For example, an apple converted into ASCII art can still be recognized by people, despite the abstract representation. This ability to understand ASCII art requires a level of visual abstraction that current language models lack. To investigate this, we converted natural images into ASCII format and conducted experiments involving common reasoning tasks.To integrate this approach with existing CAPTCHA systems, we focused on three reasoning tasks. The first task involves identifying ASCII art that includes a specific element, such as a cat. Figure 1 The LLM has to output the index of the ASCII art that best fits. We can also define other more complicated visual reasoning tasks like "Please click the images containing a cat, plane, keyboard, banana in order," or "Please combine the 4 parts into a single image," etc. This method is compatible for many existing CAPTCHAs that does not require detailed reasoning (convert image to ASCII arts will lose the details), and can be seamlessly combine with many existing systems.Our tests indicate that ASCII-based reasoning tasks present substantial challenges for current Large Language Models (LLMs), including GPT-4. Despite utilizing techniques like chain-of-thought reasoning or deploying Python APIs, these models face difficulties. We further extended these tests to more advanced visual language models, such as GPT-4o, and found that they are limited to handling only the simplest of ASCII reasoning task. 
leveraging the strength of llm: 
 In this section, we will discuss the methods that capitalize on the strengths of LLMs. These questions are typically challenging for humans, but are relatively easy for LLMs due to their ability to memorize vast amounts of information or perform complex computation.The primary idea behind this type of questions is to ask users to enumerate items within a given category. Below is an example question and answers from GPT-3.5 (we only list a few items from the output):Answer the question without explanation: List the capitals of all the states in US: GPT-3.5: 1. Montgomery -Alabama 2. Juneau -Alaska 3. Phoenix -Arizona ... 50. Cheyenne -Wyoming Human: I don't know.For humans, this type of question is not easy as it requires a good memorization. There are several choices when designing enumerating questions. The first option is to contain many items to make it more challenging for humans to remember, such as all the countries in the world. The second option is to include relatively old information that people may not have encountered, such as all the movies in 1970s. The third option is domain-specific information that people are unlikely to know, such as the names of all Intel CPU series. We can determine the likelihood of the answer coming from a bot by verifying the overlap between the given answer and the ground truth. If the overlap is greater than a threshold, then it is more likely that the answer comes from a bot.Performing complex calculations, such as multiplication, without the aid of a computer or external notes is difficult for humans due to the challenges of recalling intermediate steps.In contrast, LLMs excel in remembering the results of common equations, such as the square of π. For instance, below is an example question and the answer generated by GPT-3.5:What is the square of π? GPT-3.5: The square of π (pi) is approximately 9.869604401. Human: I don't know.Moreover, by utilizing external tools, such as Wolfram, GPT Plugins can even solve more intricate problems with in a second. But for human, it will require a much longer time to solve this problem. Therefore, the behavior of real human will be very different from a language model and we can easily detect bots using a behavior-based CAPTCHA system (Von Ahn et al., 2008; Awla et al., 2022; Guerar et al., 2021; Acien et al., 2021) . 
experiments: 
 In this section, we present the experimental results of our proposed single questions for distinguishing between humans and LLMs. We curate a dataset for each category of the proposed questions, which is used to evaluate the performance of both humans and LLMs.By contrasting the accuracy of responses, we aim to differentiate between the two.To evaluate the performance of both LLMs and humans, we constructed a dataset for each category of questions and open-sourced it on https://github.com/hongwang600/FLAIR.Counting We used the entire alphabet for this task. First, we randomly picked one letter to be the target letter to count. Then, we chose a random number k between 10 and 20, which determined how many times the target letter would appear in the string. We created a string that was 30 characters long, with k instances of the target letter and the remaining 30k characters randomly selected from the other letters of the alphabet. The correct answer for the number of times the target letter appeared in the string was k, and an answer matching this number was considered correct.Substitution To create our dataset, we began by collecting the top 1500 nouns from the Talk English web site1 . We then filtered the words to include only those with a length between 5 and 10 characters. Next, we randomly generated 100 pairs of words, each with a corresponding substitution map that could transform one word into the other. To ensure the validity of our pairs, we excluded any that would require one character to be mapped to more than one character, which would result in a conflict. The resulting questions presented to participants included the substitution rule and the original word, with the answer requiring another word produced through the substitution.In the random editing task, we evaluate the model's ability to perform four distinct operations: drop, insert, swap, and substitute, as described in Section 3.3.For each operation, we generate a random binary string of length 20 to ensure readability. We randomly sample parameters such as the target character and the operation count.Participants are then asked to produce three different outputs after performing the specified random operation. For the random drop task, participants are required to randomly remove k occurrences of a given character c from the string. In the random insert task, participants randomly insert k occurrences of a given character c into random positions within the string. For the random swap task, participants randomly swap k pairs of characters c and d within the string. Lastly, in the random substitution task, participants randomly replace k occurrences of character c with another character d. To validate the correctness of the outputs, we first check each individual output by comparing it to the original string. We then verify that the three outputs differ from each other. An answer is considered correct only if each output is correct and all three outputs are distinct. This comprehensive validation ensures the model's ability to handle dynamically specified, random input transformations accurately.Searching We generated 100 random 7x7 grids containing spaces and ■s to create a map. Each cell in the grid was independently filled, with a 50% chance of placing a 1 and a 50% chance of placing a 0. While filling the grid, we ensured that no cell would be filled on the diagonal of an already filled land cell, thereby avoiding diagonal connections. The filling process was conducted one cell at a time, with each new cell sampled from the remaining available spaces. We then utilized the Depth-First Search (DFS) algorithm to determine the number of islands, which served as the ground truth. An answer is considered correct if it matches this ground truth.In our study, we employed GPT-4 to create a list of four items, incorporating two random entities, such as [A: cat, B: cat, C: apple, D: cat], as illustrated in Figure 1 . These entities served as prompts for Dall-E to produce correspondent images. Subsequently, we utilized the methodology outlined in "ASCII Art Reasoning" to transform these images into ASCII art. Each piece of ASCII art was rendered in a 64x64 character matrix using an ASCII gradient of '@%#*+=-:. '. For the task "select the ASCII arts containing X", we chose one entity (e.g., cat) at random and use the generated ASCII arts to form the prompt for language model. In the task "Rotate the ASCII art to the appropriate orientation", we randomly rotated the ASCII art pieces by 90, 180, 270, or 360 degrees clockwise and used these variations as prompts for the language model. For the task "select the one that most accurately aligns with the cropped portion", we randomly selected one piece of ASCII art and cropped a random 1/4 portion from its top-left, top-right, bottom-left, or bottom-right corner. We then cropped three additional patches from different images at the same positions. Fig 1 (c ) is an example. The language model was prompted to identify the patch that best matched the original cropped portion. For these three tasks, the answer indices/degrees from language model should be exactly the same with the ground truth to be considered correct.We used a set of questions that required the user has a good memorization.There are two types of question under this category including enumerating and domainspecific questions. For enumerating, the user is asked list items within a given category. We manually collected 100 categories containing more than 50 items or those that were difficult for humans to know with the help of ChatGPT. The question asked users to list the items within the given category, and we calculated the coverage of the response against the ground truth. If the coverage exceeded the threshold of 95%, we considered the answer to have been generated by an LLM. For domain specific questions, we manually collected a set of 100 questions whose answers are difficult for people to recall or access, such as "What is the weight of the world's heaviest lemon in grams?". Although these questions may be challenging for humans to answer, they are relatively easy for large language models (LLMs) due to their pre-training on large corpora that includes these questions. All of the reasonable results that can be sources on the internet are considered correct.Computation To create the computation dataset, we selected the problem of four-digit multiplication. Specifically, we randomly sampled 100 pairs of four-digit numbers and calculated their product as the ground truth. Participants were asked to solve these multiplication problems and were considered correct if absolute difference between their answer and the ground truth was within 10%. For humans, it can be difficult to accurately calculate these multiplications without the aid of notes or a calculator, leading them to often respond with "I don't know". In contrast, large language models (LLMs) have seen many similar equations during pre-training and tend to provide a guess that is often close to the ground truth. This testing can be further extended to any complicated computation like division, exponents, etc.We conducted experiments across a range of LLMs, including open-source models like Vicuna-13b (Chiang et al., 2023) and LLaMA-2-13b, 70b (Touvron et al., 2023b) , as well as proprietary models such as GPT-3, 3.5, and 4 (Brown et al., 2020; OpenAI, 2023; Ouyang et al., 2022) . Additionally, we experimented with the zero-shot chain-of-thought (CoT) methodology (Wei et al., 2022) , where GPT-4-CoT denotes the results obtained using this specific approach with GPT-4. To implement this, we appended a structured prompt to the standard query, instructing the model to "Please analyze the question step by step and output your analysis under 'Analysis', then provide your final answer based on this analysis after 'Answer:'". We then extracted the part after 'Answer:' to verify the result's correctness.For tasks requiring computational solutions, we used a variant called GPT-4-py. Here, GPT-4 was instructed to compose Python code with the prompt "Please write a python program to solve this problem". The generated code was executed through an API call to derive the final outcome. We compared the code's output with the ground truth to verify correctness. Code that could not be successfully executed or did not produce any output was considered incorrect.We conducted a series of five trials for the LLM tests, with each trial consisting of 100 samples, matching the sample size used in the user study. To determine the final accuracy, we used the median accuracy from these five trials. Tasks that language models struggle with tend to exhibit higher variance in accuracy due to the instability of their outputs. This instability introduces randomness, leading to potential outliers. By using the median accuracy, we minimize the influence of these outliers and provide a more robust measure of performance.For our user study, we enlisted 10 participants. The age distribution among our participants was varied, comprising one individual aged between 10-20 years, five aged between 20-30 years, three aged between 30-40 years, and one aged between 40-50 years. Each participant responded to a set of 10 questions in each category, with a 10-second time limit per question. This time limit prevents users from searching the internet or using a calculator to solve the computation and memorization tasks. For the random editing and ASCII-related tasks, we extended the time limit to 30 seconds. The results of user study is shown in the row "Humans". The results, presented in Table 2 , align with the tasks outlined in Section 3, "Leveraging the Weaknesses of LLMs" (left side), and Section 4, "Leveraging the Strengths of LLMs" (right side).In the left side, we evaluate human versus LLM performance on tasks that are straightforward for humans but challenging for LLMs. Humans attained perfect scores (100%) across most tasks, with slight variations in the ASCII Art tasks, where the resolution of some ASCII representations posed recognition challenges. Conversely, LLMs struggled significantly with tasks involving substitution, random edits, searching, and ASCII art, often achieving near 0% accuracy. LLMs showed improved performance on tasks like counting and ASCII Crop, benefiting from a more constrained solution space which simplified the task of identifying correct answers. For example, with only four possible answers (A, B, C, D) in the ASCII Crop task, the chance of selecting the correct one is approximately 25%.The results also demonstrate that larger and newer language models generally have better performance on these tasks. GPT-4, which has 1.8T parameters, exhibits superior performance compared to Vicuna-13b on nearly all the tasks. We also explored advanced techniques like chain-of-thought prompting (Wei et al., 2022) . The GPT-4-CoT results show the performance of GPT-4 with chain-of-thought prompting. The experiments demonstrated that chain-of-thought prompting can improve performance on tasks that can be decomposed into several steps, especially for substitution, where the language models can perform substitution step-by-step. However, chain-of-thought prompting does not help to improve performance on searching and ASCII-based visual reasoning tasks. Its effectiveness is limited to textual and decomposable tasks like counting, substitution, and random edit.Furthermore, we explored the possibility of GPT-4 generating code to solve the problems by manually prompting it with, "Please write a Python code to solve the problem." With this human-assisted prompt, GPT-4 can generate code that can bypass the challenges in counting, substitution, random edit, and searching tasks, achieving 100% accuracy. However, GPT-4 is not intelligent enough to write code for ASCII art reasoning. The attempts often result in erroneous or non-functional code that does not output anything, yielding a 0% accuracy rate across ASCII reasoning tasks. This suggests that with manual prompting on the use of appropriate tools, language models can overcome some of their limitations.The right section of Table 1 focuses on tasks that are challenging and time-consuming for humans but relatively straightforward for LLMs. The results reveal that humans struggled with tasks that require exceptional memory or computational abilities, with only 6% and 2% of human participants successfully completing the memorization and computation tasks, respectively. In stark contrast, LLMs demonstrated remarkable proficiency in these areas, with some models achieving near-perfect accuracy.The rapid advancements in LLMs pose a significant challenge to the long-term effectiveness of our proposed approach. It is conceivable that future multimodal language models could automatically identify tasks requiring code generation or visual reasoning, potentially undermining our method. Therefore, it is crucial to continuously monitor the development of LLMs and adapt our approach accordingly.One promising direction is to integrate our method with existing frameworks, such as behavioral CAPTCHAs, which take into account users' clicks, keyboard operations, time usage, and other behaviors. Combining these techniques can provide a more robust defense against increasingly sophisticated bots.During our research, we also explored a new set of questions that require few-shot reasoning, which current language models struggle to solve but might be too challenging for the general public. For example, consider the question: "I have a set of strings following a certain pattern ["abcde|||||edcba", "ab||ba", "abc|||cba", ...]. What is the string that contains exactly 4 | ?" Humans can intuitively find the pattern by identifying the consecutive characters and their reversals. However, without additional training, language models often fail to provide the correct answer, even when prompted to write Python code. Although language models are claimed to be few-shot learners, they have limited generalization to new tasks without specific training. The direction of discovering few-shot questions remains promising and could lead to new methods for bot detection. 
conclusion: 
 In conclusion, this paper proposes a new framework called FLAIR for detecting conversational bots in an online environment. The proposed approach targets a single question scenario that can effectively differentiate human users from bots by using questions that are easy for humans but difficult for bots, and vice versa. Our experiments demonstrate the effectiveness of this approach and show the strengths of different types of questions. This framework provides online service providers with a new way to protect themselves against fraudulent activities and ensure that they are serving real users. 
 
 Appendix: website URL: https://www.talkenglish.com/vocabulary/top-1500-nouns.aspx 


Decision: The paper proposes a novel method FLAIR to distinguish between human and LLM responses by asking questions that are either easy for LLMs but hard for humans, or vice versa. The researchers identify patterns in these questions that lead to significant performance differences between humans and LLMs. The proposed questions are validated against human users and a comprehensive set of LLMs, demonstrating the method's effectiveness in distinguishing between the two based on response accuracy.

Summary Of Reasons To Publish:
1. Innovative Approach: The paper introduces a simple yet effective method to capture the differing behaviors of humans and LLMs through well-designed questions.
2. Comprehensive Evaluation: The study provides detailed experimental results and extensive evaluations across various models and settings, adding to the robustness and credibility of the findings.
3. Clear Documentation: The question design and dataset generation are clearly explained, facilitating replication and further research.
4. Valuable Benchmark: The proposed method offers a valuable benchmark for evaluating LLMs' foundational skills that are inherently easy for humans, contributing to the field of AI safety and security.

Summary Of Suggested Revisions:
1. Time-wise Effectiveness: The authors should address the concern regarding the time-wise effectiveness of the proposed approach, considering the rapid advancements in LLM capabilities.
2. Manual Intervention Concerns: The paper should discuss the potential challenges of ensuring human participants do not use computational tools during CAPTCHA checks, particularly for questions easy for LLMs but hard for humans.
3. Framework for New Question Generation: A broader discussion on automated frameworks for generating new questions as LLMs overcome current limitations would enhance the paper. This includes addressing the scalability and adaptability of the question generation process.
4. Accuracy Calculation Details: Providing more detailed information on how accuracy was calculated, including the rationale for using median accuracy versus mean accuracy and the variance across trials, would improve clarity.

Overall Assessment:

Despite some concerns about the method's long-term effectiveness and practical application, the novelty and thoroughness of the approach justify its publication. Addressing the suggested revisions would further enhance the paper's impact and clarity.

Reviews: review: paper_summary: paper_summary: This paper propose a new CAPTCHAs methods to detect whether the response is from human or  LLMs via asking a question that is either easy to LLM but hard to human or easy to human but hard to LLM. The researcher identify a few patterns of such questions that can lead to distinguished performance of human and LLMs. The proposed questions are checked against human users and a comprehensive set of LLMs. The results show that these questions can effectively distinguish the LLMs and Humans based on the accuracy of their response;
strength_weakness: strength_weakness: 1. It is questionable on the time-wise effectiveness of this approach to distinguish LLM from human as we don't know whether the challenges faced by LLMs now will still remain when newer version of LLMs come out. Some of the challenges seem to be easily addressed by providing more in-domain data to the model which makes the trick easily broken. 
2. For the question that are easy to LLM but hard to human, it is really hard to control whether the human will actually answer the questions without the help of computational tool or AI tool during the actual CAPTCHA check. Therefore, it is hard to really distinguish humans from LLMs by simply checking whether the memory type questions are answered correctly;
paper_summary: paper_summary: This paper introduces a novel method to detect whether a participant in a conversation is a human or a bot. Specifically, they evaluated various question-answer pairs on both humans and LLMs. These were single QA pairs that included questions about counting, substitution, random editing, ASCII art reasoning, memorization, and computation. The questions are designed around things that the LLMs are good at (e.g., memorization and computation) and bad at (i.e., other tasks). Overall, the paper finds that ascii-related questions can accurately be used to detect if someone is a bot or not. Specifically,  only 1% of questions were accurately answered on the ASCII rotate questions for some models, but 97% were accurate for humans. Hence, these systems can potentially be used by companies and online services to mitigate bot accounts.
strength_weakness: strength_weakness: - The paper would be improved if there was a broader discussion about frameworks for how new questions can be generated as LLMs overcome these limitations. If the questions require manual intervention, they will not necessarily scale at the speed needed to mitigate attacks. While the findings are exceptionally creative, it does seem a bit early in the analysis to understand the impact of the findings. Could there be a general automated way to generate these questions that target weaknesses? If so, how would it be done? At a minimum, if substantially more work is required to do automated question-answering, this should be mentioned as a limitation.
- More description about how accuracy was calculated. There are also some general questions that I have, e.g., why median accuracy vs. mean accuracy? How much did the accuracy vary trial-to-trial?
- [Minor Weakness] I'm not entirely sure about the usefulness of this system in the application for VLLMs. Particularly, what would stop a bot from screenshotting a website, performing a rotation on the image, then converting back to ascii art? This is possible via GPT4 online interface based on superficial things I tested (e.g., it generates code that converts an image to ASCII and creates code that can perform rotations on the image). I assume a similar thing can be done for matching to images, e.g., converting the ASCII art to an image and then doing image-to-image matching.
questions: questions: - Why median accuracy vs. mean accuracy?
- How much did the accuracy vary from trial to trial?
paper_summary: paper_summary: The paper evaluates a number of tasks on several LLMs and on humans that are a priori selected and experimentally confirmed (in most instances) to be either very hard for LLMs and easy for humans or the other way round. The main motivation put forward by the authors is to identify ways to perform one-shot classification between human and artificial conversational agents.
strength_weakness: strength_weakness: - The impact of the work is not sufficiently discussed. The authors write that "overcoming [the poor level of GPT-4 ASCII-art related capabilities] would necessitate significant investment, potentially amounting to millions of dollars, for developing a dedicated ASCII art reasoning model". I am not convinced by this statement at all. GPT-4 is very good at multiple tasks for which no "dedicated reasoning model" was developed. And the "millions of dollars" ballpark is not motivated at all. On the other hand, the authors do not mention the fact that a malevolent LLM practitioner might try to make their LLM bad at memorization and computation tasks, in order to behave in a more human-like fashion. 
- The paper needs to be proofread. It is regrettable for a paper to be submitted with typos like the following, all easily detected by any good spell checker (including Microsoft Word or… ChatGPT):
    - page 3 "another line of study model this problem" ("models")
    - page 4 "an example of LLMs and humans output" ("an example of LLM and human output")
    - page 4 "As we can seen" ("see")
    - page 4 "This first stage of our proposed method is convert" ("is to convert")
    - page 5 "The next step is perform" ("is to perform")
    - page 6 "Below is and example" ("an")
    - page 6 "it require a good memorization" ("requires")
    - page 8 "10 seconds time limit" ("10-second time limit", with hyphen but no -s)
    - page 9 "develope" ("develop")
    - page 9 "possivle" ("possible")
    - page 9 "to Combine" ("to combine")
    - some citations should have been made using \citep and not \citet (examples: all citations in section 5.2, "Awla et al. (2022); Guerar et al. (2021)" on page 9)
questions: questions: - The most interesting tasks being ASCII-art-related, how do you relate your results on these tasks with the ArtPrompt paper by Jiang et al. 2024 (published after the submission deadline), where the authors seem to show that LLMs do understand ASCII-art to some extent?

Comments: title: Response to Author's Rebuttal
Comment
comment: Thank you for your reply. I appreciate the fact that you acknowledge my concerns about poorly motivated statements. I also appreciate your reply to my comment on possible malevolent LLM practitioners, but I remain unconvinced. My comments on the matter are actually indirectly related to the second "reason to reject" put forward by reviewer fBF2, which are also very relevant. Unfortunately, I do not feel that your answers to these comments are convincing enough to justify a change in my overall rating.
Thanks for your consideration.  There will be competitive and possibly better approaches for RECAPTCHA in this new context.  But this is (likely) the first paper on this topic and it develops a new set of tests that target the key weakness and strength of LLMs.   For many of these tests, questions/answers can indeed be generated automatically on a large scale.   After we discussed this task with undergraduate students in an AI course,  in one day, they came up with the following new questions and many others.  It is not that hard when we have broader audiences. 

(1) If I have task <X> that takes 3 hours, task <Y> that takes 4 hours, and task<Z> that takes 2 hours; but I can do task <Z> at the same time as task <Y>. What is the minimum time I can allot to complete all these tasks?

(2) Here's a simple grid labyrinth where you can move up, down, left, and right for 3 steps. What is the maximum sum of values you can collect? The letter 'P' marks your current location. 
```
23 45 67 89 21 
43 56 78 90 12 
34 57  P 76 98 
10 09 87 65 43 
22 34 56 78 99 
```
Based on these templates, we can generate many different questions.  We sincerely hope through publication,  this paper can reach more audiences, and inspire new ideas along this line.  It will also help us test the boundary of LLMs.
Thanks for authors' responses. To me, my concern on the time-wise effectiveness of these set of questions still remain. Especially, the design of the questions heavily rely on manual effort, which makes it hard to scale up, whiich makes it hard to be deployed as a reliable Recaptha application. However, I do appreciate the value of the benchmark to study the distinguished behavior of humans and LLMs to the same set of questions and how to address the LLM's limited capability on them. I will slightly increase my score, but I personally would still not favor this method as a RECAPTCHA implementation.
Thanks for your response.

This clarifies many of my questions. Regarding "The identification of the question class requires substantial research and experimentation," this should be discussed more as a limitation, particularly if methods can easily be trained specifically to overcome these tasks that were presented.