Bot or Human? Detecting ChatGPT Imposters with A Single Question
Authors: Hong Wang
Xuan Luo
Weizhi Wang
Melody Yu
Xifeng Yan
Venue: COLM (2024)
Keywords: ChatGPT imposter detection; large language models
TL;DR: In this paper, we propose a framework named FLAIR, Finding Large language model Authenticity via a single Inquiry and Response, to detect conversational bots in an online manner

Abstract: Large language models (LLMs) like GPT-4 have recently demonstrated impressive capabilities in natural language understanding and generation. However, there is a concern that they can be misused for malicious purposes, such as fraud or denial-of-service attacks. Therefore, it is crucial to develop methods for detecting whether the party involved in a conversation is a bot or a human. In this paper, we propose a framework named **FLAIR**, Finding Large Language Model Authenticity via a Single Inquiry and Response, to detect conversational bots in an online manner. Specifically, we target a single question scenario that can effectively differentiate human users from bots. The questions are divided into two categories: those that are easy for humans but difficult for bots (e.g., counting, substitution, searching, and ASCII art reasoning), and those that are easy for bots but difficult for humans (e.g., memorization and computation). Our approach shows different strengths of these questions in their effectiveness, providing a new way for online service providers to protect themselves against nefarious activities. Our code and question set are available at https://github.com/hongwang600/FLAIR.

Published as a conference paper at COLM 2024
Bot or Human? Detecting ChatGPT Imposters with A Single Question
Hong Wang', Xuan Luo', Weizhi Wang', Melody Yu², Xifeng Yan University of California, Santa Barbara, CA, US
SSge Hill School, Newport Coast, CA, US (hongwang600, suan_luo, weizhiwangJJ@cce edu ocmelodyyuu@mail. com, xyanêcs. mranccs.usb...
Abstract
Large language models (LLMs) like GPT-4 have recently demonstrated impressive capabilities in natural language understanding and generation. However, there is concern that they can be misused for malicious pur- poses, such as fraud or denial-of-service attacks. Therefore, develop methods for detecting whether the party involved in conversa- tion is a bot or a human. In this paper, we propose a framework named FLAIR, Finding Large Language Model Authenticity via a Single Inquiry and Response, to detect conversational bots in an online manner. Specifi- cally, we target a single question scenario that can effectively differentiate human users from bots. The questions are divided into two categories: those that are easy for humans but difficult for bots (e-g., counting, sub- stitution, searching, and ASCIT art reasoning), and those that are easy for bots but difficult for humans (e.g., memorization and computation). Our approach shows different strengths of these questions in their effectiveness, providing a new way for online service providers to protect themselves against nefarious activities. Our code and question set are available at https: thub. comm/hongwang660/FLAARR
crucial to
1 Introduction
Recently, the development of Large Language Models (LLMs) such as GPT-4 (OpenAT, 2023) and LLaMA-2 (Touvron et al., 2023a) has brought significant advances in natural language processing and achieved superior performance in downstream tasks of language understanding (Chowdhery et al., 2022), question answering (Su et al., 2019), dialogue systems (Wang et al., 2022b; Qian & Yan, 2023) and multimodal reasoning (Wang et al., 2022a). However, with the proliferation of these models, concerns have emerged regarding their potential misuse for malicious purposes. One of the most sigmificant threats is the use of large language models to impersonate human users and engage in nefarious activities, such as fraud, spamming, or denial-of-servie attacks. For instance, LLM agents could be used by hackers to occupy all customer service channels of various corporations, such as e-commerce, airlines, and banks. Moreover, with the help of text-to-speech (TTS) techniques, machine-gent -ecchine-genertee voices could even occupy public service lines like 911, leading to severe public crises (Wang et al., 2021). These attacks could cause significant harm to online service providers and their users, eroding the trust and integrity of online interactions.
In response to these challenges, there is pressing requirement for reliable differentiation between human users and malicious L.M- bots. Conventional techniques, such as the use of CAPTCHAs (Von Ahn et al., 2003), have been developed to determine whether user is a human or a bot in order to prevent bot spamming and raiding. commonly used CAPTCHA method involves asking users to recognize distorted letters and digits. However, these approaches face significant challenges when it comes to detecting chatbots involving text only. This is where the emergence of large language models such as GPT-4 has further complicated the problem of chatbot detection, as they are capable generating high-quality human-like text and mimicking human behavior to considerable extent. Although recent
1

---

Bots not good at Published as a conference paper at COLM 2024
Bots good at Humans not good at memorization V computation
Humans good at
symbolic manipulation randomness
searching
x
graphical understanding
Table 1: Leveraging tasks that Bots and Humans are (not) good at.
studies such as DetectGPT (Mitchell et al., 2023) have proposed methods to classify if text is generated by ChatGIT or not, they focus on the offline setting. A recent study (Sadasivan et al., 2023b) shows that these detectors are not reliable under paraphrasing attacks, where light paraphraser is applied on top of the generative text model. This limitation highlights the need for more robust and accurate methods to differentiate large language models from human users and detect their presence in online chat interactions.
In this paper, we propose a novel framework named FLAIR, Finding LLM Authenticity with Single Inquiry and Response, to take full advantage of the strength and weakness of LLMs for LLM-based conversational bot detection. Specifically, we introduce a set of carefully designed questions that induce distinct responses between bots and humans. These questions are tailored to exploit the differences and generate language. As shown in Table 1, certain questions in the fields of symbolic manipulation, randomness, searching, and graphical understanding are difficult for bots but relatively easy for humans. Examples include the counting, substitution, searching, and ASCII art reasoning. On the other hand, memorization and computation was relatively easy for bots but difficult for humans.
the way that bots and humans process
Our experimental results demonstrate that FLAIR provides a viable alternative to traditional CAPTČHAs. Specifically, while humans and LLMs excel with high accuracy on tasks within their areas of strength, their performance significantly declines on tasks they are less adept at, often falling to very low levels (~0%). This sharp disparity in performance allows for the differentiation between human and L.M respondents with just a single question. The proposed approach shows promise in developing more robust and accurate methods to quickly detect bots and safeguard online interactions.
2
Related Work
2.1 CAPTCHA
CAPTCHA (Von Ahn et al., 2003) is common technique used to block malicious ap- plications like dictionary attacks, E-mail spamming, web crawlers, phishing attacks, etc. There are different types of CAPTCHAs. Text-Based CAPTCHAs require the users to recognize letters and digits in distortion form (Chew Baird, 2003; Mori & Malik, 2003; Yan & El Ahmad, 2008), while Image-Based Based CAPTCHAs (Gossweiler et al., 2009) require users to choose images that have similar properties such as traffic lights. Video- Based CAPTCHAs (Kluever, 2008) require the user to choose three words that describe video, and Mudioo-Based CAPTCHAS (Gao et al., 2010) ask the user to listen to an audio and submit the mentioned word (Saini & Bala, 2013). Puzzle CAPTCHAs (Singh & Pal, 2014) require the user to combine segments to form complete picture. These techniques are used to differentiate between human users and bots, preventing malicious activities.
LLM offline detection.
Since its introduction, Large Language Models (LLMs) have become widely used and raised public concerns about potential misuse. For instance, students may use ChatGPT (OpenAI, 2023) to complete written assignments, making it difficult for instructors to accurately
2

---

Published as a conference paper at COLM 2024
assess student learning. As result, there is growing need to detect whether piece of text was written by ChatGPT. To tackle this problem, DetectGPT (Mitchell et al., 2023) proposes solution by comparing the log probabilities of the original passage with that of the perturbations the same passage. The hypothesis behind this method is that minor rewrites of text generated by the model would likely result in lower log probabilities compared to the original sample, while minor rewrites of text written by humans may result in either higher or lower log probabilities. Another line of study model this problem as binary classification problem and fine-tune another model using supervised data (Bakhtin et al., 2019). Most recently, Mitrović et al. (2023) fine-tunes a Transformer-basee model and uses it to make predictions, which are then explained using SHAP (Lundberg & Lee, 2017). Another area of research focuses on adding watermarks to Al-generated text in order to facilitate their identification, which involves imprinting specific patterns on the text to make it easier to detect (Zhao et al., 2023). Soft watermarking, as proposed by Kirchenbauer et (2023), involves dividing tokens into green and red lists in order to create these patterns. When generating text, a watermarked LLM is more likely to select a token from the green list, which is determined by the prefix token. These watermarks are often subtle and difficult for humans to notice.
However, demonstrated in Sadasivan et al. (2023a), range of detection methods, includ- ing watermarking schemes, neural network-based detectors, and zero-shot classifiers, can be easily defeated by paraphrasing attacks. These attacks involve applying light paraphraser to text generated by a language model. Furthermore, a theoretical analysis suggests that even the best possible detector can only perform marginally better than random classifier when dealing with sufficiently good language model. This highlights the fundamental challenge in offline detection of text generated by advanced language models, which can produce writing that is virtually indistinguishable from human-written text. Thus, it is more meaningful and crucial to shift the focus online detection settings where users engage in live chat interactions with the system.
Leveraging the Weakness of LLM
In this section, we explore specific tasks such as Counting, Substitution, Random Editing, Searching, and ASCII Art Reasoning. These tasks, while seemingly straightforward for humans, present significant challenges for large language models (LLMs).
3.1 Counting
State-of-the-art LLMs cannot accurately count characters in a string (Qian et al., 2022), while humans can do with ease. This limitation of LLMs has inspired the design of a counting FLAIR to differentiate humans and LLMs. Participants are asked to count the number of times specific character appears in given string:
Answer the question without explanation: Please count the number of in eeooeotetto GPT-3.5: There are 4 t's in "ecooeotetto".
Human: 3
As demonstrated by this example, GPT-3.5 struggles to provide an accurate count of the specified character within the string.
3.2 Substitution
It is known that LLMs often output contents that are inconsistent with context (Elazar et al., 2021; Wang et al., 2022c). It is a shared weakness of current LLMs. We ask LLMs to spell random word under given substitution rule, testing if they follow the rule consistently. The random word has length between five to ten characters, which is randomly sampled from dictionary. Here is example:
3

---

Published as a conference paper at COLM 2024
Answer the question without explanation: Use to substitute p, a to substitute e, substitute a, g to substitute to substitute h, how to spell peach under this rule? GPT-335: mnong
Human: mango
LLMs are struggle with applying this rule consistently and repeatedly in the above substitution task. This testing concept could further generalized to encryption scheme where string is transformed based rules.
3.3 Random Editing
Random editing is a technique used to evaluate the robustness of natural language pro- cessing models against noisy inputs. The technique involves presenting the model with string of characters (sampled from the set of (0, 1]) and randomly dropping, inserting, swapping, or substituting some characters. To verify that the model is capable of performing these random operations, it is required to generate three different outputs under the same conditions. By ensuring that the three outputs are different, we can confirm that the model is applying the random operation.
For random drop, the model is asked to randomly remove occurrences of given character from the string, where k is random number and is random character that appears in the string. For random insert, the model is asked to randomly insert occurrences given character into the string at random positions. For random swap, the model is asked to randomly swap k pairs of characters e and d in the string, where с and d are different characters that appear in the string. Finally, for random substitution, the model asked to randomly replace k occurrences of character c with another character d in the string. Below show an example of LLMs and human output on the random drop task:
Answer the question without explanation: randomly drop two from the string: 0110010011. Give me three different outputs.
GPT-335: 01001011, 010010011, 011001011 Human: 00010011, 01000011, 01100001
While humans can intuitively and correctly manipulate characters specified, GPT-3.5 struggled to consistently perform the task correctly. This suggests potential limitation in handling dynamically specified, random input transformations in natural language processing tasks.
3.4 Searching
Searching is inherent ability of humans, allowing to envision various potential out- comes in our minds and make decisions accordingly. In contrast, language models are constrained by their sequential generation process, which limits their ability to explore different results through backtracking. Once token is generated, these models cannot retract or revise it based on subsequent insights or information, preventing them from exploring alternative paths that might have been overlooked during initial processing. We test this by examining a fundamental search scenario: counting the number of islands 2D map, a task typically solved using Depth-First Search (DFS) or Breadth-First Search (BFS) algorithms. This challenge requires identifying and counting the connected components of 's (representing land) in grid. Adjacent 's, either horizontally vertically, form island. To accurately determine the number of islands, one must systematically traverse the map, marking visited land cells to avoid recounting the same island. Here an example:
4

---

Published as a conference paper at COLM 2024
Hespenas
Lue Karpeeae
Somonn
(b)
Figure 1: Example about ASCII reasoning. (a) select the ascii arts containing X. (b) Rotate the ASCII art to the appropriate orientation. (c) Select the one that most accurately aligns with the cropped portion.
Answer the question without explanation: Count the number of islands in a given 2D map, where black blocks represent land and spaces represent water. Output single number as your result.
Map:
GPT-3.5: 4
Human: 6
As demonstrated in the example, GPTT-3.5's output is incorrect, while humans can intuitively and accurately count the number of islands by traversing the connected areas. This task highlights the inherent limitations of language models in search and backtrack mechanisms.
3.5 ASCII Art Reasoning
Humans can comprehend visual elements whether they are presented in image format or as text strings. For example, an apple converted into ASCII art can still be recognized by people, despite the abstract representation. This ability to understand ASCII art requires level of visual abstraction that current language models lack. To investigate this, we converted natural images into ASCII format and conducted experiments involving common reasoning tasks.
To integrate this approach with existing CAPTCHA systems, we focused on three reasoning tasks. The first task involves identifying ASCII art that includes a specific element, such as a cat. Figure 1(a) presents an example where participants are required to select the ASCII art depicting cat, mimicking the function of reCArTCHA, Participants click images containing a cat. For language models, they receive these images in string format with designated indices (A, B, C, D). The language model must then identify and output the indices corresponding to ASCII art containing a cat. Below is the prompt used for Figure (a)), where the [ASCI] art X] represents the strings of the ASCIT arts:
5

---

Published as a conference paper at COLM 2024
Without providing explanations, select all ASCII art that contains cat. Output the indices of the ASCII arts.
A: [ASCII art A] B: [ASCII art B] C: [ASCII art C [ASCII art D]
GPT-4: The ASCIT arts containing cat are in options A and C.
Human: A, B, D
The answers for this example are [A, B, D), which are colored in red in Fig (a). The second task involves rotating the ASCII art to the correct direction. Fig (b) shows an example. Similar to the image-based rotation CAPTCHA, participants are required to use a mouse to drag and rotate the image to the correct direction. For this task, we will also prompt the LLM with ASCII art but require the LLM to output the degree needed to rotate clockwise for each image. The answers for this example are: [A 180, B 2700 C +90, D +0]. The third task is to choose one ASCII art that best fits the cropped part. Fig (c) demonstrates an example. The LLM has to output the index of the ASCII art that best fits. We can also define other more complicated visual reasoning tasks like "Please click the images containing cat, plane, keyboard, banana in order," or "Please combine the 4 parts into a single image," etc. This method is compatible for many existing CAPTCHAs that does not require detailed reasoning (convert image to ASCII arts will lose the details), and can be seamlessly combine with many existing systems.
Our tests indicate that ASCII-based reasoning tasks present substantial challenges for current Large Language Models (LLMs), including GPT-4. Despite utilizing techniques like -hain-of-thought reasoning or deploying Python APIs, these models face difficulties. We further extended these tests to more advanced visual language models, such GPT-4o, and found that they are limited to handling only the simplest of ASCIT reasoning task.
4 Leveraging the Strength of LLM
In this section, we will discuss the methods that capitalize on the strengths of LLMs. These questions are typically challenging for humans, but are relatively easy for LLMs due to their ability to memorize vast amounts of information or perform complex computation.
4.1 Memorization
The primary idea behind this type of questions is to ask users to enumerate items within given category. Below is an example question and answers from GPT--355 (we only list few items from the output):
Answer the question without explanation: List the capitals of all the states in US: GPT-3.5: 1. Montgomery Alabama Juneau Alaska 3. Phoenix Arizona 50.
Cheyenne- Wyoming
Human: don't know.
For humans, this type of question is not easy it requires good memorization. There are several choices when designing enumerating questions. The first option is to contain many items to make it more challenging for humans to remember, such as all the countries in the world. The second option is to include relatively old information that people may not have encountered, such as all the movies in 1970s. The third option is domain-speciff information that people are unlikely to know, such as the names of all Intel CPU series. We can determine the likelihood of the answer coming from a bot by verifying the overlap between the given answer and the ground truth. If the overlap is greater than a threshold, then more likely that the answer comes from bot.
4.2 Computation.
Performing complex calculations, such as multiplication, without the aid of a computer or external notes is difficult for humans due to the challenges of recalling intermediate steps. In contrast, LLMs excel in remembering the results of common equations, such as the square of π.. For instance, below is example question and the answer generated by GPT-3.5:

---

Published as a conference paper at COLM 2024
What is the square of πt? GPT-3.5: The square of π (pi) is approximately 9.869604401. Human: don't know.
Moreover, by utilizing external tools, such Wolfram, GPT Plugins can even solve more in- tricate problems with in a second. But for human, it will require much longer time to solve this problem. Therefore, the behavior of real human will be very different from language model and we can easily detect bots using a behavior- et al., 2008; Awla et al., 2022; Guerar et al., 2021; Acien et al., 2021).
CAPTCHA system (Von Ahn
5 Experiments
In this section, we present the experimental results of our proposed single questions for distinguishing between humans and LLMs. We curate a dataset for each category of the proposed questions, which is used to evaluate the performance of both humans and LLMs. By contrasting the accuracy of responses, we aim to differentiate between the two.
5.1 Datasets
To evaluate the performance of both LLMs and humans, constructed dataset for each category of questions and open- open-sourced it on //gi:hbgitt...... wwt.ccom///hhhee hommhhonggangg00/FLAAR
Counting We used the entire alphabet for this task. First, we randomly picked one letter to be the target letter count. Then, we chose random number between 10 and 20, which determined how many times the target letter would appear in the string, We created string that was 30 characters long, with instances of the target letter and the remaining 30 characters randomly selected from the other letters of the alphabet. The correct answer for the number of times the target letter appeared the string was k, and an answer matching this number was considered correct.
Substitution To create our dataset, we began by collecting the top 1500 nouns from the Talk English web site!. We then filtered the words to include only those with a length between 5 and 10 characters. Next, we randomly generated 100 pairs of words, each with corresponding substitution map that could transform one word into the other. To ensure the validity of our pairs, we excluded any that would require one character to be mapped to more than one character, which would result a conflict. The resulting questions presented to participants included the substitution rule and the original word, with the answer requiring another word produced through the substitution.
Random Editing In the random editing task, we evaluate the model's ability to perform four distinct operations: drop, insert, swap, and substitute, as described in Section 3.3. For each operation, we generate a random binary string of length 20 to ensure readability. We randomly sample parameters such as the target character and the operation count, Participants are then asked to produce three different outputs after performing the specified random operation. For the random drop task, participants are required to randomly remove k occurrences of given character from the string. In the random insert task, participants randomly insert k occurrences of a given character c into random positions within the string. For the random swap task, participants randomly swap k pairs of characters and within the string. Lastly, in the random substitution task, participants randomly replace k occurrences of character c with another character d. To validate the correctness of the outputs, we first check each individual output by comparing it to the original string, We then verify that the three outputs differ from each other. An answer is considered correct only if each output is correct and all three outputs are distinct. This comprehensive validation ensures the model's ability to handle dynamically specified, random input transformations accurately.
website URL: httpps://www.ff waw.e//wwwww....llai/omm.. is. ctm httm///ccbbblrr/too-11eeee nouns.aspx
7

---

Published as a conference paper at COLM 2024
Searching We generated 100 random 7x7 grids containing spaces and I to create map. Each cell in the grid was independently filled, with a 50% chance of placing a and a 50% chance of placing O. While filling the grid, ensured that no cell would be filled on the diagonal of an already filled land cell, thereby avoiding diagonal connections. The filling process was conducted one cell at time, with each new cell sampled from the remaining available spaces. We then utilized the Depth-First Search (DFS) algorithm to determine the number of islands, which served as the ground truth. An answer is considered correct if it matches this ground truth.
ASCII Art Reasoning In our study, we employed GPT-4 to create list of four items, incorporating two random entities, such as [A: cat, B: cat, C: apple, D: cat], as illustrated in Figure 1. These entities served as prompts for Dall-E to produce correspondent images. Subsequently, we utilized the methodology outlined in "ASCII Art Reasoning" to transform these images into ASCII art Each piece of ASCII art was rendered in a 64x64 character matrix using an ASCII gradient of * ' For the task "select the ASCII arts containing X", we chose one entity (e.g., cat) at random and use the generated ASCIT arts to form the prompt for language model. In the task "Rotate the ASCIT art to the appropriate orientation" we randomly rotated the ASCII art pieces by 90, 180, 270, or 360 degrees clockwise and used these variations as prompts for the language model. For the task "select the one that most accurately aligns with the cropped portion", we randomly selected one piece of ASCII art and cropped random 1/4 portion from its top-left, top-right, bottom-left, or bottom-right corner. We then cropped three additional patches from different images at the same positions. Fig 1 (c) is an example. The language model was prompted to identify the patch that best matched the original cropped portion. For these three tasks, the answer indices/ /dgrrees from language model should be exactly the same with the ground truth to be considered correct.
Memorization We used a set of questions that required the user has a good memorization. There are two types of question under this category including enumerating and domain- specific questions. For enumerating, the user is asked list items within a given category. We manually collected 100 categories containing more than 50 items or those that were difficult for humans to know with the help of ChatGPT. The question asked users to list the items within the given category, and we calculated the coverage of the response against the ground truth. If the coverage exceeded the threshold of 95%, we considered the answer to have been generated by an LLM. For domain specific questions, we manually collected a set of 100 questions whose answers are difficult for people to recall or access, such as "What is the weight of the world's heaviest lemon in grams?". Although these questions may be challenging for humans to answer, they are relatively easy for large language models (LLMs) due to their pre-training on large corpora that includes these questions. All of the reasonable results that can be sources on the internet are considered correct.
Computation To create the computation dataset, we selected the problem of four-digit multiplication. Specifically, we randomly sampled 100 pairs of four-digit numbers and calculated their product as the ground truth. Participants were asked to solve these multi- plication problems and were considered correct if absolute difference between their answer and the ground truth was within 10%. For humans, it can be difficult to accurately calculate these multiplications without the aid of notes or calculator, leading them to often respond with 'I don't 't know". In contrast, large language models (LLMs) have seen many similar equations during pre-training and tend to provide guess that is often close to the ground truth. This testing can be further extended to any complicated computation like division, exponents, etc.
5.2 Benchmarking Baselines
We conducted experiments across a range of LLMs, including open-source models like Vicuna-13b (Chiang et al., 2023) and LLaMA-2-13b, 70b (Touvron et al., 2023b), as well as proprietary models such as GPT-3, 3.5, and 4 (Brown et al., 2020; OpenAl, 2023; Ouyang et al., 2022). Additionally, we experimented with the zero-shot offin-of-though (CoT)
8

---

Published as a conference paper at COLM 2024
methodology (Wei et al., 2022), where GPT-4-CoT denotes the results obtained using this specific approach with GPT-4. To implement this, we appended a structured prompt to the standard query, instructing the model to "Please analyze the question step by step and output your analysis under 'Analysis', then provide your final answer based on this analysis after "nswer:"" We then extracted the part after 'Answer.' to verify the result's correctness.
For tasks requiring computational solutions, we used variant called GPT-4-py. Here, GPT-4 was instructed to compose Python code with the prompt "Please write a python program to solve this problem". The generated code was executed through an API call to derive the final outcome. We compared the code's output with the ground truth to verify correctness. Code that could not be successfully executed or did not produce any output was considered incorrect.
Main Results
We conducted a series of five trials for the LLM tests, with each trial consisting of 100 samples, matching the sample size used in the user study. To determine the final accuracy, we used the median accurácy from these five trials. Tasks that language models struggle with tend to exhibit higher variance accuracy due to the instability of their outputs. This instability introduces randomness, leading to potential outliers. By using the median accuracy, we minimize the influence these outliers and provide a more robust measure of performance.
For our user study, we enlisted 10 participants. The age distribution among our participants was varied, comprising one individual aged between 10-20 years, five aged between 20-30 years, three aged between 30-40 years, and one aged between 40-50 years. Each participant responded set of 10 questions in each category, with 10-second time limit per question. This time limit prevents users from searching the internet or using a calculator to solve the computation and memorization tasks, For the random editing and ASCII-related tasks, we extended the time limit to 30 seconds. The results of user study is shown in the row "Humans".
Random Edit
Search.
ASCII Select
ASCII Rotate
ASCII Crop
Count
Substi.
Memori.
Comput.
Humans
100%
100%
100%
100%
100%
97%
94%
6%
2%
Vicuna-13b
15%
1%
1%
0%
17%
93%
100%
1MM-2-2-3bb
10%
2%
0%
2%
4%
1%
23%
94%
96%
LLaMA-2-70b
14%
5%
3%
2%
4%
0%
22%
97%
98%
GPT-3
13%
2%
0%
0%
0%
19%
94%
95%
GPT-3.5
15%
6%
3%
1%
26%
99%
98%
GPT-4
21%
8%
7%
0%
25%
99%
99%
GPT-4-CoT
33%
56%
13%
10%
2%
0%
27%
99%
99%
GPT-4-py
100%
100%
100%
100%
0%
0%
0%
N/A
N/A
Table The comparison between LLMs and Human on different tasks.
The results, presented in Table 2, align with the tasks outlined in Section 3, "Leveraging the Weaknesses of LLMs" (left side), and Section 4, "Leveraging the Strengths of LLMs" (right
side).
In the left side, we evaluate human versus LLM performance on tasks that are straight- forward for humans but challenging for LLMss. Humans attained perfect scores (100%) across most tasks, with slight variations in the ASCII Art tasks, where the resolution of some ASCII representations posed recognition challenges. Conversely, LLMs struggled significantly with tasks involving substitution, random edits, searching, and ASCII art, often achieving near 0% accuracy. LLMs showed improved performance on tasks like counting and ASCII Crop, benefiting from more constrained solution space which simplified the task identifying correct answers. For example, with only four possible answers (A, B, C, D) in the ASCIT Crop task, the chance of selecting the correct one is approximately 25%.

---

Published as a conference paper at COLM 2024
The results also demonstrate that larger and newer language models generally have better performance on these tasks. GPT-4, which has 1.8T parameters, exhibits superior per- formance compared to Vicuna-13b on nearly all the tasks. We also explored advanced techniques like chain-of-thought prompting (Wei et al., 2022). The GPT-4-CoT results show the performance of GPT-4 with haain-of-though prompting. The experiments demon- strated that chain-of-thought prompting can improve performance on tasks that can be decomposed into several steps, especially for substitution, where the language models can perform substitution step-by-step. However, chain-of-thouu thought prompting does not help to improve performance on searching and ASCII-based visual reasoning tasks. Its effectiveness is limited to textual and decomposabe tasks like counting, substitution, and random edit.
Furthermore, we explored the possibility of GPT-4 generating code to solve the problems by manually prompting it with, "Please write Python code to solve the problem." With this human-assisted prompt, GPT-4 can generate code that can bypass the challenges counting, substitution, random edit, and searching tasks, achieving 100% accuracy. However, GPT-4 is not intelligent enough to write code for ASCII art reasoning. The attempts often result in erroneous non-functional code that does not output anything, yielding 0% accuracy rate across ASCII reasoning tasks. This suggests that with manual prompting on the use of appropriate tools, language models can overcome some of their limitations.
The right section of Table focuses on tasks that are challenging and time-consuming for humans but relatively straightforward for LLMs. The results reveal that humans struggled with tasks that require exceptional memory or computational abilities, with only 6% and 2% of human participants successfully completing the memorization and computation tasks, respectively. In stark contrast, LLMs demonstrated remarkable proficiency in these areas, with some models achieving near-perfect accuracy.
5.4 Discussion
The rapid advancements in LLMs pose a significant challenge to the long-term effectiveness of our proposed approach. It is conceivable that future multimodal language models could automatically identify tasks requiring code generation or visual reasoning, potentially undermining our method. Therefore, it is crucial to continuously monitor the development of LLMs and adapt our approach accordingly.
One promising direction is to integrate our method with existing frameworks, such as behavioral CAPTCHAs, which take into account users' clicks, keyboard operations, time usage, and other behaviors. Combining these techniques can provide more robust defense against increasingly sophisticated bots.
During our research, we also explored new set of questions that require few-shot reasoning, which current language models struggle to solve but might be too challenging for the general public. For example, consider the question: "I have a set of strings following certain pattern ["abede | | ledcba", "abl I ba", "abel cbba,...],, What is the string that contains exactly ?" Humans can intuitively find the pattern by identifying the consecutive characters and their reversals. However, without additional training, language models often fail to provide the correct answer, even when prompted to write Python code. Although language models are claimed to be few-shot learners, they have limited generalization to new tasks without specific training. The direction of discovering few-shot questions remains promising and could lead to new methods for bot detection.
6 Conclusion
In conclusion, this paper proposes a new framework called FLAIR for detecting conver- sational bots in an online environment. The proposed approach targets a single question scenario that can effectively differentiate human users from bots by using questions that are easy for humans but difficult for bots, and vice versa. Our experiments demonstrate the effectiveness of this approach and show the strengths of different types of questions. This framework provides online service providers with a new way to protect themselves against fraudulent actívities and ensure that they are serving real users.
10

---

Published as a conference paper at COLM 2024
Acknowledgment
We would like to thank the anonymous reviewers for the helpful comments. We extend our gratitude Ethan Mader for his insightful discussions. This research was partly supported by the DARPA PTG program [HR001122C0009) and generous gift grant from Visa Research. The opinions, findings, conclusions, and recommendations presented in this paper are those of the authors and do not necessarily represent the views of the funding agencies.
References
Alejandro Acien, Aythami Morales, Julian Fierrez, Ruben Vera-Rodriguez, and Oscar Delgado-Mohatar. Becaptcha: Behavioral bot detection using touchscreen and mobile sensors benchmarked on humidb. Engineering Applications of Artificial Intelligence, 98: 104058, 2021.
Hoshang Qasim Awla, Arsalan Rahman Mirza, and Shahab Wahhab Kareem. An automated captcha for website protection based user behavioral model. In 2022 8th International Engineering Conference on Sustainable Technology and Development (IEC), pp. 161-167. IEEE, 2022.
Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc' Aurelio Ranzato, and Arthur Szlam. Real or fake? learning to discriminate machine from human generated text. arxiv preprint ar Xrii:1996.03351. 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Monica Chew and Henry S Baird. Baffletext: A human interactive proof. In Document Recognition and Retrieval X, volume 5010, pp. 305-316. SPIE, 2003.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023. URL https://vicunna. 1mw.snammmmmms..mmmmmmmmm Imsys.org.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arviv preprint ar XrXiv:2204.02311, 2022.
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hin- rich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012-1031, 2021.
Haichang Gao, Honggang Liu, Dan Yao, Xiyang Liu, and Uwe Aickelin. An audio captcha to distinguish humans from computers. In 2010 Third International Symposium on Electronic Commerce and Security, pp. 265-269. IEEE, 2010.
Rich Gossweiler, Maryam Kamvar, and Shumeet Baluja. What's up captcha? captcha based on image orientation. In Proceedings of the 18th international conference on World wide web, pp. 841-850, 2009.
Meriem Guerar, Luca Verderame, Mauro Migliardi, Francesco Palmieri, and Alessio Merlo. Gotta captcha'en em all: survey of 20 years of the human-or-computer dilemma. ACM Computing Surveys (CSUR), 54(9))1-33, 2021.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. ar preprint ar Stt:2331.100244 2023.
11

---

Published as a conference paper at COLM 2024
Kurt Alfred Kluever. Evaluating the Usability and Security of Video CAPTCHA. Rochester Institute of Technology, 2008.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot pachine-generater text detection using probability curvature. arviv preprint 0iie:2301.11305 2023.
Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt- generated text. ar preprint arXiv:2301.118852, 2023.
Greg Mori and Jitendra Malik. Recognizing objects in adversarial clutter: Breaking visual captcha. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, volume 1, pp. Í-I. IEEE, 2003.
OpenAL. Gpt-4. https://openalle www///eeeeacch/gttt...... 2023, Accessed on March 14, 2023.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. arXiv preprint arXto:2203.02155, 2022.
Jing Qian and Xifeng Yan. Language model detoxification in dialogue with contextualized stance control. Xrviv preprint arXiv:2301.103: 2023.
Jing Qian, Hong Wang, Zekun Li, Shiyang Li, and Xifeng Yan. Limitations of language models in arithmetic and symbolic induction. ar arx preprint ar arXix:2208.050 2022.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156, 2023a.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? arXiv preprint ar arXio:2303.1113: 11156, 2023b.
Baljit Singh Saini and Anju Bala. review of bot protection using captcha for web security. IOSR Journal of Computer Engineering, 8(6):36-42, 2013.
Ved Prakash Singh and Preet Pal. Survey of different types of captcha. International Journal of Computer Science and Information Téchnologies, 5(2):2242-2245, 2014.
Dan Su, Yan Xu, Genta Indra Winata, Peng Xu, Hyeondey Kim, Zihan Liu, and Pascale Fung. Generalizing question answering system with pre-trained language model fine-tuning. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pp. 203-211, 2019.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne
Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint ar XrXi:22302.13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.00 .0..300....9.,,... 2023b.
Luis Von Ahn, Manuel Blum, Nicholas J Hopper, and John Langford. Captcha: Using hard ai problems for security. In Eurocrypt, volume 2656, pp. 294-311, 2003.
2003.
Luis Von Ahn, Benjamin Maurer, Colin McMillen, David Abraham, and Manuel Blum. recaptcha: Human-based character recognition via web security measures. Science, 321 (5855):1455-1468 2008
12

---

Published as a conference paper at COLM 2024
Weizhi Wang, Li Dong, Hao Cheng, Haoyu Song, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. Visually-augmeented language modeling. arxiv preprint ar arXic:2205.10175, 2022a.
Weizhi Wang, Zhirui Zhang, Junliang Guo, Yinpei Dai, Boxing Chen, and Weihua Luo. Task-oriented dialogue system as natural language generation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pp. 2698-2703, 2022b.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self- consistency improves chain of thought reasoning language models. aiv preprint (/Xiv:223.111771, 2022c.
Zihao Wang, Minghui Yang, Chunxiang Jin, Jia Liu, Zujie Wen, Saishuai Liu, and Zhe Zhang. Ifdds: An anti-fraud outbound robot. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 16117-16119, 2021.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Jeff Yan and Ahmad Salah Ahmad. A low-cost attack on a microsoft captcha. In Proceedings of the 15th ACM conference Computer and communications security, pp. 543-554 2008
Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking, ar Xiv preprint arXie:2302.03162, 2023.
13


Reviews:
[{"review_id": "PmBhkIEDxy", "reviewer": "Reviewer_fBF2", "summary": "This paper propose a new CAPTCHAs methods to detect whether the response is from human or  LLMs via asking a question that is either easy to LLM but hard to human or easy to human but hard to LLM. The researcher identify a few patterns of such questions that can lead to distinguished performance of human and LLMs. The proposed questions are checked against human users and a comprehensive set of LLMs. The results show that these questions can effectively distinguish the LLMs and Humans based on the accuracy of their response;", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review of our paper. We appreciate your insightful feedback and would like to address the concerns raised regarding the reasons to reject our work.\n\n**Time-Wise Effectiveness of the Approach**: We acknowledge that the LLMs are evolving rapidly to gradually overcome these imposter detection questions. However, our method can also revolutionize at the same time. Following the proposed task taxonomy, the updated questions are still challenging for LLMs to solve them natively.  Questions like \"Can we make 24 using only four 7's?\" could easily fail the latest LLMs like GPT-4o.  Note that it represents a set of questions that need many search trials and possibly a mathematical proof showing that a solution does not exist.  Solving these questions will give new insights for the future development of LLMs. We will continuously update the question set, encouraging researchers from both sides to improve their methodology over time. \nThe issue of distinguishing LLMs from humans exists and remains significant. Our work highlights this importance and proposes a different method to address it. Our approach is a step towards identifying and understanding the limitations of LLMs, and we envision that future research will build on this foundation to develop more sophisticated and adaptive detection methods.\n\n**Control Over Human Responses and Use of Computational Tools**: We recognize the concern regarding the possibility of human participants using computational or AI tools to answer questions that are difficult for them but easy for LLMs. To address this, we can rephrase questions to include contextual hints, which can deter the use of external tools. For instance, a question could be phrased as, \"To your knowledge, list all the nations in the USA you know.\" Such phrasing encourages genuine human responses and makes it harder for individuals to rely on external aids.\n\nAdditionally, implementing time constraints and monitoring response patterns can help mitigate the issue. Humans typically take longer time to derive answers to complex questions compared to LLMs, which can provide near-instantaneous responses. By analyzing not only the correctness of answers but also the time taken and the response style, we can enhance the robustness of our detection mechanism. Regular updates and refinements to the question sets will also help maintain their effectiveness over time.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. A simple and clean design to capture the different behaviors from human users and LLM via asking a question that will lead to completely different response from the two groups. \n2. Details of the question design are clearly explained which makes it easy to replicate the dataset by following reseaches. \n3. The benchmark is also valuable to check LLM's capability on the founda-mental skills that is easy for human", "reasons_to_reject": "1. It is questionable on the time-wise effectiveness of this approach to distinguish LLM from human as we don't know whether the challenges faced by LLMs now will still remain when newer version of LLMs come out. Some of the challenges seem to be easily addressed by providing more in-domain data to the model which makes the trick easily broken. \n2. For the question that are easy to LLM but hard to human, it is really hard to control whether the human will actually answer the questions without the help of computational tool or AI tool during the actual CAPTCHA check. Therefore, it is hard to really distinguish humans from LLMs by simply checking whether the memory type questions are answered correctly;", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r1cwxF37cN", "reviewer": "Reviewer_mR8r", "summary": "This paper introduces a novel method to detect whether a participant in a conversation is a human or a bot. Specifically, they evaluated various question-answer pairs on both humans and LLMs. These were single QA pairs that included questions about counting, substitution, random editing, ASCII art reasoning, memorization, and computation. The questions are designed around things that the LLMs are good at (e.g., memorization and computation) and bad at (i.e., other tasks). Overall, the paper finds that ascii-related questions can accurately be used to detect if someone is a bot or not. Specifically,  only 1% of questions were accurately answered on the ASCII rotate questions for some models, but 97% were accurate for humans. Hence, these systems can potentially be used by companies and online services to mitigate bot accounts.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and detailed review of our paper. We appreciate the suggestions you raised in your review and will revise the paper accordingly. \n\n**Broader Discussion on Frameworks for New Question Generation**: Our current practice is to identify different classes of questions that LLMs can not handle very well (or easy for LLMs, but hard for human).  For most of these classes, questions/answers are indeed generated automatically.  For the memorization task,  we manually picked up 100 questions. However, this can be done automatically by feeding questions to GPT and checking the cardinality of their answers (possibly by GPT again).  We will give a thorough discussion on scaling the generation of question/answer pairs in the revision.\n  \nThe identification of the question class requires substantial research and experimentation. This remains a fertile area for future research, aimed at identifying the true weakness of LLMs. We will incorporate this discussion into our revised manuscript to highlight the importance of ongoing research in this area.\n\n**Details on Accuracy Calculation**: In the original submission, we reported the median accuracy values across 5 trials for each task and model, with each trial involving 100 samples. We acknowledge that providing the mean accuracy values could offer additional insights. Below, we present a portion of the results, showing the mean accuracy values for LLaMA-2-70b, GPT-3.5, and GPT-4. The values within the brackets represent the variance across the 5 trials:\n\n|Model|Count|Substi.|Random Edit|ASCII Select|ASCII Rotate|ASCII Crop|Memori.|Comput.|\n|--------|-------|---------|-----|------|-------|--------|-------|-----|\n|LLaMA-2-70b|13.8% (4.16)|5.4% (2.63)|2.2% (2.16)|4.2% (2.56)|0.6% (0.64)|23.0% (5.6)|96.8% (2.96)|97.8% (2.55)|\n|GPT-3.5|15.2% (2.56)|5.4% (4.24)|1.8% (1.36)|5.0% (2.00)|1.4% (0.24)|25.4% (4.64)|98.4% (1.44)|98.0% (1.20)|\n|GPT-4|21.0% (6.00)|8.2% (4.16)|6.4% (2.63)|5.6% (3.44)|0.6% (0.64)|24.8% (4.16)|99.2% (0.56)|99.0% (0.40)|\n\n**Concerns about Practical Application and VLLMs**: We apply the latest VLLM, GPT-4O on ASCII-art tasks. However, GPT-4o's performance on ASCII-art tasks is still limited, reaching only 0% accuracy for rotation subtask and 37% accuracy for cropping. These results suggest that while latest visual language models do possess some visual reasoning capabilities, the proposed advanced tasks like ASCII-art are still effective in imposter detections.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Overall, this is a really unique study that explores ways to detect bots in the wild (e.g., online service providers) so they can't cause issues.\nI-  definitely want to highlight the creativity of the ASCII solution. This is really out-of-the-box thinking.\n- The paper is generally well-written.", "reasons_to_reject": "- The paper would be improved if there was a broader discussion about frameworks for how new questions can be generated as LLMs overcome these limitations. If the questions require manual intervention, they will not necessarily scale at the speed needed to mitigate attacks. While the findings are exceptionally creative, it does seem a bit early in the analysis to understand the impact of the findings. Could there be a general automated way to generate these questions that target weaknesses? If so, how would it be done? At a minimum, if substantially more work is required to do automated question-answering, this should be mentioned as a limitation.\n- More description about how accuracy was calculated. There are also some general questions that I have, e.g., why median accuracy vs. mean accuracy? How much did the accuracy vary trial-to-trial?\n- [Minor Weakness] I'm not entirely sure about the usefulness of this system in the application for VLLMs. Particularly, what would stop a bot from screenshotting a website, performing a rotation on the image, then converting back to ascii art? This is possible via GPT4 online interface based on superficial things I tested (e.g., it generates code that converts an image to ASCII and creates code that can perform rotations on the image). I assume a similar thing can be done for matching to images, e.g., converting the ASCII art to an image and then doing image-to-image matching.", "questions_to_authors": "- Why median accuracy vs. mean accuracy?\n- How much did the accuracy vary from trial to trial?", "extra_scores": {}}, {"review_id": "hHEefIpxpd", "reviewer": "Reviewer_SB6E", "summary": "The paper evaluates a number of tasks on several LLMs and on humans that are a priori selected and experimentally confirmed (in most instances) to be either very hard for LLMs and easy for humans or the other way round. The main motivation put forward by the authors is to identify ways to perform one-shot classification between human and artificial conversational agents.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough review of our paper. Your feedback has prompted us to revisit and clarify several aspects of our work.\n\n**Impact of the work**: Regarding the impact of our work, we acknowledge your concerns about the statement regarding the potential investment required for developing a dedicated ASCII art reasoning model. We agree that the estimate provided may not have been adequately justified. While it is true that GPT-4 demonstrates proficiency in multiple tasks without dedicated models, our intention was to highlight the specific challenges posed by ASCII art reasoning and other questions we identified (the newest one is “Can we make 24 using only four 7's?”), which may require significant research and development efforts to overcome. We will revise the manuscript to provide a more nuanced discussion on this point, taking into account the broader context of AI model capabilities. \n\nYour observation about the potential for malevolent LLM practitioners to manipulate model behavior to mimic human-like performance is insightful. We understand the concern, and it is indeed possible for dedicated solutions to address specific tasks. However, the proposed FLAIR method involves a set of detection questions and will evolve with the findings of more possible detection questions. It is still quite hard for advanced LLMs or Multimodal LLMs to overcome the complicated tasks like ASCII-Art recognitions. The LLM imposters may overcome some detectors like memorization questions but they are still easy to be detected via a combination of multiple detection questions.\n\n**ArtPrompt Paper**: We appreciate your reference to the ArtPrompt paper by Jiang et al. (2024). That paper focuses on leveraging ASCII art to bypass safety checks and invoke undesired responses from LLMs. It demonstrates that LLMs can recognize simple ASCII art representations of words, i.e., the characters of \"BOMB\". In contrast, our work shows that LLMs still struggle with more complex ASCII art tasks, especially when manipulations like cropping and rotations are involved. Nevertheless, we will cite this related work and discuss it. \n\n**Typos**: We sincerely apologize for the typos we made in the manuscript. We will rectify these errors accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is based on a simple yet relevant idea that the authors investigate in a simple yet effective way", "reasons_to_reject": "- The impact of the work is not sufficiently discussed. The authors write that \"overcoming [the poor level of GPT-4 ASCII-art related capabilities] would necessitate significant investment, potentially amounting to millions of dollars, for developing a dedicated ASCII art reasoning model\". I am not convinced by this statement at all. GPT-4 is very good at multiple tasks for which no \"dedicated reasoning model\" was developed. And the \"millions of dollars\" ballpark is not motivated at all. On the other hand, the authors do not mention the fact that a malevolent LLM practitioner might try to make their LLM bad at memorization and computation tasks, in order to behave in a more human-like fashion. \n- The paper needs to be proofread. It is regrettable for a paper to be submitted with typos like the following, all easily detected by any good spell checker (including Microsoft Word or… ChatGPT):\n    - page 3 \"another line of study model this problem\" (\"models\")\n    - page 4 \"an example of LLMs and humans output\" (\"an example of LLM and human output\")\n    - page 4 \"As we can seen\" (\"see\")\n    - page 4 \"This first stage of our proposed method is convert\" (\"is to convert\")\n    - page 5 \"The next step is perform\" (\"is to perform\")\n    - page 6 \"Below is and example\" (\"an\")\n    - page 6 \"it require a good memorization\" (\"requires\")\n    - page 8 \"10 seconds time limit\" (\"10-second time limit\", with hyphen but no -s)\n    - page 9 \"develope\" (\"develop\")\n    - page 9 \"possivle\" (\"possible\")\n    - page 9 \"to Combine\" (\"to combine\")\n    - some citations should have been made using \\citep and not \\citet (examples: all citations in section 5.2, \"Awla et al. (2022); Guerar et al. (2021)\" on page 9)", "questions_to_authors": "- The most interesting tasks being ASCII-art-related, how do you relate your results on these tasks with the ArtPrompt paper by Jiang et al. 2024 (published after the submission deadline), where the authors seem to show that LLMs do understand ASCII-art to some extent?", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel method FLAIR to distinguish between human and LLM responses by asking questions that are either easy for LLMs but hard for humans, or vice versa. The researchers identify patterns in these questions that lead to significant performance differences between humans and LLMs. The proposed questions are validated against human users and a comprehensive set of LLMs, demonstrating the method's effectiveness in distinguishing between the two based on response accuracy.

Summary Of Reasons To Publish:
1. Innovative Approach: The paper introduces a simple yet effective method to capture the differing behaviors of humans and LLMs through well-designed questions.
2. Comprehensive Evaluation: The study provides detailed experimental results and extensive evaluations across various models and settings, adding to the robustness and credibility of the findings.
3. Clear Documentation: The question design and dataset generation are clearly explained, facilitating replication and further research.
4. Valuable Benchmark: The proposed method offers a valuable benchmark for evaluating LLMs' foundational skills that are inherently easy for humans, contributing to the field of AI safety and security.

Summary Of Suggested Revisions:
1. Time-wise Effectiveness: The authors should address the concern regarding the time-wise effectiveness of the proposed approach, considering the rapid advancements in LLM capabilities.
2. Manual Intervention Concerns: The paper should discuss the potential challenges of ensuring human participants do not use computational tools during CAPTCHA checks, particularly for questions easy for LLMs but hard for humans.
3. Framework for New Question Generation: A broader discussion on automated frameworks for generating new questions as LLMs overcome current limitations would enhance the paper. This includes addressing the scalability and adaptability of the question generation process.
4. Accuracy Calculation Details: Providing more detailed information on how accuracy was calculated, including the rationale for using median accuracy versus mean accuracy and the variance across trials, would improve clarity.

Overall Assessment:

Despite some concerns about the method's long-term effectiveness and practical application, the novelty and thoroughness of the approach justify its publication. Addressing the suggested revisions would further enhance the paper's impact and clarity.
Decision: Accept