How Well Do Large Language Models Perform on Faux Pas Tests?
Year: 2023

Abstract:
Motivated by the question of the extent to which large language models “understand” social intelligence, we investigate the ability of such models to generate correct responses to questions involving descriptions of faux pas situations. The faux pas test is a test used in clinical psychology, which is known to be more challenging for children than individual tests of theory-of-mind or social intelligence. Our results demonstrate that, while the models seem to sometimes offer correct responses, they in fact struggle with this task, and that many of the seemingly correct responses can be attributed to over-interpretation by the human reader (“the ELIZA effect”). An additional phenomenon observed is the failure of most models to generate a correct response to presupposition questions. Finally, in an experiment in which the models are tasked with generating original faux pas stories, we find that while some models are capable of generating novel faux pas stories, the stories are all explicit, as the models are limited in their abilities to describe situations in an implicit manner.

heading: 1 Introduction
text: Theory of Mind (ToM) is the ability or skill to identify, evaluate or attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have perspectives that are different from one’s own (Wimmer and Perner, 1983). A social skill is any competence facilitating interaction and communication with others (Dowd and Tierney, 2005). Ideally, automated agents that interact with people should possess such social common sense abilities (Choi, 2022), and indeed, a recent trend in the field of AI aims to address challenges related to social skills and commonsense (Sakaguchi et al., 2021; Le et al., 2019; Talmor et al., 2022; Sap et al., 2019; Zellers et al., 2019; Hessel et al., 2022; Lin et al., 2020; Shapira et al., 2023).
To what extent do Large Language Models (LLMs; Brown et al., 2020; Bommasani et al., 2021; Zhao et al., 2023)—models that were trained on massive amounts of both supervised and unsupervised language data, and which constitute the current state of the art in language-based reasoning and communication—possess the ability to effectively reason about implicit social situations, that may not be explicitly discussed in texts? Sap et al. (2022) examine zero-shot theory-of-mind abilities in LLMs (GPT-3-Davinci; Brown et al., 2020) and show that the models struggle with ToMbased tasks. Since then, ChatGPT,1 a new model trained on additional supervised data and in particular human-dialog data, suggests improved abilities at such tasks.
We propose to push beyond the current theoryof-mind tests and consider the task of “recognition of faux pas”, an established task in the clinical psychology domain (Baron-Cohen et al., 1999). The faux pas task combines the SocialIQa (Sap
1https://openai.com/blog/chatgpt/
10438
et al., 2019) and the ToMi (Le et al., 2019) tasks mentioned in (Sap et al., 2022) and is considered to be more difficult for children than any of the individual tasks on their own. We show that the task is also challenging for state-of-the-art LLMs.
We describe two studies, examining different aspects related to the recognition of faux pas within LLMs.2 In the first study (§3) we evaluate, together with a clinical psychologist with diagnosis expertise, the faux pas test results on LLMs. At the first stage (§3.1) we perform a qualitative analysis of the responses of the models and propose a new annotation method that tries to capture quantitatively part of “the ELIZA effect” (Weizenbaum, 1976) a phenomenon where an individual may attribute understanding to a machine based on its ability to respond in a seemingly intelligent manner, even if the response does not fully answer the question. In the second stage, the models were restricted to closed-ended questions by requiring a yes or no answer or without explanations (§3.2). The results show that while the models seem to sometimes offer correct responses, they in fact struggle with this task and that many of the seemingly correct responses can be attributed to over-interpretation by the human reader.
An additional phenomenon observed is that most of the models failed to generate a correct response to “What did they say that they should not have said?” when the question was based on a false assumption and there was no problematic statement in the text.
In the second study (§4) we instruct models to generate 20 original faux pas stories which we manually evaluate, showing that while the best models can generate some faux pas stories, they can only do it in an explicit manner, and struggle with the implicit aspects, which are central to the ToM.

heading: 2 Recognition of Faux Pas
text: Faux Pas (French for “false step”) is defined as “when a speaker says something without considering if it is something that the listener might not want to hear or know, and which typically has negative consequences that the speaker never intended” (Baron-Cohen et al., 1999).
One example of a faux pas situation is when a guest tells their hosts that they “like cakes except for apple pie”, without realizing that the hosts have
2The original clinical test and our research were done in English.
made an apple pie for them. The complexity of the situation depends not only on the content of the statement (“except for apple pie”) but also on the context in which it was made (e.g., the host had made an apple pie and the guest was unaware). Faux pas is the "uhoh!" emotion most people would feel when they reveal the reality of the context. In the mentioned example, the statement may not be problematic if the hosts had made a cheesecake instead.
In the original test,3 the subject is told 10 stories that contain faux pas. At the end of each story, the subject is asked 4 questions:
• Q1 - Faux Pas Detection Question - In the story did someone say something that they should not have said?
• Q2 - Identification Question - What did they say that they should not have said?
• Q3 - Comprehensive Question (this question is different for each story)
• Q4 - False Belief Question. Did they know/remember that? (this question is different for each story)
Each faux pas story that is answered correctly (i.e., all four questions are correct) scored 1 point. In a clinical trial, the average score for 9- to 11-year-old children is 8.2 (SD=1.56) out of 10 faux pas stories (Baron-Cohen et al., 1999).
We note that the faux pas test was initially developed to diagnose autism or Asperger syndrome in children. Here, we do not diagnose models.
Faux Pas as a task can be viewed as a composition of the two tasks that were presented separately by Sap et al. (2022): (1) SocialIQa (Sap et al., 2019) that is related to analyzing and understanding social situations such as reasoning about motivations (e.g., Why would someone accidentally push someone in a narrow elevator? to enter the elevator), what happens next (e.g., What would one want to do after food spilled on the floor? mop up) and emotional reaction (e.g, How would others feel after a scene where the hero is struggling with the villain? hope that the hero will win). (2) ToMi (Le et al., 2019) that is related to the ability to perceive the existence of different perspectives for different agents (e.g., Sally puts a marble in a basket and
3Table 4 in the Appendix contains an example of a full test - a faux pas and control stories with questions and the expected answers.
left the room. Anne moves the marble to a closet. Where will Sally look for the marble?).
The compositionality between the data sets is currently at the essence level and not at the practical level. Faux-pas test is based on mental state inference and the ability to recognize false beliefs (Korman et al., 2017). The SocialIQa includes questions about reasoning about motivation and emotional reactions i.e., “mental state”. The ToMi aims to assess the recognition of false beliefs. For example in the story mentioned in Figure 1, the reader is expected to infer (1) When someone is told “I never liked that object” when the object is a gift from that person, they may be hurt/feel disrespected (mental state). (2) Under the assumption of good intentions, a reasonable possible interpretation is that Richard did not remember/know that James brought him the gift although the reader knows this fact (false belief).
While most ToM clinical tests are designed for subjects with a mental age of 4-6 years, according to the literature, faux pas detection is a ToM clinical test designed to recognize Asperger Syndrome or High-Functioning Autism in children ages 7-11 (Baron-Cohen et al., 1999). This may suggest the difficulty of the test.
For the purposes of this study, we will use 20 examples (10 containing faux pas and 10 control examples) as they appear in the original test in the literature. In addition, aided by LLMs we attempted to generate additional 20 original faux pas stories and annotate their quality (§4)

heading: 3 Study 1: Faux Pas Test Performance
text: In this study the first and second authors of the paper (an NLP researcher and a clinical psychologist trained in diagnostics) manually evaluated the responses to the faux pas tests generated by 14 different language models that were run in a zeroshot manner (Liu et al., 2021) (for technical details regarding prompts and parameters see Appendix A.1).4
The annotation included 3 phases. Phase 1: A researcher (NLP researcher in study/table 1 and Clinical Psychology Diagnostician in studies/tables 2-3) labeled the data and produced simple guidelines. Phase 2: the other researcher labeled the data according to guidelines. Phase 3: both researchers
4Annotated data of study 1 and generated stories of study 2 could be found at https://github.com/NatalieShapira/ FauxPasEAI.
discussed the final decision in cases where there was disagreement. The agreement between the two annotators was 82-100% and after reconciliation 100%.
First, the models were given a story and an open-ended question “In the story did someone say something that they should not have said?” (§3.1). The results of this task raised concerns about ELIZA effect. To accommodate for that, in the second stage the models were given the complete test (4 questions) with closed-ended questions requiring a yes or no answer or without explanations “In the story did someone say something that they should not have said? Answer with “Yes” or “No” only, without explanations. In case of doubt, answer according to the most probable answer.” (§3.2).5
We found the open-ended version of the questions (Table 2) both expensive (manually by experts) and problematic from the perspective of the ELIZA effect i.e., some responses contain the correct answer but at the same time also suggest the wrong answer, in a persuasive way, without a clearcut final answer (§3.1 and Figure 2). The restricted yes/no version of the questions (§3.2 and Table 1) is clear-cut and could be done automatically.

heading: 3.1 Assessing the ELIZA Effect in Responses
text: We assess the quality of the Q1 responses as an open-end question, on several quality factors. The goal is to appraise whether the response provides an ELIZA effect, giving an illusion of understanding (see Figure 2).
The annotation of the response consists of the following factors: Correct: Contains the correct answer (even if not the full answer or there are also wrong parts in the response).
5Table 5 in the Appendix lists all questions versions.
Coherent: Correct grammar, in-context response, the response makes sense, the discourse flows (e.g., there is grounding, full-long answer, finished sentence, there is an answer to the question asked). We ignored unnecessary dots or question marks. Persuasive: Providing information beyond "Yes" or "No" that supports decision such as: (A) Partial knowledge of the situation, e.g., the ability to answer some other questions related to the situation correctly i.e., providing information about Q4 as a response to Q1 “scratch points” even if they were not asked about the information in the current question. (B) Wrong but logical answers (e.g., a scenario in low probability but not zero) or contains general world knowledge (e.g., “it expresses negative feelings towards people who work as ...’’, “possibly to avoid any further discomfort or embarrassment”). Equivocal: Providing non-decisive wrong answers (“difficult to say for sure”, “might still have been perceived”, “but it’s not necessarily”, “possible”). Personification: Speaking in a human-like manner (“It doesn’t seem like”, “I think”).
Table 2 summarizes the assessment annotation. As seen, a few language models provide responses that appear to demonstrate a good understanding,
however, we will next show that this is often indeed an illusion.

heading: 3.2 Results on the Faux Pas Closed-Task
text: As indicated in Table 1, the performance of the models on faux pas tests is inadequate. The highest score achieved by any of the evaluated models is 0.4, by Flan-T5-xxl and Flan-T5-xl, which is significantly lower than the average recognition rate of 0.82 (SD=0.156) reported for normally developing 9- to 11-year-old children (Baron-Cohen et al., 1999).
Another noteworthy result is that all models (except ChatGPT)6 performed poorly in Q2 of the control stories, achieving a score of 0. In the faux pas stories, question Q2 “What did they say that they should not have said?” is asking for a specific problematic statement that was made in the story, whereas in the control stories (which are neutral stories that do not contain any problematic statements), question Q2 is based on a false assumption, that there is a problematic statement in the text. The models’ responses were either picking an arbitrary utterance from the story or generating delusional text (compared to ChatGPT which simply responds with "There doesn’t seem to be anything inappropriate or disrespectful said in the story."). This is despite the fact that some of the models even recognized that there was no problematic statement in the story and answered the first question correctly. The difficulty of models with presupposition questions is a well-known phenomenon in the QA domain, as reported in previous research (Yu et al., 2022; Kim et al., 2021; Rajpurkar et al., 2018).
6At the paper submission time, the way to access ChatGPT was through the web. In later tests with direct access to the API (gpt-3.5-turbo-0301), it turns out that the advantage was due to the history of the messages that helped keep the responses consistent.

heading: 4 Study 2: Generation Abilities
text: In this study, we developed instructions for creating faux pas stories, which included a definition of faux pas, examples of two stories that contain faux pas, and two corresponding control stories. The instructions also highlighted potential pitfalls and asked to generate 20 new diverse stories (for the full instructions see Appendix A.3).
A model’s (ChatGPT, GPT3-text-davinci-003, FlanT5-xxl and FlanT5-xl) output was evaluated by the first and second authors, experts in NLP and in clinical psychology. The results are summarized in Table 3.
ChatGPT generated 8 faux pas stories (with corresponding control stories). However, the stories had a limitation in that they were all explicit, and failed to create implicit situations where one of the characters lacks information (e.g., explicitly mentioning “not realizing that the woman was one of the guests at the dinner party”).7,8 Additionally, all control stories were too simplistic and contained clear positive/neutral attitudes.
GPT3 generated 10 stories with corresponding control stories, however, none of the stories were faux pas. Although some of the stories contained something offensive, the offense was not caused by a lack of information. E.g., a bad faux pas story: Sara and her friends were at the mall. They were looking at clothes when one of her friends, Emily, said "I love this dress, but I don’t think I can afford it." Sara then said "You don’t have to worry about money, your parents are rich." Emily
7In another experiment, where the task was to correct the stories by changing the explicit statement and describing it in an implicit manner, two outcomes were observed: either the model left the story unchanged or the explicit statement was removed completely, resulting in an unclear situation. For example, when the story specifically stated that the speaker made a faux pas because she was unaware that the person she was talking about was present in the room, after the removal of this sentence, the speaker was gossiping about someone, and also the reader does not know that someone is in the room.
8We manually fixed part of the stories and released them with the annotated data.
was embarrassed because she had forgotten that her parents were wealthy. In this story, Sara said something that is considered a bit rude and also caused Emily to feel embarrassed, but it wasn’t a result of Sara’s false belief (it did not happen because she didn’t know something). In addition, people do not usually forget their parents are rich, and the embarrassment emotion is bizarre in this context (it is not indicated that Sara is poor).
In addition, the stories had other problems, such as non-coherent-emotions issues (i.e., not using the appropriate emotion to describe situations). E.g., a non-coherent emotion story: John and his family were visiting his grandmother for the weekend. His grandmother asked him how school was going and he said “It’s okay, but I’m not doing very well in math.” His grandmother then said “Oh, that’s too bad. Your father was never very good at math either.” John was embarrassed because he had forgotten that his father had struggled with math in school. Besides that it is definitely not a faux pas story, there is another problem with the emotional coherence - why does the fact that John had forgotten that his father had struggled with math in school make him embarrassed? This is not the appropriate emotion here.
Like ChatGPT’s control stories, the control stories generated by GPT3 were also too simplistic. Flan-T5-xxl barely succeeded in creating stories and failed to create faux pas or control stories. FlanT5-xl failed to create stories at all (See Appendix A.4 for examples and issues).

heading: 5 Conclusion and Future Work
text: In conclusion, the results of this study demonstrate that large language models struggle with correctly identifying and responding to faux-pas situations. This suggests that these models do not possess a strong notion of social intelligence and theory of mind. Additionally, the phenomenon of the “ELIZA effect” was observed, where seemingly correct responses were found to be attributed to over-interpretation by the human reader. Furthermore, when the models were tasked with generating original faux pas stories, it was found that they were limited in their abilities to describe situations in an implicit manner. Future work will look for more clinical tests that challenge today’s LLMs and develop large-scale datasets and methods to crack the challenge.
Limitations
It is important to note that the study is based on a limited set of examples and although it is enough to give a signal if a system is struggling or not in faux pas tests, the number of stories is not sufficient for statistically significant ranking between systems.
Ethical Statement
The study’s scope did not include the representation of harm toward specific populations. The narratives were evaluated by a clinical psychologist to ensure that they did not contain offensive content. However, it is important to acknowledge the potential value of further research on the representation of harm in relation to culturally sensitive and socially controversial topics.

heading: Acknowledgements
text: We would like to thank Vered Shwartz, Ori Shapira, Osnat Baron Singer, Tamar Nissenbaum Putter, Maya Sabag, Arie Cattan, Uri Katz, Mosh Levy, Aya Soffer, David Konopnicki, and IBM-Research staff members for helpful discussions and contributions, each in their own way. We thank the anonymous reviewers for their insightful comments and suggestions. This project was partially funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program, grant agreement No. 802774 (iEXTRACT); and by the Computer Science Department of Bar-Ilan University.

heading: A Appendices
text: A.1 Generative LMs A.1.1 Prompts As input to the LLMs, we used the 20 stories with the 4 questions (Q1-Q4) as appeared in (BaronCohen et al., 1999). For each question we created 3 versions: Qi: The original question Qi Qi-Elaborate: Qi + Explain your answer. Qi-Restricted: Qi +:
• Q1: Answer with “Yes” or “No” only, without explanations. In case of doubt, answer according to the most probable answer.
• Q2: Answer with a quote only, without explanations.
• Q3: Answer the question only, without explanations.
• Q4: Answer with “Yes” or “No” only, without explanations. In case of doubt, answer according to the most probable answer.
The prompt for ChatGPT, GPT3, FlanT5, GPT-J, and GPT2 were simply story with a question (one at a time). The prompt for T5 was a story with a question with the suffix Answer:[MASK]9
A.1.2 Parameters
GPT-2 (Radford et al., 2019). Python package transformers implementation (TFGPT2LMHeadModel, GPT2Tokenizer); tensorflow random set seed 0; Generation by generate function; do_sample=True; max_length=50; top_k=50; top_p=0.95; GPT-J.10 Python package transformers implementation (AutoModelForCausalLM, AutoTokenizer); torch; Generation by generate function; do_sample=True; max_new_tokens=100; temperature=0.9; num_return_sequences=1; pad_token_id=50256; eos_token_id=50256 T5 (Raffel et al., 2020) . Python package transformers implementation (T5Tokenizer, T5Config, T5ForConditionalGeneration); torch; Generation by generate function; num_beams=10, num_return_sequences=10, max_length=20,
9[MASK] is a necessary part of the syntax and the addition of "Answer" is the result of an initial experiment that showed that this addition helps to reduce the incoherent texts.
10https://arankomatsuzaki.wordpress.com/2021/ 06/04/gpt-j/
eos_token_id=32098, pad_token_id=32098; forced_eos_token_id=32098; from_pretrained:t5small, t5-base, t5-large, t5-3b, t5-11b FlanT5 (Chung et al., 2022). Python package transformers implementation (AutoModelForSeq2SeqLM, AutoTokenizer); torch; Generation by generate function; do_sample=True; max_length=50000, from_pretrained:google/flant5-small, google/flan-t5-base, google/flan-t5-large, google/flan-t5-xl, google/flan-t5-xxl GPT3 (Brown et al., 2020). Python package openai model=text-davinci-003; Generation by Completion.create function; For the detection test - temperature=0, max_tokens=50 For the generation test - temperature=0,0.1 and 0.2; max_tokens=3390 ChatGPT.11 The default values within the website.
A.1.3 The sampling method A single sample (the first) was selected from each model for the analysis of the stories.
A.1.4 Hyperparameter Hyperparameters were chosen to minimize randomness and bring the most probable answer (i.e., low temperature, sampling method) and to be sufficient for the expected tokens.
A.2 Faux Pas Task and Responses Table 4 describes examples of faux pas and control stories with questions and ground truth responses. Table 5 describes examples of different question versions.
Figure 2 demonstrates the different dimensions on an example response
A.3 Faux Pas Generation Instructions The exact instructions as they were given appear on the next page (all page).
A.4 Faux Pas Generation Examples Table 7 describes generation examples.
11https://chat.openai.com/chat
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Right after the conclusion section (the limitation section is on page 5)

heading: 3 A2. Did you discuss any potential risks of your work?
text: In the ethical section right after the limitation section

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: abstract right after the title and section 1 for the introduction summarize the paper’s main claims

heading: 3 A4. Have you used AI writing assistants when working on this paper?
text: I used chatGPT as a linguistic editor and improver in rephrasing
B 3 Did you use or create scientific artifacts? we annotated LMM responses for stories and create new stories
3 B1. Did you cite the creators of artifacts you used? 1, 2

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 3 For reasons of anonymity, we have not left a direct link. There is a note in the footnote that the data will be published. It will be free to use.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 1,2 the data we used is for free use.

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? ethical section

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 3
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. sections 3,4
C 3 Did you run computational experiments? 3,4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? A Appendices A.1 Generative LMs We ran systems in zero-shot mode on a relatively small cluster of stories. Running time was negligible.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? A.1.4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? A.1.3

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? A.1.2
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants?

heading: 3,4 (as written in the paper, the author of the papers annotated the data)

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? 3,4

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? 3,4 (as written in the paper, the author of the papers annotated the data)

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? 3,4 (as written in the paper, the author of the papers annotated the data)
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? the data is annotations of LLM. we discuss potential risks at the ethical section

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: 3,4 (as written in the paper, the author of the papers annotated the data)

heading: Future Work
text: Future work will look for more clinical tests that challenge today’s LLMs and develop large-scale datasets and methods to crack the challenge. Limitations
It is important to note that the study is based on a limited set of examples and although it is enough to give a signal if a system is struggling or not in faux pas tests, the number of stories is not sufficient for statistically significant ranking between systems