GPT -4 is Judged More Human than Humans in Displaced and Inverted T uring Tests (2025)

Abstract: Everyday AI detection requires differentiating between humans and AI in informal, online conversations. At present, human users most often do not interact directly with bots but instead read their conversations with other humans. We measured how well humans and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. We found that both AI and displaced human judges were less accurate than interactive interrogators, with below chance accuracy overall. Moreover, all three judged the best-performing GPT-4 witness to be human more often than human witnesses. This suggests that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations.

## introduction in 1950, alan turing devised "the imitation game" as a test to indirectly investigate the question, "can machines think?" in a classic turing test, a human interrogator engages in a text-only conversation with two witnesses: one human and one machine. if the interrogator is unable to accurately differentiate between the human and the computer, the computer passes the test and can be considered intelligent. since turing's original paper, the turing test has sparked an intense debate that has been pivotal in constructing modern understandings and conceptions of intelligence, shaping the fields of computer science, cognitive science, artificial intelligence, robotics, philosophy, psychology, and sociology @xcite . beyond its controversial role as a test of intelligence, the turing test also serves as a measure of whether humans can detect ai in conversational settings, or whether ai models can successfully deceive human interlocutors into thinking that they are human. recent empirical work has found that interrogators could not reliably determine whether a gpt-4-based agent was human or ai in a turing test @xcite . models that can successfully impersonate people bring attendant risks, including persuasion and deception. especially in informal settings, which may include online forums such as reddit or quora, chatting platforms such as discord, and social media platforms, an llm-based agent could contribute to misinformation, scams, or be maliciously misused at a wide scale. in these open-access settings, posts, messages, or chats containing llmgenerated content would not only have persuasive power over an active interrogator but also over several consumers of the content @xcite . this motivates conducting variations of the turing test in more ecologically valid settings to determine how effective people are in discriminating between humans and ais in realistic scenarios. an ordinary turing test provides the interrogator with a key advantage not always present in passive consumption of ai-generated text: they can adapt their questions to adversarially test the witness in real time. here, we ask how well human and ai judges perform without this advantage, when they only have access to a transcript of a turing test interview conducted by a separate participant. ## study 1: inverted turing test we first place gpt-3.5 and gpt-4 in the role of the evaluator in an inverted turing test. because the models are not actively interrogating the witness, we refer to this more passive role as the adjudicator. we presented both models with transcripts of previously-run turing tests @xcite and elicited judgements about whether the witness was a human or an ai. we pre-registered 5 hypotheses on osf. first, we asked whether gpt-4 accuracy would be significantly greater or less than human accuracy. second, in the original study, interactive human interrogators were no better than chance at determining whether one gpt-4 witness was human or ai. we asked whether gpt-4 would also fail to accurately identify this witness. third, in order to test whether gpt-4 could distinguish between real humans and well-designed systems, we asked whether the model would judge real humans to be human more often than it judged the best gpt-4 witness to be human. fourth, in order to understand the role that the length of the conversations plays in their identifiability, we asked whether gpt-4 accuracy as an adjudicator would be higher for longer transcripts. finally, to investigate the role of llm model quality in judging transcripts, we asked whether gpt-4 would have higher accuracy as an adjudicator than gpt-3.5. ## study 2: displaced turing test the original turing test was conceived as a test of intelligence-a crucial component of which is the interactive and adversarial nature of the interrogator. however, in many real-life scenarios, people do not have the opportunity to directly query potential ai agents: for instance, when reading a conversation between two interlocutors on social media. to investigate people's accuracy in identifying ai systems in this more passive context, we mirrored the inverted turing test study with humans-a design we call the displaced turing test. displaced human adjudicators evaluated the same sample of transcripts and made judgements about whether the witness was human or an ai. we pre-registered a similar set of questions for this design on osf. we asked whether displaced human adjudicators would be more or less accurate than the original interactive interrogators, and whether they would be significantly more accurate than gpt-4 as an adjudicator. in addition, we asked whether they would be above chance in determining that the same best gpt-4 witness was an ai, and whether they would judge actual humans to be human more frequently than the best gpt-4 witness. finally, we asked whether transcript length would influence adjudicator accuracy. ## conclusion we conducted an inverted turing test, in which gpt-3.5 and gpt-4 judged whether one interlocutor in a transcript was human, and mirrored this approach in a displaced test, where human adjudicators read the same transcripts. we found that both ai adjudicators and displaced human adjudicators were less accurate than interactive interrogators who had conducted the original turing test, but not more or less accurate than each other. this suggests that neither ai nor humans are reliable with detecting ai-contributions to online conversations.