Big Escape Benchmark: Evaluating Human-Like Reasoning in Language Models via Real-World Escape Room Challenges (2025)

Abstract: Large Language Models (LLMs) have recently demonstrated remarkable reasoning capabilities across a wide range of tasks. While many benchmarks have been developed on specific academic subjects, coding, or constrained visual tasks, they often fail to fully capture the breadth, diversity, and dynamic nature of real-world human reasoning. Further, the creation of high-quality, complex multimodal reasoning benchmarks typically requires significant manual effort and expert annotation, which is costly and time-consuming.To address these limitations, we introduce Big Escape Bench, a novel multimodal reasoning benchmark derived from popular reality shows and television programs. Big Escape Bench leverages unique characteristics of TV content, providing a rich source of challenging and realistic multimodal reasoning problems. Key advantages include: questions guaranteed to be human-solvable and of moderate difficulty; problems reflecting diverse, real-world scenarios and knowledge domains; high inherent quality due to content generated by professional program teams.Notably, we develop an automated pipeline to construct the data from these programs into a standardized benchmark format, significantly reducing the manual effort compared to traditional dataset construction. We have conducted extensive experiments to evaluate state-of-the-art (SOTA) LLMs and Multimodal Large Language Models (MLLMs) on Big Escape Bench. Our results reveal a surprising performance gap: while the questions are easily solved by human viewers (about 60% in accuracy), the performance of even the most advanced models (best 40.50% in accuracy) is significantly lower than human-level accuracy. This highlights that despite recent progress, MLLMs still face substantial challenges in robustly performing the kind of diverse, dynamic, and context-dependent reasoning that is trivial for humans in routine situations. Big Escape Bench serves as a valuable tool for identifying current limitations of MLLMs and fostering future research towards more human-like multimodal reasoning.

## introduction recent years have witnessed unprecedented progress in the reasoning capabilities of llms @xcite and mllms @xcite @xcite , with state-of-the-art (sota) systems achieving human-competitive performance on specialized tasks such as mathematical problem solving @xcite @xcite @xcite , code generation @xcite @xcite , and constrained visual question answering @xcite @xcite . however, these successes often rely on benchmarks that prioritize narrow, domain-specific expertise (e.g., math @xcite for math, hu-maneval @xcite for coding) or static, artificially constructed multimodal tasks (e.g., image captioning or vqa datasets). however, such benchmarks are not sufficient to capture the breadth, diversity, and dynamic nature of real-world reasoning, where humans seamlessly integrate multimodal information, adapt to novel contexts, and apply commonsense knowledge to solve open-ended problems. a critical gap persists in evaluating models on reasoning tasks that mirror the complexity of human challenges. existing benchmarks face several key limitations: (a) the scope of many existing benchmarks is limited, disproportionately emphasizing performance in specific technical domains, such as math and code, while overlooking the assessment of more general, contextually embedded reasoning abilities critical for realworld understanding. (b) benchmarks constructed through static, manual processes often result in homogeneous question sets, thereby failing to capture the innovation and rich variability inherent in dynamic, real-world scenarios. (c) the development of complex and high-fidelity multimodal reasoning datasets typically incurs substantial human costs, stemming from the requirement for labor-intensive annotation and expert validation processes. for instance, benchmarks like mmmu @xcite or gpqa @xcite , while comprehensive, focus on academic subjects and rely on curated, domain-specific content. this leaves open the question of whether current models can generalize to more diverse, complex, and real-world reasoning demands. to address these challenges, we introduce big escape benchmark, a novel multimodal reasoning benchmark derived from popular reality shows and television programs (e.g., the great escape and the 1% club). tv content has unique characteristics that offer untapped resources for benchmarking: questions are designed by professional production teams to challenge human contestants, ensuring they are inherently solvable, contextually grounded, and dynamically varied. by leveraging these resources, big escape benchmark offers significant benefits, including (1) human-aligned difficulty: all problems are vetted for solvability by human participants, ensuring a balanced evaluation of model capabilities without artificial extremes (e.g., trivial or impossibly niche questions); (2) diverse and real-world knowledge: questions span broad domains (e.g., logic, commonsense, cultural references) and tasks, reflecting the integrative demands of real-life reasoning; beyond the conceptual strengths of big escape benchmark, the benchmark collection pipeline also introduces methodological innovation. specifically, we develop an automated pipeline to extract, preprocess, and standardize tv content into a scalable benchmark, minimizing manual annotation while preserving the richness of the original material. we leverage an automated pipeline that begins with accurate transcript generation using tools like videolingo, followed by gpt-4o-mini @xcite for refinement. subsequently, a sophisticated llm, claude-3.7-sonnet @xcite , is employed to analyze dialogue and extract problem instances along with relevant clues from the video content. importantly, this approach not only reduces costs but also enables future expansion to new programs or regions. we have conducted extensive experiments evaluating multiple advanced llms (e.g., deepseek v3 @xcite , grok 3 beta (x.ai, 2025)) and mllms (e.g., qwen2.5-vl-instruct @xcite , gpt-4olatest @xcite , gemini-2.5 (google, 2025), o4-mini (openai, 2025)) on our big escape benchmark. while human viewers can easily solve these problems with high accuracy (about 60%), the performance of even the most advanced models (e.g., leading proprietary models like claude-3.7-sonnet @xcite and gemini-2.5-pro (google, 2025)) test falls considerably short, trailing human performance by over 30%. our analysis reveals a significant performance gap between open-source and proprietary models. we also find that while model scaling and the integration of sophisticated reasoning mechanisms can yield high performance, these approaches often encounter diminishing returns or introduce efficiency trade-offs. furthermore, we observe that wrong reasoning ideas, rather than incorrect information extraction, are a primary driver of model failures; indeed, models with strong reasoning capabilities can exhibit a tendency to overthink textual information. this stark contrast underscores that despite rapid advancements, llms and mllms still face substantial challenges in robustly performing the diverse, dynamic, and context-dependent reasoning at which humans excel. ## conclusion we introduce big escape benchmark, a novel multimodal reasoning benchmark derived from reality tv shows, addressing the diversity, dynamism, and creation-cost limitations of current benchmarks. big escape benchmark features human-solvable, diverse, high-quality problems via an automated pipeline. experiments revealed a significant performance gap: humans achieve approximately 60% accuracy, while top models reach only about 40.50%. this highlights that even advanced mllms struggle with human-like, context-dependent reasoning. our analysis indicates that flawed reasoning approaches are the primary error source. big escape benchmark offers a valuable tool to identify mllm limitations and guide future research towards more robust multimodal reasoning.