X - L e B ench: A Benchmark for Extremely Long Egocentric Video Understanding (2025)

Abstract: Long-form egocentric video understanding provides rich contextual information and unique insights into long-term human behaviors, holding significant potential for applications in embodied intelligence, long-term activity analysis, and personalized assistive technologies. However, existing benchmark datasets primarily focus on single, short (e.g., minutes to tens of minutes) to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. To address this, we introduce X-LeBench, a novel benchmark dataset meticulously designed to fill this gap by focusing on tasks requiring a comprehensive understanding of extremely long egocentric video recordings. Our X-LeBench develops a life-logging simulation pipeline that produces realistic, coherent daily plans aligned with real-world video data. This approach enables the flexible integration of synthetic daily plans with real-world footage from Ego4D—a massive-scale egocentric video dataset covers a wide range of daily life scenarios—resulting in 432 simulated video life logs spanning from 23 minutes to 16.4 hours. The evaluations of several baseline systems and multimodal large language models (MLLMs) reveal their poor performance across the board, highlighting the inherent challenges of long-form egocentric video understanding, such as temporal localization and reasoning, context aggregation, and memory retention, and underscoring the need for more advanced models.

## introduction understanding long-form egocentric videos captured from a first-person perspective over extended periods holds significant potential for advancing various domains such as embodied intelligence, long-term activity analysis, and personalized assistive technologies @xcite @xcite . these videos provide rich contextual information and unique insights into human behaviors as they unfold naturally throughout the day. the ability to analyze such extensive recordings is key to developing more personalized agent systems that can construct long-term memory, anticipate user needs, and interact seamlessly in real-world settings @xcite @xcite . however, most existing datasets @xcite @xcite @xcite , predominantly feature short individual video clips captured from third-person views, making them insufficient for understanding the continuous, nuanced context of daily human activities or for indepth human-centered research that requires a firstperson perspective. while more recent benchmarks and datasets focus on egocentric video understanding @xcite @xcite @xcite @xcite , they remain limited in capturing continuous, long-term human activities. for instance, ego4d @xcite features massivescale egocentric videos covering a wide range of daily activities, with video durations ranging from 07:30 -07:32 c walks inside the house, washes hands, and dries them with a towel. 18:35 -18:44 c prepares a meal in the kitchen, eats, and watches television in the living room. figure 1 : example of generated video life logs in x-lebench. generated video life logs consist of multiple videos with corresponding timestamps. the visualization shows the time organization and content allocation of data. user queries. without extensive, continuous contextual data, evaluating the robustness of their performance becomes difficult. creating benchmark datasets that span several hours of egocentric video is necessary but presents significant challenges. data acquisition is a primary hurdle, requiring participants to wear recording devices for extended periods is labor-intensive and raises privacy concerns. device limitations, including storage constraints and reliability issues, further complicate continuous video capture. additionally, annotating long-form videos is timeconsuming and prone to annotator fatigue, affecting label accuracy and consistency. to address these challenges, we introduce x-lebench, a versatile and scalable benchmark dataset designed for evaluating tasks on extremely long egocentric videos. x-lebench features a lifelogging simulation pipeline that simulates extended video logs by integrating short (seconds) or moderately long (hours) video clips with dynamically generated daily plans. leveraging large language models (llms), it generates realistic, contextually rich schedules aligned with real-world activities based on adjustable input settings. specifically, this simulation integrates synthetic daily plans with actual footage from ego4d, then iteratively optimizes the simulation process based on retrieved information, producing video life logs that mirror daily activities in rich contexts with duration extended to dozens of hours. a generated sample is shown in fig. 1 , showcasing video segments with different timestamps. notably, x-lebench offers a customizable and scalable design, enabling the synthesis of datasets with various durations and content to accommodate diverse research needs. our initial evaluations of baseline systems and multi-modal llm (mllm) reveal consistently poor performance on x-lebench, highlighting the inherent difficulties of long-form egocentric video understanding and underscoring the need for more advanced models capable of interpreting and analyzing ultra-long egocentric videos. our contributions are: (1) we present x-lebench, the first benchmark dataset that encompasses ultra-long egocentric video recordings; (2) we introduce a novel and customizable pipeline that simulates realistic, hours-long egocentric video life logs by integrating synthetic daily plans with real-world footage; (3) we conduct extensive evaluations of existing models on x-lebench, exposing significant performance gaps and key challenges for future research. ## x-lebench in real-world scenarios, continuous egocentric lifelogging is often constrained by hardware limitations and privacy concerns, making uninterrupted long-term recordings infeasible. consequently, extended life logs must be reconstructed from multiple video segments captured at different times. existing benchmark datasets lack the structural continuity and task setup required for studying temporally extended, cross-hour behaviors and reasoning. building on this insight, we develop the lifelogging simulation pipeline, enabling scalable and customizable simulation of hours-long video life logs, while maintaining reality and contextual coherence. we also define tasks that better reflect real-world long-term requirements and go beyond ego4d's original scope. the following sections detail our methodology for x-lebench creation. ## conclusion we introduce x-lebench, the first benchmark for ultra-long egocentric video understanding, featuring a customizable life-logging simulation pipeline. by curating 432 video life logs from ego4d, categorized into short, medium, and long durations. x-lebench provides a diverse, structured dataset for long-form video analysis. it includes tasks in temporal localization, summarization, counting, and ordering, offering a rigorous evaluation framework. we hope to advance research in long-form egocentric video processing, fostering the development of more robust and temporally aware ai models.