OpenGVL - Benchmarking Visual Temporal Progress for Data Curation
Authors: name: Paweł Budzianowski, name: Emilia Wiśnios, name: Gracjan Góral, name: Igor Kulakov, name: Viktor Petrenko, name: Krzysztof Walas

Data scarcity remains one of the most limiting factors in driving progress in
robotics. However, the amount of available robotics data in the wild is growing
exponentially, creating new opportunities for large-scale data utilization.
Reliable temporal task completion prediction could help automatically annotate
and curate this data at scale. The Generative Value Learning (GVL) approach was
recently proposed, leveraging the knowledge embedded in vision-language models
(VLMs) to predict task progress from visual observations. Building upon GVL, we
propose OpenGVL, a comprehensive benchmark for estimating task progress across
diverse challenging manipulation tasks involving both robotic and human
embodiments. We evaluate the capabilities of publicly available open-source
foundation models, showing that open-source model families significantly
underperform closed-source counterparts, achieving only approximately 70% of
their performance on temporal progress prediction tasks. Furthermore, we
demonstrate how OpenGVL can serve as a practical tool for automated data
curation and filtering, enabling efficient quality assessment of large-scale
robotics datasets. We release the benchmark along with the complete codebase at
github.com/budzianowski/opengvl{OpenGVL}.