Game-Time: Evaluating Temporal Dynamics in Spoken Language Models
Authors: name: Kai-Wei Chang, name: En-Pei Hu, name: Chun-Yi Kuan, name: Wenze Ren, name: Wei-Chih Chen, name: Guan-Ting Lin, name: Yu Tsao, name: Shao-Hua Sun, name: Hung-yi Lee, name: James Glass
Organization: Massachusetts Institute of Technology

Conversational Spoken Language Models (SLMs) are emerging as a promising
paradigm for real-time speech interaction. However, their capacity of temporal
dynamics, including the ability to manage timing, tempo and simultaneous
speaking, remains a critical and unevaluated challenge for conversational
fluency. To address this gap, we introduce the Game-Time Benchmark, a framework
to systematically assess these temporal capabilities. Inspired by how humans
learn a language through language activities, Game-Time consists of basic
instruction-following tasks and advanced tasks with temporal constraints, such
as tempo adherence and synchronized responses. Our evaluation of diverse SLM
architectures reveals a clear performance disparity: while state-of-the-art
models handle basic tasks well, many contemporary systems still struggle with
fundamental instruction-following. More critically, nearly all models degrade
substantially under temporal constraints, exposing persistent weaknesses in
time awareness and full-duplex interaction. The Game-Time Benchmark provides a
foundation for guiding future research toward more temporally-aware
conversational AI. Demos and datasets are available on our project website
https://ga642381.github.io/Game-Time.