# VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT
## 1 Introduction  
Given a linguistic query, video temporal grounding (VTG) aims to locate the most relevant temporal segments from an untrimmed video, each containing a start and end timestamp. An illustrative example of VTG is shown in Figure 0(a). This task [1, 2] has numerous practical applications in daily life, such as how it can help video platform users easily skip to relevant portions of a video. The field of natural language has witnessed a significant leap forward with the advent of GPT-4 [3]. This development has spurred the rise of large language models (LLMs) such as LLaMA [4] and Baichuan2 [5]. Concurrently, GPT-based (Generative Pre-trained Transformer) models like MiniGPT4 [6] and LLAVA [7] have made significant strides in vision and multimodal applications. A recent work, LLAViLo [8], reveals that training adapters alone can effectively leverage the video understanding capabilities of LLMs. However, this method requires designing a sophisticated fine-tuning strategy specifically for VTG, thereby introducing additional computing costs. Existing VTG methods [1, 9, 10, 11, 12] primarily adopt supervised learning, which demands massive training resources and numerous annotated video-query pairs, as illustrated in Figure 0(b). However, developing datasets for VTG is time-consuming and expensive; for instance, Moment-DETR [1] spent 1455 person-hours and USD 16,600 to create the QVhighlights dataset. Furthermore, ground-truth (GT) queries often contain human biases, such as (1) Bias from erroneous word spellings, as depicted in Figure 1(a). The misspelled word _\"ociture\"_ in original query would be tokenized by language models into _\"o\"_, _\"cit\"_, _\"ure\"_, leading to model misunderstanding; (2) Bias due to incorrect descriptions, as shown in Figure 1(b). Here, the action _\"turns off the lights\"_ mentioned in the query does not occur in the video. In this paper, we propose a tuning-free zero-shot method named VTG-GPT to address the above issues. As shown in Figure 1c, VTG-GPT completely satisfies zero-shot settings, adopting a direct feed-forward approach without training or fine-tuning. To minimize biases arising from human-annotated queries, we employ Baichuan2 [5] to rephrase the original query and obtain debiased queries. As illustrated in Figure 2, the erroneous word \"_ociture_\" in query (a) has been accurately revised to \"_image_\", and the non-existent action \"_turn off the lights_\" in query (b) has been effectively refined to \"_a darkened environment_\". Furthermore, considering that videos inherently contain more redundant information than text, and inspired by the human approach to understanding video linguistically, we apply MiniGPT-v2 [6] to transform visual content into more precise textual descriptions. Finally, to generate accurate temporal proposals, we design a proposal generator that models debiased queries and image captions in the textual domain. In summary, our main contributions include:  
(1) To the best of our knowledge, we are the first zero-shot method to utilize GPT on VTG without training or fine-tuning. (2) We present a novel framework, VTG-GPT, which effectively leverages GPT to mitigate human prejudice in annotated queries. Furthermore, VTG-GPT distinctively models debiased queries and video content within the linguistic domain to generate temporal segments. (3) Comprehensive experiments demonstrate that VTG-GPT significantly surpasses SOTA (State-of-the-Art) methods in zero-shot settings. More importantly, this method achieves competitive performance comparable to supervised methods. Figure 1: (**a**) An illustrative example of a video temporal grounding (VTG) task. (**b**) Previous methods require training for all modules. (**c**) Our proposed VTG-GPT operates without any training or fine-tuning. Moreover, it employs GPT to reduce bias in human-annotated queries. Figure 2: Human biases in ground-truth queries arise from (**a**) misspelled words and (**b**) incorrect descriptions.