## 1 Introduction  
Temporal sentence grounding aims to determine the start and end positions in an untrimmed video corresponding to natural language descriptions. This task provides access to specific video content without having to watch the entire video, making it valuable for various computer vision applications, including video surveillance [1], sports analytics [2], and short-term video recommendation [3], etc. While recent fully supervised algorithms [4; 5; 6; 7; 8; 9] have made significant breakthroughs, they demand extensive annotations of temporal boundaries and language queries for each video, which can be both expensive and time-consuming. As a result, weakly supervised learning methods have attracted increasing attention in recent years. Weakly supervised learning approaches rely solely on video and language queries during the training stage, making them more scalable and practical for real-world applications. Most existing weakly supervised approaches for temporal sentence grounding typically utilize scoring-based (Figure 1a), reconstruction-based (Figure 1b), and RL-based strategies (Figure 1c) to identify the temporal moments. Scoring-based methods [10; 11; 12; 13; 14; 15] typically commence by generating a series of candidate video proposals with a sliding window. Subsequently, they learn scores that indicate the potential alignment between video proposals and the language query. Reconstruction-based methods [16; 11; 17] minimize the reconstruction loss between partially masked queries and video proposals, and ultimately selecting the proposal that best reconstructs the query. RL-based methods [8; 18; 19] refine candidate proposals by learning effective policies. Although these methods have achieved significant performance improvements, there still face inherent limitations. Firstly, scoring-based methods use a sliding window to generate a set of video clips, leading to a substantial imbalance between the numbers of negative and positive samples. Such an unbalanced distribution can cause the network to be more inclined toward locating negative samples rather than accurately identifying positive ones, which contradicts conventional understanding [20]. Furthermore, scoring-based methods heavily depend on alignment modules that illustrate the consistency between video-query pairs [21]. Secondly, while reconstruction-based methods avoid the issue of generating a large number of negative samples, they tend to rely more heavily on video features due to language bias and prioritize the localization of frequently occurring words [22; 23; 24]. Thirdly, existing reinforcement learning (RL)-based methods lack a coherent strategy for the movement policies involved in selecting candidate proposals, introducing significant randomness into the system. Furthermore, most current methods employ these three strategies independently, overlooking their potential complementarity. To tackle the aforementioned challenges, we introduce a multi-policy movement framework (MMF), a pioneering RL-based framework with a Gaussian-based Dual-Alignment Module (GDAM) and multi-policy movement strategy (MMS). The MMF emulates human behavior when localizing temporal boundaries based on the language query. It begins from an initial window position and then adjusts the window step by step towards the corresponding position of the query. Specifically, the GDAM integrates both scoring-based and reconstruction-based alignment approaches. It redefines the reward function to more accurately reflect the semantic alignment between the current window's video segment and the query. This combination harnesses the strengths of both methods, enabling a more comprehensive assessment of cross-modal feature consistency from both global and local perspectives. Additionally, to adapt the training process, we refine the alignment scoring using a dynamic Gaussian mask that allocates adaptive attention to different positions within the video features, thereby facilitating accurate localization of temporal boundaries.