## 3 Proposed Method
### Gaussian-Based Dual-Alignment Module  
The goal of the GDAM is to score the semantic consistency between the video segment and language query. According to the alignment score, we can determine if the window position of the current movement is optimal. We take advantages of both scoring-based and reconstruction-based methods to evaluate the cross-modal alignment in global and local perspectives. **Scoring-based alignment:** We employ the scoring-based strategy to predict the alignment score of the video-textual pairs in the neural network by optimizing a rank loss, encouraging scores of matched video-query pairs to be higher than the ones of unmatched pairs. Specifically, we use the cross-modal fusion feature in visual space \(V_{c}^{\prime\prime}\) to compute the alignment score. Denoting the cross-modal fusion feature of the matched video as \(V_{pos}^{\prime\prime}\) and the one of the unmatched videos as \(V_{neg}^{\prime\prime}\), the rank loss is formulated as  
\[\mathcal{L}_{rank}=\max\Big{(}0,\delta-O\Big{(}V_{pos}^{\prime\prime})+O\Big{(} V_{neg}^{\prime\prime}\Big{)}\Big{)} \tag{4}\]  
where \(O(\cdot)\) is the scoring function, and \(\delta\) is the margin parameter. In this module, we compute the global alignment between the whole video and query, namely, employing the global features \(V_{global}^{\prime\prime}\) of matched and unmatched videos. Different from existing scoring-based methods [10; 11; 12] which directly predict a scaler as the alignment score, we learn a score vector with a dynamic Gaussian mask \(G_{\mu,\sigma}\), assigning varied attention values to different positions of the candidate window according to the training process,  
\[O\big{(}V_{c}^{\prime\prime}\big{)}=\
u\big{(}FC_{3}\big{(}V_{c}^{\prime\prime }\big{)}\big{)}^{\top}\cdot G_{\mu,\sigma} \tag{5}\]  
where \(\
u\) is the sigmoid function, \(FC_{3}\) is the fully connected layer, and \(\mu\) and \(\sigma\) are the mean and variance of the Gaussian distribution, respectively. The dynamic Gaussian mask varies with the training process,  
\[G_{\mu,\sigma}= \begin{cases}\mathcal{N}(\mu,\sigma),&e<E\\ (1-\mathcal{N}(\mu,\sigma)/\max(\mathcal{N}(\mu,\sigma)),&e\geq E\end{cases}  \tag{6}\]  
where \(e\) is the training epoch, \(E\) is the hyper-parameter, the mean \(\mu\) is set to \(L/2\), and the variance \(\sigma\) is  
\[\sigma= \begin{cases}e^{2},&e<E\\ (20-e)^{2},&e\geq E\end{cases}  \tag{7}\]  
which increases before \(E\) epochs and then decreases. As shown in Figure 3, at the beginning of the training stage, the dynamic Gaussian mask allocates higher attention to the center of the video, which makes the alignment process rely more on the main content of the video. After \(E\) epochs of training, the normal distribution is inverted and gives higher attention to the boundaries of the video and thus helps to determine accurate temporal positions in the later process. By rectifying the score vector with the dynamic Gaussian mask, the final score \(O(V_{c}^{\prime\prime})\) reflects the alignment of the video segment and language query reasonably, and the model is more stable to converge. In addition, it is intuitive that the alignment score of the middle part of the video is usually higher than the part near the boundaries. Therefore, we add a guidance constraint on the score vector, encouraging elements near the middle dimension to have higher values than side dimensions,  
\[\mathcal{L}_{guidance}=\sum_{i=0}^{d/2}\frac{1}{\mathcal{O}[i]-O[i+1]}+\sum_{i=d /2}^{d}\frac{1}{\mathcal{O}[i+1]-O[i]} \tag{8}\]  
where \(O[i]\) is the \(i\)th element of the score vector. **Reconstruction-based alignment:** The reconstruction-based method tries to reconstruct the masked query from the video, and the reconstruction loss indicates the semantic alignment between the video and query pairs. We compute the reconstruction loss based on the cross-modal fusion feature in the textual space and the whole video. Specifically, the input query is first randomly masked and then encoded by Equation (2).