## 2 Methodology
### Audio-Guided Space-Time Memory Network  
Space-time memory networks excel at tracking targets using given segmentation masks [22]. Unlike semi-supervised video object segmentation methods, their key advantage is the ability to store and retrieve historical information, which both speeds up the process and improves tracking accuracy. If the space-time memory network model is sufficiently lightweight, it can achieve real-time target tracking. However, beyond the challenges of tracking accuracy and speed, practical application environments introduce additional complexities. Ubiquitous environmental noise and interfering sound sources can lead the model to segment silent targets. To address this issue, this study proposes an audio-guided space-time memory network (AG-STMNet), as illustrated in Figure 4. The network consists of two key components: the space-time memory network, which conducts video object segmentation guided by the fusion mask, and the audio feature extraction and fusion module. In the space-time memory network module, the historical frame object masks are encoded by the memory encoder, generating corresponding keys and values. These keys and values are then chronologically concatenated and converted to the memory key embedding \(k^{M}\) and memory value embedding \(v^{M}\). Upon the arrival of the current frame, the query encoder is employed to encode it, producing the corresponding key embedding \(k^{Q}\) and value embedding \(v^{Q}\). Subsequently, a spatial matching mechanism is utilized to determine whether each pixel belongs to the foreground object, thereby completing the segmentation of the target object in the current frame. This process effectively computes the spatio-temporal attention of each pixel in the query image with respect to the pixels in the historical frames. The spatio-temporal memory query process is mathematically expressed by the following formula [22]:  
\[\mathbf{y}_{i}=[v^{Q}_{i},\frac{1}{Z}\sum_{j}f(k^{Q}_{i},k^{M}_{j})v^{M}_{j}] \tag{4}\]  
Figure 3: The mask generator operates through the following detailed procedure. First, an image is input into Fastsam to generate a preliminary segmentation mask. Second, the image is cropped based on the mask to extract the candidate region’s image block. Next, the image block and its synchronized audio segment are processed by the Wav2 CLIP model to produce corresponding image embeddings and an audio embedding. Finally, the image embeddings, audio embedding, and image block patches are fed into the mask-scoring module, where the final mask is generated according to the predefined scoring mechanism. \(y\) represents the query output, where \(i\) and \(j\) denote the indices of each position in the query embedding and memory embedding feature map, respectively. The notation \([\cdot,\cdot]\) indicates the concatenation and stacking operation, while \(f\) denotes the similarity calculation function. \[f(k_{i}^{Q},k_{j}^{M})=\exp(k_{i}^{Q}\circ k_{j}^{M}) \tag{5}\]  
The \(\circ\) operation represents the dot product operation, and Z denotes the normalization factor. \[Z=\sum_{j}f(k_{i}^{Q},k_{j}^{M}) \tag{6}\]  
In the aforementioned formula, \(k^{Q}\in\mathbb{R}^{H\times W\times C/8}\), \(v^{Q}\in\mathbb{R}^{H\times W\times C/2}\), \(k^{M}\in\mathbb{R}^{T\times H\times W\times C/8}\), \(v^{M}\in\mathbb{R}^{T\times H\times W\times C/2}\), where H denotes the height of the image, W represents the width of the image, C signifies the number of dimensions in the feature map, and T corresponds to the number of historical frames retained. The audio feature extraction and fusion module consists of two one-dimensional convolutional layers. The first layer maps the input dimension to an intermediate dimension, followed by dimensionality reduction via max pooling. The second one-dimensional convolutional layer then maps the intermediate dimension to the output dimension, extracting the final audio feature.