## 3 Experiments and Results
### Audio-Guided Space-Time Memory Network
#### 3.2.2 Experiments  
(1) Training  
To enhance the accuracy and generalization capability of AG-STMNet while reducing its parameter count for greater lightweight efficiency, a two-stage training method was employed in this experiment. In the first stage, ResNet18 was used as the backbone network, and the space-time memory network component was trained on the MS COCO dataset. Subsequently, the encoder parameters of the trained space-time memory network were frozen, and an audio feature extraction and fusion module was integrated into the decoder. Fine-tuning was then performed using the Ref-AVSbench dataset. During the initial training of the space-time memory network, video sequences were processed frame-by-frame in groups of three frames [22]. The detailed procedure is as follows: (1) Key-value pairs are extracted from the first frame. (2) The second frame uses the memory prediction mask from the first frame to generate the prediction probability for the current frame. After calculating the loss for the second frame, the memory is updated. (3) The third frame integrates the memories of the preceding two frames for prediction, and the loss for the third frame is computed. The total loss is obtained by summing the prediction losses of the second and third frames, with the loss formula as follows:  
\[\text{loss}=n2\_loss+n3\_loss=L(n2\_\log it,n2\_label)+L(n3\_\log it,n3\_ label) \tag{7}\]  
\[L(p,y)=-[y*\log(p)+(1-y)*\log(1-p)] \tag{8}\]  
Let \(L\) denote the binary cross-entropy loss function, where \(p\) represents the predicted probability and \(y\) denotes the label value. In the second training stage, the audio feature extraction fusion module was constructed and integrated into the AG-STMNet architecture alongside the space-time memory network module. To ensure training stability, the parameters of the STMNet memory encoder and query encoder were frozen. The Ref-AVSbench dataset was used for training, with binary cross-entropy loss as the model's loss function. The space-time memory network was configured to store three historical frames, and the threshold value for the audio feature extraction fusion module was set to 0.5. Additionally, the label mask of the first frame in the dataset was used as initial input. (2) Testing and Evaluation  
(1) Audio Feature Extraction and Fusion Module Test: The Ref-AVSbench test dataset was imported into the trained network. Consistent with the training setup, the space-time memory network stored three historical frames, the audio feature extraction and fusion module threshold was set to 0.5, and the label mask from the first frame was used as the model input. The network's performance was evaluated based on the mIoU and F-score of the model output. (2) Audio Feature Extraction and Fusion Module Ablation Experiment: The audio module was removed from AG-STMNet, and only the image and mask data from the Ref-AVSbench test dataset were used for evaluation. The testing procedure remained identical to the previous setup, and the impact of the audio module on the model's performance was analyzed using the output metrics. (3) Complete Model Test: The mask generator was integrated with AG-STMNet to form a complete model. The Ref-AVSbench test dataset was imported to evaluate the complete model's performance. During testing, the output of the audio-visual fusion mask generator served as the input to AG-STMNet, while all other settings remained unchanged. The experimental results are presented in the table below:  
Based on the data presented in Table 4, the following interpretations can be drawn regarding the model performance:  
(a) STMNet without the Audio Module: The model achieves an mIoU score exceeding 70%, which substantiates the effectiveness of the trained network architecture. This baseline configuration demonstrates robust capability in generating accurate segmentation masks for the current frame by leveraging historical information and image queries.