# Video Captioning Based on Channel Soft Attention and Semantic Reconstruct
## 4 Experimental Evaluation
### Results and Analysis
#### 4.4.2 Results on MSVD Dataset  
Before comparing it with other methods, we did some comparative experiments to prove the effectiveness of our method, and report the results of the different combinations of our method in Table 3. We first evaluate the original SeqVLAD method and its combination with channel soft attention, semantic LSTM decoder, and semantic reconstructor, where \"SVLAD\" denotes the SeqVLAD, \"CSA\" denotes the channel soft attention, \"SLSTM\" denotes the semantic LSTM which semantic information has been added, and \"SR\" denotes the semantic LSTM with reconstructor. The cluster center is set to 32 because of the good overall performance shown in paper [6]. As Table 3 shows, a single channel soft attention (SVLAD\({}_{CSA}\)) can make a significant improvement in METEOR with 3.85%. The single semantic LSTM (SVLAD\({}_{SLSTM}\)) and semantic reconstructor (SVLAD\({}_{SR}\)) gain over the original SeqVLAD 2.34% and 2.96%, respectively. The channel soft attention improved \begin{table}
\begin{tabular}{c c c} \hline \hline
**Module** & **Parameters** & **MACs** \\ \hline CA & \(2.05\times 10^{5}\) & \(4.93\times 10^{5}\) \\ SeqVLAD & \(1.08\times 10^{6}\) & \(1.97\times 10^{7}\) \\ SDN & \(1.39\times 10^{7}\) & \(3.48\times 10^{7}\) \\ Decoder & \(2.72\times 10^{7}\) & \(5.81\times 10^{8}\) \\ Reconstructor & \(9.77\times 10^{5}\) & \(1.47\times 10^{5}\) \\ CSA-SR & \(4.43\times 10^{7}\) & \(7.83\times 10^{8}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters and multiply–accumulate operations (MACs) of modules. Figure 4: BLEU-4 effects of parameter \(\lambda\) in semantic reconstructor. CSA-SR denotes the complete model based on channel soft attention and semantic reconstructor, and SR is a simplified CSA-SR which without channel soft attention. model has a higher score in CIDEr than others. It shows that channel soft attention makes more improvement. However, semantic reconstructor got a higher score in BLEU-4. This is because the semantic reconstructor has fully used the semantic tags. These tags indicate the most common words in the dataset. In generated captions, the more matched words, the higher the BLEU-4 score. The channel soft attention combined with semantic LSTM decoder (\(\text{SVLAD}_{CSA-SLSTM}\)) does not have much improvement compared to \(\text{SVLAD}_{CSA}\). Channel soft attention with semantic reconstructor (\(\text{SVLAD}_{CSA-SR}\)) has shown the best overall performance. Then our method is compared with some state-of-the-art methods. The main ideas of the methods in Table 4 are introduced as follow:  
1. S2 VT [20] is an encoder-decoder framework with two layers of LSTM, where the first layer encodes the video representation and the second generates video descriptions. 2. SA [21] which is based on temporal attention focuses on both the local and global temporal structure of the video, and select the most relevant temporal segments automatically. 3. h-RNN [44] introduces a hierarchical structure for sentence generator and paragraph generator. Both sentence generator and paragraph generator are made up of the recurrent neural network. 4. HRNE [45] is a hierarchical recurrent neural encoder with a soft attention mechanism and explores the temporal transitions between frame chunks with different granularities. 5. SCN-LSTM [11] detects semantic tags before training caption models, and uses semantic compositional networks to embed semantic information for caption generating. 6. PickNet [46] selects the useful part of video frames for the encoder-decoder framework. It aims to reduce the computation and the noise caused by redundant frames. 7. TDDF [47] uses a task-driven dynamic fusion mechanism to reduce the ambiguity in the video description and refines the degree of the description of the video content. 8. RecNet [24] is a dual learning model that has forward flow and backward flow while training a model. The video representation will be reconstructed using the output of the decoder. 9.