# A Review of Video Object Detection: Datasets, Metrics and Methods
## 3 Video Object Detection Methods
### Attention-Related  
For video object detection, it is known that exploiting the temporal context relationship is quite important. This relationship needs to be established based on a long-duration video, which requires a large amount of memory and computational resources. In order to decrease the computational resources, an attention mechanism was introduced for feature map alignment. This mechanism was first proposed for machine translation in [118; 119] and was then applied to video object detection in [25; 74; 75; 76; 77]. Figure 6: Temporal Conv LSTM architecture. State S-1 and Output H-1 are retrieved from the memory. Forget Gate, Input Gate and Output Gate operate the \(3\times 3\) convolutions followed by the activation function. Some methods only take the global or local temporal information into consideration. Specifically, the method RDN in [75] only makes use of the local temporal information. The methods SELSA in [77], and OGEMN in [74] only utilize the global temporal information. While the other methods of PSLA in [76], and MEGA in [25] use both the global and local temporal information. Relation Distillation Networks (RDN) presented in [75] propagate and aggregate the feature maps based on object relationships in video. In RDN, ResNet-101 [112] and ResNeXt-101-64 x 4d [120] are utilized as the backbone to extract feature maps and object proposals are generated with the help of a Region Proposal Network (RPN) [15]. The feature maps of each proposal on the reference frame are augmented on the basis of supportive proposals. A prominent innovation in this work is to distill the relation with multi-stage reasoning consisting of a basic and an advanced stage. In the basic stage, the supportive proposals consisting of Top K proposals of a current frame and its adjacent frames are used to measure the relation feature of each reference proposal obtained on the current frame to generate refined reference proposals. In the advanced stage, supportive proposals with high objective scores are selected to generate advanced supportive proposals. Features of selected supportive proposals are aggregated with the relation against all supportive proposals. Then, such aggregated features are employed to strengthen the reference proposals obtained from the basic stage. Finally, the aggregated features of reference proposals obtained from the advanced stage are used to generate the final classification and bounding box regression. In addition, the detection box linking is used in a post-processing stage to refine the detection outcome. Evaluated on the ImageNet VID dataset, RDN achieved a detection accuracy of 81.8% and 83.2% mAP, respectively, with ResNet-101 and ResNeXt-101 for feature extraction. With linking and rescoring operations, it achieved an accuracy of 83.8% and 84.7% mAP, respectively. A module (SELSA) was introduced in [77] to exploit the relationship between the proposals in the entire sequence level, and then related feature maps were fused for classification and regression. More specifically, the features of the proposals were extracted on different frames and then a clustering module and a transformation module were applied. The similarities of the proposals were computed across frames and the features were aggregated according to the similarities. Consequently, more robust features were generated for the final detection. In [74], OGEMN was presented and used object-guided external memory to store the pixel and instance level features for further global aggregation. In order to improve the storage-efficiency aspect, only the features within the bounding boxes were stored for further feature aggregation. In [25], MEGA was introduced, utilizing the global and local information inspired by how humans go about object detection in video using both global semantic information and local localization information.