# A Review of Video Object Detection: Datasets, Metrics and Methods
## 3 Video Object Detection Methods
### Tracking-Based  
Inspired by the fact that tracking is an efficient way to utilize the temporal information, several methods [78; 79; 81] have been developed to detect objects on fixed interval frames and track them in frames in between. The improved methods in [26] and [80] detect interval frames adaptively and track the other frames.  
A framework named CDT was presented in [79], combining detection and tracking for video object detection. This framework consisted of an object detector, a forward tracker and a backward tracker. Initially, objects were detected by the image object detector. Then, each detected object was tracked by the forward tracker, and undetected objects were stored by the backward tracker. In the entire process, the object detector and the tracker cooperated with each other to deal with the appearance and disappearance of objects.  
Another framework named CaTDet with a high computational efficiency was presented in [78]. This framework is shown in Figure 8, which includes a tracker and a detector. CaTDet uses a tracker to predict the position of objects with a high confidence in a next frame. The processing steps of CaTDet  
Figure 7: PSLA (progressive parse local attention) framework [76].  
are: (i) Every frame is inputted to a proposal network to output potential proposals in the frame. (ii) Object position in a next frame is predicted with a high confidence using the tracker. (iii) In order to obtain the calibrated object information, the outputs of the tracker and the proposal network are combined and inputted to a refinement network.  
More specifically, based on the observation that objects detected in one video frame would most likely appear in a next frame, a tracker was used to predict the positions on the next frame with the historical information. In case new objects appeared in a current frame, a computationally efficient proposal network similar to RPN was utilized to detect proposals. In addition, to address situations such as motion blur and occlusion, the temporal information was used by a tracker to predict future positions. The results obtained by combining the tracker and the proposal network was then refined by a refinement network. Only the regions of interest were refined by the refinement network to save computation time while maintaining accuracy.  
Similar to CDT and CaTDet, recent approaches for the detection and tracking of objects in video involve rather complex multistage components. In [81], a framework using a ConvNet architecture was deployed in a simple but effective way by performing tracking and detection simultaneously. More specifically, first R-FCN [19] was employed to extract the feature maps shared between detection and tracking. Then, proposals in each frame were obtained by using RPN based on anchors [15]. RoI pooling [15] was utilized for the final detection. In particular, a regressor was introduced to extend the architecture. Position-sensitive regression maps from both frames were used together with correlation maps as the input to an RoI tracking module, in which the box relationship between the two frames was outputted. For video object detection, the framework in [81] was evaluated on the ImageNet VID dataset achieving an accuracy of 82.0% mAP.  
Similarly, inspired by the observation that object tracking is more efficient than object detection, a framework (D or T) was covered in [80], see Figure 9, which includes a scheduler network to determine the operation (detecting or tracking) on a certain frame. Compared with the baseline frame skipping (detecting on fixed interval frames and tracking on intermediate frames), the scheduler network with light weights and a simple structure was found to be more effective on the ImageNet VID dataset. Moreover, the adaptive mechanism in [26] (TRACKING ASSISTED) was used to select key frames. Detection on key frames involved the utilization of an accurate detection network and detection on non-key frames was assisted by the tracking module.  
Figure 8: CaTDet framework [78].