# 3D Attention Based YOLO-SWINF for Real-Time Video Object Detection

Pradeep Moturi, Mukund Khanna, Kunal Singh

► To cite this version:

Pradeep Moturi, Mukund Khanna, Kunal Singh. 3D Attention Based YOLO-SWINF for Real-Time Video Object Detection. 19th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2023, León, Spain. pp.491-502, 10.1007/978-3-031-34111-3_41. hal-05449820

HAL Id: hal-05449820

https://inria.hal.science/hal-05449820v1

Submitted on 9 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
3D Attention Based YOLO-SWINF
for Real-Time Video Object Detection

Pradeep Moturi ID, Mukund Khanna ID, and Kunal Singh(✉) ID

AI@Scale, Fractal Analytics, Mumbai, India
{pradeep.moturi,mukund.khanna,kunal.singh}@fractal.ai
https://fractal.ai/

**Abstract.** Video object detection has a lot of applications that require detections in real-time, but these applications are unable to leverage the high accuracy of current SOTA video object detection models due to their high computational requirements. A popular approach to overcome this limitation is to reduce the frame sampling rate, but this comes at the cost of losing important temporal information from these frames. Thus, the most widely used object detection models for real-time applications are image-based single-stage models. Therefore, there is a need for a model that can capture the temporal information from the other frames in a video to boost detection results while still staying real-time. To this end, we propose a YOLOX based video object detection model YOLO-SWINF. Particularly, we introduce a 3D-attention based module that uses a three-dimensional window to capture information across the temporal dimension. We integrate this module with the YOLOX backbone to take advantage of the single-stage nature of YOLOX. We extensively test this module on the ImageNet-VID dataset and show that it has an improvement of 3 AP points over the baseline with just less than 1 ms increase in inference time. Our model is comparable to current real-time SOTA models in accuracy while being the fastest. Our YOLO-SWINF-X model achieves 80.4% AP at 38FPS on NVIDIA 1080Ti GPU.

**Keywords:** Video object detection · 3D Attention module · Real-time · YOLOX

# 1 Introduction

One of the main focuses of deep learning and computer vision research is object detection in videos. Most real-world applications of object detection like people tracking, self-driving cars, surveillance etc., require detection models to be run on a set of input frames in the form of videos. These videos often suffer from a degradation in quality due to various reasons like motion blur, camera defocus, and occlusion which significantly increases the difficulty of detection. Another constraint in these applications is that the detections are often needed in real-time. The current state-of-the-art deep learning models deal with these issues by
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** AP50 vs Inference time in milliseconds with the x axis being in log scale and our YOLO-SWINF models are tested on NVIDIA 1080 Ti GPU while the other models inference times are tested on NVIDIA 2080Ti GPU</figcaption></figure>

combining information from other frames, but these models are computationally
intensive meaning they are often not real-time. These problems become even
more apparent when these large-scale models need to be deployed on computa-
tionally limited platforms. This necessitates a trade-off between model complex-
ity and inference time by using smaller models which are faster but have worse
performance. So, this presents an intriguing application-based research challenge
of designing object detection models that can provide a better trade-off between
detection accuracy and inference time while simultaneously being able to gather
information from other frames of the video to solve the issue of video quality
degradation.

One simple way of getting detection results in real-time is to use single-
stage object detectors like the YOLO [2, 4, 7, 8, 14] series models on each frame
of the video. These models localise and categorise the objects in an image all
at once and are thus able to meet the real-time requirements but fall short of
the required standard for accuracy as they do not inculcate results from other
frames. Another approach is to use image based two-stage object detectors. These
detectors choose region proposals in the image and subsequently do classification
on these regions. Two-stage detectors [15] are more accurate than one-stage
detectors, but do not get results in real-time because of their two-stage nature.
Like single-stage detectors they also do not combine the temporal information
from other frames and are susceptible to the same problem. To use two-stage
detectors one approach is to skip the processing frames (in other words, by
reducing the sampling frequency for processing). Yet doing so also means passing
up on potentially crucial information in between frames. And to get detections
for the skipped frames, the detected boxes need to be interpolated which further
degrades their performance.
As a result, using simple single frame object detectors is not ideal. Hence, a model is needed that can incorporate temporal information, offer greater accuracy than one-stage detectors, and be faster than two-stage detectors. The current state-of-the-art methods for video object detection (VOD), such as MEGA [3], and SELSA [23], which use the temporal information between the frames to identify objects are extremely slow and are not real-time even on most GPUs. They use two-stage detectors as backbone to identify the region proposals in the target frame and the neighbouring frames, and then use a feature aggregation module to combine the features across the temporal dimension. The complexity of the frame aggregation method and the fact that existing VOD detectors are two-stage are the key drawbacks of present VOD technologies, which result in slow inference speed. They can solve the problem of frame aggregation, but it comes at the cost of speed.

**Contribution:** In this research, we present a 3D attention based temporal fusion block that was incorporated at the end of the backbone of YOLOX [4] models to create light-weight VOD models that can exploit temporal information effectively and under real-time restrictions. Because of its anchor-free, single-stage design, the YOLOX [4] model offers a fair trade-off between detection accuracy and inference speed, which is why we chose it to evaluate our module. As can be observed from Fig. 1. We achieve an AP50 of 80.4% while running at 38 FPS on NVIDIA 1080Ti GPU. Furthermore, our medium model is comparable to baseline YOLOX-X model in performance and can run at over 4× the FPS.

## 2 Related Work

This section will explore the current approaches in object detection which are related to this work. Video object detection is an extension of image-based detectors to a continuous stream of images; therefore, we also explore single frame object detectors. Single frame object detectors can be broadly classified into two categories- Two stage and One stage detector. As the name suggests, two-stage detectors first identify the key regions in an image which may have an object using a RPN and then extract the features from that region using techniques like ROIpool and ROIalign. The final detections are obtained using a classification and regression head which uses these region features as an input. Some of the well-known two-stage detectors in academia are RCNN [16], Faster RCNN [15] and Mask RCNN [9]. Although these models have good performance in object detection tasks, they fail in real-world scenarios when the frames suffer from degradation which reduces their performance. Thus, these models, are not suited for video object detection. Whereas one stage object detector directly gives the class probability and bounding box coordinates from the single input image, without the intermediate step of extracting region proposals. There are many one-stage object detectors such as the YOLO [2, 4, 7, 14] series, SSD [22],
RetinaNet [20] and FCOS [26]. These detectors are faster than two-stage object detectors since they do the detection in one go. However, these models lack the accuracy provided by two-stage object detectors. Like two-stage detectors these models also heavily depend on the quality of the input frames and are unable to solve issues like motion blue or occlusion reducing their usefulness for video object detection tasks.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Architecture diagram of SwinF module where X represents the input image size, the training is done in two phases, during phase 1 the baseline YOLOX model is trained i.e., the backbone, the neck and the head</figcaption></figure>

To overcome this problem of degradation of video frames, temporal data from other frames in the video can be used for more accurate identification of keyframe contamination. As the name implies, VOD approaches use data from nearby frames to identify objects. To extract and aggregate this data to get cues from various frames, there are numerous methods available.

The detectors like DFF [29], FGFA [28], THP [27], and MANet [21] integrate the motion information in the video by explicitly calculating the optical flow of the images and aggregating the features accordingly. One major drawback with this class of detectors is that training a model for flow extraction is expensive and they are inherently slow because the model needs to compute the optical flow of each image to get detections. The accuracy of the detections also depends on the accuracy of the optical flow estimations.

Techniques like REPP [18], Seq-NMS [6], Seq-Box [1] matching are box level post-processing methods that link objects across frames by feature similarity and enhance detections scores for the frames. Other techniques like, TrackNet [10] and Global Correlation Network [11], simultaneously detect a 3D tube enclosing a moving object in a video segment by extending the faster R-CNN framework [15]. Thus, they do detection and tracking invadjust an end-to-end manner.
Most of the existing detectors in this class are based on two-stage detectors which makes them unsuitable for real-time applications or they are post-processing techniques which are used after the detections are already available. RNN-based detectors consist of methods like Mobile-VOD [12] which combine fast single image object detection with LSTM layers to create an inter-weaved recurrent convolutional neural network. ROLO [5] uses a convolutional network as a backbone to extract bounding boxes from each frame and feed these features into an LSTM network. These methods apply LSTMs as post-processing on top of network outputs which effectively makes them two-stage and reduced the inference speed. Lastly, Attention based models utilize attention mechanism to capture temporal dependencies. The models like MEGA [3]- and SELSA [23] focusing on carrying temporal information in form of attention on region proposals. SELSA proposed a long-range feature aggregation approach based on semantic similarity between the region level features. MEGA proposes a long-range memory module which enables the key frame to get access to the information from global and local frames. Although these approaches give a considerable boost to detection accuracy, they are based on two stage detectors and suffer from slow inference speed.

# 3 Proposed Method

As previously discussed, the major drawback of existing video object detection methods is that they are not real-time due to their two-stage nature while the single-stage detectors lack the detection accuracy. To overcome this limitation, we propose YOLO-SWINF to collate spatial and temporal information from neighbouring frames in a video. We use YOLOX [4] models as our baseline, because they are single stage detectors. A YOLOX model consists of a CSP-Darknet backbone, a PAFPN neck and a decoupled head. The PAFPN neck takes as input three feature maps from the backbone at three different levels for feature aggregation and outputs 3 feature maps to the decoupled head. The head then processes these outputs to get detections. Followed by this, object confidence thresholding and Non-Max Suppression (NMS) are applied to obtain final object bounding boxes. We place our Swin [25] based SwinF module at the end of the backbone and just before the PAFPN neck. The overall architecture is depicted in Fig. 2. The input to the YOLOX model is a batch of feature maps with B sets where each set consists of N neighbouring frames randomly sampled from a random video. Thus, the input to the model is of shape (B, N, 3, 576, 576). This tensor is flattened along the first two dimensions to get the input of the backbone as shape (B*N, 3, 576, 576), effectively making the batch size as B*N. This tensor is passed through the backbone to get an output of shape (B*N, C, H, W) where C is the number of channels and HxW is the feature map size. The input to the attention module requires each batch to have all the neighbouring frames, thus the input is split back into shape (B, N, C, H, W).
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** End to End design of YOLO-SWINF, we place our block at the end of ‘Dark5’ output of the CSPDarknet platform; our module YOLO-SWINF stands for YOLO SWIN Fusion</figcaption></figure>

The attention module is a SWIN block consisting of a window attention layer with a 3D window of shape (N, S, S) followed by an MLP layer which brings the total number of windows to (H/S * W/S * 1). Each layer is preceded by a layer normalization and followed by a dropout layer. We found that setting the dropout probability to zero gave better results, so dropout is disabled. The input is permuted to channel first dimension because we want the 3D window to traverse along the temporal dimension. This ensure that each window is looking at all the frames when doing attention. The window size in the spatial dimension, S is set to 6 to get a fair-trade off between computational complexity and accuracy. After the attention enhanced feature maps are calculated for all the frames in a batch, a residual connection is used to add the input feature maps to the attention enhanced feature maps. The hypothesis is that in cases when there is too much object movement then the attention feature maps might end up focusing on the wrong object. Thus, we use a residual connection to ensure that some of the previous context is also carried forward.

The output feature maps are again of shape (B, N, C, H, W). But the PAFPN neck requires the input to be 4 dimensional of shape (batch size, feature map size). Thus, the output is again flattened across the first two dimension to get the input to the PAFPN neck as (B*N, C, H, W). These enhanced feature maps are normally passed through the neck and decoupled head, and the detections are obtained just like in a normal YOLOX model. The complete architecture is explained in Fig. 3.
# 4 Experiments

We initialize our baseline YOLOX models with COCO [19] pre-trained weights provided by YOLOX. We mix the images in the ImageNet-VID [17] dataset with those in the ImageNet-DET [17] dataset belonging to the same class in the training data. We use all the images in the ImageNet-DET dataset and 1/10th of the images in the ImageNet-VID dataset to train the baseline model. The ImageNet-VID dataset consists of 3,862 videos for training and 555 videos for validation. There are 30 classes in the ImageNet-VID dataset, i.e., a subset of the 200 categories of ImageNet-DET. All the base models are trained for 7 epochs with the first epoch being used for warmup as we found that for a small dataset like Imagenet 7 epochs were enough for the model to reach convergenc. SGD optimizer is used with a batch size of 16 on a single A4000 GPU. A cosine learning rate-scheduler is used to alter the learning rate. Mosaic [2] and Mixup [2] augmentations are used for the first 5 epochs with the last two epochs being done without any strong data augmentation. An image size of 576 × 576 is used with multi-scale training enabled.

<table><caption>**Table 1.** Comparison with baseline YOLOX model results</caption><thead><tr><th>Model</th><th>AP50</th><th>Inference time (ms)</th></tr></thead><tbody><tr><td>YOLOX-S</td><td>69.5</td><td>4.56</td></tr><tr><td>YOLOX-M</td><td>74.1</td><td>8.93</td></tr><tr><td>YOLOX-L</td><td>76.1</td><td>15.04</td></tr><tr><td>YOLOX-X</td><td>77.6</td><td>24.69</td></tr><tr><td><strong>YOLO-SWINF-S</strong></td><td><strong>73.2</strong></td><td><strong>5.14</strong></td></tr><tr><td><strong>YOLO-SWINF-M</strong></td><td><strong>77.1</strong></td><td><strong>9.65</strong></td></tr><tr><td><strong>YOLO-SWINF-L</strong></td><td><strong>79.3</strong></td><td><strong>16.00</strong></td></tr><tr><td><strong>YOLO-SWINF-X</strong></td><td><strong>80.4</strong></td><td><strong>25.81</strong></td></tr></tbody></table>

We add the SwinF module at the end of the backbone and before the PAFPN neck. For training this model we freeze the entire base model and stop gradient computation for the backbone to save memory and training time. Only the attention module is trained using SGD for 7 epochs with the first epoch being used for warmup. Again we found that the model converges before the 7th epoch in each case. No strong data augmentation like Mosaic or Mixup is used and multi-scale training is also disabled because all strong data augmentation was found to negatively affect the accuracy. All perspective and affine transforms are also uniform across a single batch of images to ensure that each window is looking at the same region in an image. To train the SwinF module we found that using just 1/3rd of the ImageNet-VID dataset was enough for the model to converge.
We use an image size of 576 × 576 and a batch size of 4 with each batch consisting of 16 frames randomly sampled from a video. Thus, each batch is selected from one video. To evaluate the performance of the models, we use the standard metrics for object detection on ImageNet-VID validation set. Specifically, we report the AP50 (AP calculated at IoU threshold of 0.5) and inference time of our models to show their efficiency. All models are validated on 1080Ti GPU with FP-16 precision unless otherwise mentioned. A confidence threshold of 0.001 is used with an NMS threshold of 0.5. The results with comparison with baseline are present in Table 1 and comparison with other methods is present in Table 2. Overall our experiments show that our module outperforms the baseline consistently by 3 AP points across small to extra large versions. YOLO-SWINF method is able to provide the best inference efficiency when compared to other SOTA models. Our YOLO-SWINF-S model is also much faster than baseline YOLOX-M model while having similar performance. Similarly our YOLO-SWINF-M model is comparable to YOLOX-X model in terms of accuracy while simultaneously being faster.

<table><caption>**Table 2.** Comparison with other SOTA methods: A-the inference time was taken from corresponding papers, B-inference time was tested on 2080Ti, our YOLO-SWINF models were tested on 1080 Ti GPU, while models with * were tested in online mode whereas ‘-’ denotes the model is not released and the speed is not reported in original paper. Thus we use a computationally slower GPU to test our inference times compared to other SOTA models</caption><thead><tr><th>Model</th><th>Backbone</th><th>AP50</th><th>Inference time (ms)</th></tr></thead><tbody><tr><td>FGFA</td><td>R101</td><td>76.3</td><td>104.2(B)</td></tr><tr><td>SELSA</td><td>X101</td><td>83.1</td><td>153.8(B)</td></tr><tr><td>RDN</td><td>R101</td><td>81.8</td><td>162.6(B)</td></tr><tr><td>RDN</td><td>X101</td><td>83.2</td><td>-</td></tr><tr><td>MEGA</td><td>R101</td><td>82.9</td><td>230.4(B)</td></tr><tr><td>MEGA</td><td>X101</td><td>84.1</td><td>-</td></tr><tr><td>TROIA</td><td>X101</td><td>84.3</td><td>285.7(B)</td></tr><tr><td>MAMBA</td><td>R101</td><td>84.6</td><td>110.3(A)</td></tr><tr><td>HVR</td><td>X101</td><td>84.8</td><td>-</td></tr><tr><td>TransVOD</td><td>R101</td><td>81.9</td><td>-</td></tr><tr><td>QueryProp</td><td>R50</td><td>80.3</td><td>21.9(A)</td></tr><tr><td>QueryProp</td><td>R101</td><td>82.3</td><td>30.8(A)</td></tr><tr><td>FastVOD-NET*</td><td>R101</td><td>79.3</td><td>34.13 (A)</td></tr><tr><td>FastVOD-NET*</td><td>R101</td><td>81.2</td><td>43.47 (A)</td></tr><tr><td>MultiFrame CenterNET*</td><td>R101</td><td>78.8</td><td>27.8 (A)</td></tr><tr><td>YOLO-SWINF-S</td><td>YOLOX</td><td>73.2</td><td>5.14</td></tr><tr><td>YOLO-SWINF-M</td><td>YOLOX</td><td>77.1</td><td>9.65</td></tr><tr><td>YOLO-SWINF-L</td><td>YOLOX</td><td>79.3</td><td>16.00</td></tr><tr><td>YOLO-SWINF-X</td><td>YOLOX</td><td>80.4</td><td>25.81</td></tr><tr><td>YOLO-SWINF-X*</td><td>YOLOX</td><td>79</td><td>25.81</td></tr></tbody></table>
# 5 Ablation Studies

## 5.1 Frame Sampling Strategy

Frame sampling strategy is important for video object detectors to get a good balance between accuracy and inference time. Two of the most widely used frame sampling strategies are global and local sampling schemes. We also run experiments using these two techniques. In the continuous frame, sampling scheme Fc continuous frames are selected from a video with a fixed frame gap D. In the random sampling scheme, Fg frames are randomly sampled from each video. We conduct experiments with varied frame counts Fg (=2,5,10,16,32) to understand the impact of global sampling. To understand the effect of continuous sampling the frame gap D (=1,3,5,7,10,15) is changed while maintaining a constant frame count Fc (=16) We perform validation experiments with the above-mentioned schemes using the YOLO-SWINF-S model, which was trained using 16 randomly selected frames. The results are shown in Table 3 and Table 4. As can be observed, the performance just using 5 global frames is better than the performance when using 16 local frames irrespective of frame gap. The best results are obtained with 16 randomly sampled frames after which the performance starts to plateau.

<table><caption>**Table 3.** Number of random frames study: YOLO-SWINF-S model is validated using different number of randomly sampled frames</caption><thead><tr><td>Number of Frames</td><td>AP50</td></tr></thead><tbody><tr><td>2</td><td>70.4</td></tr><tr><td>5</td><td>72.3</td></tr><tr><td>10</td><td>72.9</td></tr><tr><td>16</td><td>73.2</td></tr><tr><td>32</td><td>73.1</td></tr></tbody></table>

<table><caption>**Table 4.** Online setting validation study where YOLO-SWINF-S model is validated using 16 continuous frames with different sampling rates</caption><thead><tr><td>Frame gap</td><td>AP50</td></tr></thead><tbody><tr><td>1</td><td>69.7</td></tr><tr><td>3</td><td>70.6</td></tr><tr><td>5</td><td>71.6</td></tr><tr><td>7</td><td>71.2</td></tr><tr><td>10</td><td>70.9</td></tr><tr><td>15</td><td>69.4</td></tr></tbody></table>
## 5.2 Head Dimension

Just like vanilla attention, SwinF also uses multiple heads to run attention in parallel as it allows for attending to parts of the sequence differently (e.g., longer-term dependencies versus shorter-term dependencies) which leads to better feature representation. The processing power of each head depends on the number of channels used to represent each embedding vector. Thus, to decide the best value for the head dimension we run ablation studies with different values of the head dimension. We train YOLO-SWINF-S with three different values of the head dimension. The results are showed in Table 5. As can be observed the best value for hidden dimension is 64 and then the AP50 decreases on further increasing the hidden dimension.

<table><caption>**Table 5.** Ablation study on head dimension where YOLO-SwinF-S model was trained using different head dimensions</caption><thead><tr><td>Head dimension</td><td>AP50</td></tr></thead><tbody><tr><td>16</td><td>73.1</td></tr><tr><td>64</td><td>73.2</td></tr><tr><td>128</td><td>72.8</td></tr></tbody></table>

## 6 Conclusion

In this paper, we present YOLOX based YOLO-SWINF models for fast and accurate VOD. A SwinF based temporal attention module was introduced to collate temporal information from neighbouring frames in a video and boost detection results for each frame. We position our module directly in front of the YOLOX backbone to ensure optimal detection accuracy. Our module uses a multi-headed attention operation with a sliding 3D window attention as described in SWIN Vision Transformer [25] to combine data from many frames, speeding up inference while also increasing accuracy. We demonstrated that, with only marginal increase in latency, YOLO-SWINF outperforms the baseline YOLOX models. To evaluate the efficacy of our model, we ran multiple experiments and ablation studies. Our YOLO-SWINF-X model achieves a high detection accuracy of 80.4% AP with an inference time of 25.81 ms on NVIDIA 1080Ti GPU while our YOLO-SWINF-S and M models could be the better choice for platforms with limited computation power.

## References

1. Belhassen, H., Zhang, H., Fresse, V., Bourennane, E.: Improving video object detection by seq-bbox matching. VISIGRAPP (2019). https://doi.org/10.1109/ISCC53001.2021.9631435
2. Bochkovskiy, A., Wang, C., Liao H.: YOLOv4: optimal speed and accuracy of object detection. ArXiv (2020). arXiv:2004.10934

3. Chen, Y., Cao, Y., Hu, H., Wang, L.: Memory enhanced global-local aggregation for video object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10334–10343 (2020). https://doi.org/10.1109/CVPR42600.2020.01035

4. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: YOLOx: exceeding yolo series in 2021. ArXiv (2021). arXiv:2107.08430

5. Guanghan, N., Zhi, Z., Chen, H., Zhihai, H., Xiaobo, R., Haohong, W.: Spatially supervised recurrent convolutional neural networks for visual object tracking. CoRR (2016). https://doi.org/10.1109/ISCAS.2017.8050867

6. Han, W., et al.: Seq-nms for video object detection. ArXiv (2016). https://doi.org/10.48550/arXiv.1602.08465

7. Jocher, G.: YOLOv5 (2021). https://github.com/ultralytics/yolov5. https://doi.org/10.5281/zenodo.4154370

8. Joseph, R., Santosh, K.D., Ross, B.G., Ali, F.: You only look once: unified, real-time object detection. CoRR (2015). https://doi.org/10.1109/CVPR.2016.91

9. Kaiming, H., Georgia, G., Piotr, D., Ross, B.G.: Mask R-CNN. CoRR (2017). https://doi.org/10.1109/ICCV.2017.322

10. Li, C., Dobler, G., Feng, X., Wang, Y.: Tracknet: simultaneous object detection and tracking and its application in traffic video analysis. ArXiv (2019) https://doi.org/10.48550/arXiv.1902.01466

11. Lin, X., Guo, Y., Wang, J.: Global correlation network: end-to-end joint multi-object detection and tracking. ArXiv (2021). https://doi.org/10.21203/rs.3.rs-1107274/v1

12. Liu, M., Zhu, M.: Mobile video object detection with temporally-aware feature maps. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5686–5695 (2018). https://doi.org/10.1109/CVPR.2018.00596

13. Qi, Q., Wang, X., Hou, T., Yan, Y., Wang, H.: FastVOD-Net: a real-time and high-accuracy video object detector. IEEE Trans. Intell. Transp. Syst. 23, 20926–20942 (2022). https://doi.org/10.1109/TITS.2022.3176721

14. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. ArXiv (2018). https://doi.org/10.1109/ACCESS.2021.3103522

15. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2015). https://doi.org/10.1109/TPAMI.2016.2577031

16. Ross, B.G., Jeff, D., Trevor, D., Jitendra, M.: Rich feature hierarchies for accurate object detection and semantic segmentation. CoRR (2013). https://doi.org/10.1109/CVPR.2014.81

17. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115, 211–252 (2014). https://doi.org/10.1007/s11263-015-0816-y

18. Sabater, A., Montesano, L., Murillo, A.: Robust and efficient post-processing for video object detection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10536–10542 (2020). https://doi.org/10.1109/IROS45743.2020.9341600

19. Yi, L.T., et al.: Microsoft COCO: common objects in context. CoRR (2014). https://doi.org/10.1007/978-3-319-10602-1_48

20. TsungYi, L., Priya, G., Ross, B.G., Kaiming, H., Piotr, D.: Focal loss for dense object detection. CoRR (2017). https://doi.org/10.1109/iccv.2017.324
21. Wang, S., Zhou, Y., Yan, J., Deng, Z.: Fully motion-aware network for video object detection. In: Proceedings of the European conference on computer vision (ECCV), pp. 542–557 (2018). https://doi.org/10.1007/978-3-030-01261-8_33

22. Wei, L., et al.: Single shot multibox detector. CoRR (2016). https://doi.org/10.1109/ICMEW.2017.8026312

23. Wu, H., Chen, Y., Wang, N., Zhang, Z.: Sequence level semantics aggregation for video object detection. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9216–9224 (2019) https://doi.org/10.1109/iccv.2019.00931

24. Zhou, Y., Bai, Y., Chen, Y.: Multiframe centernet heatmap ROI aggregation for real-time video object detection. IEEE Access 10, 54870–54877 (2022). https://doi.org/10.1109/ACCESS.2022.3174195

25. Ze, L., et al.: Video swin transformer. CoRR (2021). https://doi.org/10.1109/cvpr52688.2022.00320

26. Zhi, T., Chunhua, S., Hao, C., Tong, H.: Fully convolutional one-stage object detection. CoRR (2016). https://doi.org/10.1109/iccv.2019.00972

27. Zhu, X., Dai, J., Zhu, X., Wei, Y., Yuan, L.: Towards high performance video object detection for mobiles. ArXiv (2018). https://doi.org/10.1109/cvpr.2018.00753

28. Zhu, X., Wang, Y., Dai, J., Wei, Y.: Flow-guided feature aggregation for video object detection. In: IEEE International Conference on Computer Vision (ICVV), pp. 408–417 (2017). https://doi.org/10.1109/iccv.2017.52

29. Zhu, X., Xiong, Y., Dai, J., Yuan, L., Wei, Y.: Deep feature flow for video recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4141–4150 (2017). https://doi.org/10.1109/cvpr.2017.441