Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
Authors: Chengcheng Wang
Wei He
Ying Nie
Jianyuan Guo
Chuanjian Liu
Yunhe Wang
Kai Han
Venue: NeurIPS 2023 poster (2023)
Keywords: YOLO
object detection
computer vision

Abstract: In the past years, YOLO-series models have emerged as the leading approaches in the area of real-time object detection. Many studies pushed up the baseline to a higher level by modifying the architecture, augmenting data and designing new losses. However, we find previous models still suffer from information fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) have alleviated this. Therefore, this study provides an advanced Gatherand-Distribute mechanism (GD) mechanism, which is realized with convolution and self-attention operations. This new designed model named as Gold-YOLO, which boosts the multi-scale feature fusion capabilities and achieves an ideal balance between latency and accuracy across all model scales. Additionally, we implement MAE-style pretraining in the YOLO-series for the first time, allowing YOLOseries models could be to benefit from unsupervised pretraining. Gold-YOLO-N attains an outstanding 39.9% AP on the COCO val2017 datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model YOLOv6-3.0-N with similar FPS by +2.4%. The PyTorch code is available at https://github.com/huawei-noah/Efficient-Computing/tree/master/Detection/Gold-YOLO, and the MindSpore code is available at https://gitee.com/mindspore/models/tree/master/research/cv/Gold_YOLO.

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
Chengcheng Wang
Wei He Ying Nie
Jianyuan Guo Chuanjian Liu Kai Han" Yunhe Wang"
Huawei Noah's 's Ark Lab
(vangchengcheng11 hewei142 ying. mie, jinnuan guo, sluchuanjian, kai han, yunhe. wang))huawee com
Abstract
In the past years, YOLO-series models have emerged the leading approaches in the area of real-time object detection. Many studies pushed up the baseline higher level by modifying the architecture, augmenting data and designing new losses, However, we find previous models still suffer from information fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) have alleviated this. Therefore, this study provides an advanced Gather- uid-Distribute mechanism (GD) mechanism, which is realized with convolution and self-attention operations. This new designed model named as Gold-YOLO, which boosts the multi-scale feature fusion capabilities and achieves an ideal balance between latency and accuracy across all model scales. Additionally, we implement the first time, allowing YOLO-series
MAB-style pretraining the YOLO-series models could be to benefit from unsupervised pretraining. outstanding 39.9% AP on the COCO val2017 datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model YOLOv6-3.04 with similar FPS by +2.4%. The Py Torch code is available at lttpss:/githhbb.. https://tthu..co//uuweeeeee www/huawei-noaa//ffiieeee
Bold-YOLO-N attains
Commpttin/ueemmaee////eecctooooo able at wttp://ww....c.oooooooooccccccea
and the MindSpore code is avail-
Introduction
Object detection fundamental vision task that aims recognize the categories and locate the positions of objects. can be widely used a wide range of applications, such as intelligent security, autonomous driving, robot navigation, and medical diagnosis. High-performance and low-latency object detector is receiving increasing attention for deployment on the edge devices.
Over the past few years, researchers have extensive research on CNN-based detection networks, gradually evolving the object detection framework from two-stage (e.g., Faster RCNN 43 and
Mask RCNN 25 to one-stage (e.g., YOLO 44)), and from anchor-based (e.g.. YOLOv3
42
and YOLOv4 22 to anchor-free (e.g., CenterNet (10, FCOS 44 and YOLOX (T)
12
studied the optimal network structure through NAS for object detection task, and [16] 23 19 explore
another way to improve the performance the model by distillation. Single-stage detection models, especially YOLO series models, have been widely welcomed in the industry due to their simple structure and balance between speed and accuracy.
Improvement of backbone is also an important research direction in the field of vision. As described in the survey 20 26 27 60 21 62] has achieved balance between precision and speed, while 9 36 22 18 has shown strong performance in precision. These backbones have improved the performance of the original model different visual tasks, ranging from high-level tasks like object
"Corresponding Author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

245-
10.01
paid-0i0-
2rld TOLP
posd VOLOiS
CDCO AR0%)
COCO APP%)
GOBIYCLOM
2005 VOLOUN
Gead VOLD YOLOUS
Gild YOLD
35
35
YOLOyS
rOLDubia 10
VOLDu6-30
YO:0u4
30
NPYDICE
30
PYYOLO
YOLOK
VOLOX
200
1000
200
600
1000
1200
1400
Throughput(FP5) on T4 (bs=32. TensorRT8)
ThroughputiFPS) on T4 (bs=32, TensorRT88
(a) TensorRT 7, FP16 Throughput (FPS), BS=32
(b) TensorRT 8, FP16 Throughput (FPS), BS=32
Comparison state-of-the-ard efficient object detcctors in Tesla T4 GPU. Both lateney and throughput (batch size of 32) are given for handy reference. (a) and (b) test with TensorRT 7 and 8, respectively.
Figure I:
detection to low-level tasks like image restoration. By using the encoder-decoder structure with the transformer, researchers have constructed a series of DETR-like object detection models, such DETR 3 and DINO [57] These models can capture long-range dependency between objects, enabling transformer-basee detectors to achieve comparable or superior performance with most refined classical detectors. Despite the notable performance of transformer based detectors, they fall short when compared to the speed of CNN-based models. Small-scale object detection models based on CNN still dominate the speed-accuracs trade-off, such as YOLOX [T]] and YOLOv6-v3
33
49 [4].
We focus on the real-time object detection models, especially YOLO series for mobile deployment. Mainstream real-time object detectors consist of three parts: backbone, neck, and head. The backbone architecture has been widely investigated 42 36 and the head architecture typically straight forward, consisting of several convolutional or fully-connected layers. The necks in YOLO series usually use Feature Pyramid Network (FPN) and its variants to fuse multi-level features. These neck modules basically follow the architecture shown in Fig. 3 However, the current approach to information fusion has a notable flaw: when there is a need to integrate information across layers (e.g., level-1 and level-3 are fused), the conventional FPN-like structure fails to transmit information without loss, which hinders YOLOs from better information fusion.
Built upon the concept of global information fusion, TopFormer [59] has achieved remarkable results in semantic segmentation tasks. In this paper, we expanding on the foundation of TopFormer's theory. propose novel Bather-and-Distribuus mechanism (GD) for efficient information exchanging in YOLOs by globally fusing multi-level features and injecting the global information into higher levels. This significantly enhances the information fusion capability of the neck without significantly increasing latency, improving the model's performance across varying object sizes. Specifically, branch and deep gather- block
GD mechanism comprises two branches: shallow pather-and-distribee and-distribute branch, which extract and fuse feature information via a convolution- and attention-basee based block, respectively. To further facilitate information flow, introduce lightweight adjacent- fusion module which combines features from neighboring levels on a local scale. Our Gold-YOr YOLO architectures surpasses the existing YOLO series, effectively demonstrating the effectiveness of our proposed approach.
To further improve the accuracy of the model, we also introduce pre-training method, where we pre-train the backbone on ImageNet 1K using the MAE method, which significantly improves the convergence speed and accuracy of the model. For example, our Gold-YOn YOLO-S with pre-training achieves 46.4% AP, which outperforms the previous SOTA YOLOv6-3.0-5 with 45.0% AP at similar speed.
2
Related works
2.1 Real-time object detectors
After years of development, the YOLO-series model has become popular in the real-time object detection area. YOLOv1-v3 440 41 42 constructs the initial YOLOs, identifies a single-stage

---

85
P5
Injact
N5
B6
Inject
P6
Inject
Backbone
B3
Inject
P3
N3
Low-GD
High-GD
Low-FAM
Low-IFM
High-FAM
High-IFM
Neck
Figure 2: The architecture of the proposed Gold-YOLO.
detection structure consisting of three parts, backbone-neck-hhad, predicts objects of different sizes through multi-scale branches, become a representative single-stage object detection model. YOLOv4 [2] optimizes the previously used darknet backbone structure and propose series of improvements, like the Mish activation function, PANet and data augmentation methods. YOLOv5 [33 inheriting the YOLOv4 2 scheme with improved data augmentation strategy and greater variety of model variants. YOLOX incorporates Multi positives, Anchor-free, and Decoupled Head into the model structure, setting new paradigm for YOLO-modes design. YOLOv6 33 32
brings the reparameterization method YOLO-series models for the first time, proposing EfficientRep Backbone and Rep-PAN Neck. YOLOv7 49) focuses on analyzing the effect of gradient paths on the model performance and proposes the E-ELAN structure enhance the model capability without destroying the original gradient paths. The YOLOv8 [14] takes the strengths of previous YOLO models and integrates them to achieve the SOTA of the current YOLO family.
2.2 Transformer-bass object detection
Vision Transformer (ViT) emerged as a competitive alternative to convolutional neural networks (CNNs) that are widely used for different image recognition tasks. DETR applies the transformer structure to the object detection task, reconstructing the detection pipeline and eliminating many hand- designed parts and NMS components to simplify the model design and overall process. Combining the sparse sampling capability of deformable convolution with the global relationship modeling capability of transformer, Deformable DETR 63 improve convergence speed while improve model speed and accuracy. DINO [57] first time introduced Contrastive denoising, Mix query selection and a look forward twice scheme. The recent RT-DETR [37] improved the encoder-decoder structure to solve the slow DETR-liK- model problem, outperforming YOLO-L/X in both accuracy and speed. However, the limitations of the DETR-like structure prevent from showing sufficient dominance in the small model region, where YOLOs remain the SOTA of accuracy and velocity balance.
2.3 Multi-scale features for object detection
Traditionally, features at different levels carry positional information about objects of various sizes. Larger features encompass now-dimensional texture details and positions of smaller objects. In contrast, smaller features contain high-ding high-dimensional information and positions of larger objects. The original idea behind Feature Pyramid Networks (FPN) proposed by (35) is that these diverse pieces of information can enhance network performance through mutual assistance. FPN provides an efficient architectural design for fusing multi-scale features through cross-scale connections and information exchange, thereby boosting the detection accuracy of objects of varied sizes.
Based on FPN, the Path Aggregation Network (PANet) (50] incorporates a bottom-up path to make information fusion between different levels more repeatable module (BiFPN) to increase the efficiency of information fusion between different levels. M2Det [1] introduced an efficient MLFPNN architecture with U-shape and Feature Fusion Modules. Ping- Yang Chen (5) improved interaction between deep and shallow layers using bidirectional fusion modules. Unlike these inter-layer works, (38 explored individual feature information using the Centralized Feature Pyramid (CFP) method. Additionally, (54) extended FPN with the Asymptotic
EfficientDet 45 presents new
Feature Pyramid Network (AFPN) to interact across non-adjacent layers. response to FPN's
limitations in detecting large objects, 31] proposed refined FPN structure. YOLO-F (6) achieved
3

---

level-1
fuse
level-1
level-2
fuse
fuse
level-2
level-3
fuse
level-3
(a) traditional neck structure
(b) traditional neck
(c) our proposed neck
Figure 3: (a) is example diagram of traditional neck information fusion structure. (b) and (c) is
AblationCAM [39] visualization
34 aligns different level features with state-of-the-an performance with single-level features. SFNet
semantic flow to improves FPN performance in model. SAFNet 30 introduced Adaptive Feature Fusion and Self-Enhanced Modules. 4 presented a parallel FPN structure for object detection with bi-directional methods in the network, the previous FPN-based fusion structures still have drawbacks in low speed, cross-level information exchange and information loss.
due to the excessive number of paths and indirect interaction
However, due to the excessive number of paths and indirect interaction methods in the network, the previous FPN-based fusion structures still have drawbacks in low speed, cross-level information exchange and information loss.
3 Method
3.1 Preliminaries
The YOLO series neck structure, as depicted in Fig employs traditional FPN structure, which comprises multiple branches for multi-scale feature fusion. However, it only fully fuse features from neighboring levels, for other layers information it can only be obtained indirectly 'recursively.. In Fig 3 it shows the information fusion structure of the conventional FPN: where existing level-1, 2. and 3 are arranged from top to bottom. FPN is used for fusion between different levels. There are two distinct scenarios when level-1 get information from the other two levels:
1) If level-1 seeks to utilize information from level-2, it can directly access and fuse this information.
2) If level-1 wants to use level-3 information, level-1 should recursively calling the information fusion module of the adjacent layer. Specifically, the level-2 and level-3 information must be fused first, then level-1 can indirectly obtain level-3 information by combining level-2 information.
This transfer mode can result in a significant loss of information during calculation. Information interactions between layers can only exchange information that is selected by intermediate layers, and not selected information is discarded during transmission. This leads to a situation where information certain level can only adequately assist neighboring layers and weaken the assistance provided to other global layers. As result, the overall effectiveness of the information fusion may be limited.
To avoid information loss in the transmission process of traditional FPN structures, we abandon the original recursive approach and construct novel gather-and and-adntrrbbute mechanism (GD). By using unified module to gather and fuse information from all levels and subsequently distribute it to different levels, we not only avoid the loss of information inherent the traditional FPN structure also enhance the neck's partial information fusion capabilities without significantly increasing latency. Our approach thus allows for more effective leveraging the features extracted by the backbone, and can be easily integrated into any cxisting backbone-nech neckboeeccck- structure.

---

In our implementation, the process gather and distribute correspond to three modules: Feature Aligmment Module (FAM), Information Fusion Module (IFM), and Information Injection Module (Inject).
The gather process involves two steps. Firstly, the FAM collects and aligns features from various levels. Secondly, IFM fuses the aligned features to generate global information.
Upon obtaining the fused global information from the gather process, the inject module distribute this information across each level and injects it using simple attention operations. subsequently enhancing the branch's 's detection capability.
To enhance the model's ability to detect objects of varying sizes, we developed two branches: low- stage suther-and-distribur branch (Low-GD) and high- These branches extract and fuse large and small size feature maps, respectively. Further details provided in Sections 4.1 and 4.2. As shown in Fig. 2 the neck's input comprises the feature maps B2, B3, B4, B5 extracted by the backbone, where B, E Nxxx x Rg, The batch size is denoted by N, the channels by C, and the dimensions by R = H x W. Moreover, the dimensions of RR,, R3, R4, and R5 are R. 1 R. R, and 3 R, respectively.
branch (High-GD).
3.2 Low-stage gather-and-distribn branch
In this branch, the output B2, B3, B4, B5 features from the backbone are selected for fusion to obtain high resolution features that retain small target information. The structure show in Fig (a)
Low-stage feature alignment module. In low-stage feature alignment module (Low-FAM), we employ the average pooling (AvgPool) operation to down-sample input features and achieve a unified size. By resizing the features to the smallest feature size of the group BR = 1 R), we obtain Falign. The Low-FAM technique ensures efficient aggregation of information while minimizing the computational complexity for subsequent processing through the transformer module.
The target alignment size is chosen based on two conflicting considerations: (1) To retain more low-level information, larger feature sizes are preferable; however, (2) as the feature size increases, the computational latency of subsequent blocks also increases. To control the latency in the neck part, it is necessary to maintain smaller feature size.
Therefore, choose the R.11 the target size of feature alignment to achieve balance between speed and accuracy.
Low-stage information fusion module. The low-stage information fusion module (Low-IFM) design comprises luyti-layer reparameterized convolutional blocks (RepBlock) and split operation. Specifically, RepBlock takes Fatign (channel= sumn( (Cm2, Cna, CBA,, CBs)) input and produces Fjuse (channel = CB4 + CBS). The middle channel is an adjustable value (e.g., 256) to accommodate varying model sizes. The features generated by the RepBlock are subsequently split in the channel dimension into Fing_P3 and F Fsnj_P4, which are then fused with the different level's 's feature.
The formula is follows:
Falign F.a ([B2, B3, B4, B5)),
Fruse RepBlock ( (Falign), Finj_Ps, Fin.PA Split(Frusc)
(2)
Information injection module. In order inject global information more efficiently into the different levels, we draw inspiration from the segmentation experience [48] and employ attention operations fuse the information, illustrated in Fig. [ Specifically, we input both local information (which refers to the feature of the current level) and global inject information (generated by IFM), denoted as Flocal and Finj. respectively. We use two different Convs with Finy for calculation, resulting in F Fglobat_eman embed and Fact- While Flocal_embed is calculated with Flocal using Conv. The fused feature Fout is then computed through attention. Due to the size differences between Fioeal and Fgtobal, we employ average pooling or bilinear interpolation to scale F Fatohat Fglohat_embed embeed and Fact according to the size of Finj. ensuring proper alignment. At the end of each attention fusion, we add the RepBlock to further extract and fuse the information.
5

---

Low-GD
High-GD
sggh
Aboila
FAM
(a) low-stage gather- and-distribute branch
(b) high-stage gather-ande outher-and-oee and-distribuut branch
Figure 4: Gather-and- Diitttrbib structure. In (a), the Low-FAM and Low-IFM is low-stage feature alignment module and low- stage information fusion module in low-stage branch, respectively. In (b), the High-FAM FAM and High-IFM is high-stage feature alignment module and high-stage information fusion module, respectively.
In low stage, Fiocal is equal to Bi, so the formula is follows:
Fglobalact act = resize (ti.niiiiieaeeseiiiiiooCoo... (Finj (i))),
(4)
Pgloball em rewseeeeooossieee bebbeeeeeebbbeeeeeee...e Pi (Finj Pi),,
(5)
Fatt_fuse_ fuse_ = Cotcoooool embed_ BaabbbeddPPppp (Bi) Fing_act_ act_ + Felobal embed_Pi Pir
(6)
Pi (eepBlock(Fatt_fueee
(7)
3.3 High-stage gather-and-distributt branch
The Hi fuses the features P4, 4b)
P4, P5} that are generated by the Low-GD, shown in
High-stage feature alignment module. The high-stage feature alignment module (High-FAM) consists of avgpool, which is utilized to reduce the dimension of input features to uniform size. Specifically, when the size of the input feature { (Rr,, Rp4. Rp5}, avgpool reduces the feature size the smallest size within the group of features (Rys 8 R). Since the transformer module extracts high-level information, the pooling operation facilitates information aggregation while decreasing the computational requirements for the subsequent step in the Transformer module.
High-stage information fusion module. The high-stage information fusion module (High-IFM) comprises the transformer block (explained in greater detail below) and a splitting operation, which involves three-step process: (1) the Fatign= derived from the High-FAM,,
combined using the transformer block to obtain the Fruse. (2) The Fi channel is reduced to sumnCCps Cps) via Convl x 1 operation. (3) The Fuse partitioned into Finj_N and Finj N5 along the channel dimension through a splitting operation, which is subsequently employed for fusion with the current level feature.
The formula is as follows:
Falign High (A3 P4, P5)). Fruse Trons (Fatign)
(8)
(9)
FAiNNA, FUiNS5 = Split(Convl x 1(Fruse)).
(10)
The transformer fusion module in Eq. comprises several stacked transformers, with the number of transformer blocks denoted by L. Each transformer block includes multi-head attention block, Feed-Forward Network (FFN), and residual connections. To configure the multi-head attention block, we adopt the same settings Le [15]. assigning head dimensions of keys K and queries Q D (e.g., 16) channels, and V = 2D (e.g., 32) channels. In order to accelerate inference, we substitute the velocity-unfriendi unfriendly operator, Layer Normalization, with Batch Normalization for each convolution. and replace all GELU activations with ReLU. This minimizes the impact of the transformer module on model's speed. To establish our Feed-Forward Network, we follow the methodologies presented in 28 (6)) for constructing the FFN block. To enhance the local connections of the transformer block, we introduce depth-wise convolution layer between the two 1xl convolution layers. We also set the expansion factor of the FEN to 2, aiming to balance speed and computational cost.

---

Inject
Inject-LAF
LAF Low-stage
LAF High-stage
Conv txt
Conv
sugpost/
nigpael/
augpool/ Blineer
argpool / biiiner
Sigmold
Sigmold
sygpool
Conv txt
bilinear
avgpool
Conv
Conv
Conv hdl
Cony tel
8_n-1
B_n
B_n+1
P_n-1
P_n
X.local
x_global
X_local
X_global
(a) Information injection module
(b) lightweight adjacent layer fusion module
Figure 5: The information injection module and lightweight adjacent layer fusion(LAF) module
Information injection module. The information injection module High-GD is exactly the same in Low-GD, In high stage, Flocal is equal to Pi, so the formula is as follows:
F Lact Ni resize SaansiieSSigoiii(Cnnn.oo Finj_Ni))), F gglobal cmbee__ii "global_ambeed_NN resize(Con@gioobll embed Ns (F((Fin (Finj_Ni)).
(11)
(12)
Fast_ Ni Con embeed_eeeddddidi (Pi) Fing_act_N + F Fglobal embeedNii
(13)
Ni RepBlock F fuse_fue__Nii
(14)
3.4 Enhanced cross-layer information flow
We have achieved better performance than existing methods using only a global information fusion structure. To further enhance the performance, drew inspiration from the PAFPN module in YOLOv6 (32 and introduced an Inject-LAF module. This module is an enhancement of the injection module and includes lightweight adjacent layer fusion (LAF) module that added the input position of the injection module.
To achieve balance between speed and accuracy, designed two LAF models: LAF low-level model and LAF high-level model, which are respectively used for low-level injection (merging features from adjacent two layers) and high-level injection (merging features from adjacent one layer). There structure is shown in Fig. (b).
To ensure that feature maps from different levels are aligned with the target size, the two LAF models our implementation utilize only three operators: bilinear interpolation to up-sample features that are too small, average pooling to down- down-sample features that are too large, and 1x1 convolution to adjust features that differ from the target channel.
The combination of the LAF module with the information injection module in our model effectively balances the between accuracy and speed. By using simplified operations, are able increase the number of information flow paths between different levels, resulting in improved performance without significantly increased the latency.
3.5 Masked image modeling pre-training
Recent methods such as BEiT [.. MAE 24 and SimMIM [52] have demonstrated the effectiveness of masked image modeling (MIM) for vision tasks. However, these methods not specifically tailored for convolutional networks (convnets). SparK [46) and ConvNeXt- exploring the potential of masked image modeling for convnets.
[1]]
are pioneers in
In this study, we adopt MIM Pre-training following the SparK's [46] methodology, which successfully identifies and overcomes two key obstacles in extending the success of MAE-style pretraining to

---

Table 1: Comparisons with other YOLO-series detectors on COCO 2017 val. FPS and lateney are measured FPI6-precision on Tesla T4 in the same environment with TensorRT 7. All our models are trained for 300 epochs. Both the accuracy and the speed performance of our models are evaluated with the input resolution 640x640. 't' represents that the self-distillation method utilized, and
'*' represents that the MIM pre- praiiigg method is utilized.
Method
Input Size
APM
AP
FPS
FPS
Lateney (Ds=1)
Params
FLOPs
YOLOv5-N
B3
640
28.0%
45.7%
602
1.7 ms
M
YOLOv5-S
640
37.4%
56.8%
376
444
16.5
YOLOvv M
13
6400
45.4%
64.1%
182
209
5.5 ms
21.2 M
49.0
YOLOv5-L
640
49.0%
113
ms
YOLOX-Tiie Tiny
11
416
32.8%
50.3%
717
1143
1.4 ms
5.1 M
6.5 G
YOLOX-S []]
640
40.5%
59.3%
333
ms
9.0
26.8
YOLOX-M [1]
640
46.9%
65.6%
155
179
6.4
25.3
73.8
YOLOX-L
640
49.7%
68.0%
94
103
10.6 ms
54.2 M
155.6
PPYOLOE-S [53
640
43.1%
59.6%
327
419
7.9 M
17.4G
PPYOLOE-M [33
640
49.0%
65.9%
152
189
6.6 ms
23.4 M
49.9
PPYOLOE-L 33
640
51.4%
68.6%
101
127
10.1 ms
52.2
110.1
YOLOv7-Tim
49
416
33.3%
49.9%
787
1196
1.3 ms
6,2 M
G
YOLOv7-Tinv
640
37.4%
55.2%
424
ms
13.7 G
YOLOv7
40
640
51.2%
69.7%
110
122
9.0
36.9
104.7
YOLOv7-EM
49
1280
56.8%
74.4%
16
17
59.6 ms
151.7 M
843.2 G
YOLOv8-N
14
640
37.3%
52.6%
561
734
1.8
3.2
G.7
YOLOv8-S
640
44.9%
61.8%
311
387
3.2 ms
11.2 M
G
YOLOv8-M
640
50,2%
67.2%
143
7.0 ms
25.9
78.9
YOLOv8-L
14
640
52.9%
69.8%
105
11.0
43.7
165.2
YOLOv6-3.0-N
640
0%
37.5%%
52.7% 53.1%
779
1187
1,3
11.4G
YOLOv6-3.0-S
640
44.3% 45.0%%
01.2%
61.8%
339
18.5 M
45.3
YOLOv6-3.0-M
32
640
49.1% 55.00%
66.1% 66.9%
175
ms
34.9
85.8
YOLOv6-3.0-1 32
640
51.8% /2.
69.2% / 70.3%*
98
116
10.3 ms
59.6 M
150.7
Gold- YOLO-N
640
39.6%
339996T
55.7% 55.9%
563
1030
1.7 ms
12.1
Gold YOLO-S
640
45.4% 46.1%%
62.5% / 63.3%
286
446
3.3 ms
21.5 M
46.0
Gold-YOLO-M
640
50.9%% 49.8%
67. 07.00%% 2%
152
220
6,4 ms
41.3 M
87.5
Gold-YOP YOLO4I.
640
51.8% 533.2%6
68.9% 70.5%
116
11.1
75.1
151.7
Gold- YOLO-S*
640
45.5% 46.4%%
62.2% 63.4%
286
ms
21.5
46.0
Gold- YOLO--
640
50.2% 51.
67.5% 68.5%!
152
220
6.4 ms
41.3 M
87.5 G
Gold-YOLO-L*
640
52.3% 53.3%%
69.6% 70.9%!
88
116
11.1 ms
75.1 M
151.7 G
convolutional networks (convnets). These challenges include the convolutional operations' inability to handle irregular and randomly masked input images, as well as the inconsistency between the single-scale nature of BERT pretraining and the hierarchical structure of convnets.
To address the first issue, unmasked pixels treated as sparse voxels of 3D point clouds and employ sparse convolution for encoding. For the latter issue, hierarchical decoder is developed to reconstruct images from multi-scale encoded features. The framework adopts UNet-style architecture decode multi-scale sparse feature maps, where all spatial positions are filled with embedded masks. We pretrain our model's backbone on ImageNet K for multiple Gold- HOLO models, and results in notable improvements
4 Experiment
4.1 Setups
Datasets. We perform extensive experiments on the Microsoft COCO datasets validate the proposed detector. For the ablation study, we train on COCO train2017 and validate on COCO v datasets. We use the standard COCO AP metric with a single scale image as input, and report the standard mean average precision (AP) result under different IoU thresholds and object scales.
Implementation details. We followed the setup of YOLOv6-3.0 32 use the same structure (except for neck) and training configurations. The backbone of the network implemented with

---

the EfficientRep Backbone, while the head utilized the Efficient Decoupled Head. The optimizer learning schedule and other setting also same as YOLOv6, i.e. stochastic gradient descent (SGD)
with momentum and cosine decay on learning rate. Warm-up, grouped weight decay strategy and the exponential moving average (EMA) are utilized. Self-distillation and anchor- aided training (AAT)
also be used in training. The strong data augmentations we adopt Mosaic [2] [3] and Mixup 58)
We conducted MIM unsupervised pretraining on the backbone using the 1.28 million ImageNet-I1 datasets 8 Following the experiment settings in Spark [46], we employed LAMB optimizer [55] and cosine-annealing learning rate strategy, with a masking ratio of 60 % and mask patch size of
32. For the Gold- YOLO-L models, we employed a batch size of 1024, while for the Gold- YOLO-M
models, a batch size of 1 1152 was used. MIM pretraining was not employed for Gold- YOLO-N due to the limited capacity of its small backbone.
All models trained 8 NVIDIA A100 GPUs, and the speed performance measured on NVIDIA Tesla T4 GPU with TensorRT.
4.2 Comparisons
Our focus is primarily on evaluating the speed performance of our models after deployment. Specifi- cally, we measure throughput (frames per second a batch size of or 32) and GPU latency, rather than FLOPs the number of parameters. To compare Gold- YOLO with other sfate-of-the-art detectors in the YOLO series, such as YOLOv5 [33. YOLOX [T]. PPYOLOE [53], YOLOv7 [49], YOLOv8 [4 and YOLOv6-3.0 [32.. we test the speed performance of all official models with FP16-precision on the same Tesla T4 GPU with TensorRT.
Gold- YOLO-N demonstrates notable advancements, achieving an improvement of 2.6%/2.4%%/6.6 compared to YOLOv8-N, YOLOv6-3.0-A and YOLOv7- Tiny (input size=416), respectively, while offering comparable superior performance in terms of throughput and latency. When compared to YOLOX-S and PPYOLOE-S, Gold YOLO-S demonstrates a notable increase AP by 5.9%/3. 1%, while operating at a faster speed of 50/27 FPS (with a batch size of 32).
Gold-YOLO-M outperforms YOLOv6-3.0-M, YOLOX-M and PPYOLOE-M by achieving 1.1%. 4.2% and 2.1% higher AP with a comparable speed. Additionally, achieves 5.7% and 0.9% higher AP than YOLOv5-M and YOLOv8-M, respectively, while achieving a higher speed. VOLL-YOLO-M outperforms YOLOv7 with significant improvement of 98FPS (batch size = 32). while maintaining the same AP. YOLO-L also achieves a higher accuracy compared to YOLOv8-L and YOLOv6- 3.0-L, with a noticeable accuracy advantage of 0.4% and 0.5% respectively, while maintaining similar FPS at batch size of 32.
4.3 Ablation study
4.3.1
Ablation study on GD structure
To verify the validity of our analysis concerning the FPN and to assess the efficacy of the proposed gather-and-disrriutee mechanism, we examined each module in GD independently, focusing on AP, number of parameters, and latency on the T4 GPU. The Low-GD predominantly targets small and medium-sized objects, whereas the High-GD primarily detect large-sized objects, and the LAF module bolsters both branches. The experimental results are displayed in Table
Table 2: Ablation study on GD structure. The test model Gold- YOLO-S on T4 GPU evaluate.
Low-GD
High-GD
LAF
AP
AP-small
AP-medium
AP-large
FPS (8um32)
Params
FLOPs
44.65%
25.13%
49.70%
60.36%
454.4
18.7 M
45.1
42.27%
20.79%
47.74%
61 09.9%
493.7
20.9 M
43.6G
44.36%
25.04%
49.53%
60.64%
4
45.57%
24.90%
50.38%
63.50%
461.8
21.5 M
4
7
46.11%
25.22%
51.23%
63.42%
446.2
21.5 M
46.0
4.3.2 Ablation study on LAF
In this ablation study, we conducted experiments to compare the effects of different module designs within the LAF framework and evaluate the influence of varying model sizes accuracy. The

---

results of our study provide evidence to support the assertion that the existing LAF structure is indeed optimal. The difference between model-1 and model-2 is whether LAF uses add or concat, and model-3 increase the model size basis of model-2. The model-4 is based on model-3 but discards LAF. The experimental results are displayed Table
Table 3: Ablation study LAF. Use TensorkT T4 GPU evaluate.
concat
add
increase model
AP-small
AP-medium
AP-large
Params
46. 46.11%
25.22%
51.23%
63.42%
446.2
21.5 M
46.0
45.43%
24.98%
50.66%
62.10%
413.2
20,9
47.5
46.49%
26.32%
51.29%
63.43%
356.0
30.8
54.1
46.47%
25.27%
50.80%
64.25%
373.8
26.4 M
52.9 G
4.3.3 Ablation study on other model and task
The GD mechanism is general concept and can be applied beyond YOLOs. We have extend GD mechanism to other models and obtain significant improvement.
On Instance Segmentation task, we replace different necks in Mask R-CNN and train/test on the COCO instance datasets. The result shown in the Table
Table Ablation study on Instance Segmentation Task.
Model
Neck
FPS
Bbox mAP
Bbox mAP:50
Segm mAP
Segm mAP:50
MaskRCNN-Restee
FPN
21.6
38.2%
58.8%
34.7%
55.7%
MaskRCNN-ResNee
AFPN
19.1
36.0%
53.6%
31.8%
50.7%
MaskRCNN-ResNetSh
PAFPN
20.2
37.9%
58.6%
34.5%
55.3%
Mrss
GD
18.7
40.7%
59.5%
36.0%
56.4%
On Semantic Segmentation task, we replace different necks in PointRend and train/test on the Cityscapes datasets. The result as shown in the Table
Table Ablation study on Semantic Segmentation Task.
Table 6: Performance of GD mechanism
on other object detection models.
Model
Neck
FPS
mIoU
mAcc
aAce
PointRend Reit
FPN
76.47
84.05
95.96
model
Neck
FPS
AP
PointRend ResNet50
GD
11.07
78.54
85.60
96.12
EfficientDet EfficientDet
BiFPN
6.0
34.4
PointRend- PointRen--ResNettO
FPN
78.30
85.71
96.23
GD
5.7
38.8
PointRend-ResNee ResNe1101
GD
7.54
80.01
86.15
96.34
object detection task, we replace different necks in EfficientDet and train/test on the COCO
datasets. The result shown in the Table
5 Conclusion
In this paper, we revisit the traditional Feature Pyramid Network (FPN) architecture and critically analyze its constraints in terms of information transmission. Following this, we subsequently developed the Gold- YOLO series models for object detection tasks, achieving state-of-the-ar results. In Gold- YOLO introduce innovative ditttr-and-distributee mechanism, strategically designed to enhance the efficacy and efficiency of information fusion and transmission, avoid unnecessary losses, thereby significantly improving the model's 's detection capabilities. We truly hope that our work will prove valuable in addressing real-world problems and may also ignite fresh ideas for researchers in this field.
10

---

Acknowledgement
We gratefully acknowledge the support of MindSpore [29]. CANN (Compute Architecture for Neural Networks) and Ascend Al Processor used for this research.
References
[1] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image trans- formers, 2022.
[2] Alexey Bochkovskiy, Chien-Ya Wang, and Hong- Yuan Mark Liao. Yolov4; Optimal speed and accuracy object detection. arXiv preprint arXiv:2004. 10934, 2020.
[3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pages 213-229. Springer, 2020.
[4] Ping- Ping-Yang Yang Chen, Ming-Chine -hiigChing Chang, Jun-Wei Hsieh, and Yong-Sheng Chen. Parallel residual bi-fusion feature pyramid network for accurate single-shot object detection. IEEE Transactions on Image Processing, 30:9099-9111, 2021.
[5] Ping-Yau Yang Chen, Jun-Wei Hsieh, Chien-Yao Wang, and Hong- Yuan Mark Liao. Recursive hybrid fusion pyramid network for real-time small object detection on embedded devices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 402-403, 2020.
[6] Qiang Chen, Yingming Wang, Tong Yang. Xiangyu Zhang, Jian Cheng, and Jian Sun. You only look one-level feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13039-13048, 2021.
[7] Yukang Chen, Tong Yang. Xiangyu Zhang, Gaofeng Meng, Xinyu Xiao, and Jian Sun. Detnas: Backbone search for object detection. Advances in Neural Information Processing Systems, 32, 2019.
[8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint UxXiv:2010.11922 2020.
[10] Kaiwen Duan, Song Bai. Lingxi Xie, Honggang Qi, Qingming Huang, and Tian. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6569 6578, 2019.
[11] Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.084300 2021.
[12] Golnaz Ghiasi, Tsung- Yi Lin, and Quoc Le. Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7036-7045, 2019.
[13] Jocher Glenn. Yolov5 release v..1. releases/tag/vol v 2022.
[14] Jocher Glenn. Ultralyties yolov8. http:///gtthbbcoo/lllaaaaalllllp 2023.

---

[15] Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervé Jégou, and Matthijs Douze. Levit: a vision transformer convnet's clothing for faster inference. In Proceedings of the IEEF/CVF international conference on computer vision, pages 12259- 12269, 2021.
[16] Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, and Chang Xu. Distilling object detectors via decoupled features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2154-2164, 2021.
[17] Jianyuan Guo, Kai Han, Yunhe Wang, Chao Zhang, Zhaohui Yang, Han Wu, Xinghao Chen, and Chang Xu. Hit-detector: Hierarchical trinity architecture search for object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattrr recognition, pages 1 11405-11414 2020.
[18] Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12175-12185, 2022.
[19] Jianyuan Guo, Kai Han, Han Wu, Chao Zhang, Xinghao Chen, Chunjing Xu, Chang Xu, and Yunhe Wang. Positive-unlabeled data purification in the wild for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2653 2663,266,222 2021.
[20] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang. An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45(1):87-110, 2022.
[21] Kai Han, Yunhe Wang. Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1580-1589 2020.
[22] Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in transformer. Advances in Neural Information Processing Systems, 34: 34:1540:4 15908-15919, 2021.
[23] Zhiwei Hao, Jianyuan Guo, Ding Jia, Kai Han, Yehui Tang, Chao Zhang. Han Hu, and Yunhe Wang. Learning efficient vision transformers via fine-grained manifold distillation. In Advances in Neural Information Processing Systems, 2022.
[24] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoeneoders are scalable vision learners, 2021.
[25] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. Proceedings of the IEEE international conference on computer vision, pages 2961-2966 2017.
[26] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[27] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704 04861, 2017.
[28] Zilong Huang, Youcheng Ben, Guozhong Luo, Pei Cheng, Gang Yu, and Bin Fu. Shuffle transformer: Rethinking spatial shuffle for vision transformer. arXiv preprint arXiv:2106.0366 2021.
[29] Huawei. Mindspore, https://www. http://wwwwmiddpor...ee cn/ 2020.
[30] Zhenchao Jin, Bin Liu, Qi Chu, and Nenghai Yu. Safnet: semi-anchor-free network with enhanced feature pyramid for object detection. IEEE Transactions on Image Processing, 29:9445 9954445--557,,0 2020.
[31] Zhenchao Jin, Dongdong Yu, Luchuan Song, Zehuan Yuan, and Lequan Yu. You should look at all objects. In European Conference Computer Vision, pages 332-349. Springer, 2022.
12

---

[32] Chuyi Li, Lulu Li, Yifei Geng, Hongliang Jiang, Meng Cheng, Bo Zhang, Zaidan Ke. Xiaoming Xu, and Xiangxiang Chu. Yolovó v3. 0: A full-scale reloading, arXiv preprint XXv:23001005580 2023.
[33] Chuyi Li, Lulu Li, Hongliang Jiang, Kaiheng Weng, Yifei Geng, Liang Li, Zaidan Ke, Qingyuan Li, Meng Cheng. Weiqiang Nie, et al. Yolov6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:220..02976 2022.
[34] Xiangtai Li, Ansheng You, Zhen Zhu, Houlong Zhao, Maoke Yang, Kuiyuan Yang. Shaohua Tan, and Yunhai Tong. Semantic flow for fast and accurate scene parsing. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28. 2020, Proceedings, Part 1 16, pages 775-793. Springer, 2020.
[35] Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125, 2017.
[36] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEF/CVF international conference on computer vision, pages 10012-10022, 2021.
[37] Wenyu Lv, Shangliang Xu, Yian Zhao, Guanzhong Wang, Jinman Wei, Cheng Cui, Yuning Du, Qingging Dang, and Yi Liu. Detrs beat yolos on real-time object detection. arXiv preprint arXiv:2304.08060 080699 2023.
[38] Y Quan, D Zhang, L Zhang, and J Tang. Centralized feature pyramid for object detection. arxiv 2022. arXiv preprint arXiv:2210.02093, 41.
[39] Harish Guruprasad Ramaswamy et al. Ablation-cam: Visual explanations for deep convolutional network via gradient-fre free localization. In Proceedings of the /EEE/CVF Winter Conference on Applications of Computer Vision, pages 983-991, 2020.
[40] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 779-788, 2016.
[41] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7263-7271, 2017.
[42] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02667 2018.
[43] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
[44] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114 14. PMLR, 2019.
[45] Mingxing Tan, Ruoming Pang, and Quoe Lc. Efficientdet: Scalable and efficient object detec- tion. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10781 10791,, 2020.
[46] Keyu Tian, Yi Jiang, Qishuai Diao, Chen Lin. Liwei Wang, and Zehuan Yuan. Designing bert for convolutional networks: Sparse and hierarchical masked modeling. arXiv preprint arXiv:2301.03580 2023.
[47] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. In Proceedings of the IEEF/CV international conference on computer vision, pages 9627-9636. 2019.
[48] Qiang Wan, Zilong Huang, Jiachen Lu, Gang Yu, and Zhang. Seaformer: Squeeze-enhanced axial transformer for mobile semantic segmentation. arXiv preprint arXiv:2301. 2023.
13

---

[49] Chien-Yan Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of- freebies sets new state-of-the-arr for real-time object detectors. arXiv preprint S3lv:2207.006966 2022.
[50] Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng. Panet: Few-shot image semantic segmentation with prototype alignment. In proceedings of the IEEE/CVF international conference computer vision, pages 9197-9206, 2019.
[51] Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders, 2023.
[52] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling, 2022.
[53] Shangliang Xu, Xinxin Wang, Wenyu Lv, Qinyao Chang, Cheng Cui, Kaipeng Deng, Guanzhong Wang, Qingqing Dang. Shengyu Wei, Yuning Du, al. Pp-yoloe: An evolved version of yolo. arXiv preprint arXiv: 162.0 2022.
[54] Guoyu Yang. Jie Lei, Zhikuan Zhu, Siyu Cheng, Zunlei Feng, and Ronghua Liang. Afpn: Asymptotic feature pyramid network for object detection. arXiv preprint arX1v:2306.15988 2023.
[55] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert 76 minutes, 2020.
[56] Kun Yuan, Shaopeng Guo, Ziwei Liu, Aojun Zhou, Fengwei Yu, and Wei Wu. Incorporating convolution designs into visual transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 579-588, 2021.
[57] Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03660 2022.
[58] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond 2017.
empirical risk minimization. arXiv preprint
[59] Wenqiang Zhang, Zilong Huang, Guozhong Luo, Tao Chen, Xinggang Wang, Wenyu Liu, Gang Yu, and Chunhua Shen. Topformer: Token pyramid transformer for mobile semantic the 2022.
segmentation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages
[60] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6848-6856, 2018.
[61] Qijie Zhao, Tao Sheng, Yongtao Wang, Zhi Tang. Ying Chen, Ling Cai, and Haibin Ling, M2det: single-shot object detector based on multi-level feature pyramid network. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 9259-92666 2019.
[62] Dehua Zheng, Wenhui Dong, Hailin Hu, Xinghao Chen, and Yunhe Wang. Less is more: Focus attention for efficient detr. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pages 6674-6683, 2023.
[63] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint ar 200.:20010.0415, 2020.
14

---

A Additional experiment
A.1 More detailed accuracy and speed data for Gold-YOLO
In this section, we report the test performace of our Gold-YOLO with or without LAF module and pre-training. FPS and latency are measured in FP16-precision on a Tesla T4 in the same environment with TensorRT 7. Both the accuracy and the speed performance of our models are evaluated with the input resolution of 640x640. The result shown in Table 7
Table 7: Test results of Gold-YOLO series model on COCO 2017 val. 'f" represents that the self-distillation method is utilized, 'o' represents that the model don't have LAF module, and 'x' represents that the MIM pre-training method is utilized.
Ap
AP
A A
Params
Gold-YOLO-N
38.37%//388225
18.37%
42.62%
55.13%
G
Gold-YOLo YOLO.S*
(41477%/4455574
61.55% / 62.92%
24.04% / 24.90%
49.34%, 50.38%
61.95% / 63.50%
462
21.5 M
45.8 G
CoL--YOLO-M
49.41% 50.26%
66.64% 67.58%
31.19% / 31.59%
54.30% / 55,26%
65.91% / 67. 67.62%%
229
41.3 M
86.8 G
GoLO-YOLO-1*
51.68%/52.6552
69.07% / 70.25%
34.86%
34.20%/
56.92% /57.70%)
69.00% 69.771
119
75.0
150.6 G
Gold-YOLO-N
39.57% /3, 99.92%%
55.70% / 55.94%
19.67% / 19.15%)
44.08% / 444.2%
56.98% / 57.75%
1030
5.6 M
12.1G
Gold YOLO-S
45.36%.
25.22
60.21%//51.225
633.22%
21.5
Gold-YOLO-M
49.77%
/ 50.86%1
67.01% 23% 68.94% 70.49%
32.32% / 31.01%
85, 29% 56, 56.24% 24%?
66.27%
67 67.83%11
220
41.3 M
8
Gold- YOLO-
51.84% 53, 16%1
34.12% 34.53%
57.36%
68.17%
77000%
116
75.1
151.7G
Gold-YOLO-5
45.52% 36%?
62.20% / 63, 36%
24 34.66% 25 26%
50.76 30.76%
30%1
24%
63 64%1
21.5
460G
Gold-YOLO-M"
50. 16% / 51.14%
67.52%
68.53%
30.52% / 32 23%%
35.544
56, 10%
67.64%
68.55%
220
41. M
Golo-YOLO-LL
52.25% 53.28811
69.61%
70.93%
33.09%
58.92%
69.92181
75.1
151.7
A.2 MIM pre-training ablation experiment
We also compared the YoLO-YOLO-S COCO 2017 validation results for different MIM pre-training epochs without self-distillation. The result shown in Table 8
Table 8: Test results on COCO 2017 val for different pre-training epoch setting.
Epoch
Ap
AP 50
AP smalt
AP
medium
AP large
400
45.39%
62.17%
25.01%
50.28%
62.74%
600
45.48%
62.18%
25.56% 24.66%
50.63%
62.85%
800
45.52%
62.20%
50.76%
63.24%
B Comprehensive Latency and Throughput Benchmark
B.1 Model Latency and Throughput on T4 GPU with TensorRT 8
Comparisons with other YOI.O-series detectors on COCO 2017 val. FPS and latency are measured in FP16 FP16-precision on Tesla T4 in the same environment with TensorRT 8.2. The result shown in Table 9
15

---

Table 9: Comparison of Latency and Throughput in YOLO series model on a T4 GPU using TensorRT
8.2.
Method
Input Size
FPS (bx=1)
FPS (bs=32)
Latency (be= 1)
YOLOv5-N
640
702
843
1.4 ms
YOLOv5-S
640
433
515
2.3 ms
YOLOv5-M
640
202
235
4.9 ms
YOLOv5-L
640
126
137
7.9 ms
YOLOX- -iny
416
766
1393
1.3 ms
YOLOX-S
640
313
489
2.6 ms
YOLOX-M
640
159
204
5.3 ms
YOLOX-L
640
104
117
9.0 ms
PPYOLOE-S
640
357
493
2.8 ms
PPYOLOE-M
640
163
210
6.1 ms
PPYOLOE-L
640
110
145
9.1 ms
YOLOv7-Tiny
640
464
568
2.1 ms
YOLOv7
640
128
135
7.6 ms
YOLOv6-3,0-N
640
785
1215
1.3
YOLOv6-3.0-S
640
345
498
2.9 ms
YOLOv6-3,0-M
640
178
238
5.6 ms
YOLOv6-3.0-L
640
105
125
9.5 ms
Gold- YOLO-N
640
637
1191
1.4 ms
VOLO-S
640
308
492
3.1 ms
Gold-YOLO-M
640
157
241
6.1 ms
Gold-YOLO-L
640
94
137
10.3 ms
B.2 Model Latency and Throughput on V100 GPU with TensorRT 7
Comparisons with other YOLO-scries detectors on COCO 2017 val. FPS and latency are measured in FP FP16-precision on Tesla V100 in the same environment with TensorRT 7.2. The result shown in Table 10.
Table 10: Comparison of Latency and Throughput in YOLO series model V100 GPU using
TensorRT 7.2.
Method
Input Size
FPS (bs=1)
FPS (bs=32)
Latency (bs=1)
YOLOv5-N
640
577
1727
1.4 ms
YOLOv5-S
640
449
1249
1.7 ms
YOLOv5-M
640
271
698
3.0 ms
YOLOv5-L
640
178
440
4.7 ms
YOLOX Tiny
416
569
2883
1.4 ms
YOLOX-S
640
386
1206
2.0 ms
YOLOX- M
640
245
600
3.4 ms
YOLOX-L
640
149
361
5.6 ms
PPYOLOE-S
640
322
1050
2.4 ms
PPYOLOE-M
640
222
566
4.0 ms
PPYOLOF-L
640
153
406
5.5 ms
YOLOv7-Tiny
640
453
1565
1.7 ms
YOLOv?
640
182
412
4.6 ms
YOLOv6-3.0-N
640
646
2660
1.2
YOLOv6-3.0-S
640
399
1330
2.0 ms
YOLOv6-3.0-X
640
203
676
4.4 ms
Y0LOv6-3.0-L
640
125
385
6.8 ms
Gold-YOLO-N
640
574
2457
1.7 ms
Gold-YOL.0s
640
391
1205
2.5 ms
Gold-YOL.-AA
640
238
633
4.0 ms
Gold-YOL, YOLO-L
640
146
365
6.6 ms
16

---

C Broader impacts and limitations
Broader impacts. The YOLO model can widely applied in fields such as healthcare and intelligent transportation. In the healthcare domain, the YOLO series models can improve the early diagnosis rates of certain diseases and reduce the cost of initial diagnosis, thereby saving more lives. In the field of intelligent transportation, the YOLO model can assist in autonomous driving of vehicles, enhancing traffic safety and efficiency. However, there are also risks associated with the military application of the YOLO model, such target recognition for drones and assisting military reconnaissance. We will make every effort to prevent the use of our model for military purposes.
Limitations. Generally, making finer adjustments on the structure will help further improve the model's performance, but this requires significant amount of computational resources. Additionally, due to our algorithm's 's heavy usage of attention operations, it may not be as friendly to some earlier hardware support.
D CAM visualization
Below are the CAM visualization results of the neck for YOLOv5, YOLOv6, YOLOv7, YOLOv8, and our Gold- YOLO, shown in Fig. 6 can be observed that our model assigns higher weights to the detected regions of the targets.
And compare the neck CAM visualization between YOLO and YOLOv6, shown in Fig. 7
Gol--YOLO-N Figure 6: The CAM visualization results of the neck for YOLOv5, YOLOv6, YOLOv7, YOLOV8, and our Gold-YOLO
COCO
YOLOv5-N
YOLOv6-N
YOLOv7-T
YOLOv8-N
17

---

GD-Neck
FPN-Neck
P4
P3
N3
N4
Figure 7: Neck CAM visualization. We can observe that features at different levels exhibit distinct preferences for objects of different sizes. In the traditional grid-structured FPN, with increasing network depth and information interaction between different levels, the sensitivity of the feature map to object positions gradually diminishes, accompanied by information loss. Our proposed GD mechanism performs global fusion separately for high-level and low-level information, resulting in globally fused features that contain abundant position information for objects of various sizes.
E
Discussion
E.1 The differences between feature alignment module of Gold-YOLO and other similar works.
Both M2Det and RHF-Net have incorporated additional information fusion modules within their alignment modules. In M2Det, the SFAM module includes an SE block, while in RHF-Net, the Spatial Pyramid Pooling block augmented with Bottleneck layer. In contrast to M2Det and RHF-Net, Gold-YOLO leans towards functional separation among modules, segregating feature alignment and feature fusion into distinct modules. Specifically, the FAM module GoldD- focuses solely on feature alignment. This ensures computational efficiency within the FAM module. And the LAF module efficiently merges features from various levels with minimal computational cost, leaving greater portion of fusion and injection functionalities to other modules.
Based on the GD mechanism, achieving SOTA performance for the YOLO model can be accomplished using simple and easily accessible operators. This strongly demonstrates the effectiveness of the approach we propose. Additionally, during the network construction process, we intentionally opted for simple and thoroughly validated structures. This choice serves prevent potential development and performance issues arising from certain operators being unsupported by deployment devices. As result, it guarantees the usability and portability of the entire mechanism.
E.2 Simple calculation operation
In the process of network construction, have drawn from and built upon the experiences of previous researchers. Rather than focusing solely on performance improvements achieved by enhancing specific operators or local structures,
emphasis lies on the conceptual shift brought about by the GD mechanism in comparison to the traditional FPN structure. Through the GD mechanism, achieving SOTA performance for the YOLO model is attainable using simple and easily applicable operators. This serves as strong evidence for the effectiveness of the proposed approach.
Additionally, during the network construction process, intentionally opted for simple and thor- oughly validated structures. This choice serves to prevent potential development and performance issues arising from certain operators being unsupported by deployment devices. As result, it guarantees the usability and portability of the entire mechanism. Moreover, this decision also creates opportunities for future performance enhancements.
The GD mechanism general concept and can be applied beyond YOLOs. We have extend GD mechanism to other models and obtain significant improvement. The experiments demonstrate
18

---

that our proposed GD mechanism exhibits robust adaptability and generalization. This mechanism
consistently brings about performance improvements across different tasks and models.
19


Reviews:
[{"review_id": "tjXfwyTIZn", "reviewer": "Reviewer_CoJm", "summary": "The paper presents a real-time object detection method for the YOLO series, introducing a 'Gather-and-Distribute' (GD) mechanism. Despite achieving good results on the COCO dataset, the paper lacks significant novelty and doesn't significantly advance multi-scale feature fusion or FPN-based methods. A deeper comparison with prior work could strengthen its scientific contribution.", "questions": "1. Could you elaborate on why YOLOv7-E6E has been chosen as the \"L\" model for comparison, given that its size is 1280? It seems that YOLOv7-X would be a more suitable choice for a fair comparison.\n\n2. The relevance of the section \"Masked image modeling pre-training\" isn't clear, as it doesn't appear to directly relate to multi-scale features and FPN-based methods. Additionally, the results from this section don't seem to add significant value. Would it be more beneficial to include additional comparisons or ablation studies for multi-scale features in lieu of this section?", "limitations": "I want to highlight the author's commendable acknowledgment of the potential military applications of their model. They clearly state their commitment to prevent such uses. This level of social impact consideration is laudable and sets a positive precedent for responsible research conduct.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The paper exhibits commendable clarity with well-written content and lucidly presented figures, making it easy to understand.\n\n2. Performance-wise, the proposed contribution demonstrates impressive results on the COCO dataset in terms of both accuracy and computational efficiency.\n\n3. The introduction of the 'Gather-and-Distribute' (GD) mechanism is particularly striking. This method enhances multi-scale feature fusion capabilities, striking an excellent balance between latency and accuracy across different model scales.", "weaknesses": "Weaknesses:\n\n1. Despite achieving strong results on the COCO dataset in terms of accuracy and efficiency, the paper lacks substantial scientific novelty. The method, while technically sound, doesn't significantly advance the field.\n\n2. The manuscript's focus is on real-time object detection and multi-scale features for the YOLO-Series. However, the related work section needs to delve more into multi-scale features.\n\n3. The concepts of Low/High-FAM and the lightweight adjacent layer fusion (LAF) module are not new in the field, having been discussed in M2Det [1] and [2] respectively.\n\nSuggestions:\n\n1. The authors need to restructure the related work section to better represent and compare with multi-scale features[1] or FPN-based methods [2-8].\n\n2. Adding more comparative analyses and surveys related to multi-scale features and FPN-based methods will establish their work as more than a minor modification of previous works.\n\n3. Despite YOLO-Series being known for speed and efficiency, GD-YOLO appears to be slower than the baseline (YOLOV6: v3). The authors should address this discrepancy.\n\n4. To reiterate, the work seems to be more application-focused with minimal contribution to the field. I recommend the authors address the points above to enhance their scientific contribution, which may change the review score. Otherwise, the work might not meet the standards of a top-tier conference.\n\n\n[1] Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., & Ling, H. (2019, July). M2det: A single-shot object detector based on multi-level feature pyramid network. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 9259-9266).\n\n[2] Chen, P. Y., Hsieh, J. W., Wang, C. Y., & Liao, H. Y. M. (2020). Recursive hybrid fusion pyramid network for real-time small object detection on embedded devices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 402-403).\n\n[3] Quan, Y., Zhang, D., Zhang, L., & Tang, J. (2023). Centralized feature pyramid for object detection. IEEE Transactions on Image Processing.\n\n[4]Yang, G., Lei, J., Zhu, Z., Cheng, S., Feng, Z., & Liang, R. (2023). AFPN: Asymptotic Feature Pyramid Network for Object Detection. arXiv preprint arXiv:2306.15988.\n\n[5] Jin, Z., Yu, D., Song, L., Yuan, Z., & Yu, L. (2022, October). You should look at all objects. In European Conference on Computer Vision (pp. 332-349). Cham: Springer Nature Switzerland.\n\n[6] Chen, Q., Wang, Y., Yang, T., Zhang, X., Cheng, J., & Sun, J. (2021). You only look one-level feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13039-13048).\n\n[7] Jin, Z., Liu, B., Chu, Q., & Yu, N. (2020). SAFNet: A semi-anchor-free network with enhanced feature pyramid for object detection. IEEE Transactions on Image Processing, 29, 9445-9457.\n\n[8] Chen, P. Y., Chang, M. C., Hsieh, J. W., & Chen, Y. S. (2021). Parallel residual bi-fusion feature pyramid network for accurate single-shot object detection. IEEE Transactions on Image Processing, 30, 9099-9111.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n\n### W-1: The paper lacks substantial scientific novelty, and doesn't significantly advance the field.\n\nThank you for your suggestions. Our core contribution lies in the proposal of the Gather-and-Distribute Mechanism (GD mechanism), which is fast and effective. Our method gathers the global information, and distribute this information into differtnt levels. The GD mechanism is performed in both high dimension and low dimension to fully enhance information exchange. By separately considering low-dimensional and high-dimensional information, we have constructed a global information fusion mechanism, thereby unifying the approaches for promoting information flow between different levels feature that were previously disparate. Our proposed GD mechanism represents a more comprehensive improvement over previous SOTA detectors.\n\nThe GD mechanism is a general concept and can be applied beyond YOLOs. We have extend GD mechanism to other models and obtain significant improvement as show in the Experiment-1, you can find in global response.\n\n### W-2: The related work section needs to delve more into multi-scale features.\n\nThank you for your suggestions. We’ll include more discussion with related works on multi-scale features in the final version. The specific content please refer to the Discuss-1 in global response.\n\n### W-3: The issue of similarity between the FAM/LAF modules and M2Det [1] / [2].\n\nThank you for raising the question.  Rather than focusing on improving specific operators or local structures, we propose a new Gather-and-Distribute (GD) mechanism compared to the traditional FPN structure, without relying on the characteristics of special operators. \n\nWe’ll include the discussion with M2Det [1] and RHF-Net [2] in our final version. The specific content please refer to the Discuss-2 in global response.\n\n## Suggestions\n\n### S-1: The authors need to restructure the related work section to better represent and compare with multi-scale features[1] or FPN-based methods [2-8].\n\nThank you for your suggestions. We will follow your advice and incorporate a more in-depth investigation and analysis of multi-scale features in the revised version. Additionally, we will cite [1-8] and other relevant articles to comprehensively outline the historical development of the relevant field. \n\nWe’ll include more discussion with related works on multi-scale features in the final version. The specific content please refer to the Discuss-1 in global response.\n\nIn addition, we tested the inference speed of some of the open source models on the V100 and plotted the accuracy-speed curve. It proves the excellent performance of our model. The experimental results can be found in Experiment-2. And the Figure 2 can be found in the global response PDF.\n\n### S-2: Adding more comparative analyses and surveys related to multi-scale features and FPN-based methods will establish their work as more than a minor modification of previous works.\n\nThank you for your suggestions. The specific content please refer to the Discuss-1 in global response.\n\n### S-3: GD-YOLO appears to be slower than the baseline (YOLOV6: v3)\n\nThank you for your feedback. In the comparison with previous generations of YOLO models, our focus lies more on the trade-off between speed and accuracy improvements, rather than solely aiming for the utmost speed or precision in a single model. Of course, we have developed a smaller and faster model (GD-YOLO-N-2), and the specific performance is detailed in the table below:\n\n| model|FPS-32|AP|AP:50|\n|-|-|-|-|\n|YOLOv6-3.0-N|1187|37.0 / 37.5|52.7 / 53.1|\n|GD-YOLO-N-2|1211|38.09 / 38.41|54.92 / 55.14 |\n\nGD-YOLO-N is built by reducing the parameter count in the neck section (removing the LAF module and High-GD branch), resulting in a smaller and faster model, GD-YOLO-N-2. This model outperforms the current SOTA YOLOv6-3.0-N in both accuracy and speed. The reason we present results for GD-YOLO-N in the article, rather than GD-YOLO-N-2, is to maintain consistent structures across models of different sizes and ensure result continuity.\n\n### S-4: The work seems to be more application-focused with minimal contribution to the field.\n\nThanks for your suggestion. Our contribution is not minimal since we propose a new scheme called GD mechanism, which is a general concept and can be applied beyond YOLOs (details can be found in the responses in W-1/2). Moreover, we think our paper is proper for NeurIPS and appeals to the research community, as the Call For Papers in NeurIPS23 stated “We invite submissions presenting new and original research on topics including but not limited to the following: Applications (e.g., vision, language, speech and audio)”. \n\n## Questions\n\n### Q-1: Why YOLOv7-E6E has been chosen as the \"L\" model for comparison\n\nThank you for sharing the information. Below is the test result for YOLOv7-X whose performance is poorer than YOLOv7-E6E. A more intuitive  Figure 1 can be found in the global response PDF.\n\n|model|FPS-32|AP|AP:50|\n|-|-|-|-|\n|YOLOv7-X|73.92|52.9|71.2|\n\n### Q-2: The value and relevance of pre-training\n\nThank you for your question. We would like to emphasize that the use of MIM pre-training is not meant to have a direct correlation with Gather-and-Distribute methods. Instead, it's intended to act as a practical and robust technique in real-world applications, enhancing the overall performance of the model without adding to the inference latency. To the best of our knowledge, we are the pioneers in applying MIM pre-training to the YOLO series. The experimental results presented in the appendix demonstrate the advantages of pre-training with ImageNet classification."}, {"review_id": "aFkFAe1e0X", "reviewer": "Reviewer_YJPg", "summary": "In this paper, the authors proposed an efficient object detection network to make a new trade-off between efficiency and effectivity. In the framework, a lightweight adjacent-layer fusion module termed as gather-and-distribute (GD) mechanism is proposed to take place the conventional neck module in general detectors. Experimental results on COCO built-on the YOLO-series off-the-peg detectors demonstrate the effectiveness of the proposed method.", "questions": "The innovation of the proposed method needs to be improved comprehensively, and there is a lack of more substantial analysis and understanding in terms of innovation in technology. Then, it is necessary to verify the scalability and robustness of the proposed method in more experiments and even more tasks. In addition, the writing and layout concept of the article still needs to be improved.", "limitations": "Applicable", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "-The bilateral interaction of information in  multi-layer layers is interesting, and I like the work that simple changes can bring significant improvement for foundational tasks.\n\n-The experimental setup of ablation studies on COCO is helpful for follow-up work in the future.", "weaknesses": "The main weakness of this paper is the limited technical novelty and the relatively inadequate experiments. First, although the authors try to explain that the proposed neck module of the GD mechanism can improve the detection accuracy of the detection task and benefit the computational efficiency, the novelty of the proposed module can be seen as a compromise similar to the stack of the existing technologies. Additionally, the experiment is carried out only on YOLO-series methods, and there is a lack of testing and verification on other datasets.\n\nThis paper needs further modification in terms of layout design, e.g., the fonts in the Fig.4 are generally small and unclear, which is very difficult for review.\n\nPage 3 Line 82-83, Improving ->can improve, introduce -> introduced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W1: The novelty of the proposed module can be seen as a compromise similar to the stack of the existing technologies, and lack of testing and verification on other datasets.\n\nThank you for your suggestions. Our core contribution lies in the proposal of the Gather-and-Distribute Mechanism (GD mechanism),  By separately considering low-dimensional and high-dimensional information, we have constructed a global information fusion mechanism, thereby unifying the approaches for promoting information flow between different levels feature that were previously disparate. Our proposed GD mechanism represents a more comprehensive improvement over previous works.\n\nIn the process of network construction, we have drawn from and built upon the experiences of previous researchers. Rather than focusing solely on performance improvements achieved by enhancing specific operators or local structures, our emphasis lies on the conceptual shift brought about by the GD mechanism in comparison to the traditional FPN structure. Through the GD mechanism, achieving SOTA performance for the YOLO model is attainable using simple and easily applicable operators. This serves as strong evidence for the effectiveness of the proposed approach.\n\nAdditionally, during the network construction process, we intentionally opted for simple and thoroughly validated structures. This choice serves to prevent potential development and performance issues arising from certain operators being unsupported by deployment devices. As a result, it guarantees the usability and portability of the entire mechanism. Moreover, this decision also creates opportunities for future performance enhancements.\n\nThe GD mechanism is a general concept and can be applied beyond YOLOs. We have extend GD mechanism to other models and obtain significant improvement as show in the Experiment-1, you can find in the Global Author Rebuttal. The aforementioned experiments demonstrate that our proposed GD mechanism exhibits robust adaptability and generalization. This mechanism consistently brings about performance improvements across different tasks and models.\n\n\n### Format problem\n\n- W-2: Fig.4 are generally small and unclear\n- W-3: Page 3 Line 82-83, Improving ->can improve, introduce -> introduced.\n\nThanks for your suggestions, we will correct these formatting and grammar problems in the revision.\n\n\n### Q-1: Lack of in-depth analysis and understanding regarding innovation, along with insufficient evidence of the scalability and robustness of the proposed method.\n\nThank you for your suggestions. We’ll include more discussion with related works on multi-scale features in the final version. The specific content please refer to the Discuss-1 in global  response.\n\nIn order to demonstrate the scalability and robustness of our proposed GD mechanism, in addition to the experiments conducted in semantic segmentation and instance segmentation tasks as mentioned in W1, we have conducted the following additional experiments to supplement our idea:\n\n| model| AP| AP-50|FPS-32|Params|FLOPs|\n|-|-|-|-|-|-|\n| GD-YOLO-S|45.4 / 46.1| 62.5 / 63.3| 446| 21.5 M | 46.00 G |\n| GD-YOLO-S-all_trans  | 46.9 / 47.0 | 64.1 / 64.4 | 258.56 | 25.2 M | 57.95 G|\n| GD-YOLO-S-all_conv| 45.6 / 46.1 | 62.5 / 63.2 | 417.76 |26.6 M| 47.06 G|\n| GD-YOLO-S-inject_all| 46.3 / 46.6 | 63.4 / 64.0 | 397.76 |23.2 M| 50.43 G|\n\nWe have developed three variants based on GD-YOLO-S:\n\n1. GD-YOLO-S-all_trans: Replaces the fusion operators in both Low-IFM and High-IFM with Transformers.\n2. GD-YOLO-S-all_conv: Replaces the fusion operators in both Low-IFM and High-IFM with Convolutions.\n3. GD-YOLO-S-inject_all: Adds Inject models to all P-level and N-level, injecting global information into each level.\n\nFrom the results of experiments 1 and 2, it is evident that the model's performance does not significantly degrade with changes in the operators within the fusion module. This demonstrates the strong robustness of the GD mechanism and its insensitivity to operators. The GD mechanism can be effortlessly combined with any operators, introducing new features by incorporating new operators.\n\nIn experiment 3, we extended the number of layers for information injection within the GD mechanism, resulting in improved accuracy at the cost of reduced speed. Additionally, in section 4.3.1 \"Ablation study on GD structure\" of the main text, we conducted detailed ablation experiments on various modules, further demonstrating the scalability of the proposed GD mechanism. In practical applications or different tasks, these modules can be freely combined and adjusted based on specific requirements to achieve optimal performance.\n\nAs per your suggestions, we will include a more in-depth analysis of multi-scale features in the revised version. And correct the grammar problems in the article, improve the graphic layout."}, {"review_id": "fVk8xrliEl", "reviewer": "Reviewer_c8e6", "summary": "This paper studies the problem of efficient object detector and proposes the Gather-and-Distribute mechanism (GD) mechanism to alleviate the information fusion problem. The experiments on the COCO dataset demonstrate the effectiveness of the proposed method.", "questions": "Please refer to the Weaknesses.", "limitations": "None", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ This paper studies an important topic, efficient object detection, and achieves a great balance between accuracy and speed in its results.\n+ The ablation studies have been provided to verify the effectiveness of the proposed module.", "weaknesses": "- The structural design of this paper is quite confusing in some aspects, such as the choice of where to inject information. For example, in Low-GD, semantic information is only injected into P3 and P4, while one would expect that global semantic information could also benefit the P5 branch. Similarly, in High-GD, information is only injected into N4 and N5. The authors should provide the rationale and advantages of this design choice to address these doubts.\n\n- In line #157, the authors mention they were inspired by [28]. However, the injection design is more closely related to Topformer, which bears greater relevance to the structure of the current paper. The authors should consider discussing and citing Topformer as another source of inspiration or relevant related work.\n\n[a] TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation, CVPR 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W-1: The structural design of this paper is quite confusing in some aspects, such as the choice of where to inject information. For example, in Low-GD, semantic information is only injected into P3 and P4, while one would expect that global semantic information could also benefit the P5 branch. Similarly, in High-GD, information is only injected into N4 and N5. The authors should provide the rationale and advantages of this design choice to address these doubts.\n\nThank you for your question. When designing the fusion mechanism, we aim for the two branches in the network to focus on low-dimensional and high-dimensional information respectively. Following this principle, we have designed and implemented two branches: Low-GD and High-GD. This design approach results in the fused features from the two branches having richer low-dimensional and high-dimensional information respectively.\n\nAdditionally, it is commonly believed that features at different levels contain positional information of objects of varying sizes. For instance, larger features encompass more low-dimensional texture information and positions of smaller objects, while smaller features contain more high-dimensional structural information and positions of larger objects. This aligns well with the characteristics of the two branches in the GD mechanism. Therefore, the global information generated by Low-GD is injected into feature maps with larger feature sizes, whereas the global information generated by High-GD is injected into feature maps with smaller feature sizes.\n\nSuch a choice is also based on a trade-off between accuracy and speed. We conducted ablation experiments to validate this idea:\n\n| model     | Inject level          | AP          | AP-50       | FPS-32 | Params | FLOPs   |\n| --------- | --------------------- | ----------- | ----------- | ------ | ------ | ------- |\n| GD-YOLO-S | [P3/P4]-[N4/N5]       | 45.4 / 46.1 | 62.5 / 63.3 | 446    | 21.5 M | 46.00 G |\n| GD-YOLO-S | [P3/P4/P5]-[N3/N4/N5] | 46.3 / 46.6 | 63.4 / 64.0 | 397.76 | 23.2 M | 50.43 G |\n\nThe experimental results indicate that as the inject level increases, the model's accuracy also increases. However, at the same time, the speed of the model decreases. The final version provided in the paper represents our optimal choice in balancing speed and accuracy.\n\n\n\n\n\n### W-2:  In line #157, the authors mention they were inspired by [28]. However, the injection design is more closely related to Topformer, which bears greater relevance to the structure of the current paper. The authors should consider discussing and citing Topformer as another source of inspiration or relevant related work.\n\nThank you for your reminder. Topformer and Seaformer are both important sources of inspiration for us. We’ll include the discussion with M2Det [1] and RHF-Net [2] in our final version:\n\nBuilt upon the concept of global information fusion, TopFormer has achieved remarkable results in semantic segmentation tasks. Expanding on the foundation of TopFormer, we have taken a step further by separately considering high-dimensional and low-dimensional information. We have meticulously designed two branches, namely Low-GD and High-GD, infusing the notion of global information fusion into the realm of object detection. As a result, we have achieved SOTA performance at object detection task."}, {"review_id": "RbpUY3OIjp", "reviewer": "Reviewer_bFVa", "summary": "In this research, the authors propose a novel Gather-and-Distribute (GD) mechanism implemented through convolution and self-attention operations. This mechanism, incorporated into the GD-YOLO model, significantly enhances multi-scale feature fusion capabilities and achieves a remarkable balance between latency and accuracy across all model scales. Notably, the researchers introduce MAE-style pretraining to the YOLO-series models for the first time, enabling unsupervised pretraining benefits. The GD-YOLO-N variant achieves exceptional results, with a 39.9% Average Precision (AP) on the COCO val2017 dataset and a remarkable 1030 Frames Per Second (FPS) on a T4 GPU. These results surpass the previous state-of-the-art model, YOLOv6-3.0-N, with a similar FPS by 2.4%.", "questions": "1. Why need to use LAMB Optimizer while already using SGD and manual scheduled Learning rate?\n2. If you use EMA, then why again, LAMB is needed?\n", "limitations": "Limitations of GD-YOLO not added; however, I would suggest adding it.  ", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "GD-YOLO is a quite impressive work, I think as it globalized the features from local and can retrieve features without tagging other layers feature with it. ", "weaknesses": "1. While GD-YOLO-N cannot use the proposed backbone for limited capacity, what will be the approaches for mobile deployment as your main objective is YOLO for mobile deployment?\n2. GD-YOLO-N implementation is not quite understandable while it misses a few parts like MIM and seems pretty same as YOLOv6 with EWA and AAT as stated.\n3. Comparison with the Transformer based model could add a bit more value to this paper.\n4. Abbreviations should be added after first introduction and then using the abbreviation is the ideal practice. For instance, in line 46, it should have been “... use Feature Pyramid Network (FPN) and…” instead of just FPN.\n5. Line 45 to 54: This para basically represents the contribution part, which can be presented in bullet terms. \n6. Line 58: “...SOTA YOLOv6…” : As per my knowledge, at this point, YOLOv8 is the SOTA. I would request for a checkup and update of this from your end. \n7. Line 74-75: What strengths YOLOv8 takes from their predecessor, it is not mentioned. \n8. Line 94: “In this study, we will…..” - it is recommended to use present tense, instead of future tense. \n9. Table 1: Best values should be bolded. \n\nMinor:\n1. Line 32: “...detectors.Despite…, Line 138: “...alignment module(Low-FAM)...” - Spacing issue. \n2. Line 257: Double comma issue. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W-1: While GD-YOLO-N cannot use the proposed backbone for limited capacity, what will be the approaches for mobile deployment as your main objective is YOLO for mobile deployment?\n\nThank you for your question. The main contribution of our work is the Gather-and-Distribute mechanism. Regarding the backbone, we are the first to pre-train the convolutional MIM in the YOLO series, aiming to explore the effect of MIM on the CNN backbone for real-time object detection task. We also compared the effects of MIM pre-training with ImageNet classification supervised pre-training on GD-YOLO-S in the appendix, and found a certain improvement. The reason for not implementing pre-training on GD-YOLO-N is that it is the most lightweight network within GD-YOLO. As such, its backbone struggles to show improvement in the MIM task.\n\nFor mobile deployment, we have demonstrated strong support on ONNX and TensorRT, which clears the path for deploying our model on mobile devices.\n\n### W-2: GD-YOLO-N implementation is not quite understandable while it misses a few parts like MIM and seems pretty same as YOLOv6 with EWA and AAT as stated.\n\nWe understand your concerns about the GD-YOLO-N implementation. As you correctly noted, GD-YOLO-N does not incorporate the Mutual Information Maximization (MIM) pre-training, unlike our other models such as GD-YOLO-S. This decision was made because GD-YOLO-N is designed to be a lightweight model. Its backbone, due to its simplicity and size, does not benefit substantially from MIM pre-training.\n\nGD-YOLO-N might appear similar to YOLOv6 with EWA and AAT because these are common techniques used to improve performance in object detection tasks. However, our work introduces the Gather-and-Distribute mechanism, which we believe adds a unique value to our model. This mechanism, in conjunction with our other design choices, aims to strike a balance between model complexity and performance, particularly for deployment on mobile devices with limited computational resources.\n\nI hope this clarifies your concerns about GD-YOLO-N. Please feel free to ask if you have further questions.\n\n### W-3: Comparison with the Transformer based model could add a bit more value to this paper.\n\nThank you for your question. We have added a chart of object detection model based on Transformer .\n\n| model                | #Params | GFLOPs | FPS bs=1 | APval | Apval-50 |\n| -------------------- | ------- | ------ | -------- | ----- | -------- |\n| RT-DETR-L            | 32      | 110    | 114      | 53    | 71.6     |\n| RT-DETR-X            | 67      | 234    | 74       | 54.8  | 73.1     |\n| DINO-Deformable-DETR | 47      | 279    | 5        | 50.9  | 69       |\n| GD-YOLO-N            | 5.6     | 12.1   | 563      | 39.9  | 55.9     |\n| GD-YOLO-S            | 21.5    | 46     | 286      | 46.4  | 63.4     |\n| GD-YOLO-M            | 41.3    | 87.5   | 152      | 51.1  | 68.5     |\n| GD-YOLO-L            | 75.1    | 151.7  | 88       | 53.3  | 70.9     |\n\nThis highlights the absolute performance advantage of GD-YOLO in the domain of small models . Despite RT-DETR significant advantages over YOLO-L/X, due to the structural constraints of the transformer, RT-DETR cannot be further downsized. This makes GD-YOLO still the best choice in the field of small models.\n\n### W-6: Line 58: “...SOTA YOLOv6…” : As per my knowledge, at this point, YOLOv8 is the SOTA. I would request for a checkup and update of this from your end.\n\nThank you for your question. It's perfectly normal to have such confusion due to the naming inconsistencies in the YOLOv6 series, which has led to misunderstandings for many. The chronological order of model releases is as follows: YOLOv6, YOLOv7, YOLOv6-2.0, YOLOv8, YOLOv6-3.0. YOLOv6-3.0 was released after YOLOv8 and significantly improved model accuracy, allowing YOLOv6-3.0 to surpass YOLOv8 on the precision-speed curve. This result is also evident in our precision-speed curve graph.\n\n### W-7: Line 74-75: What strengths YOLOv8 takes from their predecessor, it is not mentioned.\n\nThank you for your question. YOLOv8 introduces a new Conv block called C2f, which compared to YOLOv5's C3 block, enhances the gradient path even further. In terms of loss calculation, YOLOv8 adopts the TaskAlignedAssigner positive sample assignment strategy and introduces the Distribution Focal Loss. YOLOv8 is more of an integration of design elements from various YOLO series models such as YOLOX, YOLOv6, and YOLOv7, and leans more towards engineering.\n\n\n\n### Format problem：\n\n- W-4: Abbreviations should be added after first introduction and then using the abbreviation is the ideal practice. For instance, in line 46, it should have been “... use Feature Pyramid Network (FPN) and…” instead of just FPN.\n- W-8: Line 94: “In this study, we will…..” - it is recommended to use present tense, instead of future tense.\n- W-9: Table 1: Best values should be bolded.\n- Minor:\n  1. Line 32: “...detectors.Despite…, Line 138: “...alignment module(Low-FAM)...” - Spacing issue.\n  2. Line 257: Double comma issue.\n\nThanks for your suggestions, we will correct these formatting and grammar problems in the revision.\n\n\n\n### Q-1: Why need to use LAMB Optimizer while already using SGD and manual scheduled Learning rate?\n\nThank you for your question. We utilize the LAMB optimizer only for MIM pre-trained backbones, following the practices in SparK. For the rest of the training process and self-distillation, we adopt SGD.\n\n### Q-2: If you use EMA, then why again, LAMB is needed?\n\nThank you for your question. As with Q-1, we only followed SparK's design in pre-training, using the LAMB optimizer.\n\n### Limitations: Limitations of GD-YOLO not added; however, I would suggest adding it.\n\nThank you for your question. Limitations can be found in section C Broader impacts and limitations in the Appendix."}, {"review_id": "PRqizzNV8Y", "reviewer": "Reviewer_3APV", "summary": "In this paper, the authors proposed a Gather-and-Distribute mechanism (GD) for efficient information exchange in YOLOs by globally fusing multi-level features and injecting the global information into higher levels. The proposed GD-YOLO architectures show good results compared with the existing YOLO series. The authors also presented a pre-training method, where we pre-train the backbone on ImageNet 1K using the MAE method, which improves the convergence speed and accuracy of the model. ", "questions": "If the best trade-off between latency and accuracy is the main goal of this method, isn't fairer to compare with YOLOv8 instead of YOLOv6, since YOLOv8 has the closest latency with GDYolo? ", "limitations": "1. The idea is pretty interesting but the advantage of this proposed method doesn't show a very convincing result. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. In this paper, the authors proposed a gather-and-distribute mechanism to replace the traditional FPN structure. By using this unified module to gather and fuse information from all levels and subsequently distribute it to different levels, they can avoid the loss of information inherent in the traditional FPN structure and also enhance the neck’s partial information fusion capabilities without significantly increasing latency. \n2. The paper is clear and organized, and easy to follow.\n3. Authors have provided a comprehensive comparison between the proposed model and YOLOs. ", "weaknesses": "1. The motivation is a little bit unclear. The advantage of this GD mechanism compared with traditional FPN are not very clear.  \n2. Based on the results, the tradeoff between accuracy and latency is pretty similar compared with baselines (YOLOv8). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W-1: The motivation is a little bit unclear. The advantage of this GD mechanism compared with traditional FPN are not very clear.\n\nThank you for your suggestions. In traditional concepts, features at different levels contain positional information of different-sized objects. Larger features encompass more low-dimensional texture information and the positions of smaller objects, while smaller features contain more high-dimensional information and the positions of larger objects. The original motivation behind traditional Feature Pyramid Networks (FPN) is that although features at different levels contain different information, these pieces of information can assist each other and enhance the network's performance. Many previous works have addressed the issue of information loss in interactions between different levels. However, due to the excessive number of paths and indirect interaction methods in the network, the previous FPN-based fusion structures still have drawbacks in low speed, cross-level information exchange and information loss.\n\nHere we focus on the efficiency of information interaction fusion and the integrity of information retention. Beyond FPN-based fusion structures, we propose a new scheme, i.e., gather-and-distribute mechanism which is new information fusion structure for detection, not a revised version of FPN. The visualized feature maps of different-level Class Activation Maps (CAM) of Figure 3 can be found in the global response PDF.\n\nWe can observe that features at different levels exhibit distinct preferences for objects of different sizes. In the traditional grid-structured FPN, with increasing network depth and information interaction between different levels, the sensitivity of the feature map to object positions gradually diminishes, accompanied by information loss. Our proposed GD mechanism performs global fusion separately for high-level and low-level information, resulting in globally fused features that contain abundant position information for objects of various sizes. When injected into different branches, this not only enriches the information in each branch but also avoids information loss caused by \"recursive\" interaction.\n\n\n\n### W-2: Based on the results, the tradeoff between accuracy and latency is pretty similar compared with baselines (YOLOv8).\n\nThank you for your question. Our model shows significant performance improvements compared to previous works in the sizes of N/S/M (nano, small, medium), including YOLOv8. For YOLO-L, our GD-YOLO-L also exhibits performance improvements of +0/+9FPS and +0.6 AP/+1.1 AP compared to YOLOv6-3.0 and YOLOv8, respectively.\n\n\n\n### Q-1: If the best trade-off between latency and accuracy is the main goal of this method, isn't fairer to compare with YOLOv8 instead of YOLOv6, since YOLOv8 has the closest latency with GDYolo?\n\nThank you for your question. It's perfectly normal to have such confusion due to the naming inconsistencies in the YOLOv6 series, which has led to misunderstandings for many. The chronological order of model releases is as follows: YOLOv6, YOLOv7, YOLOv6-2.0, YOLOv8, YOLOv6-3.0. YOLOv6-3.0 was released after YOLOv8 and significantly improved model accuracy, allowing YOLOv6-3.0 to surpass YOLOv8 on the precision-speed curve. This result is also evident in our precision-speed curve graph.\n\n\n\n### L-1: The idea is pretty interesting but the advantage of this proposed method doesn't show a very convincing result.\n\nThank you for your question. Our core contribution lies in the proposal of the Gather-and-Distribute Mechanism (GD mechanism),  which is fast and effective.  Our method gathers the global information, and distribute this information into differtnt levels.  The GD mechanism is performed in both high resolution and low resolution to fully encourage information exchange.  By separately considering low-dimensional and high-dimensional information, we have constructed a global information fusion mechanism, thereby unifying the approaches for promoting information flow between different levels feature that were previously disparate.  Our proposed GD mechanism represents a more comprehensive improvement over previous SOTA detectors.\n\nThe GD mechanism is a general concept and can be applied beyond YOLOs.  We have extend GD mechanism to other models and obtain significant improvement as show in the Experiment-1, you can find in the Global Author Rebuttal.  The aforementioned experiments demonstrate that our proposed GD mechanism exhibits robust adaptability and generalization.  This mechanism consistently brings about performance improvements across different tasks and models."}]

Author rebuttal:
### Experiment-1:

- Instance Segmentation Task

  Replace different Necks in Mask R-CNN and train/test on the COCO instance dataset.

  | model| Neck|FPS|Bbox mAP | Bbox mAP:50 | Segm mAP | Segm mAP:50 |
  |-|-|-|-|-|-|-|
  | MaskRCNN-ResNet50| FPN| 21.6|38.2| 58.8| 34.7| 55.7|
  | MaskRCNN-ResNet50| AFPN| 19.1|36.0|53.6| 31.8| 50.7|
  | MaskRCNN-ResNet50|PAFPN | 20.2|37.9|58.6| 34.5| 55.3|
  | MaskRCNN-ResNet50| GD| 18.7| 40.7|59.5| 36.0|56.4|

- Semantic Segmentation Task

  Replace the Neck in PointRend and train/test on the Cityscapes dataset.

  | model|Neck|FPS| mIoU| mAcc|aAcc|
  | - |-|-|-|-|-|
  | PointRend-ResNet50|FPN|11.21| 76.47| 84.05| 95.96|
  | PointRend-ResNet50| GD|11.07| 78.54| 85.60| 96.12|
  | pointrend-ResNet101| FPN|8.76| 78.3| 85.705|96.23|
  | PointRend-ResNet101| GD|11.07|80.01|86.15|96.34|

- Performance of GD mechanism on other object detection models

  Replace necks in efficientdet and train/test on the COCO instance dataset.

  | model|Neck|FPS|AP|
  |-|-|-|-|
  | EfficientDet|BiFPN| 6.0|34.4 |
  | EfficientDet|GD|5.7|38.8|

### Experiment-2:

- FPN-like model comparison

  | Model|FPS| AP-val2017 |
  |-|-|-|
  | M2Det| 15.178| 37.8|
  | AFPN-R50-640x640 | 28.1| 39|
  | AFPN-R50-800x1000 | 23.8| 41|
  | AFPN-R101-800x1000 | 16.5| 42.3|
  | YOLOF| 36.63| 37.7|
  | YOLOF-R101| 25.38| 39.8|
  | YOLOF-X101| 10.88| 42.3|
  | CFPNet-S| 331| 41.1|
  | CFPNet-M| 165| 46.4|
  | CFPNet-L| 95| 49.4|
  | GD-YOLO-N| 684| 39.9|
  | GD-YOLO-S| 337| 46.4|
  | GD-YOLO-M| 177| 51.1|
  | GD-YOLO-L| 110| 53.3|

### Discuss-1: A exploration of the motivation behind GD, as well as in-depth discussions on FPN and multi-scale features.

Traditionally, features at different levels carry positional information about objects of various sizes. Larger features encompass low-dimensional texture details and positions of smaller objects. In contrast, smaller features contain high-dimensional information and positions of larger objects. The original idea behind Feature Pyramid Networks (FPN) is that these diverse pieces of information can enhance network performance through mutual assistance. Previous works have addressed the information loss problem when interacting between different levels. For instance, M2Det[1] introduced an efficient MLFPN architecture with U-shape and Feature Fusion Modules. Ping-Yang Chen[2] improved interaction between deep and shallow layers using bidirectional fusion modules. Unlike these inter-layer works, [3] explored individual feature information using the Centralized Feature Pyramid (CFP) method. Additionally, [4] extended FPN with the Asymptotic Feature Pyramid Network (AFPN) to interact across non-adjacent layers. In response to FPN's limitations in detecting large objects, [5] proposed a refined FPN structure. YOLO-F [6] achieved state-of-the-art performance with single-level features. SAFNet [7] introduced Adaptive Feature Fusion and Self-Enhanced Modules. [8] presented a parallel FPN structure for object detection with bi-directional fusion.

However, due to the excessive number of paths and indirect interaction methods in the network, the previous FPN-based fusion structures still have drawbacks in low speed, cross-level information exchange and information loss.

Here we focus on the efficiency of information interaction fusion and the integrity of information retention. Beyond FPN-based fusion structures, we propose a new scheme, i.e., gather-and-distribute mechanism which is new information fusion structure for detection, not a revised version of FPN. As shown in the visualized feature maps of different-level Class Activation Maps (CAM) in the appendix.

We can observe that features at different levels exhibit distinct preferences for objects of different sizes. In the traditional grid-structured FPN, with increasing network depth and information interaction between different levels, the sensitivity of the feature map to object positions gradually diminishes, accompanied by information loss. Our proposed GD mechanism performs global fusion separately for high-level and low-level information, resulting in globally fused features that contain abundant position information for objects of various sizes. When injected into different branches, this not only enriches the information in each branch but also avoids information loss caused by "recursive" interaction.

### Discuss-2: Differences in the feature alignment module between GD-YOLO and other similar works

Both M2Det [1] and RHF-Net [1] have incorporated additional information fusion modules within their alignment modules. In M2Det, the SFAM module includes an SE block, while in RHF-Net, the Spatial Pyramid Pooling block is augmented with a Bottleneck layer. In contrast to M2Det and RHF-Net, GD-YOLO leans towards functional separation among modules, segregating feature alignment and feature fusion into distinct modules. Specifically, the FAM module in GD-YOLO focuses solely on feature alignment. This ensures computational efficiency within the FAM module. And the LAF module efficiently merges features from various levels with minimal computational cost, leaving a greater portion of fusion and injection functionalities to other modules. 

Based on the GD mechanism, achieving SOTA performance for the YOLO model can be accomplished using simple and easily accessible operators. This strongly demonstrates the effectiveness of the approach we propose. Additionally, during the network construction process, we intentionally opted for simple and thoroughly validated structures. This choice serves to prevent potential development and performance issues arising from certain operators being unsupported by deployment devices. As a result, it guarantees the usability and portability of the entire mechanism. 

Meta-review: This work proposes an efficient object detection method, named GD-YOLO. The authors introduce a Gather-and-Distribute mechanism to boost the multi-scale feature fusion capability. Extensive experiments show the great performance of this method. The reviewers acknowledge its outstanding performance but also point out the lack of novelty (by Reviewer CoJm and YJPg), and unclear motivation (by Reviewer c8e6 and 3APV). After the rebuttal, the concerns of Reviewer CoJm are addressed and he/she raises the rating to “Borderline Accept”. Considering its scores (2 Weak Accept, 2 Borderline accept, and 1 Borderline reject), I lean towards acceptance. I also suggest the authors revise the paper according to the reviewers' suggestions and further clarify the motivation of the “Gather-and-Distribute mechanism”.
Decision: Accept (poster)