Original: Effectiveness of Vision Transformer for Fast and Accurate Single-Stage Pedestrian Detection
Revised: Effectiveness of Vision Transformer for Fast and Accurate Single-Stage Pedestrian Detection

Original: Abstract
Revised: Abstract

Original: Vision transformers have demonstrated remarkable performance on a variety of computer vision tasks.
Revised: Vision transformers have demonstrated remarkable performance on a variety of computer vision tasks.

Original: In this paper, we illustrate the effectiveness of the deformable vision transformer for single-stage pedestrian detection and propose a spatial and multi-scale feature enhancement module, which aims to achieve the optimal balance between speed and accuracy.
Revised: In this paper, we illustrate the effectiveness of the deformable vision transformer for single-stage pedestrian detection and propose a spatial and multi-scale feature enhancement module, which aims to achieve the optimal balance between speed and accuracy.

Original: Performance improvement with the use of vision transformers on various commonly used single-stage structures is demonstrated.
Revised: Performance improvement with the use of vision transformers on various commonly used single-stage structures is demonstrated.

Original: The design of the proposed architecture is investigated in depth.
Revised: The design of the proposed architecture is investigated in depth.

Original: Comprehensive comparisons with state-of-the-art single- and two-stage detectors on various pedestrian datasets are performed.
Revised: Comprehensive comparisons with state-of-the-art single- and two-stage detectors on various pedestrian datasets are performed.

Original: The proposed detector achieves leading performance on Caltech and Citypersons datasets among single- and two-stage methods using fewer parameters than the baseline.
Revised: The proposed detector achieves leading performance on Caltech and Citypersons datasets among single- and two-stage methods using fewer parameters than the baseline.

Original: The log-average miss rates for Reasonable and Heavy are decreased to 2.6% and 28.0% on the Caltech test set, and 10.9% and 38.6% on the Citypersons validation set respectively.
Revised: The log-average miss rates for Reasonable and Heavy are decreased to 2.6% and 28.0% on the Caltech test set, and 10.9% and 38.6% on the Citypersons validation set respectively.

Original: The proposed method outperforms SOTA two-stage detectors in the Heavy subset on the Citypersons validation set with considerably faster inference speed.
Revised: The proposed method outperforms SOTA two-stage detectors in the Heavy subset on the Citypersons validation set with considerably faster inference speed.

Original:  Introduction
Revised:  Introduction

Original: Pedestrian detection is a popular task subordinate to object detection in computer vision.
Revised: Pedestrian detection is a popular task subordinate to object detection in computer vision.

Original: This task aims towards accurately locating and classifying pedestrians in images or videos obtained via visiblelight cameras, infrared cameras etc.
Revised: This task aims towards accurately locating and classifying pedestrians in images or videos obtained via visiblelight cameras, infrared cameras etc.

Original: In this paper, we focus on the detection based on RGB images.
Revised: In this paper, we focus on the detection based on RGB images.

Original: Pedestrian detection is very important as it serves as the prerequisite of various vision tasks [1], such as human-centric tasks (person re-identification [2, 3], person search [4], human pose estimation [5] etc.) and more generic multi-object tracking [6].
Revised: Pedestrian detection is very important as it serves as the prerequisite of various vision tasks [1], such as human-centric tasks (person re-identification [2, 3], person search [4], human pose estimation [5] etc.) and more generic multi-object tracking [6].

Original: It has been applied to autonomous driving [7, 8], video surveillance [9] and action tracking.
Revised: It has been applied to autonomous driving [7, 8], video surveillance [9] and action tracking.

Original: Pedestrian detection suffers from significant occlusion and varying scales.
Revised: Pedestrian detection suffers from significant occlusion and varying scales.

Original: Intra- and inter-class occlusion occur when a pedestrian is occluded by other pedestrians or objects like cars, bicycles etc.
Revised: Intra- and inter-class occlusion occur when a pedestrian is occluded by other pedestrians or objects like cars, bicycles etc.

Original: Both significantly reduce the discriminative features and destroy the regular shape of pedestrians.
Revised: Both significantly reduce the discriminative features and destroy the regular shape of pedestrians.

Original: For varying scales, large pedestrians tend to have more informative features but are difficult to be fully extracted from a vast region while features of small targets are compact but relatively ambiguous with less preserved details.
Revised: For varying scales, large pedestrians tend to have more informative features but are difficult to be fully extracted from a vast region while features of small targets are compact but relatively ambiguous with less preserved details.

Original: In summary, the fluctuating amount and varying shape of effective features are the core problems.
Revised: In summary, the fluctuating amount and varying shape of effective features are the core problems.

Original: They challenge the capabilities of feature extraction modules, which act as a long-standing bottleneck in pedestrian detection.
Revised: They challenge the capabilities of feature extraction modules, which act as a long-standing bottleneck in pedestrian detection.

Original: To deal with these problems, two-stage methods [10–15] based on Fast [16] and Faster R-CNN [16] have pervaded pedestrian detection tasks owing to the high detection accuracy.
Revised: To deal with these problems, two-stage methods [10–15] based on Fast [16] and Faster R-CNN [16] have pervaded pedestrian detection tasks owing to the high detection accuracy.

Original: These methods first make coarse predictions of targets via the Region Proposal Network (RPN), then refine the bounding boxes and predict the final scores based on the features inside the proposal regions.
Revised: These methods first make coarse predictions of targets via the Region Proposal Network (RPN), then refine the bounding boxes and predict the final scores based on the features inside the proposal regions.

Original: In addition to methods for general object detection, pedestrian detectors take the advantage of unique characteristics of targets, such as the mask of visible parts [17, 10] and key points of human bodies [18].
Revised: In addition to methods for general object detection, pedestrian detectors take the advantage of unique characteristics of targets, such as the mask of visible parts [17, 10] and key points of human bodies [18].

Original: However, the inference speed of these methods is limited by the repeated predictions which makes them hard to be applied to real-world scenes.
Revised: However, the inference speed of these methods is limited by the repeated predictions which makes them hard to be applied to real-world scenes.

Original: To achieve faster inference, single-stage approaches, which only make one round prediction, are developed [19] and applied to pedestrian detection [20–24].
Revised: To achieve faster inference, single-stage approaches, which only make one round prediction, are developed [19] and applied to pedestrian detection [20–24].

Original: However, they suffer from decreased detection accuracy.
Revised: However, they suffer from decreased detection accuracy.

Original: The miss rates of two-stage methods in the Reasonable subset on Caltech are reduced to less than 4% [14, 10, 25, 18], while those for single-stage methods are larger than 4.5%
Revised: The miss rates of two-stage methods in the Reasonable subset on Caltech are reduced to less than 4% [14, 10, 25, 18], while those for single-stage methods are larger than 4.5%

Original: For the Citypersons validation set, the miss rates for the former are less than 40% in the Heavy subset, which is much lower than the latter (42% [24]).
Revised: For the Citypersons validation set, the miss rates for the former are less than 40% in the Heavy subset, which is much lower than the latter (42% [24]).

Original: In this paper, our aim is to improve the detection accuracy, especially in Reasonable and Heavy subsets, and to narrow the gap between single- and two-stage methods with fast inference.
Revised: In this paper, our aim is to improve the detection accuracy, especially in Reasonable and Heavy subsets, and to narrow the gap between single- and two-stage methods with fast inference.

Original: Typical single-stage detectors use anchors (SSD [26]) or are anchor-free (CSP [22]).
Revised: Typical single-stage detectors use anchors (SSD [26]) or are anchor-free (CSP [22]).

Original: The former generates rectangular bounding boxes with different aspect ratios and scales centered at each pixel of the feature maps at certain levels.
Revised: The former generates rectangular bounding boxes with different aspect ratios and scales centered at each pixel of the feature maps at certain levels.

Original: Anchor scales are designed smaller at lower levels to facilitate the detection of small objects.
Revised: Anchor scales are designed smaller at lower levels to facilitate the detection of small objects.

Original: These methods predict the offsets w.r.t.
Revised: These methods predict the offsets w.r.t.

Original: the upper left position, height and width of the anchor.
Revised: the upper left position, height and width of the anchor.

Original: The latter method, taking CSP as an example, only predicts the logarithm height and offsets w.r.t.
Revised: The latter method, taking CSP as an example, only predicts the logarithm height and offsets w.r.t.

Original: centers of each pixel.
Revised: centers of each pixel.

Original: For anchor-based single-stage detectors, ALFNet [21] refines anchors progressively with stacked prediction blocks to remedy the lack of proposal regions.
Revised: For anchor-based single-stage detectors, ALFNet [21] refines anchors progressively with stacked prediction blocks to remedy the lack of proposal regions.

Original: In the past year, most research focuses on the fusion of representative features [27–32] to improve single-stage methods.
Revised: In the past year, most research focuses on the fusion of representative features [27–32] to improve single-stage methods.

Original: For example, [30] enhances features via increasing semantic information at a low level and enriches the localization information at a high level.
Revised: For example, [30] enhances features via increasing semantic information at a low level and enriches the localization information at a high level.

Original: Similarly, [32] fuses the feature maps with different scales in adequate proportions.
Revised: Similarly, [32] fuses the feature maps with different scales in adequate proportions.

Original: [29, 31] also explore new strategies to aggregate multi-level features.
Revised: [29, 31] also explore new strategies to aggregate multi-level features.

Original: These feature enhancements are mainly performed along the dimension of feature level due to the intrinsically unbalanced feature information between shallow and deep feature levels.
Revised: These feature enhancements are mainly performed along the dimension of feature level due to the intrinsically unbalanced feature information between shallow and deep feature levels.

Original: However, this unbalance also exists in two-stage detectors.
Revised: However, this unbalance also exists in two-stage detectors.

Original: As such, this is not the particular reason for the poorer accuracy of single-stage methods.
Revised: As such, this is not the particular reason for the poorer accuracy of single-stage methods.

Original: The general architecture of single- and two-stage detectors are compared in Figure 1.
Revised: The general architecture of single- and two-stage detectors are compared in Figure 1.

Original: Assuming that the training strategies and the detection head are the same for both methods, the difference in the architecture lies in the information fed into the detection head.
Revised: Assuming that the training strategies and the detection head are the same for both methods, the difference in the architecture lies in the information fed into the detection head.

Original: For two-stage methods, both positions and features of the proposal regions are fed into the detection head.
Revised: For two-stage methods, both positions and features of the proposal regions are fed into the detection head.

Original: These proposals contain potential pedestrians.
Revised: These proposals contain potential pedestrians.

Original: Thereby, the detection head classifies spatially target-focused features with fewer background interruptions and refines the bounding boxes by predicting small offsets from the proposal positions.
Revised: Thereby, the detection head classifies spatially target-focused features with fewer background interruptions and refines the bounding boxes by predicting small offsets from the proposal positions.

Original: For single-stage methods, each pixel in the feature map serves as the ‘proposal region’ with no pre-estimated positions.
Revised: For single-stage methods, each pixel in the feature map serves as the ‘proposal region’ with no pre-estimated positions.

Original: These receptive fields of these pixels share the same size, which may be too small to include sufficient information for large targets or so large that the background information overwhelms the useful features.
Revised: These receptive fields of these pixels share the same size, which may be too small to include sufficient information for large targets or so large that the background information overwhelms the useful features.

Original: This is more challenging for the classifier compared to two-stage methods.
Revised: This is more challenging for the classifier compared to two-stage methods.

Original: Additionally, single-stage methods have to regress from scratch, which is more difficult than simple refinement.
Revised: Additionally, single-stage methods have to regress from scratch, which is more difficult than simple refinement.

Original: Thus, the lack of spatially target-focused feature representation and the prediction of bounding boxes from scratch are the two key bottlenecks hindering the improvement of single-stage detectors.
Revised: Thus, the lack of spatially target-focused feature representation and the prediction of bounding boxes from scratch are the two key bottlenecks hindering the improvement of single-stage detectors.

Original: To make the features fed into the detection head concentrate on the targets or other useful information automatically without the assistance of proposal regions, in this paper we take the advantage of vision transformers.
Revised: To make the features fed into the detection head concentrate on the targets or other useful information automatically without the assistance of proposal regions, in this paper we take the advantage of vision transformers.

Original: Vision transformers describe the pairwise dependency of each entity in the feature map with attention weights.
Revised: Vision transformers describe the pairwise dependency of each entity in the feature map with attention weights.

Original: The output weighted averaged feature is the adaptive aggregation of important entities (with higher attention weights) while the disturbing information (with lower attention weights) is suppressed.
Revised: The output weighted averaged feature is the adaptive aggregation of important entities (with higher attention weights) while the disturbing information (with lower attention weights) is suppressed.

Original: Using such attention mechanism on top of the backbone enables the single-stage detector to supplement spatially filtered features easily for subsequent classification and regression.
Revised: Using such attention mechanism on top of the backbone enables the single-stage detector to supplement spatially filtered features easily for subsequent classification and regression.

Original: In this case, the modified detector makes the best use of the fast inference originating from single-stage methods and more effective features.
Revised: In this case, the modified detector makes the best use of the fast inference originating from single-stage methods and more effective features.

Original: Our main contributions are as follows
Revised: Our main contributions are as follows

Original: : • Demonstrate the effectiveness of the deformable vision transformer in improving the accuracy of commonly used single-stage detectors on pedestrian datasets.
Revised: : • Demonstrate the effectiveness of the deformable vision transformer in improving the accuracy of commonly used single-stage detectors on pedestrian datasets.

Original: • Extend the application of vision transformers on top of the backbone in pedestrian detection tasks.
Revised: • Extend the application of vision transformers on top of the backbone in pedestrian detection tasks.

Original: • Achieve the best performance among single-stage detectors on the Caltech test set and Citypersons validation set while maintaining fast inference and reducing the number of parameters.
Revised: • Achieve the best performance among single-stage detectors on the Caltech test set and Citypersons validation set while maintaining fast inference and reducing the number of parameters.

Original: • Narrow the gap of detection accuracy between the single- and two-stage methods in pedestrian detection.
Revised: • Narrow the gap of detection accuracy between the single- and two-stage methods in pedestrian detection.

Original:  Related works
Revised:  Related works

Original: Currently, vision transformers are used to establish general-purpose backbones (ViT [33] and Swin
Revised: Currently, vision transformers are used to establish general-purpose backbones (ViT [33] and Swin

Original: Transformer [34]) or stack on top of the backbone (DETR [35]).
Revised: Transformer [34]) or stack on top of the backbone (DETR [35]).

Original: Considering that we focus on pedestrian detection, this paper explores the latter case.
Revised: Considering that we focus on pedestrian detection, this paper explores the latter case.

Original: DETR consists of the convolution backbone, six encoders and decoders and the prediction head.
Revised: DETR consists of the convolution backbone, six encoders and decoders and the prediction head.

Original: It is an inspiring end-to-end detector but is memory-consuming.
Revised: It is an inspiring end-to-end detector but is memory-consuming.

Original: DETR requires massive memory to store the self-attention weights within each
Revised: DETR requires massive memory to store the self-attention weights within each

Original: Multi-Head Self-Attention (MHSA) layer.
Revised: Multi-Head Self-Attention (MHSA) layer.

Original: The memory cost is linear to the number of attention heads and is square to the number of pixels in the down-sampled feature map.
Revised: The memory cost is linear to the number of attention heads and is square to the number of pixels in the down-sampled feature map.

Original: Additionally, first and second order momentums in optimization introduce further memory cost in the training procedure.
Revised: Additionally, first and second order momentums in optimization introduce further memory cost in the training procedure.

Original: In pedestrian detection, more attention heads and relatively large down-sampled feature maps are preferred to enhance detection accuracy, especially for small targets.
Revised: In pedestrian detection, more attention heads and relatively large down-sampled feature maps are preferred to enhance detection accuracy, especially for small targets.

Original: This results in memory explosion using DETR.
Revised: This results in memory explosion using DETR.

Original: To this end, deformable DETR [36] is proposed.
Revised: To this end, deformable DETR [36] is proposed.

Original: It only needs the attention weights at several sampling locations rather than each pixel in the feature map.
Revised: It only needs the attention weights at several sampling locations rather than each pixel in the feature map.

Original: The memory cost of the attention weights is linear to the number of pixels, which makes training with high resolutions possible.
Revised: The memory cost of the attention weights is linear to the number of pixels, which makes training with high resolutions possible.

Original: Experiments show that the deformable DETR outperforms the Faster R-CNN [37] and
Revised: Experiments show that the deformable DETR outperforms the Faster R-CNN [37] and

Original: DETR on COCO 2017 validation set [38].
Revised: DETR on COCO 2017 validation set [38].

Original: So far, vision transformers show great potential, but they have rarely been applied to pedestrian detection in the form of DETR or its variants.
Revised: So far, vision transformers show great potential, but they have rarely been applied to pedestrian detection in the form of DETR or its variants.

Original: This is because it has been observed that they perform worse than commonly used Faster R-CNN on CrowdHuman dataset [39] and require tenfold training time [40].
Revised: This is because it has been observed that they perform worse than commonly used Faster R-CNN on CrowdHuman dataset [39] and require tenfold training time [40].

Original: Although [40] proposed the use of dense object queries and the rectified attention field to enhance scale-adaptive feature extraction in the decoding phase, the modified deformable DETR still shows a limited advantage over the traditional Faster R-CNN.
Revised: Although [40] proposed the use of dense object queries and the rectified attention field to enhance scale-adaptive feature extraction in the decoding phase, the modified deformable DETR still shows a limited advantage over the traditional Faster R-CNN.

Original: This implies that rigidly putting the whole six encoder-decoder pairs into the pedestrian detector may not be cost-effective.
Revised: This implies that rigidly putting the whole six encoder-decoder pairs into the pedestrian detector may not be cost-effective.

Original: As an example, BoTNet [41] only substitutes the convolution layers in residual bottleneck blocks in the last stage of ResNet [42] for MHSA layers, but it produces strong performance on ImageNet validation set [43].
Revised: As an example, BoTNet [41] only substitutes the convolution layers in residual bottleneck blocks in the last stage of ResNet [42] for MHSA layers, but it produces strong performance on ImageNet validation set [43].

Original: Inspired by this and the aforementioned limitation, our work only utilizes a single encoder of the deformable vision transformer as an adaptive feature extractor and applies it to commonly used single-stage detectors for better detection accuracy and fast inference.
Revised: Inspired by this and the aforementioned limitation, our work only utilizes a single encoder of the deformable vision transformer as an adaptive feature extractor and applies it to commonly used single-stage detectors for better detection accuracy and fast inference.

Original:  Method
Revised:  Method

Original: Deformable Vision Transformer Encoder :
Revised: Deformable Vision Transformer Encoder :

Original: The deformable vision transformer encoder (Figure 2) takes the L feature maps (cid:8) z l (cid:9)
Revised: The deformable vision transformer encoder (Figure 2) takes the L feature maps (cid:8) z l (cid:9)

Original: L l =1 with height H l and width W l at scale l and reference points, which are the positions of grid centers of the feature maps, as inputs.
Revised: L l =1 with height H l and width W l at scale l and reference points, which are the positions of grid centers of the feature maps, as inputs.

Original: It outputs the enhanced feature maps with the same resolution as the input.
Revised: It outputs the enhanced feature maps with the same resolution as the input.

Original: The input feature maps are first added by fixed encoded positional [35] and learnable level information [36] to disambiguate spatial and scale positions, then projected to the query feature map z q via a linear layer.
Revised: The input feature maps are first added by fixed encoded positional [35] and learnable level information [36] to disambiguate spatial and scale positions, then projected to the query feature map z q via a linear layer.

Original: The feature maps also generate the value feature maps z v with a linear layer but without encoding.
Revised: The feature maps also generate the value feature maps z v with a linear layer but without encoding.

Original: The query feature map z q , value feature map z v together with the pre-generated reference points are sent to the Multi-Scale Deformable
Revised: The query feature map z q , value feature map z v together with the pre-generated reference points are sent to the Multi-Scale Deformable

Original: Attention (MSDA) layer to enhance spatially adaptive features, followed by a Feed-Forward Network (FFN).
Revised: Attention (MSDA) layer to enhance spatially adaptive features, followed by a Feed-Forward Network (FFN).

Original: In summary, the encoder supplements the semantic information of input feature maps via the embedded attention layer and FFN.
Revised: In summary, the encoder supplements the semantic information of input feature maps via the embedded attention layer and FFN.

Original: Multi-Scale Deformable Attention : The MSDA layer sums the selected entities at sampling locations in the value feature map z v with predicted attention weights by each corresponding query entity.
Revised: Multi-Scale Deformable Attention : The MSDA layer sums the selected entities at sampling locations in the value feature map z v with predicted attention weights by each corresponding query entity.

Original: These attention weights W are the linear projection of query features z q followed by a softmax operator along scale and sampling point dimensions.
Revised: These attention weights W are the linear projection of query features z q followed by a softmax operator along scale and sampling point dimensions.

Original: For a single query entity, it only needs N h N l N p attention weights, representing the significance of selected value features at different attention heads, scales, and points.
Revised: For a single query entity, it only needs N h N l N p attention weights, representing the significance of selected value features at different attention heads, scales, and points.

Original: Selections are decided by the sampling locations which are the summation of reference points p and sampling offsets ∆ p which is the embedding of the query features.
Revised: Selections are decided by the sampling locations which are the summation of reference points p and sampling offsets ∆ p which is the embedding of the query features.

Original: At each float sampling location, the selected value feature is bilinear interpolated for accuracy and training offset predictor.
Revised: At each float sampling location, the selected value feature is bilinear interpolated for accuracy and training offset predictor.

Original: With weight, sampling locations and selected value features prepared, the q -th element of the separate output feature z
Revised: With weight, sampling locations and selected value features prepared, the q -th element of the separate output feature z

Original: so ,h ∈ R N q × c v ( N q = (cid:80) Ll =1 H l W l ) at attention head h
Revised: so ,h ∈ R N q × c v ( N q = (cid:80) Ll =1 H l W l ) at attention head h

Original: (total
Revised: (total

Original: N h heads) is z
Revised: N h heads) is z

Original: so ,h q =
Revised: so ,h q =

Original: N p (cid:88) p =
Revised: N p (cid:88) p =

Original: [Equation]
Revised: [Equation]

Original: where p , q , l and h index the sampling offsets, elements of the deformable attention feature z o , the scale of value v and attention head.
Revised: where p , q , l and h index the sampling offsets, elements of the deformable attention feature z o , the scale of value v and attention head.

Original: W plhq is a value from the weight W ∈ R N q × N h
Revised: W plhq is a value from the weight W ∈ R N q × N h

Original: × L × N p .
Revised: × L × N p .

Original: p ∈ R N q × L × 2 and ∆ p ∈ R N q × N h
Revised: p ∈ R N q × L × 2 and ∆ p ∈ R N q × N h

Original: × L × N p × 2 are the reference points and sampling offsets.
Revised: × L × N p × 2 are the reference points and sampling offsets.

Original: p ql and ∆ p plhq denote the position of a single reference point and one of its corresponding sampling offsets respectively.
Revised: p ql and ∆ p plhq denote the position of a single reference point and one of its corresponding sampling offsets respectively.

Original: The separate output features from N h attention heads are projected to the q -th element of the final output deformable attention feature
Revised: The separate output features from N h attention heads are projected to the q -th element of the final output deformable attention feature

Original: z
Revised: z

Original: o by a linear layer expressed as
Revised: o by a linear layer expressed as

Original: [Equation]
Revised: [Equation]

Original: where W ′ h ∈ R c
Revised: where W ′ h ∈ R c

Original: ×
Revised: ×

Original: c
Revised: c

Original: v denotes the learnable weight for the h -th attention head.
Revised: v denotes the learnable weight for the h -th attention head.

Original: Proposed Feature Enhancement Module :
Revised: Proposed Feature Enhancement Module :

Original: The features are enhanced owing to the self-attention mechanism to supplement the spatially adaptive features across multiple scales.
Revised: The features are enhanced owing to the self-attention mechanism to supplement the spatially adaptive features across multiple scales.

Original: The proposed module simply consists of convolution/deconvolution and normalization layer pairs ahead of and after the deformable encoder, and a final feature fusion step 3.
Revised: The proposed module simply consists of convolution/deconvolution and normalization layer pairs ahead of and after the deformable encoder, and a final feature fusion step 3.

Original: In this module, input feature maps from the backbone are first upsampled with deconvolution layers or encoded with the convolution layer to generate multi-scale feature maps (cid:8) z l (cid:9) 3 l =1 .
Revised: In this module, input feature maps from the backbone are first upsampled with deconvolution layers or encoded with the convolution layer to generate multi-scale feature maps (cid:8) z l (cid:9) 3 l =1 .

Original: They are followed by group normalization to prevent the Internal Covariate Shift (ICS) that might be inducted by subsequent linear operations in the deformable encoder.
Revised: They are followed by group normalization to prevent the Internal Covariate Shift (ICS) that might be inducted by subsequent linear operations in the deformable encoder.

Original: The encoder yields enhanced multi-scale feature maps.
Revised: The encoder yields enhanced multi-scale feature maps.

Original: Enhanced features are upsampled to keep the resolution as ( H/ 4 , W/ 4) for accurate detection.
Revised: Enhanced features are upsampled to keep the resolution as ( H/ 4 , W/ 4) for accurate detection.

Original: They are normalized via
Revised: They are normalized via

Original: L2Norm [22] before concatenation along the channel dimension.
Revised: L2Norm [22] before concatenation along the channel dimension.

Original: This makes the features at different scales contribute equally to the final feature representation fed into the detection head.
Revised: This makes the features at different scales contribute equally to the final feature representation fed into the detection head.

Original: Concatenated features are compressed along the channel dimension to reduce the network parameters.
Revised: Concatenated features are compressed along the channel dimension to reduce the network parameters.

Original: The output feature maps can be fed into the detection head used in SSD, CSP etc.
Revised: The output feature maps can be fed into the detection head used in SSD, CSP etc.

Original: Except for this standard structure, the use of convolution/deconvolution layers can be adjusted according to the resolution of input, and the concatenation step can be removed if predictions are made at separate levels.
Revised: Except for this standard structure, the use of convolution/deconvolution layers can be adjusted according to the resolution of input, and the concatenation step can be removed if predictions are made at separate levels.

Original: Training :
Revised: Training :

Original: For anchor-free cases, namely CSP, the loss function follows [22].
Revised: For anchor-free cases, namely CSP, the loss function follows [22].

Original: The overall loss consists of three parts as Equation(3) where L c , L h and L o stand for the center heatmap loss, height map loss and offset loss.
Revised: The overall loss consists of three parts as Equation(3) where L c , L h and L o stand for the center heatmap loss, height map loss and offset loss.

Original: Weights for each loss λ c , λ s and λ o are set as 0.01, 1 and 0.1 [22].
Revised: Weights for each loss λ c , λ s and λ o are set as 0.01, 1 and 0.1 [22].

Original: For anchor-based cases, namely SSD or ALFNet, the multi-task loss function is formulated with two objectives as
Revised: For anchor-based cases, namely SSD or ALFNet, the multi-task loss function is formulated with two objectives as

Original: Equation(4)
Revised: Equation(4)

Original: [21] where λ cls is experimentally set as 0.01 in the following experiments.
Revised: [21] where λ cls is experimentally set as 0.01 in the following experiments.

Original: [Equation]
Revised: [Equation]

Original: Inference : For anchor-free single-stage methods, the predicted width is the height multiplied by the uniform aspect ratio 0.41 [22].
Revised: Inference : For anchor-free single-stage methods, the predicted width is the height multiplied by the uniform aspect ratio 0.41 [22].

Original: If not specified, bounding boxes with scores above 0.01 are kept and merged by Non-Maximum Suppression (NMS) with the IoU threshold of 0.5.
Revised: If not specified, bounding boxes with scores above 0.01 are kept and merged by Non-Maximum Suppression (NMS) with the IoU threshold of 0.5.

Original:  Experiments 4.1 Settings
Revised:  Experiments 4.1 Settings

Original: Datasets : The proposed detector is evaluated on two commonly used public pedestrian datasets:
Revised: Datasets : The proposed detector is evaluated on two commonly used public pedestrian datasets:

Original: Caltech [44] and Citypersons [45].
Revised: Caltech [44] and Citypersons [45].

Original: The Caltech dataset is an approximately 10 hours of 480x video taken in a single urban city.
Revised: The Caltech dataset is an approximately 10 hours of 480x video taken in a single urban city.

Original: The standard training set contains about 250k frames with 350k bounding boxes.
Revised: The standard training set contains about 250k frames with 350k bounding boxes.

Original: In our experiments, the training data augmented by 10 folds containingimages with 13674 persons and the standard test set containing 4024 images with corresponding new annotations [46] and fixed aspect-ratio for bounding boxes [47] are used.
Revised: In our experiments, the training data augmented by 10 folds containingimages with 13674 persons and the standard test set containing 4024 images with corresponding new annotations [46] and fixed aspect-ratio for bounding boxes [47] are used.

Original: Citypersons training set recorded across 18 different cities, 3 seasons and various weather conditions with 19654 persons in 2975 high resolution (1024x2048) images.
Revised: Citypersons training set recorded across 18 different cities, 3 seasons and various weather conditions with 19654 persons in 2975 high resolution (1024x2048) images.

Original: The validation set contains 500 images across 3 cities.
Revised: The validation set contains 500 images across 3 cities.

Original: Training Details : If not specified, the ResNet50/VGG16 pre-trained on ImageNet, Adam with moving average weights [48] and step learning rate schedule are applied.
Revised: Training Details : If not specified, the ResNet50/VGG16 pre-trained on ImageNet, Adam with moving average weights [48] and step learning rate schedule are applied.

Original: Data augmentation techniques including random horizontal flip with a probability of 0.5 and scaling are applied.
Revised: Data augmentation techniques including random horizontal flip with a probability of 0.5 and scaling are applied.

Original: For Caltech, additional random color distortion and cropping are implemented.
Revised: For Caltech, additional random color distortion and cropping are implemented.

Original: The input images are rescaled to 336x448 and 640x1280 for Caltech and Citypersons datasets respectively.
Revised: The input images are rescaled to 336x448 and 640x1280 for Caltech and Citypersons datasets respectively.

Original: The detectors are trained with a single NVIDIA GeForce RTX 3090 GPU for 10 and 75 epochs with batch size 16 and 4 on
Revised: The detectors are trained with a single NVIDIA GeForce RTX 3090 GPU for 10 and 75 epochs with batch size 16 and 4 on

Original: Caltech and Citypersons respectively using the anchor-free CSP detection head.
Revised: Caltech and Citypersons respectively using the anchor-free CSP detection head.

Original: The base learning rate is 0.5e-4 and decreased by a factor of 0.5 after 6 and 60 epochs.
Revised: The base learning rate is 0.5e-4 and decreased by a factor of 0.5 after 6 and 60 epochs.

Original: Initialization is performed with a randomly chosen and fixed seed.
Revised: Initialization is performed with a randomly chosen and fixed seed.

Original: Tools provided by [47] are used in the experiments.
Revised: Tools provided by [47] are used in the experiments.

Original: Metrics : Log-average Miss Rate (denoted as MR − 2 or miss rate in this paper) over False Positive
Revised: Metrics : Log-average Miss Rate (denoted as MR − 2 or miss rate in this paper) over False Positive

Original: Per Image (FPPI) in the range [10 − 2 , 10 0 ] is calculated over Reasonable, Small, Heavy, and All subsets defined in Table 1.
Revised: Per Image (FPPI) in the range [10 − 2 , 10 0 ] is calculated over Reasonable, Small, Heavy, and All subsets defined in Table 1.

Original: The lower the MR − 2 the better.
Revised: The lower the MR − 2 the better.

Original: [Table]
Revised: [Table]

Original: 4.2 Ablation Experiments
Revised: 4.2 Ablation Experiments

Original: Effectiveness of Enhanced Feature Maps : For generality, the proposed module is applied to three commonly used single-stage structures in pedestrian detection: anchor-based, progressive refinement, and anchor free.
Revised: Effectiveness of Enhanced Feature Maps : For generality, the proposed module is applied to three commonly used single-stage structures in pedestrian detection: anchor-based, progressive refinement, and anchor free.

Original: For anchor-based single-stage methods, like SSD300 [26], Table 2 shows that the proposed module shows stable and significant improvement in all three subsets with an IoU threshold of 0.5 in NMS.
Revised: For anchor-based single-stage methods, like SSD300 [26], Table 2 shows that the proposed module shows stable and significant improvement in all three subsets with an IoU threshold of 0.5 in NMS.

Original: For refined anchors, we append two Convolutional Predictor Blocks [21] after the feature maps at the last three stages of the backbone and an extra stage.
Revised: For refined anchors, we append two Convolutional Predictor Blocks [21] after the feature maps at the last three stages of the backbone and an extra stage.

Original: The second block refines the coarse anchors predicted by the first block.
Revised: The second block refines the coarse anchors predicted by the first block.

Original: Table 3 demonstrates that even the anchors are refined progressively to remedy the lack of proposal regions, the separate enhancement at each level can bring improvement in certain subsets (level 0 improves Reasonable and All subsets, level 1, 2 improves Heavy subset).
Revised: Table 3 demonstrates that even the anchors are refined progressively to remedy the lack of proposal regions, the separate enhancement at each level can bring improvement in certain subsets (level 0 improves Reasonable and All subsets, level 1, 2 improves Heavy subset).

Original: The combination of multi-level inputs brings in stable improvement in all the subsets.
Revised: The combination of multi-level inputs brings in stable improvement in all the subsets.

Original: For anchor-free methods, We evaluated the influence of the proposed module on the baseline CSP [22] detector on Caltech (Table 4) and Citypersons (Table 5).
Revised: For anchor-free methods, We evaluated the influence of the proposed module on the baseline CSP [22] detector on Caltech (Table 4) and Citypersons (Table 5).

Original: Results from both datasets indicate that with the enhanced feature maps, the miss rates are decreased significantly in Reasonable, Heavy and
Revised: Results from both datasets indicate that with the enhanced feature maps, the miss rates are decreased significantly in Reasonable, Heavy and

Original: All subsets, in particular, the Heavy subset witnesses a decrease of up to 7.9%, and the Reasonable subset up to 3.1%.
Revised: All subsets, in particular, the Heavy subset witnesses a decrease of up to 7.9%, and the Reasonable subset up to 3.1%.

Original: The additional enhancement module increases the inference time, however, they contain fewer learnable parameters as shown in Table 6.
Revised: The additional enhancement module increases the inference time, however, they contain fewer learnable parameters as shown in Table 6.

Original: As the combination of CSP achieves the best results, subsequent experiments follow this implementation.
Revised: As the combination of CSP achieves the best results, subsequent experiments follow this implementation.

Original: According to the above, the proposed module is effective for general single-stage detectors on pedestrian datasets, owing to the multi-scale deformable self-attention mechanism to enhance spatially adaptive features across levels.
Revised: According to the above, the proposed module is effective for general single-stage detectors on pedestrian datasets, owing to the multi-scale deformable self-attention mechanism to enhance spatially adaptive features across levels.

Original: [Table]
Revised: [Table]

Original: Feature Maps Scales : Different combinations of multi-scale feature maps (cid:8) z l (cid:9) 3 l =1 with three downsampling ratios ( 1 / 4 , 1 / 8 and 1 / 16 ) are compared in Table 7.
Revised: Feature Maps Scales : Different combinations of multi-scale feature maps (cid:8) z l (cid:9) 3 l =1 with three downsampling ratios ( 1 / 4 , 1 / 8 and 1 / 16 ) are compared in Table 7.

Original: In this comparison, only z o
Revised: In this comparison, only z o

Original: [Table] [Table]
Revised: [Table] [Table]

Original: which has the same resolution as z 3 upsampled to ( H/ 4 , W/ 4) , and is fed into the detection head without intermediate concatenation and L2Norm.
Revised: which has the same resolution as z 3 upsampled to ( H/ 4 , W/ 4) , and is fed into the detection head without intermediate concatenation and L2Norm.

Original: Results show that feature maps with ratios 1 / 4 , 1 / 8 , and 1 / 6 for each scale produce the best performance, which is yielded by upsampling c 1 and cby two times while maintaining the scale of c 3 .
Revised: Results show that feature maps with ratios 1 / 4 , 1 / 8 , and 1 / 6 for each scale produce the best performance, which is yielded by upsampling c 1 and cby two times while maintaining the scale of c 3 .

Original: Enhanced Feature Map Scales : Fix the downsampling ratios of (cid:8) z l (cid:9) 3 l =1 as 1 / 4 , 1 / 8 , and 1 / 16 , feed different collections of multi-scale enhanced feature maps { z o l } 3 l =1 to the detection head.
Revised: Enhanced Feature Map Scales : Fix the downsampling ratios of (cid:8) z l (cid:9) 3 l =1 as 1 / 4 , 1 / 8 , and 1 / 16 , feed different collections of multi-scale enhanced feature maps { z o l } 3 l =1 to the detection head.

Original: Note that the deformable encoder keeps the resolutions of the input feature maps, the downsampling ratios for { z o l } 3 l =1 are 1 / 4 , 1 / 8 and 1 / 6 respectively.
Revised: Note that the deformable encoder keeps the resolutions of the input feature maps, the downsampling ratios for { z o l } 3 l =1 are 1 / 4 , 1 / 8 and 1 / 6 respectively.

Original: All the enhanced feature maps are first upsampled to ( H/ 4 , W/ 4) if needed, followed by normalization when multiple feature maps are utilized.
Revised: All the enhanced feature maps are first upsampled to ( H/ 4 , W/ 4) if needed, followed by normalization when multiple feature maps are utilized.

Original: They are then processed in three ways: 1.
Revised: They are then processed in three ways: 1.

Original: Cat: Concatenate them along the channel dimension followed by a compression layer to reduce the number of channels to 256. 2.
Revised: Cat: Concatenate them along the channel dimension followed by a compression layer to reduce the number of channels to 256. 2.

Original: Add: Implement element-wise sum of the enhanced feature maps.
Revised: Add: Implement element-wise sum of the enhanced feature maps.

Original: Sep: No fusion operations across multi-scales; send them separately to the detection head which doubles the number of predictions.
Revised: Sep: No fusion operations across multi-scales; send them separately to the detection head which doubles the number of predictions.

Original: Table 8 compares various strategies to fuse the multi-scale enhanced feature maps.
Revised: Table 8 compares various strategies to fuse the multi-scale enhanced feature maps.

Original: It shows that concatenation followed by L2Norm produces the overall best results in both Reasonable and Heavy subsets on the Caltech test set.
Revised: It shows that concatenation followed by L2Norm produces the overall best results in both Reasonable and Heavy subsets on the Caltech test set.

Original: Choice of the Normalization Method Applied to (cid:8) c l (cid:9) 3 l =1 : As Table 8 shows, GN performs best in the Reasonable subset while L2N in the Heavy subset.
Revised: Choice of the Normalization Method Applied to (cid:8) c l (cid:9) 3 l =1 : As Table 8 shows, GN performs best in the Reasonable subset while L2N in the Heavy subset.

Original: Considering that the miss rate of GN is 11.6% lower in the Reasonable subset and only 1.2% higher in the Heavy subset compared to L2N, GN is utilized in our experiments before the encoder as presented in Figure 3.
Revised: Considering that the miss rate of GN is 11.6% lower in the Reasonable subset and only 1.2% higher in the Heavy subset compared to L2N, GN is utilized in our experiments before the encoder as presented in Figure 3.

Original: Number of Encoders :
Revised: Number of Encoders :

Original: Based on the settings in the last part, different numbers of encoders are tested.
Revised: Based on the settings in the last part, different numbers of encoders are tested.

Original: These encoders are connected in series.
Revised: These encoders are connected in series.

Original: Table 9 presents that although more encoders can provide higher-level semantic information, the best result is observed when only a single encoder is applied.
Revised: Table 9 presents that although more encoders can provide higher-level semantic information, the best result is observed when only a single encoder is applied.

Original: This phenomenon supports the design of BotNet [41] to some extent, which indicates that following the whole set of six encoders and decoders in (deformable) DETR may not be suitable for specific
Revised: This phenomenon supports the design of BotNet [41] to some extent, which indicates that following the whole set of six encoders and decoders in (deformable) DETR may not be suitable for specific

Original: [Table]
Revised: [Table]

Original: + Enhancement module
Revised: + Enhancement module

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: object detection tasks.
Revised: object detection tasks.

Original: A single encoder can also work effectively with the least number of learnable parameters, which prevents overfitting.
Revised: A single encoder can also work effectively with the least number of learnable parameters, which prevents overfitting.

Original: 4.3 Comparison with the state-of-the-arts
Revised: 4.3 Comparison with the state-of-the-arts

Original: Table 10 and Table 11 show that the proposed module achieves the lowest miss rates in Reasonable and Heavy subsets and leading performance in the All subset on both datasets among presented single-stage detectors.
Revised: Table 10 and Table 11 show that the proposed module achieves the lowest miss rates in Reasonable and Heavy subsets and leading performance in the All subset on both datasets among presented single-stage detectors.

Original: For the Caltech dataset, the lowest miss rate (3.7%) of the proposed detector in the Reasonable subset is 0.2% smaller than that of the two-stage KGSNet presented in the upper part of Table 10.
Revised: For the Caltech dataset, the lowest miss rate (3.7%) of the proposed detector in the Reasonable subset is 0.2% smaller than that of the two-stage KGSNet presented in the upper part of Table 10.

Original: With pretraining on the Citypersons dataset, the miss rates on all three subsets of the Caltech dataset are reduced significantly and reach the lowest compared to other pretrained detectors as shown in the bottom part of Table 10.
Revised: With pretraining on the Citypersons dataset, the miss rates on all three subsets of the Caltech dataset are reduced significantly and reach the lowest compared to other pretrained detectors as shown in the bottom part of Table 10.

Original: For the Citypersons dataset, the proposed two detectors even outperform the competitive two-stage detectors in the Reasonable and especially Heavy subset with a miss rate of 38.6% which is 1.1% lower than that of the best of two-stage detectors.
Revised: For the Citypersons dataset, the proposed two detectors even outperform the competitive two-stage detectors in the Reasonable and especially Heavy subset with a miss rate of 38.6% which is 1.1% lower than that of the best of two-stage detectors.

Original: Overall, with the enhanced spatially adaptive and multi-scale features, the gap between single- and two-stage detectors in Reasonable and Heavy subsets on different detectors has been narrowed.
Revised: Overall, with the enhanced spatially adaptive and multi-scale features, the gap between single- and two-stage detectors in Reasonable and Heavy subsets on different detectors has been narrowed.

Original: Surprisingly, the proposed single-stage method outperforms the accurate two-stage methods on certain pedestrian datasets, such as the Citypersons dataset.
Revised: Surprisingly, the proposed single-stage method outperforms the accurate two-stage methods on certain pedestrian datasets, such as the Citypersons dataset.

Original: It should be noted that two-stage methods usually produce overall better accuracy than single-stage methods with the advantage of the region proposal network and refined bounding boxes and at the expense of inference time.
Revised: It should be noted that two-stage methods usually produce overall better accuracy than single-stage methods with the advantage of the region proposal network and refined bounding boxes and at the expense of inference time.

Original: Apart from accuracy, fast inference also matters for pedestrian detection in practical scenes.
Revised: Apart from accuracy, fast inference also matters for pedestrian detection in practical scenes.

Original: The combination of
Revised: The combination of

Original: CSP and the proposed module has a simple structure, which is easy to perform and effective.
Revised: CSP and the proposed module has a simple structure, which is easy to perform and effective.

Original: On the contrary, the leading two-stage KGSNet, for example, takes the advantage of the additional proposal
Revised: On the contrary, the leading two-stage KGSNet, for example, takes the advantage of the additional proposal

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: generation network, the refined bounding boxes, the key-point detector and the super-resolution network.
Revised: generation network, the refined bounding boxes, the key-point detector and the super-resolution network.

Original: With these components, the inference speed of KGSNet is 5.9 FPS and 3.2 FPS (Titan X
Revised: With these components, the inference speed of KGSNet is 5.9 FPS and 3.2 FPS (Titan X

Original: GPU, not including the time of using ALFNet to generate the candidate proposals) on Caltech and
Revised: GPU, not including the time of using ALFNet to generate the candidate proposals) on Caltech and

Original: Citypersons datasets [18] while ours achieves 29.5 FPS and 6.8 FPS (RTX 3090 GPU) respectively.
Revised: Citypersons datasets [18] while ours achieves 29.5 FPS and 6.8 FPS (RTX 3090 GPU) respectively.

Original: Therefore, the enhanced single-stage pedestrian detector is cost-effective with fast inference and competitive accuracy compared to complicated two-stage methods
Revised: Therefore, the enhanced single-stage pedestrian detector is cost-effective with fast inference and competitive accuracy compared to complicated two-stage methods

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  Conclusion
Revised:  Conclusion

Original: The paper proposes a module to enhance spatial and multi-scale features based on a single encoder of the deformable vision transformer to improve the detection accuracy of single-stage pedestrian detectors with fast inference.
Revised: The paper proposes a module to enhance spatial and multi-scale features based on a single encoder of the deformable vision transformer to improve the detection accuracy of single-stage pedestrian detectors with fast inference.

Original: This module is effective on commonly used single-stage structures, including (progressively refined) anchor-based and anchor-free cases.
Revised: This module is effective on commonly used single-stage structures, including (progressively refined) anchor-based and anchor-free cases.

Original: With the CSP detection head, more than 40% parameters are reduced in the detection neck compared to CSP; however, this combination still achieves the best results in Reasonable and Heavy subsets among presented single-stage detectors on both Caltech and Citypersons datasets.
Revised: With the CSP detection head, more than 40% parameters are reduced in the detection neck compared to CSP; however, this combination still achieves the best results in Reasonable and Heavy subsets among presented single-stage detectors on both Caltech and Citypersons datasets.

Original: Utilising pre-training, the miss rates for these subsets can be decreased to 2.6% and 28.0% respectively, which are far better than other single-stage methods and comparable to two-stage methods.
Revised: Utilising pre-training, the miss rates for these subsets can be decreased to 2.6% and 28.0% respectively, which are far better than other single-stage methods and comparable to two-stage methods.

Original: The proposed method outperforms
Revised: The proposed method outperforms

Original: SOTA two-stage detectors in the Heavy subset by 1.1% on Citypersons with slightly decreased inference speed.
Revised: SOTA two-stage detectors in the Heavy subset by 1.1% on Citypersons with slightly decreased inference speed.

Original: This demonstrates that single-stage detectors can be improved if spatially adaptive and multi-scale features are jointly adopted, making them cost-effective and promising.
Revised: This demonstrates that single-stage detectors can be improved if spatially adaptive and multi-scale features are jointly adopted, making them cost-effective and promising.

Original: It should be mentioned that false positives appear if the attention points of a negative reference point extend to the target areas.
Revised: It should be mentioned that false positives appear if the attention points of a negative reference point extend to the target areas.

Original: Although the performance has been improved with the current module, the generation of attention weights and sampling locations can be carefully designed to suppress the false positives for further improvements.Checklist
Revised: Although the performance has been improved with the current module, the generation of attention weights and sampling locations can be carefully designed to suppress the false positives for further improvements.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] The code will be published online in the future.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [No] (d) Did you include the total amount of compute and the type of resources used (e

Original: g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?

Original: [TODO] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] The data are available online.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]

Original: A Appendix
Revised: A Appendix

Original: Memory Cost of Self-attention Weights in DETR : Figure 4 presents the structure of the encoder, decoder, and embedded Multi-Head Self-Attention (MHSA) layer.
Revised: Memory Cost of Self-attention Weights in DETR : Figure 4 presents the structure of the encoder, decoder, and embedded Multi-Head Self-Attention (MHSA) layer.

Original: DETR has six encoder-decoder pairs.
Revised: DETR has six encoder-decoder pairs.

Original: Each MHSA layer has a self-attention weight tensor produced by the multiplication of Query and Key as shown in Figure 4.
Revised: Each MHSA layer has a self-attention weight tensor produced by the multiplication of Query and Key as shown in Figure 4.

Original: The memory to store this tensor during training under different hyperparameter settings and optimization strategies are plotted in Figure 5.
Revised: The memory to store this tensor during training under different hyperparameter settings and optimization strategies are plotted in Figure 5.

Original: It shows that more attention heads, and especially large downsampling ratios, increase the memory cost significantly.
Revised: It shows that more attention heads, and especially large downsampling ratios, increase the memory cost significantly.

Original: Additionally, Adam and AdamW optimizers, which are commonly used to train vision transformers, take more memory compared to simple SGD.
Revised: Additionally, Adam and AdamW optimizers, which are commonly used to train vision transformers, take more memory compared to simple SGD.

Original: For pedestrian detection tasks, we normally choose head=8, downsampling ratio=0.25 and Adam optimizer, it takes more than 10 3 GB memory to store a single attention weight tensor even when the batch size is 1.
Revised: For pedestrian detection tasks, we normally choose head=8, downsampling ratio=0.25 and Adam optimizer, it takes more than 10 3 GB memory to store a single attention weight tensor even when the batch size is 1.

Original: Thus, deformable DETR is used in our work to save memory resources.
Revised: Thus, deformable DETR is used in our work to save memory resources.

Original: Architecture : The overall architecture of the proposed enhancement module combined with the CSP detection head is illustrated in Figure 6.
Revised: Architecture : The overall architecture of the proposed enhancement module combined with the CSP detection head is illustrated in Figure 6.

Original: the backbone neck
Revised: the backbone neck

Original: Training :
Revised: Training :

Original: For anchor-free methods, the same ground truth and loss functions as CSP are utilized.
Revised: For anchor-free methods, the same ground truth and loss functions as CSP are utilized.

Original: The overall loss consists of three parts as:
Revised: The overall loss consists of three parts as:

Original: [Equation]
Revised: [Equation]

Original: 5) where the center heatmap loss L c , height map loss L h and offset loss L o are formulated as
Revised: 5) where the center heatmap loss L c , height map loss L h and offset loss L o are formulated as

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (10) where p ij ∈ (0 , 1) denotes the predicted score in the center heatmap; s k and o k are the predicted height and offset at k -th pedestrian with the ground truth t k
Revised: (10) where p ij ∈ (0 , 1) denotes the predicted score in the center heatmap; s k and o k are the predicted height and offset at k -th pedestrian with the ground truth t k

Original: and ot k ; M is a Gaussian mask map with variances proportional to the height and width of each one of the K pedestrians.
Revised: and ot k ; M is a Gaussian mask map with variances proportional to the height and width of each one of the K pedestrians.

Original: The two hyper-parameters β and γ are set as 4 and 2 empirically in accord with CSP settings.
Revised: The two hyper-parameters β and γ are set as 4 and 2 empirically in accord with CSP settings.

Original: Weights λ c , λ s and λ o for losses in equation Eq.
Revised: Weights λ c , λ s and λ o for losses in equation Eq.

Original: (5) are set as 0.01, 1 and 0.1 according to CSP.
Revised: (5) are set as 0.01, 1 and 0.1 according to CSP.

Original: For anchor-based methods the overall loss consists of two parts:
Revised: For anchor-based methods the overall loss consists of two parts:

Original: [Equation]
Revised: [Equation]

Original: where focusing parameters α and γ are set as 0.25 and 2 [51].
Revised: where focusing parameters α and γ are set as 0.25 and 2 [51].

Original: S + and S − stand for positive and negative sets.
Revised: S + and S − stand for positive and negative sets.

Original: Smooth L1 loss is used for localization.
Revised: Smooth L1 loss is used for localization.

Original: Visualization : Figure 7 presents some detected pedestrian samples under various circumstances including relatively large, small, and significantly inter- and intra- class occluded ones.
Revised: Visualization : Figure 7 presents some detected pedestrian samples under various circumstances including relatively large, small, and significantly inter- and intra- class occluded ones.

Original: To better understand how the Multi-Scale Deformable Attention (MSDA) layer works, the sampling locations and their attention weights are presented in Figure 8.
Revised: To better understand how the Multi-Scale Deformable Attention (MSDA) layer works, the sampling locations and their attention weights are presented in Figure 8.

Original: The center of a single pedestrian corresponds to two reference points in feature maps at the first and third scale with size ( H/ 4 , W/ 4) and ( H/ 16 , W/ 16) .
Revised: The center of a single pedestrian corresponds to two reference points in feature maps at the first and third scale with size ( H/ 4 , W/ 4) and ( H/ 16 , W/ 16) .

Original: Each reference point corresponds to learned sampling locations across 8 heads and 3 scales with learned attention weights which assign large values to important features.
Revised: Each reference point corresponds to learned sampling locations across 8 heads and 3 scales with learned attention weights which assign large values to important features.

Original: The
Revised: The

Original: MSDA layer accumulates the features at these locations with the weights to fuse the multi-scale and spatial adaptive information together.
Revised: MSDA layer accumulates the features at these locations with the weights to fuse the multi-scale and spatial adaptive information together.

Original: According to Figure 8, the distribution patterns of sampling locations are similar in presented samples, however, the differences lie in the attention weights.
Revised: According to Figure 8, the distribution patterns of sampling locations are similar in presented samples, however, the differences lie in the attention weights.

Original: It is observed that high scales tend to focus on a sparser area with mild attention weights while low scales concentrate on the area close to the reference point with relatively sharp weights.
Revised: It is observed that high scales tend to focus on a sparser area with mild attention weights while low scales concentrate on the area close to the reference point with relatively sharp weights.

Original: As the heat maps at a higher scale (third) show, important sampling locations of the relatively large pedestrian seem to be more reasonable as the head, shoulder and leg are highlighted.
Revised: As the heat maps at a higher scale (third) show, important sampling locations of the relatively large pedestrian seem to be more reasonable as the head, shoulder and leg are highlighted.