Original: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds
Revised: MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds

Original: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomous driving.
Revised: Abstract 3D object detection from the LiDAR point cloud is fundamental to autonomous driving.

Original: Large-scale outdoor scenes usually feature significant variance in instance scales, thus requiring features rich in long-range and fine-grained information to support accurate detection.
Revised: Large-scale outdoor scenes usually feature significant variance in instance scales, thus requiring features rich in long-range and fine-grained information to support accurate detection.

Original: Recent detectors leverage the power of window-based transformers to model long-range dependencies but tend to blur out fine-grained details.
Revised: Recent detectors leverage the power of window-based transformers to model long-range dependencies but tend to blur out fine-grained details.

Original: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.
Revised: To mitigate this gap, we present a novel Mixed-scale Sparse Voxel Transformer, named MsSVT, which can well capture both types of information simultaneously by the divide-and-conquer philosophy.

Original: Specifically, MsSVT explicitly divides attention heads into multiple groups, each in charge of attending to information within a particular range.
Revised: Specifically, MsSVT explicitly divides attention heads into multiple groups, each in charge of attending to information within a particular range.

Original: All groups’ output is merged to obtain the final mixed-scale features.
Revised: All groups’ output is merged to obtain the final mixed-scale features.

Original: Moreover, we provide a novel chessboard sampling strategy to reduce the computational complexity of applying a window-based transformer in 3D voxel space.
Revised: Moreover, we provide a novel chessboard sampling strategy to reduce the computational complexity of applying a window-based transformer in 3D voxel space.

Original: To improve efficiency, we also implement the voxel sampling and gathering operations sparsely with a hash map.
Revised: To improve efficiency, we also implement the voxel sampling and gathering operations sparsely with a hash map.

Original: Endowed by the powerful capability and high efficiency of modeling mixed-scale information, our single-stage detector built on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectors on Waymo.
Revised: Endowed by the powerful capability and high efficiency of modeling mixed-scale information, our single-stage detector built on top of MsSVT surprisingly outperforms state-of-the-art two-stage detectors on Waymo.

Original: Our project page: https://github.com/dscdyc/MsSVT .
Revised: Our project page: https://github.com/dscdyc/MsSVT .

Original:  Introduction
Revised:  Introduction

Original: 3D object detection has received increasing attention due to its successful autonomous driving applications.
Revised: 3D object detection has received increasing attention due to its successful autonomous driving applications.

Original: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturally irregular and unordered.
Revised: Unlike 2D images with a regular structure of pixels, LiDAR point clouds are naturally irregular and unordered.

Original: Hence directly applying CNN-like operations [11, 12] to them can be difficult.
Revised: Hence directly applying CNN-like operations [11, 12] to them can be difficult.

Original: To solve this, many researchers have rasterized point clouds into regular voxel grids [24] and employed 3D CNNs to extract high-dimensional voxel features.
Revised: To solve this, many researchers have rasterized point clouds into regular voxel grids [24] and employed 3D CNNs to extract high-dimensional voxel features.

Original: With the recent rise of vision transformer (ViT) [42] on 2D images, some attempts have been made to generalize global or more efficient window-based transformers to 3D voxels [23] or pillars [7].
Revised: With the recent rise of vision transformer (ViT) [42] on 2D images, some attempts have been made to generalize global or more efficient window-based transformers to 3D voxels [23] or pillars [7].

Original: These methods successfully seek long-range context by utilizing transformers’ powerful abilities in modeling long-range information.
Revised: These methods successfully seek long-range context by utilizing transformers’ powerful abilities in modeling long-range information.

Original: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details, especially in sparse 3D space, crucial to accurate object recognition and localization.
Revised: However, they ignore that blindly increasing receptive fields would easily blur fine-grained details, especially in sparse 3D space, crucial to accurate object recognition and localization.

Original: Standard window-based transformers update the features of queries in a local window by attending to keys from the same window.
Revised: Standard window-based transformers update the features of queries in a local window by attending to keys from the same window.

Original: Hence simultaneously aggregating long-range context and fine-grained details require enlarging the window size to embrace local and distant voxels.
Revised: Hence simultaneously aggregating long-range context and fine-grained details require enlarging the window size to embrace local and distant voxels.

Original: Nevertheless, directly gathering all the voxels within the window as keys suffers a cubical growth of the computational load with the window size.
Revised: Nevertheless, directly gathering all the voxels within the window as keys suffers a cubical growth of the computational load with the window size.

Original: Some attempts alleviate this by sampling only a certain number of key voxels [23].
Revised: Some attempts alleviate this by sampling only a certain number of key voxels [23].

Original: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels (Fig. 1 b), thus bias mainly on long-range context.
Revised: While a trivial sampling strategy quickly leads to the sparse sampling of local voxels (Fig. 1 b), thus bias mainly on long-range context.

Original: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.
Revised: To mitigate the above, we try to set up multiple key windows of varying sizes centered on a query window and sample the same number of local and distant key voxels separately from the smaller and larger windows.

Original: As a result, we can keep finer granularity in the local region to retain fine-grained details while collecting distant voxels roughly to enlarge the receptive field (Fig.
Revised: As a result, we can keep finer granularity in the local region to retain fine-grained details while collecting distant voxels roughly to enlarge the receptive field (Fig.

Original: 1 (c)).
Revised: 1 (c)).

Original: With the sampled voxels ready, the next question is how to effectively attend to voxels from different windows and simultaneously capture long-range context and fine-grained details.
Revised: With the sampled voxels ready, the next question is how to effectively attend to voxels from different windows and simultaneously capture long-range context and fine-grained details.

Original: We argue that the divide-and-conquer philosophy can satisfactorily resolve this issue.
Revised: We argue that the divide-and-conquer philosophy can satisfactorily resolve this issue.

Original: Specifically, inspired by the recent findings [42, 59, 26] that transformers learn different levels of self-attention by different heads, we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides the transformer heads into multiple groups.
Revised: Specifically, inspired by the recent findings [42, 59, 26] that transformers learn different levels of self-attention by different heads, we propose a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which explicitly divides the transformer heads into multiple groups.

Original: Different head groups accept voxels sampled from windows of different sizes, so they are each in charge of capturing information of a particular scale.
Revised: Different head groups accept voxels sampled from windows of different sizes, so they are each in charge of capturing information of a particular scale.

Original: Combining the outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context, and fine-grained details.
Revised: Combining the outputs from all the head groups, we can capture mixed-scale information, i.e. , long-range context, and fine-grained details.

Original: We also design a novel scale-aware relative position encoding strategy to adaptively adjust the position encoding used in each head group according to the range of the keys.
Revised: We also design a novel scale-aware relative position encoding strategy to adaptively adjust the position encoding used in each head group according to the range of the keys.

Original: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).
Revised: We provide some resulting attention maps by two different head groups (Fig. 1 (d), (e)).

Original: It is also worth mentioning that the mixed-scale attention enables information exchange across local windows, making MsSVT more compact by saving additional shift window operation commonly required by window-based transformers [21, 7].
Revised: It is also worth mentioning that the mixed-scale attention enables information exchange across local windows, making MsSVT more compact by saving additional shift window operation commonly required by window-based transformers [21, 7].

Original: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reduce computational costs in two ways.
Revised: Moreover, to improve the efficiency of applying transformers in 3D voxel space, we strive to reduce computational costs in two ways.

Original: First, we propose a novel chessboard sampling (CBS) strategy to reduce the number of query voxels that need to be sampled within the query window, to reduce computational costs without losing information.
Revised: First, we propose a novel chessboard sampling (CBS) strategy to reduce the number of query voxels that need to be sampled within the query window, to reduce computational costs without losing information.

Original: Specifically, we partition the query window into chess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.
Revised: Specifically, we partition the query window into chess-like spaced, and termed as " × ", " ⃝ ", " △ ", " □ " positions separately.

Original: During each attention layer, only one specific position of voxels is sampled and updated by serving as queries, and the updates of the other voxels can be obtained by interpolation.
Revised: During each attention layer, only one specific position of voxels is sampled and updated by serving as queries, and the updates of the other voxels can be obtained by interpolation.

Original: Four positions are selected in the circular.
Revised: Four positions are selected in the circular.

Original: Thus, we can update all the voxels without introducing deviation.
Revised: Thus, we can update all the voxels without introducing deviation.

Original: Second, we take advantage of non-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-empty sites in 3D space.
Revised: Second, we take advantage of non-empty voxels’ sparsity by performing mixed-scale window-based attention solely on non-empty sites in 3D space.

Original: Furthermore, we parallelize the search and feature gathering for non-empty voxels using hash mapping for further acceleration.
Revised: Furthermore, we parallelize the search and feature gathering for non-empty voxels using hash mapping for further acceleration.

Original: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [50] with our
Revised: We build a 3D detector by replacing the original sparse 3D CNN backbone in SECOND [50] with our

Original: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].
Revised: MsSVT and conduct extensive experiments on the large-scale Waymo open dataset [38].

Original: Benefiting from the powerful capability of abstracting mixed-scale voxel features, our single-stage detector based on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.
Revised: Benefiting from the powerful capability of abstracting mixed-scale voxel features, our single-stage detector based on MsSVT surprisingly outperforms state-of-the-art two-stage detectors.

Original: We summarize our contributions as follows:
Revised: We summarize our contributions as follows:

Original: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneously abstracts voxel features with long-range context and fine-grained details.
Revised: • We present a novel Mixed-scale Sparse Voxel Transformer (MsSVT), which simultaneously abstracts voxel features with long-range context and fine-grained details.

Original: • We design an efficient chessboard sampling strategy to vastly reduce the computational cost of applying a voxel-based transformer in 3D space and sparsely implement all operations to improve efficiency.
Revised: • We design an efficient chessboard sampling strategy to vastly reduce the computational cost of applying a voxel-based transformer in 3D space and sparsely implement all operations to improve efficiency.

Original: • Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors on Waymo.
Revised: • Our MsSVT-based single-stage detector outperforms state-of-the-art two-stage detectors on Waymo.

Original:  Related work
Revised:  Related work

Original: 3D detection on point clouds.
Revised: 3D detection on point clouds.

Original: The mainstream 3D object detectors are based on voxels [3, 51, 52, 37, 17, 14, 60, 50, 44, 43] or pillars [16, 32].
Revised: The mainstream 3D object detectors are based on voxels [3, 51, 52, 37, 17, 14, 60, 50, 44, 43] or pillars [16, 32].

Original: VoxelNet [60] utilizes PointNet [27, 32] to aggregate features within each voxel and then apply sparse 3D convolution to generate detection results.
Revised: VoxelNet [60] utilizes PointNet [27, 32] to aggregate features within each voxel and then apply sparse 3D convolution to generate detection results.

Original: SECOND [50] investigates improved sparse convolution to further improve speed.
Revised: SECOND [50] investigates improved sparse convolution to further improve speed.

Original: Pointpillar [16] converts the point cloud into pillars such that 2D CNNs can be applied to trade off between accuracy and efficiency.
Revised: Pointpillar [16] converts the point cloud into pillars such that 2D CNNs can be applied to trade off between accuracy and efficiency.

Original: Rapoport-Lavie et al .
Revised: Rapoport-Lavie et al .

Original: [28] introduce the Cylindrical Coordinates to leverage the natural scanning pattern from LiDAR sensors.
Revised: [28] introduce the Cylindrical Coordinates to leverage the natural scanning pattern from LiDAR sensors.

Original: Chen et al .
Revised: Chen et al .

Original: [2] further operate on both Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS) voxel representation.
Revised: [2] further operate on both Bird Eye View (BEV) and Range View (RV) in the unified Hybrid-Cylindrical-Spherical (HCS) voxel representation.

Original: Reconfigurable Voxels [45] improves the local neighbors searching of each voxel using a random walk scheme.
Revised: Reconfigurable Voxels [45] improves the local neighbors searching of each voxel using a random walk scheme.

Original: Voxel-FPN
Revised: Voxel-FPN

Original: [15] and Pillar-in-Pillar [41] adopt a multi-scale voxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate FPN [20] to aggregate multi-resolution feature maps.
Revised: [15] and Pillar-in-Pillar [41] adopt a multi-scale voxelization strategy to rasterize the input point cloud into multi-sized voxels and incorporate FPN [20] to aggregate multi-resolution feature maps.

Original: Two-stage detectors [19, 35, 36, 31] refine the bounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features, yielding state-of-the-art performance.
Revised: Two-stage detectors [19, 35, 36, 31] refine the bounding boxes output by a single-stage detector by aggregating raw point clouds or voxel features, yielding state-of-the-art performance.

Original: Vision transformer.
Revised: Vision transformer.

Original: Transformer [42, 5] has recently achieved great success in computer vision [6, 1, 21, 18, 61, 53, 46].
Revised: Transformer [42, 5] has recently achieved great success in computer vision [6, 1, 21, 18, 61, 53, 46].

Original: Swin-transformer [21] restricts self-attention to non-overlapping local windows while allowing cross-window connection to improve efficiency.
Revised: Swin-transformer [21] restricts self-attention to non-overlapping local windows while allowing cross-window connection to improve efficiency.

Original: SSA [29] divides attention heads into multiple groups to aggregate image features with different granularities.
Revised: SSA [29] divides attention heads into multiple groups to aggregate image features with different granularities.

Original: Guo et al.
Revised: Guo et al.

Original: [9] and
Revised: [9] and

Original: Zhao et al.
Revised: Zhao et al.

Original: [58] make the first step towards introducing the transformer for point cloud analysis.
Revised: [58] make the first step towards introducing the transformer for point cloud analysis.

Original: Recently, many approaches [47, 57, 25, 10, 22, 49] apply local self-attention to learn richer 3D feature representation.
Revised: Recently, many approaches [47, 57, 25, 10, 22, 49] apply local self-attention to learn richer 3D feature representation.

Original: Our work extends the window-based attention on 3D voxels by introducing scale-aware attention learning equipped with novel sampling strategies for the queries and the keys to improve both accuracy and efficiency.
Revised: Our work extends the window-based attention on 3D voxels by introducing scale-aware attention learning equipped with novel sampling strategies for the queries and the keys to improve both accuracy and efficiency.

Original: Voxel transformer for 3D detection.
Revised: Voxel transformer for 3D detection.

Original: VoTr [23] introduces a voxel-based transformer backbone that performs self-attention on sparse voxels with local and dilated attention mechanisms.
Revised: VoTr [23] introduces a voxel-based transformer backbone that performs self-attention on sparse voxels with local and dilated attention mechanisms.

Original: Our work improves VoTr by introducing window-based attention and optimizing sparse operation.
Revised: Our work improves VoTr by introducing window-based attention and optimizing sparse operation.

Original: The recent
Revised: The recent

Original: SST [7] follows a single-stride design and the swin-transformer architecture, which performs well on small objects.
Revised: SST [7] follows a single-stride design and the swin-transformer architecture, which performs well on small objects.

Original: Nevertheless, SST is implemented based on pillars.
Revised: Nevertheless, SST is implemented based on pillars.

Original: The single window size is not conducive to capturing multi-scale features, resulting in unsatisfactory performance on Vehicle when simultaneously detecting multiple categories.
Revised: The single window size is not conducive to capturing multi-scale features, resulting in unsatisfactory performance on Vehicle when simultaneously detecting multiple categories.

Original: In comparison, our MsSVT can capture mixed-scale information to boost the detection of objects of various scales.
Revised: In comparison, our MsSVT can capture mixed-scale information to boost the detection of objects of various scales.

Original:  Method
Revised:  Method

Original: This section first gives details of the MsSVT block, then its efficient sparse implementation, and finally the 3D detector based on MsSVT.
Revised: This section first gives details of the MsSVT block, then its efficient sparse implementation, and finally the 3D detector based on MsSVT.

Original: [Figure]
Revised: [Figure]

Original: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.
Revised: We gather the non-empty voxels within the query window and apply Chessboard Sampling (CBS) to sample the queries.

Original: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.
Revised: For the keys, we gather the non-empty voxels from the key windows of different sizes separately, and get multiple sets of keys through Balanced Multi-window Sampling, with each set representing information of a specific scale.

Original: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.
Revised: Keys from windows of different sizes are assigned to different head groups to perform scale-aware attention learning, thus simultaneously capturing both long-range context and fine-grained details.

Original: 3.1 Mixed-scale Sparse Voxel Transformer
Revised: 3.1 Mixed-scale Sparse Voxel Transformer

Original: Fig.
Revised: Fig.

Original: 2 illustrates the overall architecture of the MsSVT block.
Revised: 2 illustrates the overall architecture of the MsSVT block.

Original: We first gather the query and the key voxels via chessboard sampling and balanced multi-window sampling, respectively.
Revised: We first gather the query and the key voxels via chessboard sampling and balanced multi-window sampling, respectively.

Original: The obtained queries and keys are then fed into multiple head groups to capture mixed-scale information through scale-aware attention learning.
Revised: The obtained queries and keys are then fed into multiple head groups to capture mixed-scale information through scale-aware attention learning.

Original: We further incorporate scale-aware relative position encoding to make better use of position information in different head groups.
Revised: We further incorporate scale-aware relative position encoding to make better use of position information in different head groups.

Original: 3.1.1 Balanced Multi-window Sampling
Revised: 3.1.1 Balanced Multi-window Sampling

Original: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query window and s 1 ,...,M are the sizes of M successively larger key windows.
Revised: Let { s k | s k ∈ Z 3 } Mk =0 denote a series of window sizes, where s 0 is the size of the query window and s 1 ,...,M are the sizes of M successively larger key windows.

Original: Let V
Revised: Let V

Original: = { v i | v i =
Revised: = { v i | v i =

Original: ( x i , f i ) } |V| i =1 be the input voxel set, with xyz coordinates x i ∈
Revised: ( x i , f i ) } |V| i =1 be the input voxel set, with xyz coordinates x i ∈

Original: Z 3 and feature vector f i ∈ R C for voxel
Revised: Z 3 and feature vector f i ∈ R C for voxel

Original: i .
Revised: i .

Original: We first partition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-empty ones as query windows with their centers denoted by { c i
Revised: We first partition the voxel set into non-overlapping 3D windows each of size s 0 , and find the non-empty ones as query windows with their centers denoted by { c i

Original: | c i ∈
Revised: | c i ∈

Original: Z 3 } Li =0 , where L is the total number of query windows.
Revised: Z 3 } Li =0 , where L is the total number of query windows.

Original: To get query voxels V c i , s 0 for the query window centered on c
Revised: To get query voxels V c i , s 0 for the query window centered on c

Original: i , one can simply gather all the non-empty voxels within the window as the queries.
Revised: i , one can simply gather all the non-empty voxels within the window as the queries.

Original: While keeping efficiency in mind, we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.
Revised: While keeping efficiency in mind, we present a novel chessboard sampling strategy, which will be detailed in Section 3.1.2.

Original: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for the neighbors for each center c i within multiple key windows of different sizes.
Revised: As for the key voxels, instead of sampling within a single large window at once as previous methods [23], which inevitably biases on either local or distant voxels, we simultaneously search for the neighbors for each center c i within multiple key windows of different sizes.

Original: For the key window of size s k , we gather no more than N P non-empty voxels V c
Revised: For the key window of size s k , we gather no more than N P non-empty voxels V c

Original: i , s k
Revised: i , s k

Original: = { v j | − s k < x j − c
Revised: = { v j | − s k < x j − c

Original: i < s k } N P j =1 , where N P is a pre-set number.
Revised: i < s k } N P j =1 , where N P is a pre-set number.

Original: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from
Revised: Furthermore, to reduce computational cost and keep balanced sampling, we adopt the farthest point sampling (FPS) algorithm to uniformly sample N K voxels from

Original: V c i , s k to obtain the final key voxels
Revised: V c i , s k to obtain the final key voxels

Original: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-set maximum number of sampled voxels.
Revised: V fps c i , s k , k = 1 , ..., M at different scales, where N K is a pre-set maximum number of sampled voxels.

Original: Benefitting from the multi-window strategy equipped with uniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, which is crucial to capturing mixed-scale information.
Revised: Benefitting from the multi-window strategy equipped with uniform sampling by FPS, we can achieve balanced sampling of key voxels at various scales, which is crucial to capturing mixed-scale information.

Original: 3.1.2 Chessboard Sampling
Revised: 3.1.2 Chessboard Sampling

Original: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels and only need to select some representative ones to reduce computational redundancy.
Revised: The prerequisite for us to sample the keys is that we do not have to preserve all the key voxels and only need to select some representative ones to reduce computational redundancy.

Original: While things are different for the queries.
Revised: While things are different for the queries.

Original: On the one hand, we should retain and update every query voxel after an attention layer; otherwise, the information could be irreversibly lost.
Revised: On the one hand, we should retain and update every query voxel after an attention layer; otherwise, the information could be irreversibly lost.

Original: But on the other hand, the computational cost and memory footprint grow dramatically with the window size, making implementation impractical if the queries are not reduced.
Revised: But on the other hand, the computational cost and memory footprint grow dramatically with the window size, making implementation impractical if the queries are not reduced.

Original: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxels each time for feature updating, and then use the sampled voxels to update those unsampled.
Revised: Since the positions of non-empty voxels remain unchanged after a series of attention blocks, an effective solution is to sample a subset of query voxels each time for feature updating, and then use the sampled voxels to update those unsampled.

Original: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.
Revised: Motivated by the above, we present Chessboard Sampling (CBS) as illustrated in Fig.

Original: Each voxel in the query window is marked with " × ", " ⃝ ", " △ ", or " □ ", in an evenly spaced pattern.
Revised: Each voxel in the query window is marked with " × ", " ⃝ ", " △ ", or " □ ", in an evenly spaced pattern.

Original: In each MsSVT block, we sample queries from the non-empty voxels with one specific type of the marks, and update the queries’ features through attention learning.
Revised: In each MsSVT block, we sample queries from the non-empty voxels with one specific type of the marks, and update the queries’ features through attention learning.

Original: Then for each unsampled non-empty voxel, we update its feature by finding the K-nearest query voxels (by default K = 3 ) and by linearly interpolating their updated features.
Revised: Then for each unsampled non-empty voxel, we update its feature by finding the K-nearest query voxels (by default K = 3 ) and by linearly interpolating their updated features.

Original: The four masks are used in the circular to sample query voxels in stacked blocks.
Revised: The four masks are used in the circular to sample query voxels in stacked blocks.

Original: This way, we retain the original structure and cover all voxels as comprehensively as possible.
Revised: This way, we retain the original structure and cover all voxels as comprehensively as possible.

Original: Note that we can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2, 1/4, or 1/8.
Revised: Note that we can apply the interval sampling on any of the x , y , z axes to achieve a sampling rate of 1/2, 1/4, or 1/8.

Original: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).
Revised: Generally, we apply it in the horizontal x - y plane (as illustrated in Fig. 3).

Original: 3.1.3 Scale-aware Head Attention
Revised: 3.1.3 Scale-aware Head Attention

Original: Given the query voxels V c
Revised: Given the query voxels V c

Original: i , s 

Revised: i , s

Original: =
Revised: =

Original: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and feature vectors F 0 ∈ R N Q × C , and the multi-scale key voxels
Revised: ( X 0 , F 0 ) with voxel coordinates X 0 ∈ Z N Q × 3 and feature vectors F 0 ∈ R N Q × C , and the multi-scale key voxels

Original: V fps c i , s k =
Revised: V fps c i , s k =

Original: ( X k , F k ) , k = 1 , ..., M with voxel coordinates
Revised: ( X k , F k ) , k = 1 , ..., M with voxel coordinates

Original: X k ∈
Revised: X k ∈

Original: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk = and values { V k } Mk =1 as
Revised: Z N K × 3 and feature vectors F k ∈ R N K × C , we first get queries Q , keys { K k } Mk = and values { V k } Mk =1 as

Original: [Equation]
Revised: [Equation]

Original: where W Q , W K , W V ∈ R C
Revised: where W Q , W K , W V ∈ R C

Original: ×
Revised: ×

Original: C are linear projections.
Revised: C are linear projections.

Original: To achieve scare-aware attention learning, we divide multiple attention heads into M groups and assign the keys from windows of different sizes to different head groups.
Revised: To achieve scare-aware attention learning, we divide multiple attention heads into M groups and assign the keys from windows of different sizes to different head groups.

Original: Accordingly, we also split the feature channels of the queries Q into M groups.
Revised: Accordingly, we also split the feature channels of the queries Q into M groups.

Original: The k -th channel group of Q , denoted by Q k =
Revised: The k -th channel group of Q , denoted by Q k =

Original: Q
Revised: Q

Original: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M , is fed into the k -th head group.
Revised: [: , ( k − 1) × C/M : k × C/M ] , k = 1 , ..., M , is fed into the k -th head group.

Original: As a result, each head group is responsible for learning to attend information at a specific scale.
Revised: As a result, each head group is responsible for learning to attend information at a specific scale.

Original: The attended feature for the k -th head group is
Revised: The attended feature for the k -th head group is

Original: [Equation]
Revised: [Equation]

Original: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents the newly proposed relative position encoding which will be detailed in Section 3.1.4.
Revised: where MHA ( · ) denotes multi-head-group attention and RPE ( · ) represents the newly proposed relative position encoding which will be detailed in Section 3.1.4.

Original: Each window size corresponds to a head group with one or more attention heads.
Revised: Each window size corresponds to a head group with one or more attention heads.

Original: We concatenate the output from all heads groups { ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implemented by multi-layer perceptron (MLP) to obtain the final mixed-scale feature Y :
Revised: We concatenate the output from all heads groups { ˜ Y k } Mk =1 to ˜ Y ∈ R L × N Q × C , which is further fed into a feed-forward network (FFN) implemented by multi-layer perceptron (MLP) to obtain the final mixed-scale feature Y :

Original: [Equation]
Revised: [Equation]

Original: where LN ( · ) represents layer normalization.
Revised: where LN ( · ) represents layer normalization.

Original: 3.1.4 Scale-aware Relative Position Encoding
Revised: 3.1.4 Scale-aware Relative Position Encoding

Original: Relative position encoding is necessary for transformer-based networks because fine-grained position information may be lost in high-level features with the deepening of the network.
Revised: Relative position encoding is necessary for transformer-based networks because fine-grained position information may be lost in high-level features with the deepening of the network.

Original: To make better use of position information to facilitate multi-scale feature learning in our case, we adopt a scale-aware adaptive relative position encoding strategy inspired by [30, 48, 55], which can generate the positional bias dynamically with scales for different head groups.
Revised: To make better use of position information to facilitate multi-scale feature learning in our case, we adopt a scale-aware adaptive relative position encoding strategy inspired by [30, 48, 55], which can generate the positional bias dynamically with scales for different head groups.

Original: Specifically, we establish a learnable embedding table T k ∈ R D × R for the k -th head group according to the size of the largest key window [21], where R is the number of possible relative position pairs and D is the feature dimension.
Revised: Specifically, we establish a learnable embedding table T k ∈ R D × R for the k -th head group according to the size of the largest key window [21], where R is the number of possible relative position pairs and D is the feature dimension.

Original: We get the relative positional bias for the queries as:
Revised: We get the relative positional bias for the queries as:

Original: [Equation]
Revised: [Equation]

Original: where I
Revised: where I

Original: k ∈
Revised: k ∈

Original: Z L × N Q × N K represents the table indices corresponding to the actual relative position between the queries and the keys, G ( · ) is the operation of gathering features according to the indices.
Revised: Z L × N Q × N K represents the table indices corresponding to the actual relative position between the queries and the keys, G ( · ) is the operation of gathering features according to the indices.

Original: Similarly, we can ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.
Revised: Similarly, we can ge the relative positional bias B Kk ∈ R L × N Q × N K for the keys.

Original: Finally, the biases act directly on the attention weights, and Eq.
Revised: Finally, the biases act directly on the attention weights, and Eq.

Original: (2) can be rewritten as:
Revised: (2) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) represents softmax.
Revised: where σ ( · ) represents softmax.

Original: This way, position embeddings can be adjusted adaptively according to different scales to guide the scale-ware head attention better.
Revised: This way, position embeddings can be adjusted adaptively according to different scales to guide the scale-ware head attention better.

Original: 3.2 Sparse Implementation
Revised: 3.2 Sparse Implementation

Original: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implement all our window center searching, window gathering, and balanced window sampling into CUDA operations.
Revised: To leverage the natural sparsity of point clouds and further improve efficiency, we sparsely implement all our window center searching, window gathering, and balanced window sampling into CUDA operations.

Original: These operations are mainly based on a hash map that establishes the mapping from coordinate space to voxel index [23].
Revised: These operations are mainly based on a hash map that establishes the mapping from coordinate space to voxel index [23].

Original: Taking the window gathering operation as an example, we query each possible position w.r.t.
Revised: Taking the window gathering operation as an example, we query each possible position w.r.t.

Original: the given center within the window, and retrieve the corresponding features if the position is a valid key in the pre-built hash map.
Revised: the given center within the window, and retrieve the corresponding features if the position is a valid key in the pre-built hash map.

Original: More details are available in the supplementary materials.
Revised: More details are available in the supplementary materials.

Original: 3.3 Detector Establishment
Revised: 3.3 Detector Establishment

Original: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.
Revised: We build our 3D backbone by stacking multiple MsSVT blocks, as shown in Fig.

Original: Noted that we set both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3D voxels into a 2D feature map, where the query is the average voxel features within the pillar window.
Revised: Noted that we set both the query and the key window size in the last MsSVT block as (1 , 1 , ∞ ) to compress the 3D voxels into a 2D feature map, where the query is the average voxel features within the pillar window.

Original: We replace the 3D backbone in SECOND [50] with MsSVT and keep the other network components unchanged.
Revised: We replace the 3D backbone in SECOND [50] with MsSVT and keep the other network components unchanged.

Original: We discard the down sampling process because MsSVT can already capture features at different scales.
Revised: We discard the down sampling process because MsSVT can already capture features at different scales.

Original: The input point cloud is first converted into regular voxels and fed into our MsSVT backbone to get mixed-scale voxel features.
Revised: The input point cloud is first converted into regular voxels and fed into our MsSVT backbone to get mixed-scale voxel features.

Original: The features are then compressed vertically and sent to the subsequent 2D RPN and detection head to get detection results.
Revised: The features are then compressed vertically and sent to the subsequent 2D RPN and detection head to get detection results.

Original:  Experiments
Revised:  Experiments

Original: In this section, we first provide architectural details of MsSVT, then compare our model with recent state-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.
Revised: In this section, we first provide architectural details of MsSVT, then compare our model with recent state-of-the-art detectors on Waymo Open [38] and KITTI [8] datasets.

Original: Thorough ablation studies and in-depth analysis are further provided to validate our design choices.
Revised: Thorough ablation studies and in-depth analysis are further provided to validate our design choices.

Original: 4.1 Architectural Details
Revised: 4.1 Architectural Details

Original: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key window sizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are set as 1 × 1 pillar as mentioned in 3.3.
Revised: MsSVT comprises four regular MsSVT blocks with a query window size of (3 , 3 , 5) and key window sizes of (3 , 3 , 5) and (7 , 7 , 7) , followed by a specialized MsSVT block where the windows are set as 1 × 1 pillar as mentioned in 3.3.

Original: We divide 8 attention heads into 2 head groups.
Revised: We divide 8 attention heads into 2 head groups.

Original: The sampling rate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .
Revised: The sampling rate of the chessboard sampling is 1/4 and the maximum number of sampled keys N K is 32 .

Original: We use the center head [56] to generate single-stage bounding boxes.
Revised: We use the center head [56] to generate single-stage bounding boxes.

Original: In addition, we also provide a two-stage version with CT3D [31].
Revised: In addition, we also provide a two-stage version with CT3D [31].

Original: More details can be found in OpenPCDet [40] since we conduct all experiments with this toolbox.
Revised: More details can be found in OpenPCDet [40] since we conduct all experiments with this toolbox.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4.2 Results on Waymo
Revised: 4.2 Results on Waymo

Original: Setups.
Revised: Setups.

Original: We first evaluate our model on large-scale Waymo Open Dataset [38].
Revised: We first evaluate our model on large-scale Waymo Open Dataset [38].

Original: The input is a single-frame point cloud with 150m × 150m detection range.
Revised: The input is a single-frame point cloud with 150m × 150m detection range.

Original: We set the detection range in the horizontal and the vertical direction as [ − 75 .
Revised: We set the detection range in the horizontal and the vertical direction as [ − 75 .

Original: 2m , 75 . 2m] and [ − 2 .
Revised: 2m , 75 . 2m] and [ − 2 .

Original: 0m , 4 . 0m] , respectively.
Revised: 0m , 4 . 0m] , respectively.

Original: The voxel size is (0 . 4m , 0 .
Revised: The voxel size is (0 . 4m , 0 .

Original: 4m , 0 .
Revised: 4m , 0 .

Original: 6m) .
Revised: 6m) .

Original: We follow the same training strategy as in [23].
Revised: We follow the same training strategy as in [23].

Original: Specifically, we train our model for 80 epochs on 20% Waymo data using the Adam optimizer [13].
Revised: Specifically, we train our model for 80 epochs on 20% Waymo data using the Adam optimizer [13].

Original: We apply the cyclic decay scheme [50], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochs and further decreased to 1 e- 5 in the remaining epochs.
Revised: We apply the cyclic decay scheme [50], by which the learning rate is increased from 1 e- 4 to 1 e- 3 during the first 40% epochs and further decreased to 1 e- 5 in the remaining epochs.

Original: We also report the results of training forepochs on 100% Waymo data using the same optimizer and learning rate scheme.
Revised: We also report the results of training forepochs on 100% Waymo data using the same optimizer and learning rate scheme.

Original: The evaluation metric is the 3D mean Average Precision (mAP) for difficulty levels of LEVEL 1 and LEVEL 2.
Revised: The evaluation metric is the 3D mean Average Precision (mAP) for difficulty levels of LEVEL 1 and LEVEL 2.

Original: Main results.
Revised: Main results.

Original: We compare our model with state-of-the-art priors in Table 1 and Table 2 .
Revised: We compare our model with state-of-the-art priors in Table 1 and Table 2 .

Original: It is worth noting that our model performs simultaneous detection of three object categories, which is more challenging than detecting a single category.
Revised: It is worth noting that our model performs simultaneous detection of three object categories, which is more challenging than detecting a single category.

Original: Using 20% training data, our single-stage MsSVT-SS significantly surpasses other single-stage counterparts, even though some of them are trained specially for one specific category.
Revised: Using 20% training data, our single-stage MsSVT-SS significantly surpasses other single-stage counterparts, even though some of them are trained specially for one specific category.

Original: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage
Revised: Noticeably, MsSVT-SS performs on par with the state-of-the-art two-stage

Original: PV-RCNN++ [36] on Vehicle , and even better ( 0 .
Revised: PV-RCNN++ [36] on Vehicle , and even better ( 0 .

Original: 8 - 1 . 5 mAP higher) on Pedestrian and Cyclist .
Revised: 8 - 1 . 5 mAP higher) on Pedestrian and Cyclist .

Original: In addition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperforms previous best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.
Revised: In addition, our two-stage MsSVT-TS performs the best on all categories, and significantly outperforms previous best-performing PV-RCNN++ by 2 . 9 and 3 . 2 mAP on Pedestrian and Cyclist , respectively.

Original: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achieve the best results compared to their corresponding counterparts.
Revised: Similarly, when using 100% data, both our single-stage MsSVT-SS and two-stage MsSVT-TS achieve the best results compared to their corresponding counterparts.

Original: Noticeably, our MsSVT-SS surpasses the recent transformer-based detector SST [7] dedicated to small object detection by large margins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.
Revised: Noticeably, our MsSVT-SS surpasses the recent transformer-based detector SST [7] dedicated to small object detection by large margins of 4 . 3 and 4 . 5 mAP on Vehicle and Cyclist , respectively.

Original: This clearly evidences the superiority of our
Revised: This clearly evidences the superiority of our

Original: MsSVT in capturing mixed-scale information over conventional transformer designs.
Revised: MsSVT in capturing mixed-scale information over conventional transformer designs.

Original: Visualization.
Revised: Visualization.

Original: We provide some qualitative results in Fig.
Revised: We provide some qualitative results in Fig.

Original: Our MsSVT can still predict accurate bounding boxes in scenes beyond the range of 50m where the points become extremely sparse (Fig.(a)).
Revised: Our MsSVT can still predict accurate bounding boxes in scenes beyond the range of 50m where the points become extremely sparse (Fig.(a)).

Original: This evidences that MsSVT successfully captures valuable contextual information in the absence of fine-grained details for long-range object detection.
Revised: This evidences that MsSVT successfully captures valuable contextual information in the absence of fine-grained details for long-range object detection.

Original: In addition, our model also performs well in complex scenes in the presence of dense objects with large-scale variations (Fig. 4 (b).
Revised: In addition, our model also performs well in complex scenes in the presence of dense objects with large-scale variations (Fig. 4 (b).

Original: It should be owed to the flexibility and robustness of MsSVT.
Revised: It should be owed to the flexibility and robustness of MsSVT.

Original: Analysis.
Revised: Analysis.

Original: We further visualize some attention maps which indicate what has been focused on by our MsSVT to understand its behavior better.
Revised: We further visualize some attention maps which indicate what has been focused on by our MsSVT to understand its behavior better.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, the head group with a smaller key window focuses on the local foreground information, while the head group with a larger key window attends more to the longer-range context.
Revised: 5, the head group with a smaller key window focuses on the local foreground information, while the head group with a larger key window attends more to the longer-range context.

Original: As a result, the two head groups complement each other, well capturing mixed-scale information to support the detection of objects of various scales
Revised: As a result, the two head groups complement each other, well capturing mixed-scale information to support the detection of objects of various scales

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4.3 Results on KITTI
Revised: 4.3 Results on KITTI

Original: Setups.
Revised: Setups.

Original: We also evaluate our model on KITTI [8].
Revised: We also evaluate our model on KITTI [8].

Original: Given an input point cloud, we reserve the points within the range of [0m , 70 . 4m] , [ − 40 .
Revised: Given an input point cloud, we reserve the points within the range of [0m , 70 . 4m] , [ − 40 .

Original: 0m , 40 .
Revised: 0m , 40 .

Original: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes, respectively.
Revised: 0m] and [ − 3 . 0m , 1 . 0m] on the x , y , z axes, respectively.

Original: The voxel size is set to (0 .
Revised: The voxel size is set to (0 .

Original: 32m , 0 . 32m , 0 .
Revised: 32m , 0 . 32m , 0 .

Original: 4m)
Revised: 4m)

Original: and other settings remain the same as in the experiments on Waymo.
Revised: and other settings remain the same as in the experiments on Waymo.

Original: We train the model for 100 epochs with the Adam optimizer.
Revised: We train the model for 100 epochs with the Adam optimizer.

Original: The learning rate is 0 . 003 , decayed by the cyclic scheme [50].
Revised: The learning rate is 0 . 003 , decayed by the cyclic scheme [50].

Original: The evaluation metric is 3D mAP for three difficulty levels (easy, moderate, and hard).
Revised: The evaluation metric is 3D mAP for three difficulty levels (easy, moderate, and hard).

Original: Main results.
Revised: Main results.

Original: Table 3 depicts that our model achieves competitive performance on all three categories.
Revised: Table 3 depicts that our model achieves competitive performance on all three categories.

Original: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by 0 .
Revised: Specifically, on Car , our single-stage MsSVT-SS surpasses the superior VoTr [23] by 0 .

Original: 5 mAP.
Revised: 5 mAP.

Original: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminent two-stage detectors.
Revised: Meanwhile, on Pedestrian and Cyclist , MsSVT-SS even outperforms some preeminent two-stage detectors.

Original: Moreover, our two-stage MsSVT-TS further increases the lead.
Revised: Moreover, our two-stage MsSVT-TS further increases the lead.

Original: These results demonstrate that our MsSVT can be well generalized to various datasets.
Revised: These results demonstrate that our MsSVT can be well generalized to various datasets.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: All ablation models are trained for 12 epochs on 20% Waymo data.
Revised: All ablation models are trained for 12 epochs on 20% Waymo data.

Original: More implementation details and ablations on hyperparameters are available in supplementary materials.
Revised: More implementation details and ablations on hyperparameters are available in supplementary materials.

Original: Balanced multi-window sampling.
Revised: Balanced multi-window sampling.

Original: We first validate our balanced multi-window sampling strategy in Table 4.
Revised: We first validate our balanced multi-window sampling strategy in Table 4.

Original: The base model listed in the first row employs a 3D version of standard windowbased attention [21] with window size (3 , 3 , 5) and without shift window scheme, adopts dilated key sampling [23], and gathers all non-empty voxels within the window as queries.
Revised: The base model listed in the first row employs a 3D version of standard windowbased attention [21] with window size (3 , 3 , 5) and without shift window scheme, adopts dilated key sampling [23], and gathers all non-empty voxels within the window as queries.

Original: We build a model variant by simply replacing the dilated key sampling with our proposed balanced multiwindow sampling.
Revised: We build a model variant by simply replacing the dilated key sampling with our proposed balanced multiwindow sampling.

Original: Results in the second row demonstrate that sampling key voxels from multiple windows of different sizes brings noticeable performance gains,
Revised: Results in the second row demonstrate that sampling key voxels from multiple windows of different sizes brings noticeable performance gains,

Original: i
Revised: i

Original: [Table]
Revised: [Table]

Original: e. , 71 . 

Revised: e. , 71 .

Original: → 73 . 44 on Pedestrian and 63 .
Revised: → 73 . 44 on Pedestrian and 63 .

Original: 31 → 66 . 88 on Cyclist .
Revised: 31 → 66 . 88 on Cyclist .

Original: [Table]
Revised: [Table]

Original: Scale-aware head attention.
Revised: Scale-aware head attention.

Original: We also validate the effectiveness of our proposed scale-aware head attention in Table 4.
Revised: We also validate the effectiveness of our proposed scale-aware head attention in Table 4.

Original: By comparing the results in the second and the third rows, one can find that enabling multiple head groups to capture information at different scales clearly boots the performance.
Revised: By comparing the results in the second and the third rows, one can find that enabling multiple head groups to capture information at different scales clearly boots the performance.

Original: [Table]
Revised: [Table]

Original: Table 4 shows that incorporating scaleaware relative position encoding (listed in the last row) further improves the performance compared to the model variant that adopts scale-agnostic position encoding.
Revised: Table 4 shows that incorporating scaleaware relative position encoding (listed in the last row) further improves the performance compared to the model variant that adopts scale-agnostic position encoding.

Original: It well supports our design motivation that the position encoding should vary with different scales
Revised: It well supports our design motivation that the position encoding should vary with different scales

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Chessboard sampling.
Revised: Chessboard sampling.

Original: Table 5 reports the performance of the model variants that apply chessboard sampling with different sampling rates and of the model without performing any sampling.
Revised: Table 5 reports the performance of the model variants that apply chessboard sampling with different sampling rates and of the model without performing any sampling.

Original: We measure the latency on Tesla-V100 GPU and
Revised: We measure the latency on Tesla-V100 GPU and

Original: Unbuntu-16.04, Python 3.7, Cuda-10.2, and Pytorch-1.8.
Revised: Unbuntu-16.04, Python 3.7, Cuda-10.2, and Pytorch-1.8.

Original: Our model is not sensitive to varying sampling rates.
Revised: Our model is not sensitive to varying sampling rates.

Original: Incorporating chessboard sampling with a sampling rate of 1/4 results in only negligible performance degradation compared to the model variant without sampling, yet largely improves the computational efficiency by a significant reduction in memory footprint (up to 33% decrease) and in latency (up to 28% decrease).
Revised: Incorporating chessboard sampling with a sampling rate of 1/4 results in only negligible performance degradation compared to the model variant without sampling, yet largely improves the computational efficiency by a significant reduction in memory footprint (up to 33% decrease) and in latency (up to 28% decrease).

Original:  Conclusion
Revised:  Conclusion

Original: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scale information to boost 3D object detection.
Revised: This paper presents MsSVT, a novel sparse voxel transformer that is capable of capturing mixed-scale information to boost 3D object detection.

Original: We begin with sampling the queries and the keys at different scales through the novel chessboard and the balanced multi-window sampling strategy, respectively.
Revised: We begin with sampling the queries and the keys at different scales through the novel chessboard and the balanced multi-window sampling strategy, respectively.

Original: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge of capturing information at a specific scale.
Revised: Then, we explicitly divide attention heads into multiple scale-aware groups, each in charge of capturing information at a specific scale.

Original: Extensive experiments have shown that our
Revised: Extensive experiments have shown that our

Original: MsSVT is superior in detecting objects of various scales and granularities thanks to the learned mixed-scale information.
Revised: MsSVT is superior in detecting objects of various scales and granularities thanks to the learned mixed-scale information.

Original: Limitations.
Revised: Limitations.

Original: MsSVT can capture mixed-scale information from multiple local windows and achieves promising performance.
Revised: MsSVT can capture mixed-scale information from multiple local windows and achieves promising performance.

Original: Yet, the sizes of the windows need to be pre-set manually.
Revised: Yet, the sizes of the windows need to be pre-set manually.

Original: We will explore the adaptive-window version of MsSVT in future work.
Revised: We will explore the adaptive-window version of MsSVT in future work.

Original: Acknowledgements
Revised: Acknowledgements

Original: This work was financially supported by the National Natural Science Foundation of China (No.
Revised: This work was financially supported by the National Natural Science Foundation of China (No.

Original: 62101032), the Postdoctoral Science Foundation of China (Nos. 2021M690015, 2022T150050) and
Revised: 62101032), the Postdoctoral Science Foundation of China (Nos. 2021M690015, 2022T150050) and

Original: Beijing Institute of Technology Research Fund Program for Young Scholars (No. 3040011182111)Checklist
Revised: Beijing Institute of Technology Research Fund Program for Young Scholars (No. 3040011182111)

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Our work is only for academic research purpose.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] Our implementation details are posted on Section 4.1 and the supplemental materials.

Original: The data are public datasets and our code will be available.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Section 4.1 and setup in 4.2 and 4.

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report average performance of 5 trials by default.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] Waymo [38] and KITTI [8] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] We don’t include any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] Waymo [38] and KITTI [8] are two standard benchmarks.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human subjects...

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A]

Original: We didn’t use crowdsourcing or conduct research with human subjects...

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A] We didn’t use crowdsourcing or conduct research with human subjects...