# REGNet: Ray-Based Enhancement Grouping for 3D Object Detection Based on Point Cloud
## 2 Related Work  
**Grid Projection/Voxelization-based Detection. ** 3D object detection is a challenging task due to the irregular, sparse, and orderless characteristics of 3D points. Most current existing work could be classified into three categories in terms of point cloud representations, i.e., voxel-based, bird's-eye-view-based, and point-based. Thanks to the success of the deep neural network, marvelous progress has been achieved in 2D object detection. However, the 2D object detection task ignores the depth information, which is important to understand the whole scene. Early 3D object detection methods project point clouds to 2D grids [35; 36; 37] or 3D voxels [8; 38], so that the most successful convolutional networks could be directly applied. Concerning bird's-eye-view based methods, most of them are applied to autonomous driving. In outdoor scenes, most objects are distributed on the same plane, so there is little mutual occlusion of objects in the top-down view. However, in indoor scenes, many objects are on top of each other, such as photos on walls or tables covering sofas. Therefore, some works project the point cloud onto a frontal view and use 2D ConvNets to tackle the problem. However, self-occlusion of objects in indoor scenes poses many challenging issues to address. Voxel-based methods transform the point cloud into 3D voxels, which have been shown to yield more reasonable performance. However, these methods suffer from high memory and computational costs, as well as quantization errors. **Point based-Detection. ** To tackle the problems noted above, recently, most methods process point clouds directly for 3D object detection. Due to the point clouds being irregular and sparse, how to extract feature representations from them is the core task of these methods. The pioneering works PointNet/PointNet++ [15; 16] provide a powerful and robust backbone for point cloud based tasks. VoteNet [32] adopts PointNet++ as the backbone and reproduces a Hough voting strategy to propose an end-to-end framework for 3D object detection. This is the first end-to-end framework based on point clouds for this task. There are numerous successors based on this work, such as BGNet [39], which further improves the traditional Hough voting mechanism [40]. The authors proposed a back-tracing strategy that generatively backtracked representative points from the vote centers and then revisited the seed point. H3 DNet [30] and MCGNet [41] recognize the lack of  
Figure 1: The overall architecture of our REGNet model. REGNet is built based on VoteNet with three new modules: (1) the foreground-aware module aims to automatically weight foreground and background features for obtaining better seed point features; (2) the voting-aware module concentrates on weighting different votes on the rays to produce better clusters; (3) the cluster-aware module is utilized for adaptively gathering different clusters for the final prediction. Best viewed in color. modeling ability with only feature extraction from a single backbone branch. They utilized a four-way backbone to extract more plausible feature representations. MLCVNet [29] proposes three sub-modules to capture multi-level contextual information in point cloud data to boost performance. VENet [28] improves the voting procedure in \"before, during and after\" stages to address the limitations of current voting schemes. **Attention Mechanism/Transformer-based Detection. ** The transformer scheme is the dominant network architecture for the tasks of neural language of processing (NLP). Due to the powerful ability of feature modelling, it is applied in the field of 2D image recognition [42; 43; 44]. Most recently, many works apply the transformer scheme into 3D object detection [45; 46]. GFNet [31] proposes a group-free strategy that adopts a powerful transformer module to replace the proposal head in VoteNet [32]. In this paper, we also resorts to the transformer scheme to build spatial context dependencies among different vote centers and clusters to boost prediction performance.