# Multiscale Feature Fusion with Self-Attention for Efficient 6D Pose Estimation
## 3 Materials and Methods
### Overall Structure of the Network
#### 3.2.3 Multiscale Point Cloud Feature Extraction Module  
Point cloud data serve as the data carrier for the entire network, and the accurate conversion of depth images into object surface point clouds is particularly important. Based on optical principles and the camera parameters, pixel points from the depth image are restored into corresponding 3D points, as shown below:  
\[ \begin{cases}z=d_{t}/s\\ x=(u-c_{x})\times z/f_{x}\\ y=(v-c_{y})\times z/f_{y}\end{cases}  \tag{1}\]  
where \((u,v)\) represents the pixel coordinates in the image, \(d_{t}\) is the depth value of the pixel, s is the camera's scaling factor, \(c_{x}\) and \(c_{y}\) represent the image coordinates of the camera's optical axis, and \(f_{x}\) and \(f_{y}\) are the camera's focal lengths along the x-axis and y-axis, respectively. \((x,y,z)\) are the 3D coordinates corresponding to the pixel. PointNet, a commonly used network for processing point cloud data, can only extract geometric information on a single scale and cannot perceive local information. To address this issue, we consider the idea of multiscale local information extraction from grouping and sampling layers in PointNet++ [46]. We simultaneously collect neighborhood information of different sizes around 3D points and integrate this information using a self-attention mechanism to create more expressive fused features. Figure 3 shows the specific process of point cloud feature extraction in our network. The multiscale point cloud extraction module consists of two parts:  
* The upper branch, which contains three consecutive multilayer perceptrons (MLPs) and is structurally similar to the traditional PointNet network. It mainly encodes the geometric information of each point in the point cloud to generate point-level features. * The lower branch of the network, which is dedicated to capturing local geometric features for each point through multiple parallel graph convolution layers. Each layer operates on neighborhoods of different sizes, allowing the network to extract geometric information across various spatial scales. To effectively integrate these multi-scale local features, we introduce a channel-wise self-attention mechanism that adaptively fuses the features based on their relative significance. After obtaining the feature maps from the graph convolution branches, we first apply global average pooling to  
Figure 3: Structure of the point cloud feature extraction module. the summed features, resulting in a compact representation. This pooled vector is then passed through two lightweight MLP layers to generate attention weights with a shape of [3,128], where the dimensions correspond to the three spatial scales and one hundred and twenty-eight feature channels. A softmax function is applied to normalize the attention weights across the scales. These weights are subsequently used to reweight the scale-specific feature maps before aggregation. This attention-driven fusion enables the network to dynamically emphasize the most informative features for each channel, thereby enhancing its discriminative capability and improving robustness to variations in scale. The structure of each graph convolution layer is shown in Figure 4. We select \(k\) neighboring points within a radius \(r\) of the center point based on the Euclidean distance, forming a point set \(Y\) of size (3,k,N). The center point is then subtracted from all neighboring points in \(Y\) to represent local features with size (3,k,N) and is then input into the multilayer perceptron (MLP) to obtain high-dimensional features \(F^{1}\) of size (128,k,N). However, a larger radius \(r\) expands the receptive field, allowing the model to capture richer contextual information, which is particularly beneficial for objects with a detailed texture. Nevertheless, an excessively large \(r\) may introduce noise or irrelevant points, potentially degrading the feature quality.