# RE-PU: A Self-Supervised Arbitrary-Scale Point Cloud Upsampling Method Based on Reconstruction
## 3 Method
### Overview  
We propose a novel point cloud upsampling method, named RE-PU, which is based on the point cloud reconstruction from a prior distribution, achieving self-supervised upsampling at arbitrary rates. The idea is simple but effective: we train a network to reconstruct a prior distribution into the input point cloud, and then we can upsample the point cloud data by increasing the number of sampled points on the prior distribution.  
The architecture of RE-PU is illustrated in Figure 1. The network can be an AutoEncoder (Figure 1), which is for multiple shapes in one model, consisting of an encoder and a decoder, where the encoder identifies input point clouds and the decoder takes the encoder's output along with a prior distribution to reconstruct the prior distribution into the input point cloud. The difference between RE-PU and a regular AutoEncoder lies in the decoder, which incorporates a prior input.  
The proposed method consists of two main stages: the first stage is to train the autoencoder with the original point cloud data and a prior distribution. And the second stage is to upsample the point cloud data with the trained autoencoder. If a separate model is trained for a single shape, the encoder can be removed, and the decoder can be trained alone (Figure 1).  
Since the object is to reconstruct the original point cloud, we do not need input and ground truth paired data, achieving self-supervised upsampling. Since the prior distribution is continuous, then we successfully transform the discrete representation, i.e., the point cloud, into a continuous representation, so we can upsample the point cloud at arbitrary rates.
Figure 3 illustrates the specific operations of the dynamic graph at each layer in the encoder. Given a point cloud \(L_{i}\in\mathbb{R}^{N*C_{i}}\) with N points in \(C_{i}\) dimensions as input, the output is a point cloud \(L_{o}\in\mathbb{R}^{N*C_{o}}\) with N points in \(C_{o}\) dimensions, where \(C_{o}\) can be equal to \(C_{i}\). Initially, for every point \(x_{i}\) within the point cloud, a directed graph is fashioned by gauging feature similarity via K-nearest neighbors (KNN), where each point is capable of referencing itself. The feature of each edge in the graph, linking point \(x_{i}\) to each point \(x_{j}\) in its neighborhood \(N(i)\), is derived by subtracting the feature of \(x_{i}\) from that of \(x_{j}\). Subsequently, the feature of the edge is concatenated with the self-feature of \(x_{i}\), culminating in a comprehensive feature representation that encapsulates both local and global information from the neighborhood graph. Following this, this feature undergoes convolutional layer processing followed by max aggregation across the neighborhood, ultimately yielding the feature \(x_{i}^{\prime}\) for the point \(x_{i}\), as depicted in Equation (1).  
\[x_{i}^{\prime}=\max_{j\in N(i)}\left(Conv(x_{j}-x_{i})\oplus x_{i}\right) \tag{1}\]  
where \(\oplus\) signifies the concatenation operation, and \(Conv\) denotes the convolutional operation.  
Following two layers of convolutional operations, the feature of each point is refined by leveraging the features of its surrounding neighborhood. Similar to PointNet [6], a max pooling operation is applied to the entire point cloud to derive its global feature, which is then fed into the decoder for further processing.