# A Novel Equivariant Self-Supervised Vector Network for Three-Dimensional Point Clouds
## 1 Introduction  
With the rapid development of autonomous driving [1, 2], virtual reality [3], and robot interaction [4, 5], the processing of 3D data is becoming increasingly important [6], and the equivariant network has a good performance in processing transformed stereoscopic data. Three-dimensional data contain depth information that is unavailable in 2D planar data, allowing machines to perform more advanced tasks, such as robotic arm manipulation [7] using a point cloud. In this task, the neural network is required to consistently provide pose information for varying object poses equivariantly, facilitating precise object grasping by the robotic arm. This requires the network to be able to effectively learn the pose information in objects, that is, the network needs to have equivariance. If a neural network has equivariance, it means that the representations it produces transform under transformations of the input [8]. The traditional neural network, which does not consider the transformation of data, is not sensitive to the transformation of 3D data in space [9]. Equivariant networks for 3D data show more rotation equivariance. If a network has rotation equivariance, it actively learns the rotational operations within the data, where rotations in the Euclidean space drive the rotation of features in the latent space. Therefore, rotation-equivariant networks can perceive the transformation of data and conveniently learn deeper spatial geometric features. Rotation invariance is a special case of rotation equivariance. Rotation-invariant networks handle rotateddata robustly, and the rotation operation applied to the data does not impact the learning of the network's latent features. It abstracts the same invariant information from the geometric characteristics of the same class of rotation data. Recent works [10; 11; 12; 13] have looked into more efficient equivariant networks, but these equivariant networks can be complex [14] or involve complex formulations and are difficult to incorporate into existing pipelines [15]. Therefore, it is hard to apply them to 3D point cloud processing networks with excellent performance [16; 17; 18; 19; 20]. Self-supervised learning leverages unlabeled data to learn latent features, eliminating the need for human-defined annotations when constructing representations [20]. It has demonstrated promising outcomes in the domains of natural language processing and computer vision, yielding notable achievements. Among them, the masked autoencoder [21; 22; 23] is a promising scheme for both languages and images. In point cloud networks, there are also advanced masked self-supervised learning frameworks such as Point-Bert [19] and Point-Mae [20]. However, due to the lack of interpretability, the internal mechanism of the network remains a black box [24], and it is hard to visualize its learned features and enhance them. In this paper, we propose an equivariant self-supervised vector network to handle pose variation information in 3D point clouds, enhance the network's generality for rotated point clouds, and conduct a theoretical analysis of the network's interpretability using information theory. We design a simple and efficient masked self-supervised learning scheme to guide the model to map the input into the latent space in an equivariant way and reconstruct the original input from the latent features equivariantly, to achieve the reconstruction of point clouds in any direction in 3D space. For the equivariance of the network, we adopt the idea of vectorization [15] and use vector neural layers in the network to achieve the transmission of equivariant features. In our self-supervised learning framework, we integrate vector neurons and innovatively extend them to the Transformer architecture of the autoencoder to construct a vector attention calculation mechanism. This enables the seamless transfer of vector information throughout the entire network.