# Point-Sim: A Lightweight Network for 3D Point Cloud Classification
## 3 Methods
### Nonparametric Attention Module (Squash and ResSimAM)  
SimAM [27] devises an energy function to discern the importance of neurons based on neuroscience principles, with most operations selected according to this energy function to avoid excessive structural adjustments. SimAM has been verified to have good performance in 2D parametric models. Due to its nonparametric character, we are considering incorporating this attention mechanism into our 3D point cloud network.  
To successfully implement attention, we need to estimate the importance of individual features. In visual neuroscience, neurons that exhibit unique firing patterns from surrounding neurons are often considered to have the highest information content. Additionally, an active neuron may also inhibit the activity of surrounding neurons, which is a phenomenon known as spatial suppression [28]. In other words, neurons that exhibit significant spatial suppression effects during visual processing should be assigned higher priority. As with SimAM, we use the following equation to obtain the minimum energy for each position:  
\[e_{t}^{*}=\frac{4(\hat{\sigma}^{2}+\lambda)}{(t-\hat{\mu})^{2}+2\hat{\sigma} ^{2}+2\lambda} \tag{12}\]  
where \(\hat{\mu}=\frac{1}{M}\sum_{i=1}^{M}x_{i},\hat{\sigma}^{2}=\frac{1}{M}\sum_{i=1 }^{M}(x_{i}-\hat{\mu})^{2}\), and M denote the feature dimensions.  
The above equation indicates that the lower the energy \(e_{t}^{*}\), the greater the difference between the neuron and its surrounding neurons, which is also more important in visual processing. The importance of neurons is represented by \(1/e_{t}^{*}\). To enhance the features, we construct a residual network. Firstly, we apply the Squash operation to smooth the features, and then we add the ResSimAM attention operation to the squashed features:  
\[\begin{split} X&=Squash(f_{i}+f_{c})\\ \tilde{X}&=sigmoid\left(\frac{1}{E}\right)\odot X+X \end{split} \tag{13}\]  
where \(E\) groups all \(e_{t}^{*}\) across all dimensions, and a sigmoid is added to restrict too large values in \(E\).  
Algorithm 1 denotes the pseudocode for the implementation of ResSimAM using PyTorch, where \(X=Squash(f)\) as \(X=\frac{\|f\|^{2}}{1+\|f\|^{2}}\frac{f}{\|f\|^{\prime}}\), and \(\|f\|\) denotes the module of \(f\).  
```
Input:\(f_{i}\), \(f_{c}\), \(\lambda\) Output:\(X\)
1 def forward(\(f_{i}\), \(f_{c}\), \(\lambda\)):
2\(X=\text{Squash}(\text{f}_{i}+\text{f}_{c})\);
3\(n=\text{X}.shape[2]-1\);
4\(d=(X-X.mean(\text{dim}=[2])).pow(2)\);
5\(v=\text{d}.sum(\text{dim}=[2])/n\);
6E_inv = d / \((4*(v+\text{lambda}))+0.5\); returnX* sigmoid(E_inv)+X;
```  
**Algorithm 1**A PyTorch-like implementation of our ResSimAM  
The Squash operation enables a nonlinear feature transformation by squeezing the input features to a certain range without changing the direction in the vector space. This squeezing helps to preserve the directional information of the feature vectors while eliminating the effect of magnitude, thereby allowing the network to better learn the structure and patterns in the data and be able to better preserve the relationships between the feature vectors. The feature squeezing operation makes each feature vector have a unit length, which helps with better similarity computation between the vectors, and by normalizing the vectors to a unit length, the magnitude difference between the vectors can be reduced, which helps in reducing the numerical instability due to the change in the length of the vectors for the subsequent similarity matching of the features and improves the generalization ability of the network.  
It is worth mentioning that Algorithm 1 does not introduce any additional parameter and can therefore work well in a nonparametric network. The energy function involved in the algorithm only requires computing the mean and variance of features, which are then brought into the energy function for calculation. This allows for the computation of weights to be completed in linear time.