SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To verify our guess, in this section, we conducted a series of ablation experiments on ModelNet40. We trained the Spiking PointNet with 4 time steps and evaluated its accuracy at every time step and all time steps respectively. The results are shown in Tab. 3. It can be seen that, the collective results outperform those obtained from individual steps, implying that the performance improvement associated with larger time steps might be more related to an ensemble learning effect, rather than a direct result of the increased time steps. In specific, the Spiking PointNet at each time step can be seen as an independent model casting a vote towards the final prediction. This ensemble learning strategy increases the robustness of the model and subsequently improves the prediction accuracy. Our study suggests that a rethinking and optimization of time steps in SNNs is warranted. The inherent ensemble learning effect, which is underappreciated in the conventional SNN design, could be a viable strategy to enhance the performance of SNNs, while also managing computational resources. Our insights provide valuable implications for future design and optimization strategies in the field of SNNs.
Table 3: The verification test for the effect of the time step on the static dataset.
1-th time step	2-th time step	3-th time step	4-th time step	Averaging all
Accuracy	83.70%	84.65%	85.70%	85.29%	86.70%
Under the perspective that the residual membrane potential of SNN, coming from the previous time step cannot transmit the temporal information for static point cloud datasets but a perturbation to increase the generalization, we further propose a membrane potential perturbation method for the framework. In specific, we add some membrane potential perturbation randomly to initial the spiking neurons of the Spiking PointNet at each epoch in the training phase, thus the generalization of the model trained with only 1 single time step will be improved like those trained with multiple time steps. The results for the trained-less-based Spiking PointNet with membrane potential perturbation are shown in Tab. 4. It can be seen that with the perturbation method, the Spiking PointNet further gets another performance lift, amounting to  $93.31\%$  and  $88.61\%$  final accuracy for ModelNet10 and ModelNet40 respectively.
Table 4: The ablation study for the membrane potential perturbation.
Dataset	Method	Training: 1 T
Inferring: 1 T	Inferring: 2 T	Inferring: 3 T	Inferring: 4 T
ModelNet10	without MPP	91.99%	92.43%	92.53%	92.32%
with MPP	91.66%	92.98%	92.98%	93.31%
ModelNet40	without MPP	86.98%	87.26%	87.21%	87.13%
with MPP	87.72%	88.46%	88.25%	88.61%
MPP denotes membrane potential perturbation.
In this section, we conduct extensive experiments on ModelNet10 and ModelNet40 [50] to demonstrate the superior performance of our method. ModelNet10 and ModelNet40 are two widely recognized public datasets used for 3D object classification, curated and maintained by a research team at Princeton University. ModelNet10 is a compact dataset comprising 4,899 3D models that span 10 distinct categories such as tables, chairs, bathtubs, and guitars. This dataset is a subset of ModelNet40, offering fewer categories but with more pronounced differences between each category. This characteristic makes ModelNet10 an excellent starting point for evaluating the performance of 3D classification algorithms. ModelNet40 is a more comprehensive dataset, containing approximately 12,311 3D models across 40 different categories, including tables, chairs, airplanes, guitars, and more. With an expanded array of categories and samples, ModelNet40 serves as a robust benchmark for gauging the performance of 3D classification algorithms in more complex and challenging tasks. We leverage the PointNet architecture for point cloud classification tasks. For all our SNN models, we set  $V_{\mathrm{th}}$  as 0.5, The initial perturbations,  $\delta$ , range from 0 to 0.5.
We first conducted thorough ablation experiments of our method against the vanilla SNN for PointNet on the ModelNet10/40 datasets. The Tab. 5 displays the performances of various methods under different training and testing time steps. On the ModelNet10 dataset, our Spiking PointNet with membrane potential perturbation (MPP) reaches an accuracy of  $93.31\%$  with a testing time step of 4, which outperforms both the one without MPP  $(92.32\%)$  and the ANN-based approach  $(92.98\%)$ . Even with a testing time step of 1, our Spiking PointNet with MPP still achieves an accuracy of  $91.66\%$ , surpassing the performance of vanilla Spiking PointNet trained with 4 time steps  $(89.62\%)$ . This validates the effectiveness of our method. Further, on the ModelNet40 dataset, our Spiking PointNet with MPP attains an accuracy of  $88.61\%$  with a testing time step of 4, also outperforming
Table 5: Comparison between our method and the vanilla SNN on ModelNet10/40 datasets.
Datasets	Methods	Training time steps	Testing time steps
1	2	3	4
ModelNet10	ANN	-	92.98%
Vanilla SNN	4	89.62%	90.83%	91.05%	91.05%
Ours without MPP	1	91.99%	92.43%	92.53%	92.32%
Ours with MPP	1	91.66%	92.98%	92.98%	93.31%
ModelNet40	ANN	-	89.20%
Vanilla SNN	4	85.59%	86.58%	86.34%	86.70%
Ours without MPP	1	86.98%	87.26%	87.21%	87.13%
Ours with MPP	1	87.72%	88.46%	88.25%	88.61%
Table 6: Energy estimation of ANN (PointNet) and SNNs (Spiking PointNet) of computation.
Method	Time step	Acc.	#Add.	#Mult.	Energy
PointNet	-	92.98%	0.03M	13.94M	1.4 × 108pJ
Spiking PointNet	1	91.66%	0.45M	0.45M	9.2 × 106pJ
4	93.31%	1.8M	1.8M	3.7 × 107pJ
the one without MPP  $(87.13\%)$  and the vanilla Spiking PointNet  $(86.70\%)$ . Similarly, even with a testing time step of 1, our Spiking PointNet with MPP achieves an accuracy of  $87.72\%$ , still superior to the performance of the vanilla one trained with 4 time steps  $(85.59\%)$ .
4.2 Energy Efficiency
In this section, we conducted a comprehensive investigation into the hardware efficiency of our proposed framework, with a focus on quantifying energy consumption in computational tasks on ModelNet10. For an ANN model, the dot product operation, or Multiply-Accumulate (MAC) operation, involves both addition and multiplication operations. However, the SNN leverages the multiplication-addition transformation advantage, eliminating the need for multiplication operations in all layers except the first layer. Remarkably, in the absence of spikes, hardware can employ sparse computation to completely avoid addition operations. To estimate energy consumption, we adopted the methodology using  $45\mathrm{nm}$  CMOS technology following [26; 39]. The MAC operation in ANN consumes  $4.6\mathrm{pJ}$  of energy, while the accumulation operation in SNN requires only  $0.9\mathrm{pJ}$ . Notably, in line with our trained-less but learning-more paradigm, we achieved a spike firing rate of  $18.7\%$  with  $k = 5$ . Based on our findings, we computed the energy cost and presented the results in Tab. 6. Our network exhibits remarkable energy efficiency, necessitating only  $9.2 \times 10^{6}\mathrm{pJ}$  of energy per forward pass, which equates to a 15.2-fold reduction in comparison to conventional ANNs. Moreover, when we conduct inference in four time steps, the performance reaches  $93.31\%$ , while the energy required is merely about 3.8 times less than that of its ANN counterpart.
In this paper, we have presented Spiking PointNet, the first spiking neural network (SNN) specifically designed for efficient deep learning on point clouds. This work was motivated by the tremendous potential of SNNs in energy efficiency and the rising demand for efficient point cloud processing techniques, especially in fields such as autonomous driving and augmented reality. We identified two main challenges hindering the application of SNNs in point cloud tasks: the intrinsic optimization difficulty of SNNs, and the high computational and memory cost of point cloud processing, especially for large time steps. To address these obstacles, we proposed a novel trained-less but learning-more paradigm. This paradigm allows for the training of Spiking PointNet with only a single time step, but is capable of achieving superior performance through multiple time step inference. Theoretical justifications and experimental analysis provided in the paper support our method's effectiveness. Additionally, we introduced a membrane potential perturbation method, which significantly en
hanced the generalization ability of the Spiking PointNet without increasing computational and storage requirements. Our extensive experiments on multiple datasets, including ModelNet10 and ModelNet40, demonstrated the robustness and superiority of Spiking PointNet. Notably, in certain scenarios, Spiking PointNet was even able to outperform its Artificial Neural Network counterparts, an uncommon achievement in the SNN field.
This work is supported by grants from the National Natural Science Foundation of China under contracts No.12202412 and No.12202413.