SlowGuess/NeurIPS_2023_OCR huggingface dataset:
AC/DC [40] achieves State-of-the-Art performance at high sparsities. Therefore, we hope to compare our method extensively with AC/DC on various settings. Accordingly, the schedule of AC/DC need slight modifications based on the original setting. According to the original work [40], dense and sparse status are alternated every 5 epochs. We scale the period according to the ratio of the training epochs versus the standard training epochs in the paper [40]. For example, if the AC/DC method is applied for 300 epochs, since the original ImageNet AC/DC setting has only 100 epochs, we lengthen the period of AC/DC from the original 5 epochs to 15 epochs. For warming-up and finetuning periods, the similar scaling is performed.
We also tried RPG pruning without loading pretrained models on ResNet-50 ImageNet experiments. Training settings are kept the same with other ResNet-50 pruning experiments. The results are provided in Table 8. The proposed RPG method could surpass all baselines even without loading pretrained models.
Algorithm	Sparsity	FLOPs	Accuracy
ResNet-50 [22]	0	1×	76.80
RPG from scratch	0.9	0.155×	75.35
RPG from scratch	0.95	0.093×	73.62
Table 8. RPG-Sparsified ResNet-50 from scratch on ImageNet.