[4] M. Belbahri, E. Sari, S. Darabi, and V. P. Nia. Foothill: A quasiconvex regularization for edge computing of deep neural networks. In F. Karray, A. Campilho, and A. C. H. Yu, editors, International Conference on Image Analysis and Recognition, 2019.
[5] Y. Bengio, N. Léonard, and A. C. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR, 2013.
[6] J. Chang, x. zhang, Y. Guo, G. MENG, S. XIANG, and C. Pan. Data: Differentiable architecture approximation. In Advances in Neural Information Processing Systems, 2019.
[7] B. Chen, T. Dao, K. Liang, J. Yang, Z. Song, A. Rudra, and C. Ré. Pixelated butterfly: Simple and efficient sparse training for neural network models. In International Conference on Learning Representations, 2022.
[8] M. Cho, K. Alizadeh-Vahid, S. Adya, and M. Rastegari. DKM: differentiable k-means clustering layer for neural network compression. In International Conference on Learning Representations, 2022.
[9] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, 2019.
[11] S. Elkerdawy, M. Elhoushi, H. Zhang, and N. Ray. Fire together wire together: A dynamic pruning approach with self-supervised mask prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[12] U. Evci, T. Gale, J. Menick, P. S. Castro, and E. Elsen. Rigging the lottery: Making all tickets winners. In International Conference on Machine Learning, 2020.
[13] J. Frankle and M. Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019.
[14] E. Frantar and D. Alistarh. SPDY: Accurate pruning with speedup guarantees. In International Conference on Machine Learning, 2022.
[15] T. Gale, E. Olsen, and S. Hooker. The state of sparsity in deep neural networks. CoRR, 2019.
[16] S. Guo, Y. Wang, Q. Li, and J. Yan. Dmcp: Differentiable markov channel pruning for neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020.
[17] Y. Guo, A. Yao, and Y. Chen. Dynamic network surgery for efficient dnns. In Advances in Neural Information Processing Systems, 2016.
[18] S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2016.
[19] S. Han, J. Pool, J. Tran, and W. J. Dally. Learning both weights and connections for efficient neural network. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, 2015.
[20] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[21] Y. He, X. Zhang, and J. Sun. Channel pruning for accelerating very deep neural networks. In The IEEE International Conference on Computer Vision, Oct 2017.
[22] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.
[23] B. H. J. Lee, D. Kim. Network quantization with element-wise gradient scaling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[24] C. Y. Jeff Pool. Channel permutations for n:m sparsity. In Advances in Neural Information Processing Systems, 2021.
[25] M. Kang and B. Han. Operation-aware soft channel pruning using differentiable masks. In International Conference on Machine Learning, 2020.
[26] M. Kang and B. Han. Operation-aware soft channel pruning using differentiable masks. In International Conference on Machine Learning, 2020.
[27] E. Kurtic, D. Campos, T. Nguyen, E. Frantar, M. Kurtz, B. Fineran, M. Goin, and D. Alistarh. The optimal bert surgeon: Scalable and accurate second-order pruning for large language models. arXiv preprint arXiv:2203.07259, 2022.
[28] A. Kusupati, V. Ramanujan, R. Somani, M. Wortsman, P. Jain, S. Kakade, and A. Farhadi. Soft threshold weight reparameterization for learnable sparsity. In International Conference on Machine Learning, July 2020.