[20] H. Li, S. De, Z. Xu, C. Studer, H. Samet, and T. Goldstein. Training quantized nets: A deeper understanding. NeurIPS - Advances in Neural Information Processing Systems, 30, 2017.
[21] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
[22] Y. Luo and S. Fan. Min-max-plus neural networks. arXiv preprint arXiv:2102.06358, 2021.
[23] P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu. Mixed precision training. In ICLR - International Conference on Learning Representations, 2018.
[24] J. N. Mitchell. Computer multiplication and division using binary logarithms. IRE Transactions on Electronic Computers, EC-11(4):512-517, 1962. doi: 10.1109/TEC.1962.5219391.
[25] T. Mogami. Deep neural network training without multiplications. In Beyond BackPropagation WS at 34th Conference on Neural Information Processing Systems, 2020.
[26] Nvidia. Nvidia Tesla V100 GPU Architecture. In Nvidia Tesla V100 GPU Architecture, 2017.
[27] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019.
[28] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS - Advances in Neural Information Processing Systems 32, pages 8024-8035. 2019.
[29] C. Sakr, N. Wang, C.-Y. Chen, J. Choi, A. Agrawal, N. Shanbhag, and K. Gopalakrishnan. Accumulation bit-width scaling for ultra-low precision training of deep networks. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bk1MjsRqY7.
[30] H. Shu, J. Wang, H. Chen, L. Li, Y. Yang, and Y. Wang. Adder attention for vision transformer. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, NeurIPS - Advances in Neural Information Processing Systems, volume 34, pages 1989-1990, 2021. URL https://proceedings.neurips.cc/paper/2021/file/a57e8915461b83adefb011530b711704-Paper.pdf.
[31] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[32] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou. Training data-efficient image transformers & distillation through attention. In ICML - International Conference on Machine Learning, pages 10347-10357. PMLR, 2021.
[33] A. Trockman and J. Z. Kolter. Patches are all you need? arXiv preprint arXiv:2201.09792, 2022.
[34] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. NeurIPS - Advances in neural information processing systems, 30, 2017.
[35] N. Wang, J. Choi, D. Brand, C.-Y. Chen, and K. Gopalakrishnan. Training deep neural networks with 8-bit floating point numbers. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper/2018/file/335d3d1cd7ef05ec77714a215134914c-Paper.pdf.
[36] R. Wightman. Pytorch image models. https://github.com/rwrightman/pytorch-image-models, 2019.
[37] S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492-1500, 2017.
[38] O. Yildüz. Training methodology for a multiplication free implementable operator based neural networks. Master's thesis, Middle East Technical University, 2017. URL https://hdl.handle.net/11511/26664.