[44] Liangyu Chen, Xin Lu, J. Zhang, Xiaojie Chu, and Chengpeng Chen. Hinet: Half instance normalization network for image restoration. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 182-192, 2021.
[45] Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. Memnet: A persistent memory network for image restoration. 2017 IEEE International Conference on Computer Vision (ICCV), pages 4549-4557, 2017.
[46] Huishuai Zhang, Da Yu, Mingyang Yi, Wei Chen, and Tie-Yan Liu. Stabilize deep resnet with a sharp scaling factor tau. 2019.
[47] Huishuai Zhang, Da Yu, Wei Chen, and Tie-Yan Liu. On the stability of multi-branch network, 2021.
[48] Thomas C. Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, G. Cottrell, and Julian McAuley. Rezero is all you need: Fast convergence at large depth. In Conference on Uncertainty in Artificial Intelligence, 2020.
[49] Hongyi Zhang, Yann Dauphin, and Tengyu Ma. Fixup initialization: Residual learning without normalization. ArXiv, abs/1901.09321, 2019.
[50] Soham De and Samuel L. Smith. Batch normalization biases residual blocks towards the identity function in deep networks. arXiv: Learning, 2020.
[51] Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tie-Yan Liu. On layer normalization in the transformer architecture, 2020.
[52] Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andy Brock, Samuel L. Smith, and Yee Whye Teh. Deep transformers without shortcuts: Modifying self-attention for faithful signal propagation. ArXiv, abs/2302.10322, 2023.
[53] Soufiane Hayou, Eugenio Clerico, Bo He, George Deligiannidis, A. Doucet, and Judith Rousseau. Stable resnet. In International Conference on Artificial Intelligence and Statistics, 2020.
[54] Boris Hanin and David Rolnick. How to start training: The effect of initialization and architecture. Advances in Neural Information Processing Systems, 31, 2018.
[55] David Balduzzi, Marcus Frean, Lennox Leary, JP Lewis, Kurt Wan-Duo Ma, and Brian McWilliams. The shattered gradients problem: If resnets are the answer, then what is the question? In International Conference on Machine Learning, pages 342-350. PMLR, 2017.
[56] Muhammad Awais, Md. Tauhid Bin Iqbal, and Sung-Ho Bae. Revisiting internal covariate shift for batch normalization. IEEE Transactions on Neural Networks and Learning Systems, 32:5082-5092, 2020.
[57] You Huang and Yuanlong Yu. An internal covariate shift bounding algorithm for deep neural networks by unitizing layers' outputs. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8462-8470, 2020.
[58] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, 2015.
[59] Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does batch normalization help optimization? (no, it is not about internal covariate shift). ArXiv, abs/1805.11604, 2018.
[60] Tim Cooijmans, Nicolas Ballas, César Laurent, and Aaron C. Courville. Recurrent batch normalization. ArXiv, abs/1603.09025, 2016.
[61] Tong Wu, Ziwei Liu, Qingqiu Huang, Yu Wang, and Dahua Lin. Adversarial robustness under long-tailed distribution. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8655-8664, 2021.
[62] Zhuo Chen, Jinglong Chen, Yong Feng, Shen Liu, Tianci Zhang, Kaiyu Zhang, and Wenrong Xiao. Imbalance fault diagnosis under long-tailed distribution: Challenges, solutions and prospects. Knowl. Based Syst., 258:110008, 2022.
[63] Xiyu He and Xiang Qian. A real-time surface defect detection system for industrial products with long-tailed distribution. 2021 IEEE Industrial Electronics and Applications Conference (IEACon), pages 313-317, 2021.
[64] Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X. Yu. Long-tailed recognition by routing diverse distribution-aware experts. ArXiv, abs/2010.01809, 2020.