[76] R. Wang, D. Tang, N. Duan, Z. Wei, X. Huang, J. Ji, G. Cao, D. Jiang, and M. Zhou. K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findings of ACL, pages 1405-1418. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021-findings-acl.121. URL https://doi.org/10.18653/v1/2021_findings-acl.121. (Cited on page 1)
[77] Z. Wang, J. Wohlwend, and T. Lei. Structured pruning of large language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.emnlp-main.496. URL https://doi.org/10.18653%2Fv1%2F2020.emnlp-main.496. (Cited on page 1)
[78] G. I. Winata, A. Madotto, J. Shin, E. J. Barezi, and P. Fung. On the effectiveness of low-rank matrix factorization for LSTM model compression, 2019. URL https://arxiv.org/abs/1908.09982. (Cited on page 1)
[79] R. Witten and E. Candès. Randomized algorithms for low-rank matrix factorizations: Sharp performance bounds. Algorithmica, 72(1):264-281, may 2015. ISSN 0178-4617. doi: 10. 1007/s00453-014-9891-7. URL https://doi.org/10.1007/s00453-014-9891-7. (Cited on page 2)
[80] T. Wolf, L. Debut, V. Sanh, J. Chaumont, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. (Cited on page 7)
[81] Z. Yao, R. Yazdani Aminabadi, M. Zhang, X. Wu, C. Li, and Y. He. ZeroQuant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168-27183, 2022. (Cited on page 7)
[82] T. Ye and S. S. Du. Global convergence of gradient descent for asymmetric low-rank matrix factorization. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 1429-1439. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/0af854284f4ab0cfea8fcfd889cbb41a-Paper.pdf. (Cited on page 2)
[83] S. Young, Z. Wang, D. Taubman, and B. Girod. Transform quantization for CNN compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-1, 2021. doi: 10.1109/tpami.2021.3084839. URL https://doi.org/10.11092%Ftpami.2021.3084839. (Cited on page 3)
[84] X. Yu, T. Liu, X. Wang, and D. Tao. On compressing deep models by low rank and sparse decomposition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 67–76, 2017. doi: 10.1109/CVPR.2017.15. (Cited on page 1)
[85] H. Zhang, Y. Bi, and J. Lavaei. General low-rank matrix optimization: Geometric analysis and sharper bounds. In Advances in Neural Information Processing Systems, volume 34, pages 27369-27380. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/e60e81c4cbe5171cd654662d9887aec2-Paper.pdf. (Cited on page 2)
[86] Y. Zhang, Z. Jiang, and L. S. Davis. Learning structured low-rank representations for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 676-683, 2013. (Cited on page 7)
[87] X. Zhou, C. Yang, H. Zhao, and W. Yu. Low-rank modeling and its applications in image analysis. ACM Computing Surveys (CSUR), 47(2):1-33, 2014. (Cited on page 7)