[1] mteb/emotion - datasets at hugging face. https://huggingface.co/datasets/mteb/emotion, . (Accessed on 05/16/2023). (Cited on page 8)
[2] mteb/imdb - datasets at hugging face. https://huggingface.co/datasets/mteb/imdb? doi=true, . (Accessed on 05/16/2023). (Cited on page 8)
[3] A. Aghajanyan, L. Zettlemoyer, and S. Gupta. Intrinsic dimensionality explains the effectiveness of language model fine-tuning, 2020. URL https://arxiv.org/abs/2012.13255. (Cited on page 1)
[4] F. Alimisis, P. Davies, and D. Alistarh. Communication-efficient distributed optimization with quantized preconditioners. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 196-206. PMLR, 18-24 Jul 2021. URL https://proceedings.mlrpress/v139/alimisis21a.html. (Cited on page 2)
[5] D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems, volume 30, pages 1709-1720. Curran Associates, Inc., 2017. (Cited on page 3)
[6] N. Alon and V. H. Vu. Anti-hadamard matrices, coin weighing, threshold gates, and indecomposable hypergraphs. J. Comb. Theory Ser. A, 79(1):133-160, jul 1997. ISSN 0097-3165. doi: 10.1006/jcta.1997.2780. URL https://doi.org/10.1006/jcta.1997.2780. (Cited on page 5)
[7] Amazon AWS. Enterprise Search Engine - Amazon Kendra - AWS, 2023. URL https://aws.amazon.com/kendra/. (Accessed on 05/16/2023). (Cited on page 8)
[8] M. Ben Noach and Y. Goldberg. Compressing pre-trained language models by matrix decomposition. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 884-889, Suzhou, China, Dec. 2020. Association for Computational Linguistics. URL https://aclanthology.org/2020.aacl-main.88. (Cited on page 1)
[9] N. I. Bernardo, J. Zhu, Y. C. Eldar, and J. Evans. Hardware-limited non-uniform task-based quantizers. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, 2023. doi: 10.1109/ICASSP49357.2023.10094884. (Cited on page 3)
[10] W.-N. Chen, P. Kairouz, and A. Ozgur. Breaking the communication-privacy-accuracy trilemma. In Neural Information Processing Systems (NeurIPS) 2020, 2020. (Cited on page 3)
[11] Y. Chi, Y. M. Lu, and Y. Chen. Nonconvex optimization meets low-rank matrix factorization: An overview. IEEE Transactions on Signal Processing, 67(20):5239-5269, 2019. doi: 10.1109/TSP.2019.2937282. (Cited on page 2)
[12] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE, 2009. (Cited on page 8)
[13] M. Derezinski, F. T. Liang, Z. Liao, and M. W. Mahoney. Precise expressions for random projections: Low-rank approximation and randomized Newton. In Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/d40d35b3063c11244fbf38e9b55074be-Paper.pdf. (Cited on page 2)
[14] T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339, 2022. (Cited on pages 7 and 9)
[15] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. (Cited on page 9)