[49] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
[50] J. Pan, T. Gao, H. Chen, and D. Chen. What in-context learning "learns" in-context: Disentangling task recognition and task learning. In Annual Meeting of the Association for Computational Linguistics, 2023.
[51] J. Pennington, R. Socher, and C. Manning. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1162.
[52] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana, 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-1202.
[53] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2020.
[54] A. Ramdas, N. García Trillos, and M. Cuturi. On wasserstein two-sample testing and related families of nonparametric tests. Entropy, 19(2):47, 2017.
[55] O. Sainz, O. L. de Lacalle, E. Agirre, and G. Rigau. What do language models know about word senses? zero-shot wsd with language models and domain inventories. ArXiv preprint, abs/2302.03353, 2023.
[56] V. Sanh, L. Debut, J. Chaumont, and T. Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. ArXiv preprint, abs/1910.01108, 2019.
[57] T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilic, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Galle, et al. Bloom: A 176b-parameter open-access multilingual language model. ArXiv preprint, abs/2211.05100, 2022.
[58] J. Shlens. A tutorial on principal component analysis. arXiv:1404.1100, 2014.
[59] M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. ArXiv preprint, abs/1909.08053, 2019.
[60] S. M. Smith. The constraining effects of initial ideas. Group creativity: Innovation through collaboration, pages 15-31, 2003.
[61] S. M. Smith and S. E. Blankenship. Incubation effects. Bulletin of the Psychonomic Society, 27 (4):311-314, 1989.
[62] S. M. Smith and S. E. Blankenship. Incubation and the persistence of fixation in problem solving. The American journal of psychology, pages 61-87, 1991.
[63] S. M. Smith and D. R. Tindell. Memory blocks in word fragment completion caused by involuntary retrieval of orthographically related primes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(2):355, 1997.
[64] K. Song, X. Tan, T. Qin, J. Lu, and T. Liu. Mpnet: Masked and permuted pre-training for language understanding. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
[65] Y. Song, C. Cui, S. Khanuja, P. Liu, F. Faisal, A. Ostapenko, G. I. Winata, A. F. Aji, S. Cahyawijaya, Y. Tsvetkov, et al. GlobalBench: A benchmark for global progress in natural language processing. ArXiv preprint, abs/2305.14716, 2023.