[22] Nora Kassner and Hinrich Schütze. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In ACL, 2020.
[23] Lisa Anne Hendricks, John Mellor, Rosalia Schneider, Jean-Baptiste Alayrac, et al. Decoupling the role of data, attention, and losses in multimodal transformers. TACL, 2021. 2
[24] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, et al. Emerging properties in self-supervised vision transformers. In ICCV, 2021. 2
[25] Kaiming He, Xinlei Chen, Saining Xie, et al. Masked autoencoders are scalable vision learners. In CVPR, 2022. 4
[26] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV, 2021.
[27] Chunyuan Li, Jianwei Yang, Pengchuan Zhang, Mei Gao, Bin Xiao, et al. Efficient self-supervised vision transformers for representation learning. In ICLR, 2022.
[28] Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, and Matthew J. Kusner. Unsupervised point cloud pre-training via occlusion completion. In ICCV, 2021. 2
[29] Shengchao Liu, David Vazquez, Jian Tang, and Pierre-Andre Noel. Flaky performances when pretraining on relational databases. arXiv:2211.05213, 2022. 2
[30] Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, et al. Evaluating protein transfer learning with tape. In NeurIPS, 2019. 2
[31] Alexander Rives, Joshua Meier, Tom Sercu, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS, 2021.
[32] Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, et al. Protrans: towards cracking the language of life's code through self-supervised deep learning and high performance computing. IEEE TPAMI, 2021. 2
[33] Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. Ogb-lsc: A large-scale challenge for machine learning on graphs. In NeurIPS Dataset and Benchmark Track, 2021. 2
[34] Qinkai Zheng, Xu Zou, Yuxiao Dong, Yukuo Cen, Da Yin, Jiarong Xu, Yang Yang, and Jie Tang. Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning. In NeurIPS Dataset and Benchmark Track, 2021.
[35] Yuanqi Du, Shiyu Wang, Xiaojie Guo, Hengning Cao, Shujie Hu, Junji Jiang, Aishwarya Varala, Abhinav Angirekula, and Liang Zhao. Graphgt: Machine learning datasets for graph generation and transformation. In NeurIPS Dataset and Benchmark Track, 2021.
[36] Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: A graph out-of-distribution benchmark. In NeurIPS Dataset and Benchmark Track, 2022.
[37] Yijian Qin, Ziwei Zhang, Xin Wang, Zeyang Zhang, and Wenwu Zhu. Nas-bench-graph: Benchmarking graph neural architecture search. In NeurIPS Dataset and Benchmark Track, 2022. 2
[38] David Duvenaud, Dougal Maclaurin, Jorge A.-Iparraguirre, Rafael Gomez-Bombarelli, et al. Convolutional networks on graphs for learning molecular fingerprints. In NIPS, 2015. 2
[39] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In ICML, 2017. 2
[40] Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, et al. Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling, 2019.
[41] Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Lio, and Petar Velicković. Principal neighbourhood aggregation for graph nets. In NeurIPS, 2020. 2