Table 7: Node classification accuracy on ogbn-arxiv-TA, Books-Children, Books-History and ElePhoto. We bold the best results for each row.
Methods	ogbn-arxiv-TA
Small Scale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	70.83	71.26	72.50	72.67	72.96	73.10	73.82	72.42	73.24	73.83	74.57
GNNs	GCN	72.03	68.45	73.39	70.81	73.30	73.56	68.15	70.44	73.25	73.95	72.57
GAT	72.25	70.97	73.48	71.67	73.40	73.38	66.56	71.01	73.37	73.72	71.50
SAGE	72.35	69.63	74.48	70.82	74.14	74.52	67.58	70.72	74.15	74.64	73.22
RevGAT	72.52	71.12	74.68	71.96	74.59	74.82	68.26	72.56	74.68	74.99	73.59
NFormer	71.91	69.45	73.56	70.43	72.80	73.12	67.11	70.04	73.12	73.12	71.88
GIN	68.42	58.09	72.30	64.88	71.94	72.63	62.05	64.47	71.88	73.10	71.25
JKNet	69.50	62.87	71.44	63.41	70.08	69.40	44.16	58.34	68.70	68.10	54.41
APPNP	71.63	59.55	74.01	65.62	73.90	74.01	52.37	64.52	73.53	74.17	69.28
MoNet	45.13	36.65	50.51	38.91	46.90	44.53	29.67	37.26	43.31	44.01	33.53
MLP	57.22	36.58	68.11	48.56	67.35	69.31	47.07	47.72	66.85	69.51	66.28
Co-Training	GCN	69.22	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	73.57	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	70.83	72.65	73.53	73.51	73.97	74.25	74.26	74.76	75.01	75.18	75.92
TDK	71.50	72.83	74.38	74.33	74.23	74.57	75.01	73.82	74.31	74.58	75.20
TCL	71.55	73.06	74.89	74.26	74.87	75.37	75.15	74.17	75.15	75.48	75.58
TMDC	71.17	73.71	75.50	75.56	76.11	75.97	75.99	75.58	75.75	75.73	76.20
Methods	Books-Children
SMALLScale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	49.85	57.03	59.90	59.09	59.91	59.80	60.26	58.28	60.65	60.93	61.61
GNNs	GCN	57.07	54.35	58.19	55.31	58.11	58.62	50.72	54.66	57.70	57.11	54.89
GAT	57.22	56.18	57.91	55.89	57.70	57.83	47.63	55.72	57.50	57.35	55.45
SAGE	57.57	55.32	59.33	55.84	58.74	58.97	49.61	55.52	58.40	58.21	56.29
RevGAT	57.86	56.42	59.28	56.42	58.67	59.01	49.63	56.59	58.90	59.26	56.34
NFormer	56.89	55.12	58.03	55.12	57.42	57.26	48.89	54.59	57.10	56.43	54.48
GIN	53.12	47.26	55.86	50.45	55.62	55.62	47.08	49.85	55.22	55.37	51.90
JKNet	53.48	48.36	51.25	45.90	52.33	49.12	34.19	42.56	51.18	44.47	36.89
APPNP	56.19	49.63	57.83	52.42	57.73	57.49	41.13	50.24	57.76	54.73	46.51
MoNet	36.81	35.18	37.57	34.87	36.43	36.02	32.29	34.22	35.60	34.70	33.16
MLP	48.34	40.33	53.18	43.14	52.55	54.57	43.55	43.11	52.43	52.61	48.55
Co-Training	GCN	54.75	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	59.70	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	54.27	57.35	60.03	59.93	60.34	60.19	60.73	60.51	60.84	62.11	61.91
TDK	53.43	56.92	60.23	60.27	60.43	60.71	61.39	59.31	61.15	61.95	62.51
TCL	54.11	56.88	60.60	60.21	60.73	61.47	61.92	59.29	61.50	62.06	62.37
TMDC	54.66	58.55	61.38	60.83	61.43	61.83	62.20	61.31	62.06	63.24	62.46
Methods	SMALLScale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	83.06	84.18	85.81	85.64	86.09	85.85	86.16	86.13	86.24	86.41	87.00
GNNs	GCN	84.52	82.08	85.14	82.46	85.04	85.15	82.31	82.54	84.95	84.51	84.22
GAT	84.21	82.85	84.68	82.97	84.49	84.76	80.71	83.10	84.36	84.47	83.78
SAGE	84.79	82.12	85.56	82.53	85.12	85.47	82.00	82.45	85.08	84.92	84.51
RevGAT	85.03	83.11	85.67	83.79	85.26	85.69	81.98	83.56	85.15	85.23	84.89
NFormer	83.59	80.96	84.49	81.16	84.59	84.46	81.46	80.15	84.29	84.23	82.99
GIN	82.62	73.69	83.60	76.29	83.19	84.01	79.81	76.47	83.29	83.34	82.89
JKNet	82.97	80.25	84.01	79.88	83.53	83.31	69.26	76.77	83.45	82.17	77.36
APPNP	84.31	78.65	85.49	79.91	85.28	85.35	78.16	79.18	84.97	84.86	82.68
MoNet	71.24	69.67	70.41	66.17	71.28	72.72	59.17	60.48	70.66	66.26	60.79
MLP	79.86	64.36	83.00	68.09	82.84	83.78	74.60	68.88	83.18	82.73	80.46
Co-Training	GCN	83.52	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	85.09	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	85.76	84.54	85.76	85.77	86.24	86.19	86.00	86.39	86.47	86.72	87.11
TDK	85.79	84.42	86.29	85.88	86.46	86.32	86.46	86.49	86.73	86.75	87.26
TCL	86.06	84.57	86.28	86.62	86.80	86.95	87.01	86.82	86.93	87.11	87.30
TMDC	86.88	85.18	86.88	86.41	86.82	86.96	86.94	86.28	86.94	87.22	87.32
Methods	Ele-Photo
SMALLScale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	73.75	76.58	77.51	77.84	77.53	78.11	78.37	77.25	77.72	79.60	79.63
GNNs	GCN	82.42	78.86	82.91	79.99	82.70	82.99	80.07	79.20	82.01	83.82	80.76
GAT	83.82	82.83	83.75	82.82	83.74	83.99	79.47	83.00	83.48	83.97	82.55
SAGE	83.25	80.90	83.50	81.79	83.27	83.81	81.44	81.05	82.77	84.15	81.88
RevGAT	84.12	83.12	84.34	82.98	84.46	84.59	80.98	82.59	84.21	85.12	81.12
NFormer	79.98	80.45	82.69	80.02	81.79	82.44	79.66	79.96	81.23	82.96	80.62
GIN	76.09	64.89	77.22	68.56	76.55	77.76	69.91	66.98	75.32	79.37	70.93
JKNet	79.68	75.29	80.41	76.53	79.72	79.13	60.13	74.35	79.18	78.61	69.57
APPNP	79.24	70.77	81.45	73.56	80.68	81.82	67.52	72.89	79.21	82.02	76.86
MoNet	76.24	66.55	72.84	68.89	73.17	73.67	57.48	66.24	71.08	73.57	61.87
MLP	58.43	47.69	64.47	51.24	62.51	65.64	54.12	49.98	60.88	66.26	58.76
Co-Training	GCN	83.32	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	86.64	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	74.30	76.09	77.81	78.27	78.54	78.33	79.27	79.21	78.95	80.32	80.45
TDK	73.99	76.89	79.69	80.18	81.04	81.26	81.34	78.44	79.26	80.82	81.33
TCL	73.86	77.74	81.85	81.47	82.85	82.47	83.07	79.56	80.74	81.47	82.33
TMDC	74.92	77.83	82.52	82.82	84.09	83.04	83.80	81.32	81.14	82.55	82.70