Table 8: Node classification f1 score on ogbn-arxiv-TA, Books-Children, Ele-Photo, and Sports-Fitness. We bold the best results for each row.
Methods	ogbn-arxiv-TA
Small Scale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	50.48	52.15	53.01	54.81	54.39	55.41	56.40	55.71	55.88	57.52	57.64
GNNs	GCN	51.38	44.40	54.08	48.91	53.44	54.32	32.38	47.17	53.14	54.35	50.97
GAT	52.26	49.33	53.97	51.41	54.11	53.61	41.04	53.79	50.62	53.88	51.32
SAGE	51.81	45.58	55.22	49.38	54.57	55.86	38.56	48.37	54.39	55.79	52.41
RevGAT	52.34	49.88	55.12	52.21	54.68	55.91	42.06	54.11	55.21	56.04	53.01
NFormer	49.95	45.56	51.02	47.75	51.16	51.58	39.55	41.12	50.03	51.16	47.89
GIN	46.92	22.56	52.28	41.88	52.24	52.31	32.76	39.14	51.39	52.58	49.88
JKNet	47.10	36.82	47.07	37.25	48.49	45.75	29.56	31.27	44.54	43.29	28.55
APPNP	50.63	33.67	54.32	41.50	54.35	54.26	40.90	40.58	53.68	54.12	42.69
MoNet	38.62	20.15	35.26	24.56	36.57	39.22	28.45	20.12	31.26	34.28	26.51
MLP	34.71	15.35	47.37	25.33	45.88	48.05	24.39	45.58	23.38	48.37	43.03
Co-Training	GCN	51.55	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	52.76	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	50.81	52.43	54.41	55.02	54.89	56.67	56.92	55.83	56.42	58.60	58.73
TDK	50.96	52.88	54.96	55.52	55.21	57.03	57.23	56.22	56.96	59.44	59.59
TCL	51.46	53.11	55.36	56.01	55.96	57.68	57.96	57.03	57.69	60.02	60.13
TMDC	51.79	53.45	55.96	56.45	56.39	58.11	58.45	57.64	58.12	60.56	60.78
Methods	Books-Children
SMall Scale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	26.26	38.09	46.17	46.44	47.55	48.25	49.54	40.70	50.51	50.67	51.34
GNNs	GCN	51.38	44.40	54.08	41.90	46.35	46.99	33.97	41.57	46.16	45.89	40.93
GAT	52.26	49.33	53.97	45.60	46.55	48.54	29.88	46.16	48.46	45.98	43.46
SAGE	51.81	45.58	55.22	43.49	47.31	48.77	33.30	43.60	46.74	47.77	42.94
RevGAT	52.35	49.96	55.52	45.78	47.56	49.01	33.41	46.26	48.59	47.82	46.29
NFormer	50.30	43.32	51.63	42.23	44.43	44.12	28.55	40.06	44.15	44.26	40.96
GIN	46.92	22.56	52.28	38.16	44.21	44.92	27.92	37.56	44.20	43.22	41.09
JKNet	47.10	36.82	47.07	30.15	38.92	30.35	7.98	23.14	36.22	26.56	13.54
APPNP	50.63	33.67	54.32	37.53	46.08	44.22	16.16	32.00	45.46	40.95	23.53
MoNet	36.89	21.56	43.38	28.56	34.45	36.51	23.65	31.12	36.64	37.99	29.92
MLP	34.71	15.35	47.37	20.75	34.88	38.87	22.54	21.14	34.15	36.85	29.97
Co-Training	GCN	52.03	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	52.96	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	36.47	42.04	48.16	44.79	47.87	48.37	49.42	46.75	48.89	51.80	51.59
TDK	33.22	40.26	46.80	47.16	49.40	49.42	51.84	45.82	50.06	52.07	52.21
TCL	34.15	40.43	48.46	42.93	49.77	49.72	51.85	44.59	50.62	52.63	51.85
TMDC	35.78	44.21	49.31	46.96	50.89	50.09	52.38	49.96	51.21	52.55	52.28
Methods	Ele-Photo
SMall Scale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	66.11	69.31	70.82	70.48	70.04	71.21	72.10	67.97	70.96	73.69	74.30
GNNs	GCN	75.03	69.32	76.44	72.12	75.38	76.49	70.22	70.90	74.91	76.84	72.93
GAT	77.24	76.22	77.26	76.54	76.97	77.30	73.02	75.99	76.77	77.47	75.66
SAGE	75.75	72.97	76.22	73.96	75.26	76.71	73.25	73.53	74.86	77.31	74.11
RevGAT	77.34	76.55	77.31	76.81	77.23	77.35	73.55	76.12	76.94	77.51	76.21
Nformer	72.26	68.12	73.22	69.57	72.15	73.28	68.12	73.49	73.56	74.45	70.12
GIN	67.95	53.35	68.75	60.26	69.45	63.50	51.77	55.90	62.82	70.48	57.46
JKNet	69.99	64.72	69.92	65.5	69.5	67.02	40.33	62.51	69.72	68.11	54.92
APPNP	69.86	56.59	72.84	61.91	71.53	73.37	48.02	59.36	68.74	73.88	63.26
MoNet	45.51	44.12	51.12	48.86	52.26	55.51	34.56	42.56	53.35	57.11	38.69
MLP	39.60	14.98	48.32	24.56	44.86	50.65	29.30	23.25	41.91	51.75	37.18
Co-Training	GCN	74.59	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	76.98	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	65.19	66.62	71.50	70.94	71.34	71.76	72.27	72.77	73.23	73.96	74.61
TDK	66.15	68.08	72.92	73.60	74.98	75.08	75.84	73.79	71.71	74.85	75.85
TCL	64.94	69.34	75.03	74.93	76.59	76.62	76.92	71.95	75.05	75.48	76.95
TMDC	66.23	70.02	76.82	76.97	78.18	77.45	78.69	74.47	75.27	77.08	76.10
Methods	Sports-Fitness
SMall Scale	Base Scale	Large Scale
BERT-Tiny	ELECTRA	DistilBERT	ELECTRA	BERT	RoBERTa	DeBERTa	ELECTRA	BERT	RoBERTa	DeBERTa
LMs	PLM	81.47	76.58	85.31	83.94	86.02	83.04	84.28	76.85	85.06	84.16	86.21
GNNs	GCN	84.93	82.24	86.15	83.46	86.16	85.83	77.96	83.54	85.70	86.07	83.71
GAT	86.45	85.46	86.58	85.57	86.65	86.26	74.23	86.85	85.80	85.77	84.56
SAGE	87.06	85.06	87.51	85.70	87.34	87.39	76.86	85.86	87.46	87.58	85.60
RevGAT	87.55	85.96	87.88	85.89	87.46	87.56	77.79	87.02	87.68	87.96	86.62
Nformer	83.69	81.15	84.56	82.20	84.42	84.66	74.12	82.26	84.49	85.02	82.26
GIN	81.59	71.31	74.91	71.51	81.95	72.47	67.06	71.80	68.77	83.83	79.86
JKNet	80.70	73.56	75.69	71.70	80.31	74.37	15.38	66.20	80.96	77.24	48.73
APPNP	83.62	71.84	83.54	73.10	84.59	82.61	36.81	73.37	85.18	84.31	77.37
MoNet	59.95	45.56	68.89	51.23	69.02	69.56	55.56	58.59	70.01	72.12	65.13
MLP	68.47	43.66	76.09	49.56	73.69	74.90	52.86	56.33	74.48	77.37	71.41
Co-Training	GCN	85.06	83.21	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
SAGE	85.87	86.21	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
TPT	TMLM	81.56	80.15	86.22	84.45	86.35	83.95	85.16	79.89	85.45	84.85	86.51
TDK	81.40	82.33	86.48	85.51	86.88	84.92	86.04	80.47	85.65	85.50	86.75
TCL	82.04	83.20	87.44	86.72	88.08	86.67	87.29	82.05	86.64	85.80	87.57
TMDC	83.15	84.21	88.56	87.69	89.01	88.89	88.56	84.51	87.89	86.43	88.78