SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We use three citation networks, labelled by CoraML, CiteSeer, Pubmed [4], two co-purchase Amazon datasets [31], labeled by Computers and Photos, two coauthor datasets [31], labeled by CoauthorCS and Physics, and a large dataset OGBN Arxiv [16]. We use the same train/val/test split of 5/15/80 as [32]. The details of the graphs and setups for the OOD detection are provided in Table 4.
Table 4: Dataset Description
CoraML	CiteSeer	PubMed	Computers	Photos	Coauthor CS	Coauthor Physics	OGBN-Arxiv
#nodes	2,995	4,230	19,717	13,752	7,650	18,333	34,493	169,343
#edges	16,316	10,674	88,648	491,722	238,162	163,788	495,924	2,315,598
#features	2879	602	500	767	745	6,805	8,415	128
#classes	7	6	3	10	8	15	5	40
# left-out-classes	3	2	1	5	3	4	2	15
D.3 Hyper-parameter tuning
We follow the same setting with [32]. In detail, we use the Adam optimizer with a learning rate of 0.01. For VGCN-Energy, we use a temperature of  $T = 1.0$ . We carefully tune three hyperparameters: the distance-based regularization weight, Dirichlet entropy weight, and activation functions. We select the best parameters for each dataset separately that returns the highest validation cross-entropy. The detailed hyperparameters configuration is as Table 5.
Table 5: Hyperparameter configurations of proposed model
Dirichlet Entropy Reg. Weight	Graph Distance Reg. Weight	Activation function
CoraML	0	10-4	GELU
CiteSeer	10-4	10-9.5	LogSigmoid
PubMed	10-5	10-4	RELU
Computers	10-5	10-4	RELU
Photos	10-5	10-11	RELU
Coauthor CS	0	10-6	RELU
Coauthor Physics	10-4	10-4.5	LogSigmoid
OGBN-Arxiv	10-5	10-8	RELU
We also consider the following activation functions in the encoding network with element-wise operations,
$$
\sigma_ {\text {R E L U}} (x) = \max  (0, x),
$$
$$
\sigma_ {\operatorname {L o g S i g m o i d}} (x) = \log \left(\left(1 + \exp (- x)\right) ^ {- 1}\right),
$$
$$
\sigma_ {\mathrm {G e L U}} (x) = x \mathrm {C D F} _ {\mathcal {N}} (x)
$$
$$
\sigma_ {\text {H a r d T a n h}} (x) = \left\{ \begin{array}{l l} - 1, x <   - 1 \\ x, - 1 \leq x \leq 1 \\ 1, x > 1 \end{array} . \right.
$$
ReLU is the most popular activation function used in the hidden layer of neural networks, which brings efficient computation by only activating neurons with positive outputs. Sigmoid is popularly used for probability prediction because its output is always in the range (0,1) with a smooth gradient. GeLU has better nonlinearity and is widely used in Natural Language processing and computer vision. HardTanh is a more computation-efficient version of Tanh.
E.1 Additional Experiments - OOD Detection
For Amazon Photos, Amazon Computers, Coauthor CS, Coauthor Physics, and OGBN Arxiv dataset, the OOD Detection results are shown in Table 6.
E.2 Additional Experiments - Misclassification Detection
For Amazon Photos, Amazon Computers, Coauthor CS, Coauthor Physics, and OGBN Arxiv dataset, the Misclassification Detection results are shown in Table 7.
E.3 Graph Distance Minimization
We plot the tSNE visualization of latent space with different distance-based regularization weights and symbol sizes denote the total evidence. We plot for coraML in Figure 2, CiteSeer in Figure 3, Coauthor CS in Figure 4, Coauthor Physics in Figure 5. With increasing weight, it tends to have a more separable latent representation for different categories while degenerate mappings occur when distance minimization is too large.
Table 6: OOD Detection (Cont.)
Data	Model	ID-ACC	AUROC	AUPR
Alea w/	Epi w/	Epi w/o	Alea w/	Epi w/	Epi w/o
Amazon Computers	LP	83.28	86.74	83.88	n.a.	67.10	63.08	n.a.
GKDE	71.41	75.14	73.58	n.a.	49.21	47.68	n.a.
VGCN-Energy	88.95	82.76	83.43	n.a.	57.49	60.64	n.a.
GKDE-GCN	82.73	77.03	70.32	n.a	49.81	45.92	n.a
GPN	88.48	82.49	87.63	74.55	56.78	67.94	48.03
Ours	89.88	83.56	89.26	71.82	58.51	71.06	43.35
Amazon Photos	LP	89.27	94.24	90.26	n.a.	90.24	85.55	n.a.
GKDE	85.94	76.51	60.83	n.a.	66.72	59.09	n.a.
VGCN-Energy	94.24	82.44	79.64	n.a.	72.60	71.71	n.a.
GKDE-GCN	89.84	73.65	69.09	n.a	62.45	59.68	n.a
GPN	94.10	82.72	91.98	76.57	74.55	86.29	64.00
Ours	94.40	83.51	92.30	78.10	77.65	87.36	65.39
Coauthor CS	LP	86.40	83.78	80.86	n.a.	74.8	71.15	n.a
GKDE	78.84	79.32	77.59	n.a.	66.30	64.69	n.a.
VGCN-Energy	93.07	85.35	87.33	n.a.	80.87	82.79	n.a.
GKDE-GCN	93.13	85.02	84.45	n.a.	80.15	77.90	n.a.
GPN	88.21	69.49	92.90	88.84	55.41	90.28	86.54
Ours	89.24	70.12	92.37	91.38	56.20	91.17	90.45
Coauthor Physics	LP	95.39	91.78	90.03	n.a.	70.58	69.63	n.a.
GKDE	93.30	87.02	84.64	n.a.	57.00	52.49	n.a.
VGCN-Energy	97.96	90.29	91.08	n.a.	63.63	69.41	n.a.
GKDE-GCN	97.95	87.38	84.62	n.a.	57.97	56.30	n.a.
GPN	97.40	85.20	94.51	89.63	61.89	83.73	66.44
Ours	97.44	85.28	94.42	90.36	62.80	83.61	70.62
OGBN Arxiv	LP	66.84	80.04	75.22	n.a.	65.21	67.69	n.a.
GKDE	51.51	68.12	65.80	n.a.	47.22	45.23	n.a.
VGCN-Energy	75.61	64.91	64.50	n.a.	42.72	42.41	n.a
GKDE-GCN	73.89	68.84	72.44	n.a.	49.71	52.23	n.a.
GPN	73.84	66.33	74.82	62.17	46.35	58.71	43.01
Ours	71.30	66.98	74.52	62.75	47.48	56.97	41.48
Alea: Aleatoric, Epi.: Epistemic, w:/ with propagation, w/o: without propagation
Table 7: AUROC and AUPR for the Misclassification Detection (Cont.)
Data	Model	AUROC	AUPR
Alea w/	Epi w/	Alea w/	Epi w/
Amazon Computers	APPNP	79.75	n.a.	45.10	n.a.
VGCN-Energy	82.08	n.a.	45.53	n.a.
GKDE-GCN	79.66	73.66	63.26	56.93
GPN	82.20	77.58	47.93	41.80
Ours	80.75	74.87	93.12	90.11
Amazon Photos	APPNP	85.74	n.a.	37.00	n.a.
VGCN-Energy	87.94	n.a.	48.35	n.a.
GKDE-GCN	84.11	75.07	54.35	45.43
GPN	87.21	83.38	46.32	37.07
Ours	84.42	81.61	96.89	96.70
Coauthor CS	APPNP	89.92	n.a.	37.98	n.a.
VGCN-Energy	89.46	n.a.	38.86	n.a.
GKDE-GCN	89.24	80.98	39.30	30.52
GPN	85.72	81.56	46.12	38.98
Ours	86.21	83.94	97.34	96.80
Coauthor Physics	APPNP	93.27	n.a.	38.14	n.a.
VGCN-Energy	92.86	n.a.	37.19	n.a.
GKDE-GCN	92.77	86.12	37.08	25.13
GPN	91.14	89.63	41.43	35.64
Ours	89.93	88.83	99.14	99.10
OGBN Arxiv	APPNP	77.55	n.a.	54.57	n.a.
VGCN-Energy	77.89	n.a.	54.87	n.a.
GKDE-GCN	77.47	77.55	61.62	62.33
GPN	75.44	72.71	55.64	52.99
Ours	75.30	72.85	83.95	81.54
Alea: Aleatoric, Epi.: Epistemic, w/: with propagation
CoraML
(b)  $10^{-6}$
(a) 0
(c)  $10^{-4}$
Figure 2: latent representation for CoraML
(d)  $10^{-2}$
(a) 0
(c)  $10^{-4}$
Figure 3: latent representation for CiteSeer
(b)  $10^{-6}$
(d)  $10^{-2}$
CoauthorCS
(a) 0
(c)  $10^{-4}$
Figure 4: latent representation for Coauthor CS
CoauthorPhysics
(b)  $10^{-6}$
(d)  $10^{-2}$
Figure 5: latent representation for Coauthor Physics
E.4 Graph Activation
In this subsection, we present the t-SNE visualizations of the learned representational space for various datasets in the following figures, without applying distance regularization. Instead, we introduce different activation functions. It is worth noting the notable distinction in quality when using the LogSigmoid activation function, which appears to be the smoothest among the activation functions employed on CiteSeer and Amazon Computers datasets. Once again, the size of each node corresponds to the square root of the learned evidence. Additionally, the color black indicates out-of-distribution (OOD) instances across all datasets, while distinct colors represent different classes.
Figure 6: Latent representation for CoraML, CiteSeer and PubMed on different graph activation functions: RELU, LogSigmoid, and HardTanh.
Figure 7: Latent representation for AmazonPhotos, AmazonComputers, CoauthorCS and CoauthorPhysics on different graph activation functions: RELU, LogSigmoid, and HardTanh.
E.5 Ablation Study
We show the full ablation study on three datasets: CoraML, CiteSeer and PubMed in Table 8.
Table 8: Ablation Study with OOD Detection task (cont.)
Data	Model	ID-ACC	AUROC	AUPR
Alea w/	Epi w/	Epi w/o	Alea w/	Epi w/	Epi w/o
CoraML	GPN	88.51	83.25	86.28	80.95	75.79	79.97	72.81
GPN-CE	89.31	82.58	83.91	80.88	76.54	77.60	76.05
GPN-CE-ACT	89.87	83.34	86.96	75.60	74.96	79.74	62.73
GPN-CE-ACT-GD	90.06	83.94	87.20	76.12	76.26	80.36	63.32
Citeseer	GPN	69.79	72.46	70.74	66.65	55.14	50.52	44.93
GPN-CE	70.98	74.20	73.75	68.41	58.12	53.55	46.60
GPN-CE-ACT	71.96	74.72	77.97	72.28	60.41	56.04	50.73
GPN-CE-ACT-GD	72.51	75.22	78.98	73.21	62.30	58.63	52.73
PubMed	GPN	94.08	71.84	73.91	71.2	57.92	67.19	59.72
GPN-CE	93.84	74.19	78.32	74.50	59.85	74.11	64.55
GPN-CE-ACT	93.84	74.19	78.32	74.50	59.85	74.11	64.55
GPN-CE-ACT-GD	93.84	75.23	81.76	77.79	60.75	78.16	69.19
* Alea: Aleatoric, Epi.: Epistemic, w/: with propagation
GPN is the original results from the GPN paper with default hyperparameters and ReLU as the middle activation function, GPN-CE is the original GPN model with re-tuned dirichlet entropy regularization weight; GPN-CE-ACT is the original GPN model with re-tuned entropy regularization weight and activation function; GPN-CE-ACT-GD/(Ours) add the distance-based regularization term and tuned the two weights and activation function.