SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Table 10, we show the rank order performance of all models on all  $d \gg n$  datasets. We find that PLATO exhibits consistent and strong performance while the performance of the baselines depends on the specific  $d \gg n$  dataset. For example, TabTransformer is the second best performing of all models on the MNSCLC dataset but the worst performing of all models on the PDAC and CH datasets. Similarly, XGBoost is the second best performing of all models on PDAC but only the tenth best performing of all models on BRCA. The baselines with the most stable performance are LASSO and Ridge Regression which rank consistently between the second and eighth best of all models.
Table 10: For datasets with  $d \gg n$ , PLATO exhibits consistent and strong performance. By contrast, the performance of the baselines varies with each dataset. For every dataset, the rank order of performance from Table 1 is shown. The best overall model is in bold and the second best model is underlined.
Dataset	MNSCLC	CM	PDAC	BRCA	CRC	CH
# of features d / # of samples n	52.2	46.1	40.3	28.2	22.6	19.7
Classic Stat ML	Ridge	8	3	4	2	3	5
Dim. Reduct.	PCA	7	12	10	7	10	8
Feat. Select.	LASSO	6	2	3	5	2	4
STG	9	4	8	4	7	7
Decision Tree	XGBoost	14	9	2	10	5	2
Graph Reg.	GraphNet	5	7	9	8	12	3
NC LASSO	4	5	5	6	9	6
Network LASSO	3	8	12	9	11	9
Param. Infer.	Diet	13	13	6	12	13	11
Tabular DL	MLP	10	6	7	11	4	12
NODE	12	10	11	3	6	10
TabTransformer	2	11	14	13	8	14
TabNet	11	14	13	14	14	13
Ours	PLATO	1	1	1	1	1	1
Table 11: For datasets with  $d\sim n$ , PLATO is competitive with baselines. For XGBoost is consistently the strongest baseline. For every dataset, the rank order of performance from Table 11 is shown. The best overall model is in bold and the second best model is underlined.
Dataset	ME	BC	SCLC	NSCLC
# of features d / # of samples n	2.0	1.8	1.7	1.1
Classic Stat ML	Ridge	9	9	9	4
Dim. Reduct.	PCA	13	12	13	7
Feat. Select.	LASSO	7.5	6	6	8
STG	5	5	7	6
Decision Tree	XGBoost	1.5	2	2	1
Graph Reg.	GraphNet	6	4	5	9
NC LASSO	4	3	4	3
Network LASSO	11	10	11	13
Param. Infer.	Diet	14	13	14	14
Tabular DL	MLP	10	8	10	10
NODE	3	11	3	12
TabTransformer	12	14	12	11
TabNet	7.5	7	8	5
Ours	PLATO	1.5	1	1	2
E Graph regularization baselines
We test the state-of-the-art graph regularization baselines GraphNet [20], Network-Constrained LASSO [40], and Network LASSO [23]. The graph regularization baselines can only consider a homogeneous graph with only features as nodes and a single edge type. For the graph regularization baselines, we thus induce a subgraph between feature nodes from the knowledge graph and collapse all edge types between feature nodes into a single edge type. In this context, GraphNet, Network-constrained LASSO, and Network LASSO correspond to a LASSO model with a mean-squared error loss and a graph regularization penalty. Let  $\lambda$  be the graph regularization coefficient,  $j$  and  $k$  be two input features, let  $E$  be the set of edges in the graph, let  $\Theta \in \mathbb{R}^d$  be the weights of the linear regression for  $d$  input features, and let  $D_{j}$  be the degree of feature node  $j$ . The graph regularization penalty for GraphNet is  $\lambda \sum_{j,k\in E}(\Theta_j - \Theta_k)^2$ , the penalty for Network-constrained LASSO is  $\lambda \sum_{j,k\in E}(\frac{\Theta_j}{\sqrt{D_j}} -\frac{\Theta_k}{\sqrt{D_k}})^2$ , and the penalty for Network LASSO is  $\sum_{j,k\in E}|\Theta_j - \Theta_k|$ .
F Number of trainable weights in PLATO vs. a multilayer perceptron
Table 12: PLATO drastically reduces the number of trainable weights compared to a multilayer perceptron (MLP) across all of the datasets. The number of trainable weights in the best model from the hyperparameter sweep is shown for each dataset.
Model	MNSCLC	CM	PDAC	BRCA	CRC	CH	ME	BC	SCLC	NSCLC
MLP	429665	416961	820097	425217	200529	589761	586945	296113	298929	594209
PLATO	17154	42498	32066	17154	28386	61890	17154	28386	32066	17154
G Dataset Details
We use 6 datasets with  $d \gg n$  and 4 datasets with  $d \sim n$  [16, 17, 30, 79]. In all datasets, a machine learning model must predict the response of a cell line or a mouse tumor model to a drug. As input, the model considers a tabular dataset. In the tabular dataset, every row corresponds to a specific cell line or mouse tumor model. Every column corresponds to a gene name. Every value corresponds to the amount of that gene in the cell line or in the mouse tumor (i.e. gene expression). In practice, the number of genes is large for all tasks and the number of cell lines or mouse tumor models is comparatively small (i.e.  $d \gg n$ ). For every row, the model also takes as input a fixed feature vector corresponding to the drug (i.e. a 200-dimensional ComplEx [65] embedding of the drug node in the knowledge graph). The output label is the response of the cell line (i.e. Ln-ic50) [30, 79, 17] or mouse tumor model (i.e. minimum average percent tumor growth "min-avg-pct-tumor-growth") to the drug [16]. Data is available at https://github.com/snap-stanford/plato.
Gene expression datasets were pre-processed following a standard process in [48]. Briefly, gene expression values underwent TMM normalization and log transformation (i.e.  $\log (x + 1)$ ). Values were made to have zero mean and unit standard deviation. Dataset abbreviations are breast carcinoma (BC) [30, 79, 17], breast carcinoma (BRCA) [16], chondrosarcoma (CH) [30, 79, 17], colorectal cancer (CRC) [16], cutaneous melanoma (CM) [16], melanoma (ME) [30, 79, 17], non-small cell lung carcinoma (MNSCLC) [16], non-small cell lung carcinoma (NSCLC) [30, 79, 17], pancreatic ductal carcinoma (PDAC) [16], and small cell lung carcinoma (SCLC) [30, 79, 17].
H Knowledge Graph Details
We compile a general biomedical knowledge graph from prior studies [44, 35, 38, 56, 63, 74, 75] to use across all datasets. A schematic of the KG is in Supplementary Figure 2. A detailed breakdown of relation types is in Supplementary Table 13. The knowledge graph is available at https://github.com/snap-stanford/plato.
The knowledge graph contains 108,447 total nodes, including 7,975 drugs, 18,370 diseases, 11,447 phenotypes, 22,319 genes, 11,153 molecular functions, 28,748 biological processes, and 4,184 cellular components. Every gene and every drug in every dataset is present as a node in the knowledge graph. The knowledge graph also contains 3,066,156 edges with 99 distinct relation types. The remaining node types and their relationships serve as broader domain knowledge.
Edges between drug nodes and gene/protein nodes were derived from Drugbank [75], Gao [16], and the Genomics of Drug Sensitivity in Cancer [79, 30, 17]. Edges between diseases and genes/proteins were derived from DisGeNet [5]. Edges between diseases and phenotypes were derived from the Human Phenotype Ontology [35]. Edges between drugs and diseases were derived from the Multiscale Interactome [56]. Edges between drugs and side effects were derived from SIDER [38]. Edges between genes/proteins and other genes/proteins were derived from BioGRID [49], [55], the Database of Interacting Proteins [57], [44], [47], [53], [82], [69], and STRING [63]. Finally, edges from genes/proteins to molecular functions, biological processes, and cellular components as well as edges between molecular functions, biological processes, and cellular components were derived from the Gene Ontology [9].
Figure 2: Knowledge graph as a unified knowledge backbone. We constructed a knowledge graph as a unified knowledge backbone across all 6 datasets. (a) Legend. For each node type, the number of nodes is given in parentheses. Between node types, the number of edges and the number of relation types are given. (b) Number of total nodes and edges across entire knowledge graph. (c) Visual schematic of knowledge graph across each node type.
Table 13: Knowledge graph relations between node types.