Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Authors define PLATO, a model for underserved scenario of short and fat datasets (d>>n). PLATO works when we have an auxiliary knowledge graph describing the d features and the relations between them. Tabular datasets with d>>n are common in physical sciences and biology where data is collected through expensive experiments. 

Plato is an MLP where the first layer is trained by leveraging the knowledge graph as follows:
1. A c-dimensional embedding M_j is learned for each feature j
2. A message passing algorithm that uses the knowledge graph is used to learn another embedding Q_j for each feature j based on its neighbors. In each round the embedding of a feature is a weighted combination of its embedding in the previous round and all its neighbors in the previous round.
3. A neural network outputs P the weights of the first layer.

Experimental results are presented on multiple datasets and compared with multiplied methods (both statistical and deep-learning based). Some experimental results are presented to explain the motivation around using a message passing protocol and large knowledge graphs describing features even if not all are in dataset.
strength_and_weaknesses: Strengths
1. Solves for an important set of applications that are underserved in deep learning focused research geared towards large datasets.
2. Elegant way of incorporating prior information via a knowledge graph (KG). While not all applications have associated knowledge graphs, a sizeable set do. KG is a distillation of several studies, and to be able to incorporate effectively is significant. 

3. Experimental results are rigorous. On multiple datasets and compared with multiplied methods (both statistical and deep-learning based) Established protocols for tabular dataset are followed.
4. The weights of the first layer of MLP are learned in 3 steps described above. An ablation study shows the importances of each step showing the added value of a message passing routine and also of adding knowledge about features not present in the datasets.
5. Clearly describes architecture, datasets, and results.

Weaknesses:
Ablation study is presented only on a single BRCA dataset. It would be valuable for readers to know that findings hold across datasets.

The intro and related work sections both spend significant ink on related work, including describing works like graph-based prediction that are only tangentially related. Instead, the intro can incorporate a summary of the contributions in the paper. Currently, the reader has to read all through methods and results to understand the significance of the contributions. Examples of items to include earlier are how message passing the graph structure of KG is used effectively to incorporate additional info. This is a significant result not sufficiently highlighted.

The current paper is not incomplete without these, but some suggestions for future work:

Study impact of missingness in KG. Is it possible to support datasets where some subset of features are not described in KG. The paper currently assumes every feature in dataset is a node in the KG. 

A study on model's sample complexity e.g., we broadly cast this problem as d>>n. e.g., can we afford to drop some rows from the dataset and the models achieve the same performance.. What is the point of diminishing returns. It would be interesting to see how few examples the model can get away with.
clarity,_quality,_novelty_and_reproducibility: Clearly written paper. Experimental results are rigorous and following best practices including hyper parameter optimization and results reported over a large number of runs. As far as I am aware, this is a novel approach to solving for an underserved set of applications common in biology and physical sciences where d>>n
summary_of_the_review: Overall, the authors present a novel approach to supervised learning tasks on datasets where d>>n. Given the novelty of the approach, significance of the setting for biological and physical sciences and elegance of the solution, I vote accept. 

The major weaknesses of the paper are largely open questions that can be addressed in a future iteration of the paper. The paper remains of interest to the community  at this iteration.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: To deal with the problem when tabular data has high dimensionality, the paper introduces a knowledge graph to help determine the weights of an MLP. The proposed PLATO method achieves good results on various datasets.

strength_and_weaknesses: To deal with the case that the dimensionality of the data is very larger than the sample size, the authors introduce an additional knowledge graph as a prior of the feature relationship. Then the weights of the MLP are better determined.

There are some suggestions for the paper:
1. Why the weights of the first layer of the MLP is more important than the others? 
2. Do the results depend on the quality of the knowledge graph? How to compare with other methods in a fair manner since an auxiliary knowledge graph is introduced.
3. Pre-train the node embedding in Eq. 2 is not with a self-supervised node embedding method is not clear enough. How will the pretraining method influences the final results?
4. The layer-wise predicted embeddings are concatenated in line 9 in the algorithm. It works like an ensemble. How much does the concatenation step improve in the whole method?
5. Please consider comparing more tabular data methods, such as FT-Transformer based on some dimensionality reduction methods.
6. It seems the column names are required. Does it limit the field of the method?
clarity,_quality,_novelty_and_reproducibility: The paper is generally clear, but the method is a bit complicated. The authors may add the size of matrices in the algorithms.

summary_of_the_review: The main idea of the paper is reasonable. An auxiliary knowledge graph will introduce more information especially since there are many features. The method is clear and the results are good. Please check the weakness part for some questions.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This submission contributes a learning method for high-dimensional data that uses an associated knowledge graph between the features to create a lower-dimension representation. The methods pretrains a data-transformation mechanism on the knowledge graph, creates attention weights, and used message passing on the corresponding graph neural network to output an embedding of input data, which consistutes the intermediate layer of an MLP. The method is benchmarked on 6 biomedical datasets, with a dozen thousands features and hundreds samples, where it outperforms classic methods.

strength_and_weaknesses: The idea of using a knowledge graph to build a graph neural net on the features is interesting.

However, this paper seems more targetted as computational biology applications, and not tabular data in general. Indeed, outside computational biology, it seems quite unlikely to find the configuration of many columns all corresponding to entries of a knowledge base. Can this paper be applied outside computational biology? If so, it would be interesting to demonstrate it.


Did the author compare to classic graph-regularized models? There is a literature that is more than 10 years old on the topic, including in computational biologie.

I do not understand where things such as the line 3 of algorithm 1 come from. I just do not understand how was the algorithm designed. By trial and error?

How are the error bars computed on the tables? Given the small number of n, I would expect much larger error bars.

PCA is not a supervised method. How can it be used in the experimental tables?


clarity,_quality,_novelty_and_reproducibility: 
The paper is reasonably clear but not stellar. In particular it does not explain how the specificities of the method where obtained.

The novelty is limited given the huge amount of papers on graph regularization.

Given that no code is available, the reproducibility is limited.

summary_of_the_review: There is an interesting idea, but this seems more like a computational biology paper than a general tabular learning but. The paper does not contribute understandible and insight on top of the graph regularization work.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This work attempts to address the curse of high dimensionality problem for the tabular data. The authors build their model, PLATO, on the well formed insight that auxiliary KG about the feature connections can improve performance as the sparsity induced in the model helps with the generalizations in the cases where d>>n. Their method predicts the first layer weights of the MLP and then the remaining layers of the MLP are trained using the loss function. They evaluate their method on different datasets and report improved performance.
strength_and_weaknesses: 
Pros:
1. Figure 1 explains their methodology in a good and succinct manner. 

Comments:
1. It will be helpful to cover some related graph recovery methods to get auxiliary graphs for more adoption. 
2. (I) In sec 3.2, the rationale given to share the parameters of the first layer does not seem principled to me.  “Typically this intuition is missed since T learns the parameters Θ [1] j and Θ [1] k associated with two features j and k independently by gradient backpropagation.” The features do have common parameters in the subsequent layers through which their commonalities can be modelled. 
3. This methodology seems to learn a projection of the input data to another space, say R^h (here referred to as layer 1 of the MLP). The method comes up with `d’ vectors in R^h space that are put together as a matrix to obtain the $\Theta[1]$. This is done by running message-passing on the KG which takes individual feature inputs as one-hot vectors to obtain a vector in R^h space.
4. How does it perform for out-of-distribution samples? 
5. Please share details about any ablation study done, which empirically shows that having the first layer obtained from this method works better than just training a simple MLP. What happens when there are more than 3-4 layers in MLP and what if there are no learnable MLP layer and just this direct projection layer? 

I have some major concerns about this methodology (pts 2&3 in comments). The rationale given by the authors seems to be hand-wavy and not grounded. I hereby request the authors to develop a mathematically rigorous approach to support their claims. 

clarity,_quality,_novelty_and_reproducibility: <mentioned in comments above.>
summary_of_the_review: I have some major concerns about this methodology (pts 2&3 in comments). The rationale given by the authors seems to be hand-wavy and not grounded. I hereby request the authors to develop a mathematically rigorous approach to support their claims.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper suggests a technique for tabular datasets with high-dimensional features but limited number of samples. Specifically, the authors suggest to utilize an auxiliary knowledge graph to predict the first layer parameters of the MLP, effectively reducing the number of trainable parameters. Three reviewers are negative and one reviewer is positive. The main concerns of negative reviewers are unclear motivation for several design choices, limited applicability from assuming a knowledge base and the lack of novelty compared to graph regularization work approaches. 
justification_for_why_not_higher_score: While the authors provided response to address the concerns of reviewers, it was not enough to convince the negative reviewers. Hence, given the current status, AC recommends reject.
Decision: Reject