Learning Enhanced Representation for Tabular Data via Neighborhood Propagation
Authors: Kounianhua Du
Weinan Zhang
Ruiwen Zhou
Yangkun Wang
Xilong Zhao
Jiarui Jin
Quan Gan
Zheng Zhang
David Wipf
Venue: NeurIPS 2022 Accept (2022)
Keywords: Tabular Prediction
Hypergraph

Abstract: Prediction over tabular data is an essential and fundamental problem in many important downstream tasks. However, existing methods either take a data instance of the table independently as input or do not fully utilize the multi-row features and labels to directly change and enhance the target data representations. In this paper, we propose to 1) construct a hypergraph from relevant data instance retrieval to model the cross-row and cross-column patterns of those instances, and 2) perform message Propagation to Enhance the target data instance representation for Tabular prediction tasks. Specifically, our specially-designed message propagation step benefits from 1) the fusion of label and features during propagation, and 2) locality-aware multiplicative high-order interaction between features. Experiments on two important tabular prediction tasks validate the superiority of the proposed PET model against other baselines. Additionally, we demonstrate the effectiveness of the model components and the feature enhancement ability of PET via various ablation studies and visualizations. The code is available at https://github.com/KounianhuaDu/PET.

Learning Enhanced Representations for Tabular Data via Neighborhood Propagation
Kounianhua Du', Weinan Zhang', Ruiwen Zhou', Yangkun Wang', Xilong Zhao Jiarui Jin
Department of Computer Science Shanghai Jiao Tong University
{774581965, wnzhang, skyriver, espylacopa, zhaoxilong, jinjiarui97)asjtu edu..uuuu.uuuuuuu.e...mooo.. cn
Quan Gan, Zheng Zhang, David Wipf
Amazon
{quagan, zhaz, daviwipf]@amazon.
Abstract
Prediction over tabular data is an essential and fundamental problem in many important downstream tasks. However, existing methods either treat a data instance of the table independently input do not jointly utilize multi-row features and labels to directly change and enhance target data representations. In this paper, we propose to construct hypergraph from relevant data instance retrieval to model the cross-row and cross-column patterns those instances, and 2) perform message Propagation Enhance the target data instance representations for Tabular predic- tion tasks. Specifically, our tailored message propagation step benefits from both the fusion of label and features during propagation, well locality-aware high- order feature interactions. Experiments on two important tabular data prediction tasks validate the superiority of the proposed PET model relative other baselines. Additionally, we demonstrate the effectiveness of the model components and the feature enhancement ability of PET via various ablation studies and visualizations. The code is available at https://githubb.cmm http:ghtwbb..oooooooaaaaaaaaaiie KounianhuaDw/PET
Introduction
Prediction over tabular data is fundamental and essential problem in many data science applications including recommender systems (Bobadilla et al., 2013; Ying et al. (2018), online advertising (Richardson et al., 2007: Zhou et al. 2018), fraud detection (Bolton and Hand 2002). question answering (Chen et al.) 2020), etc. Most existing methods seek to capture the patterns of feature interactions within an instance independently using tree models (Chen and Guestrin. 2016) deep networks (Guo et al. 2017).
Recently, as shown by Papernot and McDaniel (2018), feeding in neighbors of the target as input can improve the robustness data representations and help the model generalize better to out-of- distribution samples. Some retrieval based methods (Pi et al.] 2020 Qin et al.) 20200 (021) then seek to utilize multiple neighboring data instances the target for label prediction. These methods take the auxiliary instances extra inputs but do not consider that such auxiliary information could enhance the target representation. Other existing graph-based methods on tabular data (You et al. 2020 Wu al. 2021 Guo et al. 2021b) aim to learn robust representations with locality structure, as proved stabilization and generalization ability of graph neural networks.
Verma and Zhang
"ork done during internship at Amazon Web Services Shanghai AI Lab. WWiinan Zhang is the corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Col Col Col
ColF
Col 1 Coll2 Col
Col
Col
Col2 Col
Col
(yypo-slys)
Models
y1
Tirget Datu Olper-
LR, GBDT, Feature Interactions...
(CLS)
Date (lypte-sdge) y
(4)
Figure 1: Difference between PET and other models. (a) The popular tree models and interaction- based models utilize single data instance for prediction. (b) The strieval-basee methods take multiple data instances as input without sufficiently mining the interaction patterns among them. (c) The proposed PET models the multiple data instances set as a hypergraph and capture their correlations with the assistance of labels.
However, they either omit the high-order product feature interactions, or else serve as a plugin while resorting to other architectures like factorization machines (Rendle 2010) to explore feature interactions. Moreover, they ignore the mutual enhancement between the representations of labels and features. In this paper, with the aim of learning enhanced data representations of tabular data, propose to construct a hypergraph among relevant data instances model the set relationships (Srinivasan et al., 2021). Additionally, we design an end-to-end graph neural network prediction model that generates high-order feature interactions with the assistance of the locality structure and label adjustment.
Specifically, we design PET, a novel architecture that Propagates and Enhances the Tabular data representations based on the hypergraph for target label prediction. We first retrieve from the observed data instances pool to get the neighboring data instances for each target. The resulting instance set can be seen as hypergraph, where sets of feature columns of data instances form hyperedges and each distinet feature value of these data instances form node. This hypergraph models the cross-row and cross-column relations of the resulting instance set. Then we conduct message propagation on the graph. The propagation serves three purposes. First, auxiliary label information from the retrieved data instances propagates through the common feature value nodes to help the target prediction. Second, feature representations get enhanced through the locality structure. Our interactive message generation, attention- aggregation, and update generate locality-awan high-order feature interactions. Third, the labels are incorporated into the propagating messages to directly adjust the feature spaces and generate label-enhanced feature representations.
The main contributions are summarized as follows:
We propose retrieval-based hypergraph to capture the feature and label correlations among tabular data instances.
We design an end-to-end graph neural network prediction model that unifies the product feature interaction, locality mining, and label enhancement.
We utilize the observed labels in the resulting set to guide the feature learning process and use the propagated labels to enhance predictions.
We evaluate the proposed PET model on two prediction tasks, i.e., binary classification and top-n ranking. over five tabular datasets, where substantial performance improvement against other strong baselines validates the superiority of PET.
2 Preliminaries and Related Work
Tabular data prediction. Tabular data prediction treats every row of the table as a data instance and every column/field as a feature attribute We consider tabular data prediction under a single table scenario in this paper, and prediction over multiple tables can be seen as the prediction over joined table. We also focus on tabular discrete data this paper, while the continuous feature values can be
We and data instance, well column and field interchangeably.

---

discretized in various ways (Guo et al.
2021a). Currently some of the most widely used models for
tabular data prediction include Gradient Boosting Decision Trees (GBDTs) (Friedman 200T and Factorization Machines (FM) (Rendle 2010). Variants of FM such as DeepFM [Guo et al. [017 and Wide & Deep (Cheng et al. 20T6) are especially popular in industrial applications. For a given row, such models simply take in the row and make predictions, i.e., these models assume that the rows are IID. Such an assumption, however, is reasonable only if the embedding representations of the tabular data can sufficiently host the high-order interaction patterns within the data instance, which is practically impossible.
Graph neural networks on tabular data. As Graph Neural Networks (GNNs) become popular, multiple attempts to apply GNNs on tabular data prediction have emerged. Since the model takes in graph that connects the rows and columns together, the prediction on single row no longer depends only on itself, but also other rows, and is therefore suitable for exploiting non-III properties in tabular data. Examples of using GNNs on tabular data include Wu et al. (202T) that constructs a mini-batch data-feature bipartite graph, You et al. (2220) that treats the (incomplete) table as an adjacency matrix of a bipartite graph, and Guo et al. (2021b) that treats rows nodes and build edges according to predefined rules. These methods omit the product feature interactions or else resort to other architecture like factorization machines to explore them. Moreover, they ignore the mutual enhancement between labels and features.
Hypergraphs and hyperedge glasiffcaation/rerress as a single node, then a row describes an n-ary relationship among the nodes. Normal graphs have trouble describing this relation since edges can only connect two nodes at a time. A hypergraph is defined as pair G = CV=VV E) with its node set V and its hyperedge set E C P(VV), where P(V) is the power set of V. meaning that each "edge" becomes simply a subset of V. regardless of the number of nodes in the "edge". We can construct hypergraph from a set of rows, where each hyperedge correspond to a row and each node correspond to the distinet feature values among all the rows. Tabular data prediction problem can be cast into hyperedge classification/reeressooo problem, where every row now corresponds to a hyperedge.
If we treat each individual value in table
generalizes graphs such that an edge can connect more than two nodes.
Hypergraph neural networks. Hypergraph neural networks are an adaptation of GNNs with a mes- sage passing paradigm wherehy node representations are used to update hyperedge representations, which in turn update the node representations again (Srinivasan et al. 2021 Bai et al. 2021 Feng jet al. (219)). Using a hypergraph neural network on tabular data allows us to obtain a representation for each row from its corresponding hyperedge, well as representation for each individual value of each column from its corresponding node.
3 Methodology
When making prediction for sample, feeding in similar or relevant samples together as input is known to contribute to robustness and generalization abilities for out-of-distributtin samples (Papernot and McDaniel 2018). In light of this, for a given data instance, our model retrieves a set of relevant data instances according to a relevance metric, aiming to take auxiliary information from relevant data instances.
The resulting instances set can be seen as a hypergraph, where each distinct feature value forms a node and a collection of them, i.e., data instance, forms a hyperedge. After the star expansion (Agarwal et al. 2006;; Srinivasan et al. 202I). we get a bipartite graph with feature value nodes on one side and data instance nodes on the other, on which we propagate and enhance representations. An advantage of message passing is that it allows us to capture higher-order interactions among nodes and hyperedges (Srinivasan al. 2021). ),, i.c. the interactions among the individual column values as well as the rows. The other advantage of message passing is that we can utilize the labels of retrieved data instances to interact with features to generate label-enhanced messages, guide the message aggregation process, and take advantage of label propagation at the same time. After message passing, the enhanced data instance representations are then used for prediction.
Figure 2 illustrates the framework of PET. Detailed descriptions of each individual component are provided in the following subsections.
3

---

Row prediction (farget instance)
Bis E0x unemployed senbossmnngg man.geeenngcecceeeee.. color marridd dig primary secordary terbor
Occupation
Marriage
Education
unerployed
Mariie
primnar
2
Pessing
abel of incident data node
OUTPUT 8c
instance
Relevant Data Instance Search
Node Embeddiigg
Poot
Star Expansion
Feed-forwere
Occupation
Marriage
Education
Label
Edimany
Hypergraph Construction
ramagament
T3
secondary
hyperedge
grggmeeett
tertary
O
O
secondary
tertary
O
F1
T4
Blue ouluooolar
Service
The workflow of PET. For each row to predict (top-left), PET retrieves fixed number of relevant data instances (bottom-left) and constructs hypergraph (bottom-right) from the resulting data instances set. After a star expansion (top-right), we get a data-feature bipartite graph with data instance nodes at the bottom and feature value nodes at the top. We then perform the proposed hypergraph neural network on the resulting graph and use the target data instance node representation for prediction.
Figure 2:
3.1 Graph Construction
For each target data instance, we retrieve K relevant data instances from the observed data instances using ElasticSearch and construct a hypergraph from the resulting instance set.
......., F} be the feature of a target data instance f, where is the number of feature fields and x'y is the feature value of the f-th field of data instance t. We first conduct a boolean query to obtain instances that have at least one common feature value with the target data instance. Then retrieve the top-K relevant instances T1.-. TK from the filtered instances set according to the relevance value (Qin et al. 2021) defined as:
Let X = { = 1,...,
R(X*, x'Xx,x)== = Σ IDF(,))) 1 (a)===))
f=1
N N(x)) + 0.5
IDF(x'))= = (1 0.5
(2)
where (()) is the indicator function, N is the number of data instances in the table, and N(x)) is the number of data instances that have feature value z in the f-th field. This metric is equivalent the BM25 (Robertson et al. 1995) metric if treat each data instance document and their feature values as the terms.
After the retrieval, the resulting instances set (X*, "x. ,". can be seen as hypergraph, where each distinct feature value in each individual field forms node and each data instance constructs a hyperedge. We then perform a star expansion (Agarwal et al. (006) on the hypergraph,
rk]. Vr {=y) 1. ..... F:j t,r,,...,, TK), and undirected edge exists between two nodes i Vp and E Vu if T X2.
i.e. construct a bipartite graph G (Vp, Vr, E) where VD = (1, ".r,...,kk
....FF:j
3.2 Message Passing and Interaction
After the graph is constructed, we propagate and enhance the representations on it. The propagation serves three purposes. First, the label information propagates through common feature value nodes to help the label prediction of the target data instance node. Second, the features get enhanced through taking in high-order information. Locality-aware high-order product feature interactions are generated through the interactive message generation and attention based aggregation. Third,
httpps//www.llasiicccoollsssssss

---

the label embeddings directly interact with the features to adjust the feature spaces and generate tubel-enhanced features and high-order interactions. Then we detail the components as follows.
Initialization. Before message passing, we initialize the node representations and edge representa- tions with trainable embedding vectors. Let 4 and yy different trainable embedding layers for feature value nodes and data instances nodes, respectively. And let Pin and Pout be the embedding layers for the edges. We first initialize the feature value node representations as
(0) (i), 2 Vp.
(3)
We also initialize the data instance node representations of the retrieved rows with a trainable embed- ding vector associated with their labels. As for the target data node, we initialize its representations with constant zero vector.
f0
1=4
1100 -
(1'),,
jE VoN(),
where y denotes the label of the data instance j.
In addition, we initialize each edge representation with the embedding of its incident data instance node label to guide the feature learning process and generate high-order interactions.
D ( (v'). T Dout (v'),
i VD.J E VF.
(0) ci
i Ve.j VD.
Message Generation. Then the interactions between edge and node representations along with the original node representations to generate the messages;
m (!) = (e (ey-") @ h
(1-1)
(6)
where @ denotes the Hadamard product, superscript / means the 1-th layer, and || denotes the concatenation operation.
As edge representations carry the label information initially, the Hadamard product term generates label-enhanced features. In addition, the edges will contain node features after updating edge representations with incident node representations, then the Hadamard product term produces high- order product feature interactions. The high-order product feature interactions prove to be important in many interaction brrrre tabular data prediction methods (Qu et al. 2018), which are often missed in other graph-base base tabular networks captured by extra factorization machine (Wu et al. 2021).
Message Aggregation. The incoming neighboring messages are then aggregated based on an attention mechanism:
W (1-1) Q nj
K(0) 1D
1)
K
mij
W mij
(7)
(
Softmax,E (/))0 x
(I) = Σ
(rl)
15
(EN()
Node Embedding Update. After receiving the aggregated neighboring messgages, the node embed- dings are updated based on the aggregated messages and their own node embeddings.
1 a (W (n (1-1) In ())).
(8)
where (-) denotes the ReLU activation function.
Edge Embedding Update. Then we update edge embeddings using their incident node embeddings as Equation (9). edge embeddings are updated using the node embeddings, the features and labels on nodes are propagated to the edges as well. After rounds of propagation, the messages generated contain high-order feature interactions and high-order feature-labe interactions.
c oww (9) W (n [1 1e (1-"))).
(1)
(9)
5

---

Initialization
is One round of propagation and update
Two rounds of propagation and update
Target row
Represeriat on of target
Higher-order
Information layer
interactions
First order features: 800
Propageaed label:
MLP
Label enhanced Ieatures:
00
Secordkorder
row (retrieved)
Doctor
mossages
Figure 3: The representation of the target node after propagation. We use the red color to represent the retrieved instance node and use the grey color to represent the target data instance node to predict. Other feature value nodes in different colors. The information flow is marked with the corresponding color. The dots-on-edges represent the feature and label information stored in edges. (best viewed in colors)
3.3 Label Prediction
Then the target data node embedding of the last layer for prediction:
dr = MLP(h(
(10)
For binary classification, the training objective is set as cross entropy:
C(Y Σ I(vr log Úe + we) log(1 ût))
(11)
where y/ is the label of the target data instance L.
3.4
Discussion
In this section, discuss the strength and potential weakness of the proposed model and offer the complexity analysis for the proposed method.
3.4.1 Model Analysis
We illustrate the representation of the target node after propagation in Figure Following the pro- posed propagation method, the target node will receive the first order features, second-order features, label-enhanced features, and the propagated labels after two rounds of propagation. Propagating more than two layers will further produce higher-order feature interactions, cross-rows feature interactions, and feature-label interactions. Furthermore, the feature messages adjusted by corresponding labels and attentively aggregated based on the locality structure.
When dealing with datasets with large number of fields, the constructed graph may be dense. In this way, we could potentially meet the oversmoothing problem, which may degrade the model performance. However, this problem can still be overcome by some methods. For example, could retrieve and construct the graph over selected fields (e.g., top 10 fields that have the largest mutual information with the labels).
3.4.2 Complexity Analysis
In this section, we offer the complexity analysis for PET. Let indicate the number of retrieved data instances, F indicate the number of feature fields, N be the number of features, and d be the embedding size.
For the retrieval stage, as discussed in RIM QQn et al. 2021), the average length of the posting lists in the inverted index is |Search(Poo]] for constructing the inverted index, it could be done offline and only once. The complexity O ((F) Search Pool)).
N The total time complexity of retrieval is O(F |SeacchPool ). As
N
The main computation of PET lies in the message passing part. The number of nodes in the graph for data instance is more than (K + 1)(F 1). and the number of edges is 2(K + 1)F. Both of

---

them are F). The time complexity of single-layer message passing of PET is F(FF.)). This makes PET scalable as the complexity linear with respect to the number of feature fields (number of table columns). In addition, the graph construction and message passing can be highly parallel: 1) The graph construction for each data instance can be done in parallel for a batch data. We batch all the resulting graphs for the data instances in the batch to form one single batched graph. Then we can apply GNN on the resulting single batched graph to make predictions parallelly. 2) For the GNN, the operation of the attention computation can be parallelized across all edges, and the computation of representations can be parallelized across all nodes and edges.
For the MLP computation, PET inputs the target node embedding for prediction. The complexity of it is (d²)), which independent of the number of feature fields. In addition, the number of model parameters of PET is light. Both other methods (e.g., DeepFM, RIM, etc.) and PET have embedding tables of the same size Nd. The main differences lie in the linear layers of models. And for PET, the number of parameters of linear layers are (d²)) which is independent of the number of fields.
4 Experiments
In this section, we show the experimental results and the corresponding settings. Generally, we experiment on two kinds of tabular data prediction tasks: top-n recommendation. The results showcase the value of our model on tabular data prediction tasks. Additionally, we conduct several ablation studies to validate the components of our model.
rate (CTR) prediction and
4.1 Setup
Table 1: Dataset statistics.
We evaluate the performance of PET on five datasets. For the CTR prediction task, we conduct experiments on three large-scale datasets, i.c.. Tmal Taobaq and Alipay For the top-n recommendation task, we experiment on two widely-used public recommendation datasets, i.e., Movielens IM and LastFM The statistics of the used datasets are summarized in Table
Datasets
Samples
Fields
Tmall
54,925.331
9
Taobao
100,150,807
4
Alipay
35,179.371
6
Moviclens- 1M
1,000,209
7
LastFM
18.993.371
5
The evaluation metrics include area under ROC curve (AUC) and negative log-likehood (LogLoss) for the CTR prediction task and hit rate (HR), normalized discounted cunulative gain (NDCG), and mean reciprocal rank (MRR) for the top-n recommendation task.
Following Qin et al. (021), we spilt the datasets according to the global timestamps. The carliest data instances are grouped into the retrieval pool. The latest data instances form the test pool. Then the remaining data instances are grouped into the train pool. To avoid unfair comparisons, the non-retrieval model takes the retrieval pool as additional train pool.
On the CTR prediction task, we compare our model against nine widely-used and strong baselines. GBDT (Chen and Guestrin 2016) is the widely-used tree model. DeepFM (Guo et al. 2017) is the inner-product interaction- based model. FATE (Wu et al.) (202T) and TabGNN (Guo et al. 2021b) are the recent graph models. DIN (hou et al. 2018) and DIEN (Zhou et al. 2019) are the attention- based sequential models. SIM (Pi et al.) 2020). UBR (Qin et al. 2020), and RIM (Qin et al., 2021) are the retrieval-based models. On the top-n recommendation task, we compare PET with six strong recommendation models, including factorization- based FPMC (Rendle et al., 2010) and TransRec (He al. 2017), and recently proposed DNN models NARM (Li et 2017). GRU4Rec (Hidasi ct al. 2016), SASRec (Kang and McAuley 2018), and RIM (Qin et al. 2021).
As for the hyperparameters, we test the number of GNN layers in (2.3). The embedding sizes of all the models are consistent to ensure the fair comparison. More detailed hyperparameters and experiment settings are provided in Appendix A.5

---

Table 2: Result comparisons with baselines on CTR prediction task. (K" 10)
Tmall
Taobao
Alipay
Models
AUC
LogLoss
Rel.Impr.
AUC
LogLoss
Rel.Impr.
AUC
LogLoss
Imp.
GBDT
0.8319
0.5103
12.08%
0.6134
0.6797
44.08%
0.6747
0.9062
32.36%
DeepFM
0.8581
0.4695
8.66%
0.6710
0.6497
31.71%
0.6971
0.6271
28.1%
FATE
0.8553
0.4737
9.01%
0.6762
0.6497
30.70%
0.7356
0.6199
21.40%
TabGNN
0.8945
0.4158
4.24%
0.7294
0.6173
21.17%
0.8086
0.5849
10.44%
DIN
0.8796
0,4292
6.00%
0.7433
0.6086
18.90%
0.7647
0.6044
16.78%
DIEN
0.8838
0.4445
5,50%
0.7506
0.6084
17.74%
0.7502
0.6151
19.03%
SIM
0.8857
0.4520
5.27%
0.7825
0.5795
12.95%
0.7600
0.6089
17.50%
UBR
0.8975
0.4368
3.89%
0.8169
0,5432
8.19%
0.7952
0.5747
12.30%
RIM
0.9138
0.3804
2.04%
0.8563
0,4644
3.21%
0,8006
0,5615
11.54%
PET
0.9324
0,3321
0.8838
0.4162
0.8930
0,4132
Table 3: Result comparisons with baselines top-n recommendation task. (K
10)
Datasets
Metric
FPMC
TransRec
NARM
GRU4Rec
SASRec
RIM
PFT
HR@ 1
0.0261
0.0275
0.0337
0.0369
0.0392
0.0645
0.0904
HR@5
0.1334
0.1375
0.1418
0.1395
0.1588
0.2515
0.2889
HR @ I0
0.2577
0.2659
0.2631
0.2624
0.2709
0.4014
0.4404
ML-IM
NDCG@5
0.0788
0.0808
0.0866
0.0872
0.0981
0.1577
0.1903
NDCG@ 10
0.1184
0.1217
0,1254
0.1265
0.1341
0,2059
0.2390
MRR
0.1041
0,1078
0,1113
0.1135
0.1193
0.1704
0.2006
HR@
0.0148
0.0563
0.0423
0.0658
0.0584
0.0915
0.1149
HS@
0.0733
0.1725
0.1394
0.1785
0.1729
0,3468
0.3621
LastFM
HR
0.1531
0.2628
0.2227
0,2581
0,2499
0.5780
0.6033
NDCG@5
0.0432
0.1148
0.0916
0.1229
0.1163
0.2165
0,2381
NDCG@
0.0685
0,1441
0.1185
0.1486
0.1409
0.2911
0.3156
MRR
0.0694
0.1303
0.1083
0.1362
0.1289
0,2210
0.2492
4.2 Overall performance comparison
We first validate the effectiveness of the proposed PET model. The main results are summarized in Tables[aand) where we can see the proposed PET performs consistently better on all datasets.
The results demonstrate the superiority of PET against the baselines both tasks. On the CTR prediction task, PET achieves relatively 2.04%, 3. 3.21%, 10.44% higher AUC over the best performed baseline on Tmall, Taobao, Alipay, respectively. The results show that PET can learn effective tabular data representation for better prediction performance. Furthermore, compared with RIM that takes exactly the same inputs with PET, the improvements of PET are statistically significant under 95% confidence level. Given that RIM also utilizes the labels of relevant rows, this empirically justifies the capability of our propagation method. On the top-n recommendation task, PET shows significant improvements on the recommendation task against other baselines, too. The results show that PET performs substantially better than the strong baselines in comparisons.
4.3
Ablation study
4.3.1 Impact of the label usages
We further study the usage of labels of PET. The results are summarized in Table 4 For fast exploration, we randomly sample 10% data from training pool and test pool on the three large CTR datasets, respectively. The best performed baseline RIM is tested on the sampled data for comparison. Both RIM and PET utilize the same set of retrieved data instances and their label information. We then examine the impact of different label usages on the sampled data. In the PET model, we initialize the embeddings data instances nodes and edges with the corresponding label embeddings. We then inspect the impacts of such initialization and the operations edges. For convenience, we calculate the homophily ratios for the resulting instances set. The homophily ratios for Tmall, Taobao, and Alipay are 0.5566. 0.61 I1 and 0.5097, respectively.
8

---

Table 4: Impact of label embeddings. (On randomly sampled data, K 10)
Tmall
Taobao
Alipay
Models
AUC
LogLoss
AUC
LogLoss
AUC
LogLoss
RIM
0.9120
0.3769
0.8587
0.4586
0.7845
0.5742
PET PET (w/o edge labels)
0.9279
0.3387
0.8762
0.4279
0.8720
0.4201
0,9291
0.3367
0.8665
0.4465
0,8558
0,4776
PET (w/o node labels)
0.9233
0.3494
0.8431
0,4847
0.8518
0,4799
PET (w/o all labels)
0.9208
0.3568
0.8416
0.4815
0.8096
0.5719
We first remove edge embedding initialization with labels and eliminate all the operations involved with edges, thus the message becomes
(1) mi h:j (1-1)
(12)
In addition, the edge embedding update step in Equation is also omitted. As such, the labels only serve from propagation among nodes. The high-order product interaction between features and label-feature interaction are omitted. As shown in Table 4 PET without edge labels performs weaker than the original PET model, which demonstrates the power of high-order product interactions and label- eebennenhanced features. Moreover, PET without edge labels still performs better than RIM, which validates the superiority of label propagation through common feature nodes against simple iitention-based aggregation.
By removing the node labels, the embeddings of all the data instance nodes initialized constant vectors. In this way, no pure first-order label embeddings are used for prediction. Since any information related with labels will be propagated to nodes after product interaction with features. One can see that PET without node labels performs better than RIM, which further justifies the effectiveness of high-order product interactions and label-enhanced features. Additionally, PET without node labels performs weaker than the original PET, which implies the power of initializing node embeddings with pure label embeddings for propagation.
We also provide PET (w/o all labels) to see the feature interactions under the framework. Concretely, randomly initialize the label values. The results show the power of feature interactions and the advantages of feeding in labels.
4.3.2 Visualization of the feature distribution
In order to further explore the representation enhancement of PET, visualize the features of PET and RIM with t-SNE (Van der Maaten and Hinton| 2008). Figure Iillsstrates the distributions of data instance embeddings and the feature embeddings of PET and RIM Tmall. For both models, randomly choose 10,000 samples from the train data and 10,000 samples from the test data. For PET, the data instance embeddings are taken from the data instance nodes (inputs of the MLP predictor) and the feature embeddings are the concatenated feature node embeddings for each data instance. For RIM, the data instance embeddings are the inputs of the final MLP predictor and the feature embeddings are the concatenated feature embeddings of each data instance.
that the distributions of train positive data points and train negative data points are more dissimilar than those of RIM. The same phenomenon happens in the distributions of test positive data points and test negative data points. This illustrates that PET gives more informative representations. (Visualizations on more datasets can be found in Appendix A.2
From Figure 4
4.3.3 Impacts of different retrieval schemes
First, we evaluate the effectiveness of current retrieval by comparing the current retrieval scheme with the random retrieval scheme. The results are summarized Table 5 One can see that the relevance-basee basse retrieval performs much better than random retrieval. This validates the effectiveness of the relevance- based retrieval and feeding in relevant rows.
We also study the impact of using different retrieval sizes K. The results are put in Appendix A.3 due to the page limit. Generally, too few retrieval samples may fail to carry enough information to enhance the representation, while too many retrieved samples will introduce more noise.

---

Ohitad centen Bhor
Detp cndesbeee (Bo)
OUSAE ITAI
Figure 4: The t-SNE visualization of data and feature embeddings Tmall.
Table 5: Impacts of the retrieval schemes.
Tmall
Taobao
Alipay
Models
AUC
LogLoss
AUC
LogLoss
AUC
LogLoss
Random retrieval
0.8433
0.4922
0.6544 0.8838
0.6572
0.7271
0.6120
Relevance retrieval
0.9324
0.3321
0.4162
0.8930
0.4132
Conclusion
In this paper, focus improving the prediction of tabular data, which is essential in many important downstream tasks. Existing methods either take each data instance independently or directly take multiple data instances input without enhancing the target data instance representation. We propose to construct retrieval-based hypergraph to model the cross-row and cross-column relations of tabular data, utilizing the propagation the resulting graph to directly change and enhance the target data instance representations. Concretely, we utilize relevance retrieval to of the hypergraph, aiming to resort relevant patterns. Then design
construct the hyperedges PET, novel architecture that propagates and enhances the tabular data representations based on the hypergraph for target label prediction. The propagation serves from three aspects: label propagates through common feature values; 2) features get enhanced through locality and high-order product feature interactions are generated through the interactive message passing framework; and 3) labels are used to generate the label-enhanced features. Experiments on two important tabular data prediction tasks validate the superiority of the proposed PET model over strong baselines.
Acknowledgments and Disclosure of Funding
The Shanghai Jiao Tong University Team is supported by Shanghai Municipal Science and Technology Major Project (2021 62177033). We would also like thank Wu Wen Jun Honorary Doctoral Scholarship from AI Institute, Shanghai Jiao Tong University.
and National Natural Science Foundation of China (62076161,
10

---

References
Sameer Agarwal, Kristin Branson, and Serge Belongie. 2006. Higher order learning with graphs. In Proceedings of the 23rd international conference on Machine learning. 17-24.
Song Bai, Feihu Zhang, and Philip HS Torr. 2021. Hypergraph convolution and hypergraph attention. Pattern Recognition 110 (2021), 107637.
Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez. 2013. Recommender systems survey. Knowledge-Bass Based Systems 46 (2013), 109-132.
Richard J. Bolton and David J. Hand. 2002. Statistical Fraud Detection: A Review. Statist. Sci. 17, 3 (2002), 235-249.
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd aem sigkdd international conference on knowledge discovery and data mining. 785-794.
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Wang. 2020. HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data.
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In DLRS@ RecSys.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. 2019. Hypergraph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 3565-.
Jerome H Friedman. 2001. Greedy function approximation: gradient boosting machine. Annals of statistics (2001), 1189-1232.
Huifeng Guo, Bo Chen, Ruiming Tang, Weinan Zhang, Zhenguo Li, and Xiuqiang He. 2021a. An embedding learning framework for numerical features in etr prediction. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2910-2918.
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a
factorization-machine based neural network for CTR prediction. IJCAI.
Xiawei Guo, Yuhan Quan, Huan Zhao, Quanming Yao, Yong Li, and Weiwei Tu. 2021b. TabGNN: Multiplex Graph Neural Network for Tabular Data Prediction. arXiv preprint arXiv:2 108.0220 (2021).
Ruining He, Wang-Cheng Kang, and Julian J. McAuley. 2017. Translation-based Recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, August 27-31, 2017, 161-169.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session- based Recommendations with Recurrent Neural Networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings.
Wang-Cheng Kang and Julian J. McAuley. 2018 Self-Attentive Sequential Recommendation, In IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018. 197-206.
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural Attentive Session-bas -baed Recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, Singapore, November 06 - 10, 2017. 1419-1428.
Nicolas Papernot and Patrick McDaniel. 2018. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. arXiv preprint arXiv: /803.04765 (2018).
Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click- Through Rate Prediction. CIKM '00: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020. ACM, 2685-2692.
11

---

Jiarui Qin, Weinan Zhang, Rong Su, Zhirong Liu, Weiwen Liu, Ruiming Tang, Xiuqiang He, and Yong Yu. 2021. Retrieval & Interaction Machine for Tabular Data Prediction. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1379-1389,
Jiarui Qin, Weinan Zhang, Xin Wu, Jiarui Jin, Yuchen Fang, and Yong Yu. 2020. User Behavior Retrieval for Click-Through Rate Prediction. In Proceedings of the 43rd International ACM SIGIR conference research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020. ACM, 2347-2356.
Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang. Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data. ACM Transactions on Information Systems (2018).
Steffen Rendle. 2010. Factorization machines. In ICDM.
Steffen Rendle, Christoph Freudenthaler, and Lars Soamidt-Thhieme 2010. Factorizing personal- ized Markov chains for next-basket recommendation. Proceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, April 26-30, 2010. 811-820.
Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting Clicks; Estimating the Click-Through Rate for New Ads. In Proceedings of the 16th International World Wide Web Conference(
Stephen Rohertson, S. Walker, S. Jones, M. M. Blancock-Beaulinn and M. Gatford. 1995. Okapi at TREC-3. In Overview of the Third Text REtrieval Conference (TREC-3). 109-126.
Balasubramaniam Srinivasan, Da Zheng, and George Karypis. 2021. Learning over Families of Sets-Hypergraph Representation Learning for Higher Order Tasks. In Proceedings of the 202 I SIAM International Conference on Data Mining (SDM). STAM, 756-764
Laurens Van der Maaten and Geoffrey Hinton. 2008 Visualizing data using I-SNE, Journal of machine learning research 9, 11 (2008).
Saurabh Verma and Zhi-Li Zhang. 2019. Stability and Generalization of Graph Convolutional Neural Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, 1539-1548.
Qitian Wu, Chenxiao Yang, and Junchi Yan. 2021. Towards Open- World Feature Extrapolation: An Inductive Graph Learning Approach. In Advances in Neural Information Processing Systems: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L.. Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In Pro- ceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
Jiaxuan You, Xiaobai Ma, Daisy Yi Ding, Mykel J. Kochenderfer, and Jure Leskovec. 2020. Handling Missing Data with Graph Representation Learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click- Through Rate Prediction. In The Thirty- -hird AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 February I, 2019. 5941-5948.
Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018 Deep Interest Network for Click-Thoughh Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018. 1059-1068.
12


Reviews:
[{"review_id": "tQ5vzfQ1CRT", "reviewer": "Reviewer_9mtT", "summary": "The authors propose a graph approach that connects an item to classify and its features with relevant items via the common values. This graph is used to propagate feature and label information and then classify the item. Comparisons are made in multiple datasets for two tasks (recommendation and classification) with several state-of-the-art approaches. The results show that the proposed approach can improve the performance on the specific metrics.", "questions": "One question I have is about the relevant item extraction. This works well for binary or categorical features but did you try with real-valued features?\n\n\n", "limitations": "As I mentioned previously the retrieval process as defined seems to limit the applicability of the approach for the kind of features the authors consider. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is an interesting approach and based on the results it seems to improve upon various state-of-the-art models. It is described quite well, even though I would like to see a bit more detailed description of the implementation. Some algorithm pseudo-code would help. Experiments are good with several models for comparison. The results are also good and not negligible I would say.\n\nSomething that is missing in the paper is the detailed description of how the baselines were tuned. For example, DeepFM is quite a strong approach but one need to tune properly from my experience. So, I am not sure if the difference is due to the approach that better captures and propagates the relevant information or just note well tuned approaches.\n\nThe authors could also have added graph approaches like for example GraphSage or other state-of-the-art ones.\n\nAlso, the retrieval of relevant instances is thoroughly studied. This has a big impact I imagine on the learning process. I would like to see here different scenarios and more analysis. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "7zuKuU8UFSD", "reviewer": "Reviewer_8FC8", "summary": "The authors propose a novel architecture for learning representations over “tabular data”. The problem setup is highly relevant as learning over tabular data has diverse practical implications in multiple fields. The proposed architecture, named “PET: Propagate and Enhance the Tabular data”, utilizes multiple components at its core - (i) relevance based retrieval to obtain $K$ instances similar to the target data instance;  (ii) message passing (over the star expansion of the generated hypergraph); (iii) label enhanced feature learning; (iv) higher-order feature interaction. To retrieve the relevant instances, the authors utilize the “relevance value” (proposed by Qin et al., 2021), which is akin to BM25 when the data instances are treated as documents and their feature values as the terms, as mentioned by the authors. The resulting set of the original instance along with the retrieved ones is processed as a hypergraph, where each distinct feature value in each individual field forms a node and each data instance constructs a hyperedge. This is further subjected to the star expansion to construct a bipartite graph. Subsequently, few steps of message passing are applied over the graph, with multiple operations involved. The messages are generated by the interaction of the edge and node representations - eq 6. The aggregation of the messages for each node is done by an attention mechanism, attributed to the widespread success of the basic attention module. The node embedding update is then performed via a transformation over the concatenation of previous layer representation with the aggregated messages. The edge embedding update follows a similar process. Finally, the target data node representation is passed through MLP for prediction and the model is learned end-to-end (cross-entropy is used for binary prediction tasks). As emphasized by the authors multiple times over the course of the paper, the proposed architecture aids the feature learning process by inducing the label information into the representations as well as allowing for higher order feature interactions with more rounds of propagation. The empirical analysis is done on 5 majorly used datasets and comparison has been performed against multiple baselines. ", "questions": "1. In section 4.3.1, the authors have performed two primary ablations by - (i) removing “edge labels” and (ii) removing “node labels”. Do the authors have some results where both “edge” and “node” labels are removed? Essentially, only the “feature interactions” are captured in this setup. This might help validate the efficacy of the structure of the proposed architecture.\n2. The authors have utilized “directional” edges in the message passing mechanism. Do the authors have supporting arguments and/or experiments for the case where the graph obtained post the “star” expansion is undirected? In this case, the edge embeddings can be a composition of the two nodes. The composition function can be a simple operation like concatenation, which has been heavily utilized in this work, or something more sophisticated - MLP.\n3. Can the authors also describe how they arrived at the hyperparameter values provided in table 8 of the appendix?\n4. From figure 7 in the appendix, it is evident that $K=15$ or $K=20$ provide the best results. Is there a specific reason that the authors have used $K=10$ as shown in the hyperparameter table 8 of appendix? \n", "limitations": "As mentioned in section A.1 of the appendix, the constructed graph can be dense when the number of fields is large. Moreover, the graph is created for **each** target instance, which can be computationally expensive. The problems of oversmoothing and oversquashing may occur in such cases. I commend the authors for explicitly stating this. I also believe that the assumption that relevance based retrieval can help realize the homophily may not always hold, as described previously. Although such scenarios can be rectified by utilizing the enhancements proposed in the recent literature focused on learning GNNs under heterophily (such as [1]), I would like to know the authors’ response to this argument. Lastly, I don’t see any serious negative societal impacts of the work. \n\n[1] ​​Graph Neural Networks with Heterophily - Jiong Zhu, Ryan A. Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K. Ahmed, Danai Koutra\n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The results provided in tables 2, 3 and 4 “strongly” demonstrate the efficacy of the proposed model. The improvements in both - CTR and top-n recommendation tasks are significant.\n2. Figure 4 provides interesting visualizations that demonstrate the efficacy of the proposed method PET and inline with the experimental results.\n3. While the authors have provided some hyperparameter and sensitivity analyses for label usage (sec 4.3.1), retrieval schemes (sec 4.3.3) and $K$ (appendix A.3), it will be also interesting to see analyses of important attributes like the number of GNN layers and embedding size as well.\n4. The authors utilize retrieved neighbors (eq 1 and 2) to obtain the representation of the target data instance under message passing framework. Furthermore, they mention explicitly in section 4.3.3 that the “relevance-based retrieval” can help achieve homophily, and thus improve performance. However, in the cases when relevance based retrieval fetches $K$ instances that are close in the input space, but have different labels, the method can give highly sub-par performance, particularly when the GNN layers are more than $2$. Such scenarios can occur when the output class label is heavily dependent on a few “fields” and the data instances have high similarity in the irrelevant fields (more in count). Can the authors provide some arguments and/or experiments under this setup?\n5. The authors have mentioned prior works utilizing GNNs over tabular data in section 2 (lines 77-79), however the comparison in tables 2 and 3 doesn’t contain these baselines. Can the authors elaborate on this? It’s strongly recommended to provide the comparison for the completeness of the work.\n6. Tables 6 and 7 of the appendix provide the corresponding standard deviations for the proposed model. I suggest the authors provide the same for the second best method, ie the best performing baseline, for completeness as well.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1LwM2wG76Se", "reviewer": "Reviewer_3NXB", "summary": "The authors propose PET, an architecture that Propagates and Enhances the Tabular data representations for label prediction using hypergraphs.  Given tabular data, the novel end-to-end system consists of of several steps, beginning with hypergraph construction based on information retrieval, followed by partitioning of labels and feature into a bipartite graph, then several rounds of message passing to refine high-level interactions between nodes and edges of the bipartite graph, and finally target node embeddings are fed to an MLP for label prediction.  PET is shown to significantly outperform a wide variety of state-of-the-art label prediction methods for two tasks: click-through rate (CTR) prediction and top-n recommendations.  Furthermore, due to the complicated nature of the end-to-end system, an extensive ablation study is conducted to understand the affects different components of the model have on downstream performance.", "questions": "What dataset are the T-SNE plots derived from?  Interestingly, PET is learning overarching structure inherent in the data, whereas RIM seems to just learn small clusters.  It would be interesting to see the influence of utilizing edge and node labels on the ability to learn this\nstructure, i.e., repeating study in Section 4.3.1 to regenerate these T-SNE plots and viewing whether PET (w/o node labels) learns less structure in the data (like RIM T-SNE plots) and the overarching structure becomes more apparent with each edge addition.\n\nCan you comment on the process for choosing PET hyperparameters for the different datasets?  As the hyperparameters differ per dataset, how much computational time does hyperparameter optimization require per dataset?\n\nPET is complicated, can the authors comment on the computational price incurred for the various steps, i.e., hypergraph construction/information retrieval, star expansion, message passing, embedding calculations, MLP training time, and hyperparameter tuning across all these various steps?  How does this compare to RIM?\n\nQuestion from the previous section: Are the various steps within PEP tractable when the number of features grows on the order of dozens/hundreds, as is commonly encountered in large-scale binary classification datasets?", "limitations": "NA", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality\nStrengths: While PET borrows inspiration from previous methods (i.e, RIM also uses information retrieval to learn inter-row -column high-level interactions), the use of hypergraph neural networks for tabular data prediction is novel and extremely interesting.  \n------------------\nQuality:\nStrengths: The approach, methodology, and experiments are all well done.  In particular, the various stages of PET make intuitive sense at drawing out and refining high-level interactions within the data to improve target node prediction.  Furthermore, the ablation study was thorough and illustrative to understand the impact of the hypergraph neural network on downstream performance, as well as visualizing the structured latent space learned in PET compared to the nearest SOTA competitor RIM.\n\nWeaknesses: There are three large concerns given PET.  While the overall model has been shown to perform extremely well, model complexity is high.  While exact runtimes and the necessary compute required to put this model into action for large-scale data are not currently discussed, it does not seem like all researchers will have access to compute resources necessary to train and deploy such a model.  \n\nThe second concern is related to the first; how does runtime scale with the number of features considered?  While the datasets utilized in the paper contain large numbers of samples, they contain very small numbers of fields.  Are the various steps within PEP tractable when the number of features grows on the order of hundreds, as is commonly encountered in large-scale binary classification datasets?\n\nFinally, it is not clear how computationally expensive hyperparameter optimization of PET is, nor how often this is necessary (each presented dataset has a different configuration of hyperparameters).\n\n------------------\nClarity:\nStrengths: The paper is very well written.\n\nWeaknesses: A necessasry discussion of hyperparameter tuning for the various methods evaluated is currently lacking in the paper.  In particular, for the competitors in Table 2 and 3, the choice of hyperparameters is not discussed.\n\n------------------\nSignificance:\nStrengths: GNNs are an extremely active area of research right now.  Coupled with the impressive performance demonstrated for two important tasks, PET has the potential to significantly influence current work looking to similarly leverage GNNs to capture high-level latent interactions among features for different prediction tasks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept