RIGNN: A Rationale Perspective for Semi-supervised Open-world Graph Classification
Authors: Xiao Luo
Yusheng Zhao
Zhengyang Mao
Yifang Qin
Wei Ju
Ming Zhang
Yizhou Sun
Venue: Accepted by TMLR ()

Abstract: Graph classification has gained growing attention in the graph machine learning community and a variety of semi-supervised methods have been developed to reduce the high cost of annotation. They usually combine graph neural networks (GNNs) and extensive semi-supervised techniques such as knowledge distillation. However, they adhere to the close-set assumption that unlabeled graphs all belong to known classes, limiting their applications in the real world. This paper goes further, investigating a practical problem of semi-supervised open-world graph classification where these unlabeled graph data could come from unseen classes. A novel approach named Rationale-Informed GNN (RIGNN) is proposed, which takes a rationale view to detect components containing the most information related to the label space and classify unlabeled graphs into a known class or an unseen class. In particular, RIGNN contains a relational detector and a feature extractor to produce effective rationale features, which maximize the mutual information with label information and exhibit sufficient disentanglement with non-rationale elements. Furthermore, we construct a graph-of-graph based on geometrical relationships, which gives instructions on enhancing rationale representations. In virtue of effective rationale representations, we can provide accurate and balanced predictions for unlabeled graphs. An extension is also made to accomplish effective open-set graph classification. We verify our proposed methods on four benchmark datasets in various settings and experimental results reveal the effectiveness of our proposed RIGNN compared with state-of-the-art methods.

Published in Transactions on Machine Learning Research (09/2023)
RIGNN: A Rationale Perspective for Semi-supervised Open- world Graph Classification
Xiao Luo
zinoluo &ss. ucla edu yuskeng, thao @stu. tu.pku. edu. zhengyong, mao @stu. Astu.pku. edu. qinyrifang Lpku. edu. juwes ppku. edu.co mzhang es Upku. edu, on yzsun Bes. uelaa.laa..............e........ edu
Yusheng Zhao
Zhengyang Mao
Yifang Qin
Wei Ju
Ming Zhang?
Yizhou Sun-
Department of Computer Science, University of California, Los Angeles School of Computer Science, Peking University
Reviewed on OpenReview: https: // openrevieww net/ forum? id= qcCE4mC2j1
Abstract
Graph classification has gained growing attention in the graph machine learning commi- nity and a variety of semi-supervised methods have been developed to reduce the high cost of annotation. They usually combine graph neural networks (GNNs) and extensive semi- supervised techniques such as knowledge distillation. However, they adhere to the closed-set assumption that unlabeled graphs all belong to known classes, limiting their applications in the real world. This paper goes further, investigating a practical problem of semi-supervise open-world graph classification where these unlabeled graph samples could come from un- classes. A novel approach named Rationale-Ifformed GNN (RIGNN) is proposed, which takes rationale view to detect components containing the most information related to the label space and classify unlabeled graphs into a known class or an unscen class. In particular, RIGNN contains a relational detector and a feature extractor to produce eflec- tive rationale features, which maximize the mutual information with label information and exhibit sufficient disentanglement with non-rationale elements. Furthermore, we construct a saph-of-graph based on geometrical relationships, which gives instructions on enhancing rationale representations. In virtue of effective rationale representations, accurate and balanced predictions for unlabeled graphs. An extension is also made to complish effective open-set graph classification. We verify our proposed methods on four benchmark datasets in various settings and experimental results reveal the effectiveness of our proposed RIGNN compared with state-of-tho-art methods.
can provide
1 Introduction
Recently, graph-structured data has become omnipresent in the real world (Chen et al. 2022b), and graph classification has received extensive attention with applications in various fields such molecular chemistry and social analysis (Hansen et al.) 2015; Ying et al. 2018a Lee et al. 2019b Ying et al. 2018b). Graph neural networks (GNNs) have been demonstrated to be eflicient and adaplable for this topic due to their strong capability of representation learning (Lu et al., 2019 Schütt et al. 2017 Gilmer et al. 2017). To specific, every node receives information from neighboring nodes, which is then aggregated for incremental node embedding updating. A readout operator is adopted to combine all of these node representations into graph-level representation after few iterations (Ying al.
2018b Lee et. al.] 2019b). In this fashion,

---

Published in Transactions on Machine Learning Research (09/2023)
Labeled Graphs
Known Classes
Unlabeled Graphs
Novel Classes
Figure 1: An illustration of our open-world setting. We are required to classify each unlabeled graph example into either one of the known classes or a corresponding novel class.
the learned graph representation is able to reveal the graph structural semantics for effective downstream classification.
Although GNNs have been empirically shown to be effective on numerous benchmarks, they are incredibly data-hungry Gilmer et al.) 2017). Considering that acquiring labels in the real world is usually expensive, one of the dominant solutions to reduce the labeling cost by semi-supervised learning, which makes use of
abundant unlabeled graphs and limited number of labeled graphs to train GNNs (Sun et al.) 2020aa] Hao
et al. 2020; Yang et al. 2022a Li et al. 2022a Yue al. 2022a]. These techniques either use knowledge distillation where a teacher model is imposed to learn generalized graph representations, pseudo-labeling to annotate unlabeled graphs using their own model. These works almost adhere to the closed-sct assumption that unlabeled graphs share the same label space as labeled graphs. Unfortunalely, the raw unlabeled set could include samples from unidentified classes in real-world applications. Towards this end, this work generalizes semi-supervised graph classification to a more practical setting called semé-superoised open-world graph classification, in which partial unlabeled graphs could belong unknown classes. In particular, we need to classify each unlabeled graph example into either one of the known classes or corresponding novel class. Figure provides example of our problem where colored graphs are with annotations and gray ones are not. Within the same scenario, similar problem named semi-supervvsed open-sel graph. classification aims to not only classify samples from known classes correctly but also detect sample graphs from novel classes without further subdivision. We also expect that a generalized algorithm can be casily extended to this similar problem.
The obstacle to semi-supervised open-world graph classification the broken closed-set assumption. Several studics open-world recognition primarily concentrate images and texts (Rizve et al. 2022; Cao ct al. to tackle new challenges as follows: (1) Complex structured data. Our problem needs to deal with both attribute-level and structure-level information with varying graph sizes, densities and homophily. Worse yet, the involution of samples from novel classes would further disturb the representation learning of samples from known classes. (2) The impact of nonerucial components. The complex data generation procedure may include crucial and nonerucial components and only the former is highly related to target label information. How extract these informative messages from graphs meanwhile reducing the impact of nonerucial components for effective classification remains open problem. (3) Serious label scarcity. We would encounter
2022 Nayeem Rizve et al. 2022). while our problem irregular graph data requires
scrious label scarcity in this problem, especially for novel classes, which could deteriorate the performance of existing semi-supervised GNN-based methods YYue et al., 2022a). Therefore, an effective strategy to extract approximate semantic information from unlabeled graphs is urgently anticipated.
This work provides a rationale perspective to tackle the problem of semi-supervised open-world graph classi- fication. In particular, we first comprehend the challenges in this problem by understanding the logic of the data generation process where graph is made up of rationale and non-rationale components. Then, a novel
2

---

Published in Transactions on Machine Learning Research (09/2023)
method named Rationale-Informeet GNN (RIGNN) is developed, which integrates rationale learning into effective graph representation learning. To be more precise, we first build a relational detector to select the crucial components and a feature extractor is utilized to extract them into rationale representations. To learn rationale information related to target semantics, we not only maximize the mutual information between rationale representations with the same semantics, but also minimize the mutual information wetw ratio- nale features and their non-rationale features generated by complementary components. Both contrastive learning and adversarial learning are adopted to implement effective rationale representation learning. In measure the pairwise distance between rationale features and then
addition, to tackle the label scarcity, construct a graph-of-graph based on geometrical relationships, which guides the enhancement of rationale representations. We further add a regularization term to guarantee accurate and balanced predictions for unlabeled graphs. Our work can also be easily exlended to accomplish effective open-set graph classification, where outliers only need to be rejected from datasets rather than elaborate classification into different novel classes. We verify our proposed methods on four benchmark datasets in various open-world and open-sct settings and experimental results reveal the effectiveness of our proposed RIGNN compared with variely of state-of-the-art the-art methods. The contribution of this paper can be summarized as follows:
New Problem: We study the problem of semi-supervised open- world graph classification, which breaks the close-set assumption for more generalized flexible real-world applications.
Novel Approach: We develop novel approach named RIGNN, which involves a rationale perspective in effective graph representation learning. Moreover, graph-of-graph is constructed to extract the semantic guidance in unlabeled graphs for the enhancement of graph representations.
Extensive Erperiments: We verify the effectiveness of our proposed RIGNN by comparing with competitive baselines on four benchmark datasets in various settings.
2
Related Work
2.1 Graph Classification
Graph neural networks (GNNs) have gained growing attention for graph machine learning problems in recent
years (Guo et al. 2022 Zhao et al. 2021 Liu et al. 2021). Graph classification is one of these fundamental
problems with extensive applications in computer vision (Jiao et al., 2022 social analysis (Wu et al. 2019)
and biology (Xia & Ku/ 2021). GNN-based approaches usually follow the message passing mechanism Ying
et al. 2018b Lee et al. 2019b), which combines structural semantics and node attributes in an iterative fashion. These node representations
then compressed into graph representation for classification using pooling procedure. Due to the restricted availability of labels in the real world, semiisuperrvsed graph classification methods have become more popular in research. These approaches use a large number of unlabeled graphs and a few of labeled graphs to maximize the performance of G.NNs (Li et al. 2019 Sun et al.) 2020a Hao et al. 2020: You et al. 2020b Ju et al.) 2022 Yang et al. 2022a). However, they do not take into account. the situation that the raw graph set could contain samples from unidentified classes. In light of this, we investigate a generalized and practical problem of semi-supervised open-world graph classification.
2.2 Rationale Extraction
Rationale extraction or rationale discovery has been incorporated into numerous machine learning appli- cations such as video question answering [i et al., 2022c). domain generalization (Zhang et al., 2028 sentiment analysis (Yue et al. 2022b), and text classification (Chan et al. 2022 The basic idea of rationale extraction is to extract the crucial part of the input, which facilitates the model performance and explainabil- ity. For example, UNIREX explores different Transformer-based rationale extractors to fit. multiple priors, which benefit language models in multiple tasks, This topic can also be combined with causality (Zuo et al. 2022) by introducing different intervention techniques (Li et al.] 2022b; Wang et al. (022). Rationale ex- traction has recently been combined with GNNs to overcome potential dut-of-distributtin shifts in graph classification (Sui et al. 2022 Yang et al. 2022b). RGCL. [Li et al. 2022b) studies the invariant rationale
3

---

Published in Transactions on Machine Learning Research (09/2023)
discovery and then generates augmented graphs from rationale-aware perspective for effective graph con- trastive learning. CIGA (Chen et al. 2022b) describes potential distribution variances graphs with causal models and extends the invariance principle to graph data. Compared with these methods, our proposed RIGNN learns rationale features based on information theory under label scarcity, which facilitates effective graph classification in both open-world and open-sel settings.
2.3 Open-set and Open-world Recognition
Open-sel recognition expects the model to reject instances from new classes while taking into account the inductive learning configuration (Sun al.) 2020b) Zhou al. 2021 Kong & Ramanan 2021 Xia et all 2021). Open-world recognition further requires to separate these rejected instances based on their se- mantics (Rizve et al. 2022 Cao et al. 2022 Nayeem Riave et al. 2022). Existing open-sel and open-world techniques can be divided into generating and discriminative models. To match realistic environments, discriminative models often modify the softmax layer utilizing one-vs-rest units (Scheirer et al.) 2012). cali- bration (Scheirer et al., 2014) and optimal transport (Rizve et al.) 2022). In contrast, generative models use conditional auto-encoders (Oza Patel 2019) and data augmentation (Ditria al. 2020) to forecast. the distribution of unobserved classes. Recently, self-supervised learning has been incorporated
learn from augmented samples (Rizve et al., 2022). Open-set recognition has been further considered simultaneously with domain shifts (aanareda Busto & Gall 2017). However, these methods usually focus on Euclidean data, while our RIGNN aims to handle complicated graph data and extract crucial features from a rationale perspective.
3 Preliminaries
3.1 Problem Definition
A graph is denoted as G EV, where V and & is the node set and edge set, respectively. X E RI*X denotes the node attribute matrix with the attribute dimension F and A gax= denotes the adjacent matrix. In the setting of semi-supervised open-world graph classification, we have dataset D, which includes a labeled subset D' = G2, g2, ... [un] containing N' labeled samples and an unlabeled subset D= = (9 G2- 9v+ ) } containing N unlabeled samples. The class set of labeled data and the whole data denoted as C and C. Closed-world semi-supervised classification implies Ci = C while in our settings we have C¹ C C, and C" C) contains novel classes. We aim learn a model, which classifies unlabeled graphs from both known and novel classes into their corresponding classes in C.
3.2 Graph Neural Networks
We provide a brief overview of graph neural networks (Kipf & Welling 2017 Xu et al. 2019), which are mainstream techniques for encoding graph-structured data. They often adopt the neighborhood aggregation stratogy to extract structural data. In particular, the updating rule for each node i E g at layer 7 is written as follows:
n AGGREGATE" :jE (foT-D:JEENN)) :jEN(O))
(1)
COMBINE()
(1)
where ((1) collects the neighboring nodes around i. w and n (1) denote the node representation and the neighborhood representation at layer AGCREGATE"() and COMBINE! denote the aggregation and combination operators at layer I, respectively. After stacking L layers, readout operation is adopted to summarize all these node representations at the final layer into a graph-level representation z E Rdd where d is the hidden dimension. In formulation,
= READOUT (L (055)
(2)
where READOUT(.) could be represents averaging or complicated pooling procedures YYng et al.) 2018b;
Lee et al. 2019b)
4

---

Published in Transactions on Machine Learning Research (09/2023)
3.3 Graph Contrastive Learning
We briefly introduce the framework of graph contrastive learning for unsupervised graph representation learning (You et al.] 2020b] 2021). Typically, these methods usually maximize the mutual information between input graphs and their representations by comparing the similarity between two augmented views of each input with the similarity between different samples. Given a dataset {G] and a stochastic augmentation operator T(-), they first construct positive pairs ( (
Ji-i with G.(²) T(G.). Then a graph encoder g()) transfer augmented graphs into representations, i.c., 2, = =g(g!-)) Given a batch B
and a temperature parameter 7, the normalized temperature-scaled cross entropy (NT-XENT) loss used to conduct contrastive learning:
log exp(2 (2
* (2)
L=
(3)
B
EBper exp(z, (1)
2,/1/1 /T)
B,EB
2
where * denotes the cosine similarity between two vectors and E calculates the empirical average over a dataset since every sample has the same probability to be sampled.
4 Methodology
4.1
Overview
This paper studies the problem of semi-supervised open-world graph classification. Although variety of methods have been put forward to address the label scarify in graph classification (Li et al. 2019 Sun et al.,
2020a; Hao et al., 2020 You et al.) 2020b Yang et al. 2022a] Xie et al., 2022.. they usually adhere to the close-world assumption that unlabeled graphs belong to known classes. This assumption restricts their applications in the real world.
Here, we propose novel method named rationale-infomme graph neural network (RIGNN) to solve this problem. The basic idea is to discover the rationale elements for effective graph representation learning. In particular, first comprehend the challenges in this problem, and then incorporate rationale discovery into graph representation learning based on information theory, which retains components related to semantics labels. Moreover, we construct graph-of-graph, which detects semantic proximity in unlabeled graphs to enhance our rationale-informed representation learning. Finally, we summarize our semi-supervised open- world learning framework and make an extension. More details can be seen in Figure 2
4.2 A Rationale View for Graph Generation
We first comprehend the challenges in this problem by illustrating the graph generation process. To begin, a graph G constructed using both rationale and non-rationale components, i.e., R and NR. Here is closely tied to intrinsic property which is highly relevant to our downstream classification. NR refers to the part irrelevant our target property, which can be varied due different backgrounds. However, previous graph classification approaches feed both R and NR as whole into mossage passing neural network, which clearly suffers from the influence of NR. Therefore, we expect to reduce the impact of NR to generate discriminative rationale representations, which is helpful to generate confident and accurate predictions Y even with Y E C
Then, we attempt to incorporate the logic in graph representation learning.
4.3 Representation learning via Rationale Extraction
To perform effective open-world classification, we need to be more cautious when generating rationale features to get rid of the impact of the non-rationale part. Since the data generation process cannot be intervened, we turn to information thcory instcad to learn invariant representation under varying non-rationale components. To achieve this, we introduce a relational detector generate the probability of each node carrying rationale information. We train the relational detector along with feature extractor, which produces rationale features
5

---

Published in Transactions on Machine Learning Research (09/2023)
Labeled Graphs
Graph Representation Learning
Rationale
Non-rationale
Graph-of-Graph Construction
Feature
Feature
Mutual Information Maximization
Rationale Detector
Contrastive Loss Leon
Masked Nodes
Unlabeled Graphs
Rationale Feature
Non-rationale
Disentanglement Loss Lais
Feature
Feature
Classfication Loss Lts
Extractor
+
Regularization Loss Lreg
Figure 2: Illustration of the proposed framework RIGNN. Our RIGNN utilizes relational detector and feature extractor to generate rationale features related to semantic labels and complementary non-rationale features. Morcover, we construct a graph-of-graph to extract the additional semantic information in the unlabeled set. The whole model is optimized using the combination of four objectives.
to not only maximize the mutual information with label information, but also disentangle with non-rationale components.
In detail, our relational detector is a message passing neural network Sa(.) with parameters 0, which first stacks graph convolution layers to generate hidden representations and then utilize a multi-layer perception (MLP) generate the probability that each node should be kept. Formally, a mask vector based on importance scores M E (0, x) is defined as:
M, (f
(4)
where (i: denotes the corresponding output of fe(-) for node i and we keep a nodes above a pre-defined threshold e. a is a predefined parameter to decide the proportion of the kept nodes. We add continuous values in the mask for efficient gradient updating. Then, the node attribute after masking would be X X o M which @ denotes the Hadamard product of two matrices. The removed information in the altribute matrix is X X c (1 M), which indicates non-rationale information. With the rationale attribute matrix and the non-rationale attribute matrix, nr using a feature extractor, which is another message passing neural network 9-(-) follows:
generate rationale fcature z" and non-rationale fcature matrix
x" = (s(
(5)
TM = (9:
(6)
To relieve the impact caused by the non-rationale part, aim to maximize the mutual information between zf and its label while minimizing the mutual information between z" and ne for disentanglement. In formulation, the objective is:
max [(>":y 2(=";2")) 0.0
(7)
However, label information is unavailable in D To tackle this, we consider two different. graph samples with an identical rationale part, i.e., 9 and g. From our previous analysis, their labels, i.e, and ğ should the same, and thus the mutual information between their rationale features should be maximized. In turn, maximizing the mutual information between rationale features with the same label would naturally result in invariant features (Bachman et al. 2019). In other words, when the labels of two graphs the same, they

---

Published in Transactions on Machine Learning Research (09/2023)
should share the same rationale components which are highly relevant to our downstream classification (by
maximizing y)). Therefore, we revise Equation 7 as follows:
max (z); Ey Y) =(="; 0.0
(8)
where 2f and Ey corresponds to the feature representation of G and G given the same label Y.
To maximize / (zt; z Y), we turn to graph rontrastive learning Yoou et al. 2020b 2021], which constructs rationale-informed positive pairs (ie., with the same label Y) from two sources. On the one hand, we consider sample pairs with the same labels as positives in D'. On the other hand, we take each original sample D" and its subgraphs (with a dropout ratio 0.2 You et al.) 2020b)) as positives since we do not have access to the label information. In formulation, define the positive as P ((i,j)ly yo yo 9j D'} and have the contrastive loss as:
Loon
PPL
E
log
E
e2ttz//T
Ginzyl
Duu
1
Σ GIED
log
EgyED /2
e
(9)
(L.))E
where T is a temperature parameter set to 0.5 following previous works YYou al. 2020b Ju al. 2022) and &; denotes the rationale feature for the subgraph of G. Here two positive sets together allow our model to elfectively leverage unlabeled data to learn more powerful embeddings. Finally, to minimize T(=", =²""
for sufficient disentanglement of rationale and non-rationale elements, we build a Jensen-Shamnon mutual information estimator T, (Sun et al. 2020ab, which is trained in an adversarial manner. In formulation, we have
Cais
Σ
sp( (x(- + DDP
Σ -pp( =p(-T,(2,,,2))) 9.9,ED
D
(10)
D:ED
where sp(x) = log(1 + ) is the softplus function. In summary, our model is optimized in a minimax game,
min max Loon Cdiss
(11)
0.0
To resolve Equation 18 we minimize two sub-objectives till the convergence follows:
ming,o Loon Latis
(12)
min, -Ldis-
From Equation on the one hand, we train the estimator for accurate measurement of mutual information. On the other hand, we update the network parameters to obtain discriminative rationale features satisfying Equation
4.4 Representation Enhancement via Graph-of-Graph
We have created rationale features which are highly related to semantic labels. Intuitively, the geometrically nearest. neighbors based on rationale features can be considered as semantic-similar graph pairs (Chen et al. 2022a). To make use of abundant unlabeled graphs, a graph-of -oofggraphh is constructed to connect indepen- dent graphs with similar semantics, providing extra semantic proximity to enhance rationale represcntation learning.
In detail, compare the rationale features of graph pairs and measure the similarity using the cosine distance:
Sú 2; * Zj.
(13)
Then, we identify k-nearest neighbors (kNNs) of labeled samples to add edges between graph samples where i denotes the number of neighbors. However, due to the label scarify of novel classes, kNNs could introduce false positives by connecting samples from novel classes to the other classes. To handle this, we filter false positives by identifying mutual nearest. neighbors (MNN) for unlabeled samples. In other words, connect

---

Published in Transactions on Machine Learning Research (09/2023)
9, and Gj when 2, (2)) 22 (z;( (i.e., 2, E MNN(2,)). Therefore, the adjacency matrix of the graph-of-graph is defined as:
{
1, 2 ENN(()), E' MMN(2 D
Au
(14)
0, otherwise
Afterward, we view connected graph pairs in the graph-of-grast as positives and add them into the positive set P. In formulation,
P - PU ((,j)A = 13.
(15)
Equation 15 enlarges the positive set, which enhances rationalc graph reprosentations with the additional guidance of semantic proximity under serious label scarcity.
4.5 Framework Summarization
Finally, we incorporate our rationale representations into open-world graph classification. To build mapping from rationale representations to label space, we add classifier ha(-) Rd R the top of ga(+) where the first |C'| scores are for the unseen classes, while the last |Cd| scores are for expected novel classes. Then we minimize the standard classification loss for labeled data and minimize the entropy for unlabeled data to generate informative distributions:
Leta = (E(h((CE((((())) y.)) + Bg H(hazi))))))
(16)
where CF(()) denote the standard cross-entropy loss and H() measures the entropy of the distribution. However, minimizing the entropy of predictions for unlabeled graphs could generate trivial solutions which assign the majority of novel samples into single class (Huang et al.) 2020). To tackle this, we introduce regularization term which minimizes the negative entropy of averaged distributions across the whole dataset:
Lreg log((C) - Hp), with p P2, .Picll
(17)
E (=(2))
where pic] Eee Ex,eo ha())]ee
denotes the summarized probability of belonging to class c in the whole datasot and log [C] can make the loss non-negative. In a mutshell, our final objective can be written as follows:
min max Leta + Laii + forg + XLeon
(18)
0.0
where À is a parameter to balance these losses. Similarly, adversarial learning is implemented using the gradient reverse layer (Zhang et al.) 2018) to optimize the whole framework as in Equation 12 In practice, we adopt mini-batch stochastic gradient descent to update the whole framework and update the graph-of- graph every cycle, and the total cycle number T. The detailed algorithm is shown in Algorithm [
Complexity. The computational complexity of our RIGNN mainly depends on the relational detector and the feature extractor. Given graph G with the number of nonzeros in the adjacency matrix denoted ||A||o. Recall that d denotes the feature dimension. L, and L, denotes the layer number of fe(-) and (-(,), respectively. [V] is the number of nodes. Obtaining rationale features and non-rationale fcatures takes O(L,, + Ly)][ + (L, + (((2) computational time. From the results, the complexity of the proposed RIGNN is linearly related to [V], || and Lp + Ly.
4.6 Extension to Open-set Graph Classification
Although RIGNN originally designed for semi-supervised open-world graph classification, can be ex- tended to open-set graph classification (Luo et al. 2023), which only needs to detect outliers in the unlabeled set. Here, we would adjust the classifier into (.) Rdd + RIC'l and detect outliers by selecting samples with small confidence scores, i.c., q = max; (g) Moreover, we will delete the regularization loss Creg since trivial solutions could not occur. Due to the existence of outliers, the classification loss is limited to
8

---

Published in Transactions on Machine Learning Research (09/2023)
labeled samples and unlabeled samples with high confidence. In formulation, we set threshold p and the set of outliers is (9e: qi S (r). The classification is modified into the following equation:
Eeta CE(hss))) v.)) + Eg u1 (hos
(19)
The final objective is modified into:
min max Eota + Cais + X
(20)
8,0
We will also utilize adversarial training for the disentanglement.
5 Experiments
In this section, we conduct various experiments on six datasets to validate the effectiveness of our RIGNN. The experimental results show the superiority of RIGNN in both open-world and open-set graph classification settings. Specifically, we will focus on the following research questions (RQs): (1) RQ1: What is the performance of our RIGNN compared to baselines in the open-world graph classification task? (2) RQ2: What is the prediction accuracy of RIGNN compared to baseline models in the open-sel graph classification task? (3) RQ3: What is the influence of rationale representation learning, contrastive learning and graph- of-graph reprosentation enhancement in the model's performance? (4) RQ4: Are there any visualization results of the rationale representation learning?
5.1 Experimental Setup
Datasets and Evaluation Protocols. We utilize four public benchmark graph datasets, i.e., COT.- DEL, Letter-high. high, MNIST, CIFAR1O, REDDIT and COLORS-3 in our experiments. Their statistics are presented in Table We divide all the classes into known classes and unknown classes with details recorded in Table 4 Tn both the open-world and open-set semi-supervised settings, partial labels are available for samples from known classes and we cannot get access to the labels of examples from the novel classes. We create two scenarios indicating different labeling ratios and denote them as Easy (a higher labeling ratio) and Hord (a lower labeling ratio), respectively. In particular, the ratio for Easy/ Hard problems is 0.8/0.5, 0.4/0.2, 0.03/0.01, 0.07 00.7/0.03, 0.7/0.3 and 0.8/0.3 for COIL-DEL, Letter-High, MNIST, CIFAR1O, REDDIT and COLORS-3, respectively (Luo et al., 2023). We report the classification accuracy to compare the performance. To be more precise, in the oprndworld setting, Hungarian algorithm (Kuhn 1955) is adopted to match these unknown classes and calculate the final prediction accuracy. In the open-set setting, we view all these novel classes as a unified class and when the model gives a correct label for samples from known classes or rejects samples from novel classes, we classify them correctly.
Baselines.
The proposed RIGNN is compared with a range of competing baselines, including graph neural
network methods (GraphSAGE (Hamilton et al.) 2017). GIN (Xu et al., 2019). GCN KKipf & Welling,
2017), ASAP (Ranjan et al. 2020 Edge Pooling (Dihhl 2019), TopK Pooling (Goo & Ji 2019a) and SAG
Pooling (Lee et al.] 2019a)) and graph contrastive learning methods (GraphCT, (You et al. 2020b). GLA (Yue)
et al. 2022a) and UGNN (Luo et al. 2023
Implementation Details. We implement the proposed RIGNN with PyTorch and train all the models with an NVIDIA RTX GPU. As for hyperparameters, we set i in the graph-of-graph construction process to 2. For the weight in the loss function, we set Section C The dimension of all hidden features is set to 128. As for the network architecture, two-layer GraphSAGE (Hamilton et al. 2017) to construct the relational detector fo and a three-layer GIN convolution for the feature extractor go- In the middle of the convolutional layer, we implement graph pooling with TopK Pooling (Gao & Jil 2019b) as default. For the Jensen-Shannon mutual information estimator T,, we concatenate the two inputs and send the feature to a two-layer MLP. A two-layer MLP is also adopted from the classifier hgg For the model training, we train the model for 100 epochs in total and utilize the entire dataset for estimating the mutual information. In terms of optimization, we employ the gradient reversal layer Ganin & Lempitsky 2015) to realize the adversarial training of 7. This approach provides
0.1. Their detailed analysis can be found in use
9

---

Published in Transactions on Machine Learning Research (09/2023)
Table 1: Open-worle classification accuracy in COIL-DEL, Legh,r-high, MNIST and CIFAR10 datasets.
Both Easy and Hard scenarios are included, and the proposed RIGNN achieves the best performance.
COIL-DEL
Letter-Hight
MNIST
CIFAR10
REDDIT
COLORS-3
Methods
Average
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
GraphSAGE
35.64
41.53
52.67
55.11
19.31
40.91
29.88
31.90
28.96
27.49
25.14
25.38
34.39
GIN
50.38
56.15
46,00
48.67
41.55
53.17
33.92
35.23
26.15
27.24
22.57
21.52
38.55
ASAP
45.00
37.56
30.00
47.11
34.40
55.40
34.90
35.93
27.03
27.16
33.33
35.00
38.57
Edge Pooling
45.90
50.00
38.22
49.78
22.02
46.58
27.58
32.23
27.03
26.57
31.90
31.29
35.76
TopK Pooling
35.64
37.44
35.33
48.89
21.03
37.33
32.65
31.43
29.09
29.00
30.67
31.62
33.34
SAG Pooling
41.28
46.54
43.78
48.89
28.90
54.00
28.94
29.81
23.85
28.46
29.71
28.19
36.03
GraphCL
48.33
53.97
44.89
48.44
35.11
56.48
33.81
35.78
26.99
27.91
34.62
34.76
40.09
GLA
51.92
35.26
44.00
48.22
42.59
55.90
33.06
35.40
29.00
29.72
35.14
34.95
41.26
RIGNN (Ours)
52.69
61.03
46.67
50.44
52.92
61.06
36.37
39.57
29.34
29.46
35.86
35.95
44.28
Improvement 1.4%
8.7%
1.5%
1.3%
24.3%
8.1%
7.2%
7.7%
1.2%
-0.8%
2.0%
2.7%
7.3%
mechanism to avoid alternative optimization, and ensure the overall training stability. The parameters 0 and & are viewed as an integrated unit during the training process. This design choice effectively mitigates the discrepancies in the update frequencies among different components. Morcover, our training process is divided into two phases. We initially warm up the model with labeled data only, ensuring that the paramoters reach a stable state before the introduction of complex interactions. Following this, the model trained jointly with all the available data. We use the Adam optimizer for its well-known efficiency and effectiveness. In the training, we use Adam (Kingmaa & Ba 2015) optimizer and set the batch size to 256, with the learning rale set to 0.001.
5.2
The Performance of RIGNN in Open-world Graph Classification (RQ1)
The open-world classification accuracy on the datasets COIL-DEL, Letter-Higg High, MNIST, CIFARIO, REDDIT and COLORS-3 compared to the baseline methods is listed in Table 1 From the results, we obtain the following observations:
Firstly, the proposed RIGNN obtains a consistent lead in both Hard and Easy scenarios on all four datasets, which demonstrates the superiority of the model. In particular, we attribute the performance gain to two aspects: better representation learning with rationale and the representation enhancement according to the constructed graph-of-graph proximity. Learning with rationale helps the model detect the most essential part of the graph and get rid of the non-rationale part, which contributes to the generalization capability of the model to unknown classes. The constructed graph-of-graph and the corresponding contrastive learning improve the model capability to detect semantic proximity among unlabeled instances and to make the best of unlabeled instances. With the enhancement brought by the hooph-of-grahh, the model is better at classifying graph instances in unknown classes.
In addition, we observe that our model achieves more significant improvement on the MNIST and CI- FAR10 datasets, which contains more nodes and edges in a graph, compared to the Letter-high dataset, which contains fewer nodes average. One possible explanation for this is that cach node in small graph plays a more important role in the class-determining process than nodes in a large graph. Large graphs like those in MNIST and CIFAR10 tend to contain more non-rationale parts, for example, the nodes representing the background in MNIST and CIFAR10. Therefore, the proposed rationale representation learning contributes less to the classification of small graphs.
Moreover, we find that existing semi- supervipe graph classification methods fail to provide satisfactory accuracy in the open-world classification task, since they are designed for the closod-world graph classi- fication. In comparison, the proposed RIGNN leverages rationale discover the most essential part in the graph related to the label space and adopts the graph-of-graph construction to make better use of
10

---

Published in Transactions on Machine Learning Research (09/2023)
Table 2: Open-set classification accuracy in COLL-DEL, Letter-high, MNIST and CIFARIO datasets. Both Easy and Hard scenarios are included, and the proposed RIGNN achieves the best performance.
COIL-DEL
Letter- High
MNIST
CIFARIO
REDDIT
COLORS-3
Methods
Average
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
GCN
22.56
33.46
32.89
50.44
20.00
37.03
30.89
36.45
31.23
33.42
31.87
38.32
33.21
GraphSAGE
37.69
39.74
40.89
58.22
19.56
41.97
32.94
35.14
24.43
26.19
31.31
34.00
35.17
GIN
41.15
44.49
48.89
57.11
29.72
62.55
27.18
33.32
31.71
35.83
31.09
33.53
39.96
SAG Pooling
41.03
18.46
16.22
52.22
12.49
63.22
36.41
38.83
32.81
31.74
39.07
40.21
42.98
GraphCL
56.28
60.64
56.22
63.56
49.81
69.97
37.27
40.98
32.71
36.02
38.17
39.94
48.46
GLA
56.54
61.03
60.22
63.11
48.30
70.45
38.31
41.18
31.12
36.80
39.03
10.89
19.17
UGNN
59.36
62.95
64.00
66.00
58.50
73.01
39.73
42.03
37.73
38.30
41.17
43.12
52.21
RIGNN (Our)
60.13
66.41
63.56
65.56
69.70
78.23
43.73
47.10
36.38
37.74
50.14
45.05
55.31
Improvement 1.3%
3.5%
-0.7%
-0.7%
19.1%
7.1%
10.1%
12.1%
-3.6%
-1.5%
20.9%
3.8%
5.9%
unlabeled graphs, which could belong to unknown classes. RIGNN has good generalization ability, and in the following, we would see that the model does well in open-set graph classification.
5.3 The Performance of RIGNN in Open-set Graph Classification (RQ2)
The performance of our RIGNN on the COLL-DEL, High MNIST CIFARIO,, REDDIT and COLORS- 3 in comparison with several baseline methods is listed in Table 2 the extended model generalizes well into the open-set graph classification task and outperforms all the listed baselines in both Hard and Easy scenarios on all four datasets. Similar to the open-world classification setting, the model gains relative improvement of about 6.4% on averago. The high performance in the open-set graph classification task shows that the extended model RIGNN is also good at detecting out-of- distribution instances, i.e., instances in unknown classes.
According to the results, we can see that
Furthermore, can observe that although semi-supervised graph classification methods generally outper- form the other GNN-based baselines, the extended RIGNN gains more improvement. This suggests that existing semi-supervised graph classification methods (e.g. GraphCL (You et al. 2020a) and GLA (Yue) let al. 2022a are able to detect out-of-distribution instances more effectively than vanilla GNNs, they do so with lower accuracy than our proposed RIGNN and more importantly, they are weak in clustering the instances in the unknown classes into reasonable clusters, as can be seen from their performance in the open-world classification task.
5.4 Ablation Study (RQ3)
In this part, extensive ablated studies are conducted on the COIL-DEL and MNIST datasets to demonstrate the effectiveness of the proposed RIGNN. Concretely, we perform the experiments in the open-set graph classification setting and remove some of the proposed modules/ modules/mechanissm mechaniechsss to test the prediction accuracy.
The three variants of RIGNN include: (1) RIGNN w/o CRL, which removes the rationale representation learning and utilizes a single message passing neural network to generate graph reprosentations; (2) RIGNN w/o D, which removes the disentanglement between rationale features and non-rationale features; (3) RIGNN w/o G, which removes the enhancement from the graph-of-graph.
(4) RIGNN w CLUE, which employs a different popular estimator CLUE [2). CLUB firstly models the distribution e (2 [2,,, and then calculates the mutual information using 1(Z", 2 Eg,eD [log ((2,"" xx)] - Eg,,9,ED [log q(2)"" xz]]
The results are summarized in Table 3 From the results, we have the following obscrvations: (1) It is evident that removing each component causes the performance to drop in all cases, which demonstrates the con- tribution of rationale representation learning, disentanglement between rationale features and non-rationale features, and graph-of-graph. representation enhancement. (2) The model experiences more significant perfor- mance drops when the rationale representation learning module removed (e.g. 7.18% absolute percentage drop in COIL-DEL Hard task and 10.60% absolute percentage drop in MNIST Hard task). This suggests
11

---

Published in Transactions on Machine Learning Research (09/2023)
Table 3: Ablation studies of our proposed RIGNN on the COIL-DEL and MNIST datasets. RIGNN w/o CRL removes the causal representation learning and utilizes single message passing neural network generate graph representations: RIGNN w/o D removes the disentanglement between causal features and non-causal features; RIGNN w/o G removes the enhancement from the graph-of-graph.
COIL-DDE
MNIST
Experiment
Hard
Easy
Hard
Easy
RIGNN w/o CRL
52.95
59.74
59.10
65.32
RIGNN w/o D
56.15 15
63.21
65.04
75.92
RIGNN w/o G
57.44
62.56
63.39
74.86
RIGNN w CLUE
57.38
64.97
68.59
77.71
RIGNN
60.13
66.41
69.70
78.23
3
4
3
Figure 3: Visualization of learned important scores generated by the relational detector (Top line: orig- inal images; middle line: superpixel-based graphs; Bottom line: important scores). The experiments are performed on dataset. MNIST, and darker nodes are relatively more important. The results show that the rationale detector in the proposed RIGNN is able to make a reasonable estimation of node importance.
that detecting rationale subgraphs in the original graph and removing non-rationale components is important for the performance in the face of unknown classes. (3) The use of contrastive learning in both rationale and graph-of-graph proximity contexts is helpful for the classification, since it can learn robust representations for the rationale part of the graph. This in alignment with the results in the table: removing either rationale contrastive learning or the graph-of-graph construction hurts the prediction accuracy. (4) We can observe that the performance of RIGNN CLUE remains relatively comparable our original model. This suggests that the choice of MI estimator does not significantly impact the performance of our model. Nevertheless, we agree that it is important to select the appropriate MI estimator based on the characteristics of the data and the specific application. Our approach allows for flexibility in choosing the MI estimator based on validation data when dealing with new datasets.
5.5 Visualization (RQ4)
In addition, we offer some visualization results to show the effectiveness of the rationale representation learn- ing in RIGNN. Concretely, we conduct experiments on the MNIST dataset and visualize the rationale-based important scores generated by the relational detector. We show the results in Figure g As can be observed from the results, the proposed rationale importance estimation in representation learning yields reasonable important scores related to their underlying patterns, which validates that our exploration of rationale factors can obtain meaningful subgraphs and thus learn effective graph representations. Moreover, the important
12

---

Published in Transactions on Machine Learning Research (09/2023)
nodes tend to come together and therefore the selected rationale subgraph tends to be connected. In contrast, the nodes in the boundary tend to not be selected.
6 Discussion
In this part, we provide a discussion between our rationale-informed framework and causality- graph representation methods. Causality has been widely utilized to learn graph-level representations in out-of- distribution generalization (Chen et al., 2022b Sui et al.} 2022), which identifies the causal mechanisms graph generation and enhances the representation learning using the intervention. In our setting, utilize mutual information maximization to discover the rationale elements in graphs without the intervention procedure, which is more close to rationale discovery instead of cansality learning. However, we believe that causality learning is still valuable in graph representation lcarning with the invariance principle, and would explore the strength in the complex scenarios with out-of-distribution shift.
7
Conclusion
This paper studies the problem of semi-supervised open-world graph classification and a novel method named RIGNN is proposed to solve the problem, which detects features that hold the most information about the label space. Our RIGNN contains a relational detector and feature extractor to provide rationale features. To capture rationale components,
maximize their mutual information with label information and require sufficient disentanglement with non-rationale components. In addition, we build graph-of-graph based on geometrical relationships that provide guidance on improving rationale representations. We also make an extension for effective open-set graph classification. Interostingly, our method contains both mutual information maximization and minimization and we will investigate more about the relationship between them for enhanced performance. Comprehensive experiments on four popular datasets evaluate the efficacy of our proposed RIGNN. We will also test model on wild datasets for more comprehensive evaluation of its robustness and versatility.
Funding Information
This work was partially supported by NSF 2211557, NSF 1937599, NSF 2119643, NSF 2303037, NSF 2312501, NASA, SRC, Okawa Foundation Grant, Amazon Research Awards, Cisco research grant, Picsart Gifts, and Snapchat Gifts.
References
Philip Bachman, Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views. In Proceedings of the Conference on Newral Information Processing Systems, volume 32, 2019.
Karsten M Borgwardt and Hans-Peter Kriegel. Shortest-pas path kernels on graphs. In Proceedings of the IEEE International Conference on Data Mining, 2005.
Kaidi Cao, Maria Brbic, and Jure Leskovec. Open semi-supervised learning. Proceediiggs of the International Conference on Learning Representations, 2022.
Aaron Chan, Maziar Sanjabi, Lambert Mathias, Liang Tan, Shaoliang Nie, Xiaochang Peng, Xiang Ren, and Hamed Firooz. Unirex: unified learning framework for language model rationale extraction. In ICML, pp. 0867-2889) 2022.
Liang Chen, Qianjin Du, Yihang Lou, Jianzhong He, Tao Bai, and Minghua Deng. Mutual nearest neighbor contrast and hybrid prototype self-training for universal domain adaptation. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 6248 6257, 2022a.
13

---

Published in Transactions on Machine Learning Research (09/2023)
Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, MA KAILI, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Learning causally invariant representations for out-of-distribution generalization on graphs. In Proceedings of the Conference on Newral Information Processing Systems, 2022b.
Frederik Diehl. Edge contraction pooling for graph neural networks. ar preprint orv: 10990, 2019.
Luke Ditria, Benjamin J Meyer, and Tom Drummond. Opengan: Open set generative adversarial networks. In Proceedings of the Asian Conference on Computer Vision, 2020.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Bench- marking graph neural networks. arXiv preprint S1::2003300988 2020.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the International Conference Machine Learning, pp. 1180-1189, 2015.
Hongyang Gao and Shuiwang Ji. Graph u-ncts. In Proceedings of the International Conference on Machine Learning, pp. 2083 2092, 2019a.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In Proceedings of the International Conference on Machine 2019b.
Learning, pp.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the International Conference Machine Learning, 2017.
Gaoyang Guo, Chaokun Wang, Bencheng Yan, Yunkai Lou, Hao Feng, Junchao Zhu, Jun Chen, Fei He, and Philip Yu. Learning adaptive node embeddings across graphs. IEEE Transactions on Knowledge and Dala Engineering, 2022.
William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proceedings of the Conference on Neural Information Processing Systems, 2017.
Katja Hansen, Franziska Biegler, Raghunathan Ramakrishnan, Wiktor Pronobis, O Anatole Von Lilienfeld, Klaus-Robert Müller, and Alexandre Tkatchenko. Machine learning predictions of molecular properties: Accurate many-body potentials and nonlocality in chemical space. Journal of Physical Chemistry Letters, 6(12):2326 2331, 2015.
Zhongkai Hao, Chengqiang Lu, Zhenya Huang, Hao Wang, Zheyuan Hu, Qi Liu, Enhong Chen, and Cheekong Lee. Asgn: An active ceni-supervise graph neural network for molecular property prediction. In Pro- ceedings of the International ACM SIGKDD Conference on Knowledge Discovery 8 Data Mining, 2020.
Jiabo Huang, Shaogang Gong, and Xiatian Zhu. Deep semantic clustering by partition confidence maximi- sation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
8849-8858 2020.
Licheng Jiao, Jie Chen, Fang Liu, Shuyuan Yang, Chao You, Xu Liu, Lingling Li, and Biao Hou. Graph representation learning meets computer vision: A survey, IEEE Transactions Artificial Intelligence, 2022.
Wei Ju, Junwei Yang, Meng Qu, Weiping Song, Jianhao Shen, and Ming Zhang. Kgnn: Harnessing kernel- based networks for semi-supervised graph classification. 2022.
Diederik P Kingma and Jimmy Ba. Adam: A mothod for stochastic optimization. Proceedings of the International Conference on Learning Representations, 2015.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations, 2017.
Shu Kong and Deva Ramanan. Opengan: Open-set recognition via open data generation. In Proceedings of the TEEE/CVF International Conference on Computer Vision, pp. 813 822, 2021.
14

---

Published in Transactions on Machine Learning Research (09/2023)
Harold W Kuhn. The hungarian method for the assignment problem. Naval rescarch logistics quarterly, 2 (1-2):83-95) 1008,99970177 1955.
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In International conference on machine learning, pp. 3743. PMLR, 2019a.
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In Proceedings of the International Conference on Machine Learning, 2019b.
Jia Li, Yu Rong, Hong Cheng, Helen Meng, Wenbing Huang, and Junzhou Huang. Semi-supervised graph classification: A hicrarchical graph perspective. In Proceedings of the Web Conference, 2019.
Jia Li, Yongfeng Huang, Heng Chang, and Yu Rong. Semi-supervised hierarchical graph classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022a.
Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan Hc, and Tat-Seng Chua. Let invariant rationale discovery inspire graph contrastive learning. In Proceedings of the International Conference on Machine Learning, pp. 13052 13065, 2022b.
Yicong Li, Xiang Wang, Junbin Xiao, and Tat-Seng Chua. Equivariant and invariant grounding for video question answering. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 4714- 4722, 2022c.
Zemin Liu, Yuan Fang, Chenghao Liu, and Steven CH Hoi. Relative and absolute location embedding for few-shot node classification on graph. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. -275, 2021.
Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang, Peize Lin, and Lixin He. Molecular property prediction: A multilevel quantum interactions modeling perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
Xiao Luo, Yusheng Zhao, Yifang Qin, Wei Ju, and Ming Zhang. Towards semi-supervised universal graph classification. IEEE Transactions on Knowledge and Data Engineering, pp. 1-13, 2023.
Mamshad Nayeem Rizve, Navid Kardan, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Openldn: Learning to discover novel classes for spen-world semi-supervised learning. In Proceedings of the Conference on Neural Information Processing Systems, 2022.
Poojan Oza and Vishal M Patel. C2ae: Class conditioned auto-encoder for open-set recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2307-2316, 2019.
Pau Panareda Busto and Juergen Gall. Open set domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 754-763, 2017.
Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar. Asap: Adaptive structure aware pooling for learning hierarchical graph representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5470-5477 2020.
Kaspar Riesen and Horst Bunke. Iam graph database repository for graph based pattern recognition and machine learning. In Joint TAPR International Workshops on Statistical Techniques Pattern Recognation (SPR) and Structural and Syntactic Pattern Recognition (SSPR), pp. 287 297. Springer, 2008.
Mamshad Nayeem Rizve, Navid Kardan, and Mubarak Shah. Towards realistic semi-supervised learning. Proceedings of the European Conference on Computer Vision, pp. 437-455, 2022.
Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E Boult. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 3777):1757-1772, 2012.
Walter J Scheirer, Lalit Jain, and Terrance Boult. Probability models for open set recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11):2317 22244, 2014.
15

---

Published in Transactions on Machine Learning Research (09/2023)
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sanceda Felix, Stefan Chmicla, Alexandre Tkatchenko, and Klaus-Robbrt Miller. Schnet: A contimuous-filter convolutional neural network for modeling quantum interactions. In Proceedings of the Conference on Neural Information Processing Systems, 2017.
Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics, pp. 488 495. PMLR, 2009.
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisseilee-lehman graph kernels. Journal of Machine Learning Research, 12(9), 2011.
Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. Causal attention for interpretable and generalizable graph classification. In Proceedings of the International ACM SIGKDD Conference Knowledge Discovery & Data Mining, pp. 1696-1705, 2022.
Fan-Yun Sun, Jordan Hoffmmnnn Vikas Verma, and Jian Tang, Infograph: Unsupervised and semi-superrviedd graph-level reprosentation learning via mutual information maximization. In Proceedings of the Interna- lional Conference on Learning Representations, 2020a.
Xin Sun, Zhenning Yang, Chi Zhang, Keck-Voon Ling, and Guohao Peng. Conditional gaussian distribution learning for open set recognition. In Proceedingss of the IEEE/CVE Conference on Compuler Vision and Pattern Recognition, pp. 13480-13489, 2020b.
Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, Min Lin, and Tat-Seng Chua. Causal representation learning for
recommendation. In WWW, pp. 3562 3571, 2022.
Jun Wu, Jingrui He, and Jiejun Xu. Net: Degree-specific graph neural networks for node and graph classi- fication. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 406 415, 2019.
Tian Xia and Wei-Shinn Ku. Geometric graph representation learning on protein structure prediction. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Dala Mining, pp. 1873-1887) 2021.
Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Instance correction for learning with open-set noisy labels. arv preprint Xrx:2:2106. 00455, 2021.
Yu Xie, Yanfeng Liang, Maoguo Gong, AK Qin, Yew-Soon Ong, and Tiantian He. Semisupervised graph neural networks for graph classification. IEEE Transactions on Cybernetics, 2022.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka, How powerful are graph neural networks? In Proceedings of the International Conference on Learning Representations, 2019.
Haoran Yang, Hongxu Chen, Shirui Pan, Lin Li, Philip S Yu, and Guandong Xu. Dual space graph contrastive learning. In Proceedings of the ACM Web Conference 2022, pp. 1238 1247, 2022a.
Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia, and Junchi Yan. Learning substructure invariance for out-of-distribution molecular representations. In Proceedings of the Conference on Neural Information Processing Systems, 2022b.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2018a.
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovee. Hierarchical graph representation learning with differentiable pooling. In Proceedings of the Conference on Neural Information Processing Systems, 2018b.
Yuming You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812- 5823, 2020a.
16

---

Published in Transactions on Machine Learning Research (09/2023)
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In Proceedings of the Conference on Neural Information. Processing Systems. 2020b.
Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang- Graph contrastive learning automated. Tn Proceedings of the International Conference Machine Learning, 2021.
Han Yue, Chunhui Zhang, Chuxu Zhang, and Hongfu Liu. Label-invariant augmentation for semi-supervised graph classification. arXiv preprint orXiv:2205.0055 098002, 2022a.
Linan Yue, Qi Liu, Yichao Du, Yanqing An, Li Wang, and Enhong Chen. Dare: Disentaanglemeen--aggmeeeeddd ralionale extraction. In NewrIPS, pp. 26603 26617, 2022b.
Shengyu Zhang, Xusheng Feng, Wenyan Fan, Wenjing Fang, Fuli Feng, Wei Ji, Shno Li, Li Wang, Shanshan Zhao, Zhou Zhao, et al. Video-audio domain generalization via confounder disentanglement. In AAAI, volume 37, pp. 15322-153322 2023.
Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. Collaborative and adversarial network for unsuper- vised domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognilion, pp. 3801 3809, 2018.
Tianxiang Zhao, Xiang Zhang, and Suhang Wang. Graphsmote: Imbalanced node classification on graphs with graph neural networks. pp. 833 841, 2021.
Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Learning placeholders for open-sel recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1401-4410, 2021.
Aoqi Zuo, Susan Wei, Tongliang Liu, Bo Han, Kun Zhang, and Mingming Gong. Counterfactual fairness with partially known causal graph. In NewrIPS, pp. 1238-1252, 2022.
17

---

Published in Transactions on Machine Learning Research (09/2023)
A
Algorithm
The algorithm of our RIGNN is summarized as below.
Algorithm 1: Training Algorithm RIGNN
Require: Training set D= D' U paramoter A;
Ensure: The prediction for all unlabeled graphs; Initialize parameters Φ and 7
for t = 1, T do
3: Calculate the similarity and calculate A using Equation 14
4: Update the positive set using Equation 15
5: repeat
Generate a batch by sampling graph examples from D' and D"; Produce rationale features and non-rationale features using Equations and Compute the overall loss with Equation [8] Update the parameters in the nelwork through back propagation; 10: until convergence
6:
11: end for
B Details of Baselines
Their details of the compared baselines are introduced as follows:
Teisneil (WL.) Kernel (Shervashidze el al. 2011 which adopts the Weisseller-LLehman algo- rithm construct mapping from the original graph to graph sequence.
Shortest-Path (SP) Kernel [Borgwardt & Kriegel 2005 which attempts to decompose graphs into various shortest paths for comparison.
Graphlet Kernel (Shervashidze et al. 2009 which calculates the number of graphlets the input graphs generate features.
GCN (Kipf & Welling 2017 which is the pioneer graph neural network method. It to adopt the nor- malized adjacent matrix for message passing.
GraphSAGE (Hamilton al. 2017). which introduces sampling into efficient message propagation.
GIN Xu al. 2019 which relates the power of message passing neural networks to the Weisfeiler- Lehman test.
SAG Pooling Lee al. 2019a), which utilizes the attention mechanism generate hierarchical sub- graphs, which can generate elfective graph representations for downstream tasks.
GraphCL You et al. 2020b which introduces four graph augmentation strategics to compare different views, and can be exlended to a semi-supervised graph classification method.
GLC Yuue et al. 2022a). which utilizes label-invariant augmentation to enhance graph classification and tests the performance for semi-supervised graph classification.
C Hyperparameter Analysis
In this part, we study the parameter sensitivity our proposed RIGNN. More specifically, we conduct experiments on the COIL-DEL and MNIST datasets for open-set graph classification. The results are shown in Figure 4 The first column shows the performance of the model & changes, while the second column
18

---

Published in Transactions on Machine Learning Research (09/2023)
67.50
Hard
Easy
67.50
Hard
Easy
Accuracy
65.00
Accuracy
65.00
62,50
62.50
60.00
60.00
57.50
57.50
0.05
0.1
0.3
k
80.00
Eay
80.00
Eay
Accuracy
Accuracy
75.00
75.00
70.00
70.00
65.00
65.00
3
0.05
0.1
0.2
0.3
0.4
k
The parameters sensitivity analysis of our RIGNN for the open-set graph classification task, and we provide the result on both Easy and Hard scenarios. The top row shows the experiments on the COIL-DEL dataset, while the bottom row presents the experiments on the MNIST dataset.
Figure 4:
shows the prediction accuracy À changes. The upper part of the figure presents the results the COIL- DEL dataset and the lower part shows the results the MNIST dataset.
As can be seen from the results, the model is generally not sensitive to these hyperparameters and perturbing them in a specific range has limited influence on the classification accuracy. For hyperparameter k; we find that the approach obtains the best performance when set to 2. Decreasing will result in a relatively sparse graph and fewer anchors for the representation enhancement via graph-of-graph, whereas increasing k will add noise to the contrastive objective, hurting the performance. Similarly, we find that RIGNN has the best performance when À is set to 0.1, which provides the appropriate weight for the contrastive learning loss.
D Details of Datasets
The details of the adopted datasets are introduced as follows:
COII.-EI. The COIL-DEL dataset (Riesen Bunke 2008) is created by Harris corner detection well Delaunay Triangulation on image data. Then, graph is constructed with nodes representing ending points and edges representing lines.
Lelter-high. The Letter-high datasel [Riesen & Bunke 2008) is made of graphs indicating fifteen capital letters, ie., A, E, F, H, I, K, L, M, N, T, V, W, X, Y, Z. In each graph sample, node denotes an endpoint, and edges denote lines. Highly distorted letters indicate the high difficulty in identifying them.
MNIST. The MNIST dataset Dwivedi et al. 2020 is adapted from a vision dataset with the same name, where we extract super-pixels images to construct nodes and a kNN graph is utilized to characterize the relationships between super-pixels.
CIFAR10. The CIFAR10 dataset (Dwivedi et al. 2020) is also a vision datasct with similar construction manner. Moreover, CIFAR10 is more challenging since it made up of larger graphs with complicated semantic information.
19

---

Published in Transactions on Machine Learning Research (09/2023)
Table 4: Statistics of the datasets used in the experiments.
Dataset
# Graphs
# Classes
# Known
# Unknown
COIL-DEL
3900
100
80
20
Letter-High
2250
15
10
MNIST
55,000
10
7
3
CIFAR10
45,000
10
7
REDDIT 11929
11
COLORS-3 10500
11
E More Visualization
We present the visualization result of the classification of the MNIST dataset. We compare our prediction with the prediction of GIN, and the result is listed in Figure 5 As can be seen from the results, the proposed RIGNN achieves better performance when there unlabeled tittof-distributiin data. The baseline model classifies the OOD instances into known classes, while our method detects the instances as out-of-distribution.
Ground Truth: 7
Ground Truth: 9 Baseline Prediction I (Wrong) Our Prediction: OOD (Right)
Baseline Prediction (Wrong) Our Prediction: OOD (Right)
Figure 5: Visualization of two graph examples from the MNIST dalaset. Our RIGNN can make the correct prediction while the baseline GIN cannot detect these out-of-distribution samples.
20


Reviews:
[{"review_id": "ehHX14DQVx", "reviewer": "Reviewer_ecKj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nThe paper is overall well-written and organized, making it easy to follow. I did not find significant grammatical errors or clarity concerns. The proposed loss is well-motivated. From the empirical perspective, the authors have shown the proposed method has better performance on open-set classification and OOD in various datasets. Some visualization results of the causal representation are also provided in the paper.\n\n## Weaknesses\n\nMost of my concerns fall in the discussion in terms of the mutual information. The authors propose to tackle the non-causal impact with the maximization of mutual information. However, several key perspectives regarding mutual information estimation are not complete:\n\nThe loss of disentanglement $\\mathcal L_{dis}$ is dependent to a mutual information estimator $T_\\gamma$ that is trained in an adversarial manner. The training stability might be a concern of the proposed CIGNN, as the scale of all components in Eq (18) is different. The authors may need to provide more details like papers training GANs in addressing the stability in training. For example, do we need separate optimizers, do $\\theta$, $\\phi$ and $\\gamma$ take different update frequencies, etc. \n\nThe mutual information estimators are usually upper-bound or lower-bound estimators. Some of them are unbiased but have high variance, while some are biased but have lower variance. Different estimators have different properties and request different sample selections to get a good approximation. How the quality of the estimator affects CIGNN is not theoretically analyzed nor empirically studied. \n\nThe mutual information estimators are sometimes sensitive to the number of samples used in the estimation. The authors may also need to conduct corresponding studies to address this point. \n\nA complementary point regarding the mutual information estimators is the relation between $\\mathcal L_{dis}$ and $\\mathcal L_{con}$, since JS-bound estimator and contrastive prediction estimator are two different estimators. It would be interesting if the authors could explore deeper on the relation between this two components. (Please note we do not request corresponding changes regarding this point.) \n\n\nAnother of my concern is the comparison with the baselines in Table 3. Several of the baselines also report the results on REDDIT-MULTI-12K and COLORS-3 datasets. Are there any specific reasons why the authors eliminate these two datasets?\n\nAs the paper tackles the open-set  problems, it could be more convincing if the authors could provide experiment results on some real-world datasets. E.g., images in the wild data. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper tackles the open-world semi-supervised graph classification where unlabeled graphs could come from unseen classes. The authors propose CIGNN, which takes a causal perspective to detect components containing the most information related to the label space and classify unlabeled graphs into known or unseen classes. The approach includes a relational detector and a feature extractor to produce causal features. In addition, a graph-of-graph is constructed to enhance causal representations. The paper also mentions the extension of the approach to open-set graph classification. Experimental results show the effectiveness of CIGNN compared to baseline methods such as GCN, UGNN. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please add the details of the training to address the training stability questions, and add potential discussions regarding the effects of $\\gamma$. \n\n2. Please add results on at least REDDIT-MULTI-12K and COLORS-3 datasets if possible. ", "broader_impact_concerns": "There are no significant broader impact concerns for this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FZzfPTrVPz", "reviewer": "Reviewer_HeaJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n\n**1.** CIGNN has the ability to effectively extract causal features (w.r.t. semantic labels) from graphs while reducing the impact of non-causal components. With its use of a relational detector and feature extractor, the framework can produce effective causal features, and the authors also propose a construction of a graph-of-graph based on geometrical relationships to enhance and visualize causal representations. \n\n**2.** Additionally, different from traditional semi-supervised learning, CIGNN can provide accurate and balanced predictions for unlabeled graphs that are even outside the predefined label class, even in the presence of label scarcity.\n\n**3.** The paper is well-structured and experimental results reveal the effectiveness of CIGNN compared with state-of-the-art methods on various benchmark datasets in various settings that may have potential implications for the causal and graph machine learning.\n\n## Weaknesses:\n\n### Major Comments:\n\nHowever, the reviewer may have the following concern regarding this paper:\n\n**1.** Regarding the SCM in Section 4.2: in literature such as Pearl (2009) and Peters et al. (2017), SCM is a framework used in causal inference and modeling to understand the relationships between variables and their causal dependencies. For example, given a DAG, we would recognize that the variable X causes variable Y if X pointed out to Y. But in figure 2, the author does not really explain the definition of the causality and non-causality when first introducing it, e.g., what does causal part feature mean in the context, with respect to which variables? Please make the motivation clearer so that readers would be convinced that the method is well-based on causality. If you are referring to causality as w.r.t. the semantic label Y at a later stage, should we exclude those features as theoretically they are non-causal instead of reducing the impact of NC? Otherwise, I would only recognize that Figure 2 as a high-level idea of finding useful (or more impactful) features for predictions.\n\n**2.** Many notations are not properly defined and used, which makes the paper less readable, please refer to minor coments for more details. Further, going through the whole paper, I feel that using the mutual information does not go beyond statistics. Say, maximizing the mutual information does not necessarily mean doing causal discovery or finding causal features. I would say this is a process of finding label-relevant/irrelevant representations from graph and it basically does not leverage rigorous causal definitions or methods such as finding the causal relationship between two nodes, or investigating the graph structure with ideas of interventions.\n\n**3.** While I like figure 4 that the author can extract and visualize the causal features but it is not necessarily helpful in its current form as readers could not see useful information from its current graph presentation. Instead, I would suggest enhancing the visual representation by highlighting the key pixels (or nodes) in real images based on these causal features. This approach will provide readers with a better understanding of the performance of CIGNN during predictions.\n\n### Minor comments:\n\n**1**. In (1), what is $k$ in your notation? Do you mean $l-1$? In (2), does $L$ necessarily mean the total layer number? Please specify these notations clearly.\n\n**2**. What does $\\mathbb{E}$ mean in (3)? Does it mean \"empirical average\" over dataset? While typically it means the expectation over a random variable. \n\n**3**. In (4.1), typo in \"…, we propose a novel method named causal-attended graph neural network (CIGNN)…\", should it be causal-informed as the abstract tells? While in the later context, authors are all using \"causal-attended\" instead of \"causal-informed\". Please make them consistent.\n\n**4**. “We first present a structure causal model (SCM)” -> structural causal model.\n\n**5**. In (4), what does $f_\\theta(i;\\mathcal{G})$ indicate without appropriately defining it? Do you mean the ith vertex of $f_\\theta(\\mathcal{G})$? Further, what is $\\alpha$ in “..retain $\\alpha|\\mathcal{V}|$ nodes.”?\n\n**6**. Define or mention Hadamard product when using $\\mathbf{X} \\odot \\mathbf{M}$. \n\n**7**. Throughout the whole paper, the mutual information between $X$ and $Y$ should be written as $I(X;Y)$, instead of $I(X,Y)$ with the comma.\n\n**8**. Eq (8) is not mathematically right – please refer to the definition of conditional mutual information such as Cover & Thomas (2006). I could understand (7) if the authors are referring a certain graph as a random variable (and its label $y$ as well), and maximization of mutual information between $z^c$ and $y$ means the learned random feature $z^c$ contains high information of the random label $y$. But when it comes to conditional mutual information, where do $\\tilde{z}^c$ and $\\tilde{G}$ come from? Is $\\tilde{G}$ also a random graph and does $\\tilde{G}$ have the same distribution as $G$? Then your interpretation might not be very appropriate as you explain that from an individual graph level. Apart from that, conditioning on $y = \\tilde{y}$ is not properly used in the information theory: a better notation would probably be something like $I(z^c_Y ; \\tilde{z}^c_Y |Y)$ where $z_Y^c$ and $\\tilde{z}_Y^c$ corresponds to the feature representation of $G$ and $\\tilde{G}$ given the same label $Y$.\n\n**9**. What is $\\epsilon$ after (9)? Might be a typo for $\\tau$.\n\n**10**. In (16), what does $\\phi_\\phi$ mean?  Might be a typo for $h_\\phi$?\n\n**11**. In experimental section, what are the label proportions of **Easy** and **Hard** Problems? Please specify the details.\n\nReferences: \n1. Pearl, J. (2009). Causal inference in statistics: An overview.\n2. Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference: foundations and learning algorithms (p. 288). The MIT Press.\n3. Thomas, M., & Joy, A. T. (2006). Elements of information theory.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a novel GNN called Causality-Informed GNN (CIGNN) for semi-supervised open-world graph classification, where unlabeled graph data could come from unseen classes. CIGNN takes a causal look to detect components containing the most information related to the label space and classify unlabeled graphs into a known class or an unseen class. In particular, CIGNN contains a relational detector and a feature extractor to produce effective causal features, which maximize the mutual information with label information and exhibit sufficient disentanglement with non-causal elements. Furthermore, the authors construct a graph-of-graph based on geometrical relationships, which gives instructions on enhancing causal representations. In virtue of effective causal representations, they can provide accurate and balanced predictions for unlabeled graphs. The proposed method is verified on four benchmark datasets in various settings, and experimental results reveal the effectiveness of CIGNN compared with state-of-the-art methods.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see **Weaknesses** above.", "broader_impact_concerns": "I don't have any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TUNOp22pWh", "reviewer": "Reviewer_4gxE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n(+) The open-world graph classification problem is interesting. \n\n(+) Good empirical performance.\n\n## Weaknesses\n\n(-) The clarity of the paper should be greatly improved.\n\n(-) The methodology seems not convincing.\n\n(-) The writing should be further polished.\n\n## Detail comments\n\nI agree that the problem studied in this paper is interesting and may be important in practical scenarios. The proposed CIGNN method also demonstrates good empirical performance against baseline methods. However, I personally find the paper hard to digest and I am not convinced by the proposed methodology. Firstly, I am not an expert in causal analysis, so I might miss something about the causal analysis part of the paper. \n\nAccording to my understanding, the goal of the paper is trying to learn two types of graph embedding, causal part $z^c$ and non-causal part $z^{nc}$. The high-level idea is that the downstream label $y$ is only determined by $z^c$ but not $z^{nc}$. This seems to be the design logic of equation 7. However, the logic to learn $z^c$ and $z^{nc}$ seems not unclear to me. I am not sure why using the approach defined in equation (4)-(6) is reasonable. \n\nThe description above equation (8) is also very unclear to me. The authors said that ``we consider two different graphs with an identical causal part, i.e., $\\mathcal{G}$ and $\\tilde{\\mathcal{G}}$. From our SCM, their labels, i.e., $y$ and $\\tilde{y}$ should be the same…’’. The message I got here is that if two graphs have the same causal embedding $z^c = \\tilde{z}^c$, then their label would be the same. However, equation (8) says we should maximize the mutual information between $ z^c,\\tilde{z}^c$ given $y=\\tilde{y}$. This seems not logically correct unless the former statement (in the quote) represents an if and only if relation. That is, $ z^c = \\tilde{z}^c \\Rightarrow y=\\tilde{y}$ does not imply $ z^c = \\tilde{z}^c \\Leftarrow y=\\tilde{y}$. I feel this part needs more explanation and to be more precise. \n\nThe way to tackle the unlabeled graph in the contrastive learning part is also not clear. How to choose the subgraph of the unlabeled graph? Why the subgraph embedding should be similar to the original graph? I feel many details are missing here. Finally, I do not understand the relationship between the contrastive learning and causal learning part of the objective. Does the author include $\\mathcal{L}_{con}$ merely to leverage the unlabeled graphs in some sense? What is the relation between these two parts?\n\nSome parts read weirdly in the paper:\n\n1. Is there a typo in equation (3)? The denominator seems weird. The summation should enumerate over $i^\\prime$ but the authors wrote $\\sum_{G_{i^\\prime}\\in B}$, where $G_{i^\\prime}$ is not defined.\n\n2. Section 4, second paragraph: “causal-attended” should be “causality-informed”? Otherwise why the authors named it CIGNN?\n\n3. Section 4: “…to make use of abundant labeled graphs…” I thought the authors claim the label is scarce in the introduction?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study the problem of semi-supervised open-world graph classification problem, where some novel classes are unseen during the training stage. The authors propose Casualty Informed GNN (CIGNN) to address the problem. Experimental results demonstrate the good empirical performance of CIGNN.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Improve the clarity of the paper. In particular address my questions above.\n\n2. Further polish the paper. Some parts read really weird (i.e., those I mentioned above) and I feel polishing the paper will also improve the readability of the manuscript.\n", "broader_impact_concerns": "I do not aware of broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel approach for open-world graph classification, where the unlabeled graph may come from classes that do not belong to the training label space. The proposed method was developed from a causal perspective, which detects components containing the most information related to the label space and classify unlabeled graphs into known or unseen classes. However, the reviewers pointed out the method does not fully explore the properties of a causal model and thus one may not need to go to the causal level to develop the proposed method. Thanks to the reviewers' comments, the authors have made substantial modifications to remove the causal inspirations and introduced a new "rationale" perspective. After this significant modification and additional experiments, I think the current version still presents a solid technical contribution, but with more sensible motivations. I would recommend acceptance of the paper contingent on addressing the following revisions: 

1. Ensure consistency in presenting the new "rationale" perspective throughout the paper. 
2. Review the entire manuscript to verify that the updated approach is well-integrated and clearly explained.
3. Discuss why the causal perspective may not be necessary in the proposed method and whether causality still have a potential in developing new and better methods in the future.



Decision: Accept (poster)