Symmetry-induced Disentanglement on Graphs
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Learning disentangled representations is important for unraveling the underlyingcomplex interactions between latent generative factors.
Disentanglement has beenformalized using a symmetry-centric notion for unstructured spaces, however,graphs have eluded a similarly rigorous treatment.
We ﬁll this gap with a newnotion of conditional symmetry based disentanglement, and leverage tools from
Lie algebras to encode graph properties into subgroups using suitable adaptationsof generative models such as Variational Autoencoders.
Unlike existing workson disentanglement, proposed models can learn to segregate the latent space intouncoupled and entangled parts.
Experiments on synthetic and real datasets revealthe ability of these models to learn effective disengaged representations, andimprove performance on downstream tasks such as few-shot classiﬁcation andmolecular generation.
Introduction
Disentanglement represents a fundamental desideratum in Artiﬁcial Intelligence, because it capturesinformation about the salient (or explanatory) factors of variation in the data, and it isolates information about each speciﬁc factor in only a few dimensions, unraveling the underlying interactions ofcomplex data [2, 54].
Disentangled representations have often been deemed responsible for providingneural models with the ability to improve real-word downstream tasks [2, 53, 17, 54, 32], and havedemonstrated empirically the ability to enable critical properties such as sample efﬁciency [18] andsystematic generalization [63, 28, 39].
Higgins et al.
[17] have recently presented a symmetry-based deﬁnition for disentanglement, whichuses tools from group theory to formalize the idea that disentanglement is achieved when the abstractand the vector spaces are equivariant.
Such symmetry-based deﬁnition presents various advantages:ﬁrst it aligns with the idea that auxiliary information can be exploited for imposing a structure on thelatent space [21, 36], secondly, it has been successfully used as a theoretical framework for recentdisentangled Variational Autoencoder (VAE) works [23, 47] [58, 64], ﬁnally, differently from otherdeﬁnitions of disentanglement [10, 12, 2] it may enable a fundamental connection with several graphneural network (GNN) approaches that leverage group symmetries [5, 35, 9, 3].
Disentanglement approaches have been initially developed on vision-based datasets [16, 6, 22, 8, 14],however, numerous recent methods have improved GNNs performance using disentanglement [11, 37,34, 27, 59].
For example, [34] propose to use a convolutional layer to identify the factor responsiblefor linking a node to its neighbors via a neighborhood routing mechanism, while [59] enable multirelational disentanglement, by encoding all the possible links between nodes with attention.
While previous work in the GNN domain has demonstrated the advantages of exploiting groupsymmetries [35, 3], investigating disentanglement from a symmetry group perspective is still anunexplored area in the GNN literature.
Furthermore, a rigorous formalization of disentanglement for graphs, which aligns the formalism provided by [17] within a GNN architecture is still missing in thecontemporary disentangled GNN literature [34, 59, 27].
In this paper, we make several contributions to ﬁll the mentioned gaps in disentangled GNNs.
• We introduce two formal deﬁnitions for
unconditional and conditional symmetry
-based
disentanglement on graphs, which provide the tools for encoding graph properties into subgroups by leveraging a Lie algebra parameterization.
Our intuition is that each generative factor can correspond to a Lie algebra coordinate element, and control a single graph property (Section 2).
• Based on the proposed Lie group disentanglement framework, we design two disentangledGNN modules (unconditional and conditional) that auto-encode both a graph representationand a Lie group, withing a VAE framework (Section 3).
• We provide the ﬁrst systematic evaluation on several disentanglement metrics [22, 16, 8,12, 42]
and demonstrate that the proposed models can outperform all contemporary GNN
baselines [34, 59]
and successfully learn to segregate the latent space into uncoupledand entangled parts (Section 5.1).
Finally, we show that our models are able to improve
performance on downstream tasks such as few-shot classiﬁcation, compression (Section 5.2) and molecular graph generation (Section 5.3).
Proposed framework
2.1 Symmetry-based disentanglement
We consider the deﬁnition for disentangled representations from Higgins et al.
Deﬁnition 1 (Symmetry-based disentanglement) .
Let us consider a generative process b = W
→ Omapping a set of world states W into a set of observations (i.e. data) O , and an inference process h =
O → Z mapping the data into the vector space.
By composing b and h , we obtain f = h ◦ b , whichis equivalent to f : W → Z .
Given group actions on W · : G × W → W and on Z · : G × Z → Z ,the function f is deﬁned equivariant between the actions on W and Z if the action commutes with f ,which can be expressed as an equivariant map: g · f ( w ) =
f ( g · w ) , ∀ g ∈ G, ∀
w ∈
W. Assuming thatthe symmetry group G can be decomposed into subgroups G 1 × . . .
× G n , a vector representation
Z is disentangled with respect to the decomposition of G if: (1) there is an action deﬁned on therepresentation set · : G ×
Z → Z , (2) the map f : W → Z is equivariant between actions on W and
Z , (3) there is a decomposition Z
= Z 1 × Z 2 . . .
×
Z n such that each Z i is affected only by G i andﬁxed for G j , j (cid:54)
= i .
Following the notation of Deﬁnition 1, we propose the notion of conditional disentanglement.
Deﬁnition 2 (Conditional symmetry-based disentanglement) .
We consider: a generative processb =
W × E → O mapping a set of disentangle-able world states W and non-disentangle states Einto a set of observations (i.e. data) O , and an inference process h
= O → Z mapping the data intothe vector space.
By composing b and h , we obtain f = b ◦ h , which maps W × E to Z .
Givengroup actions on W · : G × W → W and on Z · : G × Z → Z , the function f is deﬁned equivariantbetween the actions on W and Z if the action commutes with f according to the following equivariantmap: g · f ( w, e ) =
f ( g · w, e ) , ∀ g ∈ G, ∀ w ∈ W, ∀ e ∈ E .
2.2 Lie algebras for graph networks
While previous works explored disentanglement on GNNs [34, 59], they lack formal frameworksfor designing disentangled GNNs, which is necessary for aligning with the notions of symmetrygroups from Deﬁnition 1.
In the next two paragraphs, we introduce Lie algebras and leverage them toprovide a disentangled GNN architecture.
Lie groups and Lie algebras A Lie group G is a group of symmetries where the symmetries arecontinuous [49].
A Lie group is associated with a Lie algebra g , which is the tangent space to theidentity element of G .
A Lie algebra is parameterized with a basis { A i } ki =1 , where every element ing can be written as A =
A 1 t 1 + . . .
+ A k t k , where t i are the coordinates.
Elements of the Lie algebracan be mapped into the Lie group with an exponential function: exp : g → G .
In the next paragraph, we show how we can connect the notion of a Lie group G with the deﬁnition of disentanglement in a
GNN model, by associating each Lie algebra coordinates t i element with a generative factor.
GNN architecture We consider an input graph G = N × X where N is the number of nodes and
X is the number of features, and we consider a Lie group of dimension L associated with k Liealgebra basis elements.
First, we compute the graph features f = Pool ( GCN ( G )) , where the GCNmaps the features to the group size as GCN : N × X → N × L and Pool reduces the number ofnodes to N 1 using a certain pooling rate, as Pool : N × L → N 1 × L .
Next, the second layer of the GCN maps the features f to k features, obtaining a latent code Z of dimension N 1 × k .
To enhancedisentanglement, Z is then fed into k GCNs, where each GCN is a map N 1 × k
→ N 1 × k , which issubsequently aggregated for each GCN as: T = agg ( { GCN j ( Z ) =
t j } kj =1 ) .
The vector T representsthe k coordinates of a Lie algebra g , which model the generative factors of the dataset.
Finally, T isfed into the exponential map, which reconstructs the variable ˆ Z , while imposing a group structure onit, by learning the basis A ∈ g , as ˆ Z =
exp A
( T ) where A ∈ g and A ( T ) =
(cid:80) ki
{+=
+}
A i t i .
This modelis further explained in Section 3.
2.3 Formalizing disentanglement on graphs
Intuitive deﬁnition
The latent representation ˆ Z is disentangled with respect to the Lie group G ,if a change in the coordinate t i is associated with a change in ˆ z i .
In other words, disentanglementmeans that the semantics ( t 1 , t 2 , . . .
, t k ) are equivariant with respect to the properties that are beingpredicted (ˆ z 1 , ˆ z 2 , . . .
, ˆ z k ) .
Next, we provide the formal deﬁnitions for unconditional and conditionalgraph disentanglement, where the former expresses ˆ Z in function of T without accounting for Z ,while the latter expresses ˆ Z based on T , after ﬁxing an encoding Z .
Deﬁnition 3 (Unconditional graph disentanglement) .
The graph embedding ˆ Z obtained by f ( ˆ Z | T )is unconditionally disentangled with respect to the Lie group coordinates T = { t j } kj =1 if: (1) there isa group action on ˆ Z : · : G × ˆ Z → ˆ Z , (2) the map f = exp A
( T ) :
T →
ˆ Z is equivariant betweenactions on T and ˆ Z , (3) there is a decomposition ˆ Z =
ˆ z 1 × ˆ z 2 . . .
× ˆ z k , where each coordinate t iaffects only the corresponding component ˆ z i .
Deﬁnition 4 (Conditional graph disentanglement) .
The graph embedding ˆ Z obtained by f ( ˆ Z | T, Z )is conditionally disentangled with respect to the Lie group coordinates T = { t j } kj =1 conditionedon Z if (1) there is a group action on ˆ Z : · : G × ˆ Z → ˆ Z , (2) the map f : exp A ( T ) ×
Z → ˆ Z isequivariant between actions on T and ˆ Z for any ﬁxed Z , and (3) exp A ( T ) factorizes into the product 115 (cid:81) ki
=1 exp( t i A i ) ×
Z such that each component exp( t i A i )
× Z is affected only by the correspondingcoordinate t i for i ∈ { 1 , 2 , . .
. , k } .
2.4 Disentanglement constraints
Since a Lie group parameterization is not sufﬁcient for a group to be decomposed into subgroups thatare parameterized independently by a single coordinate [64, 58], we provide the necessary constraintsfor enforcing disentanglement.
Commutative penalty First, disentanglement can be achieved by enforcing a coordinate t i to beidentiﬁed by single group exp( t i A i ) , representing a single property variation in the data ˆ z i .
Suchproperty is expressed by the equivalence between the exponential map of the sum, and the productover exponential maps, which [64] proved to be achieved when there is commutativity between the
Lie algebra basis.
Namely, if A i
A j =
A j A i then exp (cid:80) ki =1 t i A i
=
(cid:81) ki
=1 exp t
i A i .
Hessian penalty Furthermore, disentanglement can be strengthened using the hessian penalty [42],which states that the Hessian matrix with respect to a disentangled representation is always zero,since there should be independence between different dimensions.
Zhu et al.
[64] adapt this penaltyin a Lie algebra parameterization setup, as H ij =
δ 2 g ( T ) δt i δt j where g ( T ) =
exp (cid:80) ki
=1 t i A i , and alsoshow that
if A i
A j = 0 , ∀ i, j then H ij = 0 .
A ,X
E G r a ph Z
E G r o up t. . .
t k
Lie Group GVAE
[Figure]
Z
GCN [Figure] µ
[Figure]
σt. . .
E Group
GCN k µ k σ k t k t. . .
t k
A
D Group
A k J
J
L
A ˆ Z exp µ σ
Z ˆ Z
Method
We design a Lie group graph VAE framework [24], as depicted in Figure 1, which considers the latentgraph encoding variable Z , the graph features X , the symmetry Lie group structure T , and the graphadjacency matrix A .
We propose two decoders, the unconditional one reconstructs the properties ˆ Zgiven T , while the conditional one reconstructs the properties ˆ Z given Z and T .
3.1 Probabilistic model
Proposition 1 (Unconditional ELBO) .
Given two latent variables Z and T , we instantiate theunconditional disentanglement from Deﬁnition 3 with a probabilistic model that maximize thelog-likelihood of the graph data G =
( X, A ) , as follows:
[Equation]
) log q
[Equation]
|
[Equation]
Proposition 2 (Conditional ELBO) .
We extend Proposition 1 with a conditional version, corresponding to the conditional disentanglement from Deﬁnition 4.
[Equation]
Propositions 1 and (2) are proved respectively in Appendix B and C.
Equations (1) and (2) are implemented with four modules, namely, two stochastic networks q ( Z |G )( E graph ), encoding graph G into vector Z and q ( T | Z ) ( E group ), encoding the group structure T givengraph encoding Z , and two deterministic networks p ( ˆ Z | T ) ( D group ), decoding ˆ Z from the groupstructure T , and p ( G| ˆ Z )
( D graph ), decoding the graph G from ˆ Z .
Note that the decoder D groupis responsible for the group structure injection, because it imposes p ( ˆ Z | T ) to be a Lie algebraparameterization.
Furthermore, in the conditional ELBO model the D group decoder is given byp ( G| ˆ Z, Z ) , where the reconstructed latent code ˆ Z , is conditioned on the ﬁxed latent variable Z .
3.2 Encoders
We ﬁrst encode the graph representation in the latent space Z , by decomposing the features Xand adjacency matrix A using an inference network E graph q
( Z | X, A ) .
We use a single 2-layer
GCN, where the ﬁrst layer reduces the graph features to the symmetry group dimension, andperforms a pooling operation to reduce the number of nodes: X = Pool ( GCN ( X, A )) , while thesecond layer reduce the number of features to k while computing the matrices µ = GCN µ ( X, A ) ,log σ = GCN σ ( X, A ) , which are then combined into
Secondly, we use E group to map the latent vector Z into a vector T , which represents the k Lie algebracoordinates.
Each { t i } ki =1 is the output of a graph convolutional encoder (GCN) over Z .
Each GCNconsists of two layers, where ﬁrst one computes Z = GCN
( Z, A ) , while the second one computesthe matrices µ = GCN µ ( Z, A ) and log σ = GCN σ ( Z, A ) , which are then combined to compute t iwith the reparameterization trick.
[Equation]
3.3 Decoders
We ﬁrst leverage D group for decoding the latent variable ˆ Z using a Lie group G and a Lie algebrag , with a Lie algebra parameterization.
Speciﬁcally, for each coordinate t i ∈ T , we learn a Liealgebra basis element { A i } ki =1 ∈ g .
The coordinates and basis are then aggregated and fed into anexponential mapping layer, to achieve a group representation, as follows.
[Equation]
For the conditional version of the ELBO, D group ˆ Z is computed by combining Z and A ( T ) , asp ( ˆ Z | Z, T )
=
σ ( agg [exp A ( T ) , Z ]) , where Z = { z j } k j =1 , agg is the mean, and σ is the ReLUnon-linear activation function.
Secondly, we apply D graph , which ﬁrst maps the latent ˆ Z into N latent variables using sigmoidfunctions σ j , and then aggregates the N latent variables to obtain the full graph representation.
[Equation]
3.4 Training
The proposed unconditional and conditional models are trained by optimizing the loss given byβ L + λh + γc , where L is the ELBO, from Equation (1) (unconditional) or (2) (conditional), whileh and c are respectively the hessian and commutative penalties, obtained as regularization terms.
Theparameters are found empirically, and we provide further insights on their effect on the model and itsperformance by providing an ablation study in the experiments in Section 5.
Related work
Symmetry-based disentanglement Initial works on symmetry disentanglement [7, 45] have proposed use reinforcement learning to achieve irreducible representations of group elements throughobservation of action transitions, while [41] does not require labelled action-transition pairs.
In arelated line of work, [64] propose to decompose a Lie group in multiple subgroups, and enforcecommutativity and independence between subgroups, while [58] impose a cyclic group structure,while encouraging such group to be isomorphic to the ground truth.
Other works focus on manifoldsproperties, for example, [43], propose a nonparametric algorithm that aims to discover a decomposition of a data manifold by investigating its holonomy group, while [13] model disentanglementas a product of the low-dimensional submanifolds underlying the data space.
Finally, [56] designa method which iteratively updates a partition, modeled as a symmetry group, until the consideredfactor is invariant, while [4] uses group representation theory for deﬁning distributed latent operators.
Differently, from the reviewed methods, we propose to analyze the graph data modality.
Disentangled GNNs [34] propose a routing mechanism aimed at separating the underlying factorsin a graph dataset, while [59] use an attention mechanism for disentangling the representationof GNNs, and provide two evaluation metrics for disentanglement on graphs.
In a related lineof work, [31] achieve disentanglement by encouraging independence in the latent space, while[1] propose a method for disentangling multi-relational graphs with an application on pedestriantrajectory prediction.
Furthermore, [11] propose a generative model for spatio-temporal graphs viadisentangled representations, while [27] leverage a contrastive framework for disentangling the latentfactors in GNNs.
Differently from the mentioned GNNs methods, we provide a formal deﬁnition ofdisentanglement for graphs, and do leverage group structures for modeling.
Graph generative models We also relate to VAEs models on graph data modality, for example [57,62] propose VAE-based Information Bottleneck methods for improving GNNs performance onclassiﬁcation tasks, by learning minimal sufﬁcient representations.
Another line of work aimsto generate molecular graphs, by proposing graph-VAE models that are provided with encoderand decoders able to handle graph structures, and to condition molecule generation on speciﬁcproperties [30, 52].
Similarly, [61] propose a deep autoregressive model that learns to generate graphsby training on a representative set of graphs and decomposes the graph generation process into asequence of node and edge formations.
We differ from these methods as we explicitly investigate thedisentanglement capabilities of generative models.
Experiments
Section 5.1 evaluates the disentanglement capabilities of the models, Section 5.2 provides a compression and few-shots classiﬁcation experiment, Section 5.3 assesses the generation capabilities onmolecular datasets.
5.1 Disentanglement evaluation
Experimental setup Vision-based disentanglement [16, 22] requires the generative factors to beknown, is based on synthetic datasets that have such property.
We follow the protocol, and constructtwo graph datasets based respectively on random graphs, where we can sample a combination ofgenerative factors that are required to synthesize node features and adjacency matrices in each graph.
More details on the datasets are in Appendix A.
We measure 5 disentanglement metrics from theliterature, including β VAE metric [16] (Beta), FactorVAE metric (FVM) [22], Mutual Information
Gap (MIG) [8], DCI Disentanglement [12], and Factor Leakage (FL) [41].
Ablation study We start by evaluating the impact of our model components on the disentanglementscores, by maintaining all parameters ﬁxed, except from one that varies.
We probe group size, hessianpenalty, commutative penalty, subspace size, beta penalty, and model depth.
In Figure 2 we report themean over 10 runs for our unconditional model on the synthetic Watts-Strogatz dataset.
We observethat when the group size is increased, there is an improvement in disentanglement across all metrics,and this can be explained by the idea that with a larger group size, the subgroups, represented bythe Lie algebra coordinate t i can control different data variations.
The hessian penalty results to beeffectively enhancing disentangled representations, while the commutativity penalty is less effective,and we can explain this result, because the former requires the Lie algebra basis elements to havemutual products of zeros while subgroup decomposition only requires their commutators to be zeros,which also conﬁrms the results obtained for image datasets [64]. [Table] [Table]
Quantitative disentanglement benchmark We compare our models with several baselines, over10 random seed runs, where we measure the mean and variance, by training for 200 epochs on theproposed synthetic datasets and measuring the quantitative metrics on 1000 data points.
Baselinesinclude GAT [55] and GCN [25] based autoencoders, as well as disentangled graph methods such as
FAC [59] and DIS [34].
Our unconditional and conditional Lie group models are named UNC and
CON and set with group size of 81 and hessian penalty 40.
In Table 1 we observe that our Lie group models are able to improve on previous baselines on allmetrics on the evaluated datasets, and the conditional version achieves the best performance in mostcases.
The FL and FVM metrics show the highest performance increase.
FL is designed to capturethe disentanglement on each generative factor, which, in our models is controlled by the group size,while in previous approaches is not considered.
FVM is the most used metrics for vision datasets,and provides a more robust assessment than the Beta metric.
We note that while previously proposeddisentangled approaches DIS and FAC demonstrate higher disentanglement when compared to GCNand GAT , they are outperformed by our methods.
5.2 Compression and few-shots classiﬁcation
Experimental setup Following the setup from [15], we consider the task of learning a compressedmodel via multiple pooling operations, and test on few-shots classiﬁcation capabilities on datasetswith multiple classes, including FRANKENSTEIN [40] (2 classes), COLORS-3 [26] (11 classes),
Mutagenicity [48] (2 classes), NCI1 [50] (2 c).
The baselines include MIA [15], which propose anattention-based pooling for compression, as well as GCN [25], and GAT [55] based autoencoders,equipped with the same pooling layer.
All models are set to have a pooling rate of 0.8, and a depth of3.
Our models are set with a group size of 81 and hessian penalty of 20.
The task consists in trainingﬁrst without labels to learn the representation, and then using the compressed model as input fortraining a GCN classiﬁer a labelled subset of the dataset, which involves 100 samples for training and100 for testing.
Results In Table 2 we report the mean and variance across 10 runs for MSE, the classiﬁcationaccuracy, and the size of the trained model.
The results show that CON and UNC outperform theconsidered baselines in terms of few-shots classiﬁcation accuracy on all datasets, while achieving asmaller compressed size.
We hypothesize that the generalization performance is due to more efﬁcientlearning due to disentanglement, in fact disentangled representations have been studied in the contextof generalization in previous work [38, 63, 28].
In terms of MSE, our models show a decrease inperformance compared to MIA , but improve on GCN and GAT based autoencoders.
[Table] [Table]
generation
The molecular generation experiment is two-fold, the ﬁrst part involves learning the distribution of themolecular data while the second one involves ﬁne-tuning the pre-trained model with a Reinforcement
Learning (RL) procedure for a downstream generation task.
We employ our disentangled modelsduring the training phase, and then follow the RL method from GCPN [60] for ﬁne-tuning tasks,which include random generation, property optimization and constrained optimization.
For alltasks, we follow the setups from previous GNN approaches [20, 51, 33] designing our experimentsbased on TorchDrug 2 and DIG [29].
We set our disentangled models with a group size of 81 and ahessian penalty of 40.
In terms of baselines, we include JT-VAE [20], GraphAF [51], GraphDF [33],
GCPN [60].
Random generation This task evaluates the quality of randomly generated samples on 4 standardmetrics in percentage, including: valid molecules with resampling (Val), valid molecules withoutresampling (Res), unique molecules (Uni), novel molecules that not appear in the training data (Nov).
Following the baselines’ setup, we evaluate on ZINC [19], QM9 [46], and MOSES [44].
We train ourmodels for 10 epochs, with a batch size of 32 and the learning rate of 0.001, and compute the metricsover 10K generated molecules.
In Table 3 we observe that our CON model achieves the highest scoreof validity withour resampling on the ZINC dataset, and we note that both UNC and CON are ableto improve the performance of GraphAF and GCPN on most metrics, while achieving comparableperformance with GraphDF.
Since our model is not optimized for molecule generation, the positiveresults of UNC and CON indicate that disentangled representations may represent way to achieve moreexpressive generation, and that encouraging the separation of generative factors into different latentcodes, may be beneﬁcial for improving the quality of the samples.
Property optimization The goal of the task is to generate novel molecules with high property scoresfor the penalized logP property, and the quantitative estimation of drug-likeness property (QED).
Thesetup involves pretraining our models for 300 epochs on ZINC random generation, applying the RLﬁne-tuning procedure from GCPN, and measuring the scores from the top 3 generated molecules.
In
Table 4 we observe that our models are able to outperform GCPN and GraphAF for both penalizedlogP and QED, and achieve comparable performance to the more advanced GraphDF.
In Figure 3 we show the molecules generated using CON .
While our model does not outperform GraphDF, this resultis important because it demonstrates that the disentangled representations learned from our modelscan be leveraged to improve the performance of the downstream task.
Constrained Optimization This task aims to modify the input molecular graph for improving itspenalized logP score while keeping the similarity between the input and modiﬁed molecules higherthan the threshold δ .
We follow the setup of GraphAF and GraphDF, which select 800 moleculesfrom ZINC with low penalized logP scores as the input molecules to be optimized.
We pretrain ourmodel on the random generation task for 300 epochs, and then ﬁne-tune with the RL procedure from
GraphAF.
We report the mean and standard deviation of the largest property improvement (Imp), andsimilarities (Sim) between them and their corresponding input molecules, as well as the success rate(Suc).
In Table 5 we compare the results for our CON model with GraphAF and GraphDF.
We observethat CON is able to outperform GraphAF
3 out of 4 times for improvement values, and achievescomparable results with GraphDF, which we motivate by the fact that GraphDF provides furtherﬁne-tuning in order to improve the baseline.
In terms of similarity and success rate, our model hascomparable results with the baselines.
[Table]
Conclusion
We formalize the notion of conditional disentanglement on graphs and propose a novel framework forgraph disentanglement by leveraging tools from Lie algebras.
Based on the new deﬁnition of disentanglement, we design a graph VAE based on a Lie group parameterization, and provide a novel ELBOcriteria for optimizing conditional disentanglement.
Our method achieves superior performance onquantitative disentanglement benchmarks when compared to contemporary disentangled GNNs andother convolutional layers.
Finally, we demonstrate strong capabilities on few-shots classiﬁcation andmolecular generation experiments.
Limitations We rely on synthetic dataset for evaluating disentanglement, because current quantitative metrics are based on the ability to sample each factor and combine them into an observation.
Since this is and open problem in representation learning [22], future work will investigate how toprovide metrics that can evaluate disentanglement on real-word datasets.
Checklist
For all authors...
[Table] (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] Provided in Appendix3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] Provided as zip
ﬁle in the supplemental material
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] We specify the experimental setups in Section
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See beginning of Section
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See beginning of Section 54.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 5.
(b) Did you mention the license of the assets?
[Yes] See Section 5.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]We include our assets in the supplemental material as zip ﬁle.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]