R ELATIONAL L EARNING WITH V ARIATIONAL B AYES
Anonymous authors
Paper under double-blind review
A BSTRACT
In psychology, relational learning refers to the ability to recognize and respond to relationship among objects irrespective of the nature of those objects.
Relational learning has long been recognized as a hallmark of human cognition and a key question in artiﬁcial intelligence research.
In this work, we propose an unsupervised learning method for addressing the relational learning problem where we learn the underlying relationship between a pair of data irrespective of the nature of those data.
The central idea of the proposed method is to encapsulate the relational learning problem with a probabilistic graphical model in which we perform inference to learn about data relationships and other relational processing tasks.
I NTRODUCTION
American Psychological Association deﬁnes relational learning as (VandenBos & APA, 2007):
Deﬁnition 1.1 ( Relational learning ) .
Learning to differentiate among stimuli on the basis of relational properties rather than absolute properties.
In other words, relational learning refers to the ability to recognize and respond to relationship (called relational property ) among objects irrespective of the nature of those objects (called absolute property ).
Relational learning has long been recognized as a hallmark of human cognition, and there has been substantial research showing that adequate cognitive capacity is necessary for relational processing (Biederman, 1987;
Medin et al., 1993; Holyoak, 2012; Doumas & Hummel, 2013;
Gentner, 2016).
As a machine learning application, relational learning can provide new insight into data analysis by dissecting information in the data into relational property and absolute property.
However, in order to discover relationship patterns among raw and unknown data, relational learning is only truly useful if it can be achieved without supervised data.
A key challenge in learning relational property with machine learning-based methods is that relational property is an abstract construct; unlike absolute property, which is based on observable data and can be quantitatively measured, relational property is an abstract quantity that is difﬁcult to objectively quantify, especially when the learning is unsupervised.
In this work, we propose an unsupervised learning method—variational relation learning (VRL)—for addressing the relational learning problem.
The proposed method is completely unsupervised, which means that the learning does not require a labeled training dataset nor training examples that have the same (or different) relational property.
At its core, VRL encapsulates the relational learning problem with a probabilistic graphical model (PGM) in which we perform inference to learn about relational property and other relational processing tasks.
Furthermore, our main learning algorithm is derived from the PGM using ﬁrst principles, which gives us the ﬂexibility to use any compatible computational inference method and still retains the desired properties of the proposed method.
Our contribution in this paper is threefold.
First, we propose a PGM that encapsulates the relational learning problem.
Second, we formulate various relational processing tasks as performing inference and learning in the PGM.
Third, we propose an efﬁcient and effective learning algorithm that can be trained end-to-end and unsupervised.
P ROBLEM DEFINITION
We begin with formulating the relational learning problem as a machine learning problem: we observed a paired dataset X
=
( a ( i ) , b ( i ) )
| i ∈
[1 ..N ] } consisting of N i.i.d samples generated from a joint distribution p ( a ∈A , b ∈B ) ; our goal is to learn a relational property between a ( i ) and b ( i ) irrespective of their absolute property.
Furthermore, we want the learning to be unsupervised, e.g., we do not require a labeled dataset, such as ( a ( i ) , b ( i ) , z ( i ) )
where z ( i ) is a target variable indicating ( a ( i ) , b ( i ) )
’s
relational property, nor do we require training examples that have the same (or different) relational property.
There are two distinct features that separate our problem formulation from other unsupervised learning problem formulations:
We dissect the information in X into relational property and absolute property ; relational property characterizes the relationship between a ( i ) and b ( i ) , whereas absolute property represents speciﬁc features that independently describe a ( i ) and b ( i ) .
Our goal is to learn a relational property among X irrespective of its absolute property, i.e., we want to learn a relational property that is decoupled from the absolute property.
In addition, we are interested in two related relational processing tasks: relational discrimination 1 and relational mapping 2 (VandenBos & APA, 2007).Relational discrimination allows us to differentiate ( a ( i ) , b ( i ) ) from ( a ( j ) , b ( j ) ) based on their relational properties, while relational mapping allows us to apply the relational property of ( a ( i ) , b ( i ) ) to a different set of data, for example, deduce that b ( j ) is related to a ( j ) in the same way that b ( i ) is related to a ( i ) .
M ETHOD
Here we introduce the proposed VRL method for addressing the relational learning problem and discuss various optimization challenges unique to VRL.
3. V ARIATIONAL RELATION LEARNING
The proposed VRL method consists of two parts: ﬁrst, we encapsulate the relational learning problem with a PGM, called VRL-PGM; we then formulate various relational processing tasks as performing inference and learning in VRL-PGM.
[Figure]
The VRL-PGM model, shown in Fig.
1, generates data a , z , and b by sampling from PDFs that come from parametric families of distributions— p θ ( a ) , p θ ( z ) , p θ ( b | a , z ) —that are differentiable almost everywhere with respect to (w.r.t.)
a , z , and θ .
In practice, we observe only a set of independent realizations { ( a ( i ) , b ( i ) )
| i ∈
[1 ..N ] } while the true parameter θ ∗ and the corresponding latent variables z ( i ) are unobserved.
A key property of VRL-PGM is that random variables a
and z are independent but not conditionally independent given b , i.e., p θ ( a , z )
=
p θ ( a ) p θ ( z ) , p θ ( a , z | b ) (cid:54) =
p θ ( a | b ) p θ ( z | b ) (Bishop, 2006).
In VRL-PGM, the absolute property can be interpreted as representing the dependency between a and b , i.e., features in a that can be used to predict b , while
Deﬁnition ( Relational mapping )
The ability to apply what one knows about one set of elements to a different set of elements.
the relational property , represented by the latent variable z , can be interpreted as any additional information not found in a but can help to better predict b .
The key requirement that the learned relational property be decoupled from the absolute property is enforced by VRL-PGM’s construction where z , which represents the relational property, and a , from which the absolute property is derived, are independent.
We note that the proposed VRL-PGM is based a directed acyclic graph (DAG) that artiﬁcially introduces causal relationship between a and b as well as z and b .
VRL-PGM reﬂects our priority and compromise for using a DAG to represent the original abstract relational learning problem: we sacriﬁce some identiﬁability of the original abstract problem but we gained a rigorous and mathematical tractable PGM while achieving our primary objective of learning a decoupled (independent) relational property.
Additional discussions on the connection between VRL-PGM and the original relational problem is included in the appendix.
Having established VRL-PGM, our primary learning objective is to approximate the unknown true likelihood function p θ ( b | a , z ) and posterior p θ ( z | a , b ) (note that by observing b , random variables z and a are no longer independent).
Learning p θ ( z | a , b ) provides us a way to infer ( a ( i ) , b ( i ) ) ’s
relational property z ( i ) ; moreover, it serves as a basis for performing relational discrimination where we compare relational properties between different pairs of data.
Learning p θ ( b | a , z ) allows us to perform relational mapping where we use the relational property of ( a ( i ) , b ( i ) ) to map from a ( j ) to b ( j ) , i.e., b ( j ) ∼ p θ ( b | a ( j ) ,
z ( i ) ) where z ( i ) ∼ p θ ( z | a ( i ) , b ( i ) ) .
3. V ARIATIONAL LOWER BOUND
We estimate the parameter for p θ ( b | a , z ) by following the maximum-likelihood (ML) principle, and approximate the true posterior p θ ( z | a , b ) with variational Bayesian approach.
More speciﬁcally, we use a variational distribution q φ ( z | a , b ) , parameterized by φ , to approximate the unknown (and often intractable) true posterior through maximizing a variational lower bound (Bishop, 2006).
To derive such a lower bound, we ﬁrst write the log-evidence as log p θ ( X ) =
log p θ ( { ( a ( i ) , b ( i ) )
| i ∈
[1 ..N ] } )
= [Equation] Ni =1 log p θ ( a ( i ) , b ( i ) ) , where each term in the summation can be expressed as:
[Equation]
The ﬁrst term on the RHS is the KL-divergence from p θ ( z | a ( i ) , b ( i ) ) to q φ ( z | a ( i ) , b ( i ) ) , which provides a measure of dissimilarity between the two distributions; the second term on the RHS continues as:
[Equation]
where in the second line we use the fact that random variables a and z are independent.
Substitute
Eq.
(2) back in (1) and rearrange terms gives us:
[Equation]
where
[Equation]
Since KL-divergence is non-negative, L ( θ, φ ; a ( i ) , b ( i ) ) (abbreviated as L ( i ) for notation compactness) serves as a lower bound for the conditional log-likelihood log p θ ( b ( i ) | a ( i ) ) .
Maximizing L ( i ) w.r.t.
φ and θ gives us both a ML estimate for p θ ( b | a , z ) (by maximizing the ﬁrst term inside the expectation in Eq.
(4)) and a lower KL-divergence (the better q φ ( z | a ( i ) , b
( i ) ) approximates the true posterior p θ ( z | a ( i ) , b ( i ) ) )
as the conditional log-likelihood log p θ ( b ( i ) | a ( i ) ) does not depend on φ .
The lower bound L ( i ) can be maximized with gradient ascend methods; however, its gradients w.r.t.
φ is difﬁcult to obtain: the expectation in Eq.
(4) is taken w.r.t.
the distribution q φ ( z | a ( i ) , b ( i ) ) , which is a function of φ (Paisley et al., 2012).
To obtain efﬁcient estimators for both L ( i ) and its gradients, we adopt the reparameterization trick developed in Kingma & Welling (2014) where the random variable z is expressed as a transformation of another random variable (cid:15) ∼ p ( (cid:15) ) that is independent of a , b , and φ :
z = g ( (cid:15) , a ( i ) , b
( i ) , φ ) where g is some differentiable and invertible transformation.
Given such a change of variable, the lower bound L ( i ) can be rewritten as:
[Equation]
where z = g ( (cid:15) , a ( i ) , b ( i ) , φ ) and (cid:15) ∼ p ( (cid:15) ) .
Note that the expectation in Eq.
(5) is taken w.r.t.
p ( (cid:15) ) and we can now approximate L ( i ) with a Monte Carlo estimator:
[Equation]
where z ( i,l ) =
g ( (cid:15) ( i,l ) , a ( i ) , b ( i ) , φ ) and (cid:15)
( i,l ) ∼ p ( (cid:15) ) .
The lower bound for a minibatches of data
X M = { ( a ( i ) , b ( i ) )
| i ∈ [1 ..M ] } can be approximated by (cid:101) L ( θ, φ ; X M ) =
NM (cid:80) Mi
=1 (cid:101) L ( i ) .
And ﬁnally, the gradients ∇ θ,φ (cid:101)
L ( θ, φ ; X M )=
NM (cid:80) Mi
=1 ∇ θ,φ (cid:101) L ( i ) can be computed in a straightforward manner and used to update the parameters θ and φ with stochastic optimization methods, such as
SGD.
3. O PTIMIZATION CHALLENGES
The proposed VRL method introduces unique challenges to the variational lower bound optimization problem (see Sønderby et al.
and Bowman et al.
for other known challenges).
To explain these challenges, we ﬁrst break down
VRL’s parameter updating process into the following steps (using a single datapoint as an example): (1) a datapoint ( a ( i ) , b ( i ) ) is selected; (2) sample z ( i ) ∼ q φ k
( z | a ( i ) , b ( i ) ) by using the current parameter φ k ; (3) evaluate (cid:101) L ( i ) by using φ k , θ k ; (4) calculate gradients g = ∇ θ k ,φ k (cid:101)
L ( i ) ; (5) use gradients g to update φ k , θ k and get new parameters φ k +1 , θ k +1 .
This parameter updating process can be depicted with an information ﬂow diagram shown in Fig.
2a.
Ideally, we would like every path in Fig.
2a to contribute to the evaluation of all the terms in its reachable nodes in order to obtain meaningful gradients for updating its associated parameters; however, there are two situations where this is not the case.
The ﬁrst situation, called p θ ( b ( i ) | a ( i ) , z ( i ) ) , p θ ( z ( i ) )
z ( i ) ∼ q φ ( z | a ( i ) , b ( i ) )
b ( i ) φ a ( i ) φ θ a ( i ) θ (a) p θ ( b ( i ) | a ( i ) , z ( i ) ) , p θ ( z ( i ) )
z ( i ) ∼ q φ ( z | a (cid:48) ( i ) , b (cid:48) ( i ) )
b (cid:48)
( i ) φ a (cid:48)
( i ) φ θ a ( i ) θ (b) information-shortcut , occurs when the learning of p θ ( b ( i ) | a ( i ) , z ( i ) ) rely entirely on the dashed path in Fig.
2a; more speciﬁcally, the dashed (shortcut) path directly propagates b ( i ) through z ( i ) to p θ ( b ( i ) | a ( i ) , z ( i ) ) and, as a result, the relational property z ( i ) may learn only to encode the absolute property of b ( i ) .
The second situation, called deterministic-mapping , occurs when b ( i ) procedure VRL
( X , p ( (cid:15) ) , D ) (cid:46)
If RPDA not available, D = { id( · ) | ( a , b )
= i
d( a , b ) } Initialize parameters θ, φ while not convergence of parameters ( θ, φ ) do
Sample minibatch X M = { ( a ( i ) , b ( i ) )
| i ∈ [1 ..M ] } from X .
Sample (cid:15) ( i,l ) ∼ p ( (cid:15) ) , r ( i ) ∼ U ( R ) , i =1 , ..., M, l =1 , ..., L .
Run RPDA and obtain ( a (cid:48) ( i ) , b (cid:48) ( i ) )
=
d ( a ( i ) , b ( i ) ; r ( i ) ) , i =1 , ..., M .
Compute gradients g = ∇ θ,φ (cid:101)
L (cid:0) θ, φ ; X M (cid:1) =
NM (cid:80) Mi
=1 ∇ θ,φ (cid:101) L ( i ) RPDA (see Eq.
Update parameters θ, φ using gradients g (e.g. SGD).
end while return θ, φ end procedure can be fully characterized by a ( i ) ; in this case, the learning of p θ ( b ( i ) | a ( i ) , z ( i ) ) may rely only on the dash-dotted path in Fig. 2a. While both situations can be viewed as a overﬁtting problem, deterministic-mapping says more about the data itself and exposes a potential limitation of VRL: a decoupled relational property is primarily learned through exploring information not found in a but can help to better predict b , and when a fully characterizes b , VRL no longer need to explore information beyond a to predict b and this may prevent VRL from learning a meaningful relational property. On the other hand, information-shortcut is caused by short-cutting the parameter updating process, which we may overcome with additional regularization techniques.
Here we propose two approaches for mitigating the information-shortcut problem by disrupting the ﬂow of information passing through the shortcut path.
In the ﬁrst approach, we restrict the ﬂow of information by constraining the expressiveness of the latent variable z ; for example, by adopting an informative prior with restrictive constraint, such as p θ ( z ) =
N ( z ; 0 , σ 2 I ) , σ (cid:28) 1 , or representing z with a discrete random variable (assuming we know a priori the underlying relational proproty are discrete).
In the second approach, we propose a novel data augmentation strategy—relationpreserving data augmentation (RPDA)—that aims to eliminate the shortcut path.
First, we deﬁne a set of relation preserving functions D = { d ( a , b ; r ) | r ∈ R (some index set) , d : A×B →A×B } where the data relationship is preserved in the following sense: p θ ( z
[Equation]
) for ( a (cid:48) , b (cid:48) )
=
d ( a , b ; r ) , ∀ r .
Assuming we have access to such a D , the proposed RPDA strategy then seek to optimize a modiﬁed lower bound estimator (cid:101) L ( i ) RPDA :
[Equation]
) where ( a (cid:48) ( [Equation] , r (
[Equation]
and z ( i,l ) = g ( (cid:15) ( i [Equation] l )
[Equation] a (cid:48) ( i [Equation] (cid:48) ( i ) , φ ) , (cid:15) ( i,l ) ∼ p ( (cid:15) ) .
Note that due to the relation preserving property of D , we have q φ ( z
( i,l ) | a (cid:48)
( i ) , b (cid:48)
( i ) )
=
q φ ( z ( i,l ) | a ( i ) , b ( i ) ) and, therefore, (cid:101)
L ( i ) RPDA is equivalent to (cid:101) L ( i ) in Eq.
When we optimize with (cid:101) L ( i ) RPDA , the parameter updating process can be redrawn in
Fig.
2b, where now the learning of p θ ( b ( i ) | a ( i ) , z ( i ) ) can no longer rely solely on the shortcut path to propagate b (cid:48)
( i ) since it differs from b ( i ) by a non-deterministic factor r ( i ) .
In practice, it may seem unrealistic to assume that we can construct a set of RPDA functions D without extensive knowledge of the underlying relational property.
However, we can treat data augmentation as a form of regularization and construct a D that reﬂects our prior knowledge and belief of the underlying system (Ronneberger et al., 2015; Perez & Wang, 2017).
For example, if we want the learning to be rotation invariance (a common theme in computer vision applications), we can construct a D that consists of image rotation augmentations, e.g., d ( a , b ; r )=(rot( a , r ) , rot( b , r )) where rot( x , r ) rotates the image x by r ∈ R =
[0 , 360) degrees (note that both a and b are rotated by the same amount).
To summarize this section, the proposed VRL method with RPDA is described in Algorithm 1.
R ELATED WORK
Machine learning approaches for relational processing have gained increasing interest and attention in the literature.
Most of these methods focus on high-level cognitive tasks, such as visual Q&A and state prediction for complex-physics systems, and derive their relational processing capabilities from learning with clever designed neural networks (Hill et al., 2019; Santoro et al., 2017; Raposo et al., 2017; Battaglia et al., 2018; 2016; Wu et al., 2015;
Reed et al., 2015; van Steenkiste et al., 2018;
Chang et al., 2016; Fragkiadaki et al., 2015).
Our work differ from these methods in two ways: (1) our primary focus is addressing the relational learning problem where we want to learn a decoupled relational property; (2) we enforce the decoupling requirement on the learned relational property with a PGM, which gives us the ﬂexibility to use any compatible inference method or function approximation and still satisfy the decoupling requirement.
Conventional unsupervised learning methods can also be appied to our problem setting (Kingma &
Welling, 2014; Goodfellow et al., 2014; Mikolov et al., 2013a;b); however, these methods learn a single representation for the data with superimposing information about their relational and absolute property.
The difﬁculty of decoupling the relational property from the learned representation constitute a major obstacle to relational reasoning.
Other related work include methods on learning a disentangled representations with applications in style-transfer, image-to-image translation, domain adaptation, etc.
(Huang et al., 2018;
Chen et al., 2016;
Tenenbaum & Freeman, 1997; Higgins et al., 2017; Bousmalis et al., 2016; Mathieu et al., 2016; Tulyakov et al., 2017; Denton & Birodkar, 2017; Villegas et al., 2017; Donahue et al., 2017;
Shen et al., 2017)
Most of these methods strive to learn a disentangled representations of content and style (or pose for video sequence data) where content is generically deﬁned as the underling spatial structure, and style as the rendering of the structure.
In comparison, our work can be viewed as learning a disentangled representations of relational and absolute property ; however, we argue that style-content separation is fundamentally different from relational-absolute separation.
More speciﬁcally, we view absolute property as incorporating both style and content information (both describe features of an individual data), while relational property provides new information on the (abstract) relationship between the paired data.
E XPERIMENT
In this section, we present experimental results from applying the proposed VRL method to a set of relational learning tasks designed with the MNIST dataset (LeCun & Cortes, 2010).
5. R ELATIONAL LEARNING TASK
To setup a relational learning task, a paired dataset X
= { ( a ( i ) , b ( i ) )
| i ∈
[1 ..N ] } was generated by the following steps: (1) the MNIST dataset { ( x ( i ) , y ( i ) )
| i ∈ [1 ..T ] } , where x ( i ) and
y ( i ) are the digit images and their labels, was augmented with applying ﬁve evenly-spaced rotations to each of the image x
( i ) to get { ( x ( i,j ) , y
( i ) )
| i ∈ [1 ..T ] , j ∈
[1 .. 5] } (examples of augmented images are shown in Fig. 3); (2) individual datapoint ( a ( i ) , b ( i ) ) of X was chosen to be randomly rotated images of x ( i ) ,
i.e., X = { ( a ( i ) =
x ( j,k ) , b ( i ) =
x ( j,l ) ) | j ∼ U ([1 ..T ]) , k, l ∼ U ([1 .. 5]) } .
Note that there are ﬁve uniquely deﬁned rotational relationships between ( a ( i ) , b ( i ) ) in X and since they are randomly selected, the rotational relationship (relational property) is decoupled from the absolute properties of ( a ( i ) , b ( i ) ) .
Here, we use X to assess VRL’s capability of discovering the underlying relational property (rotational relationships) irrespective of their absolute property.
5. I MPLEMENTATION
For these experiments, we adopted a two-dimensional latent variable z ∈ R 2 and let the prior p θ ( z ) be the bivariate normal distribution.
We let p θ ( b | a ( i ) , z
( i,l ) ) be a multivariate Bernoulli distribution whose probability parameters are computed from a given a ( i ) and
z ( i,l ) with an autoencoder-like neural network f dec θ (cid:0)
f enc θ ( a ( i ) ) ,
z ( i,l ) (cid:1) .
We let the approximated posterior q φ ( z | a (cid:48)
( i ) , b (cid:48)
( i ) ) be a bivariate Gaussian distribution with a diagonal covariance
N ( z ; µ ( i ) , ( σ ( i ) ) 2 I ) where µ ( i ) and σ ( i ) are the output of a neural network f qφ ( a (cid:48)
( i ) , b (cid:48)
( i ) ) .
For RPDA, we constructed a D that consists of image rotation augmentations: D = { (rot( a , r ) , rot( b , r )) | r ∈ [0 , 360) } .
Detailed experimental setup is described in Appendix.
5. R ESULTS
Relational discrimination.
We trained VRL on X and used the approximated posterior q φ ( z | a , b ) to infer the relational property of a hold-out dataset.
Figure 4a shows a scatter plot of the relational property inferred by VRL where we can see that the approximated posterior accurately cluster(discriminate) data with the same(different) rotational relationship together(apart).
Here, we compared VRL with variational autoencoder (VAE) (Kingma & Welling, 2014) and
InfoGAN (Chen et al., 2016), both of which can learn data representations of X in a completely unsupervised manner.
In the application of VAE, we adopted a 2-D latent space and use the encoder trained on
X to inferred the latent varible of a hold-out dataset; the resulting scatter plot is shown in Fig.
4b where we see that VAE failed to discriminate data based on their relational property (rotational relationship).
Next, InfoGAN has demonstrated its ability to learn disentangled representations −−  2 z 1 −−   z+
+ + + + (a) Relational properties inferred by VRL −−  4 z−−
z+ + + + + [Figure]
(represented by structured latent codes c 1 , c 2 , ..., c L ) through generative modelling.
Although inferring latent codes for a given data point is a non-trivial task for InfoGAN, we can examine the learned latent representation by manipulating the latent codes and visually inspect the generated random samples.
We followed the examples in Chen et al.
and modeled the latent codes with one categorical code c 1 ∼ Cat ( K = 10 , p = 0 . 1) (model discontinuous variation in data) and two continuous codes c 2 , c 3 ∼ Unif ( − 1 , 1) (capture continuous variations in style).
Figure 5 shows examples of generated images from manipulating the latent codes; it is clear that none of the latent codes (or a combination of them) distinctively capture the full range of relational property (rotational relationship).
Figures 4b and 5 illustrate a major challenge of using VAE and InfoGAN (and other related methods) for relational learning: these methods learn a single representation that encodes both relational and absolute property of X and it is difﬁcult to dissect the relational property from the learned representation.
Detailed experimental setup for both VAE and InfoGAN is described in
Appendix.
[Figure]
Relational mapping.
We evaluated VRL’s learned likelihood function p θ ( b | a , z ) by visualizing the predicted images given a and z .
We chose a from a hold-out dataset and z from: (1) direct sampling in the latent space; (2) relational property inferred from a source datapoint ( a s , b s ) .
Figure 6a shows predicted images with z sampled from the latent space shown in Fig.
4a.
Figure 6b shows examples of relational mappings from a ( c ) to b ( r,c ) by applying the relational property inferred from a source datapoint ( a s , b ( r ) s ) .
At ﬁrst glance, the results in Fig.
6b resemble that of style-transfer, but they are fundamentally different: in style-transfer, the image b ( r,c ) is generated by applying the style of b ( r ) s to the content of a ( c ) , whereas VRL generates image b ( r,c ) by applying the relationship of ( a s , b ( r ) s ) to the image a ( c ) .
It is evident from Fig.
6b that predicted images b ( r,c ) do not share similar style to b ( r ) s , but rather the same rotational relationship w.r.t.
a ( c ) and a s .
[Figure]
[Equation]
[Figure]
D ISCUSSION AND CONCLUSION
The proposed method comes with both advantages and disadvantages: the main advantage of VRL lies in its relational learning capabilities; however, this may also be one of its disadvantages.
More speciﬁcally, VRL can learn a decoupled relational property even when it is coupled with the absolute property, i.e., VRL is oblivious to the coupling information between the two properties (an example is provided in Appendix).
Nevertheless, such information may be of interest to the user, and in this regard, VRL provides only a partial view of the data.
In conclusion, the proposed VRL method is an efﬁcient and effective unsupervised learning method for addressing the relational learning problem where our goal is to learn a decoupled relational property.
By dissecting the data information into decoupled relational and absolute property , we hope VRL can bring new insight into everyday data analysis and ultimately ﬁnd applications for a wide variety of problems.
A A DDITIONAL REMARKS ON VRL-PGM
The central idea of the proposed VRL method is to encapsulate the relational learning problem with a probabilistic graphical model—VRL-PGM—and then formulate various relational processing tasks as performing inference and learning in the graphical model.
Here we discuss aspects of the original relational learning problem (see Section 2) that differ from the proposed VRL-PGM (see Section 3.1).
First, the original problem speciﬁes that the relational property be decoupled from both a ’s and b ’s absolute properties; however, the latent variable z that is used to represent relational property in VRL-PGM is only independent of a but not b . Second, we note that the original problem is inherently undirected with no cause-effect relationship between a and b , whereas VRL-PGM is based a directed acyclic graph (DAG) that artiﬁcially introduces conditional dependency between a and b . However, we argue that the application of VRL does not require the true conditional dependency between ( a , b ) be known in advance only that it is maintained consistently throughout learning and inference, i.e.,
VRL can be applied in the same way to learn about the relational property between ( b , a ) , where we swap a and b .
We further remark that the unique optimization challenges—deterministic-mapping and information-shortcut—introduced by VRL (discussed in detail in Section 3.3) can be viewed as the consequence of the compromises we made with using DAG for VRL.
More speciﬁcally, deterministic-mapping can viewd as caused by the causal relationship VRL-PGM introduced between a and b whereas information-shortcut can be viewed as caused by the causal relationship VRL-PGM introduced between z and b
B E XPERIMENTAL SETUP
Recall that we adopted a two-dimensional latent variable z ∈ R 2 and let the prior p θ ( z ) be the bivariate normal distribution.
For binary valued data (e.g., MNIST dataset), we let the likelihood function p θ ( b ∈ B | a ( i ) ,
z
( i,l ) ) of VRL be a multivariate Bernoulli distribution whose probability parameters p ( i,l ) are computed from a given a ( i ) and
z ( i,l ) with an autoencoder-like neural network f dec θ
[Equation]
f enc
[Equation]
a ( i ) ) , z ( i,l ) (cid:1) : [Equation] where Conv ( · ) is a strided (stride 2) convolutional layer and Conv T ( · ) is a transposed convolutional layer.
We used batch-normalization after most layers and leaky rectiﬁed linear units (with leaky rate 0.01) as nonlinear activation function.
We let the approximated posterior q φ ( z | a (cid:48)
( i ) , b (cid:48)
( i ) ) of VRL be a bivariate Gaussian distribution with a diagonal covariance
N ( z ; µ ( i ) , ( σ ( i ) ) 2 I ) where µ ( i ) and σ ( i ) are the output of a neural network f qφ ( a (cid:48) ( i ) , b (cid:48)
( i ) ) :
[Equation]
[ a [Equation] ( i )
[Equation]
b (cid:48)
( i ) ]
[Equation] 3 x [Equation] x 8)
[Equation]
[Equation] x [Equation] x
[Equation] [Equation] [ µ ( i )
[Equation] σ ( i ) ] . We sampled from the posterior z ( i,l ) ∼ N ( z ; µ ( i ) , ( σ ( i ) ) 2 I ) using z ( i,l ) =
g ( (cid:15) ( i,l ) , a (cid:48) ( i ) , b (cid:48) ( i ) , φ ) = µ ( i ) + σ ( i ) (cid:12) (cid:15) ( i,l ) where (cid:15) ( i,l ) ∼ p ( (cid:15) )
=
N ( 0 , I ) and set L = 1 .
We used image rotation augmentations: D = { (rot( a , r ) , rot( b , r )) | r ∈ [0 , 360) } for constructing RPDA.
In this case, the learning objective (cid:101) L ( i ) RPDA in Eq.
(7) can further be derived as (see next section for the derivation): [Equation] + ( [Equation]
[Equation]
p (
[Equation]
(8) where ( µ ( i ) , σ ( i ) )
=
f qφ ( a (cid:48) ( i ) , b (cid:48) ( i ) ) ,
[Equation]
=
[Equation]
[Equation]
Parameters θ and φ were jointly trained to maximize Eq.
(8) using Adam optimizer (learning rate = 0.0001, β 1=0 .
9 , β 2=0 . 999 ) (Kingma & Ba, 2014); minibatches of size
M =100 were used.
In our VAE implementation we used the following network architecture for encoder, Enc ( a ( i ) , b ( i ) ) , and decoder, Dec ( z ( i,l ) ) :
[Equation] i
[Equation]
We used Adam optimizer with the same hyperparameter setup as before.
For InfoGAN implementation, we followed the setup described in Chen et al.
(2016) except that each datapoint consists a paired MNIST image ( a ( i ) , b ( i ) ) .
B. D ERIVATION OF TRAINING OBJECTIVE FUNCTION
Our derivation for Eq.
(8) largely follow the work of Kingma & Welling (2014); here for completeness we outline the key steps.
First, Eq.
(4) in Section 3.2 can equivalently be expressed as:
[Equation]
where z = g ( (cid:15) , a ( i ) , b ( i ) , φ ) , (cid:15) ∼ p ( (cid:15) ) , and g is some differentiable and invertible transformation.
We can approximate L ( i ) in Eq.
(9) with a Monte Carlo estimator (cid:101) L ( i ) :
[Equation]
where z ( i,l ) =
g ( (cid:15) ( i,l ) , a ( i ) , b ( i ) , φ ) and (cid:15)
( i,l ) ∼ p ( (cid:15) ) .
Based on the RPDA functions D = { d ( a , b ; r ) | r ∈ R } and its relation preserving assumption that q φ ( z | a ( i ) , b ( i ) )
=
q φ ( z | a (cid:48)
( i ) , b (cid:48)
( i ) ) for ( a (cid:48) ( i ) , b (cid:48)
( i ) )
=
d ( a ( i ) , b ( i ) ; r ) , ∀ r (see Section 3.3), we can express (cid:101) L ( i ) in Eq.
(10) equivalently as
[Equation]
where z
[Equation]
i
[Equation]
The ﬁrst term in Eq.
(11) is the KL-divergence from p θ ( z ) to q φ ( z | a (cid:48)
( i ) , b (cid:48)
( i ) ) , which can be computed analytically given that we assume p θ ( z ) =
N ( 0 , I ) and q φ ( z | a (cid:48) ( i ) , b (cid:48) ( i ) ) =
N ( z ; µ ( i ) , ( σ ( i ) ) [Equation] I [Equation]
[Equation]
The likelihood function p θ ( b ( i ) | a ( i ) , z
( i,l ) ) is deﬁned as a multivariate Bernoulli distribution whose probability parameters p ( i,l ) are computed from the neural network f dec θ (cid:0) f enc θ ( a ( i ) ) ,
z ( i,l ) (cid:1) and we have:
[Equation]
By substituting Eq.
(12) and (13) back in (11) and recall that z ( i,l )
=
g ( (cid:15) ( i,l ) , a (cid:48) ( i ) , b (cid:48) ( i ) , φ ) = µ ( i ) + σ ( i ) (cid:12) (cid:15) ( i,l ) , p ( (cid:15) )
=
N ( 0 , I ) , D = { (rot( a , r ) , rot( b , r )) | r ∈ [0 , 360) } , L =1 , we arrive at Eq.
C A DDITIONAL EXPERIMENTAL RESULTS C. MNIST DATASET
Here, we provide additional experimental results based on the paired MNIST dataset constructed in
Section 5.
First, to further test the robustness of the proposed method, we considered a scenarios where the underlying relational property is coupled with the absolute property.
An example dataset X 2 was constructed with the rotational relationship between each of the datapoint completely determined by its digit label:
X 2 = { ( a ( i ) =
x ( j,k ) , b ( i ) =
x ( j,l +1) )
| j ∼ U ([1 ..T ]) , k ∼ U ([1 .. 5]) , l = ( k + (cid:98) y
( j ) / 2 (cid:99) )
mod 5 } .
In this case, it is possible to infer the relational property solely based on the image representation of the digit (absolute property of a ( i ) ), for example, a ( i ) ∈
→ 0 ◦ (read: if a ( i ) is recognized as either digit 0 or 1 , the rotational relationship between ( a ( i ) , b ( i ) ) is 0 ◦ ) , a ( i ) ∈ [ ’2’, ’3’ ]
→ 72 ◦ , a ( i ) ∈ [ ’4’, ’5’ ] → 144 ◦ , a ( i ) ∈ [ ’6’, ’7’ ] → 216 ◦ , a ( i ) ∈ [ ’8’, ’9’ ] → 288 ◦ .
The question then arise: is VRL capable of learning a decoupled relational property even when it is coupled with the absolute property?
To test this idea, we trained VRL (with the same setup as that described in Appendix B) on X 2 but validated it on a hold-out dataset with a decoupled relational property (much like how X was constructed).
The resulting scatter plot is shown in Fig.
7, where we can see that the approximated posterior q φ ( z | a , b ) accurately cluster(discriminate) data with the same(different) rotational relationship together(apart).
This result shows that VRL was indeed capable of learning a decoupled relational property irrespective of the digit representation of a ( i ) .
If this were not the case, we would expect to see a scatter plot with heavily overlapped relationship labels since the validation dataset was constructed with random rotational relationships.
Examples
[Figure]
of images predicted by the learned likelihood function p θ ( b | a , z )
are shown in Fig.
8, where
Fig.
8a shows predicted images based on direct sampling in the latent space (shown in Fig. 7), and
Fig.
8b shows examples of relational mappings.
Figures 8a and 8b further corroborate our claim that
VRL is capable of learning a decoupled relational property and generalizing it to unseen data, e.g.,
VRL trained on X 2 learned to rotate any digit by any amount despite not having seen most of the digit-rotation pairs during training.
Note that this experiment is an example of deterministic-mapping discussed in Section 3.3, nevertheless VRL was able to utilize all paths in Fig.
2b effectively to learn about the data.
Next, we setup a more complex relational learning task that includes both rotational and resizing relationships between ( a ( i ) , b ( i ) ) : each MNIST image x ( i ) was augmented with ﬁve evenly-spaced rotations and three different resizing transformations to get { ( x ( i,j,k ) , y ( i ) )
| i ∈ [1 ..T ] , j ∈
[1 .. 5] , k ∈ [1 .. 3] } .
Examples of augmented images are shown in Fig.
9, where top, middle, and bottom row [Figure] [Equation] [Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation] [Equation] images are resized by a factor of × 0 . 66 , × 1 , and × 1 . 5 , respectively. We considered a relational learning task where each datapoint ( a ( i ) , b ( i ) ) has a decoupled rotational and/or resizing relational property; more speciﬁcally we constructed a dataset X 3 in the following way: {
[Equation]
a [Equation] [Equation] ) } where
V = [Equation] [1 , 2] , if m =[2 , 3] , if m =
Note that in this case, b ( i ) is either the same size as a ( i ) or is × 1 . 5 larger than a ( i ) , and there are a total of 10 different relationships between ( a ( i ) , b ( i ) ) in X 3 (combinations of 5 rotational andresizing transformations).
We trained VRL on X 3 with the same model setup as that described in
Appendix B and used the trained model to infer the relational property of a hold-out dataset.
The inference result is shown in Fig.
10, where we can see that the approximated posterior q φ ( z | a , b ) accurately cluster(discriminate) data with the same(different) relationship together(apart).
Examples of images predicted by the learned likelihood function p θ ( b | a , z ) are shown in Fig.
11, where
Fig.
11a shows predicted images based on direct sampling in the latent space (shown in Fig. 10), and Fig.
11b shows examples of relational mappings.
These results are consistent with the ﬁndings presented and discussed in Section 5.3.
Finally, we present an example with a continuous relational property.
Based on the MNIST dataset { ( x ( i ) , y ( i ) )
| i ∈ [1 ..T ] } , we constructed a paired dataset X 4 in the following way:
[Equation]
where rot( x ( j ) , r ( i ) ) rotates the image x ( j ) by r ( i ) ∈ [0 , 360) degrees.
In this case, b ( i ) is a random rotation of a ( i ) and there is a continuous (and decoupled) rotational relationship between ( a ( i ) , b ( i ) ) [Figure] z+0+ +144+ +288+0,x1. +72, x1.5 +144, x1.
+216, x1.5 +288, x1.
(cid:35) ( blue ):0 ◦ , (cid:79) :72 ◦ , +:144 ◦ ,
[Figure] [Figure] , ♦ : [Figure]
(cid:50) :
[Figure]
, × 1 . 5 , (cid:3) :72 ◦ , × 1 . 5 , (cid:1) :144 ◦ , × 1 . 5 , (cid:52) :216 ◦ , × 1 . 5 , (cid:35) ( cyan ):288 ◦ , × 1 . 5 ).
[Figure]
[Equation]
[Figure]
in X 4 .
We trained VRL on X 4 with the same model setup as that described in Appendix B and used the trained model to infer the relational property of a hold-out dataset.
A scatter plot of the relational property infered by the approximated posterior q φ ( z | a , b ) is shown in Fig.
12, and examples of images predicted by the learned likelihood function p θ ( b | a , z ) are shown in Fig.
13, where Fig. 13a shows predicted images based on direct sampling in the latent space (denoted by markers “ × ” in
Fig. 12), and Fig.
13b shows examples of relational mappings.
From Fig.
12 and 13 we can see that
VRL learned a decoupled relational property that encodes a continuous data (rotational) relationship; however, there is a small region in Fig.
12 with overlapping relational property that leads to an ambiguous interpretation of the rotational relationship ( 120 ◦ vs. 240 ◦ ).
This ambiguity is likely [Figure] [Figure] caused by compressing the continuous data (rotational) relationship down to a two-dimensional latent space, z ∈ R 2 , and motivates us to adopt a higher-dimensional latent space, e.g., z ∈ R 3 .
Figureshows inference result from repeating the previous experiment but with setting z ∈ R 3 ; we can see that VRL learned a three-dimensional relational property that unambiguously represents the underlying continuous data (rotational) relationship.
C. Y ALE FACE DATASET
Here we present another relational learning example based on the Yale face dataset (Belhumeur et al., 1997).
The Yale face dataset consists of 15 subjects, each with 8 facial expressions andlighting conﬁgurations.
We extracted three facial expressions (happy, surprised, sad) of each subject to form { x ( i,j ) | i ∈ [1 .. 15] , j ∈ [1 .. 3] } .
Examples of face images are shown in Fig.
15a, where we center-cropped, resized to 64 × 64 , and normalized pixel values to be within [0 , 5] .
We constructed a dataset X F where each datapoint ( a ( i ) , b ( i ) ) represents a subject with different facial expressions {+z 1 − 2 − 1 0 z− 2 − 1 0 1z− 2 −0 z 1 − 2 − 1 0 1z 2 − 2 − 1 0 z− 2 −0 z 1 − 2 − 1 0 1 2 z 2 − 2 − 1 0 1z− 2 −0 z 1 − 2 − 1 0 z− 2 − 1 0 1z− 2 −0 z− 2 − 1 0 1z 2 − 2 − 1 0 z− 2 −0 z− 2 − 1 02 z− 2 − 1 0 1z− 2 −
+} ( emotions ):
[Equation]
Our initial intention was to apply VRL to X F to learn about the “emotional change” between
[Figure] [Figure] ( a ( i ) , b ( i ) ) irrespective of the subject; however, because X F is an extremely limited dataset that consists of only 45 different images (15 subjects, each with 3 facial expressions), we settled for a less ambitious goal of learning an undirected emotional change, i.e., “happy → sad” =
“sad → happy” (simply denote as “happy-sad”).
In this case, there are three different emotional changes between ( a ( i ) , b ( i ) ) in X F : “happy-sad”, “happy-surprised”, “surprised-sad”.
To setup a VRL model for training, we adopted a two-dimensional latent variable z ∈ R 2 and let the prior p θ ( z ) be the[-z 1 − 2 − 1 0 z− 2 − 1 0 1z− 2 −0 z 1 − 2 − 1 0 1z 2 − 2 − 1 0 z− 2 −0 z 1 − 2 − 1 0 1 2 z 2 − 2 − 1 0 1z− 2 −0 z 1 − 2 − 1 0 z− 2 − 1 0 1z− 2 −0 z− 2 − 1 0 1z 2 − 2 − 1 0 z− 2 −0 z− 2 − 1 02 z− 2 − 1 0 1z− 2 −
 [Figure]-] bivariate normal distribution.
Since this is a real-valued dataset, we let p θ ( b ( i ) | a ( i ) , z
( i,l ) ) be a multivariate Gaussian distribution with a ﬁxed diagonal covariance
N ( b ( i ) ; µ ( i,l ) b , σ 2 I ) where σ = 0 .
1 and µ ( i,l ) b is computed from a given a ( i ) and
z ( i,l ) with an autoencoder-like neural network
[Equation]
and we have:
[Equation]
Again [Equation] we let the approximated posterior q φ ( z | a (cid:48)
( i ) , b (cid:48)
( i ) ) be a bivariate Gaussian distribution with a diagonal covariance
N ( z ; µ ( i ) , ( σ ( i ) ) 2 I ) where µ ( i ) and σ ( i ) are the output of a neural network f
[Equation]
[Equation]
Next, based on the premise that we are only interested in learning an undirected emotional change
[Equation]
we have p θ ( z | a , b ) = p θ ( z | b , a ) and this prompted us to construct RPDA functions D with random image rotation and swapping operations: D = { swap (rot( a , r ) , rot( b , r )) | r ∈ [0 , 360) } where
[Equation]
Finally, we combine the above settings with z ( i,l ) = µ ( i ) + σ ( i ) (cid:12) (cid:15)
( i,l ) , p ( (cid:15) )
=
N ( 0 , I ) , L = 1 , and substituting Eq.
(12) and (14) back in (11) to derive the following lower bound estimator:
[Equation]
where ( µ ( i ) , σ ( i ) )
=
f qφ ( a (cid:48) ( i ) , b (cid:48) ( i ) ) ,
[Equation]
The rest of the training setup remains the same as that described in
Appendix B. We trained VRL on X F and then used the approximated posterior q φ ( z | a , b ) to infer the relational property of X F .
The inference result is shown in Fig.
15b, where we can see that VRL learned a relational property that accurately differentiates emotional changes irrespective of the subject.
It is worth noting that
VRL does not learn the facial expression for each of ( a ( i ) , b ( i ) )
independently and then classify them based on their difference, but instead directly learns the emotional relationship between ( a ( i ) , b ( i ) ) .
D A BLATION STUDY
In this work, we introduced relation-preserving data augmentation (RPDA) as a practical data augmentation strategy for overcoming the information-shortcut problem—an unique overﬁtting problem introduced by VRL (see Section 3.3).
In order to evaluate the contribution of RPDA, we performed ablation study on the paired MNIST dataset X constructed in see Section 5.1.
First, recall that in Section 5.2 we constructed RPDA functions D with image rotation augmentations and optimized the following lower bound estimator (cf.
Eq.
[Equation]
where
[Equation]
In the ﬁrst ablation study, we experimented with removing RPDA from the VRL training, which amounts to optimizing with the original lower bound estimator (cid:101) L ( i ) in Eq.
With the rest of the training setup remained unchanged (see Appendix B), we trained VRL on X without RPDA and used the trained model to infer the relational property of a hold-out dataset; the inference result is shown in Fig. 16a.
In the second ablation study, we repeated the VRL training, but instead of following the proposed RPDA strategy, we applied RPDA functions D in a conventional data augmentation routine.
More speciﬁcally, we optimized the lower bound (cid:101) L ( i ) in Eq.
(6) over a minibatch of data augmented with D , X (cid:48)
M = { ( a (cid:48) ( i ) , b (cid:48) ( i ) ) | i ∈ [1 ..M ] } , which leads to optimizing with the following lower bound estimator:
[Equation]
where
[Equation]
With the rest of the training setup remained unchanged (see Appendix B), we trained VRL on X with maximizing (cid:101) L ( i ) DA and used the trained model to infer the relational property of a hold-out dataset; the inference result is shown in Fig. 16b.
Comparing the results from both ablation studies, shown in −−  2 z−−   z+
+ + + + (a) −−  2 z−−   z+
+ + + + (b)
[Equation] [Equation]
Fig.
16, with inference result from VRL training with RPDA, shown in Fig.
4a, we can make the following observations: ﬁrst, RPDA is a critical component necessary for VRL to learn a meaningful and decoupled relational property, especially when ﬂexible function approximations such as deep neural networks are used; second, by looking at the difference between (cid:101) L ( i ) DA and (cid:101)
L ( i ) RPDA (observe that (cid:101) L ( i ) DA applies RPDA functions D to every term in Eq.
(17), whereas (cid:101) L ( i ) RPDA applies D only to selected terms in Eq.
(16) according to the RPDA strategy), we can draw the conclusion that the improvements brought by RPDA (and its key innovation) comes not from what data augmentation functions are applied but how they are applied.