L IKELIHOOD - BASED P ERMUTATION I NVARIANT L OSS F UNCTION FOR P ROBABILITY D ISTRIBUTIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation.
Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any additional sequential algorithm.
The proposed method, Set Cross Entropy, has a natural information-theoretic interpretation and is related to the metrics deﬁned for sets.
We evaluate the proposed approach in two object reconstruction tasks and a rule learning task.
I NTRODUCTION
Sets are fundamental mathematical objects which appear frequently in the real-world dataset.
However, there are only a handful of studies on learning a set representation in the machine learning literature.
In this study, we propose a new objective function called Set Cross Entropy (SCE) to address the permutation invariant set generation.
SCE measures the cross entropy between two sets that consists of multiple elements, where each element is represented as a multi-dimensional probability distribution in [0 , 1] ⊂ R (a closed set of reals between 0,1).
SCE is invariant to the object permutation, therefore does not distinguish two vector representations of a set with the different ordering.
The SCE is simple enough to ﬁt in one line and can be naturally interpreted as a formulation of the log-likelihood maximization between two sets derived from a logical statement.
The SCE loss trains a neural network in a permutation-invariant manner, and the network learns to output a set.
Importantly, this is not to say that the neural network learns to represent a function that is permutation-invariant with regard to the input.
The key difference in our approach is that we allow the network to output a vector representation of a set that may have a different ordering than the examples used during the training.
In contrast, previous studies focus on learning a function that returns the same output for the different permutations of the input elements.
Such scenarios assume that an output value at some index is matched against the target value at the same index.
This characteristic is crucial in the tasks where the objects included in the supervised signals (training examples for the output) do not have any meaningful ordering.
For example, in the logic rule learning tasks, a ﬁrst-order logic horn clause does not care about the ordering inside the rule body since logical conjunctions are invariant to permutations, e.g. father ( c , f ) ← ( parent ( c , f ) ∧ male ( f )) and father ( c , f ) ← (
male ( f ) ∧ parent ( c , f )) are equivalent.
To apply our approach, no special engineering of the network topology is required other than the standard hyperparameter tuning.
The only requirement is that the target output examples are the probability vectors in [0 , 1] N × F , which is easily addressed by an appropriate feature engineering including autoencoders with softmax or sigmoid latent activation.
We demonstrate the effectiveness of our approach in two object-set reconstruction tasks and the supervised theory learning tasks that learn to perform the backward chaining of the horn clauses.
In particular, we show that the SCE objective is superior to the training using the other set distance metrics, including Hausdorff and set average (Chamfer) distances.
B ACKGROUNDS AND R ELATED W ORK 2. L EARNING A S ET R EPRESENTATION
Previous studies try to discover the appropriate structure for the neural networks that can represent a set.
Notable recent work includes permutation-equivariant / invariant layers that addresses the permutation in the input (Guttenberg et al., 2016;
Ravanbakhsh et al., 2016; Zaheer et al., 2017).
Let X be a vector representation of a set { x 1 , . . .
, x n } and π be an arbitrary permutation function for a sequence.
A function f ( X ) is permutation invariant when ∀ π ; f ( X ) =
f ( π ( X )) .
Zaheer et al.
(2017) showed that functions are permutation-invariant iff it can be decomposed into a form
[Equation]
where ρ, φ are the appropriate mapping function.
However, as mentioned in the introduction, the aim of these layers is to learn the functions that are permutation-invariant with regard to the input permutation, and not to reconstruct a set in a permutation-invariant manner (i.e. ignoring the ordering).
In other words, permutationequivariant/invariant layers are only capable of encoding a set.
Probst (2018) recently proposed a method dubbed as “Set Autoencoder”.
It additionally learns a permutation matrix that is applied before the output so that the output matches the target.
The target for the permutation matrix is generated by a Gale-Shapley greedy stable matching algorithm, which requires O ( n 2 ) runtime.
The output is compared against the training example with a conventional loss function such as binary cross entropy or mean squared error, which requires the ﬁnal output to have the same ordering as the target.
Therefore, this work tries to learn the set as well as the ordering between the elements, which is conceptually different from learning to reconstruct a set while ignoring the ordering.
Another line of related work utilizes Sinkhorn iterations (Adams & Zemel, 2011;
Santa Cruz et al., 2017; Mena et al., 2018)
in order to directly learn the permutations.
Again, these work assumes that the output is generated in a speciﬁc order (e.g. a sorting task), which does not align with the concept of the set reconstruction.
2. S ET D ISTANCE
Set distances / metrics are the binary functions that satisfy the metric axioms.
They have been utilized for measuring the visual object matching or for feature selection (Huttenlocher et al., 1993;
Dubuisson & Jain, 1994; Piramuthu, 1999).
Note that, however, in this work, we use the informal usage of the terms “distance” or “metric” for any non-negative binary functions that may not satisfy the metric axioms.
There are several variants of set distances.
Hausdorff distance between sets (Huttenlocher et al., 1993) is a function that satisﬁes the metric axiom.
For two sets X and Y , the directed Hausdorff distance with an element-wise distance d ( x, y ) is deﬁned as follows:
[Equation]
The element-wise distance d is Euclidean distance or Hamming distance, for example, depending on the target domain.
Set average (pseudo) distance (Dubuisson & Jain, 1994, Eq.(6)), also known as Chamfer distance, is a modiﬁcation of the original Hausdorff distance which aggregates the element-wise distances by summation.
The directed version is deﬁned as follows:
[Equation]
Set average distance has been used for image matching, as well as to autoencode the 3D point clouds in the euclidean space for shape matching (Zhu et al., 2016).
S ET C ROSS E NTROPY
Inspired by the various set distances, we propose a straightforward formulation of likelihood maximization between two sets of probability distributions.
In what follows, we deﬁne the cross entropy between two sets X, Y ∈ [0 , 1] N × F , where [0 , 1] is a closed set of reals between 0 and 1.
Let X = (cid:8) X (1) , X (2) , . . .
(cid:9) be the training dataset, and Y be the output matrix of a neural network.
Assume that each X ∈ X consists of N elements where each element is represented by F features, i.e. X = { x 1 . . .
x N } , x i ∈ R F .
We further assume that x i ∈ [0 , 1] F by a suitable transformation, which can be done by the feature learning with sigmoid activation added to the latent layer.
The set X actually takes the vector representation, which essentially makes X ∈ [0 , 1] N × F .
In this paper, we focus on the binomial distribution.
However, the proposed method naturally extends to the multinomial case.
For simplicity, we assume that the number of elements in the set X and Y is known and ﬁxed to
N .
Therefore Y is also a matrix in [0 , 1] N × F .
Furthermore, we assume that X is preprocessed and contains no duplicated elements.
In practice, if | X | varies across the dataset, it sufﬁces to take N max = max X ∈X | X | , the largest number of elements in X across the dataset X , and add the dummy, distinct objects d 0 . . .
d N max to ﬁll in the blanks.
For example, when there are N objects of F features and we want to normalize the size of the set to N (cid:48) ( > N ) , one way is to add an additional axis to the feature vector ( F +features) where the additional F + 1 -th feature is 0 for the real data and 1 for the dummy data, and the additional N (cid:48) − N objects are generated in an arbitrary way (e.g. as a binary sequence
For measuring the similarity between the two probability vectors x, y ∈ [0 , 1] F , the natural loss function would be the cross entropy H( x, y ) or, equivalently, the negative log likelihood.
[Equation]
However, applying it directly to the matrices X, Y unnecessarily limits the global optima of this loss because it does not consider the permutations between N objects, e.g., for X
=
[ o 1 , o 2 , o 3 ] ,
Y =
[ o 2 , o 3 , o 1 ] is not the global minima of H( X, Y ) .
Previous approach (Probst, 2018) tried to solve this problem by learning an additional permutation matrix that “ﬁxes” the order, basically requiring to memorize the ordering.
We take a different approach of directly ﬁxing this loss function.
The target objective is to maximize the probability of two sets being equal, thus ideally, at the global minima, two sets X and Y should be equal.
Equivalence of two sets is deﬁned as:
[Equation]
However, under the assumption that | X | = | Y | = N and X contains N distinct elements (no duplicates),
X ⊆ Y is a sufﬁcient condition for X = Y .
(Proof: If X ⊆ Y and X (cid:54)⊇ Y , there are some y (cid:48)
∈ Y
such that y (cid:48) (cid:54)∈ X .
Since N distinct elements in X are also included in Y , y (cid:48) becomes Y ’s N + 1 -th element, which contradicts | Y | = N .
Note that this proof did not depend on the distinctness of Y ’s elements.)
Under this condition, therefore,
[Equation]
We now translate this logical formula into the corresponding log likelihood as follows:
[Equation]
∵
[Equation]
x
[Equation]
y i are mutually exclusive
[Equation]
This Set Cross Entropy has the following characteristics:
First, compared to the original cross entropy loss, whose global minima is limited to the data point that preserves the same ordering of the elements, SCE increases the number of global minima exponentially by making every permutations of the point also the global minima.
Next, notice that logsumexp is a smooth upper approximation of the maximum, therefore SH( X, Y ) is upper-bounded by the set average equivalent,
[Equation]
Intuitively, this is because Eq.9 returns a value which does not account for the possibility that the current closest y = arg min y H( x, y ) of x may not converge to the x in the future during the training.
We illustrate this by comparing two examples: Let X
= { [0 , 1] , [0 , 0] } , Y 1 = { [0 . 1 , 0 . 5] , [0 . 1 , 0 . 5] } and Y 2 = { [0 . 1 , 0 . 5] , [0 . 9 , 0 . 5] } .
The set cross entropy Eq.8 reports the smaller loss for SH( X, Y 1 ) =
− log 0 .
81 ≈ 0 .
09 than for SH( X, Y 2 )
= − log 0 .
25 ≈ 0 .
This is reasonable because the global minima is given when the ﬁrst axis of both y s are 0 — Y 2 should be more penalized than Y 1 for the 0 .
9 in the second element.
In contrast, Eq.9 considers only the closest element ( arg min
y ∈
Y H( x, y ) =
[0 . 1 , 0 . 5] ) for each x , therefore returns the same loss = − log 0 .
≈ 0 .
69 for both cases, ignoring [0 . 9 , 0 . 5] completely.
In fact, Eq.9 has zero gradient at Y = { [0 , 0 . 5] , [ y, 0 . 5] } for any y ∈ [0 , 1] .
Furthermore, the following inequality suggests that the traditional cross entropy between the matrices X and Y is an even looser upper bound of Eq.9.
Here, x i , y i are the i -th element of the vector representation of X and Y , respectively:
[Equation]
This gives a natural interpretation that ignoring the permutation reduces the cross entropy.
E VALUATION 4. O BJECT S ET R ECONSTRUCTION
The purpose of the task is to obtain the latent representation of a set of objects and reconstruct them, where each object is represented as a feature vector.
We prepared two datasets originating from classical AI domains: Sliding tile puzzle (8-puzzle) and Blocksworld.
Learning to reason about the object-based, set representation of the environment is crucial in the robotic systems that continuously receive the list of visible objects from the visual perception module (e.g. Redmon et al.
(2016, YOLO)).
In a real-world systems, appropriate handling of the set is necessary because it is unnatural to assume that the objects in the environments are always reported in the same order.
In particular, the objects even in the same environment state may be reported in various orders if multiple such modules are running in parallel in an asynchronous manner.
In this experiment, we show that the permutation invariant loss function like SCE is necessary for learning to reconstruct a set in such a scenario.
In this setting, a network is required to reconstruct a set from a single latent representation, while the objects as the target output may be randomly reordered each time the same set is observed and presented to the neural network.
8 P UZZLE
Each feature vector as an object consists of 15 features, 9 of which represent the tile number (object
ID) and the remaining 6 represent the coordinates.
Each data point has 9 such vectors, corresponding to the 9 objects in a single tile conﬁguration.
The entire state space of the puzzle is 362880 states.
We generated 5000 states and used the 4500 states as the training set.
[Equation]
We prepared an autoencoder with the permutation invariant layers (Zaheer et al., 2017) as the encoder and the fully-connected layers as the decoder.
Since it uses a permutation-invariant encoder, the latent space is already guaranteed to learn a representation that is invariant to the input ordering.
The key question here is then whether they can be robustly trained against the random permutations in the training examples for the output.
We tested the reconstruction ability in four scenarios: (1)
In the ﬁrst scenario, the dataset is provided in a standard manner.
In the second scenario, we augment the input dataset by repeating the elements 5 times and randomly reorder the object vectors in each set.
The randomized dataset is used as the input to the network, while the target output is still the original dataset (repeatedtimes, without reordering).
The purpose of this experiment is to verify the claim of the Deep Set (Zaheer et al., 2017) that it is able to handle the input in a permutation invariant manner.
In order to compensate the datasize difference, the maximum training epoch is reduced by 1/5 times compared to the ﬁrst scenario.
In the third scenario, we apply the similar operation to the target output of the network.
Essentially we always feed the input in the same ﬁxed order while forcing it to learn from the randomized target output.
Each time the same data is presented, the target output has the different ordering while the input has the ﬁxed ordering.
Therefore, the training should be performed in such a way that the ordering in the output is properly ignored.
(4) Finally, in the fourth scenario, the ordering in both the input and the output are randomized.
We trained the same network with four different loss functions, (a) the traditional cross entropy H , (b) Set Cross Entropy SH , (c) directed set average of the cross entropy A 1H and (d) the directed
Hausdorff measure of the cross entropy H 1H , resulting in 16 training scenarios in total.
We performed the same experiment 10 times and took the statistics.
The purpose of this is to address the potential concern about the stability of the training.
We kept the same set of training/testing data, and the only difference between the runs is the random seed.
We ﬁrst measured the Set Cross Entropy value between the test dataset and its reconstruction in the above 16 scenarios.
Table 1 shows the results.
The training with the standard cross entropy loss ( H ) succeeds in cases (1,2) while failed in cases (3,4) .
The case (2) reproduces the claim in (Zaheer et al., 2017) that it encodes the input in an permutation-invariant manner, while it failed in the latter cases because the training is not permutation-invariant with regard to the output.
In contrast, the training with the Set Cross Entropy loss succeeds in all cases.
This shows that the permutation-invariant loss function is necessary for training a network with a dataset consisting of sets.
The training with set average distance A 1H also reduces the Set Cross Entropy because it is an upper-bound approximation of the Set Cross Entropy.
However, in one of the 10 runs, A 1H did not converge, showing that the A1H (baseline) could be unstable, possibly due to the issue explained in the example at the end of section 3.
In contrast, the training with Hausdorff distance failed to learn the representation at all.
[Table] [Table]
measured
[Table]
SH )
[Table]
test errors out of 10 runs.
Best results in bold .
SH and A 1H both succeeded to achieve a good log likelihood sufﬁciently often.
The set average A 1H however suffered from a divergence in one training instance, showing its potential instability.
The traditional cross entropy
H fails to converge when the output is presented in a different order in each iteration.
Hausdorff distance failed to converge in all cases.
We next measured the rate of the successful reconstruction among the entire dataset.
The “successful reconstruction” is deﬁned as follows:
Recall that every data point is a discrete binary vector in the 8-Puzzle dataset while the output of the network is a continuous N × F matrix of reals betweenand 1.
Therefore, we round the output of the network to 0 / 1 and directly compare the result with the input.
If every object vector in a set is matched by some of the output object vector, then it is counted as a success.
Similar results were obtained in Table 6: SH and A 1H both succeeded to achieve a high success rate, while other two metrics completely failed.
(We rerun the experiment, therefore the divergence of A 1H in the previous experiment did not happen this time.)
Finally, to address the claim that the network is able to learn from the dataset with the variable set size, we performed an experiment which applies the dummy-vector scheme (Sec. 3).
In this experiment, we modiﬁed the dataset to model such a scenario by randomly dropping one to ﬁve elements out of 9 elements.
The maximum number of elements is 9.
The dropping scheme is speciﬁed as follows:
Out of the 5000 states generated in total (including the training / testing dataset), approximately half of the states have 9 tiles, 1 / 4 of the states have 8 tiles, ... and 1 / 2 5 of the states havetiles.
The elements to drop are selected randomly.
The results in Table 3 shows that the training with our proposed SH loss function achieves the best success ratio for the reconstruction.
The reconstruction includes the dummy vectors, indicating that the network is able to represent not only the elements in the set but also the number of the missing elements.
B LOCKSWORLD
In order to test the reconstruction ability for the more complex feature vectors, we prepared a photorealistic Blocksworld dataset (Fig. 2) which contains the blocks world states rendered by Blender 3D engine.
There are several cylinders or cubes of various colors and sizes and two surface materials (Metal/Rubber) stacked on the ﬂoor, just like in the usual STRIPS (McDermott, 2000) Blocksworld domain.
In this domain, three actions are performed: move a block onto another stack or on the
[Table]
Best
Worst
Target
[Table]
Fixed
Random
Fixed
Random
Input
[Table]
Fixed
Random
Fixed
Random
Fixed
Random
Fixed
Random
H 0.
SH 1.
A 1H 1.
H 1H 0.
Median
[Table]
Fixed
[Table]
Fixed
[Table]
Fixed
Random
Fixed
[Table]
Fixed
[Table]
Fixed
Random
H 0.
SH 1.
A 1H 1.
H 1H 0.
[Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
ﬂoor, and polish / unpolish a block i.e. change the surface of a block from Metal to Rubber or vice versa.
All actions are applicable only when the block is on top of a stack or on the ﬂoor.
The latter actions allow changes in the non-coordinate features of the object vectors.
[Figure]
Pre
Suc
The dataset generator produces a 300x200 RGB image and a state description which contains the bounding boxes (bbox) of the objects.
Extracting these bboxes is a object recognition task we do not address in this paper, and ideally, should be performed by a system like YOLO (Redmon et al., 2016).
We resized the extracted image patches in the bboxes to 32x32 RGB, compressed it into a feature vector of 1024 dimensions with a convolutional autoencoder, then concatenated it with the bbox ( x 1 , y 1 , x 2 , y 2 ) which is discretized by 5 pixels and encoded as 1-hot vectors (60/40 categories for x / y -axes), resulting in 1224 features per object.
The generator is able to enumerate all possible states (80640 states for 5 blocks and 3 stacks).
We used 2250 states as the training set and 250 states as the test set.
We also veriﬁed the results qualitatively.
Some reconstruction results are visualized in Fig.
These visualizations are generated by pasting the image patches decoded from the ﬁrst 1024 axes of the reconstructed 1224-D feature vectors in a position speciﬁed by the reconstructed bounding box in the last 200 axes.
[Figure]
Results in Table 4 shows that the training with SH and A 1H achieved a better test error compared to the other metrics.
[Table]
In Table 5, as another metric with the more intuitive sense, we measure the difference between the visualization results of the input and the output (as shown in Fig. 3) by the Root Mean Squared Error of the pixel values averaged over RGB, pixels and the dataset.
Each pixel is represented in the [0 , 1] range (closed set of reals between 0 and 1), thus the 0.1 on the table means that pixels differ by 0.
on average.
[Table]
the
[Table]
4. R ULE L EARNING ILP T ASKS
The purpose of this task is to learn to generate the prerequisites (body) of the ﬁrst-order-logic horn clauses from the head of the clause.
Unlike the previous tasks, this task is not an autoencoding task.
The bodies are considered as a set because the order of the terms inside a body does not matter for the clause to be satisﬁed.
The main purpose of this experiment is to show the effectiveness of our approach on set generation, not to demonstrate a more general neural theorem proving system.
An interesting avenue of future work is to see how our approach can help the existing work on neural theorem proving.
We used a Countries dataset (Bouchard et al., 2015) that contains 163 countries and trained the models for n -hop neighbor relations.
For example, for n = 2 , given a head neighbor2 ( austria , germany , belgium ) as an input, the task is to predict the body { neighborOf ( austria , germany ) , neighborOf ( germany , belgium ) } , which is a set of two terms.
This is a weaker form of a more general backward chaining used in Neural Theorem Proving (Rockt¨aschel & Riedel, 2017) because the output does not contain free variables.
In the n -neighbor scenario, the input is a 2 + 163( n + 1) -dimensional vector, which consists of a one-hot label of 2 categories for the predicate of the head, and n + 1 one-hot labels of 163 categories for the arguments of the head.
For example, a head neighbor2 ( austria , germany , belgium ) spends 2 dimensions for identifying the predicate neighbor2 , and three 1-hot vectors of 163 categories for representing austria , germany , belgium .
The output is a n × 328 matrix, where each row represents a binary predicate (328 = 2 + 2 · 163) .
This is again because the answer is { neighborOf ( austria , germany ) , neighborOf ( germany , belgium ) } : There are 2 elements in the set, thus the output is a 2 × 328 matrix.
Each element uses 2 dimensions for identifying the predicate head neighborOf and two 1-hot vectors of 163 categories for the arguments (e.g. austria and germany ).
We trained the network with the neighbor - n datasets ranging from n = 2 to n = 5 (see the result table for the detailed domain characteristics).
The softmax output of the network is parsed back to the symbolic representation by selecting the index that gives the maximum probability, then compared against the test examples as a set.
We counted the ratio of the clauses across the test set where every body term matches against one of the output terms.
The output data (body terms) may have an arbitrary ordering, and we have another variant similar to the previous experiment: In the randomized body order dataset, the dataset is repeated 5 times, while the ordering of the terms inside each body is randomly shufﬂed.
Table 6 shows that the network with Set Cross Entropy achieved the best accuracy, set average generally comes in the second and the traditional cross entropy struggles.
This trend was observed not only in the the randomized-body-ordering dataset, which observes the same body in a different order in each iteration, but also in the ﬁxed-body-ordering dataset.
This shows that the Set Cross
Entropy relaxes the search space by adding more global minima and making the training easier.
D ISCUSSION
Vinyals et al.
(2016) repeatedly emphasized the advantage of limiting the possible equivalence classes of the outputs by engineering the training data for solving the combinatorial problems.
For example, they pre-sorted the training example for the Delaunay triangulation (set of triangles) by the lexicographical order and trained an LSTM model with the standard cross entropy (Vinyals et al., 2015).
However, this is an ad-hoc method that depends on the particular domain knowledge and, as we have shown, the difﬁculty of learning such an output was caused by the loss function that considers the ordering.
Moreover, we showed that the standard cross entropy and the set average metrics are the less tighter upper bound of the proposed Set Cross Entropy and also that it empirically outperforms the standard cross entropy in the theory learning task, even if a speciﬁc ordering is imposed on the output.
One limitation of the current approach is that the set cross entropy contains a double-loop, therefore takes O ( N 2 ) runtime for a set of N objects.
However, unlike the algorithm proposed in Probst (2018), which uses a sequential Gale-Shapley algorithm which also uses O ( N 2 ) runtime, our loss function can be efﬁciently implemented on GPUs because it consists of a simple combination of logsumexp and summation.
Still, improving the runtime complexity is an important direction for future work because the other set reconstruction tasks, including 3D point clouds datasets like Shapenet (Chang et al., 2015), may contain a much larger number of elements in each set.
A promising candidate for tacking this difﬁculty is to combine Set Cross Entropy with Approximate k -Nearest Neighbor (Indyk & Motwani, 1998) methods, especially the Locality Sensitive Hashing (Wang et al., 2016, LSH).
LSH can preprocess and divide the target output X into the subsets within a certain radius and we can limit the inner loop to each subset.
The resulting method can be seen as the midpoint of Set Cross Entropy and set average because set average (Eq.9) is the special case of this extension that uses the nearest neighbor ( min y ∈
Y H ( x, y ) )
and worked reasonably well in the tasks evaluated in this paper.
The main obstacle for this approach would be to extend Set Cross Entropy to a metric variant that satisﬁes the metric axioms (non-negativity, identity, symmetry, the triangular inequality) that is required for LSH methods in general.
One candidate in this direction is a variant of Jensen-Shannon divergence called S2JSD (Endres & Schindelin, 2003), which satisﬁes the metric axioms and has a LSH method (Mao et al., 2017).
Another direction for future work is to use the Long Short Term Memory (Hochreiter & Schmidhuber, 1997) for handling the sets without imposing the shared upper bound on the number of elements in a set, which has been already explored in the literature (Vinyals et al., 2015; 2016).
Since our approach is agnostic to the type of the neural network, they are orthogonal to our approach.
C ONCLUSION
In this paper, we proposed Set Cross Entropy, a measure that models the likelihood between the sets of probability distributions.
When the output of the neural network model can be naturally regarded as a set, Set Cross Entropy is able to relax the search space by making the permutations of a global minima also the global minima, and makes the training easier.
This is in contrast to the existing approaches that try to correct the ordering of the output by learning a permutation matrix, or an
[Table]
H 0.
SH 0.
A 1H 0.
H 1H 0.
n = 4 , neighbor4 ( a, b, c, d, e ) :- . . .
Dataset
[Table] [Table]
clauses
[Table]
1000 clauses.
[Table]
H
[Table] [Table]
SH 0.
[Table] [Table]
[Table] [Table]
[Table]
1H 0.
[Table] [Table]
1H 0.
n = 5 , neighbor5 ( a, b, c, d, e, f ) :- . . .
Dataset:
[Table] [Table]
2250 clauses; Test
[Table]
250 clauses.
[Table]
H 0.
[Table] [Table]
SH 0.
A 1H 0.
[Table]
n = 5 , neighbor5 ( a, b, c, d, e, f ) :
[Table]
Dataset:
[Table]
ground clauses; Training: 9000 clauses; Test: 1000 clauses.
[Table]
ad-hoc methods that reorder the dataset using the domain-speciﬁc expert knowledge.
Training based on the Set Cross Entropy is also robust against the dataset which contains vectors whose internal ordering may change time to time in an arbitrary manner.
We demonstrated the effectiveness of the approach by comparing Set Cross Entropy against the normal cross entropy, as well as the other set-based metrics such as Hausdorff distance or set average (Chamfer) distance.
set average distance was shown to upper-bound Set Cross Entropy, and while it performed comparably well in the object reconstruction task, it was outperformed by Set Cross Entropy in the rule learning task.
Training a neural network with Hausdorff distance turned out to be particularly hard, and it failed in many scenarios, showing that it is not suitable as a loss function for the set reconstruction tasks considered in this paper
A PPENDIX 6. N ETWORK M ODEL FOR 8 P UZZLE
As mentioned in the earlier sections the network has a permutation invariant encoder and the fullyconnected decoder.
The input to the network is a 9 × 15 matrix, where the ﬁrst dimension represents the objects and the second dimension represents the features of each object.
The encoder has two 1D convolution layers of 1000 neurons with ﬁlter size 1, modeling the element-wise network ρ .
The output of these layers is then aggregated by taking the sum of the ﬁrst dimension.
The result is then fed to two another fully-connected layers of width which maps to the latent layer of 100 neurons.
All encoder layers are activated by
ReLU.
The latent representation is regularized and activated by Gumbel-Softmax Maddison et al.
Jang et al.
(2017) as the input is a categorical model.
The decoder consists of three fully-connected layers with dropout and batch normalization as shown below: fc(1000), relu, batchnorm, dropout(0.5), fc(1000), relu, batchnorm, dropout(0.5), dense(135), reshape( 9 × 15 )
The last layer is then split into 9 × 9 , 9 × 3 , 9 × 3 matrices and separately activated by softmax, reﬂecting the input dataset (Fig. 1).
6. N ETWORK M ODEL FOR B LOCKSWOLRD
The same network as the 8-puzzle was used, except that the input and the output is a 5 × 1224 matrix.
The activations of the last layer is different:
The ﬁrst 1024 features are activated by a sigmoid function, while the 200 features are divided into 40, 60, 40, 60 dimensions (for the one-hot bounding box information) and are separately activated by softmax.
6. F EATURE E XTRACTION FOR B LOCKSWORLD
The 32x32 RGB image patches in the Blocksworld states are compressed into the feature vectors that are later used as the input.
The image features are learned by a convolutional autoencoder depicted in Fig.and Fig.
Input( 32 × 32 × 3 ),
GaussianNoise(0.1), conv2d(ﬁlter=16,kernel= 3 × 3 ,), relu,
MaxPooling2d( 2 × 2 ), conv2d(ﬁlter=16, kernel= 3 × 3 ,), relu,
MaxPooling2d( 2 × 2 ), conv2d(ﬁlter=16, kernel= 3 × 3 ,), sigmoid
Input( 8 × 8 × 16 ), conv2d(ﬁlter=16, kernel= 3 × 3 ,), relu,
UpSampling2d( 2 × 2 ), conv2d(ﬁlter=16, kernel= 3 × 3 ,), relu,
UpSampling2d( 2 × 2 ), conv2d(ﬁlter=16, kernel= 3 × 3 ,), relu, fc(
[Equation]
), sigmoid, reshape(
[Equation] [Equation]
6. N ETWORK M ODEL FOR R ULE L EARNING
We removed the encoder and the latent layer from the above models, and connected the input directly to the decoder.
While the decoder has the same types of layers, the width is shrinked to 400.
In the nneighbor scenario, the input is a 2 + 163( n + 1) vector, which consists of a one-hot label of 2 categories for the predicate of the head, and n + 1 one-hot labels of 163 categories for the arguments of the head.
163 categories corresponds to the number of countries in the Countries dataset (Bouchard et al., 2015).
The output is a [ n, 328] matrix, where each row represents a binary predicate (328 = 2 + 2 · 163) .
6. E XAMPLE A PPLICATION OF THE P ERMUTATION -I NVARIANT R EPRESENTATION & R ECONSTRUCTION
To address the practical utility of “set reconstruction” or “set autoencoding”, we added a new experiment.
We modiﬁed Latplan (Asai & Fukunaga, 2018) neural-symbolic classical planning system, a system that operates on a discrete symbolic latent space of the real-valued inputs and runs
Dijkstra’s/A* search using a state-of-the-art symbolic classical planning solver.
We modiﬁed Latplan to take the set-of-object-featurevector input rather than images.
It is a high-level task planner (unlike motion planning / actuator control) that has implications on robotic systems, which perceives a set of inputs already preprocessed by the external system.
For example, the image input is ﬁrst fed into Object Recognition system (e.g. YOLO, (Redmon et al., 2016)) and the planner receives a set of feature vectors extracted from the image patches segmented from the raw image, rather than feeding the image input directly to the planning system.
Latplan system learns the binary latent space of an arbitrary raw input (e.g. images) with a GumbelSoftmax variational autoencoder, learns a discrete state space from the transition examples, and runs a symbolic, systematic search algorithm such as Dijkstra or A* search which guarantee the optimality of the solution.
Unlike RL-based planning systems, the search agent does not contain the learning aspects.
The discrete plan in the latent space is mapped back to the raw image visualization of the plan execution, which requires the reconstruction capability of (V)AE.
A similar system replacing Gumbel Softmax VAE with Causal InfoGAN was later proposed (Kurutach et al., 2018).
We replaced Latplan’s Gumbel-Softmax VAE with our autoencoder used in the 8-Puzzle and the
Blocksworld experiments (Appendix, Sec.
6.1,Sec. 6.2).
Our autoencoder also uses Gumbel Softmax in the latent layer, but it uses (Zaheer et al., 2017) encoder and is trained with Set Cross Entropy.
When the network learned the representation, it guarantees that the planner ﬁnds a solution because the search algorithm being used (e.g. Dijkstra) is a complete, systematic, symbolic search algorithm, which guarantees to ﬁnd a solution whenever it is reachable in the state space.
If the network cannot learn the permutation-invariant representation, the system cannot solve the problem and/or return the humancomprehensive visualization.
This makes the speciﬁc permutation-invariant representation using (Zaheer et al., 2017) and the proposed Set Cross Entropy necessary when the input is given as a set of future vectors.
8 P UZZLE
First, the training was performed on a dataset in which the object vector ordering is randomized.
The autoencoder compresses the 15 × 9 = 135 -bit binary representation (object vectors) into a permutationinvariant 100-bit discrete latent binary representation.
We provided 5000 states for training the autoencoder, while the search space consists of 362880 ( = 9! )
states and 967680 transitions.
Note that each state have 9!
variations due to the permutations in the order the tiles and the locations are reported.
This also increases the number of transition quadratically ( (9!) 2 ).
We generated 40 problem instances of 8-puzzle each generated by a random walk from the goal state.instances consist of 20 instances each generated by a 7-steps random walk and another 20 by 14 steps.
We solved 40 instances using Fast Downward classical planner Helmert (2004) with blind heuristics in order to remove the effect of heuristics.
We compared the number of problems successfully solved by two variations of Latplan where each uses the autoencoder trained with Set Average and Set Cross Entropy, respectively, for encoding the input into binary latent space.
Both version managed to solve all instances because both Set Average and Set
Cross Entropy managed to train the AE from 5000 examples with a sufﬁcient accuracy.
All solutions were correct (checked manually).
Since the search algorithm being used is optimal, the quality of the solution was also identical.
B LOCKSWORLD
We solved 30 planning instances in a 4-blocks, 3-stacks environment.
The instances are generated by taking a random initial state and choosing a goal state by the 3, 7, or 14 steps random walks (10 instances each).
The system correctly solved all instances, where the correctness of the plans are checked manually.
The same planner conﬁguration was used for all instances.
The search space consists of 5760 states and 34560 transitions, and each state have 4!
variations due to permutations of 4 blocks.
We provided 1000 randomly selected states for training the autoencoder.
We compared the number of problems successfully solved by Latplan between two variations of Latplan using the autoencoder trained with Set Average and Set Cross Entropy, respectively.
Out of 30 instances,
Latplan+Set Avg returned XX plans, out of which XX plans were correct, while Latplan+SCE managed to solve all instances correctly.
As the autoencoder trained by Set Average had larger reconstruction error, it sometimes fails to capture the essential feature of the input, causing the system to return an invalid plan.
The common error was changing the surface of the blocks or swapping the blocks without a proper action needed, e.g., moving more than two blocks, move a block and polish another block in a single time step, etc.
Random walk steps
The number of solved instances used for generating (out of 10 instances each) the problem instances
SH
[Table]