A S EMANTIC L OSS F UNCTION FOR D EEP L EARNING WITH S YMBOLIC K NOWLEDGE
Anonymous authors
Paper under double-blind review
A BSTRACT
This paper develops a novel methodology for using symbolic knowledge in deep learning.
From ﬁrst principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints.
This loss function captures how close the neural network is to satisfying the constraints on its output.
An experimental evaluation shows that our semantic loss function effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classiﬁcation.
Moreover, it signiﬁcantly increases the ability of the neural network to predict structured objects, such as rankings and shortest paths.
These combinatorial concepts are tremendously difﬁcult to learn, and beneﬁt from a combination of neural learning and symbolic reasoning methods.
I NTRODUCTION
The widespread success of representation learning raises the question of which AI tasks are amenable to deep learning, which require classical model-based symbolic reasoning, and whether we can beneﬁt from an integration of both.
In recent years, signiﬁcant effort has gone towards various ways of using representation learning to solve tasks that were previously tackled by symbolic methods.
Such efforts include neural computers, Turing machines, and differentiable programming (e.g., Weston et al.
Reed & De Freitas (2015); Graves et al.
; Riedel et al.
(2016), relational embeddings, deep learning for graph data, and neural theorem proving (e.g., Bordes et al.
; Neelakantan et al. (2015); Duvenaud et al.
; Niepert et al.
(2016), and many more.
Other work has sought to augment deep learning with (symbolic) knowledge (e.g., Hu et al.
Stewart & Ermon (2017); M ´ arquez-Neila et al.
(2017); Minervini et al.
; Wang et al.
This paper considers learning tasks where we have symbolic knowledge connecting the different outputs of a neural network.
This knowledge takes the form of a constraint (or sentence) in Boolean logic.
It can be as simple as an exactly-one constraint for one-hot output encodings, or as complex as a structured output prediction constraint for intricate combinatorial objects such as rankings, subgraphs, and paths.
Our goal will be for the neural network to learn how to make predictions subject to these constraints, and use the symbolic knowledge to improve its performance.
Most neuro-symbolic approaches aim to simulate or learn symbolic reasoning in an end-to-end deep neural network, or capture symbolic knowledge in a vector-space embedding.
This choice is partly motivated by the need for smooth differentiable models; adding symbolic reasoning code (e.g., SAT solvers) to a deep learning pipeline destroys this property.
Unfortunately, while making reasoning differentiable, the precise logical meaning of our knowledge is often lost.
In this paper, we take a distinctly different approach, and tackle the problem of differentiable but sound logical reasoning from ﬁrst principles.
Starting from a set of intuitive axioms, we derive a differentiable semantic loss function that captures how well the outputs of a neural network match a given constraint.
This function precisely captures the meaning of the constraint, and is independent of its syntax .
Next, we show how this semantic loss gives signiﬁcant practical improvements in semi-supervised classiﬁcation.
The semantic loss deﬁned over the exactly-one constraint in this setting permits us to obtain a learning signal from vast amounts of unlabeled data.
The key idea is that the semantic loss helps us improve how consistently we are able to classify the unlabeled data.
This simple addition to the loss function of standard deep learning architectures yields (near-)state-of-the-art performance in semi-supervised classiﬁcation.
[Figure]
Our ﬁnal set of experiments study the beneﬁts of the semantic loss function for complex structured output learning tasks, such as preference learning and path prediction in a graph (Chang et al., 2009;
Daum´e et al., 2009; Choi et al., 2015; Graves et al., 2016).
In these scenarios, the task is two-fold: learn both the structure of the output space, and the actual classiﬁcation function within that space.
By capturing the structure of the output space with logical constraints, and minimizing the semantic loss for this constraint during learning, we are able to learn networks that are much more likely to correctly predict structured objects.
B ACKGROUND AND M OTIVATION
We write uppercase letters for random variables and lowercase letters for their instantiation.
Sets of variables are written in bold uppercase, and their joint instantiation in bold lowercase.
Concatenations of sets denote their union (e.g. XY ).
To formally deﬁne and introduce our semantic loss, we will make use of certain concepts in propositional logic.
We write uppercase letters ( X , Y ) for Boolean or random variables and lowercase letters ( x , y ) for their instantiation.
Sets of variables are written in bold uppercase ( X , Y ), and their joint instantiation in bold lowercase ( x , y ).
A literal is a variable ( x ) or its negation ( ¬ x ).
A logical sentence is constructed in the usual way, from variables and logical connectives ( ∧ , ∨ , etc.).
A state or world x is an instantiation to all variables X .
A state or world x satisﬁes a sentence α , denoted x |
=
α , if the sentence evaluates to true in the world, as deﬁned in the usual way.
We say that a sentence α entails another sentence β , denoted α | = β if all worlds that satisfy α also satisfy β .
We say that α is logically equivalent to β , denoted α ≡ β , if α | = β and β | = α .
With p we denote the output row vector of a neural net.
Each value in p represents the probability of a label and falls in [0 , 1] .
The most common output units for this in deep learning are softmax and sigmoid units, both of which are used in this paper.
We will use the notation for states x to sometimes refer to the assignment, to the logical sentence enforcing the assignment, or the binary vector capturing that same assignment, as these are all equivalent notions.
This paper studies three different output constraints (visualized in Figure 1) of varying difﬁculty, and their effects on learning.
First, we will examine the one-hot constraint.
This can be thought of as a multi-class classiﬁcation.
It states that for a set of indicators X
= { X 1 , . . .
X n } , one and exactly one of those indicators must be true, with the rest being false.
This is enforced through a logical constraint by conjoining sentences of the form ¬ X 1 ∨ ¬ X 2 for all pairs of variables, encoding the requirement that at most one variable is true, and a single sentence X 1 ∨ · · · ∨ X n , encoding the requirement that at least one variable is true.
Our experiments will also examine the valid simple path constraint.
It states for a given sourcedestination pair and edge indicators, that the edge indicators which are set to true must form a valid simple path from source to destination.
Finally, we explore the ordering constraint, which requires that a set of n 2 indicator variables represent a total ordering over n variables, effectively encoding a permutation matrix.
For a full description of the path and ordering constraints, we refer to Section 5.
S EMANTIC L OSS
Our goal in this section is to ﬁnd a semantic loss function that bridges the gap between the continuous world of neural networks, and the symbolic world of propositional logic.
We do so by ﬁrst postulating intuitive high-level properties that we seek in such a function, and that illustrate its desired behavior.
A second set of postulates establishes a correspondence between constraints and data.
Finally, we uniquely deﬁne the semantic loss function used throughout this paper.
The semantic loss L s ( α, p ) is a function of a sentence α in propositional logic, deﬁned over variables
X =
{ X 1 , . . .
, X n } , and a vector of probabilities p for the same variables X .
Element p
i denotes the predicted probability of variable X i , and corresponds to a single output of the neural network.
For example, the semantic loss between the one-hot constraint described in the previous section, and a neural net output vector p , is intended to capture how close the prediction p is to having exactly one output set to true (that is, 1).
3. H IGH -L EVEL P ROPERTIES
The ﬁrst axiom says that there is no loss when the logical constraint α is always true (it is a logical tautology), regardless of the predicted probabilities p .
Axiom 1 (Truth) .
The semantic loss of a true sentence is zero: ∀ p , L s ( true , p ) = 0 .
Next, when enforcing two constraints on disjoint sets of variables, we want to ability to compute the semantic loss of the two constraints separately, and sum the result to obtain their joint semantic loss.
Axiom 2 (Additive Independence) .
Let α be a sentence over X with probabilities p .
Let β be a sentence over Y disjoint from X with probabilities q .
The semantic loss between sentence α ∧ β and the joint probability vector [ p q ] decomposes additively: L s ( α ∧ β, [ p q ]) =
L s ( α, p )+L s ( β, q ) .
A consequence Axioms 1 and 2 is that the probabilities of variables that are not used on the constraint do not affect the semantic loss.
Proposition 6 in Appendix A formalizes this intuition.
To maintain logical meaning, we postulate that semantic loss is monotone in the order of implication.
Axiom 3 (Monotonicity) .
If α | = β , then the semantic loss L s ( α, p ) ≥ L s ( β, p ) for any vector p .
Intuitively, as we add stricter requirements to our logical constraint, going from β to α and making it harder to satisfy, the semantic loss cannot decrease.
For example, when β enforces the output of our network to encode a subtree of a graph, and we tighten that requirement in α to be a path, then the semantic loss cannot decrease.
Every path is also a tree and any solution to α is a solution to β .
A ﬁrst consequence of the monotonicity axiom is that logically equivalent sentences must incur an identical semantic loss for the same probability vector p .
Hence, the semantic loss is indeed a semantic property of the logical sentence, and does not depend on the syntax of the sentence.
Proposition 1.
If α ≡ β , then the semantic loss L s ( α, p ) =
L s ( β, p ) for any vector p .
A second consequence is that semantic loss must be non-negative (see Proposition 5 in Appendix A).
3. D ATA -S ENTENCE C ORRESPONDENCE
A state x is equivalently represented as a data instance, as well as a logical constraint that enforces a value for every variable in X .
When both the constraint and the predicted vector represent the same state (for example, X 1 ∧ ¬ X 2 ∧ X 3 vs. [1 0 1] ), there should be no semantic loss.
Axiom 4 (Identity) .
For any state x , there is zero semantic loss between its representation as a sentence, and its representation as a deterministic vector: ∀ x , L s ( x , x ) = 0 .
The axioms above together imply that any vector satisfying the constraint must incur zero loss.
For example, when our constraint α requires that the output vector encodes an arbitrary total ranking, and the vector x correctly represents a single speciﬁc total ranking, there is no semantic loss.
Proposition 2 (Satisfaction) .
If x | = α , then the semantic loss L s ( α, x ) = 0 .
As a special case, logical literals ( x or ¬ x ) constrain a single variable to take on a single value, and thus play a role similar to the labels used in supervised learning.
Such constraints require an even tighter correspondence:
the semantic loss must act like a classical loss function (i.e., cross entropy).
Axiom 5 (Label-Literal Correspondence) .
The semantic loss of a single literal is proportionate to the cross-entropy loss for the equivalent data label:
L s ( x, p ) ∝ − log( p ) and L s ( ¬ x, p ) ∝ − log(1 − p ) .
Appendix A states two additional axioms, on the symmetry between values and the symmetry between variables, as well as a Lemma 7 that ties together the multiplicative constants mentioned in
Axiom 5.
Finally, this allows us to prove the following form of the semantic loss for a state x .
Lemma 3.
For state x and vector p , we have L s ( x , p ) ∝ − (cid:80) i : x |
=
X i log p i − (cid:80) i : x | =
¬ X i log(1 − p i ) .
3. A G ENERAL D EFINITION
Lemma 3 falls short of a full deﬁnition of semantic loss for arbitrary sentences.
One can deﬁne additional axioms to pin down L s .
For example, the following axiom is highly desirable.
Axiom 6 (Differentiability) .
For any ﬁxed α , the semantic loss L s ( α, p ) is monotone in each probability in p , continuous and differentiable.
Appendix A makes the notion of semantic loss precise by stating one additional axiom.
It is based on the observation that the state loss of Lemma 3 is proportionate to a log-probability.
In particular, it corresponds to the probability of obtaining state x after independently sampling each X i with probability p
i .
We have now derived the semantic loss function from ﬁrst principles as follows.
Deﬁnition 1 (Semantic Loss) .
Let p be a vector of probabilities, one for each variable in X , and let α be a sentence over X .
The semantic loss between α and p is
[Equation]
Theorem 4 (Uniqueness) .
The semantic loss function in Deﬁnition 1 satisﬁes Axioms 1–9 and is the only function that does so, up to a multiplicative constant.
Intuitively, the semantic loss is proportionate to a negative logarithm of the probability of generating a state that satisﬁes the constraint, when sampling values according to p .
Hence, it is the selfinformation (or “surprise”) of obtaining an assignment that satisﬁes the constraint (Jones, 1979).
S EMI -S UPERVISED C LASSIFICATION
The most straightforward constraint imposed on most real-world machine learning tasks is mutual exclusion over outputs.
That is, for a given example exactly one binary label must be true.
This problem setting is often referred to as multi-class supervised classiﬁcation.
Over the past few years, the machine learning community has made great strides in this area, mostly due to the invention of assorted deep learning representations and their associated regularization terms (Krizhevsky et al., 2012;
He et al., 2016).
Many of these models take large amounts of fully labeled data for granted, and indeed big data is becoming an indispensable condition for discovering accurate representation (Hastie et al., 2009).
However, due to the curse of dimensionality, further accuracy improvement tends to require exponentially more data, threatening the sustainability of the current learning paradigm.
Therefore, there is a growing interest into utilizing unlabeled data to augment a model’s predictive power (Stewart & Ermon, 2017; Bilenko et al., 2004).
In this section, we show why semantic loss naturally qualiﬁes for this task.
4. I LLUSTRATION
To illustrate the beneﬁt of semantic loss in the semi-supervised setting, we begin our discussion with a tiny toy example.
Suppose we are working on a binary classiﬁcation task, consisting of around twenty samples with more than half unlabeled (see Figure 2).
Ignoring the unlabeled samples, a simple linear classiﬁer learns to distinguish the two classes by maximizing the distance to labeled examples.
Though unlabeled, those samples marked by green squares in the plot can be informative, as they must carry some properties which give them a particular label.
This is the crux of semantic loss.
It asserts that a model must conﬁdently assign a consistent class even to unlabeled data.
Indeed, encouraging the model to do so results in a more accurate separation line, as illustrated by the comparison between the left and right side of Figure 2.
We further explore this idea and apply it to some real-world classiﬁcation tasks in the following sections.
[Figure]
4. A LGORITHM
Our proposed method intends to be generally applicable and compatible with any feedforward neural network.
The semantic loss is simply another regularization term that can directly be plugged into an existing loss function.
More speciﬁcally, the new overall loss becomes existing loss + w · semantic loss with w denoting the associated weight.
Note when the constraint over the output space is rather simple (and admits a small number of solutions), the semantic loss can be directly computed using
Deﬁnition 1.
However, once the constraint becomes more complex, a more systematic reasoning approach is required.
Section 5 discusses complex constraints in more detail.
4. E XPERIMENT D ETAILS
In this section, we evaluate our work in the semi-supervised setting by comparing it with several competitive models.
As most semi-supervised learning methods build on top of some supervised models, it is no surprise that changing the underlying model would signiﬁcantly affect the ﬁnal model’s representation power.
For the sake of comparison, we add semantic loss to the same base models used in ladder nets (Rasmus et al., 2015), which currently achieves state-of-the-art results on MNIST and CIFAR-10 (Krizhevsky & Hinton, 2009).
Speciﬁcally, the base model on MNIST is a fully-connected multiple layer perceptron (MLP), with layers of size 784-1000-500-250-250-250-10.
On CIFAR-10, it is a 10-layer convolutional neural network (CNN) with 3-by-3 padded ﬁlters.
After every 3 layers, features are subject to a 2-bymax-pool layer with strides of 2.
Furthermore, ReLu (Nair & Hinton, 2010), normalization (Ioffe &
Szegedy, 2015), and Adam optimization (Kingma & Ba, 2015) for weight updates with a learning rate of 0.002 are always used on both.
Images are preprocessed with a standardization process and
Gaussian noise with a standard deviation of 0.3 is added to every pixel.
We refer to Appendix B for more details.
For all experiments in this section, we use the standard 10,000 test samples provided in the original dataset as a held-out test set and randomly split the standard 60,000 training samples (50,000 for
CIFAR-10) into a 10,000-sample validation set with the rest as a training set.
From the training set, we randomly choose N = 100 , 500 , 1000 , 4000 , or all, depending on which experiment we run, and remove labels from the rest.
We balance classes in the labeled samples to ensure no particular class is over-represented.
The validation set is used for tuning the weight associated with semantic loss, the only hyper-parameter that causes noticeable difference in performance for our method.
We perform a grid search over { 0 . 001 , 0 . 005 , 0 . 01 , 0 .
05 , 0 . 1 } to ﬁnd the optimal.
Empirically, 0.
always gives the best or nearly the best results and we report its results on all experiments.
4.3. MNIST
A commonly used test-bed for general semi-supervised learning algorithms is the permutation invariant MNIST classiﬁcation task.
This setting disallows any use of prior information about the
[Table]
spatial arrangement of the input pixels.
In other words, it excludes many data augmentation techniques that involve geometric distortion of images.
When evaluating on MNIST
[Table]
we run experiments for 10 epochs, with a batch size of 20 where half are labeled.
Furthermore, experiments are repeated 10 times with different random seeds to make sure results are consistent.
Results are reported in Table 1.
Two baselines are created. The ﬁrst is purely supervised MLP, which makes no use of unlabelled data. The second is the classic self training method for semi-supervised learning. After everyiterations, the unlabeled examples which have more than 95% likelihood of belonging to some single class, as predicted by the underlying MLP, is assigned with such a psuedo-label.
Note when N = 100 , MLP with semantic loss gains around 20% improvement over the purely supervised learning baseline and the improvement is even larger when comparing with the self-training method.
Considering the only change is an additional loss term, such results are very encouraging.
Though indeed falling behind ladder nets, the 0 .
5% difference may just be an artifact of more careful tuning of hyper-parameters or learning rates.
In the coming experiments, we extend our work to other more challenging datasets, so a mor methodological comparison between the two can be conducted.
Before that, we want to share a few more thoughts on how semantic loss works.
Observe that when N = all, the experiment degrades into the fully supervised setting, in which semantic loss does not provide us with any improvement in classiﬁcation accuracy.
One likely explanation is that by overﬁtting the training dataset, the MLP inadvertently learns the domain constraint, particularly given that softmax cross entropy is used as the primary loss function.
Interpreted as a representation of a categorical distribution, softmax normalizes outputs, implying they are disjoint.
In other words, softmax can be viewed as an implicit encoding of mutual exclusion.
However, there does not exist a natural way to extend softmax loss to unlabeled samples, whereas semantic loss uses these unlabeled samples to force the underlying classiﬁer to actually make decision across all samples.
4.3. FASHION
FASHION (Xiao et al., 2017) is a dataset consisting of Zalando’s article images, aiming to serve as a drop-in replacement for MNIST.
Arguably, it is a more challenging benchmark, as at least it has not been overused and requires use of more modern machine learning advancements to achieve good performance.
As the two share the same image size and structure, methods developed in MNIST should be able to directly perform on FASHION without heavy modiﬁcations.
Because of this, we use the same hyper-parameters when evaluating our method.
However, for the sake of fairness, we subject ladder nets to a small-scale parameter tuning in case its performance is more volatile.
As in the previous experiment, we run our method for 10 epochs, whereas ladder nets needepochs to converge.
Again, experiments are repeated 10 times and standard deviation of classiﬁcation accuracy is reported, except when N = all it is close to 0 and omitted to save table space.
Results (see Table 2) show utilizing semantic loss results in an enormous 17% improvement over the baseline when only 100 labels are provided.
Moreover, our method also compares favorably to ladder nets, except when the setting degrades to be fully supervised.
Note our method already nearly reaches its maximum accuracy with 500 provided labels, which is only 1% of the training dataset. [Table] [Table] [Table]
4.3. CIFAR-
In order to show the general applicability of our proposed method [Table]
we extend our work to CIFAR-
[Table]
a dataset consisting of 32-by-32 RGB images in 10 classes
[Table]
on which a simple MLP would not have enough representation power to capture the huge variance across objects within the same class.
Coping with this spike in task difﬁculty, we switch our underlying model to a 10-layer CNN as mentioned at the beginning of this section.
When evaluating our work on CIFAR-
[Table]
we use a batch size of 100 samples of which half are unlabeled.
Experiments are run for 100 epochs.
However, due to the constraint of computational resources
[Table]
we can only afford one trial.
Note we make slight modiﬁcations to the underlying model used in ladder nets to reproduce similar baseline performance.
Please refer to Appendix B for details.
As shown in Table 3, our method compares favorably to ladder nets.
However, due to the slight difference in terms of performance between two baselines, a direct comparison would be methodologically ﬂawed.
Instead, we compare the net improvements over baselines.
In terms of this measure, our method scores a gain of 4 . 66% whereas Ladder Net gains 2 .
4. D ISCUSSION
Overall, the experiments outlined so far have demonstrated the competitiveness and general applicability of our proposed method on semi-supervised learning tasks.
It achieves state-of-the-art results on two challenging datasets, FASHION and CIFAR-10, while being close on MNIST.
Considering the simplicity of our method, such results are encouraging.
Another advantage of directly tweaking the loss function is computational efﬁciency.
Without any overhaul of the structure of model itself, we incur almost no additional computational overhead.
Conversely, this same property makes our method very sensitive to the underlying model’s performance.
Without the underlying predictive power of a well-performing supervised learning model, penalties associated with not being conﬁdent in decisions would not necessarily result in the same effects we have observed.
L EARNING WITH C OMPLEX C ONSTRAINTS
While much of current machine learning research is focused on problems such as multi class classiﬁcation, there remain a multitude of difﬁcult problems involving highly constrained output domains.
As mentioned in our section on semi-supervised learning, semantic loss shows next to no improvement on the supervised single class classiﬁcation problem.
This led us to seek out more difﬁcult problems to illustrate that semantic loss can also be highly informative in the supervised case, provided the output domain has a sufﬁciently complex space.
As a result of semantic loss being deﬁned by a Boolean formula, it can be used on any output domain which can be fully described in this manner, regardless of complexity.
Here, we will develop a framework for tractable semantic loss on highly complex constraints, and evaluate it on some difﬁcult examples. [Table] [Table] semantic loss
[Table]
[Table]
5. A T RACTABLE S EMANTIC L OSS
Our goal here is to develop a method for computing both the semantic loss and its gradients in a tractable manner.
Examining Deﬁnition 1 of semantic loss, we see that the right-hand side is a wellknown automated reasoning task called weighted model counting (WMC) (Chavira & Darwiche, 2008; Sang et al., 2005).
A key property of WMC is that its partial derivatives can be computed in terms of other, slightly modiﬁed WMCs.
Furthermore, we know of circuit languages that compute weighted model counts, and that are amenable to backpropagation (Darwiche, 2003).
We use the language and circuit compilation techinques described in Darwiche (2011) to build a Boolean circuit representing our semantic loss.
Due to certain properties of this circuit form, we can use it to compute both the values and the gradients of the semantic loss in time linear in the size of the circuit (Darwiche & Marquis, 2002).
Once we have constructed this function, we can add it to our standard loss function as described in Section 4.2.
5. E XPERIMENTAL E VALUATION
Our ambition when evaluating semantic loss’s performance on complex constraints is not to achieve state-of-the-art performance on any particular problem, but rather to highlight its effect.
To this end, we evaluate our method on problems with difﬁcult output space, where the model could no longer ﬁt directly from data, and purposefully use simple MLPs for evaluation.
5.2. G RIDS
We begin with a classic algorithmic problem, ﬁnding the shortest path in a graph.
Speciﬁcally, we use a 4-by-4 grid G =
( V, E ) with uniform edge weights.
We randomly remove some edges for each example to increase difﬁculty.
Formally, our input is a binary vector of length | V | + | E | , with the ﬁrst | V | variables indicating sources and destinations, and the next | E | which edges are removed.
Similarly, the label for a given example is a binary vector of length | E | indicating which edges are in the shortest path.
Finally, we require through our constraint that the output form a valid simple path between the desired source and destination.
To compile the valid simple path constraint, we use the method given by Nishino et al.
(2017) to generate pairwise simple paths, and logically merge them to enforce the correct source and destination.
For more details on the constraint as well as the data generation procedure, see Appendix C.
To evaluate, we use our generated dataset of 1600 examples, with a 60/20/20 training/validation/testing split.
The only hyper parameter that needed to be tuned was the weight given to semantic loss, which after trying { 0 . 001 , 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 5 , 1 } was selected to be 0.5 based on validation results.
Table 4 compares test accuracy between a 5-layer MLP baseline, and the same model augmented with semantic loss.
Examining both types of accuracy in this table 1 illustrates the effect of semantic loss.
In the case of individual labels, semantic loss has little effect, and in fact slightly reduces the accuracy as it combats the standard sigmoid cross entropy.
In regards to getting paths exactly right, however, the semantic loss has a huge effect in guiding the network to jointly learn true paths, rather than optimizing each binary output individually.
5.2. P REFERENCE L EARNING
The next problem we examine is that of predicting a complete order of preferences.
That is, for a given set of user features, we would like to predict how the user would rank their preference over a at all. ﬁxed set of items.
We encode a preference ordering over n items as a ﬂattened binary matrix { X ij } , where for each i, j ∈ { 1 , . . . , n } , it denotes that item i is at position j
(Choi et al., 2015).
Clearly, not all conﬁgurations of outputs correspond to a valid ordering.
For data, we use preference rankings over 10 types of sushi for 5000 individuals, taken from
P REF L IB (Mattei & Walsh, 2013).
We take the ordering over 6 types of sushi as input features to predict the ordering over the remaining 4 types, with splits identical to those in Shen et al.
We again split the data 60/20/20 into train/test/split, and employ a 3 layer MLP as our baseline.
We initially chose the semantic loss weight from { 0 . 001 , 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 5 , 1 } to be 0.1 based on validation, and then further tuned the weight to 0.25.
Table 5 compares the baseline to the same
MLP augmented with semantic loss for valid permutations.
Again, we see that semantic loss has a marginal effect on the per label accuracy, but massively improves the network’s ability to predict correct orderings.
C ONCLUSIONS
Both reasoning and semi-supervised learning are often identiﬁed as key challenges for deep learning going forward.
In this paper, we developed a principled way of combining automated reasoning for propositional logic with existing deep learning architectures.
Moreover, we showed that our semantic loss function provides signiﬁcant beneﬁts during semi-supervised classiﬁcation, as well as deep structured prediction for highly complex output spaces.
A A XIOMATIZATION OF S EMANTIC L OSS : D ETAILS
This appendix provides further details on our axiomatization of the semantic loss.
Proposition 5 (Non-Negativity) .
Semantic loss is non-negative.
Proof.
Because α | = true for all α , the monotonicity axiom implies that ∀ p , L s ( α, p ) ≥ L s ( true , p ) .
By the truth axiom, L s ( true , p ) = 0 , and therefore L s ( α, p ) ≥ 0 for all choices of α and p .
Proposition 6 (Locality) .
Let α be a sentence over X with probabilities p .
For any Y disjoint from
X with probabilities q , the semantic loss L s ( α, [ p q ]) =
L s ( α, p ) .
Proof.
Follows from the additive independence and truth axioms.
Set β
= true in the additive independence axiom, and observe that this sets L s ( β, q ) = 1 because of the truth axiom.
Proof of Proposition 2.
The monotonicity axiom specializes to say that if x |
= α , we have that ∀ p , L s ( x , p ) ≥ L s ( α, p ) .
By choosing p to be x , this implies L s ( x , x ) ≥ L s ( α, x ) .
From the identity axiom, L s ( x , x ) = 0 , and therefore 0 ≥ L s ( α, x ) .
Proposition 5 bounds the loss from below as L s ( α, x ) ≥ 0 .
Axiom 7 (Value Symmetry) .
For all p and α , we have that L s ( α, p ) =
L s (¯ α, 1 − p ) where ¯ α replaces every variable in α by its negation.
Axiom 8 (Variable Symmetry) .
Let α be a sentence over X with probabilities p .
Let π be a permutation of the variables X , let π ( α ) be the sentence obtained by replacing variables x by π ( x ) , and let π ( p ) be the corresponding permuted vector of probabilities.
Then, L s ( α, p ) =
L s ( π ( α ) , π ( p )) .
The value and variable symmetry axioms together imply the equality of the multiplicative constants in the label-literal duality axiom for all literals.
Lemma 7.
There exists a single constant K such that L s ( X, p ) =
− K log( p ) and L s ( ¬ X, p ) =
− K log(1 − p ) for any literal x .
Proof.
Value symmetry implies that L s ( X i , p ) =
L s ( ¬ X i , 1 − p ) .
Using label-literal correspondence, this implies K 1 log( p i ) =
K 2 log(1 − (1 − p i )) for the multiplicative constants K 1 and K 2 that are left unspeciﬁed by that axiom.
This implies that the constants are identical.
A similar argument based on variable symmetry proves equality between the multiplicative constants for different i .
Proof of Lemma 3.
A state x is a conjunction of independent literals, and therefore subject to the additive independence axiom.
Each literal’s loss in this sum is deﬁned by Lemma 7.
The following and ﬁnal axiom requires that the semantic loss is proportionate to the logarithm of a function that is additive for mutually exclusive sentences.
Axiom 9 (Exponential Additivity) .
Let α and β be mutually exclusive sentences (i.e., α ∧ β is unsatisﬁable), and let f s ( K, α, p ) =
K − L s ( α, p ) .
Then, there exists a positive constant K such that f s ( K, α ∨ β, p ) =
f s ( K, α, p ) + f s ( K, β, p ) .
Proof of Theorem 4.
The truth axiom states that ∀ p , f s ( K, true , p ) = 1 for all positive constants K .
This is the ﬁrst Kolmogorov axiom of probability.
The second Kolmogorov axiom for f s ( K, ., p ) follows from the additive independence axiom of semantic loss.
The third Kolmogorov axiom (for the ﬁnite discrete case) is given by the exponential additivity axiom of semantic loss.
Hence, f s ( K, ., p
) is a probability distribution for some choice of K , which implies the deﬁnition up to a multiplicative constant.
[Table]
B S PECIFICATION OF THE C ONVOLUTIONAL N EURAL N ETWORK M ODEL
Table 6 shows the slight architectural difference between the CNN used in ladder nets and ours.
The major difference lies in the choice of ReLu.
Note we add standard padded cropping to preprocess images and an additional fully connected layer at the end of the model, neither is used in ladder nets.
We only make those slight modiﬁcation so that the baseline performance reported by (Rasmus et al., 2015) can be reproduced.
C S PECIFICATION OF C OMPLEX C ONSTRAINT M ODELS C. G RIDS
To compile our grid constraint, we ﬁrst use Nishino et al.
(2017) to generate a constraint for each source destination pair.
Then, we conjoin each of these with indicators specifying which source and destination pair must be used, and ﬁnally we disjoin all of these together to form our constraint.
To generate the data, we begin by randomly removing one third of edges.
We then ﬁlter out connected components with fewer than 5 nodes to reduce degenerate cases, and proceed with randomly selecting pairs of point to create data points.
The predictive model we employ as our baseline is a 5 layer MLP with 50 hidden sigmoid units per layer.
It is trained using Adam Optimizer, with full data batches (Kingma & Ba, 2015).
Early stopping with respect to validation loss is used as a regularizer.
C. P REFERENCE L EARNING
We split each user’s ordering into their ordering over sushis 1,2,3,5,7,8, which we use as the features, and their ordering over 4,6,9,10 which are the labels we predict.
The constraint is compiled directly from logic, as this can be done in a straightforward manner for an n-item ordering.
The predictive model we use here is a 3 layer MLP with 25 hidden sigmoid units per layer.
It is trained using Adam Optimizer with full data batches (Kingma & Ba, 2015).
Early stopping with respect to validation loss is used as a regularizer.