Year: 2021

Abstract:
Human defined concepts are inherently transferable, but it is not clear under what conditions they can be modelled effectively by non-symbolic artificial learners. This paper argues that for a transferable concept to be learned, the system of relations that define it must be coherent across domains. This is to say that the learned concept-specific relations ought to be consistent with respect to a theory that constrains their semantics and that such consistency must extend beyond the representations encountered in the source domain. To demonstrate this, we first present formal definitions for consistency and coherence, and a proposed Dynamic Comparator relation-decoder model designed around these principles. We then perform a proposed Partial Relation Transfer learning task on a novel data set, using a neural-symbolic autoencoder architecture that joins sub-symbolic representations with modular relation-decoders. By comparing against several existing relation-decoder models, our experiments show that relation-decoders which maintain consistency over unobserved regions of representational space retain coherence across domains, whilst achieving better transfer learning performance.

heading: 1 INTRODUCTION
text: Humans are capable of learning concepts such that they can be applied to many different scenarios (Inhelder & Piaget, 1964; Piaget, 2005; Lake et al., 2017). An important characteristic is that human-like concepts remain coherent across contexts, whereby their logical consistency in one context is retained in another (Nye et al., 2021). As an example, consider the concept of ordinality which permits comparison over ordered sets, e.g. “A is larger than B”, and pertains to a multitude of properties: position, size, volume, reach, etc. So long as one of these properties can be attributed to an object, a set of objects can be compared on that basis; in this sense ordinality generalises between objects. All in all, if the concept of ordinality were to be learned in its most general form, it should be coherent across properties and objects.
In this paper, we seek to define the conditions that allow a learned concept to transfer well across properties and objects in the case of sub-symbolic learners (d’Avila Garcez & Lamb, 2020; Santoro et al., 2021; Greff et al., 2020). We define consistency and coherence of sub-symbolic learners borrowing from analogous definitions from symbolic AI. We propose a neural-symbolic autoencoder architecture consisting of a neural encoder for objects coupled with modular relation-decoders (Serafini & Garcez, 2016; Donadello et al., 2017; Badreddine et al., 2020; Wang et al., 2017; Nickel et al., 2016a; Dai et al., 2020), and we show that this architecture is capable of achieving an improved transfer learning performance by being coherent across properties and objects.
We therefore claim that retaining consistency across domains dictates concept coherence, offering a more fine-grained measure of transfer learning than accuracy alone. The proposed architecture is a practical instantiation of this formalisation and is evaluated in this paper on a new Partial Relation Transfer (PRT) task and data set. We begin by expressing the symbolic application of a set of relations to some domain of interest as a model-theoretic structure, such as A is larger than B, and defining an analogous soft-structure for non-symbolic learners where relations are modelled by relation-decoders that compute beliefs. We then propose formal definitions for consistency and coherence of soft-structures which provide a practical consistency score calculation to the evaluation of autoencoders. Finally, we present a benchmark PRT learning task with the use of a new BlockStacks data set derived from the CLEVR data set rendering agent. We then compare our proposed archi-
tecture with several existing relation-decoder models on transfer learning tasks from the MNIST data set to BlockStacks. Our experiments show that relation-decoders which maintain consistency over unobserved regions of representational space retain coherence across domains whilst achieving better transfer learning performance. The contributions of this paper are:
• A formal definition of coherence and consistency for sub-symbolic learners with a practical evaluation score.
• A neural model and learning task for partial relation transfer including a new data set to evaluate concept coherence.
• A comprehensive critical evaluation of results in comparison with multiple state-of-the-art relation-decoder models.
In Section 2 we provide the required background, Section 3 introduces soft-structures and formally defines coherence and consistency, Section 4 provides a practical consistency loss and Section 5 then outlines our neural-symbolic architecture. After detailing the PRT task in Section 6, we present results in Section 7 and complete the paper in Section 8 with a discussion and concluding remarks, including limitations and future work. We provide related work in Appendix A.

heading: 2 PRELIMINARIES
text: Notations: We reserve uppercase calligraphic letters to denote sets and lowercase versions of the same letter to denote their elements, e.g., S = {s1, . . . , sn} is a set S of n elements si. We indicate with |S| = n the cardinality of S. We use uppercase roman letters to denote a random variable e.g., S, and use the uppercase calligraphic version of the same letter (S) to denote the set from which the random variable takes values according to some corresponding probability distribution, pS , over the elements of the set, such that ∑|S| i=1 pS(si) = 1 for a discrete S. For brevity, we may write pS(si) as p(si), where the random variable is implied by the argument. We use bold font lowercase letters to denote vector elements, e.g., si ∈ Rd is an d-dimensional vector element from the set S = Rd. Logic and model-theoretic background: We assume a formal language L composed of variables, predicates (i.e. relations), logical connectives ¬ (negation), ∨ (disjunction) and ∧ (conjunction),→ (implication) and universal quantification ∀ (for all) with their conventional meaning (see Shapiro & Kouri Kissel (2021)). Relations express relational knowledge over elements of a domain. For instance, r(s1, s2) states that elements s1 and s2 are related through the binary relation r. The meaning of relations is defined by an interpretation, ISσ over elements of an non-empty domain S. Together a S and ISσ form a structure Sσ .
Definition 1 (Signature, Interpretation, Structure) The signature of a language L is σ = {r ∈ L : r is a relation}, whose elements have arity given by ar : σ → N , where N is the set of natural numbers. For each r ∈ σ, ar(r) denotes the arity of r. Given a signature σ and a non-empty domain S, an interpretation ISσ of σ over elements of S assigns to each relation r ∈ σ a set ISσ (r) ⊆ Sar(r). A structure is a tuple Sσ = (S, ISσ ).
Note that for a fixed domain S and signature σ, different interpretations yield different structures. We construct universally quantified first-order formulae (called sentences) using the signature σ of L, whose truth-value is defined with respect to a given structure Sσ . To do so, we first consider ground instances of a formula. These are given by replacing all the variables in the formula with elements from the domain S. For instance, r(s1, s2), where s1 and s2 are elements of S, is a ground instance of an atomic formula r(i, j) where i and j are variables in L. Given a structure Sσ = (S, ISσ ) a relation r and a tuple (s1, . . . , sar(r)) ∈ Sar(r), a ground instance r(s1, . . . , sar(r)) is true in the structure Sσ if and only if (s1, . . . , sar(r)) ∈ ISσ (r). The truth value of a sentence in a given structure Sσ depends on the truth value of its respective ground instances. Specifically, a sentence is true in a structure Sσ if and only if all of its ground instances are true in Sσ . When a sentence, τ , is true in a structure, Sσ , we say that the structure satisfies τ , denoted as Sσ |= τ . A set of sentences form a theory, T . A model of T is a structure that satisfies every sentence in T .
Definition 2 (Model of a theory) Let T be a theory written in a language L and let Sσ = (S, ISσ ) be a structure, where σ is the signature of L. Sσ is a model of T if and only if Sσ |= τ for every sentence τ ∈ T .
Example 1 Suppose we have the structure Sσ = (S, ISσ ), where S is a domain of images of handwritten digits and σ the signature of binary relations σ = {isGreater, isEqual, isLess, isSuccessor, isPredecessor}, or for short σ = {G, E, L, S, P}. Let T be the theory that defines ordinality including, for instance, the sentence ∀i, j.G(i, j)→ ¬E(i, j) (if a digit is greater than another then they are not equal). Any structure Sσ = (S, ISσ ) with interpretations ISσ of σ that captures a total order over the elements of S is a model of T .

heading: 3 APPROXIMATING STRUCTURES THAT HAVE REAL-WORLD DOMAINS
text: In this section we turn our attention to the challenge of learning a model over a real-world domain, given a signature and theory. Here a learner must determine an appropriate interpretation over real-world data, such as images or other perceptions. This can be challenging because, firstly, we may only have a partial description of the interpretation, and secondly data may be noisy and contain information that is not relevant to the theory. For instance MNIST, a relatively simple data set by current standards, consists of stylistic details such as line thickness and digit skew (Chen & Batmanghelich, 2020), which are irrelevant to the notion of ordinality, which makes obtaining the structure from Example 1 more complicated. Nevertheless, statistical machine learning models are able to discover commonalities in data which help to infer the underlying semantics (i.e. interpretation) and disregard the noise. Following the convention in disentanglement literature (Bengio et al., 2013; Kingma & Welling, 2014; Higgins et al., 2017; 2018), we take the assumption that realworld observations S are drawn from some conditional distribution pS|Z, where Z is a latent random variable, itself drawn from prior pZ. It is therefore useful to define a domain encoding of the form,
ψS : S → Z, (1) tasked with approximating the conditioned expectation of the posterior, i.e. ψS(s) = E[pZ|S(Z|s)]. Since obtaining an interpretation from domain encodings, for a given signature, may require dealing with noise, we express the interpretation of relations over real-world data by belief functions over the space Z (Paris & Vencovská, 2015; Paris, 1994) , and refer to these as relation-decoders:
φr : Zar(r) → (0, 1) (2) with φ = {φr : r ∈ σ}. Concretely, for a binary relation r and ordered pair (si, sj) ∈ S2, φr(ψS(si), ψS(sj)) describes the belief that (si, sj) ∈ ISσ (r). A belief φr(ψS(si), ψS(sj)) ≈ 1 signifies a strong belief that (si, sj) ∈ ISσ (r) and φr(ψS(si), ψS(sj))≈ 0 signifies a strong belief that (si, sj) /∈ ISσ (r). Together, ψS and φ allow us to define a belief-based analogue to a structure.
Definition 3 (Soft-Structure/Soft-Substructure) Given signature σ, a possibly infinite set Z and relation-decoders φ, a soft-structure is a tuple Z̃σ = (Z, φ). For (finite) domain S and encoding ψS : S → Z , S̃σ = (ψS(S), φ) is a (finite) soft-substructure of Z̃σ , with sub-domain ψS(S) = {ψS(s)|s ∈ S} ⊆ Z .
A soft-structure can be used to learn a structure over a real-world domain through learning ψS and φ. Clearly, a finite soft-substructure is a soft-structure. To determine the degree to which a softstructure supports any given structure we introduce the following measure:
p(Sσ|S̃σ) = ∏ r∈σ ∏ O∈Sar(r) (φr(ψS(O))) γrO,Sσ (1− φr(ψS(O)))1−γ r O,Sσ (3)
where γrO,Sσ = 1 if O ∈ ISσ (r), and 0 otherwise; we use φr(ψS(O)) as shorthand for φr(ψS(s1), . . . , ψS(sn)) for n = ar(r). Eqn. 3 expresses the assumption that, given a finite softstructure, the beliefs in what constitutes the (different) interpretations of (different) relations are independent of one another. It is straightforward to show that ∑ Sσ p(Sσ|S̃σ) = 1 (summed over all possible structures with domain S and signature σ) and so it can be treated as a probability measure, where p(Sσ|S̃σ) ≈ 1 means that there is a high probability that the interpretation sampled from S̃σ will be ISσ . If we have a theory T over σ then it is natural to ask with what weight S̃σ supports any given structure that is a model of T . In the following, we use model weight, ΓS̃σT , to describe the support given by S̃σ to models of T :
ΓS̃σT = ∑
Sσ∈MTS
p(Sσ|S̃σ) (4)
where MTS is the set of all structures with domain S that are models of T . This lets us compare soft-structures, wherein a good soft-structure will be one that has a high model weight:
Definition 4 ( -Consistency of Soft-Structure) Given a finite soft-structure S̃σ , if 1 − ΓS̃σT ≤ then we say that the soft-structure is -consistent with theory T .
We propose -consistency as an appropriate quantified measure of the notion of consistency presented in (Nye et al., 2021). A consistent soft-structure S̃σ ensures that φ gives high belief only to interpretations that satisfy, i.e., are consistent with, T . However, this expression is limited to the domain encodings of S̃σ , i.e. ψS(S). Going a step further, for a concept to be learned in a manner comparable to what a human might learn, we would expect that this consistency carries over to new domains and their corresponding soft-structures, as defined in what follows.

heading: 3.1 COHERENCE BETWEEN SOFT-STRUCTURES
text: In this section we define the notion of coherence between soft-structures, which aims to characterise what it means for a concept to be learned in a human-like manner. As a motivating case, consider a situation where we have already learned a soft-structure that has high model weight with models from Example 1. Now suppose we are given a new domain of images, Y , showing single block stacks of differing height, and let us again use the signature of ordinal relations and T from Example 1. Lastly, let IYσ be a corresponding interpretation that orders images according to block stack height and is a model of T . We can summarise this with the following two structures:
Xσ = (X , IXσ ) ∈MTX and Yσ = (Y, IYσ ) ∈MTY , (5)
where Xσ is the structure from Example 1 with a domain of handwritten digits and Yσ is our new structure, with a domain of block stack images. These can be modelled by soft-structures:
X̃σ = (ψX (X ), φ) and Ỹσ = (ψY(Y), φ), (6)
which use domain-specific encoders, ψX and ψY , but share the same relation-decoders. As we know that X̃σ has a high model weight and since φ is shared with Ỹσ , a natural question to ask is: under what conditions will a φ that is consistent over domain-encodings ψX (X ) also be consistent over ψY(Y)? If this is the case, then we know that the high model weight in X̃σ is reciprocated for Ỹσ . Concretely, we are interested in when the following coherence condition holds.
Definition 5 ( -Coherence across soft-structures) Two soft-structures, X̃σ and Ỹσ that share relation-decoders φ, are said to be -coherent with respect to a theory T , if they are both at least -consistent with T .
Coherence between X̃σ and Ỹσ as defined above means that the concept of ordinality that applies to digit ordering can also be applied to block stack height ordering. It is desirable that learning ordinality on the domain of digits produces a coherent concept of ordinality with respect to other ordinal properties, such as height. Since it is possible that ψS(X ) and ψS(Y) produce unique encodings, coherence relies on φ’s ability to generalise over possibly disjoint subsets of Z1.

heading: 4 A PRACTICAL CONSISTENCY LOSS AND -PROXY
text: Calculating Eqn. 4 can quickly become intractable as it involves computing φ beliefs for every grounding and comparing these with every interpretation that is a model of the theory of interest. We therefore want to derive an efficient consistency loss and a calculable -proxy, that can act as a proxy estimate for a soft-structure’s -consistency/coherence with a given theory, without needing access to every model and without requiring an exhaustive calculation over every grounding. In this section, we present the -proxy derivation outline and defer the expanded derivation to Appendix H.
Suppose we have a fixed domain S and a theory T , whose sentences use relations from a signature σ. A structure Sσ will be a model of T if and only if each ground instance of each formula of
1If soft-structure Z̃σ (defined over the full space Z) is consistent, then coherence is guaranteed between all possible soft-substructures.
T is satisfied. Let k ∈ {1, ...,K0} be the index associated with each unique grounding of domain elements to the variables of T . Further, takeBT to be a Boolean random variable denoting the truthvalue of T , so that the probability of the theory being either satisfied (bT = 1) or violated (bT = 0) across ground instances, under a soft-structure S̃σ can be expressed as p(bT |S̃σ, k). Notably, by the definition of a theory, ground instances of formulae will always hold as true for any model of T , i.e. p(bT = 1|Sσ, k) = 1 if Sσ ∈ MTS . Similarly, when S̃σ is consistent with T then we should also find p(bT = 1|S̃σ, k) ≈ 1. We thus define our consistency loss as an expectation of the binary cross entropy between p(BT |Sσ, k) and p(BT |S̃σ, k), which, given p(bT = 0|Sσ, k) = 0 for any k grounding, simplifies to the expected negative log-likelihood of satisfying T under a randomly sampled grounding,
L(T , S̃σ) = Ek∼p(k)[− ln p(bT = 1|S̃σ, k)]. (7)
where p(k) = 1K0 is taken to be uniform over the possible unique groundings. However, we still require an -proxy measure based on this loss, to enable practical evaluation of concept coherence. To achieve this, we define Γ̄S̃σT = exp(−L(T , S̃σ)) and use its relationship with Γ S̃σ T to define a proxy bound,
ln 1
1− ̄ ≥ L(T , S̃σ) (8)
where ̄ ≥ 1− Γ̄S̃σT . In our results, we take -proxy coherence to be the uppermost bound of ln 1
1−̄ between source and target domains.

heading: 5 NEURAL MODEL
text: The critical components of a soft-structure, S̃σ , are its domain-encoder ψS and modular relationdecoders φ. Together these form an autoencoding architecture which, given a domain of images S ⊂ RW×H and with d-dimensional latent space Z = Rd, converts sub-symbolic encodings from ψS into a modular relational representation via decodings for each φr, r ∈ σ. Additionally, to retain information in Z pertaining to S which is beyond the requirements of φ, we include an additional domain-decoder, which produces domain reconstructions Ŝ. The overall neural model is depicted by Figure 1, where we use ψencS to refer to the domain-encoder and ψ dec S for the domain-decoder. To train the neural model, we assume a ground truth interpretation ISσ is given, allowing us to directly maximise Eqn. 3 via negative log-likelihood loss:
LS̃σ = − log p(Sσ|S̃σ), (9)
To obtain informative latent representations for S, we use a Variational AutoEncoder (VAE), specifically the β-VAE, given its simplicity and demonstrated ability to separate distinct factors in the latent representation (Higgins et al., 2017; Burgess et al., 2017; Kingma & Welling, 2014). The β-VAE achieves this by optimising the ELBO objective, LELBOβ-VAE, but with an additional β scalar hyperparameter that can be thought of as a disentanglement pressure, forcing distinct explanatory factors to align with different axes of the latent space. We provide the full ELBO loss, with a detailed explanation, in Appendix C. We combine losses over each model component to give the following aggregate objective: Ljoint = LELBOβ-VAE − λLS̃σ (10) where λ is a scalar weighting parameter. Together with the LELBOβ-VAE, the choice of relation-decoder model will shape how the domain-encodings are structured (Gutiérrez-Basulto & Schockaert, 2018). Our evaluation considers a selection of relation-decoder models that cover a range of representational flexibility. Amongst these is our proposed low-complexity, but nonetheless expressive, Dynamic Comparator (DC) model. The overall DC model is composed of two modes, a distance-based measure, φ†r, that measures the distance between two inputs relative to a reference point, and a steplike function, φ‡r, that determines the sign of the difference between two points, optionally with an offset. Although we can use any functions that have the required characteristics for φ† and φ‡, in this paper we use the following implementation:
φDCr (zi, zj) = ar,0·φ†r + ar,1 · φ‡r where (11) φ†r = f0 ( −ηr,0(‖ur (zi − zj + b†r)‖2) ) and φ‡r = f1 ( ηr,1 · u>r (zi − zj + b‡r) ) . (12)
Here ar = Softmax(Ar) ∈ (0, 1)2 is an attention weighting between the two modes, f0 and f1 are an exponential and sigmoid function, respectively ; ur = Softmax(Ur) ∈ (0, 1)m is an attention mask which is applied to m-dimensional embeddings; b†r, b ‡ r ∈ Rm are learnable bias terms that enables an offset to each mode; and ηr,0 ∈ R+ are non-negative and ηr,1 ∈ R any-valued scalar terms, respectively. Lastly, denotes the Hadamard product and ‖ · ‖2 is the L2-norm. The key innovation behind DC is its ability to model each of the ordinal relations whilst encouraging generalised consistency across the full latent subspace, as defined by each ur. This is achieved without explicit weight sharing, wherein relation-decoders discover parametric relationships between relations from the data. We provide an additional depiction and analysis of DC in Appendix D.1, which further illustrates these effects.

heading: 6 EXPERIMENTAL DESIGN
text: In this section we describe an experimental design used to compare soft-structure coherence when choosing different relation-decoder implementations.
Partial Relation Transfer (PRT): The evaluation involves a proposed PRT task across two softstructures X̃σ and Ỹσ . Each shares a common signature σ and relation-decoders φ but have disjoint domains X and Y , respectively. The experimental procedure involves first learning φ on source domain X , together with its domain-specific autoencoder. In the second phase, we train a new domain-specific autoencoder on the target domain, Y , alongside a selection of the now learned φ relation-decoders but with fixed-parameters. The selected relation-decoders act as training guides for ψencY , whilst held-out relation-decoders can be evaluated against zero-shot transfer performance. For domain X we employ the MNIST handwritten digits data set (LeCun & Cortes, 2010), and for domain Y we use a proposed BlockStacks data set, which includes singular multi-colored cube stacks of differing height, each containing one randomly positioned red cube (see Appendix B for further details and examples). The shared signature includes the ordinal relations, i.e. σ ={G, E, L, S, P}, and is applied to digit ordering in MNIST and red cube position ordering in BlockStacks. We provide results against a theory of ordinality, as explored in Example 1 - we provide a formal specification of this theory in Appendix G. When guiding ψencY to perform a similar mapping to ψencX , i.e. from domain to a similar ordinal subspace as defined by φ, we could use the full φ set of relation-decoders. However, this is not necessary from a logical standpoint, as our system of relations can all be expressed in terms of isSuccessor. We therefore only employ the isSuccessor relation-decoder as a fixed-parameter guide for ψencY .
Neural model components: Together with DC, existing relation-decoder models compared here are: TransR (Lin et al., 2015), HolE (Nickel et al., 2016b), NTN (Socher et al., 2013). We addition-
ally include a basic neural-network baseline, NN. To produce domain-encodings, all experiments use a β-VAE. We provide further details for all models in Appendix D. Due to a convergence issue when using a pretrained DC with fixed parameters, a flexible fitting procedure was necessary, in which we enable the DC parameters to train in the target domain, but with the additional loss term ‖ρ∗ − ρ‖, between pretrained ρ∗ and untrained parameters ρ, respectively. In all cases we evaluated the final parameter values in the target domain and found them to be approximately equivalent to the ρ∗. We did not apply this method to the other models as they were all able to fit the isSuccessor relation in the target domain.
Hyperparameters: In the source domain we explore β values between {1, 4, 8, 12}, and set λ = 103 and in the target domain we first normalise losses (see Appendix D.4) and set β = 10−4 and λ = 10−2 as these produced good reconstructions whilst also ensuring optimisation against LỸσ . In all experiments, we fix Z = R10.

heading: 7 KEY RESULTS
text: In this section, Figure 2 firstly shows standard PRT prediction accuracies per relation in both the source and target domain. Figure 3 then presents consistency losses for three color-coded data splits: data-embeddings (blue), where all inputs are encodings of a domain’s test data; interpolation (green), where we obtain an empirical mean and variance for the domain’s data-embeddings and sample from a corresponding Gaussian distribution; and extrapolation (red), where we sample from regions strictly outside the smallest, axis aligned hyper-rectangle that encloses all data-points. Finally, Table 1 concludes with a clear -proxy coherence comparison between relation-decoders2.
Relation-decoder PRT accuracy performance: Figure 2-top provides relation-decoder prediction accuracy in both the source MNIST (left), and target BlockStacks (right), domains. Key observations are that DC produces excellent PRT performance, whilst NN, NTN and HolE all see some degradation from their source accuracies on relations other than isSuccessor. TransR seems to maintain an target accuracy profile similar to its performance in the source domain, but this is significantly below the performance of other models in the source domain.We include β’s impact on these performances in Figure 2-bottom. Barring DC which has little discernible change in either domain, PRT performance is significantly impacted by β in all models, but has little effect in the source domain. TransR shows a strong positive correlation between target domain accuracy and β, whereas the remaining models produce their best PRT performances with intermediate disentanglement pressure.
2We take φr prediction values above 0.5 to signify a truth prediction and those below 0.5 to signify falsity.
Consistency Across (Con-A): To interrogate further how β affects each model, Figure 3-top presents consistency losses against formulae that constrains truth value assignments across relations, under a theory of ordinality, referred to as Con-A3. Results are referenced to both source (left) and target (right) domain embeddings. With reference to the source domain, we note that DC shows excellent Con-A in all regions. Most other models have worse interpolation and extrapolation consistency. Increasing β appears to improve interpolation and extrapolation performance for models NN, NTN and TransR, but there are indications that this trend does not persist into the largest β = 12 value. On the other hand, HolE shows a negative correlation between β and Con-A performance, across all data-splits. Although DC sustains impressive Con-A results for target domain data-embeddings (right), results for all other models are notably worse with respect to their source data-embeddings performances and are instead comparable with their interpolation or extrapolation results in the source domain. It may therefore be possible to anticipate/diagnose poor transfer performance by evaluating interpolation and extrapolation consistency, which suggests that DC’s strong consistency generalisation indeed enables it to learn a transferable concept.
Consistency Individual (Con-I): Figure 3-bottom presents stacked consistency losses, for formula: transitivity (white), asymmetry (magenta) and reflexivity (black); results are averaged over individual relations and are together grouped under label Con-I, given that they refer to constraints on individual relations. Losses are again partitioned between source domain (left) and target domain (right). We firstly observe that DC and NN share the best overall Con-I performance profiles, with TransR following closely. DC and TransR both show comparable data-embedding versus interpolation/extrapolation performance, whereas NN, NTN and HolE suffer from degradation across these splits. Interestingly, these results show that: DC only suffers on transitivity, NN and TransR mainly struggle to model transitivity but show additional loss for asymmetry and HolE demonstrates difficulty in modelling each of the Con-I sub-stack. With regards to β’s impact, it is not possible to determine a correlation for DC. However, NN and NTN demonstrate a negative correlation of β against overall Con-I, with comparable response for each underlying sub-stack. TransR shows a significant Con-I extrapolation improvement with increased β and HolE is for the most part adversely impacted as β is increased. Similar trends can be seen for target Con-I performance. However, notably, many models show improvements, in particular in Con-I (asymmetry). This could be due to more precise target domain data-embeddings, as a result of using a single pretrained relation-decoder.
-proxy coherence: Table 1 provides a comparison between optimal -proxy coherences achieved for each relation-decoder model, as defined in Section 4. Results are partitioned according to each consistency type and an Aggr(egate) value, which gives best summed consistency, together with best β = β∗ values. DC clearly outperforms all other models in -proxy coherence across all types. NN achieves strong aggregated -coherence compared with NTN, HolE and TransR outperforming across Con-A and Con-I-tr. Although NTN and HolE have similar aggregate -proxy coherence,
3Truth tables for each consistency formula is given in Appendix G
TransR performs generally worse. This may be caused by TransR producing weaker beliefs in comparison to other models, as this can result in a worse overall consistency level. Looking at β∗ profiles, we see that most models achieve optimum aggregate -proxy coherence at β = 8, other than DC which performs better at β = 1. Overall, this is in agreement with the β profiles given by Figure 2-bottom (right). However, we can see that β∗ profiles for Con-A based -proxy coherence are in more direct agreement - as TransR achieves its best at β = 12 - suggesting that Con-A invariance is more important to concept transfer.

heading: 8 DISCUSSION AND CONCLUDING REMARKS
text: In this work, we introduced the notion of a soft-structure, which can learn a structure over real-world domains, through the use of a domain-encoder coupled with modular relation-decoders. We subsequently provided formal definitions, defining what it means for a concept to be coherent in terms of domain-invariance of soft-structure consistency with respect to a theory. We then outlined a neural model and experimental procedure that together allowed us to investigate how concept coherence differs when choosing different implementations for underlying relation-decoders and its impact on concept transfer. Our results suggest that increasing regularisation over relation-decoder models, either in the form of disentanglement pressure or relation-decoder model capacity, seems to improve their ability to learn coherent concepts. Firstly, strong PRT transfer for DC and NN (given an appropriately high β setting) showed that both relation-decoder models are able to minimise Eqn. 9 in the source domain and retain good performance in the target domain. Consistency profiles over partial theories (subsets of the sentences that comprise the overall theory of ordinality), covering multiple data-splits, then further suggested that a relation-decoder’s ability to retain consistency over interpolated/extrapolated regions with respect to the observed data-encodings (during training) greatly impact concept coherence. Finally, an -proxy coherence comparison showed that DC achieved excellent coherence with an aggregated -proxy of 0.34, which mirrors its strong PRT performance. NN achieved a score of 34.09, which, although significantly worse than DC, is a marked improvement over the remaining models. All in all, the empirical analysis in this work provides strong evidence towards the hypothesis that the transferability of a concept depends on its coherence, as measured by the retention of consistency across domains.
Limitations and future work: Firstly, this work only considered binary relations, and in particular unary relations, such as digit classification, are not considered. Additionally, we have only considered a fixed signature which is learned “all at once” in a source domain. In practical applications, however, it is quite possible that ordinality would be discovered gradually, either through incremental learning of the relations that form a ‘complete’ signature for ordinality, or through gradual refinement of pre-learned relations after being progressively exposed to different contexts in which ordinality applies. This necessitates a continual learning procedure which has not been explored in this work and would be useful as an addition in future work. Further, even in a single domain, ordinality can be applied to multiple properties (e.g. for BlockStacks we have: block stack height, block size, position of stacks, etc.) and future work can explore our framework’s ability to initialize multiple instances of our signature, each applied to a different ordinal property. Lastly, we have only explored a signature for ordinality, whereas other fundamental properties are easy to find, such as periodic (e.g. rotation) and unordered categorical (e.g. shape) properties. These aspects are not explored in this work and would certainly be interesting for future work.

heading: 9 ETHICS STATEMENT
text: The authors declare that this work does not include any of the following: involvement of human subjects, sensitive data, harmful insights, methodologies and applications. The results, data sets and methodologies are objectively nondiscriminatory, unbiased and fair. This work does not breach any privacy or security guidelines or laws, nor any other legal restrictions. The authors declare that there are no conflicts of interest and/or external motivations through sponsorship.

heading: 10 REPRODUCIBILITY STATEMENT
text: To ensure reproducibility of this work, the experimental code has been open sourced, together with the proposed BlockStacks data set and rendering code;4 references for all other employed data sets are provided in the main text. Hyperparameter configurations are specified in both Section 6 and Appendix D and the experimental procedure is detailed in Section 6.

heading: A RELATED WORK
text: Relational representations play a prominent role in Knowledge Graph Embedding, wherein sets of relation-decoders are jointly learned in order to obtain a semantic latent representation for data points (Socher et al., 2013; Trouillon et al., 2016; 2019; Bordes et al., 2013; Nickel et al., 2016a; Wang et al., 2017; Dai et al., 2020; Kazemi & Poole, 2018; Abboud et al., 2020). Although these typically do not use a shared autoencoder as we do in this paper, Schlichtkrull et al. (2018) did adopt an autoencoding framework, where a graph neural network is used as the encoder, however they did not work with visual data and the model was only applied to single data sets. Similarly, disentanglement is also concerned with semantic representation learning (Bengio et al., 2013) , and has been explored using a variety of methods including both Generative Adversarial Networks (Chen et al., 2016) and VAEs (Burgess et al., 2017; Higgins et al., 2017; Chen et al., 2018; Ridgeway & Mozer, 2018; Eastwood & Williams, 2018; Kumar et al., 2018; Locatello et al., 2019). Disentangled representations have been evaluated in terms of there transferability in (van Steenkiste et al., 2019; Steenbrugge et al., 2018; Locatello et al., 2020). A bridge between these two fields, wherein relation-decoders are employed as a semi-supervision to VAEs can be found in (Karaletsos et al., 2016; Chen & Batmanghelich, 2020; 2019), where (Karaletsos et al., 2016) use multiple relationdecoders but compute a triplet comparison based query and (Chen & Batmanghelich, 2020; 2019) only include a single binary relation and use function forms that are not sufficient to model the full set of relations that we include in this work. Neither presents a comprehensive analysis of resulting concept coherence. Lastly, we note that our experimental setup is most remnant of domain adaptation (Redko et al., 2019). To the best of our knowledge, no work has compared relation-decoders in their ability to learn coherent concepts, as measured by their consistency across domains.

heading: B BLOCKSTACKS DATASET DESCRIPTION
text: The BlockStacks dataset consists of 12,000 images (200×200 pixels but resized in code to 128×128) of individual block stacks, of varying height (between 1-10 blocks), block colors (uniformly sampled from options: { gray, blue, green, brown, purple, cyan, yellow}) and position (uniformly sampled from x, y range (-3,-3) to (3,3)), but with the requirement that each instance consists of a single red block at a random height (see Figure 4 for example images). These were rendered using the CLEVR rendering agent with the help of code from (Asai, 2018). The dataset is divided into 9000:1500:1500 train, validation and test splits.
C EXPLANATION OF THE β-VAE
The VAE is derived by introducing an approximate posterior qα(Z|X), from which a lower bound (commonly referred to as the Evidence LOwer Bound (ELBO)) on the true marginal log pθ(X) can be obtained by using Jensen’s inequality (Kingma & Welling, 2014). The VAE maximises the log-probability by maximising this lower bound, given by:
LELBOβ-VAE = Eqα(Z|X)[log pθ(X|Z)]− βDKL(qα(Z|X)‖pθ(Z)), (13)
where qα(Z|X) is typically modelled as a neural-network encoder with parameters α. Similarly pθ(X|Z) is often modelled as a neural-network decoder with parameters θ and is calculated as a Monte Carlo estimation. A reparameterization trick is used to enable differentiation through an otherwise undifferentiable sampling from qα(Z|X) (see (Kingma & Welling, 2014)). In the β-VAE (Higgins et al., 2017; Burgess et al., 2017), an additional β scalar hyperparameter was added as it was found to influence disentanglement through stronger distribution matching pressure with respect to the prior pθ(Z), where this prior is typically set to an isotropic zero-mean Gaussian N (0, I)). When β = 1 we obtain the standard VAE objective (Kingma & Welling, 2014).

heading: D MODEL DESCRIPTIONS
text: In this section we firstly present an in-depth analysis of the key innovations presented by DC which provides insight into how it can learn a coherent notion of ordinality. We then provide model details for each of the compared relation-decoders in the main results and the β-VAE architecture that we employ for each data set.
D.1 DYNAMIC COMPARATOR ANALYSIS
Figure 5 depicts how DC is able to learn the isGreater, isLess, isEqual, isSuccessor and isPredecessor family of binary ordinal relations, assuming each corresponding relation-decoder has learned a common one-hot mask on the zeroth dimension i.e. uG = uE = . . . = uP = [1, . . . , 0], such that activations only depend on the zi,0 − zi,1 difference. An important capability of DC is its ability to select, via ar an appropriate functional mode, either φ†r or φ ‡ r, depending on the type of relation it needs to model. As shown by Figure 5, isEqual exhibits its reflexive, symmetric and transitive characteristics, whilst isGreater and isLess both carry transitivity but are asymmetric and irreflexive. Furthermore, the use of a subtraction between zi and zj (which, via mask u ends up only being a subtraction between their zeroth dimensions) leads to a relative comparison, not an absolute comparison, which generalises to arbitrary zi and zj sampled from anywhere in Z . Note that there is no built in parameter sharing, meaning each relation-decoder (for each individual relation r) is trained independently and has its own set of ar,ur, ηr,0, ηr,1, b†r and b ‡ r parameters. However, our experiments show that DC reliably obtains settings such that e.g. uG = uE, or aG = aL = [0, 1], or b ‡ G = −b ‡ L and so on. DC is thus able to discover the interdependencies between families of relations. By learning to indirectly ‘tie’ together parameters in this way, whilst still being expressive enough to model each type of relation, DC can facilitate a data-driven binding between relation-decoder outputs. This helps ensure consistent generalisation across a latent subspace, as defined by the common/overlapped ur masks.
D.2 RELATION-DECODER IMPLEMENTATIONS
TransR (Lin et al., 2015): φTransRr (zi, zj) = ‖hr + r − tr‖22
with, hr = Mrzi and tr = Mrzj .
where for zi, zj ∈ Rdz vectors, Mr ∈ Rdz×dz and r ∈ Rdz . As we want to obtain a [0,1] output, we modify TransR through φTransR +
r = σ(c−φTransRr ), where σ is a sigmoid function and c is a scalar that ensures that at φTransRr (zi, zj) = 0, then φ TransR+ r (zi, zj) ≈ 1. In all experiments we set c = 10.
NTN (modified version of (Socher et al., 2013) from (Donadello et al., 2017; Serafini & Garcez, 2016)):
φr(z1, . . . ,zn) = σ ( u>r [tanh(z c>Mrz c + Vrz c + br)] )
(14)
where ur ∈ Rk,Mr ∈ Rn·dz×n·dz×k,Vr ∈ Rk×n·dz) and br ∈ Rk. The only hyperparameter to consider is k, which controls the NTN’s capacity - in all experiments, we set this to 1. If k > 1, zc>Mrz
c produces a k-dimension vector by applying the bilinear operation to each of the k Mr slices. Here zc ∈ Rn·dz is a concatenation of the inputs z1, . . . ,zn, which was introduced in (Donadello et al., 2017; Serafini & Garcez, 2016). In contrast, the original NTN (see (Socher et al., 2013)) is only applicable to binary relations and does not include the outer sigmoid.
HolE (Nickel et al., 2016b): φHolEr (zi, zj) = σ(r >(zi ? zj))
where r ∈ Rdz and ? : Rdz × Rdz → Rd denotes the circular correlation operator and is given by,
[zi ? zj ]k = d−1∑ m=0 zi,mzj,(k+m) mod d
NN: a simple four-layer neural-network with layer sizes lin = 2dz, l1 = 2dz and l2 = dz , with ReLU activations (Nair & Hinton, 2010). The final output layer, lout, is a single value passed through a sigmoid function, to bound the output within (0,1).
D.3 β-VAE CONFIGURATION
The model configurations used for both MNIST and BlockStacks data sets are given in Table 2.
D.4 Ljoint CONFIGURATION
In the source domain, we vary β values between {1, 4, 8, 12} and fix λ = 103. In the target domain, we fix β to 10−4 and λ = 10−2 and normalise the LELBOβ-VAE reconstruction term by dividing by a factor 1√
H·W ·C , for height H , width W and color channels C, and normalize the distribution matching term by a factor 1dz , for latent representation size dz .
To train relation-decoders over a given domain S, it is necessary to supervise estimates of φr(ψ
enc S (O)), O ∈ S2, against corresponding ground-truth labels, γrO,Sσ . However, doing so for
every O ∈ S2 can easily become intractable and we instead only sample a subset of possible S2 tuples. Our sampling strategy involves first selecting a ratio R = |B||S| where B ⊂ S 2 is a set of O tuples. We then sample relation-decoder specific subsets Br where |Br| = |B||σ| , to ensure a balanced distribution of tuples between relation-decoders. Furthermore, we ensure that each Br contains a balanced ratio of γrO,Sσ = 1 versus γ r O,Sσ = 0 instances. We found that each |Br| set can be small without jeopardising the final relation-decoder performance level, allowing us to use R = 1 for MNIST experiments and R = 3 for BlockStacks experiments.
Finally, in all experiments we use a β-VAE trained for up to 300,000 steps, following accepted practice from (Locatello et al., 2019; Steenbrugge et al., 2018), together with any included relationdecoders. However, to ensure computation efficiency across experiments, we employ an early stopping procedure, where if the validation score does not increase over 30 and 120 training epochs for MNIST and Blockstacks experiments, respectively, we end the training early.

heading: E SUPPLEMENTARY RESULTS
text: β effect on intrinsic relation-decoder characteristics: We have seen how β impacts PRT accuracy but it is not clear how this is facilitated. To understand the way in which β affects each relation-decoder we produce a gradient-conformity evaluation, based on the intuition that collections of relation-decoder outputs will have to shift together in order to maintain consistency, facilitated
via a certain conformity in their gradients of input against output. For instance, suppose we have xi,xj ∈ X such that φG(xi,xj) ≈ 0 and φE(xi,xj) ≈ 1. If we then take xi,xk ∈ X such that φG(xi,xj) ≈ 1, then we must have φE(xi,xj) ≈ 0. In fact, any time φG outputs close 1, we require φL and φE output close to 0, since only one of these three relations can be true for any common arguments. At the decision boundaries, we require that their collective outputs conform such that they are each modified appropriately, ensuring that they continue to satisfy any applicable constraints. We therefore compute a gradient based analysis for arbitrary relation-decoder inputs against the overall T belief state, which should be consistently ≈ 1, to see how much this varies as β is increased. Gradient-conformity (GC) is calculated as:
GC = ∣∣∣∣ dTi dj||di||2 ||dj ||2 ∣∣∣∣ , where di = dφridzc ∣∣∣∣ zc=zcn and dj = dφrj dzc ∣∣∣∣ zc=zcn , ∀i 6= j (15)
where zcn is the concatenation of the nth sample of zi and zj from the latent space (and, more specifically, a particular data split). Figure 6 presents source domain referenced GC measures for each model, with the same data split schematic as in Figure 3. We see that for DC, GC is close to 1 for all β with no discernible change. All other models show a weaker GC with positive correlation between GC and β. TransR and NN achieve significantly higher GC than NTN and HolE. It appears that models that achieve a GC greater than 0.5 perform better at the overall PRT learning task.

heading: F HOW DOES EACH MODEL IMPACT THE RETENTION OF
text: DOMAIN-DEPENDENT INFORMATION
Figure 7 shows results for BlockStacks overall block height prediction accuracy when training on fixed encodings of each block stack, after isSuccessor relation-decoder alignment as been applied (using a pretrained fixed-parameter relation-decoder). Note β is fixed in the target domain, so the only moving part are the choices of pretrained models which have been previously trained with varied source β values. Note also that DC has an unfair advantage here, as the steered fitting approach allows more flexibility to the VAE learning phase - for this reason the result is only included in the appendix. Since we are interested in capturing general representations that encode both domaindependent and domain-independent information, we use each target encoder ψtenc obtained from each PRT experiment and produce encodings for the full BlockStacks test set. The resulting encodings are then divided into a new train and test subset, used to train both a Sci-Kit Learn Linear regressor and Support Vector Machine regressor with a RBF kernel (Pedregosa et al., 2011). We present the resulting Mean Squared Errors (MSE) in Figure 7, with Ordinary Least Squares (OLS) (a) and Support Vector Regression (SVR) (b).
There are a number of noteworthy details: firstly, DC shows no dependence on β and leads to a lower MSE across all settings; second, excluding DC, for all models we observe an optimum MSE at β = 8, with TransR reaching DC MSE performance for OLS and NN doing the same for SVR. These results indicate that lower MSE can be obtained by using non-linear regression, which indicates that to some degree, the block stack height factor is not encoded linearly, regardless of selected model. Next, by contrasting with Figure 6, these results suggest that models with higher GC lead to embeddings that are more amenable to domain-specific factor prediction. However, the parabolic trend, where increasing β to 12 leads to an increase in error, is in agreement with Figure 2-bottom-right, which showed that most models do not improve at PRT for the largest β. This is perhaps due to a loss of mutual information between input and latent representation, as the distribution matching loss outweighs reconstruction in the LELBOβ-VAE.
G SPECIFICATION FOR THEORY OF ORDINALITY
To support our claim that we can use only the isSuccessor relation as the target encoder guide due to its logical relationship the remaining relations, we include here the logical clauses:
∀i, j, k (isSuccessor(i, j) ∧ isSuccessor(k, j)→ isEqual(i, k)) ∀i, j (isSuccessor(i, j)→ isGreater(i, j))
∀i, j, k (isSuccessor(i, j) ∧ isGreater(j, k)→ isGreater(i, k)) ∀i, j (isSuccessor(i, j)↔ isPredecessor(j, i)) ∀i, j (isPredecessor(i, j)→ isLess(i, j))
∀i, j, k (isPredecessor(i, j) ∧ isLess(j, k)→ isLess(i, k)).
Therefore, by knowing all of the successor relations between data instances, it should be possible to infer the remaining relationships that they share.
For completeness, we provide the truth tables for each of the sub-theories that our consistency losses evaluate against. We only include configurations that are valid under the constraints, indicated by ⊂ T = T , where this notation highlights the fact each incomplete set of constraints form a subset of the overall theory T . Firstly, the truth-table that describes constraints shared between relation truth-values is given by the following, ∀i, j:
G(i, j) E(i, j) L(i, j) S(i, j) P(i, j) ⊂ T T F F F F T T F F T F T F T F F F T F F T F F T F F T F T T
where we use the same relation abbreviations as in the main text results.
Next, we provide each of the three consistency individual (Con-I) truth-tables. These are referred to as being “individual” due to the fact that they describe constraints applied to the truth-state of a single relation. For transitivity, given by the rule e.g. G(i, j) ∧ G(j, k) → G(i, k), we have that ∀i, j:
G(i, j) G(j, k) G(i, k) ⊂ T F F F T F F T T T F F T T F T T F T F T F T T T T T T T
(16)
For asymmetry, where S(i, j)→ ¬S(j, i), we have ∀i, j:
S(i, j) S(j, i) ⊂ T F F T T F T F T T
(17)
.
Finally, for reflexivity, given by E(i, i) → > (in this case describing that an object is always equal to itself) we have ∀i:
E(i, i) ⊂ T T T
(18)
Truth-table matrices for each of the above truth-tables can be obtained by replacing T with 1 and F with 0. We provide the full set of individual constraints that are applicable to each relation covered in this paper are given by Table 3.

heading: H EXPANDED -PROXY DERIVATION
text: In this section, we present the expanded justification for reporting − ln ̄ consistency and coherence as a proxy for -consistency/coherence as defined in Section 3. For notational clarity, in the following we omit ψS , such that φr(ψS(O)) is abbreviated to φr(O).
In the following, we make no assumptions about the sizes of domain S, signature σ and arities of each r ∈ σ. Further, we take T to be an arbitrary theory over σ consisting of universally quantified formula, and the validity of each ground instances of atomic formula with respect to T , can be expressed by a single ground truth-table matrix, T ∈ {0, 1}K0×K1×K2 , wherein each slice, Tk,:,: gives a unique grounding of domain objects to the variables, v, required by T . For each grounding of the K0 = |S||v| possible groundings, there are K1 = 2l unique truth-assignments to the l atomic formulae that constitute T , giving K2 = l+ 1 assignments per Tk,t,: row - one per atomic formulae and an additional value to denotes whether the particular row satisfies T . T can be obtained by taking any truth-table from the previous section and switching true (T) for 1 and false (F) for 0, and producingK0 copies for each assignment of domain elements to the variables. Given this truth-table matrix, notice that a structure Sσ can be composed by selecting a single row of T for each grounding (kth slice), giving a vector ckt = Tk,t,1:l. If the structure is a model of T , i.e. Sσ ∈ MTS , then only rows with Tk,t,K2 = 1 are allowed. Taking t
+ to be the set of rows such that Tk,t,K2 = 1 (which is identical for each k) for t ∈ t+, we can then rewrite ΓS̃σT in terms of samples from T :
ΓS̃σT = ∑
Sσ∈MTS
∏ r∈σ ∏ O∈Sar(r) φr(O) γrO,Sσ (1− φr(O))1−γ r O,Sσ (Eqn. 3)
= ∑
Sσ∈MTS
K0∏ k=1 ∑ t∈t+ 1t′k,Sσ (t) l∏ m=1 f(φrm , Okm, cktm) N(φrm ,Okm,cktm,Sσ)−1 (19)
with f(φrm , Okm, cktm) = φrm(Okm)
cktm (1− φrm(Okm))1−cktm . (20) In the above, 1t′k,Sσ (t) is an indicator function which equals 1 if t = t ′ k,Sσ and 0 otherwise, for active row t′k,Sσ under structure Sσ and grounding k. 1t′k,Sσ (t) has the role of only including the single summand where t corresponds with t′k,Sσ . N(φrm , Okm, cktm,Sσ) is a function that counts the number of repeat products of term f(φrm , Okm, cktm), such that the appropriate root can be applied. We use rm to denote the relation for atomic formula at column m and Okm its corresponding arguments, under grounding k; and we use cktm to denote the truth-assignment of the atomic formula for column m, as designated by row t.
At this point, we are left with an expression for ΓS̃σT in terms of truth-table matrix T entries, which is more reminiscent of L(T , S̃σ) as defined in Section 4. However, we must go further to expose the relationship between ΓS̃σT and L(T , S̃σ) for arbitrary T expressed by T . We will now show that the consistency loss L(T , S̃σ) gives the negative log-likelihood of satisfying T given a grounding k ∈ {1, . . . ,K0}, which can be further seen as a relaxation of ΓS̃σT to sum over all rows t ∈ t+ and without normalising via the N(φrm , Okm, cktm,Sσ)−1 exponent. With Boolean random variable BT denoting whether T is (bT = 1) or is not (bT = 0) satisfied, the consistency loss for a softstructure S̃σ against theory T is given by,
L(T , S̃σ) = Ek∼U [{1,...,K0}][H(p(BT |Sσ, k), p(BT |S̃σ, k))] Eqn. 7 base
which can be expanded to,
L(T , S̃σ) = − K0∑ k=1 1 K0 p(bT = 1|Sσ, k) ln p(bT = 1|S̃σ, k) (21)
+ (1− p(bT = 1|Sσ, k)) ln 1− p(bT = 1|S̃σ, k).
where Sσ ∈ MTS . Given Sσ ∈ MTS , then p(bT = 1|Sσ, k) = 1 always holds, which means the negative case in Eqn. 21 can be ignored, yielding the following simplified form:
L(T , S̃σ) = − K0∑ k=1 1 K0 ln p(bT = 1|S̃σ, k)
= −Ek∼U [1,...,K0][ln p(bT = 1|S̃σ, k)]. Eqn. 7
and so L(T , S̃σ) is simply the negative log-likelihood of sampling a satisfied theory (bT = 1) from soft-structure S̃σ , for randomly sampled grounding k. Next, we show the similarities between L(T , S̃σ) and ΓS̃σT by looking at the likelihood p(bT = 1|S̃σ, k). First, we define Γ̄ S̃σ T by isolating the likelihood:
exp(−L(T , S̃σ)) = K0∏ k=1 p(bT = 1|S̃σ, k) 1 K0
. = Γ̄S̃σT (22)
We then expand p(bT = 1|S̃σ, k) to:
p(bT = 1|S̃σ, k) = K1∑ t=1 p(bT = 1|ckt)p(ct|S̃σ, k)
= ∑ t∈t+ p(ckt|S̃σ, k) (23)
where t+ is defined as before. For all other t 6= t+, p(bT = 1|ckt) = 0 and so this acts as a filter, yielding:
Γ̄S̃σT = K0∏ k=1 ∑ t∈t+ p(ckt|S̃σ, k) 1 K0 . (24)
p(ckt|S̃σ, k) is calculated by evaluating the belief of each relation-decoder against the expected truth-assignment as defined by truth-table row ckt:
p(ckt|S̃σ, k) = l∏
m=1
φrm(Okm) cktm(1− φrm(Okm))1−cktm
= f(φrm , Okm, cktm)
where rm is the relation for atomic formula associated with columnm (which is the same for each k slice and t row) and Okm is the grounding of this entry for slice k (which is the same across rows). Putting it all back together, we finally have that:
Γ̄S̃σT = K0∏ k=1 ∑ t∈t+ l∏ m=1 f(φrm , Okm, cktm) 1 K0 , (25)
which makes the similarities between ΓS̃σT and Γ̄ S̃σ T clear and exposes their relationship.In particular, for the special case where |MTS | = 1, the outer sum for Γ S̃σ T can be removed, and the remaining differences between ΓS̃σT and Γ̄ S̃σ T are the sum over t
+ rows and difference in exponent over f(φrm , Okm, cktm). For ΓS̃σT to be maximised, through p(Sσ|S̃σ) ≈ 1, we would find that
S̃σ maximally supports only the rows associated with Sσ for each k grounding. Notice that Γ̄S̃σT is again bound to (0,1) and achieves Γ̄S̃σT ≈ 1 when Γ S̃σ T ≈ 1. We use the correspondence between ΓS̃σT and Γ̄ S̃σ T to define a practical -proxy consistency measure as follows. We firstly re-express
-consistency/coherence but for Γ̄S̃σT and a different ̄. We then trace this back to L(T , S̃σ) so a bound in terms of the consistency loss can be reported as the overall -proxy. Together this yields the following:
̄ ≥ 1− Γ̄S̃σT
ln 1
1− ̄ ≥ − ln(Γ̄S̃σT )
≥ L(T , S̃σ) (26)
and we arrive at an -proxy of the form ln 11−̄ , which is reported in the main text.

Review summaries:
The main motivation of this paper is the observation that once we humans have learned the meaning of a relation, we can apply it to different contexts and scenarios effortlessly. For instance, once we understand the meaning of a concept "larger than", we can apply this in many context such as integers, person lengths, building sizes, etc. The authors argue that it is interesting to study how we can build learning systems that similarly are able to transfers learned relations across domains.  In order to express relations, a language based on first-order logic is developed, and semantics is given to this language using an interpretation function, mapping relation symbols to the tuples of elements over which these relations hold. In order to facilitate learning these relations, the authors develop a quantitative account of the logic. They develop so-called "soft structures", which can essentially learn a symbolic structure over real-world domains. This learning is done using the Variational Auto-Encoder framework, where an encoder maps the real-world input (for instance, an image), to a latent space Z. The interpretation of relations over the real world data is then implemented by "relation decoders" (one decoder for each relation), which are belief functions assigning a number in (0, 1) to relations. In order to train such models, they define a measure on how well a soft structure support a symbolic structure, and they use this as a term in the loss function. They also provide a formal account of what it means for two soft structures to be "coherent", which intuitively mean that relations that apply to one domain (e.g., ordinality in numbers) also can be applied to another domain (e.g., ordinality in block sizes).  They authors then derive an efficient consistency loss, since the original one is intractable because it requires computing beliefs over all interpretations of the relations. They then propose a Dynamic Comparator relation decoder model, and a new learning task called "Partial Relation Transfer learning". They compare against several existing relation decoder models, and empirically show that their relation decoder retains coherence across domains, while achieving better transfer learning performance.

This work introduces a more general definition of structures namely soft structures that can be used to learn structure over richer domains in a data-driven manner. Further, they introduce suitable definitions of coherence and consistency for the proposed soft structures. Next, the paper proposes a neural architecture that is essentially, jointly learning relation encodings jointly with representations for set elements. The key argument of the paper is that transfer is to be expected across shared relations despite their overloaded usage across domains — for example, greater than (GE) can have completely different interpretations in the source / target domains. 

This paper proposes a loss based on theory for the relation prediction problem, using the sentence of the relationship as the supervision. The paper designs an relation transfer learning experiment. By supervising one relation, other relations can be transferred.