SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mathcal{T} \in \mathbb{R}^{d_e \times d_r \times d_e}$  denotes the core tensor,  $\times_n$  denotes the tensor product along the  $n$ -th mode, and  $d_e, d_r$  denote respectively the embedding sizes of entities and predicates (which might not be equal). To see how this parametrization generalises that of CP, RESCAL and COMPLEX, consider for example the score function of CP on  $d$ -dimensional embeddings. It can be obtained by (i) setting the core tensor  $\mathcal{T}$  to be a diagonal tensor having ones on the superdiagonal, and (ii) having two distinct embedding instances for each entity that are used depending on their role (either subject or object) in a triple. The embeddings  $\mathbf{e}_s, \mathbf{e}_o \in \mathbb{R}^{d_e}$  (resp.  $\mathbf{w}_r \in \mathbb{R}^{d_r}$ ) are rows of the matrix  $\mathbf{E} \in \mathbb{R}^{|\mathcal{E}| \times d_e}$  (resp.  $\mathbf{W} \in \mathbb{R}^{|\mathcal{R}| \times d_r}$ ), which associates an embedding to each entity (resp. predicate).
Constructing the circuit. For the construction of the equivalent circuit it suffices to (i) create an input unit for each  $i$ -th entry of an embedding for subjects, predicates and objects, as to implement a look-up table that computes the corresponding embedding value for an entity or predicate, and (ii) transform the tensor multiplications into corresponding sum and product units. We start by introducing the input units  $l_{i}^{S}, l_{j}^{R}$  and  $l_{k}^{O}$  for  $1 \leq i, k \leq d_{e}$  and  $1 \leq j \leq d_{r}$  as parametric mappers over variables  $S, R$  and  $O$ , respectively. The input units  $l_{i}^{S}$  and  $l_{k}^{O}$  (resp.  $l_{j}^{R}$ ) are parameterised by the matrix  $\mathbf{E}$  (resp.  $\mathbf{W}$ ) such that  $l_{i}^{S}(s; \mathbf{E}) = e_{si}$  and  $l_{i}^{O}(o; \mathbf{E}) = e_{oi}$  for some  $s, o \in \mathcal{E}$  (resp.  $l_{j}^{R}(r; \mathbf{W}) = w_{ri}$  for some  $r \in \mathcal{R}$ ). To encode the tensor products in Eq. (4) we introduce  $d_{e}^{2} \cdot d_{r}$  product units  $\phi_{ijk}$ , each of them computing the product of a combination of the outputs of input units.
$$
\phi_ {i j k} (s, r, o) = l _ {i} ^ {S} (s) \cdot l _ {j} ^ {R} (r) \cdot l _ {k} ^ {O} (o)
$$
Finally, a sum unit  $\phi_{\mathrm{out}}$  parameterised by the core tensor  $\mathcal{T} \in \mathbb{R}^{d_e \times d_r \times d_r}$  computes a weighted summation of the outputs given by the product units, i.e.,
$$
\phi_{\mathrm{out}}(s,r,o) = \sum_{\substack{(i,j,k)\in \\ [d_{e}]\times [d_{r}]\times [d_{e}]}}\tau_{ijk}\cdot \phi_{ijk}(s,r,o)
$$
Table A.1: Score functions as compact circuits. Asymptotic size of circuits encoding the score functions of CP, RESCAL, COMPLEX and TUCKER, with respect to the embedding size. For TUCKER,  $d_{e}$  and  $d_{r}$  denote the embedding sizes for entities and predicates, respectively.
KGE Model	Circuit Size	KGE Model	Circuit Size
CP	O(d)	RESCAL	O(d2)
COMPLEX	O(d)	TUCKER	O(d2·dr)
where  $[d]$  denotes the set  $\{1,\dots ,d\}$  and  $\tau_{ijk}$  is the  $(i,j,k)$ -th entry of  $\mathcal{T}$ . We now observe that the constructed circuit  $\phi_{\mathrm{out}}$  encodes the TUCKER score function (Eq. (4)), as  $\phi_{\mathrm{TUCKER}}(s,r,o) = \phi_{\mathrm{out}}(s,r,o)$  for any input triple  $(s,r,o)\in \mathcal{E}\times \mathcal{R}\times \mathcal{E}$ .
Circuit evaluation and properties. Evaluating the score function of TUCKER corresponds to performing a feed-forward pass of its circuit representation, where each computational unit is evaluated once, as we illustrate in Fig. A.1. As such, the cost of evaluating the score function is proportional to the size of its circuit representation, i.e.,  $\mathrm{cost}(\phi) \in \Theta(|\phi|)$  where  $|\phi| \in \mathcal{O}(d_e^2 \cdot d_r)$  is the number of edges. In Table A.1 we show how the sizes of the circuit representation of the other score functions increase with respect to the embedding size. Finally, since each product unit  $\phi_{ijk}$  is defined on the same scope (see Def. 1)  $\{S, R, O\}$  and fully decompose it into its inputs (i.e., into  $\{S\}, \{R\}, \{O\}$ ), and the inputs of the sum unit  $\phi_{\mathrm{out}}$  are all defined over the same scope, we have that the circuit satisfies smoothness and decomposability (Def. 2).
Furthermore, in Lem. A.1 we show that the circuit representations of CP, RESCAL, TUCKER and COMPLEX and the proposed GeKCs (Section 4) satisfy a structural property known as omnicompatability (see Def. B.2). In a nutshell, the score functions of the aforementioned KGE models and GeKCs are circuits that fully decompose their scope  $\{S,R,O\}$  into  $(\{S\},\{R\},\{O\})$ . The satisfaction of this property will be useful to prove both Thm. 1 and Thm. 2 later in this appendix.
Lemma A.1 (KGE models and derived GeKCs are omni-compatible). The circuit representation of the score functions of CP, RESCAL, TUCKER, COMPLEX and their GeKCs counterparts obtained by non-negative restriction (Section 4.1) or squaring (Section 4.2) are omni-compatible (see Def. B.2).
Proof. To begin, we note that to comply with Def. B.2 every omni-compatible circuit shall contain product units that fully factorise over their scope. In other words, for every product unit  $n$  with scope  $\operatorname{sc}(n) = \mathbf{X}$ , its scope shall decompose as  $(\{X_1\}, \{X_2\}, \ldots, \{X_{|\operatorname{sc}(n)|}\})$ . To see why, consider a circuit  $\phi$  with a product unit  $n$  whose scope is decomposed as  $\operatorname{sc}(n) = (\mathbf{X}, \mathbf{Y})$ . It is easy to construct another circuit  $\phi'$  that is not compatible with  $\phi$  by having a product unit  $m$  with scope  $\operatorname{sc}(m) = \operatorname{sc}(n)$  decomposed in a way that it cannot be rearranged by introducing additional decomposable product units (Def. 2), e.g.,  $\operatorname{sc}(m) = (\mathbf{Z}, \mathbf{W})$  with  $\mathbf{Z} \cap \mathbf{X} \neq \emptyset$  and  $\bar{\mathbf{W}} \cap \mathbf{X} \neq \emptyset$ . As such, every omni-compatible circuit over  $\mathbf{X}$  must be representable in the form  $\sum_{i=1}^{N} \theta_i \prod_{k=1}^{|X|} l_{ik}(X_k)$  without any increase in its size.
Now, it is easy to verify that the circuit representations of CP, RESCAL, TUCKER and COMPLEX follow the above form, with a different number  $N$  of product units feeding the single sum unit, but each one decomposing its scope  $\{S,R,O\}$  into  $(\{S\},\{R\},\{O\})$  (see Fig. 2). From this it follows that  $\mathrm{CP}^+$ ,  $\mathrm{RESCAL}^+$ ,  $\mathrm{TUCKER}^+$  and  $\mathrm{COMPLEX}^+$  are omni-compatible as well, as they share the same structure of their energy-based counterpart, while just enforcing non-negative activations via reparametrisation (see Section 4.1).
Finally, we note that  $\mathrm{CP}^2$ , RESCAL $^2$ , TUCKER $^2$  and COMPLEX $^2$  are still omni-compatible because the square operation yields the following fully-factorised representation:  $(\sum_{i=1}^{N}\theta_{i}\prod_{k=1}^{|\mathbf{X}|}l_{ik}(X_{k}))^{2} = \sum_{i=1}^{N}\sum_{j=1}^{N}\theta_{i}\theta_{j}\prod_{k=1}^{|\mathbf{X}|}l_{ik}(X_{k})\prod_{k=1}^{|\mathbf{X}|}l_{jk}(X_{k})$  which can be easily rewritten as  $\sum_{h=1}^{N^2}\omega_h\prod_{k=1}^{|\mathbf{X}|}l_{hk}(X_k)$  where now  $h$  ranges over the Cartesian product of  $i \in [N]$  and  $j \in [N]$ ,  $\omega_h$  is the product of  $\theta_{i}\theta_{j}$  and  $l_{hk}$  is a new input unit that encodes  $l_{ik}(X_k)l_{jk}(X_k)$  for a certain variable index  $k$ .
A.2 Efficient Summations over Circuits
Proposition 2 (Efficient Summations). Let  $\phi$  be a smooth and decomposable circuit over  $\mathbf{X} = \{S, R, O\}$  that encodes the score function of a KGE model. The sum  $\sum_{s \in \mathcal{E}} \sum_{r \in \mathcal{R}} \sum_{o \in \mathcal{E}} \phi(s, r, o)$  or any other summation over subjects, predicates or objects can be computed in time  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|) \cdot |\phi|)$ .
Proof. A proof for the computation of marginal probabilities in smooth and decomposable probabilistic circuits (PCs) defined over discrete variables in linear time with respect to their size can be found in [13]. This proof also applies for computing summations in smooth and decomposable circuits that do not necessarily correspond to marginal probabilities [76]. The satisfaction of smoothness and decomposability (Def. 2) in a circuit  $\phi$  permits to push outer summations inside the computational graph until input units are reached, where summations are actually performed independently and on smaller sets of variables (i.e.,  $\{S\}$ ,  $\{R\}$ ,  $\{O\}$  in our case), and then to evaluate the circuit only once.
Here we take into account the computational cost of summing over each input unit (see proof of Prop. 1), which is  $\mathcal{O}(|\mathcal{E}|)$  (resp.  $\mathcal{O}(|\mathcal{R}|))$  for those defined on variables  $S, O$  (resp.  $R$ ). Since the size of the circuit  $|\phi|$  must be at least the number of input units, we retrieve that the overall complexity for computing summations as stated in the proposition is  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|) \cdot |\phi|)$ .
As an example, consider the CP score function computing  $\phi_{\mathrm{CP}}(s,r,o) = \langle \mathbf{e}_s,\mathbf{w}_r,\mathbf{e}_o\rangle$  for some triple  $(s,r,o)$  and embeddings  $\mathbf{e}_s,\mathbf{w}_r,\mathbf{e}_o\in \mathbb{R}^d$ . We can compute  $\sum_{s\in \mathcal{E}}\sum_{r\in \mathcal{R}}\sum_{o\in \mathcal{E}}\phi_{\mathrm{CP}}(s,r,o)$  by pushing the outer summations inside the trilinear product, i.e., by computing it as  $\langle \sum_{s\in \mathcal{E}}\mathbf{e}_s,\sum_{r\in \mathcal{R}}\mathbf{w}_r,\sum_{o\in \mathcal{E}}\mathbf{e}_o\rangle$ , which requires time  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|)\cdot d)$ .
A.3 Efficient Summations over Squared Circuits
Theorem 1 (Efficient summations of squared GeKCs). Performing summations as stated in Prop. 2 on  $\mathrm{CP}^2$ , RESCAL $^2$ , TUCKER $^2$  and COMPLEX $^2$  can be done in time  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|) \cdot |\phi|^2)$ .
Proof. In Lem. A.1 we showed that the circuit representations  $\phi$  of CP, RESCAL, TUCKER and COMPLEX are omni-compatible (see Def. B.2). As a consequence,  $\phi$  is compatible (see Def. B.1) with itself. Therefore, Prop. B.1 ensures that we can construct the product circuit  $\phi \cdot \phi$  (i.e.,  $\phi^2$ ) as a smooth and decomposable circuit having size  $\mathcal{O}(|\phi|^2)$  in time  $\mathcal{O}(|\phi|^2)$ . Since  $\phi^2$  is still smooth and decomposable, Prop. 2 guarantees that we can perform summations in time  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|) \cdot |\phi|^2)$ .
A.4 Circuits encoding Domain Constraints
In Def. A.1 we introduce the concepts of support and determinism, whose definition is useful to describe constraint circuits in Def. A.2.
Definition A.1 (Support and Determinism [13, 76]). In a circuit the support of a computational unit  $n$  over variables  $\mathbf{X}$  computing  $\phi_n(\mathbf{X})$  is defined as the set of value assignments to variables in  $\mathbf{X}$  such that the output of  $n$  is non-zero, i.e.,  $\mathrm{supp}(n) = \{\mathbf{x} \in \mathrm{val}(\mathbf{X}) \mid \phi_n(\mathbf{X}) \neq 0\}$ . A sum unit  $n$  is deterministic if its inputs have disjoint supports, i.e.,  $\forall i, j \in \mathrm{in}(n), i \neq j: \mathrm{supp}(i) \cap \mathrm{supp}(j) = \emptyset$ .
Definition A.2 (Constraint Circuit [1]). Given a propositional logic formula  $K$ , a constraint circuit  $c_{K}$  is a smooth and decomposable PC over variables  $\mathbf{X}$  with deterministic sum units (Def. A.1) and indicator functions as input units, such that  $c_{K}(\mathbf{x}) = \mathbb{1}\{\mathbf{x} \models K\}$  for any  $\mathbf{x} \in \mathsf{val}(\mathbf{X})$ .
In general, we can compile any propositional logic formula into a constraint circuit (Def. A.2) by leveraging knowledge compilation techniques [19, 18, 52]. For domain constraints (Def. 4) this compilation process is straightforward, as we detail in the following proposition and proof.
Proposition A.1 (Circuit encoding domain constraints). Let  $K = K_{r_1} \vee \ldots \vee K_{r_m}$  be a disjunction of domain constraints defined over a set of predicates  $\mathcal{R} = \{r_1, \dots, r_m\}$  and a set of entities  $\mathcal{E}$  (Def. 4). We can compile  $K$  into a constraint circuit  $c_K$  (Def. A.2) defined over variables  $\mathbf{X} = \{S, R, O\}$  having size  $\mathcal{O}(|\mathcal{E}| \cdot |\mathcal{R}|)$  in the worst case and  $\mathcal{O}(|\mathcal{E}| + |\mathcal{R}|)$  in the best case.
Proof. Let  $K = K_{r_1} \vee \ldots \vee K_{r_m}$  be a disjunction of domain constraints (Def. 4) where
$$
K _ {r} \equiv S \in \kappa_ {S} (r) \wedge R = r \wedge O \in \kappa_ {O} (r) \equiv (\vee_ {u \in \kappa_ {S} (r)} S = u) \wedge R = r \wedge (\vee_ {v \in \kappa_ {O} (r)} O = v).
$$
Note that the disjunctions in  $K$  are deterministic, i.e., only one of their argument can be true at the same time. This enables us to construct the constraint circuit  $c_{K}$  such that  $c_{K}(s,r,o) = \mathbb{1}\{(s,r,o)\models K\}$  for any triple by simply replacing conjunctions and disjunctions with product and sum units, respectively. Note that  $c_{K}$  is indeed smooth and decomposable (Def. 2), as the inputs of the sum units are product units having scope  $\{S,R,O\}$  that are fully factorised into  $(\{S\} ,\{R\} ,\{O\})$  Moreover,  $K$  is a disjunction of  $|\mathcal{R}|$  conjunctive formulae having  $\mathcal{O}(|\mathcal{E}|)$  terms, and therefore  $|c_{K}| = \mathcal{O}(|\mathcal{E}|\cdot |\mathcal{R}|)$  in the worst case. In the best case of every predicate sharing the same subject and object domains  $\kappa_{S},\kappa_{O}\subseteq \mathcal{E}$ , we can simplify  $K$  into a conjunction of three disjunctive expressions, i.e.,
$$
K \equiv (\bigvee_ {u \in \kappa_ {S}} S = u) \wedge (\bigvee_ {r \in \mathcal {R}} R = r) \wedge (\bigvee_ {v \in \kappa_ {O}} O = v)
$$
that can be easily compiled into a constraint circuit  $c_{K}$  having size  $\mathcal{O}(|\mathcal{E}| + |\mathcal{R}|)$ , by again noticing that disjunctions are deterministic. In real-world KGs like ogbl-biokg [32] several predicates share the same subject and object domains, and this permits to have much smaller constraint circuits.
A.5 Efficient Integration of Domain Knowledge in GeKCs
Proof. In Lem. A.1 we showed that the GeKCs  $\phi_{\mathrm{pc}}$  derived from CP, RESCAL, TUCKER and COMPLEX via non-negative restriction (Section 4.1) or squaring (Section 4.2) are omni-compatible (see Def. B.2). As a consequence,  $\phi_{\mathrm{pc}}$  is always compatible with  $c_{K}$  regardless of the encoded logical constraint  $K$ , since constraint circuits are by definition smooth and decomposable (Def. A.2). By applying Prop. B.1, we retrieve that we can construct  $\phi_{\mathrm{pc}} \cdot c_{K}$  as a smooth and decomposable circuit of size  $\mathcal{O}(|\phi_{\mathrm{pc}}| \cdot |c_{K}|)$  and in time  $\mathcal{O}(|\phi_{\mathrm{pc}}| \cdot |c_{K}|)$ . As the resulting product circuit is smooth and decomposable, Prop. 2 guarantees that we can compute its partition function  $Z_{K} = \sum_{s \in \mathcal{E}} \sum_{r \in \mathcal{R}} \sum_{o \in \mathcal{E}} (\phi_{\mathrm{pc}}(s, r, o) \cdot c_{K}(s, r, o))$  in time  $\mathcal{O}((|\mathcal{E}| + |\mathcal{R}|) \cdot |\phi_{\mathrm{pc}}| \cdot |c_{K}|)$ .
B Circuits
B.1 Tractable Product of Circuits
In this section, we provide the formal definition of compatibility (Def. B.1) and omni-compatibility (Def. B.2), as stated by Vergari et al. [76]. Given two compatible circuits, Prop. B.1 guarantees that we can represent their product as a smooth and decomposable circuit efficiently.
Definition B.1 (Compatibility). Two circuits  $\phi, \phi'$  over variables  $\mathbf{X}$  are compatible if (1) they are smooth and decomposable, and (2) any pair of product units  $n \in \phi, m \in \phi'$  having the same scope can be rearranged into binary products that are mutually compatible and decompose their scope in the same way, i.e.,  $(\operatorname{sc}(n) = \operatorname{sc}(m)) \Longrightarrow (\operatorname{sc}(n_i) = \operatorname{sc}(m_i), n_i$  and  $m_i$  are compatible) for some rearrangements of the inputs of  $n$  (resp.  $m$ ) into  $n_1, n_2$  (resp.  $m_1, m_2$ ).
Definition B.2 (Omni-compatibility). A circuit  $\phi$  over variables  $\mathbf{X}$  is omni-compatible if it is compatible with any smooth and decomposable circuit over  $\mathbf{X}$ .
Proposition B.1 (Tractable product of circuits). Let  $\phi, \phi'$  be two compatible (Def. B.1) circuits. We can represent the product circuit  $\phi \cdot \phi'$  computing the product of the outputs of  $\phi$  and  $\phi'$  as a smooth and decomposable circuit having size  $\mathcal{O}(|\phi| \cdot |\phi'|)$  in time  $\mathcal{O}(|\phi| \cdot |\phi'|)$ . Moreover, if both  $\phi$  and  $\phi'$  are omni-compatible (Def. B.2), then also the product circuit  $\phi \cdot \phi'$  is omni-compatible.
Prop. B.1 allows us to compute the partition function and any other marginal probability in GeKCs obtained via squaring efficiently (see Section 4.2 and Thm. 1). In addition, Prop. B.1 is a crucial theoretical result that allows us to inject logical constraints in GeKCs in a way that enable computing the partition function exactly and efficiently (see Section 5 and Thm. 2).
C From KGE Models to PCs
C.1 Interpreting Non-negative Embedding Values
In Fig. C.1 we interpret the embedding values of GeKCs obtained via non-negative restriction -  $\mathrm{CP^{+}}$ , RESCAL+, TUCKER+, COMPLEX+-(Section 4.1) as the parameters of unnormalised categorical distributions over entities (elements in  $\mathcal{E}$ ) or predicates (elements in  $\mathcal{R}$ ).
Figure C.1: Non-negative embeddings parameterise categorical distributions. 2-dimensional embeddings of GeKCs obtained via non-negative restrictions (Section 4.1) can be seen as the parameters of two categorical distributions over entities (left) or predicates (right) up to renormalisation.
C.2 Realising the Non-negative Restriction of COMPLEX
As anticipated in Section 4.1, for the COMPLEX [65] score function restricting the real and imaginary parts to be non-negative is not sufficient to obtain a PC due to the presence of a subtraction, as showed in the following equation.
$$
\begin{array}{l} \phi_ {\text {C o m p l e x}} (s, r, o) = \left\langle \operatorname {R e} \left(\mathbf {e} _ {s}\right), \operatorname {R e} \left(\mathbf {w} _ {r}\right), \operatorname {R e} \left(\mathbf {e} _ {o}\right) \right\rangle + \left\langle \operatorname {I m} \left(\mathbf {e} _ {s}\right), \operatorname {R e} \left(\mathbf {w} _ {r}\right), \operatorname {I m} \left(\mathbf {e} _ {o}\right) \right\rangle \\ + \left\langle \operatorname {R e} \left(\mathbf {e} _ {s}\right), \operatorname {I m} \left(\mathbf {w} _ {r}\right), \operatorname {I m} \left(\mathbf {e} _ {o}\right) \right\rangle - \left\langle \operatorname {I m} \left(\mathbf {e} _ {s}\right), \operatorname {I m} \left(\mathbf {w} _ {r}\right), \operatorname {R e} \left(\mathbf {e} _ {o}\right) \right\rangle \\ \end{array}
$$
Here  $\mathbf{e}_s, \mathbf{w}_r, \mathbf{e}_o \in \mathbb{C}^d$  are the embeddings associated to the subject, predicate and object, respectively. Under the restriction of embedding values to be non-negative, we ensure that  $\phi_{\mathrm{COMPLEX}}(s,r,o) \geq 0$  for any input triple by enforcing the additional constraint
$$
\left\langle \operatorname {R e} \left(\mathbf {e} _ {s}\right), \operatorname {R e} \left(\mathbf {w} _ {r}\right), \operatorname {R e} \left(\mathbf {e} _ {o}\right) \right\rangle \geq \left\langle \operatorname {I m} \left(\mathbf {e} _ {s}\right), \operatorname {I m} \left(\mathbf {w} _ {r}\right), \operatorname {R e} \left(\mathbf {e} _ {o}\right) \right\rangle , \tag {6}
$$
which can be simplified into the two distinct inequalities
$$
\forall u \in \mathcal {E} \quad \operatorname {R e} \left(e _ {u i}\right) \geq \operatorname {I m} \left(e _ {u i}\right) \quad \text {a n d} \quad \forall r \in \mathcal {R} \quad \operatorname {R e} \left(w _ {r i}\right) \geq \operatorname {I m} \left(w _ {r i}\right).
$$
In other words, we want the real part of each embedding value to be always greater or equal than the corresponding imaginary part. We implement this constraint in practice by reparametrisation of the imaginary part in function of the real part, i.e.,
$$
\forall u \in \mathcal {E} \quad \operatorname {I m} \left(e _ {u i}\right) = \operatorname {R e} \left(e _ {u i}\right) \cdot \sigma \left(\theta_ {u i}\right) \tag {7}
$$
$$
\forall r \in \mathcal {R} \quad \operatorname {I m} \left(w _ {r i}\right) = \operatorname {R e} \left(w _ {r i}\right) \cdot \sigma \left(\gamma_ {r i}\right) \tag {8}
$$
where  $\sigma(x) = 1 / (1 + \exp(-x)) \in [0,1]$  denotes the logistic function and  $\theta_{ui}, \gamma_{ri} \in \mathbb{R}$  are additional parameters associated to entities  $u \in \mathcal{E}$  and predicates  $r \in \mathcal{R}$ , respectively. The reparametrisation of the imaginary parts using Eqs. (7) and (8) is a sufficient condition for the satisfaction of the constraint showed in Eq. (6), and also maintains the same number of learnable parameters of COMPLEX.
C.3 Sampling from GeKCs with Non-negative Parameters
Parameters interpretation. Sum units with non-negative parameters in smooth and decomposable PCs can be seen as marginalised discrete latent variables, similarly to the latent variable interpretation in mixture models [55, 53]. That is, the non-negative parameters of a sum unit are the parameters of a (possibly unnormalised) categorical distribution over assignments to a latent variable. For  $\mathrm{CP}^+$  and  $\mathrm{RESCAL}^+$  (Section 4.1), the non-negative parameters of the sum unit encode a uniform and
unnormised categorical distribution, as they are all fixed to 1 (see Fig. 2). By contrast, in  $\mathrm{TUCKER}^+$  these parameters are the vectorisation of the core tensor  $\mathcal{T}$  (see the proof of Prop. 1), and hence they are learned. The input units of  $\mathrm{CP}^+$ ,  $\mathrm{RESCAL}^+$  and  $\mathrm{TUCKER}^+$  can be interpreted as unnormalised categorical distribution over entities or predicates, as detailed in Appendix C.1.
Sampling from  $\mathbf{CP}^{+}$ , COMPLEX $^{+}$ , TUCKER $^{+}$ . Thanks to the latent variable interpretation, ancestral sampling in  $\mathbf{CP}^{+}$ , RESCAL $^{+}$  and TUCKER $^{+}$  can be performed by (1) sampling an assignment to the latent variable associated to the single sum unit, i.e., one of its input branches, (2) selecting the corresponding combination of subject-predicate-object input units, and (3) sampling a subject, predicate and object respectively from each of the indexed unnormalized categorical distributions.
C.4 Learning Complexity of GeKCs
In Table C.1 we summarise the complexities of computing the PLL and MLE objectives (Eqs. (1) and (2)) for KGE models and GeKCs. Asymptotically, GeKCs manifest better time and space complexities with respect to the number of entities  $|\mathcal{E}|$ , batch size  $|B|$  and embedding size. This makes GeKCs more efficient than traditional KGE models during training, both in time and memory (see Section 4.3 and Fig. 3).
Table C.1: Summary of complexities for exactly computing the PLL and MLE objectives. Time and space complexity of computing  $\log p(o\mid s,r)$  and the partition function  $Z$ . These complexities are respectively lower bounds of the complexities of computing the PLL and MLE objectives, as we have that  $|\mathcal{E}|\gg |\mathcal{R}|$  for large real-world KGs. For CP, RESCAL and COMPLEX and GeKCs derived from them,  $d$  denotes the size of both entity and predicate embeddings. For TUCKER and GeKCs derived from it,  $d_{e}$  and  $d_{r}$  denote the embedding sizes for entities and predicates, respectively.