SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Emanuele Marconato
DISI and DI
University of Trento and University of Pisa
Trento, Italy
emanuele.marconato@unitn.it
Stefano Teso
CIMeC and DISI
University of Trento
stefano.teso@unitn.it
avergari@exseed.ed.ac.uk
Andrea Passerini
DISI
andrea passerini@unitn.it
Neuro-Symbolic (NeSy) predictive models hold the promise of improved compliance with given constraints, systematic generalization, and interpretability, as they allow to infer labels that are consistent with some prior knowledge by reasoning over high-level concepts extracted from sub-symbolic inputs. It was recently shown that NeSy predictors are affected by reasoning shortcuts: they can attain high accuracy but by leveraging concepts with unintended semantics, thus coming short of their promised advantages. Yet, a systematic characterization of reasoning shortcuts and of potential mitigation strategies is missing. This work fills this gap by characterizing them as unintended optima of the learning objective and identifying four key conditions behind their occurrence. Based on this, we derive several natural mitigation strategies, and analyze their efficacy both theoretically and empirically. Our analysis shows reasoning shortcuts are difficult to deal with, casting doubts on the trustworthiness and interpretability of existing NeSy solutions.
Neuro-Symbolic (NeSy) AI aims at improving the robustness and trustworthiness of neural networks by integrating them with reasoning capabilities and prior knowledge [1, 2]. We focus on NeSy predictors [3, 4], neural structured-output classifiers that infer one or more labels by reasoning over high-level concepts extracted from sub-symbolic inputs, like images or text [5, 6, 7, 8, 9, 10, 11]. They leverage reasoning techniques to encourage - or even guarantee - that their predictions comply with domain-specific regulations and inference rules. As such, they hold the promise of improved systematic generalization, modularity, and interpretability, in that learned concepts can be readily reused in different NeSy tasks, as done for verification [12], and for explaining the model's inference process to stakeholders [13, 14, 15]. On paper, this makes NeSy predictors ideal for high-stakes applications that require both transparency and fine-grained control over the model's (in- and out-of-distribution) behavior, such as medical diagnosis [16], robotics [17] and self-driving cars [18].
Much of the promise of these models relies on learned concepts being high quality. The general consensus is that the prior knowledge constrains learned concepts to behave as expected [19] and issues with them are often tackled heuristically [20]. It was recently shown that, however, NeSy predictors can attain high accuracy by leveraging concepts with unintended semantics [21]. Following
Figure 1: Reasoning shortcuts undermine trustworthiness. An autonomous vehicle has to decide whether to  $Y =$  stop or  $Y =$  go based on three binary concepts extracted from an image  $\mathbf{x}$ , namely  $C_1 =$  red light,  $C_2 =$  green light and  $C_3 =$  presence of pedestrians (shown in pink). Left: In Task 1, the prior knowledge  $K = (\text{pedestrian} \lor \text{red} \Rightarrow \text{stop})$  instructs the vehicle to stop whenever the light is red or there are pedestrians on the road. The model can perfectly classify an (even exhaustive) training set by acquiring a reasoning shortcut that classifies pedestrians as red lights. Right: The learned concepts are then reused to guide an autonomous ambulance with the additional rule that in emergency situations red lights can be ignored, with potentially dire consequences. Our work identifies the causes of RSs (Section 4) and several mitigation strategies (Section 5).
Marconato et al. [21], we refer to these as reasoning shortcuts (RSs). RSs are problematic, as concepts encoding unintended semantics compromise generalization across NeSy tasks, as shown in Fig. 1, as well as interpretability and verification of NeSy systems [12]. Moreover, the only known mitigation strategies are based on heuristics [20, 11].
The issue is that RSs – and their root causes – are not well understood, making it difficult to design effective remedies. In this paper, we fill this gap. We introduce a formal definition of RSs and theoretically characterize their properties, highlighting how they are a general phenomenon affecting a variety of state-of-the-art NeSy predictors. Specifically, our results show that RSs can be shared across many NeSy architectures, and provide a way of counting them for any given learning problem. They also show that RSs depend on four key factors, namely the structure of the prior knowledge and of the data, the learning objective, and the architecture of the neural concept extractor. This enables us to identify several supervised and unsupervised mitigation strategies, which we systematically analyze both theoretically and empirically. Finally, we experimentally validate our findings by testing a number of representative NeSy predictors and mitigation strategies on four NeSy data sets.
Contributions. Summarizing, we: (i) Formalize RSs and identify four key root causes. (ii) Show that RSs are a general issue impacting a variety of NeSy predictors. (iii) Identify a number of mitigation strategies and analyze their effectiveness, or lack thereof. (iv) Empirically show that RSs arise even when the data set is large and unbiased, and evaluate the efficacy of different mitigation strategies, highlighting the limits of unsupervised remedies and the lack of a widely applicable recipe.
2 The Family of Neuro-Symbolic Predictors
Notation. Throughout, we indicate scalar constants  $x$  in lower-case, random variables  $X$  in upper case, and ordered sets of constants  $\mathbf{x}$  and random variables  $\mathbf{X}$  in bold typeface. Also,  $\mathbf{x}_{i:j}$  denotes the subset  $\{x_{i},\ldots ,x_{j}\}$ , [n] the set  $\{1,\dots ,n\}$ , and  $\mathbf{x}\models \mathsf{K}$  indicates that  $\mathbf{x}$  satisfies a logical formula  $\mathsf{K}$ .
NeSy predictors. A NeSy predictor is a model that infers  $n$  labels  $\mathbf{Y}$  (taking values in  $\mathcal{V}$ ) by reasoning over a set of  $k$  discrete concepts  $\mathbf{C}$  (taking values in  $\mathcal{C}$ ) extracted from a sub-symbolic continuous input  $\mathbf{X}$  (taking values in  $\mathcal{X}$ ). Reasoning can be implemented in different ways, but overall its role is to encourage the model's predictions to comply with given prior knowledge  $\mathsf{K}$ , in the sense that predictions  $\mathbf{y}$  that do not satisfy the knowledge are generally avoided. The prior knowledge  $\mathsf{K}$  is assumed to be provided upfront and correct, as formalized in Section 3. Normally, only supervision on the labels  $\mathbf{Y}$  is available for training, with the concepts  $\mathbf{C}$  treated as latent variables.
Example 1. In MNIST-Addition [7], given a pair of MNIST images [22], say  $\mathbf{x} = (\mathbf{z},\mathbf{z})$ , the model has to infer the concepts  $\mathbf{C} = (C_1,C_2)$  encoding the digit classes, to predict their sum  $Y$ , in this case 8. Reasoning drives the model towards complying with constraint  $\mathsf{K} = (Y = C_1 + C_2)$ .
The concepts are modeled by a conditional distribution  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  parameterized by  $\theta \in \Theta$ , typically implemented with a neural network. The predicted concepts can be viewed as "soft" or "neural" predicates with a truth value ranging in [0, 1]. As for the reasoning step, the most popular strategies
involve penalizing the model for producing concepts and/or labels inconsistent with the knowledge at training time [8, 23, 24] or introducing a reasoning layer that infers labels from the predicted concepts and also operates at inference time [7, 9, 25, 10]. In either case, end-to-end training requires to differentiate through the knowledge. Mainstream options include softening the knowledge using fuzzy logic [26, 6, 27, 11] and casting reasoning in terms of probabilistic logics [28, 7, 10].
To investigate the scope and impact of RSs, we consider three representative NeSy predictors. The first one is DeepProbLog (DPL) [7], which implements a sound probabilistic-logic reasoning layer on top of the neural predicates. DPL is a discriminative predictor of the form [21]:
$$
p _ {\theta} (\mathbf {y} \mid \mathbf {x}; \mathsf {K}) = \sum_ {\mathbf {c}} u _ {\mathsf {K}} (\mathbf {y} \mid \mathbf {c}) \cdot p _ {\theta} (\mathbf {c} \mid \mathbf {x}) \tag {1}
$$
where the concept distribution is fully factorized and the label distribution is uniform $^1$  over all label-concept combinations compatible with the knowledge, that is,  $u_{\mathsf{K}}(\mathbf{y} \mid \mathbf{c}) = \mathbb{1}\{\mathbf{c} \models \mathsf{K}[\mathbf{Y} / \mathbf{y}]\} / Z(\mathbf{c}; \mathsf{K})$  where the indicator  $\mathbb{1}\{\mathbf{c} \models \mathsf{K}[\mathbf{Y} / \mathbf{y}]\}$  guarantees all labels inconsistent with  $\mathsf{K}$  have zero probability, and  $Z(\mathbf{c}; \mathsf{K}) = \sum_{\mathbf{y}} \mathbb{1}\{\mathbf{c} \models \mathsf{K}[\mathbf{Y} / \mathbf{y}]\}$  is a normalizing constant. Inference amounts to computing a most likely label  $\operatorname{argmax}_{\mathbf{y}} p_{\theta}(\mathbf{y} \mid \mathbf{x}; \mathsf{K})$ , while learning is carried out via maximum (log-)likelihood estimation, that is given a training set  $\mathcal{D} = \{(\mathbf{x}, \mathbf{y})\}$ , maximizing
$$
\mathcal {L} \left(p _ {\theta}, \mathcal {D}, K\right) := \frac {1}{| \mathcal {D} |} \sum_ {(\mathbf {x}, \mathbf {y}) \in \mathcal {D}} \log p _ {\theta} (\mathbf {y} \mid \mathbf {x}; K). \tag {2}
$$
In general, it is intractable to evaluate Eq. (1) and solve inference exactly. DPL leverages knowledge compilation [29, 30] to make both steps practical.
Our analysis on DPL can be carried over to other NeSy predictors implementing analogous reasoning layers [31, 32, 33, 10, 34]. Furthermore, we show that certain RSs affect also alternative NeSy approaches such as the Semantic Loss (SL) [8] and Logic Tensor Networks (LTNs) [6], two state-of-the-art penalty-based approaches. Both reward a neural network for predicting labels  $\mathbf{y}$  consistent with the knowledge, but SL measures consistency in probabilistic terms, while LTNs use fuzzy logic to measure a fuzzy degree of knowledge satisfaction. See Appendix A for a full description.
3 Reasoning Shortcuts as Unintended Optima
It was recently shown that NeSy predictors are vulnerable to reasoning shortcuts (RSs), whereby the model attains high accuracy by leveraging concepts with unintended semantics [21].
Example 2. To build intuition, consider MNIST-Addition and assume the model is trained on examples of only two sums:  $\pmb{0} + \pmb{1} = 1$  and  $\pmb{0} + \pmb{2} = 2$ . Note that there exist two distinct maps from images to concepts that perfectly classify such examples: one is the intended solution  $(\pmb{0} \mapsto 0, \pmb{1} \mapsto 1, \pmb{2} \mapsto 2)$ , while the other is  $(\pmb{0} \mapsto 1, \pmb{1} \mapsto 0, \pmb{2} \mapsto 1)$ . The latter is unintended.
The ground-truth data generating process. In order to properly define what a RS is, we have to first define what non-shortcut solutions are. In line with work on causal representation learning [35, 36, 37], we do so by specifying the ground-truth data generation process  $p^*(\mathbf{X}, \mathbf{Y}; \mathsf{K})$ . Specifically, we assume it takes the form illustrated in Fig. 2. In short, we assume there exist  $k$  unobserved ground-truth concepts  $\mathbf{G}$  (e.g., in MNIST-Addition these are the digits 0 to 9) that determine both the observations  $\mathbf{X}$  (the MNIST images) and the labels  $\mathbf{Y}$  (the sum). We also allow for extra stylistic factors  $\mathbf{S}$ , independent from  $\mathbf{G}$ , that do influence the observed data (e.g., calligraphic style) but not the labels. The ground-truth concepts are discrete and range in  $\mathcal{G} = [m_1] \times \dots \times [m_k]$ , whereas the style  $\mathbf{s} \in \mathbb{R}^q$  is continuous. The training and test examples  $(\mathbf{x}, \mathbf{y})$  are then obtained by first sampling  $\mathbf{g}$  and  $\mathbf{s}$  and then  $\mathbf{x} \sim p^*(\mathbf{X} | \mathbf{g}, \mathbf{s})$  and  $\mathbf{y} \sim p^*(\mathbf{Y} | \mathbf{g}; \mathsf{K})$ . Later on, we will use  $\mathrm{supp}(\mathbf{G})$  to denote the support
Figure 2: The groundtruth data generation process (in black) and a NeSy predictor (in blue).
of  $p^*(\mathbf{G})$ . We also assume that the ground-truth process is consistent with the prior knowledge  $\mathsf{K}$ , in the sense that invalid examples are never generated:  $p^*(\mathbf{y} \mid \mathbf{g}; \mathsf{K}) = 0$  for all  $(\mathbf{g}, \mathbf{y})$  that violate  $\mathsf{K}$ .
What is a reasoning shortcut? By definition, a NeSy predictor  $p_{\theta}(\mathbf{Y} \mid \mathbf{X}; \mathsf{K})$  - shown in blue in Fig. 2 - acquires concepts with the correct semantics if it recovers the ground-truth concepts, i.e.,
$$
p _ {\theta} (\mathbf {C} \mid \mathbf {x}) \equiv p ^ {*} (\mathbf {G} \mid \mathbf {x}) \quad \forall \mathbf {x} \in \mathcal {X} \tag {3}
$$
In practice, the distribution of labels given concepts needs not be uniform [10].
A model satisfying Eq. (3) easily generalizes to other NeSy prediction tasks that make use of the same ground-truth concepts  $\mathbf{G}$ , as  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  can be reused for solving the new task [21, 38]. It is also interpretable, in the sense that as long as stakeholders understand the factors  $\mathbf{G}$ , they can also interpret concept-based explanations of the inference process that rely on  $\mathbf{C}$  [39]. Naturally, there may exist many concept distributions that violate Eq. (3) and that as such do not capture the correct semantics. However, all those that achieve sub-par log-likelihoods can be ideally avoided simply by improving learning or supplying more examples. We define RSs as those concept distributions for which these strategies are not enough.
Definition 1. A reasoning shortcut is a distribution  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  that achieves maximal log-likelihood on the training set but does not match the ground-truth concept distribution,
$$
\mathcal {L} \left(p _ {\theta}, \mathcal {D}, \mathrm {K}\right) = \max  _ {\theta^ {\prime} \in \Theta} \mathcal {L} \left(p _ {\theta^ {\prime}}, \mathcal {D}, \mathrm {K}\right) \quad \wedge \quad p _ {\theta} (\mathbf {C} \mid \mathbf {X}) \neq p ^ {*} (\mathbf {G} \mid \mathbf {X}) \tag {4}
$$
This makes RSs difficult to improve by regular means and also hard to spot based on predictions alone. Yet, since the concepts do not recover the correct semantics, RSs can compromise systematic generalization and interpretability. For instance, the shortcut concepts learned in Example 2 would fail completely at MNIST-Addition tasks involving digits other than 0, 1, and 2 and also at other arithmetic tasks, like multiplication, involving the same digits, as we show experimentally in Section 6. An examples of RSs in a high-stakes scenario is shown in Fig. 1.
4 Properties of Reasoning Shortcuts
Counting deterministic RSs. We begin by assessing how many RSs exist in an idealized setting in which the ground-truth generative process is simple, we have access to the true risk, and the concepts  $\mathbf{C}$  have been specified correctly, that is,  $\mathcal{C} = \mathcal{G}$ , and then proceed to work out what this entails. Specifically, we work with these assumptions:
A1. The distribution  $p^*(\mathbf{X} \mid \mathbf{G}, \mathbf{S})$  is induced by a map  $f: (\mathbf{g}, \mathbf{s}) \mapsto \mathbf{x}$ , i.e.,  $p^*(\mathbf{X} \mid \mathbf{G}, \mathbf{S}) = \delta\{\mathbf{X} = f(\mathbf{g}, \mathbf{s})\}$ , where  $f$  is invertible, and smooth over  $\mathbf{s}$ .
A2. The distribution  $p^*(\mathbf{Y} \mid \mathbf{G}; \mathsf{K})$  is induced by a map  $\beta_{\mathsf{K}}: \mathbf{g} \mapsto \mathbf{y}$ . This is what happens in MNIST-Addition, as there exists a unique value  $y$  that is the sum of any two digits  $(g_1, g_2)$ .
Our analysis builds on a link between the NeSy predictor  $p_{\theta}(\mathbf{Y} \mid \mathbf{X}; \mathsf{K})$  and the concept extractor  $p_{\theta}(\mathbf{C} \mid \bar{\mathbf{X}})$ , which depend on  $\mathbf{X}$ , and their analogues  $p_{\theta}(\mathbf{Y} \mid \mathbf{G}; \mathsf{K})$  and  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$  that depend directly on the ground-truth concepts  $\mathbf{G}$ . This link is formalized by the following lemma:
Lemma 1. It holds that: (i) The true risk of  $p_{\theta}$  can be upper bounded as follows:
$$
\mathbb {E} _ {(\mathbf {x}, \mathbf {y}) \sim p ^ {*} (\mathbf {X}, \mathbf {Y}; \mathsf {K})} [ \log p _ {\theta} (\mathbf {y} \mid \mathbf {x}; \mathsf {K}) ] \leq \mathbb {E} _ {\mathbf {g} \sim p (\mathbf {G})} \left(- K L [ p ^ {*} (\mathbf {Y} \mid \mathbf {g}; \mathsf {K}) \| p _ {\theta} (\mathbf {Y} \mid \mathbf {g}; \mathsf {K}) ] - H [ p ^ {*} (\mathbf {Y} \mid \mathbf {g}; \mathsf {K}) ]\right) \tag {5}
$$
where KL is the Kullback-Leibler divergence and  $\mathsf{H}$  is the Shannon entropy. Moreover, under A1 and A2,  $p_{\theta}(\mathbf{Y} \mid \mathbf{X}; \mathsf{K})$  is an optimum of the LHS of Eq. (5) if and only if  $p_{\theta}(\mathbf{Y} \mid \mathbf{G}; \mathsf{K})$  is an optimum of the RHS. (ii) Under A1, there exists a bijection between the deterministic concept distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  that are constant over the support of  $p(\mathbf{X} \mid \mathbf{g})$ , for each  $\mathbf{g} \in \mathrm{supp}(\mathbf{G})$ , and the deterministic distributions of the form  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$ .
All proofs can be found in Appendix B. Lemma 1 implies that the deterministic concept distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  of NeSy predictors  $p_{\theta}(\mathbf{Y} \mid \mathbf{X}; \mathsf{K})$  that maximize the LHS of Eq. (5), including those that are RSs, correspond one-to-one to the deterministic distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$  yielding label distributions  $p_{\theta}(\mathbf{Y} \mid \mathbf{G}; \mathsf{K})$  that maximize the RHS of Eq. (5). Hence, we can count the number of deterministic RSs by counting the deterministic distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$ :
Theorem 2. Let  $\mathcal{A}$  be the set of mappings  $\alpha: \mathbf{g} \mapsto \mathbf{c}$  induced by all possible deterministic distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$ , i.e., each  $p_{\theta}(\mathbf{C} \mid \mathbf{G}) = \mathbb{1}\{\mathbf{C} = \alpha(\mathbf{G})\}$  for exactly one  $\alpha \in \mathcal{A}$ . Under A1 and A2, the number of deterministic optima  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$  of Eq. (5) is:
$$
\sum_ {\alpha \in \mathcal {A}} \mathbb {1} \left\{\bigwedge_ {\mathbf {g} \in \operatorname {s u p p} (\mathbf {G})} \left(\beta_ {\mathrm {K}} \circ \alpha\right) (\mathbf {g}) = \beta_ {\mathrm {K}} (\mathbf {g}) \right\} \tag {6}
$$
Intuitively, this sum counts the deterministic concept distributions  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  - embodied here by the maps  $\alpha$  - that output concepts predicting a correct label for each example in the training set. The
Table 1: Impact of different mitigation strategies on the number of deterministic optima: R is reconstruction, C supervision on C, MTL multi-task learning, and DIS disentanglement. All strategies reduce the number of  $\alpha$ 's in Eq. (6), sometimes substantially, but require different amounts of effort to be put in place. Actual counts for our data sets are reported in Appendix C.2.
MITIGATION	REQUIRES	CONSTRAINT ON α	ASSUMPTIONS	RESULT
None	-	∧g∈supp(G) ((βK∘α)(g) = βK(g))	A1, A2	Theorem 2
MTL	Tasks	∧g∈supp(G) ∧t∈[T] ((βK(t)∘α)(g) = βK(t)(g))	A1, A2	Proposition 4
C	Sup. on C	∧g∈SCsupp(G) ∧i∈I (αi(g) = gi)	A1	Proposition 5
R	-	∧g, g′∈supp(G):g≠g′ (α(g) ≠ α(g'))	A1, A3	Proposition 6
ground-truth distribution  $p^{*}(\mathbf{G} \mid \mathbf{X})$  is one such distribution, so the count is always at least one, but there may be more, and all of these are RSs. Eq. (6) gives us their exact number. This formalizes the intuition of Marconato et al. [21] that, as long as the prior knowledge  $K$  admits the correct label  $y$  to be inferred from more than one concept vector  $c$ , there is room for RSs. So far, we have assumed Eq. (2) is computed as in DPL. However, RSs are chiefly a property of the prior knowledge, and as such also affect NeSy predictors employing different reasoning procedures or different relaxations of the knowledge. We show this formally in Appendix A. Deterministic RS are also important because – in certain cases – they define a basis for all reasoning shortcuts:
Proposition 3. For probabilistic logic approaches (including DPL and SL): (i) All convex combinations of two or more deterministic optima  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$  of the likelihood are also (non-deterministic) optima. However, not all convex combinations can be expressed in DPL and SL. (ii) Under A1 and A2, all optima of the likelihood can be expressed as a convex combination of deterministic optima. (iii) If A2 does not hold, there may exist non-deterministic optima that are not convex combinations of deterministic ones. These may be the only optima.
Combining Proposition 3 (i) with Theorem 2 gives us a lower bound for the number of non-deterministic RSs, in the sense that if there are at least two deterministic RS, then there exist infinitely many non-deterministic ones. An important consequence is that, if we can somehow control what deterministic RSs affect the model, then we may be able to implicitly lower the number of non-deterministic RSs as well. However, Proposition 3 implies that there may exist non-deterministic RSs that are unrelated to the deterministic ones and that as such cannot be controlled this way.
5 Analysis of Mitigation Strategies
The key factors underlying the occurrence of deterministic RSs appear explicitly in Eq. (6). These are: (i) the knowledge  $\mathsf{K}$ , (ii) the structure of  $\mathrm{supp}(\mathbf{G})$ , (iii) the objective function  $\mathcal{L}$  used for training (via Lemma 1), and (iv) the architecture of the concept extractor  $p_{\theta}(\mathbf{C} \mid \mathbf{X})$ , embodied in the Theorem by  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$ . This gives us a starting point for identifying possible mitigation strategies and analyzing their impact on the number of deterministic RSs. Our main results are summarized in Table 1.
5.1 Knowledge-based Mitigation
The prior knowledge  $\mathsf{K}$  is the main factor behind RSs and also a prime target for mitigation. The most direct way of eliminating unintended concepts is to edit  $\mathsf{K}$  directly, for instance by eliciting additional constraints from a domain expert. However, depending on the application, this may not be feasible: experts may not be available, or it may be impossible to constrain  $\mathsf{K}$  without also eliminating concepts with the intended semantics.
A more practical alternative is to employ Multi-Task Learning (MTL). The idea is to train a NeSy predictor over  $T$  tasks sharing the same ground-truth concepts  $\mathbf{G}$  but differing prior knowledge  $\mathsf{K}^{(t)}$ , for  $t \in [T]$ . E.g., one could learn a model to predict both the sum and product of MNIST digits, as in our experiments (Section 6). Intuitively, by constraining the concepts to work well across tasks, MTL leaves less room for unintended semantics. The following result confirms this intuition:
Proposition 4. Consider  $T$  NeSy prediction tasks with knowledge  $\mathsf{K}^{(t)}$ , for  $t \in [T]$  and data sets  $\mathcal{D}^{(t)}$ , all sharing the same  $p^*(\mathbf{G})$ . Under A1 and A2, any deterministic optimum  $p_{\theta}(\mathbf{C} \mid \mathbf{G})$  of the MTL loss (i.e., the average of per-task losses) is a deterministic optimum of a single task with prior
knowledge  $\bigwedge_t K^{(t)}$ . The number of deterministic optima amounts to: