Question: The one flaw in this paper is that you only compare with DeepProbLog, why do you drop NeurASP after the first table?
Evidence: Previous DPPLs, DeepProbLog and NeurASP, introduced the Neural Predicate as an annotated-disjunction or as a propositional atom, respectively, to acquire conditional class probabilities, P (C|X), via the softmax function at the output of an arbitrary DNN. As mentioned in the introduction, this approach has certain limitations concerning inference capabilities. To resolve this issue, we introduce Neural-Probabilisitic Predicates (NPPs).
Formally, we denote with
a Neural-Probabilistic Predicate h. Thereby, (i) npp is a reserved word to label an NPP, (ii) h a symbolic name of either a PC, NN or a joint of a PC and NN (cf. Fig.), e.g., color_attr is the name of an NPP of Fig.. Additionally, (iii) x denotes a "term" and (iv) v 1 , . . . , v n are placeholders for each of the n possible outcomes of h. For example, the placeholders for color_attr are the color attributes of an object (Red, Blue, Green, etc.).
An NPP abbreviates an arithmetic literal of the form c = v with c ∈ {h(x)} and v ∈ {v 1 , . . . , v n }. Furthermore, we denote with Π npp a set of NPPs of the form stated in (Eq. 1) and r npp the set of all rules c = v of one NPP, which denotes the possible outcomes, obtained from an NPP in Π npp , e.g. r color_attr = {c = Red, c = Blue, c = Green, ...} for the example depicted in Fig..
Rules of the form npp (h(x), [v 1 , . . . , v n ]) ← Body are used as an abbreviation for application to multiple entities, e.g. multiple slots for the task of set prediction (cf. Fig.). Hereby, Body of the rule is identified by (tautology, true) or ⊥ (contradiction, false) during grounding. Rules of the form Head ← Body with r npp appearing in Head are prohibited for Π npp .
In this work, we largely make use of NPPs that contain probabilistic circuits (specifically SPNs) which allow for tractable density estimation and modelling of joint probabilities. In this way, it is possible to answer a much richer set of probabilistic queries, i.e. P (X, C), P (X|C) and P (C|X).
In addition to this, we introduce the arguably more interesting type of NPP that combines a neural module with a PC. Hereby, the neural module learns to map the raw input data into an optimal latent representation, e.g. object-based slot representations. The PC, in turn, learns to model the joint distribution of these latent variables and produces the final probability estimates. This type of NPP nicely combines the representational power of neural networks with the advantages of PCs in probability estimation and query flexibility.
For making the different probabilistic queries distinguishable in a SLASH program, we introduce the following notation. We denote a given variable with + and the query variable with −. E.g., within the running example of set prediction (cf. Fig.), with the query color_attr(+X, −C) one is asking for P (C|X). Similarly, with color_attr(−X, +C) one is asking for P (X|C) and, finally, with color_attr(−X, −C) for P (X, C).
To summarize, an NPP can consist of neural and/or probabilistic modules and produces querydependent probability estimates. Due to the flexibility of its definition, the term NPP contains the predicates of previous works, but also more interesting predicates discussed above. The specific "flavor" of an NPP should be chosen depending on what type of probability estimation is required. Lastly, NPPs have the unified loss function of the negative log-likelihood:
(2) whereby we are assuming the data to be i.i.d., ground truth x i to be the all-ones vector, ξ to be the parameters of the NPP and P (X,C) ξ are the predictions xi obtained from the PC encoded in the NPP.
In addition to this, we introduce the arguably more interesting type of NPP that combines a neural module with a PC. Hereby, the neural module learns to map the raw input data into an optimal latent representation, e.g. object-based slot representations. The PC, in turn, learns to model the joint distribution of these latent variables and produces the final probability estimates. This type of NPP nicely combines the representational power of neural networks with the advantages of PCs in probability estimation and query flexibility.
Answer: We would like to explain our intention behind this decision. Throughout our experiments, we want to show the importance of PCs in LP. Thus, utilizing DeepProbLog in Table 1(b) as the only baseline, we have demonstrated the rich features of PCs as NPPs, since a DNN would require additional techniques to handle missing data. PCs are therefore syntactical sugar of SLASH in this regard.