Original: Semantic Probabilistic Layers for Neuro-Symbolic Learning
Revised: Semantic Probabilistic Layers for Neuro-Symbolic Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: We design a predictive layer for structured-output prediction (SOP) that can beplugged into any neural network guaranteeing its predictions are consistent with aset of predeﬁned symbolic constraints.
Revised: We design a predictive layer for structured-output prediction (SOP) that can beplugged into any neural network guaranteeing its predictions are consistent with aset of predeﬁned symbolic constraints.

Original: Our S emantic P robabilistic L ayer (SPL)can model intricate correlations, and hard constraints, over a structured outputspace all while being amenable to end-to-end learning via maximum likelihood.
Revised: Our S emantic P robabilistic L ayer (SPL)can model intricate correlations, and hard constraints, over a structured outputspace all while being amenable to end-to-end learning via maximum likelihood.

Original: SPLs combine exact probabilistic inference with logical reasoning in a cleanand modular way, learning complex distributions and restricting their support tosolutions of the constraint.
Revised: SPLs combine exact probabilistic inference with logical reasoning in a cleanand modular way, learning complex distributions and restricting their support tosolutions of the constraint.

Original: As such, they can faithfully, and efﬁciently, modelcomplex SOP tasks beyond the reach of alternative neuro-symbolic approaches.
Revised: As such, they can faithfully, and efﬁciently, modelcomplex SOP tasks beyond the reach of alternative neuro-symbolic approaches.

Original: We empirically demonstrate that SPLs outperform these competitors in terms ofaccuracy on challenging SOP tasks including hierarchical multi-label classiﬁcation,pathﬁnding and preference learning, while retaining perfect constraint satisfaction.
Revised: We empirically demonstrate that SPLs outperform these competitors in terms ofaccuracy on challenging SOP tasks including hierarchical multi-label classiﬁcation,pathﬁnding and preference learning, while retaining perfect constraint satisfaction.

Original:  Introduction
Revised:  Introduction

Original: Modularity is among the major factors that propelled the Cambrian explosion of deep learning [34].
Revised: Modularity is among the major factors that propelled the Cambrian explosion of deep learning [34].

Original: By stacking multiple differentiable layers together, practitioners are able to train deep classiﬁers inan end-to-end fashion with little effort.
Revised: By stacking multiple differentiable layers together, practitioners are able to train deep classiﬁers inan end-to-end fashion with little effort.

Original: However, despite its ﬂexibility, this modular approach tolearning does not guarantee that the predictions of these models conform to our expectations asto what makes sense .
Revised: However, despite its ﬂexibility, this modular approach tolearning does not guarantee that the predictions of these models conform to our expectations asto what makes sense .

Original: On the contrary, unconstrained deep classiﬁers are notorious for leading topredictions that are inconsistent with the logical constraints governing an underlying domain.
Revised: On the contrary, unconstrained deep classiﬁers are notorious for leading topredictions that are inconsistent with the logical constraints governing an underlying domain.

Original: This is even more evident in, and crucial for, structured output prediction (SOP) tasks, whereclassiﬁers have to predict hundreds of mutually constrained labels [69, 7].
Revised: This is even more evident in, and crucial for, structured output prediction (SOP) tasks, whereclassiﬁers have to predict hundreds of mutually constrained labels [70, 7].

Original: Consider for example aclassical SOP task such as multi-label classiﬁcation (MLC) [70].
Revised: Consider for example aclassical SOP task such as multi-label classiﬁcation (MLC) [71].

Original: Learning a multi-label classiﬁer thatdisregards the correlations among labels, e.g., by considering them fully independent given the inputs,yields sub-optimal results [5].
Revised: Learning a multi-label classiﬁer thatdisregards the correlations among labels, e.g., by considering them fully independent given the inputs,yields sub-optimal results [5].

Original: In more challenging tasks such as hierarchical MLC (HMLC) [67] orpathﬁnding [56], leveraging the domain’s logical constraints (encoding, e.g., the label hierarchy oracyclicity and connectedness of a path) at training time can improve prediction accuracy [41], but itcannot guarantee that the predictions are always consistent with the constraints at inference time [31].
Revised: In more challenging tasks such as hierarchical MLC (HMLC) [68] orpathﬁnding [57], leveraging the domain’s logical constraints (encoding, e.g., the label hierarchy oracyclicity and connectedness of a path) at training time can improve prediction accuracy [42], but itcannot guarantee that the predictions are always consistent with the constraints at inference time [31].

Original: Fig.
Revised: Fig.

Original: 1 illustrates this problem in the context of pathﬁnding: constraint-unaware neural networks systematically fail to predict label conﬁgurations that form a valid path.
Revised: 1 illustrates this problem in the context of pathﬁnding: constraint-unaware neural networks systematically fail to predict label conﬁgurations that form a valid path.

Original: In many safety-criticalscenarios such as protein function [59] and interaction prediction [61], and drug discovery [19, 23],predicting inconsistent solutions can not only be harmful but also highly expensive [4, 33].
Revised: In many safety-criticalscenarios such as protein function [60] and interaction prediction [62], and drug discovery [19, 23],predicting inconsistent solutions can not only be harmful but also highly expensive [4, 33].

Original: , due to their discrete nature, injecting logical constraints into deep neural networks while retaining modularity and differentiability is extremely challenging, as demonstrated in the neuro-symbolic learning literature [62].
Revised: , due to their discrete nature, injecting logical constraints into deep neural networks while retaining modularity and differentiability is extremely challenging, as demonstrated in the neuro-symbolic learning literature [63].

Original: One such attempt has been to learn neural networks that satisfy the logical constraints by explicitly minimizing a differentiable loss term, the probability that the networks violates the constraint for any given prediction.
Revised: One such attempt has been to learn neural networks that satisfy the logical constraints by explicitly minimizing a differentiable loss term, the probability that the networks violates the constraint for any given prediction.

Original: And while successful, such approaches
Revised: And while successful, such approaches

Original: [Figure]
Revised: [Figure]

Original: R ES N ET
Revised: ET

Original: [Figure]
Revised: [Figure]

Original: S EMANTIC L OSS

Original:  SPL (ours) even moderately logicalconstraints
Revised: [Figure] SPL [Figure] logical

Original: Motivated by these observations,we introduce a
Revised: Motivated by these observations, introduce a

Original: S emantic P robabilistic Layer constraints
Revised: S emantic P robabilistic L

Original: Contributions.
Revised: Contributions.

Original: Summarizing, we: ( i ) Identify six desiderata that neuro-symbolic predictors forconstrained SOP tasks ought to satisfy, and show that state-of-the-art approaches fall short of oneor more of them (Table 1); ( ii ) Introduce SPL, a novel semantic probabilistic layer that leveragesprobabilistic circuits to satisfy all six desiderata, i.e., that can be plugged into neural networks toensure predictions to comply to given logical constraints, while retaining efﬁciency, expressivity,differentiability, and fully probabilistic semantics; ( iii ) We provide empirical evidence of the effectiveness of SPL in several challenging neuro-symbolic SOP tasks, such as HMLC and pathﬁnding,where it outperforms state-of-the-art neuro-symbolic approaches, often by a noticeable margin.
Revised: Summarizing, we: ( i ) Identify six desiderata that neuro-symbolic predictors forconstrained SOP tasks ought to satisfy, and show that state-of-the-art approaches fall short of oneor more of them (Table 1); ( ii ) Introduce SPL, a novel semantic probabilistic layer that leveragesprobabilistic circuits to satisfy all six desiderata, i.e., that can be plugged into neural networks toensure predictions to comply to given logical constraints, while retaining efﬁciency, expressivity,differentiability, and fully probabilistic semantics; ( iii ) We provide empirical evidence of the effectiveness of SPL in several challenging neuro-symbolic SOP tasks, such as HMLC and pathﬁnding,where it outperforms state-of-the-art neuro-symbolic approaches, often by a noticeable margin.

Original:  Designing a probabilistic layer for neuro-symbolic SOP
Revised:  Designing a probabilistic layer for neuro-symbolic SOP

Original: Notation.
Revised: Notation.

Original: In the following, we denote scalar constants x in lower case, random variables X inupper case, vectors of constants x in bold and vectors of random variables X in capital boldface.
Revised: In the following, we denote scalar constants x in lower case, random variables X inupper case, vectors of constants x in bold and vectors of random variables X in capital boldface.

Original: 57{ ϕ } denotes the indicator function that evaluates to 1 if the statement ϕ holds and to 0 otherwise.
Revised: 57{ ϕ } denotes the indicator function that evaluates to 1 if the statement ϕ holds and to 0 otherwise.

Original: We denote by x | =
Revised: We denote by x | =

Original: K that the value assignment x to variables X satisﬁes a logical formula K .
Revised: K that the value assignment x to variables X satisﬁes a logical formula K .

Original: Neuro-symbolic SOP.
Revised: Neuro-symbolic SOP.

Original: We tackle SOP tasks in which a neural net classiﬁer must learn to associateinstances x ∈ R D to L interdependent labels, identiﬁed by the vector y ∈ { 0 , 1 } L .
Revised: We tackle SOP tasks in which a neural net classiﬁer must learn to associateinstances x ∈ R D to L interdependent labels, identiﬁed by the vector y ∈ { 0 , 1 } L .

Original: We assume thatwe can abstract any neural classiﬁer into two components: a feature extractor f that maps inputs
Revised: We assume thatwe can abstract any neural classiﬁer into two components: a feature extractor f that maps inputs

Original: X to a M -dimensional embedding Z = f ( X ) and a predictive ﬁnal layer that outputs the labeldistribution p
Revised: X to a M -dimensional embedding Z = f ( X ) and a predictive ﬁnal layer that outputs the labeldistribution p

Original: ( Y | Z ) .
Revised: ( Y | Z ) .

Original: For example, the simplest, and yet widely adopted [48, 76, 31], predictivelayer in neural classiﬁers for SOP considers labels Y i to be conditionally independent from each othergiven Z , i.e.,. p ( Y | Z ) =
Revised: For example, the simplest, and yet widely adopted [49, 77, 31], predictivelayer in neural classiﬁers for SOP considers labels Y i to be conditionally independent from each othergiven Z , i.e., p ( Y | Z ) =

Original: (cid:81)
Revised: (cid:81)

Original: Li =1 p ( Y i | Z ) .
Revised: Li =1 p ( Y i | Z ) .

Original: We refer to this as fully independent layer (FIL).
Revised: We refer to this as fully independent layer (FIL).

Original: In a FIL, p ( Y i =
Revised: In a FIL,p ( Y i =

Original: y i | z ) is computed as σ ( w (cid:62) i z ) where w i ∈ R M is a vector of parameters and σ ( x ) isthe sigmoid function 1 / (1 + e x ) .
Revised: y i | z ) is computed as σ ( w (cid:62) i z ) where w i ∈ R M is a vector of parameters and σ ( x ) is thesigmoid function 1 / (1 + e x ) .

Original: We are interested in dependencies between labels that can occur both as correlations , as is the casein MLC [21], and as logical constraints encoded by logical formulas.
Revised: We are interested in dependencies between labels that can occur both as correlations , as is the casein MLC [21], and as logical constraints encoded by logical formulas.

Original: For example, in a HMLCtask [31] one logical constraint can encode the fact that observing a label for the class cat and dog,implies observing the label for their superclass animal
Revised: For example, in a HMLCtask [31] one logical constraint can encode the fact that observing a label for the class cat and dog,implies observing the label for their superclass animal

Original: [Equation]
Revised: [Equation]

Original: Speciﬁcally, we assume symbolic knowledge to be supplied in the form of constraints encoded as alogical formula K over the labels Y and optionally over a subset of the discrete input variables in X ,
Revised: Speciﬁcally, we assume symbolic knowledge to be supplied in the form of constraints encoded as alogical formula K over the labels Y and optionally over a subset of the discrete input variables in X ,

Original: [Table]
Revised: [Table]

Original: if any (e
Revised: if any (e

Original: [Table]
Revised: [Table]

Original: g., in our experiments
Revised: g., in our experiments

Original: [Table]
Revised: [Table]

Original: the predicted simple path is constrained to lie within the subset ofedges appearing in the input graph, see Sec. 5).
Revised: the predicted simple path is constrained to lie within the subset ofedges appearing in the input graph, see Sec. 5).

Original: On the other hand
Revised: On the other hand

Original: [Table]
Revised: [Table]

Original: we expect a model to learn thelabel dependencies from data.
Revised: we expect a model to learn thelabel dependencies from data.

Original: We call such tasks neuro-symbolic SOP tasks.
Revised: We call such tasks neuro-symbolic SOP tasks.

Original: Desiderata for neuro-symbolic SOP.
Revised: Desiderata for neuro-symbolic SOP.

Original: To tackle this setting, we seek an algorithmic strategy forreplacing the predictive layer in any neural network classiﬁer with little effort, with the aim of injectingcomplex symbolic knowledge and allowing for ﬂexible probabilistic reasoning.
Revised: To tackle this setting, we seek an algorithmic strategy forreplacing the predictive layer in any neural network classiﬁer with little effort, with the aim of injectingcomplex symbolic knowledge and allowing for ﬂexible probabilistic reasoning.

Original: We formalize theseobservations into the following six desiderata for our predictive layer:
Revised: We formalize theseobservations into the following six desiderata for our predictive layer:

Original: D1.
Revised: D1.

Original: Probabilistic :
Revised: Probabilistic :

Original: The layer should enjoy sound probabilistic semantics, and deliver normalizedprobabilistic predictions to facilitate maximum-likelihood learning and sound decisionmaking by virtue of calibrated probabilistic predictionsD2.
Revised: The layer should enjoy sound probabilistic semantics, and deliver normalizedprobabilistic predictions to facilitate maximum-likelihood learning and sound decisionmaking by virtue of calibrated probabilistic predictionsD2.

Original: Expressive : It should be able to compactly encode intricate correlations between labels.
Revised: Expressive : It should be able to compactly encode intricate correlations between labels.

Original: D3.
Revised: D3.

Original: Consistent : It should always output predictions that are consistent with the prespeciﬁedsymbolic knowledge, i.e., for all x and y , if ( x , y ) (cid:54)| =
Revised: Consistent : It should always output predictions that are consistent with the prespeciﬁedsymbolic knowledge, i.e., for all x and y , if ( x , y ) (cid:54)| =

Original: K then p ( y | x ) = 0 .D4.
Revised: K then p ( y | x ) = 0 .D4.

Original: General : It should support rich logical constraints over the labels expressed in some formallanguage, e.g., propositional logic.
Revised: General : It should support rich logical constraints over the labels expressed in some formallanguage, e.g., propositional logic.

Original: D5.
Revised: D5.

Original: Modular : It should be applicable to any off-the-shelf (and possibly pretrained) neuralnetwork in a modular fashion, enabling end-to-end learning and rapid prototyping.D6.
Revised: Modular : It should be applicable to any off-the-shelf (and possibly pretrained) neuralnetwork in a modular fashion, enabling end-to-end learning and rapid prototyping.D6.

Original: Efﬁcient : The time required by the predictor to compute a prediction should be linear in thesize of the predictor and of the hard constraint representation.
Revised: Efﬁcient : The time required by the predictor to compute a prediction should be linear in thesize of the predictor and of the hard constraint representation.

Original: For example, FILs are clearly probabilistic (D1), modular (D5), and efﬁcient (D6), but at the costof being incapable of modeling intricate correlations and logical constraints and thus generatinginconsistent predictions (D2–D4) (see also Fig. 1).
Revised: For example, FILs are clearly probabilistic (D1), modular (D5), and efﬁcient (D6), but at the costof being incapable of modeling intricate correlations and logical constraints and thus generatinginconsistent predictions (D2–D4) (see also Fig. 1).

Original: Table 1 summarizes how the other popular andeffective approaches to neuro-symbolic SOP nowadays fall short of one of more desiderata as well.
Revised: Table 1 summarizes how the other popular andeffective approaches to neuro-symbolic SOP nowadays fall short of one of more desiderata as well.

Original: We discuss this in detail in Sec. 4.
Revised: We discuss this in detail in Sec. 4.

Original: To the best of our knowledge, our proposed semantic probabilisticlayers (SPLs) are the ﬁrst algorithmic solution to satisfy all above desiderata.
Revised: To the best of our knowledge, our proposed semantic probabilisticlayers (SPLs) are the ﬁrst algorithmic solution to satisfy all above desiderata.

Original: SPL.
Revised: SPL.

Original: At
Revised: At

Original: a
Revised: a

Original: high level, SPL realizes the above desiderata in a single layer that combines exact probabilistic inference with logical reasoning in a clean and modular way, learning complex distributionsand restricting their support to solutions of the constraint.
Revised: high level, SPL realizes the above desiderata in a single layer that combines exact probabilistic inference with logical reasoning in a clean and modular way, learning complex distributionsand restricting their support to solutions of the constraint.

Original: Deﬁnition 2.1 (Semantic probabilistic layer (SPL)) .
Revised: Deﬁnition 2.1 (Semantic probabilistic layer (SPL)) .

Original: Given an input conﬁguration x , a SPL decomposes the computation of the probability of a label conﬁguration as:
Revised: Given an input conﬁguration x , a SPL decomposes the computation of the probability of a label conﬁguration as:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Here, q Θ ( y | f ( x )) is a module to perform probabilistic reasoning by encoding an expressivedistribution over the labels parameterized by Θ ; c K ( x , y ) is a module to ensure consistency of thepredictions by encoding logical constraints K and being non-zero only when K is satisﬁed, i.e.,c K ( x , y )
Revised: Here, q Θ ( y | f ( x )) is a module to perform probabilistic reasoning by encoding an expressivedistribution over the labels parameterized by Θ ; c K ( x , y ) is a module to ensure consistency of thepredictions by encoding logical constraints K and being non-zero only when K is satisﬁed, i.e.,c K ( x , y )

Original: = 1 { ( x , y )
Revised: = 1 { ( x , y )

Original: | = K } ; and Z ( x ) is a renormalization term, also called the partition function.
Revised: | = K } ; and Z ( x ) is a renormalization term, also called the partition function.

Original: Fig.
Revised: Fig.

Original: 2 illustrates the computational graph of our SPL at training time.
Revised: 2 illustrates the computational graph of our SPL at training time.

Original: In order to satisfy all D1-D6, wewill realize both q Θ and c K as circuits [72, 12], constrained computational graphs that enable tractablecomputations.
Revised: In order to satisfy all D1-D6, wewill realize both q Θ and c K as circuits [73, 12], constrained computational graphs that enable tractablecomputations.

Original: Differently from FILs, q Θ in SPLs can encode an expressive joint distributions overthe labels and therefore attain full expressiveness by scaling the number of parameters Θ (D2).
Revised: Differently from FILs, q Θ in SPLs can encode an expressive joint distributions overthe labels and therefore attain full expressiveness by scaling the number of parameters Θ (D2).

Original: Consistency is guaranteed by the component c K : by multiplying it to the joint probability of a label
Revised: Consistency is guaranteed by the component c K : by multiplying it to the joint probability of a label

Original: [Figure]
Revised: [Figure]

Original: The by marginalizing product
Revised: The by marginalizing product

Original: Θ , K , so conﬁguration the resulting product distribution r Θ , K ( y , x ) =
Revised: Θ , K , so conﬁguration the resulting product distribution r Θ , K ( y , x ) =

Original: q Θ ( y | f ( x )) ·
Revised: q Θ ( y | f ( x )) ·

Original: c K ( x , y ) will have itssupport effectively cut by K , and thus cannot allocate any probability mass to inconsistent predictions(D3).
Revised: c K ( x , y ) will have itssupport effectively cut by K , and thus cannot allocate any probability mass to inconsistent predictions(D3).

Original: Additionally, c K will allow to encode general propositional logical constraints in a compactcomputational graph (D4).
Revised: Additionally, c K will allow to encode general propositional logical constraints in a compactcomputational graph (D4).

Original: Lastly, the product r Θ , K ( x , y ) is fully differentiable and allows SPL to bean off-the-shelf replacement for other predictive layers (see Fig. 2) and enables end-to-end learning(D5).
Revised: Lastly, the product r Θ , K ( x , y ) is fully differentiable and allows SPL to bean off-the-shelf replacement for other predictive layers (see Fig. 2) and enables end-to-end learning(D5).

Original: By renormalizing r Θ , K ( x , y ) and outputting normalized probabilities, SPL enables the exactcomputation of gradients for Θ , which can therefore be trained by maximum likelihood.
Revised: By renormalizing r Θ , K ( x , y ) and outputting normalized probabilities, SPL enables the exactcomputation of gradients for Θ , which can therefore be trained by maximum likelihood.

Original: Thanks to recent advancements in the literature on circuits, we can compute the partition function
Revised: Thanks to recent advancements in the literature on circuits, we can compute the partition function

Original: Z ( x ) efﬁciently in time linear in the size of r Θ , K , thus preserving efﬁciency (D6) and not compromising on the other desiderata.
Revised: Z ( x ) efﬁciently in time linear in the size of r Θ , K , thus preserving efﬁciency (D6) and not compromising on the other desiderata.

Original: This will also yield correct (and consistent) predictions at testtime, when an SPL computes the MAP state y ∗ =
Revised: This will also yield correct (and consistent) predictions at testtime, when an SPL computes the MAP state y ∗ =

Original: argmax y r Θ , K ( y , x ) / (cid:80) y r Θ , K ( y , x ) .
Revised: argmax y r Θ , K ( y , x ) / (cid:80) y r Θ , K ( y , x ) .

Original: The nextsection clariﬁes how to implement the modules of SPL as circuits while satisfying these desiderata.
Revised: The nextsection clariﬁes how to implement the modules of SPL as circuits while satisfying these desiderata.

Original:  Realizing SPLs with tractable circuit representations
Revised:  Realizing SPLs with tractable circuit representations

Original: The components of SPLs are circuits , a large class of computational graphs that can represent bothfunctions and distributions [12, 17].
Revised: The components of SPLs are circuits , a large class of computational graphs that can represent bothfunctions and distributions [12, 17].

Original: Circuits subsume many tractable generative and discriminativeprobabilistic models—from Chow-Liu and latent tree models [13, 10], to hidden Markov models(HMMs) [58], sum-product networks (SPNs) [57], decision trees [38, 14], and deep regressors [37]—as well as many compact representations of logical formulas, such as (ordered) binary decisiondiagrams [3], sentential decision diagrams (SDDs) [16] and others [17].
Revised: Circuits subsume many tractable generative and discriminativeprobabilistic models—from Chow-Liu and latent tree models [13, 10], to hidden Markov models(HMMs) [59], sum-product networks (SPNs) [58], decision trees [39, 14], and deep regressors [38]—as well as many compact representations of logical formulas, such as (ordered) binary decisiondiagrams [3], sentential decision diagrams (SDDs) [16] and others [17].

Original: The key idea behind SPLs is to leverage this single formalism to represent both an expressive jointdistribution for q Θ ( y | f ( x )) and a compact encoding of the logical constraints for c K ( x , y ) , whileensuring the exact and efﬁcient evaluation of Eq.
Revised: The key idea behind SPLs is to leverage this single formalism to represent both an expressive jointdistribution for q Θ ( y | f ( x )) and a compact encoding of the logical constraints for c K ( x , y ) , whileensuring the exact and efﬁcient evaluation of Eq.

Original: This can be achieved by ensuring that thesecomputational graphs abide certain structural properties: smoothness , decomposability , determinismand compatibility [17, 71].
Revised: This can be achieved by ensuring that thesecomputational graphs abide certain structural properties: smoothness , decomposability , determinismand compatibility [17, 72].

Original: Next, we introduce probabilistic circuits for modeling q Θ (Sec. 3.1) andconstraint circuits for c K (Sec. 3.2), while in Sec.
Revised: Next, we introduce probabilistic circuits for modeling q Θ (Sec. 3.1) andconstraint circuits for c K (Sec. 3.2), while in Sec.

Original: 3.4 we propose a more efﬁcient implementationof SPL utilizing a single circuit.
Revised: 3.4 we propose a more efﬁcient implementationof SPL utilizing a single circuit.

Original: 3.1 Representing expressive distributions with probabilistic circuits
Revised: 3.1 Representing expressive distributions with probabilistic circuits

Original: We start by introducing circuits for joint probability distributions, and then extend the discussion toconditional distributions, which we use to implement q Θ ( Y | f ( X )) in SPLs.
Revised: We start by introducing circuits for joint probability distributions, and then extend the discussion toconditional distributions, which we use to implement q Θ ( Y | f ( X )) in SPLs.

Original: Deﬁnition 3.1 (Circuits) .
Revised: Deﬁnition 3.1 (Circuits) .

Original: A circuit h over variables Y is a computational graph encoding a parameterized function h Θ ( Y ) by combining three kinds of computational units: input functionalunits, sum units, and product units.
Revised: A circuit h over variables Y is a computational graph encoding a parameterized function h Θ ( Y ) by combining three kinds of computational units: input functionalunits, sum units, and product units.

Original: An input functional n represents a base parametric functionh n ( sc ( n ); λ ) over some variables sc ( n ) ⊆
Revised: An input functional n represents a base parametric functionh n ( sc ( n ); λ ) over some variables sc ( n ) ⊆

Original: Y , called its scope, and it is parameterized by λ .
Revised: Y , called its scope, and it is parameterized by λ .

Original: Sum andproduct units n elaborate the output of other units, denoted in ( n ) .
Revised: Sum andproduct units n elaborate the output of other units, denoted in ( n ) .

Original: Sum units are parameterized by ωand compute the weighted sum of their inputs (cid:80) c ∈ in ( n ) ω
Revised: Sum units are parameterized by ωand compute the weighted sum of their inputs (cid:80) c ∈ in ( n ) ω

Original: c h c ( sc ( n )) , while product units compute 148 (cid:81) c ∈ in ( n ) h c ( sc ( n )) .
Revised: c h c ( sc ( n )) , while product units compute 148 (cid:81) c ∈ in ( n ) h c ( sc ( n )) .

Original: The parameters Θ of a circuit encompass the parameters of all input functionals( λ ) and sum units ( ω ).
Revised: The parameters Θ of a circuit encompass the parameters of all input functionals( λ ) and sum units ( ω ).

Original: Y Y
Revised: Y Y

Original: Y
Revised: Y

Original: Y× ×
Revised: Y× ×

Original: Y
Revised: Y

Original: Y× × q g 1 { Y 1 =0 } 1 { Y 1 =1 } 1 { Y 2 =0 } 1 { Y 2 =1 } × × ×  1 { Y 3 =0 } 1 { Y 3 =1 } ×× c
Revised: Y× × q g 1 { Y 1 =0 } 1 { Y 1 =1 } 1 { Y 2 =0 } 1 { Y 2 =1 } × × × × 1 { Y 3 =1 } 1 { Y 3 =0 } × c

Original: For any input y , the value of h Θ ( y ) can be evaluated by propagating the output of the input unitsthrough the computational graph and reading out the value of the last unit.
Revised: For any input y , the value of h Θ ( y ) can be evaluated by propagating the output of the input unitsthrough the computational graph and reading out the value of the last unit.

Original: The support of h is the setof all states y for which the output is non-zero, i.e., supp ( h ) =
Revised: The support of h is the set of all states y for which the output is non-zero, i.e., supp ( h ) =

Original: { y ∈ val ( Y )
Revised: { y ∈ val ( Y )

Original: | h ( y) (cid:54) = 0 }.
Revised: | h ( y (cid:54) = 0 }

Original: Deﬁnition 3.2 (Probabilistic circuits (PCs)) .
Revised: Deﬁnition 3.2 (Probabilistic circuits (PCs)) .

Original: A circuit q is a PC if it encodes a (possibly unnormalized)probability distribution, i.e., q Θ ( y ) is non-negative for all conﬁgurations y of Y .
Revised: A circuit q is a PC if it encodes a (possibly unnormalized)probability distribution, i.e., q Θ ( y ) is non-negative for all conﬁgurations y of Y .

Original: From here on, we will assume PCs to have positive sum parameters ω and whose input units modelvalid distributions, e.g., Bernoullis, as these conditions are sufﬁcient for satisfying Def. 3.2.
Revised: From here on, we will assume PCs to have positive sum parameters ω and whose input units modelvalid distributions, e.g., Bernoullis, as these conditions are sufﬁcient for satisfying Def. 3.2.

Original: Moreover,w.l.o.g.
Revised: Moreover,w.l.o.g.

Original: we will assume the sum and product units to be organized into alternating layers, and thatevery product unit n receives only two inputs c 1 , c 2 , i.e., q n ( X ) =
Revised: we will assume the sum and product units to be organized into alternating layers, and thatevery product unit n receives only two inputs c 1 , c 2 , i.e., q n ( X ) =

Original: q c 1 ( Y ) · q c 2 ( Y ) .
Revised: q c 1 ( Y ) · q c 2 ( Y ) .

Original: These conditionscan easily be enforced in polynomial time [73, 74].
Revised: These conditionscan easily be enforced in polynomial time [74, 75].

Original: We are speciﬁcally interested in smooth anddecomposable PCs, as they will be enabling efﬁcient inference in SPL (Sec. 3.3).
Revised: We are speciﬁcally interested in smooth anddecomposable PCs, as they will be enabling efﬁcient inference in SPL (Sec. 3.3).

Original: Deﬁnition 3.3 (Smoothness & Decomposability) .
Revised: Deﬁnition 3.3 (Smoothness & Decomposability) .

Original: A circuit is smooth if for every sum unit n , its inputsdepend on the same variables: ∀ c 1 , c 2 ∈ in ( n ) , sc ( c 1 ) =
Revised: A circuit is smooth if for every sum unit n , its inputsdepend on the same variables: ∀ c 1 , c 2 ∈ in ( n ) , sc ( c 1 ) =

Original: sc ( c 2 ) .
Revised: sc ( c 2 ) .

Original: It is decomposable if the inputs ofevery product unit n depend on disjoint sets of variables: in ( n ) = { c 1 , c 2 } , sc ( c 1 ) ∩ sc ( c 2 ) = ∅ .
Revised: It is decomposable if the inputs ofevery product unit n depend on disjoint sets of variables: in ( n ) = { c 1 , c 2 } , sc ( c 1 ) ∩ sc ( c 2 ) = ∅ .

Original: Smooth and decomposable PCs are both expressive and efﬁcient: they can encode distributions withhundred millions of parameters and be effectively learned by gradient ascent [54].
Revised: Smooth and decomposable PCs are both expressive and efﬁcient: they can encode distributions withhundred millions of parameters and be effectively learned by gradient ascent [55].

Original: The structure oftheir computational graph can be either speciﬁed manually [57, 54, 52] or acquired automaticallyfrom data [73, 60, 15], e.g., by ﬁrst learning a latent tree model and then compiling the latter into acircuit [43].
Revised: The structure oftheir computational graph can be either speciﬁed manually [58, 55, 53] or acquired automaticallyfrom data [74, 61, 15], e.g., by ﬁrst learning a latent tree model and then compiling the latter into acircuit [44].

Original: These circuits are competitive with intractable models such as variational autoencodersand normalizing ﬂows scores on several benchmarks [42].
Revised: These circuits are competitive with intractable models such as variational autoencodersand normalizing ﬂows scores on several benchmarks [43].

Original: As proposed by Shao et al.
Revised: As proposed by Shao et al.

Original: [64], any (smooth and decomposable) PC q Θ ( Y ) encoding a jointdistribution over the labels Y can be turned into a (smooth and decomposable) conditional circuit,conditioned by input variables X , by letting its parameters be a function of X .
Revised: [65], any (smooth and decomposable) PC q Θ ( Y ) encoding a jointdistribution over the labels Y can be turned into a (smooth and decomposable) conditional circuit,conditioned by input variables X , by letting its parameters be a function of X .

Original: Deﬁnition 3.4 (Neural conditional circuits [63]) .
Revised: Deﬁnition 3.4 (Neural conditional circuits [64]) .

Original: A conditional circuit q ( Y ; Θ = g ( X )) 
Revised: A conditional circuit q ( Y ; Θ = g ( X )) models theconditional distribution p

Original: modelsthe conditional distribution p ( Y | X ) via a differentiablegating function g that maps every inputconﬁguration x to the set of parameters of Θ of p  .
Revised: ( Y | X ) via a differentiable function g that maps every input conﬁgurationx to the set of parameters of Θ of p , also called the gating function .

Original: An example of a smooth and decomposable conditional circuit is shown in Fig.
Revised: An example of a smooth and decomposable conditional circuit is shown in Fig.

Original: This designimmediately allows us to implement q Θ ( Y | f
Revised: This designimmediately allows us to implement q Θ ( Y | f

Original: ( X )) in SPL as a conditional PC whose gating functionmaps the feature embedding space R K to the parameter space R | Θ | + , realizing q ( Y ; Θ = g ( f ( X ))) .
Revised: ( X )) in SPL as a conditional PC whose gating functionmaps the feature embedding space R K to the parameter space R | Θ | + , realizing q ( Y ; Θ = g ( f ( X ))) .

Original: As such, the gating function g creates a clean interface between any pre-trained feature extractorf and the PC q (Fig. 3).
Revised: As such, the gating function g creates a clean interface between any pre-trained feature extractorf and the PC q (Fig. 3).

Original: While one can devise g in several ways, we strive for simplicity in ourexperiments and adopt vanilla multi-layer perceptrons (MLPs) whose ﬁnal activations are eithersigmoids, if they have to predict the parameters λ of the Bernoulli input distributions of q , or softmax,if they output the sum unit parameters ω (Def. 3.1).
Revised: While one can devise g in several ways, we strive for simplicity in ourexperiments and adopt vanilla multi-layer perceptrons (MLPs) whose ﬁnal activations are eithersigmoids, if they have to predict the parameters λ of the Bernoulli input distributions of q , or softmax,if they output the sum unit parameters ω (Def. 3.1).

Original: 3.2 Encoding logical formulas with constraint circuits
Revised: 3.2 Encoding logical formulas with constraint circuits

Original: The next step is to translate a logical constraint K into a smooth and decomposable circuit c K ( x , y ) .
Revised: The next step is to translate a logical constraint K into a smooth and decomposable circuit c K ( x , y ) .

Original: To this end, we employ a special type of PCs, deﬁned as follows.
Revised: To this end, we employ a special type of PCs, deﬁned as follows.

Original: Deﬁnition 3.5 (Constraint circuits) .
Revised: Deﬁnition 3.5 (Constraint circuits) .

Original: A PC c over variables X ∪ Y is a constraint circuit encodingprior knowledge K if it computes 1 { ( x , y ) |
Revised: A PC c over variables X ∪ Y is a constraint circuit encodingprior knowledge K if it computes 1 { ( x , y ) |

Original: = K } for every conﬁguration ( x , y ) .
Revised: = K } for every conﬁguration ( x , y ) .

Original: As a practical way to realize such a circuit, we will consider constraint circuits that have all sumunit parameters equal to 1 and input functionals that are indicator functions over their scope, e.g.,c n ( z ) = 1 { z | = ϕ ( n ) } where Z is the scope of the input and ϕ ( n ) a constraint over it.
Revised: As a practical way to realize such a circuit, we will consider constraint circuits that have all sumunit parameters equal to 1 and input functionals that are indicator functions over their scope, e.g.,c n ( z ) = 1 { z | = ϕ ( n ) } where Z is the scope of the input and ϕ ( n ) a constraint over it.

Original: Furthermore,we require each sum unit in it to be deterministic .
Revised: Furthermore,we require each sum unit in it to be deterministic .

Original: Deﬁnition 3.6 (Determinism) .
Revised: Deﬁnition 3.6 (Determinism) .

Original: A sum unit n is deterministic if its inputs have disjoint supports, i.e.,∀ c 1 , c 2 ∈ in ( n ) , c 1 (cid:54)
Revised: A sum unit n is deterministic if its inputs have disjoint supports, i.e.,∀ c 1 , c 2 ∈ in ( n ) , c 1 (cid:54)

Original: =
Revised: =

Original: c 2 = ⇒ supp ( c 1 ) ∩ supp ( c 2 ) = ∅ .
Revised: c 2 = ⇒ supp ( c 1 ) ∩ supp ( c 2 ) = ∅ .

Original: Fig.
Revised: Fig.

Original: 3 shows an example of a deterministic constraint circuit.
Revised: 3 shows an example of a deterministic constraint circuit.

Original: Thanks to determinism, we can readilytranslate classical compact representations for logical formulas such as (ordered) binary decisiondiagrams [3, 8] and sentential decision diagrams (SDDs) [16] into constraint circuits as deﬁned above.
Revised: Thanks to determinism, we can readilytranslate classical compact representations for logical formulas such as (ordered) binary decisiondiagrams [3, 8] and sentential decision diagrams (SDDs) [16] into constraint circuits as deﬁned above.

Original: This becomes evident when they are written in the language of negation normal form [17] and theirand gates (resp. or gates) are replaced with product units (resp.
Revised: This becomes evident when they are written in the language of negation normal form [17] and theirand gates (resp. or gates) are replaced with product units (resp.

Original: sum units) [12].
Revised: sum units) [12].

Original: A logic constraintcan therefore be represented as a constraint circuit for SPLs, by utilizing any of the many toolsavailable for OBDDs [68] or SDDs [9, 50].
Revised: A logic constraintcan therefore be represented as a constraint circuit for SPLs, by utilizing any of the many toolsavailable for OBDDs [69] or SDDs [9, 51].

Original: Sec.
Revised: Sec.

Original: B illustrates in detail how to compile the exampleconstraint of Eq.
Revised: B illustrates in detail how to compile the exampleconstraint of Eq.

Original: (1) into the constraint circuit of Fig.
Revised: (1) into the constraint circuit of Fig.

Original: 3 in this way.
Revised: 3 in this way.

Original: 3.3 Efﬁcient inference in SPLs
Revised: 3.3 Efﬁcient inference in SPLs

Original: As discussed above, PCs can be expressive (D1) and are modular (D5), while constraint circuitsensure consistency (D3) for general constraints (D4).
Revised: As discussed above, PCs can be expressive (D1) and are modular (D5), while constraint circuitsensure consistency (D3) for general constraints (D4).

Original: What remains to be shown to complete SPLs isthat the product supports efﬁcient normalization (D2) and inference (D6), speciﬁcally that it allowsfor the efﬁcient evaluation of the normalization constant of r Θ , K , and its MAP state.
Revised: What remains to be shown to complete SPLs isthat the product supports efﬁcient normalization (D2) and inference (D6), speciﬁcally that it allowsfor the efﬁcient evaluation of the normalization constant of r Θ , K , and its MAP state.

Original: To this end, weneed to introduce the notion of compatibility between the two circuits [71].
Revised: To this end, weneed to introduce the notion of compatibility between the two circuits [72].

Original: Deﬁnition 3.7 (Compatible circuits in SPLs) .
Revised: Deﬁnition 3.7 (Compatible circuits in SPLs) .

Original: A smooth and decomposable conditional PC q ( Y ; Θ )is compatible over variables Y with a smooth and decomposable constraint circuit c K ( Y , X ) ifany pair of product units n ∈ p and m ∈
Revised: A smooth and decomposable conditional PC q ( Y ; Θ )is compatible over variables Y with a smooth and decomposable constraint circuit c K ( Y , X ) ifany pair of product units n ∈ p and m ∈

Original: m with the same scope over Y can be rearrangedto be mutually compatible and decompose in the same way: ( sc ( n ) = sc ( m ))
Revised: m with the same scope over Y can be rearrangedto be mutually compatible and decompose in the same way: ( sc ( n ) = sc ( m ))

Original: =
Revised: =

Original: ⇒ ( sc ( n i ) =
Revised: ⇒ ( sc ( n i ) =

Original: sc ( m i ) , n i and m i are compatible ) for some rearrangement of the inputs of n (resp.
Revised: sc ( m i ) , n i and m i are compatible ) for some rearrangement of the inputs of n (resp.

Original: m ) into n 1 ,
Revised: m ) into n 1 ,

Original: n 2(resp.
Revised: n 2(resp.

Original: m 1 , m 2 ).
Revised: m 1 , m 2 ).

Original: Fig.
Revised: Fig.

Original: 3 shows two circuits q
Revised: 3 shows two circuits q

Original: and c that are compatible.
Revised: and c that are compatible.

Original: Theorem 3.1 (Efﬁcient inference in SPLs) .
Revised: Theorem 3.1 (Efﬁcient inference in SPLs) .

Original: If q ( Y ; Θ ) and c K ( Y , X ) are two smooth, decomposableand compatible circuits, then computing Eq.
Revised: If q ( Y ; Θ ) and c K ( Y , X ) are two smooth, decomposableand compatible circuits, then computing Eq.

Original: (2) can be done in O ( | q || c | ) time.
Revised: (2) can be done in O ( | q || c | ) time.

Original: Furthermore, if they 217 are also deterministic, then computing the MAP state can be done in
Revised: Furthermore, if they 217 are also deterministic, then computing the MAP state can be done in

Original: O ( | q || c | ) time.
Revised: O ( | q || c | ) time.

Original: The proof can be found in Sec.
Revised: The proof can be found in Sec.

Original: A.
Revised: A.

Original: How do we come up with compatible circuits?
Revised: How do we come up with compatible circuits?

Original: One option is tohave a PC q that is compatible with every possible smooth and decomposable circuit c .
Revised: One option is tohave a PC q that is compatible with every possible smooth and decomposable circuit c .

Original: To do so,we can represent q as a mixtures of M fully-independent models; i.e., (cid:80) Mi
Revised: To do so,we can represent q as a mixtures of M fully-independent models; i.e., (cid:80) Mi

Original: =1 ω i (cid:81) j q ( Y j ; Θ i ) .
Revised: =1 ω i (cid:81) j q ( Y j ; Θ i ) .

Original: Thisadditional sum unit can be enough to be more expressive than a FIL and already delivers more accuratepredictions than any competitor, as our experiments in pathﬁnding show (Sec. 5).
Revised: Thisadditional sum unit can be enough to be more expressive than a FIL and already delivers more accuratepredictions than any competitor, as our experiments in pathﬁnding show (Sec. 5).

Original: An example of sucha circuit is shown in Fig.
Revised: An example of sucha circuit is shown in Fig.

Original: Another sufﬁcient condition for compatibility is that both q and c share theexact same hierarchical scope partitioning [71], sometimes called a vtree or variable ordering [12, 55].
Revised: Another sufﬁcient condition for compatibility is that both q and c share theexact same hierarchical scope partitioning [72], sometimes called a vtree or variable ordering [12, 56].

Original: This can be done easily if one ﬁrst compiles logical constraints into OBDDs or SDDs and then uses amechanized algorithm to build q as in [54] to create a compatible structure.
Revised: This can be done easily if one ﬁrst compiles logical constraints into OBDDs or SDDs and then uses amechanized algorithm to build q as in [55] to create a compatible structure.

Original: Additionally, to ensure qis a deterministic PC, we could exploit the mechanized construction proposed in Shih and Ermon [65].
Revised: Additionally, to ensure qis a deterministic PC, we could exploit the mechanized construction proposed in Shih and Ermon [66].

Original: Computing the exact MAP state, however, is of less concern as approximate inference algorithms,e.g., beam search decoding [75] or iterative pruning [11], are nowadays a commodity in deep learningframeworks.
Revised: Computing the exact MAP state, however, is of less concern as approximate inference algorithms,e.g., beam search decoding [76] or iterative pruning [11], are nowadays a commodity in deep learningframeworks.

Original: For non-deterministic PCs, we compute the MAP state with a faster approximation byreplacing non-deterministic sum units with max units [51].
Revised: For non-deterministic PCs, we compute the MAP state with a faster approximation byreplacing non-deterministic sum units with max units [52].

Original: This runs in time linear in the size of r ,and yet delivers state-of-the-art accuracies in our experiments Sec. 5.
Revised: This runs in time linear in the size of r ,and yet delivers state-of-the-art accuracies in our experiments Sec. 5.

Original: 3.4 A single-circuit SPL
Revised: 3.4 A single-circuit SPL

Original: The two-circuit design we proposed for SPLs provides a clear and theoretically-backed interfacebetween neural networks and probabilistic and symbolic reasoning.
Revised: The two-circuit design we proposed for SPLs provides a clear and theoretically-backed interfacebetween neural networks and probabilistic and symbolic reasoning.

Original: This setup, however, cansometimes be wasteful, as it requires to compute the product of two circuits and renormalize.
Revised: This setup, however, cansometimes be wasteful, as it requires to compute the product of two circuits and renormalize.

Original: Wecircumvent this issue by designing a single-circuit implementation of SPL.
Revised: Wecircumvent this issue by designing a single-circuit implementation of SPL.

Revised: [Table]

Original: Deﬁnition 3.8 (Single-circuit SPL) .
Revised: Deﬁnition 3.8 (Single-circuit SPL) .

Original: Given an input conﬁguration x , a single-circuit SPL computesp ( y | x ) =
Revised: Given an input conﬁguration x , a single-circuit SPL computesp ( y | x ) =

Original: c K ( Y , X ; Ω = g ( f ( X )))
Revised: c K ( Y , X ; Ω = g ( f ( X )))

Original: where c K is a neural conditional constraint circuit whosesum-unit parameters Ω are non-unitary values parameterized via a gating function g .
Revised: where c K is a neural conditional constraint circuit whosesum-unit parameters Ω are non-unitary values parameterized via a gating function g .

Original: [Table]

Original: In a nutshell, we can directly realize SPL by compiling a complex logical constraints (D4) into adeterministic constraint circuit c K , as before, and then parameterizing it with a gating function ofthe network embeddings f ( X ) , i.e., allowing its sum units to be non-unitary and input dependent.
Revised: In a nutshell, we can directly realize SPL by compiling a complex logical constraints (D4) into adeterministic constraint circuit c K , as before, and then parameterizing it with a gating function ofthe network embeddings f ( X ) , i.e., allowing its sum units to be non-unitary and input dependent.

Original: Since the support of c K is already restricted to exactly match the constraint K (D3), parameterizing Ωinduces an expressive probability distribution over the label conﬁgurations that are consistent with
Revised: Since the support of c K is already restricted to exactly match the constraint K (D3), parameterizing Ωinduces an expressive probability distribution over the label conﬁgurations that are consistent with

Original: K (D2).
Revised: K (D2).

Original: We can further guarantee that the circuit’s output are normalized probabilities (D1,D6) byenforcing the parameters ω of each sum unit to form a convex combination [53].
Revised: We can further guarantee that the circuit’s output are normalized probabilities (D1,D6) byenforcing the parameters ω of each sum unit to form a convex combination [54].

Original: This can be easilydone by utilizing a softmax activation function for g .
Revised: This can be easilydone by utilizing a softmax activation function for g .

Original: One of the advantages of the two-circuit implementation of SPLs is that the size of the circuitq Θ can be easily increased to improve the capacity of the model (Sec. 3.1).
Revised: One of the advantages of the two-circuit implementation of SPLs is that the size of the circuitq Θ can be easily increased to improve the capacity of the model (Sec. 3.1).

Original: The single-circuitimplementation is not as ﬂexible, as normally the number of parameters is determined by thecomplexity of the constraint circuit, which depends entirely on the compilation step.
Revised: The single-circuitimplementation is not as ﬂexible, as normally the number of parameters is determined by thecomplexity of the constraint circuit, which depends entirely on the compilation step.

Original: In this case,one option is to overparameterize the neural conditional circuit by introducing additional sum units,hence allowing it to capture more modes in the distribution.
Revised: In this case,one option is to overparameterize the neural conditional circuit by introducing additional sum units,hence allowing it to capture more modes in the distribution.

Original: We detail this process is Sec.
Revised: We detail this process is Sec.

Original: C. A sideeffect of overparameterization is that it relaxes determinism, meaning that MAP inference needs tobe approximated, as described in Sec. 3.3.
Revised: C. A sideeffect of overparameterization is that it relaxes determinism, meaning that MAP inference needs tobe approximated, as described in Sec. 3.3.

Original: Additionally, training a gating function to map relativelysmall embeddings to large parameter vectors in overparameterized circuits, can slow down training.
Revised: Additionally, training a gating function to map relativelysmall embeddings to large parameter vectors in overparameterized circuits, can slow down training.

Original: In such cases, a two-circuit implementation of SPL is to be preferred .
Revised: In such cases, a two-circuit implementation of SPL is to be preferred .

Original:  Related works
Revised:  Related works

Original: In this section, we position SPLs against state-of-the-art approaches for enforcing constraints onneural network predictions.
Revised: In this section, we position SPLs against state-of-the-art approaches for enforcing constraints onneural network predictions.

Original: In-depth surveys on this topic can be found in [18] and [33].
Revised: In-depth surveys on this topic can be found in [18] and [33].

Original: Energy-based models.
Revised: Energy-based models.

Original: Deep energy-based models (EBMs) replace FILs with an unnormalizedfactor graph [39] that captures higher-order label dependencies [40] (D2) but at the cost of foregoingprobabilistic semantics (D1) and efﬁciency (D6).
Revised: Deep energy-based models (EBMs) replace FILs with an unnormalizedfactor graph [40] that captures higher-order label dependencies [41] (D2) but at the cost of foregoingprobabilistic semantics (D1) and efﬁciency (D6).

Original: EBMs are typically unconcerned with hard constraints (D3).
Revised: EBMs are typically unconcerned with hard constraints (D3).

Original: Neural approaches for segmentation [44] and parsing [27, 78, 79] remedy to this byreplacing the factor graph with a full-ﬂedged intractable (discriminative) graphical model [39].
Revised: Neural approaches for segmentation [45] and parsing [27, 79, 80] remedy to this byreplacing the factor graph with a full-ﬂedged intractable (discriminative) graphical model [40].

Original: Togain efﬁciency, one can restrict EBMs to simpler graphical models (e.g., chains, trees), compromisingexpressiveness (D2) and the ability to model non-trivial logical constraints (D3, D4).
Revised: Togain efﬁciency, one can restrict EBMs to simpler graphical models (e.g., chains, trees), compromisingexpressiveness (D2) and the ability to model non-trivial logical constraints (D3, D4).

Original: Loss-based methods.
Revised: Loss-based methods.

Original: A prominent strategy consists of penalizing the network for producing inconsistent predictions using an auxiliary loss [18, 33].
Revised: A prominent strategy consists of penalizing the network for producing inconsistent predictions using an auxiliary loss [18, 33].

Original: While popular, loss-based methods, howevercannot guarantee that the predictions will be consistent at test time.
Revised: While popular, loss-based methods, howevercannot guarantee that the predictions will be consistent at test time.

Original: Common losses include translating logical constraints into a differentiable fuzzy logic [24, 25], as exempliﬁed by DL2 [28].
Revised: Common losses include translating logical constraints into a differentiable fuzzy logic [24, 25], as exempliﬁed by DL2 [28].

Original: Althoughefﬁcient (D6), this solution is not probabilistically sound (D1) and crucially is not syntax-invariant :different encodings of the same formula (e.g., conjunctive vs. disjunctive normal form) yield differentlosses [30, 23].
Revised: Althoughefﬁcient (D6), this solution is not probabilistically sound (D1) and crucially is not syntax-invariant :different encodings of the same formula (e.g., conjunctive vs. disjunctive normal form) yield differentlosses [30, 23].

Original: Closer to our SPL, the Semantic Loss (SL) [76] avoids this issue by penalizing thethe probability θ i associated to the i -th label by the neural network via the loss term
Revised: Closer to our SPL, the Semantic Loss (SL) [77] avoids this issue by penalizing thethe probability θ i associated to the i -th label by the neural network via the loss term

Original: [Equation]
Revised: [Equation]

Original: When K is compiled into a constraint circuit c K one retrieves −Z ( x ) for a two-circuit version of SPLthat is as expressive as FIL as it assumes independent labels via a conditional PC (cid:81)
Revised: When K is compiled into a constraint circuit c K one retrieves −Z ( x ) for a two-circuit version of SPLthat is as expressive as FIL as it assumes independent labels via a conditional PC (cid:81)

Original: i q ( Y i ; θ i ) .
Revised: i q ( Y i ; θ i ) .

Original: Theneuro-symbolic entropy (N E S Y E NT )
Revised: Theneuro-symbolic entropy (N E S Y E NT )

Original: [2] extends L SL by an entropy term that improves (but still
Revised: [2] extends L SL by an entropy term that improves (but still

Original: doesnot guarantee) consistency.
Revised: doesnot guarantee) consistency.

Original: It still makes the same independence assumptions over labels (D2).
Revised: It still makes the same independence assumptions over labels (D2).

Revised: Predicted paths do not exactly match the ground truth are still valid paths and yield very close costs to the ground truth.

Revised: Competitors’ predictions can have higher Hamming scores but be invalid.

Revised: More examples in Sec.

Revised: D.3.

Revised: [Table]

Original: Consistency layers.
Revised: Consistency layers.

Original: Approaches ensuring consistency by embedding the constraints into the predictive layer as in SPLs include MultiplexNet [36] and HMCCN [31].
Revised: Approaches ensuring consistency by embedding the constraints into the pre- dictive layer as in SPLs include MultiplexNet [36] and HMCCN [31].

Original: MultiplexNet is able to
Revised: MultiplexNet is able toencode only constraints in disjunctive normal form, which is problematic for generality (D4) andefﬁciency (D6) as neuro-symbolic SOP tasks involve an intractably large number of clauses – e.g. ourpathﬁnding experiments involves billions of clauses. HMCCN encodes label dependencies as fuzzyrelaxation and is the current state-of-the-art model for HMLC [31].

Original: [Table] encode Even 
Revised: Even its recent extension [32] isrestricted to a certain family of constraints (D4) that can be represented with fuzzy logic.

Original: Other approaches.
Revised: Other approaches.

Original: Other common approaches to neuro-symbolic SOP require to invoke a solver toeither obtain the MAP state or to compute (often only approximately) the gradient of the loss [22, 56,49 ].
Revised: Other common approaches to neuro-symbolic SOP require to invoke a solver toeither obtain the MAP state or to compute (often only approximately) the gradient of the loss [22, 57,50 ].

Original: SPLs have no such requirement.
Revised: SPLs have no such requirement.

Original: Some neuro-symbolic approaches [62] constrain the outputs ofneural networks within complex logical reasoning pipelines to solve tasks harder than neuro-symbolic
Revised: Some neuro-symbolic approaches [63] constrain the outputs ofneural networks within complex logical reasoning pipelines to solve tasks harder than neuro-symbolic

Original: SOP.
Revised: SOP.

Original: For instance, DeepProblog [45] uses Prolog’s backward chaining algorithm for ﬁrst orderlogical rules whose probabilistic weights are predicted by the network.
Revised: For instance, DeepProblog [46] uses Prolog’s backward chaining algorithm for ﬁrst orderlogical rules whose probabilistic weights are predicted by the network.

Original: In modern implementations of
Revised: In modern implementations of

Original: Problog, grounding a ﬁrst order program and then compiling it into constraint circuits [26] producesa conditional circuit akin to those we use in SPLs, but in which ( i ) only input distributions areparameterized and ( ii ) increasing the parameter count is not considered a straightforward operation.
Revised: Problog, grounding a ﬁrst order program and then compiling it into constraint circuits [26] producesa conditional circuit akin to those we use in SPLs, but in which ( i ) only input distributions areparameterized and ( ii ) increasing the parameter count is not considered a straightforward operation.

Revised: Scallop [37] provides a more scalable approach to deepproblog by considering only the top- k proofs.

Revised: We leave to future work how we could quickly compile only a speciﬁc query as DeepProblog/Scallopdo, to deal with ﬁrst-order representations efﬁciently

Original:  Experiments
Revised:  Experiments

Original: We evaluate SPLs on standard neuro-symbolic SOP benchmarks such as simple path prediction ,preference learning [76], shortest path ﬁnding in Warcraft [56] and HMLC [31].
Revised: We evaluate SPLs on standard neuro-symbolic SOP benchmarks such as simple path prediction ,preference learning [77], shortest path ﬁnding in Warcraft [57] and HMLC [31].

Original: We compare SPLsagainst several state-of-the-art loss- and layer-based approaches (Sec. 4) by applying them to thesame base neural network architecture as feature extractor f .
Revised: We compare SPLsagainst several state-of-the-art loss- and layer-based approaches (Sec. 4) by applying them to thesame base neural network architecture as feature extractor f .

Original: As we are interested in measuring howclose to the ground truth and how safe the predictions of all models are, we report the percentageof E XACT matches of the predicted labels, also called subset accuracy [70], and the percentage of
Revised: As we are interested in measuring howclose to the ground truth and how safe the predictions of all models are, we report the percentageof E XACT matches of the predicted labels, also called subset accuracy [71], and the percentage of

Original: C ONSISTENT predictions, also called “Constraint” [76].
Revised: C ONSISTENT predictions, also called “Constraint” [77].

Original: Note that SPLs as other consistency layers,is guaranteed to always output 100% consistent predictions.
Revised: Note that, like other consistency layers, SPLs are guaranteed to always output 100% consistent predictions.

Original: Additionally, we report the H AMMINGscore [70] mainly to maintain compatibility with previous experimental settings [76, 2].Note that
Revised: Additionally, we report the H AMMING score [71], mainly to maintain compatibility with previous experimental settings [77, 2].

Original: Hamming scores do not imply the consistency of a prediction and naturally favor competitors thatassume label independence and thus can minimize the per-label cross-entropy [21] (Table 3).
Revised: This metric does not consider consistency of predictions and naturally favors competitors that assumelabel independence and thus can minimize the per-label cross-entropy [21] (Table 3).

Original: Sec.
Revised: Sec.

Original: Dcollects all experimental details such as architectures and hyperparameters used for each experiment.
Revised: D collectsall experimental details such as architectures and hyperparameters used for each experiment.

Original: Simple path prediction & preference learning We start by comparing SPLs against loss-basedapproaches, reproducing the neuro-symbolic benchmarks of Xu et al.
Revised: Simple path prediction & preference learning. We start by comparing SPLs against loss-basedapproaches, reproducing the neuro-symbolic benchmarks of Xu et al.

Original: [76] for simple path predictionand preference learning.
Revised: [77] for simple path predictionand preference learning.

Original: In the ﬁrst experiment, given a source and destination node in an unweightedgrid G =
Revised: In the ﬁrst experiment, given a source and destination node in an unweightedgrid G =

Original: ( V, E ) , the neural net needs to ﬁnd the shortest unweighted path connecting them.
Revised: ( V, E ) , the neural net needs to ﬁnd the shortest unweighted path connecting them.

Original: Weconsider a 4 × 4 grid.
Revised: Weconsider a 4 × 4 grid.

Original: The input ( x , y ) is a binary vector of length | V | + | E | , with the ﬁrst | V | 315 variables indicating the source and destination nodes, and the subsequent | E | variables indicating a 316 subgraph G (cid:48)
Revised: The input ( x , y ) is a binary vector of length | V | + | E | , with the ﬁrst | V | 318 variables indicating the source and destination nodes, and the subsequent | E | variables indicating a 319 subgraph G (cid:48)

Original: ⊆ G .
Revised: ⊆ G .

Original: Each label is a binary vector of length | E | encoding the unique shortest path in
Revised: Each label is a binary vector of length | E | encoding the unique shortest path in

Original: G (cid:48) .
Revised: G (cid:48) .

Original: For each example, we obtain G (cid:48) by dropping one third of the edges in the graph G uniformly atrandom, ﬁlter out the connected components with fewer than 5 nodes, to reduce degenerate cases,and then sample a source and destination node uniformly at random from G (cid:48) .
Revised: For each example, we obtain G (cid:48) by dropping one third of the edges in the graph G uniformly atrandom, ﬁlter out the connected components with fewer than 5 nodes, to reduce degenerate cases,and then sample a source and destination node uniformly at random from G (cid:48) .

Original: The dataset consists of1600 such examples, with a 60 / 20 / 20 train/validation/test split.
Revised: The dataset consists of1600 such examples, with a 60 / 20 / 20 train/validation/test split.

Original: In the preference learning task, given a user’s ranking over a subset of items, the network has topredict the user’s ranking over the remaining items.
Revised: In the preference learning task, given a user’s ranking over a subset of items, the network has topredict the user’s ranking over the remaining items.

Original: We encode an ordering over n items as a binarymatrix Y ij , where for each i, j ∈ 1 , . . .
Revised: We encode an ordering over n items as a binarymatrix Y ij , where for each i, j ∈ 1 , . . .

Original: , n , Y ij indicates whether item i is the j th element in theordering.
Revised: , n , Y ij indicates whether item i is the j th element in the [Table] ordering.

Original: The input x consist of the user’s preference over 6 sushi types, and the model has to predictthe user’s preferences (a strict total order) over the remaining 4 .
Revised: The input x consist of the user’s preference over 6 sushi types, and the model has to predictthe user’s preferences (a strict total order) over the remaining 4 .

Original: We use preference ranking data over10 types of sushi for 5 , 000 individuals, taken from [46], and a 60 / 20 / 20 split.
Revised: We use preference ranking data over10 types of sushi for 5 , 000 individuals, taken from [47], and a 60 / 20 / 20 split.

Original: [Table] We employ a 5-layer and 3-layer MLP as a baseline for the simple path prediction, and preferencelearning resp. , equipped with FIL layer and additionally with the Semantic Loss [76] (MLP+ L SL ) orits entropic extension [2] (MLP+N E S Y E NT ).
Revised: We employ a 5-layer and 3-layer MLP as a baseline for the simple path prediction, and preference learning, respectively, equipped with FIL layer and additionally with the Semantic Loss [77](MLP+ L SL ) or its entropic extension [2] (MLP+N E S Y E NT ).

Original: We compile the logical constraints into an SDD [16]and then turn it into a the same constraint circuit c K that is used for L SL , N E S Y E NT (Sec. 4) andour 1-circuit implementation of SPLs.
Revised: We compile the logical constraints intoan SDD [16] and then turn it into a the same constraint circuit c K that is used for L SL , N E S Y E NT(Sec. 4) and our 1-circuit implementation of SPLs.

Original: Table 2 clearly shows that the increased expressiveness of SPL, coming from overparameterizing c K , allows to outperform all competitors while guaranteeingconsistent predictions as expected.
Revised: Table 2 clearly shows that the increased expressiveness of SPL, coming from overparameterizing c K , allows to outperform all competitors whileguaranteeing consistent predictions, as expected.

Original: Warcraft Shortest Path.
Revised: Warcraft Shortest Path.

Original: Next, we evaluate SPL on the more challenging task of predicting theminimum cost path in a weighted 12 × 12 grid imposed over terrain maps of Warcraft II [56].
Revised: Next, we evaluate SPL on the more challenging task of predicting theminimum cost path in a weighted 12 × 12 grid imposed over terrain maps of Warcraft II [57].

Original: Eachvertex is assigned a cost corresponding to the type of the underlying terrain (e.g., earth has lower costthan water).
Revised: Eachvertex is assigned a cost corresponding to the type of the underlying terrain (e.g., earth has lower costthan water).

Original: The minimum cost path between the top left and the bottom right vertices of the grid isencoded as an indicator matrix, and serves as a label.
Revised: The minimum cost path between the top left and the bottom right vertices of the grid isencoded as an indicator matrix, and serves as a label.

Original: As in [56] we use a ResNet18 [35] with FILoptionally with L SL as a baseline.
Revised: As in [57] we use a ResNet18 [35] with FILoptionally with L SL as a baseline.

Original: Given the largest size of the compiled constraint circuit c K in thiscase 10 10 , we use a two-circuit implementation of SPL.
Revised: Given the largest size of the compiled constraint circuit c K in thiscase 10 10 , we use a two-circuit implementation of SPL.

Original: Results in Fig.
Revised: Results in Fig.

Original: 1 and Table 3 are striking: notonly SPL outperforms competitors by a large margin – approx.
Revised: 1 and Table 3 are striking: notonly SPL outperforms competitors by a large margin – approx.

Original: +23 % over FIL and +19 % over the
Revised: +23 % over FIL and +19 % over the

Original: SL – but also consistently delivers meaningful paths that are very close to the ground truth in termsof cost, even when they encode very different routes.
Revised: SL – but also consistently delivers meaningful paths that are very close to the ground truth in termsof cost, even when they encode very different routes.

Original: See Sec.
Revised: See Sec.

Original: D.1 for a gallery of these examples.
Revised: D.3 for a gallery of these examples.

Original: Hierarchical Multi-Label Classiﬁcation.
Revised: Hierarchical Multi-Label Classiﬁcation.

Original: Lastly, we follow the experimental setup of Giunchigliaand Lukasiewicz [31] and evaluate SPL on 12 real-world HMLC tasks spanning four different domains: , 2 medical images, , and 1 text 
Revised: Lastly, we follow the experimental setup of Giunchigliaand Lukasiewicz [31] and evaluate SPL on 12 real-world HMLC tasks spanning four different domains: 8 functional genomics, 2 medical images, 1 microalgea classiﬁcation, and 1 text categorization.

Original: Fig.
Revised: Fig.

Original: 3 shows an example of a hierarchy of classes.
Revised: 3 shows an example of a hierarchy of classes.

Original: These tasks are especially challenging due to thelimited number of training samples, the large number of output classes, ranging from 56 to 4130 , aswell as the sparsity of the output space.
Revised: These tasks are especially challenging due to thelimited number of training samples, the large number of output classes, ranging from 56 to 4130 , aswell as the sparsity of the output space.

Original: For numeric features we replaced missing values by theirmean, and for categorical features by a vector of zeros, and standardized all features.
Revised: For numeric features we replaced missing values by theirmean, and for categorical features by a vector of zeros, and standardized all features.

Original: We used thevalidation splits to determine the number of layers in the gating function as well as the overparameterization, keeping all other hyperparameters ﬁxed.
Revised: We used thevalidation splits to determine the number of layers in the gating function as well as the overparameterization, keeping all other hyperparameters ﬁxed.

Original: The ﬁnal models were obtained by trainingusing a batch size of 128 and early stopping on the validation set.
Revised: The ﬁnal models were obtained by trainingusing a batch size of 128 and early stopping on the validation set.

Original: We compare our single-circuit SPLagainst HMCNN which was shown to outperform several other state-of-the-art HMLC approachesin Giunchiglia and Lukasiewicz [31].
Revised: We compare our single-circuit SPLagainst HMCNN which was shown to outperform several other state-of-the-art HMLC approaches in Giunchiglia and Lukasiewicz [31].

Original: We show the effect of increasing the expressivenss of SPLvia overparameterization in an ablation test in Sec.
Revised: We study the effect of increasing the expressivenss of SPL viaoverparameterization in Sec.

Original: D.2.
Revised: D.4.

Original: The results in Table 5 highlight that SPLsigniﬁcantly outperforms HMCNN in terms of exact match on 11 data sets and performs comparablyon 1 ,all while achieving nearly identical Hamming score.
Revised: The results in Table 5 highlight that SPL signiﬁcantly outperforms HMCNN in terms of exact match on 11 data sets performing comparably on 1 ,

Original:  Conclusion
Revised:  Conclusion

Original: Our SPLs offer the ﬁrst clear interface for integrating complex probabilitistic reasoning and logicalconstraints on top of any neural network classiﬁer while retaining efﬁcient inference and training.
Revised: SPLs offer the ﬁrst clear interface for integrating complex probabilitistic reasoning and logicalconstraints on top of any neural network classiﬁer while retaining efﬁcient inference and training.

Original: They improve by a noticeable margin the current state-of-the-art on challenging neuro-symbolic
Revised: They improve by a noticeable margin the current state-of-the-art on challenging neuro-symbolic

Original: SOP benchmarks such as pathﬁnding and HMLC.
Revised: SOP benchmarks such as pathﬁnding and HMLC.

Original: This opens up a number of interesting researchdirections.
Revised: This opens up a number of interesting researchdirections.

Original: First, SPLs can be extended to incorporate logical constraints over multiple networksand representable by ﬁrst-order formulas [45], which we plan to explore in future works, making thecircuit construction pipeline totally transparent to users [1] while possibly automatically learningconstraints from data [20, 47].
Revised: First, SPLs can be extended to incorporate logical constraints over multiple networksand representable by ﬁrst-order formulas [46], which we plan to explore in future works, making thecircuit construction pipeline totally transparent to users [1] while possibly automatically learningconstraints from data [20, 48].

Original: Second, we are interested in leveraging SPLs to inject scalable logical
Revised: Second, we are interested in leveraging SPLs to inject scalable logicalconstraints into large language models [6] thus equipping them with probabilistic reasoning [29, 78].

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes] Please see section 1 for a summary of our contributions
Revised: [Yes] Please see section 1 for a summary of our contributions

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] Please see Supplementary
Revised: [Yes] Please see Supplementary

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] Please see
Revised: [Yes] Please see

Original: Supplementary
Revised: Supplementary

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main exper-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main exper-

Original: imental results (either in the supplemental material or as a URL)?
Revised: imental results (either in the supplemental material or as a URL)?

Original: [Yes] Please seesupplementary material
Revised: [Yes] Please see supplementary material

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] Please see section 5 for details regarding the dataset splits.
Revised: [Yes] Please see section 5 for details regarding the dataset splits.

Original: All
Revised: All

Original: other details are included in the supplementary.
Revised: other details are included in the supplementary.

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] We reported error bars and signiﬁcance testing for the
Revised: [Yes] We reported error bars and signiﬁcance testing for the

Original: hierarchical multi-label classiﬁcation experiment, please see
Revised: hierarchical multi-label classiﬁcation experiment, please see

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: of GPUs, internal cluster, or cloud provider)
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [Yes] Please see supplementary material

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: [Table]

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: [Table]

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table]
Revised: Board

Original: Board (IRB) approvals, if applicable?
Revised: (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
Revised: c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]