Title: Grounding Neural Inference with Satisfiability Modulo Theories
Venue: NeurIPS 2023
Accepted: yes
Keywords: Satisfiability Modulo Theories

Solver Layer

Combinatorial Problem

MAXSAT

SAT
Abstract: Recent techniques that integrate solver layers into Deep Neural Networks (DNNs) have shown promise in bridging a long-standing gap between inductive learning and symbolic reasoning techniques. In this paper we present a set of techniques for integrating Satisfiability Modulo Theories (SMT) solvers into the forward and backward passes of a deep network layer, called SMTLayer.
Using this approach, one can encode rich domain knowledge into the network in the form of mathematical formulas.
In the forward pass, the solver uses symbols produced by prior layers, along with these formulas, to construct inferences; in the backward pass, the solver informs updates to the network, driving it towards representations that are compatible with the solver's theory.
Notably, the solver need not be differentiable. We implement SMTLayer as a Pytorch module, and our empirical results show that it leads to models that 1) require fewer training samples than conventional models, 2) that are robust to certain types of covariate shift, and 3) that ultimately learn representations that are consistent with symbolic knowledge, and thus naturally interpretable.

Grounding Neural Inference with Satisfiability Modulo Theories

Abstract

Recent techniques that integrate solver layers into Deep Neural Networks (DNNs) have shown promise in bridging a long-standing gap between inductive learning and symbolic reasoning techniques. In this paper we present a set of techniques for integrating Satisfiability Modulo Theories (SMT) solvers into the forward and backward passes of a deep network layer, called SMTLayer . Using this approach, one can encode rich domain knowledge into the network in the form of mathematical formulas. In the forward pass, the solver uses symbols produced by prior layers, along with these formulas, to construct inferences; in the backward pass, the solver informs updates to the network, driving it towards representations that are compatible with the solver's theory. Notably, the solver need not be differentiable. We implement SMTLayer as a Pytorch module, and our empirical results show that it leads to models that 1) require fewer training samples than conventional models, 2) that have consistent performance against certain types of covariate shift, and 3) that ultimately learn representations that are consistent with symbolic knowledge, and thus naturally interpretable. Our code is available at https://github.com/cmu-transparency/smt-layer

1 Introduction

Deep neural networks (DNNs) have recently made significant strides, achieving surprising levels of performance on tasks like question-answering [40], text summarization [27], and code generation [4, 21]. However, the ability of models that perform well on these benchmarks to consistently demonstrate sound logical reasoning, even on tasks that may appear to be more simple and self-contained, remains in question [6, 31, 39]. For example, a prediction market on whether GPT-4 will be able to consistently solve 'easy' Sudoku puzzles from the LA Times has remained open for several months at the time of this writing, despite the prompt-tuning efforts of traders on the market [19].

One way to address this issue is to encode logical constraints that are essential for certain inference tasks symbolically, making them available to the model either during training or inference. A promising approach to this type of 'neuro-symbolic' learning is to integrate solver layers into the model [35, 34, 13, 18], both during training and inference. During training, the model must learn a representation of the data that is compatible with the symbolic knowledge, and during inference this

grounded representation is used to provide information from which a symbolic reasoning engine (i.e., a 'solver') can extract accurate results.

The most straightforward way to incorporate a solver layer into a deep model is to learn representations that are compatible with symbols used by the solver. For example, if one wanted to leverage symbolic domain knowledge to classify images of birds, or diagnose ailments from CT scans, then one could train a model as in 'concept bottlenecking' [14]. This requires detailed supervised labels, which may be prohibitively expensive to obtain and keep consistent with a potentially evolving domain theory.

We present a set of techniques for incorporating a Satisfiability Modulo Theories (SMT) solver into a DNN layer so that symbolic knowledge can be leveraged to learn such a compatible representation, without requiring label supervision . Our method, unlike prior methods, does not require fine-grained labels to learn representations [14]. For example, for learning Visual Sudoku, regular supervised learning requires labels of individual digits, rather than just the solution to a set of Sudoku puzzles, to learn a dedicated digit classifier first. Our work sidesteps the need to manually break a problem apart and obtain intermediate labels to supervise on, and instead allows for learning a predictor end-to-end from just labels on the targeted task. Our approach is general, and can handle a broad range of domain knowledge encoded as SMT constraints, provided that they interface with the surrounding neural network layers over propositional variables. Unlike related prior work [35], our approach does not approximate the solver's behavior by formulating a differentiable relaxation. Rather, as the solver works on a set of constraints, we extract information that is geared towards checking the correctness of preceding layers, and use that information to construct training updates (Section 4.2).

We present two different approaches for this, one based on unsatisfiable cores, and another based on weighted MaxSMT (Section 4.2). There are several advantages to this approach. Aside from the mild interface constraints mentioned earlier (i.e., solver and neural layers interface with each other via boolean variables), our approach does not place any restrictions on the theory solver embedded in the layer, such as linearity [34] or even decidability-if the solver is capable of efficiently discharging the relevant constraints, then the layer can operate as intended. Because there is no need to provide a differentiable relaxation for each theory or solver technique that one may want to incorporate, we can leverage the continuous and unabated progress being made in solver technology.

We implement our approach as a PyTorch [22] layer, using the Z3 [7] SMT solver as the solver layer to solve SMT and MaxSMT constraints. On three applications involving vision and natural language: visual arithmetic, algebraic equation solving, and a so-called natural language 'liar's puzzle,' we demonstrate that our implementation can be incorporated into DNN architectures to solve problems more effectively than conventional DNNs (Section 5). In particular, our results show that the data needed to train a DNN with symbolic knowledge may be much simpler than may be necessary otherwise, and that while doing so is more expensive computationally, often times the more efficient (i.e., not involving MaxSMT) algorithms perform well in practice.

Our contributions are as follows:

We present SMTLayer , a framework for incorporating an SMT solver into a DNN, as a layer that leverages symbolic knowledge during training and inference.

Our empirical evaluation, over four types of tasks requiring logical reasoning, e.g. visual Sudoku, shows that models using SMTLayer require significantly less training data, can be trained more efficiently, and generalize a lot better on the much larger test sets compared to those based on closely-related prior work [35, 13].

Section 3 provides background, Section 4 describes SMTLayer , Section 5 gives our empirical evaluation, and Appendix A.1 (supplementary material) discusses the broader impacts of our work.

2 Related Work

Solver Layers. Vlastelica et al. [34] integrate a black-box and non-differentiable combinatorial solver on top of a deep network. To propagate the gradient through the solver on the backward pass, they linearly interpolate the loss w.r.t the solver's input and define the gradient of the solver as the slopes of the line segments. CSL solves a set of problems where the solver's objective must be linear w.r.t its input, e.g. finding the shortest path and travel salesman problem (TSP). Further, the authors assume that the only labels available are the outputs of solvers, e.g. the minimum cost in TSP, and

hence their tool has to discover the label for the output of the network itself. These requirements limit the choices one has for the solver layer. Wang et al. [35] present SATNet, a network architecture with a differentiable approximate MAXSAT solver layer. Their approximation is based on a coordinate descent approach to solving the semidefinite program (SDP) relaxation of the MAXSAT problem. SATNet does not assume that the logical structure of the problem is given, and instead attempts to learn it.

Neural Logic Programming. While SATNet integrates a logic-based solver on top of a network, DeepProbLog takes the opposite approach, extending the capability of a probabilistic logic programming language with neural predicates [18]. In the context of our work, the logic program can be viewed as a 'solver layer' that explicitly encodes symbolic knowledge. Scallop [13] extends DeepProbLog to scale without sacrificing accuracy compared to DeepProbLog. Similarly to DeepProbLog, each possible result of the sum of two digits in MNIST is given a probability, in the form of a weighted Boolean formula. They prune unlikely clauses of the formula, represented by proofs, only keeping the topk most likely. Likelihood is computed using weighted model counting [13, 3]. These techniques are well-suited to problems that benefit from probabilistic Datalog, but have inherent limitations: they cannot handle quantifiers, general negation, and the range of supported first-order theories is more restrictive.

More recent work explores different directions in training models to perform neural-symbolic reasoning. For example, SMT solvers and MCMC sampling are utilized to support network training, side-stepping shortcut satisfaction [15, 16]. Hoernle et al. [12] introduce MultiplexNet, a deep model that ensures the satisfaction of symbolic constraints during the inference stage and adds an additional dimension to the neuro-symbolic learning landscape. Marconato et al. [20] has investigated how to mitigate the challenges and limitations associated with neuro-symbolic reasoning shortcuts and Yang et al. [37] use straight-through-estimators and logical constraints for neural network learning.

Differentiable Logic & Semantic Losses. Another recent direction has explored differentiable logics [9, 33, 32], loss functions that incorporate logical semantics [1, 36], or training methods guided by symbolic solvers [10, 28]. These approaches provide ways of integrating symbolic knowledge into training, by making logical formulas differentiable, and therefore amenable to optimization when included in a loss function. This line of work does not explicitly aim to make use of symbolic information during inference. In contrast, the information that our approach extracts from the solver during training is used to condition the model towards a representation that will allow it to communicate effectively with the solver during inference. Additionally, we do not require the logical formulas, or the solver, to be differentiable.

3 Background

Let X denote a domain of features, Y a domain of labels, and D a distribution over X × Y . Formally, D is a probability measure on a space given by a σ -algebra over subsets of ℘ ( X × Y ) . The goal of a learning algorithm A is to find a function h : X → Y that, for ( x, y ) ∼ D , can be used to predict y when given x . To do this, A is given a set of training examples S = ( x 1 , y 1 ) , . . . , ( x m , y m ) sampled i.i.d. from D , and uses some criterion to select h from a hypothesis class H of functions. We refer to h as the model learned by A on S . When the learning algorithm A is clear from the context, we will write h S to denote the model produced from the given sample. Throughout this paper, we will generally assume that the loss is either the 0-1 loss /lscript 01 or binary cross-entropy /lscript bce .

A theory T consists of a signature Σ of constant, predicate, and function symbols, as well as a set of axioms over Σ . Formulas in a theory are composed of elements of Σ , variables, and logical symbols such as quantifiers and Boolean operations. We use the term decision procedure to refer to an algorithm that is given an open T -formula, and returns true if it is satisfiable, and false otherwise. Additionally, it may return an assignment to all of the variables that demonstrates satisfiability, or if the formula is not satisfiable, then it may return an unsatisfiable core , which is a subset of clauses taken from the formula's representation in conjunctive normal form that remains unsatisfiable. Loosely, we also refer to such an algorithm as a 'solver', but this term is more general, and could also refer to an algorithm that identifies the maximal set of clauses, possibly weighted by some user-defined values, that are satisfiable when conjoined.

Features

X

Z

0010000111

Neural

Network

φ

(

z

1

‖

. . .

‖

z

10

,

y

)

≡

a

=

1

z

1

>

0

‖

. . .

‖

1

z

5

>

0

∧

b

=

1

z

5

>

0

‖

. . .

‖

1

z

10

>

0

∧

a

+

b

=

y

Prediction Logic

φ

Y

{

01011

}

Satisfying

Assignments

Algorithm 1:

F

φ

smt

(

z

)

Forward Pass

Inputs:

z

∈

R

p

layer input

φ

(

z

0

, . . . , z

p

-

1

, y

0

, . . . , y

q

-

1

)

T

-formula

Output:

y

∈

R

q

1

begin

2

z

b

[

z

[

i

]

>

0

:

i

= 0

. . . p

-

1]

3

ˆ

φ

φ

(

z

b

[0]

, . . . , z

b

[

p

-

1])

4

if

ˆ

φ

is satisfiable

then

5

y

b

[0]

, . . . , y

b

[

q

]

solve

( ˆ

φ

, y

0

, . . .

)

6

y

[

y

b

[

i

]

>

0

:

i

= 0

. . . q

-

1]

7

else

8

y

0

9

end

10

return

y

11

end

4 Constructing SMTLayer

In this section, we present SMTLayer , a set of algorithms for computing the forward and backward passes of a layer whose behavior is defined by a set of user-defined SMT constraints. SMTLayer does not have trainable parameters, and its functionality is wholly defined by a set of SMT constraints φ that are provided by the model designer. SMTLayer can be used in modern deep-learning frameworks as a drop-in replacement for more conventional neural network layers, e.g., dense, convolutional, and LSTM [11] are prominent examples of widely-used layers.

Section 4.1 provides a high-level overview of our approach, Section 4.2 describes them in detail, and Section 4.3 begins an analysis of this setting that we hope future work will continue developing.

4.1 Overview

We envision SMTLayer being used primarily at the top of a DNN taking inputs from a stack of conventional DNN layers that convert raw input features into ground terms for the constraints φ ( z 0 , . . . , z p -1 , y 0 , . . . , y q -1 ) embedded in SMTLayer , and producing outputs that are consistent with φ and the given ground terms. Figure 1 shows an illustrative example, with the previously-studied problem of MNIST addition [18, 13].

During the forward pass the outputs of the previous layer are mapped to designated free variables z 0 , . . . , z p -1 . The layer then checks the satisfiability of φ , a formula in an appropriate combination of first-order theories, after substituting these ground terms for the z i , and the output of the layer consists of the solver's model for y 0 , . . . , y q -1 . These outputs are converted from Boolean to floating-point values by mapping false to -1 and true to 1. At the moment, the only restriction on φ that our layer requires is that z and y be vectors of Booleans, so that they can be appropriately mapped to continuous values; any other symbols appearing in φ can come from arbitrary domains (e.g. strings) supported by the underlying SMT solver.

In the backward pass, the layer receives the gradient of its output with respect to the function whose derivative is being computed, which we will assume is the binary cross-entropy loss /lscript ( y, y /star ) . Unless stated otherwise, we will assume this loss for the remainder of the section. This gradient is used, along with the inputs and outputs of the corresponding forward pass, to first compute an amended output ˆ y which corresponds to an output that would have yielded a smaller loss. Because the outputs are Boolean, it is always possible to determine the ground truth output y /star from this information. Using ˆ y , the layer determines which of components of its inputs are inconsistent with φ and ˆ y , and provides the corresponding gradients to the previous layer. Section 4.2 details the manner in which these gradients are computed.

4.2 SMTLayer , forward and backward

We now present the details of the forward and backward passes of SMTLayer . There are two algorithms for each pass, F φ smt and F φ max are forward passes, and B φ core , B φ max are backward passes. F φ max and B φ max both make use of MaxSMT solvers, whereas F φ smt and B φ core rely on satisfiability solvers (SMT). They are all compatible with each other. That is, F φ smt (and F φ max ) can be used with either B φ max or B φ core . Algorithms for F φ smt and B φ core are included in this section, and the MaxSMT-based variants are detailed in Appendix A.2.

Forward pass. Algorithm 1 illustrates F φ smt , our SMT-based forward pass, and Algorithm 3 for computing F φ max is given in Appendix A.2. Both of the algorithms are parameterized by a user-provided first-order formula φ , and take a single vector-valued input consisting of unscaled floating-point values ( logits ). These values are cast to Booleans by taking their sign on line 2 of both algorithms, and they are equated with the corresponding free variables z 0 , . . . , z p -1 .

The key difference between F φ max and F φ smt is the way in which they handle inputs that are inconsistent with φ when interpreted as Booleans. F φ smt addresses this by providing an output that is also inconsistent with φ , i.e. a vector of zeroes, effectively signaling that the network below it did not provide consistent inputs. Alternatively, we can interpret the values provided by the network as Booleans enriched with 'confidence' values. To obtain MaxSMT weights, Algorithm 3 scales the input logits to a formal probability distribution via the softmax function (line 3). With this approach, SMTLayer will always provide a valid, although not necessarily correct, output that is consistent wrt φ with the inputs of which the network below is most 'confident.' (line 4).

Backward pass. The backward pass is responsible for computing the gradient of the loss with respect to the layer inputs. It is given the gradient of the loss with respect to the layer outputs, and is assumed to have memoized the inputs and outputs from the forward pass. The gradients returned by this pass are then used by the backward pass of previous layers, and ultimately to derive updates to trainable parameters.

The key issue in designing a backward pass for SMTLayer is the geometry of the functions computed by either forward pass. For any vector v ∈ { -1 , 0 , 1 } p and x, x ′ with sign ( x ) = sign ( x ′ ) = v , then B φ · ( x ) = B φ · ( x ′ ) , so these functions are piece-wise constant step functions ranging over the corners of the R q unit hypercube. Thus, while they are differentiable almost everywhere, the gradient is not helpful for training because it is always zero. Prior work

B

φ

core

(

z, y,

∂

y

/lscript

(

y, y

/star

))

unsat core-based backward pass

Inputs:

z

∈

R

p

input of forward pass

y

∈

R

q

output of forward pass

∂

y

/lscript

(

y, y

/star

)

gradient with respect to output

φ

(

z

0

, . . . , z

p

-

1

, y

0

, . . . , y

q

-

1

)

a

T

-formula.

Output:

∂

z

/lscript

(

y, y

/star

)

∈

R

p

approximate gradient of

/lscript

1

begin

2

G

z

∂

z

/lscript

(

z,

sign

(

z

))

3

ˆ

y

=

sign

(

y

)

-

2

·

sign

(

∂

y

/lscript

(

y, y

/star

)

)

4

if

sign

(

y

)

=

sign

(ˆ

y

)

then

5

z

b

[

z

[

i

]

>

0 :

i

= 0

. . . p

-

1]

ˆ

y

b

[ˆ

y

[

i

]

>

0 :

i

= 0

. . . q

-

1]

6

φ

z

,

φ

y

∧

0

i<p

z

i

=

z

b

[

i

]

,

∧

0

i<q

y

i

= ˆ

y

b

[

i

]

7

I

arg min

I

⊆

[0

,p

)

1

(

¬

φ

∨

¬

φ

y

∨

∨

i

∈

I

z

i

=

z

b

[

i

])

· |

I

|

8

foreach

i

∈

I

do

9

G

z

[

i

]

∂

z

[

i

]

/lscript

(

z

[

i

]

,

1

-

sign

(

z

[

i

]))

10

end

11

foreach

i

∈

¯

I

do

12

G

z

[

i

]

0

13

end

14

end

15

return

G

z

16

end

/negationslash on integrating such functions into deep networks [35, 34] addresses this problem by relaxing the function computed by the forward pass, so that its gradients are hopefully more informative.

In contrast, B φ core (Algorithm 2) and B φ max (Algorithm 4 in Appendix A.2) do not attempt to provide gradients for a relaxation of the forward pass. Instead, they use information provided by the solver in its computation of the forward pass to identify which components of the input may have contributed to higher loss. The gradient is then computed by constructing a variant of the input provided to the forward pass, which differs on the identified components, and returning the gradient of the BCE loss of the original input on this variant. The two algorithms differ in the information that they extract from the solver, i.e., either solutions to a MaxSMT instance or an unsatisfiable core.

Both algorithms begin by initializing the gradient to be the loss between the logit inputs, and their hard labels (line 2). Recall that we assume the loss /lscript is binary cross-entropy, so the result will not

/negationslash

be zero. The purpose of this initialization is to emulate the dynamics of training with cross-entropy loss with a conventional layer; when the rounded output matches the target, the loss is not zero, and training will continue to move the parameters in a direction that makes them agree 'more' with the hard target.

/negationslash

One line 3, they then use the provided gradient from the subsequent layer together with the memoized output from the forward pass to construct ˆ y , a 'corrected' output that satisfies /lscript (ˆ y, y /star ) ≤ /lscript ( y, y /star ) . To understand why, observe that the sign of ˆ y computed on line 3 of both algorithms must be equal to that of y /star . This follows from two facts: (1) at any coordinate i where y [ i ] = y /star [ i ] , sign ( ∂ y /lscript ( y, y /star ))[ i ] = sign ( y )[ i ] ; (2) at any coordinate i where y [ i ] = y /star [ i ] , sign ( ∂ y /lscript ( y, y /star ))[ i ] = -1 · sign ( y )[ i ] . If the sign of ˆ y is the same as that of y , then both algorithms return the initialized gradient. Otherwise, they extract information from the solver using z and ˆ y .

Algorithm 2 that runs B φ core identifies a set of constraints z i = z b [ i ] that are inconsistent with φ ∧ φ y . Note that line 7 specifies a minimal unsatisfiable core, but this is not necessary. All that is needed is that none of the clauses in the core be superfluous, i.e., deleting any singleton clause from I will cause it to be satisfiable. If a superfluous clause remains in the core, then the gradient returned for the corresponding input will have the incorrect sign, which may lead to issues with training. B φ core then updates the gradient at each input identified in the core using the loss of z [ i ] with respect to 1 -sign ( z [ i ]) , which will lead to updates in a direction that would have modified the input such that i was not in the unsat core. The indices not in the unsat core have their gradients set to zero, as their absence in the core is not evidence that these inputs were correct or incorrect.

Algorithm 4 (in Appendix A.2) that runs B φ max instead constructs a set of clauses φ y that constrain the free y 0 , . . . , y q -1 to take the values of ˆ y b , the Boolean conversion of ˆ y . It then computes the softmax values of the absolute incoming logits | z | , and uses them to find the maximally-weighted set of clauses ( softmax ( | z | )[ i ] , z i = z b [ i ]) that are consistent with φ ∧ φ y . Intuitively, these are the inputs that the previous layer is most confident in that can be made consistent with the corrected label ˆ y by changing some of the less confident inputs. B φ max then updates the initialized gradient at each index for which the solution to this instance does not match the sign of the original input.

4.3 Analysis

To understand the settings where SMTLayer will provide optimal results, we introduce a class of decomposable learning problems (Definition 1).

Definition 1 (Decomposable problem) . Let T be a first-order theory with constants in Z . An ERM problem D , H is decomposable by T if there exists a function f : X → Z , companion hypothesis class H f ⊆ X → Z , and T -formula φ such that:

For any h ∈ H , there exists h f ∈ H f and h ′ such that h = h ′ ◦ h f .

There exists a random function g : ℘ ( Y ) → Y such that for any n > 0 and ∀ S in the support of D n ,

In (2), f is called the grounding function and φ is called the prediction logic .

Intuitively, a learning problem defined in terms of a distribution D and hypothesis class H is decomposable if members of H can be decomposed into functions that are responsible for grounding and prediction, and D can be expressed in terms of a grounding function and a first-order formula φ . There are a few important things to note. First, there is no requirement that the grounding function f be a member of H f . While this may be realized at times, we should not assume that the data is actually generated by a function in the class that one learns over. In fact, we do not assume that f is efficiently computable, as it may correspond to a natural process, or an aspect of data generation that is not understood well enough to make computational claims.

Second, for a given x , there may be more than one satisfying assignment for y to φ ( f ( x ) , y ) . The function g in (2) accounts for this, requiring only that when solutions to φ ( f ( x ) , y ) are sampled by g , the result is distributed identically to D . This paper will focus on the case where satisfying

assignments for y are unique, as these are more in line with 'classic" ERM classification problems. We leave exploration of the more general setting to future work.

We note that if the grounding function is known, can be computed efficiently, and φ is efficiently solvable, then the learning problem effectively has a closed-form solution. Rather, we assume that only φ and perhaps g are known, and a sample of D is given. The remaining challenge is to identify a grounding hypothesis h f ∈ H f for which the construction in (2) is an effective solution to the end-to-end learning problem posed by D , H . This stands in contrast to traditional ERM, in which a good solution h ∈ H must either solve both grounding and prediction, or find a 'shortcut' that manages to predict D as well as the decomposition.

Convergence. Regarding the backward passes, Theorem 2 below demonstrates that when φ satisfies certain conditions, and the companion hypothesis class H f satisfies conditions that are sufficient to guarantee convergence with SGD, then training with F φ smt and B φ max will converge to the optimal solution in the number of iterations. The proof of this theorem is based on the observation that when the conditions on φ are met, then training with B φ max obtains the same solution that would be obtained if the labels of φ were available for supervised learning, so it is possible to use the solver's solutions interchangeably with the correct supervised labels. Thus, the conditions on H f are sufficient to ensure the stated convergence, as stated in a well-known result outlined in Chapter 14 of [26].

It is also worth noting that Theorem 2 does not necessarily hold if B φ core is used instead of B φ max . The reason is that there may be many unsatisfiable cores that are locally minimal in cardinality, and gradients are set only for inputs that appear in the computed core. These gradients will not match those of the loss on a grounding sample, so the training dynamics are likely to be different. We believe that training with B φ core may have more in common with block coordinate descent than gradient descent, and save a more detailed exploration of the topic for future work.

Theorem 2. Let D , H be a T -decomposable problem with grounding function f and prediction logic φ where:

Z and Y are Cartesian products of Booleans.

/negationslash

For any ( x, y ) ∼ D and y ′ = y , φ ( f ( x ) , y ′ ) is T -equivalent to false and there is exactly one z such that φ ( z, y ) is T -equivalent to true.

H f is a convex set and for all h f ∈ H f , ‖ h f ‖ ≤ B , and the loss /lscript ( h f ( · ) , z ) is M -Lipschitz and convex in x for any fixed z .

Then for any /epsilon1 > 0 , selecting h f by minimizing either L S ( F φ smt ( h f ( · ))) with τ ≥ M 2 B 2 / /epsilon1 2 iterations of stochastic gradient descent, with gradients provided by B φ max , and learning rate η = √ B 2 / M 2 τ yields a grounding hypothesis ˆ h f ∈ H f that satisfies: E [ L D ( ˆ h f )] ≤ min h f ∈ H f L D ( h f ) + /epsilon1 . The randomness in this expectation is taken over the choices of the SGD algorithm.

5 Experimental Evaluation

In this section we present an empirical evaluation of SMTLayer on four learning problems that can be decomposed into perceptual and symbolic subtasks. Our results demonstrate the following primary findings. 1) SMTLayer is effective: on every benchmark, it provides superior results over 'conventional' learning that takes place without encoded symbolic knowledge. 2) SMTLayer has distinct advantages over prior approaches. Compared with SATNet [35], it requires significantly less training data to converge, and in all cases yields a more accurate model; compared with Scallop [13], it is less computationally expensive, requires less training data, and it is more expressive in terms of the knowledge that it can encode; compared to approaches that incorporate symbolic knowledge into training, but do not use it during inference [1], SMTLayer gives superior results on test data. 3) Models trained with SMTLayer may be more robust to certain types of covariate shift that occur relative to the symbolic component of the problem; when SMTLayer succeeds at learning a representation, it will continue to produce correct inferences provided the perceptual component remains stationary.

5.1 Datasets

Additional details on the datasets and corresponding architectures used in our evaluation can be found in Appendix A.4. Hyper-parameters used for training are in Appendix A.5.

MNIST Addition. The MNIST addition problem is illustrated in Figure 1, and is similar to the benchmark described by [13]. For training, we use 'MNIST + p %" to denote a training set of size 60,000 that contains p % of the possible pairs of digits. So p = 100 indicates all possible pairs of digits are used, and for p = 10 , we only use pairs of the same digit. We use p = 10 , 25 , 50 , 75 and 100 in our experiments. In all cases, we use the same test set of all possible pairs of digits.

Visual Algebra. The task is to solve for the variable x in a graphical depiction of the equation ax + b = c , where a, b and c are randomly-chosen numbers, and each symbol is depicted visually using EMNIST [5] and HASY graphics [29]. Similar to MNIST addition, the training sample selects a and b uniformly from pairs of the same digit, and x uniformly from the odd numbers between 0 and 9. The test sample was generated by sampling a, b , and x uniformly.

Liar's Puzzle. The liar's puzzle is comprised of three sentences spoken by three distinct agents: Alice, Bob, and Charlie. One of the agents is 'guilty' of an unspecified offense, and in each sentence, the corresponding agent either states that one of the other parties is either guilty or innocent. It is assumed that two of the agents are honest, and the guilty party is not. The objective is to identify the guilty party. A formal characterization of the underlying logic is given in Appendix A.4. We note that the logic has non-stratified occurrences of negation, so it cannot be encoded with Scallop.

Visual Sudoku. This task is to complete a 9 × 9 Sudoku board where each entry is an MNIST digit. We use the dataset from the SATNet evaluation [35], and examine three configurations obtained by sampling 10%, 50%, and 100% of the original training set. Although there are examples of Sudoku solvers implemented as logic programs, we were not able to implement one in Scallop without violating stratified negation. When calculating accuracy, we check if the entire board is correct.

5.2 Setup

We implemented a prototype of our approach using Pytorch [22] and Z3 [7], which will be made available in open-source when this paper is published. When training models with SMTLayer , we use SGD with Nesterov momentum at rate 0 . 9 and gradient clipping rate 0.1. Before training a model with SMTLayer (or a comparison technique, unless stated otherwise), we first pre-train the neural network by replacing SMTLayer with a dense network containing one hidden layer of 512 neurons. This can potentially limit the number of training updates needed at lower layers, but will not result in a model with a representation that is compatible with symbolic knowledge, so further training is needed. Results in Table 1 (left) were averaged over five runs of training. Results in Table 1 (right) for SATNet [35] and semantic strengthening (Ahmed et al. [1]) were reported in the original papers; details on our fine-tuning of GPT2-XL are given in Appendix A.6.

5.3 Results

Overall performance. In terms of accuracy, Table 1 (left) shows that SMTLayer outperforms prior work in terms of accuracy, training time, or both, on all configurations. SMTLayer is consistently faster than Scallop, nearly 4 × in the case of visual algebra. The per-epoch time to train the SATNet models is less expensive than SMTLayer , but this is not always conclusive. In the case of visual sudoku, the 10% SMTLayer model achieved superior error rates in 20 epochs, compared with 100 epochs for the 100% SATNet model. We also point out that although Theorem 2 suggests that Algorithm 4 might have learning advantages over Algorithm 2 found this not to be the case on these datasets. All of the results in Table 1 (left) were trained with Algorithm 2, and test inference was done using Algorithm 1.

Solvers during inference. Table 1 (right) shows the performance of several approaches for solving (non-visual) Sudoku puzzles presented as text. The purpose of this comparison is to highlight the advantages of using symbolic information during inference, as done by SMTLayer , SATNet, and Scallop, versus only during training, as in the case of Semantic Strengthening [1]. We include several

transformer-based approaches in this comparison, as there is widespread interest in their ability to perform this type of reasoning, despite not making explicit use of symbolic information. Although Semantic Strengthening and transformer-based approaches could be used to solve Visual Sudoku (in the latter case, via a vision encoder-decoder), we are not aware of a public implementation of either, so these numbers represent an upper-bound on what their performance on that benchmark would be.

The top row shows the performance of Z3 on the benchmark test set, and is included as a baseline for the time needed to solve a puzzle. SATNet, which includes a differentiable relaxation of a SAT solver, clearly performs the best of the learning-based approaches, and beats SMTLayer 's performance of Vis. Sudoku because there is no possibility of grounding error. While it still must learn the rules of Sudoku, it has a strong prior that enables it to do so very well; on the other hand, Semantic Strengthening leverages Sudoku constraints during training, but does not learn a complete neural representation of them to use during inference. The OpenAI models can recite the rules of Sudoku when prompted, but are rarely successful in applying them to a specific puzzle (see Appendix A.6 for details). The GPT2-XL models tuned on the 9K benchmark instances do better, but it is only when they are tuned on significantly more data (1M) that they begin to approach acceptable levels of performance. It is important to note that this is only true when given 100 attempts, making them more costly than approaches that leverage solvers.

Training sample size. Because SMTLayer encodes explicit knowledge that is essential to correct inference on these datasets, our approach is able to perform well in data-impoverished settings where the training sample is insufficient to fully specify the symbolic component of the learning task. This is readily apparent across the results in Table 1: in the MNIST addition and first visual algebra configuration, SMTLayer yields a model that performs nearly perfectly despite not being given a sufficient sample in most cases. Because SATNet must learn the symbolic component, it is at a disadvantage, and in these settings performs similarly to a conventional model. In theory, Scallop should be able to perform as well as SMTLayer , as it also encodes explicit knowledge. However, it is unable to learn a useful model for either visual algebra configuration, and does not learn the correct representation for MNIST addition until it sees half of the possible digit pairs during training.

SMTLayer does particularly well on the visual Sudoku dataset introduced by [35]. When trained on just 10% of the original sample, it learns a function that exceeds the performance of the SATNet model by a healthy margin, which continues to grow as it is exposed to more training data. On the other hand, we found that SATNet failed to converge with less than the full original training sample.

Robustness & interpretability. The reason that SMTLayer is able to perform well, and often near the optimum, in configurations that other approaches perform poorly on, is that it learns a representation that is consistent with the symbolic knowledge encoded in the SMTLayer . For example, the constraints that we use for MNIST addition, visual algebra, and visual sudoku all encode

digits as bitvectors. In order to make a correct inference, the neural network must learn to encode MNIST digits in their correct bitvector representation. If learning succeeds at this, then there are two positive outcomes that follow. First, the model's representation will be inherently interpretable , because it will coincide with the provided symbolic domain knowledge. Second, the resulting model is naturally robust to covariate shift that does not affect the distribution of perceptual data that the network grounds, but that does affect the statistics of their composition.

This type of shift is on display in the MNIST 10% and visual algebra experiments, where at training time, the model only sees pairs of same-numbered digits, and at test time it is exposed to a substantially different distribution of digit pairs or formulas. We verified this by examining the representations learned by SMTLayer and Scallop on MNIST Addition 10%; it is unreasonable to expect SATNet to learn an interpretable representation. As expected, SMTLayer produces the correct representation in proportion to the accuracy of a typical MNIST model ( ≈ 99% ), whereas Scallop's representation was correct roughly 50% of the time. However, architecture plays a role in this robustness, as shown in the SMTLayer results for the second visual algebra configuration. Because the network is shown the full instance, and not the individual digits, it learns the training bias. Despite having access to the symbolic formulas in SMTLayer , it cannot disentangle the perceptual symbols from their covariance.

6 Limitations

A major constraint imposed by our approach arises from the SMTLayer 's dependence on Boolean vectors as a means to interact with the theoretical framework. This compels the conversion of continuous values into discrete forms, a process that risks compromising inference accuracy and adding unnecessary complexity to the theoretical underpinnings. The integration of SMTLayer also limits the amount of parallelism that can be used during training and inference, as the SMT solver must be run on CPU cores. While it may be possible with significant engineering effort to move some of this functionality to GPU cores, there will likely always be a need to run portions of SMTLayer on general-purpose CPU cores, raising the cost and decreasing efficiency in batched settings.

7 Conclusion

Our approach for integrating logical theories into deep learning, SMTLayer , provides a pragmatic solution to the problem of incorporating symbolic knowledge into learning for training and inference, which we demonstrate on several problems involving both perceptual tasks-vision and natural language-and logical reasoning. Notably, we show that models which incorporate symbolic knowledge during training and inference can outperform conventional models as well as prior work in this area, especially in settings where training data is limited. Continued progress on automated reasoning techniques has played a pivotal role in the development of several fields over the past decades, and our hope is that the contributions in this paper will aid in progress towards realizing their potential in challenges that surpass the capabilities of existing learning techniques.

Acknowledgements. This work was supported by the U.S. Army Research Office under MURI Grant W911NF-21-1-0317, and the National Science Foundation under Grant No. CNS-1943016.

References

Kareem Ahmed, Kai-Wei Chang, and Guy Van den Broeck. Semantic strengthening of neurosymbolic learning. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , apr 2023.

Oscar Chang, Lampros Flokas, Hod Lipson, and Michael Spranger. Assessing satnet's ability to solve the symbol grounding problem. Advances in Neural Information Processing Systems , 33:1428-1439, 2020.

Mark Chavira and Adnan Darwiche. On probabilistic inference by weighted model counting. Artificial Intelligence , 172(6-7):772-799, 2008.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021.

Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. EMNIST: an extension of MNIST to handwritten letters. CoRR , abs/1702.05373, 2017.

Ernest Davis. Mathematics, word problems, common sense, and artificial intelligence. arXiv preprint arXiv:2301.09723 , 2023.

Leonardo De Moura and Nikolaj Bjørner. Z3: An efficient smt solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems , pages 337-340. Springer, 2008.

Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine , 29(6):141-142, 2012.

Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin Vechev. DL2: Training and querying neural networks with logic. In Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 1931-1941. PMLR, 2019.

Kshitij Goyal, Sebastijan Dumancic, and Hendrik Blockeel. Sade: Learning models that provably satisfy domain constraints. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part V , pages 410-425. Springer, 2023.

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput. , 9(8):1735-1780, nov 1997.

Nick Hoernle, Rafael Michael Karampatsis, Vaishak Belle, and Kobi Gal. Multiplexnet: Towards fully satisfied logical constraints in neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 36, pages 5700-5709, 2022.

Jiani Huang, Ziyang Li, Binghong Chen, Karan Samel, Mayur Naik, Le Song, and Xujie Si. Scallop: From probabilistic deductive databases to scalable differentiable reasoning. Advances in Neural Information Processing Systems , 34:25134-25145, 2021.

Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning , pages 5338-5348. PMLR, 2020.

Zenan Li, Zehua Liu, Yuan Yao, Jingwei Xu, Taolue Chen, Xiaoxing Ma, L Jian, et al. Learning with logical constraints but without shortcut satisfaction. In The Eleventh International Conference on Learning Representations , 2022.

Zenan Li, Yuan Yao, Taolue Chen, Jingwei Xu, Chun Cao, Xiaoxing Ma, L Jian, et al. Softened symbol grounding for neuro-symbolic systems. In The Eleventh International Conference on Learning Representations , 2022.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations , 2019.

Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic programming. Advances in Neural Information Processing Systems , 31, 2018.

Manifold Markets. Will a prompt that enables GPT-4 to solve easy Sudoku puzzles be found? Available at: https://manifold.markets/Mira/ will-a-prompt-that-enables-gpt4-to . Retrieved 5/17/23.

Emanuele Marconato, Stefano Teso, and Andrea Passerini. Neuro-symbolic reasoning shortcuts: Mitigation strategies and their limitations. arXiv preprint arXiv:2303.12578 , 2023.

Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations , 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32 . 2019.

Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages 1532-1543, 2014.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019.

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery amp; Data Mining (KDD) , 2020.

Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms . Cambridge University Press, 2014.

Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems , 33:3008-3021, 2020.

Stefano Teso, Roberto Sebastiani, and Andrea Passerini. Structured learning modulo theories. Artificial Intelligence , 244:166-187, 2017.

Martin Thoma. The hasyv2 dataset. CoRR , abs/1701.08380, 2017.

Sever Topan, David Rolnick, and Xujie Si. Techniques for symbol grounding with satnet. Advances in Neural Information Processing Systems , 34:20733-20744, 2021.

Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Large language models still can't plan (a benchmark for llms on planning and reasoning about change). arXiv preprint arXiv:2206.10498 , 2022.

Emile van Krieken, Erman Acar, and Frank van Harmelen. Analyzing differentiable fuzzy logic operators. Artificial Intelligence , 302:1-46, January 2022.

Peter Varnai and Dimos V Dimarogonas. On robustness metrics for learning stl tasks. In 2020 American Control Conference (ACC) , pages 5394-5399. IEEE, 2020.

Marin Vlastelica, Anselm Paulus, Vít Musil, Georg Martius, and Michal Rolínek. Differentiation of blackbox combinatorial solvers. arXiv preprint arXiv:1912.02175 , 2019.

Po-Wei Wang, Priya Donti, Bryan Wilder, and Zico Kolter. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In International Conference on Machine Learning , pages 6545-6554. PMLR, 2019.

Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. A semantic loss function for deep learning with symbolic knowledge. In Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 5502-5511. PMLR, 10-15 Jul 2018.

Zhun Yang, Joohyung Lee, and Chiyoun Park. Injecting logical constraints into neural networks via straight-through estimators. In International Conference on Machine Learning , pages 25096-25122. PMLR, 2022.

Dennis Yurichev. SAT/SMT by example, 2020. Available at https://sat-smt.codes/ (January, 2023).

Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, and Guy Van den Broeck. On the paradox of learning to reason from data. arXiv preprint arXiv:2205.11502 , 2022.

Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 11328-11339. PMLR, 13-18 Jul 2020.

Peer reviews:
comments: SUMMARY:
This paper proposes SMTLayer, a layer that incorporates SMT solvers (Z3 in this case) into neural networks. The layer itself is not differentiable. The forward and backward passes of SMTLayer are derived thoroughly. Experiments show that this innovation results in overall more robust, interpretable, and efficient architectures on some tasks where symbolic reasoning is heavily relied on.

STRENGTHS:
- The idea itself is straightforward and the derivations seem well-presented.
- The experiments do confirm the claims, and outperform baseline methods by large margins.

### 12 Aug update:
Rebuttal updates and modifications are satisfactory. Bumping up the rating.

WEAKNESSES:
Only some minor weaknesses regarding writing:

- Do not say "[30] showed that ...". If a citation plays a grammatical role, use the authors' names instead.
- ~~Some claims can be made more concrete~~:
  - ~~Line 13: "that are robust to certain types of covariate shift" what types of covariate shift exactly?~~
  - ~~Line 65: "four diverse applications," what applications, how diverse exactly?~~

QUESTIONS:
NA

LIMITATIONS:
The paper mentioned a few things as future work, but should do well to comment more on the limitations of the proposed approach.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies neuro-symbolic learning tasks on weakly supervised setting (i.e., lacking direct label supervision of neural networks). To incorporate symbolic knowledge into training, this work integrates SMT solvers into the forward and backward passes of a deep network layer. The key idea is to establish the surrogate gradient of SMT solver-based reasoning, allowing for the back-propagation of this SMT layer. Experimental evaluations on four tasks demonstrate the improvement over several existing approaches.\n\nSTRENGTHS:\n- The paper presents a well-motivated and easy-to-implement method for incorporating symbolic knowledge into network training.\n- The formalization derives some good theoretical results, and has proofs presented in the appendix.\n\nWEAKNESSES:\n- Theorem 2 requires both the hypothesis set and the loss function to be convex, which is impractical. Moreover, it would be better to decompose this theorem into two parts.\n  - The first part is to discuss the convergence of the SGD algorithm. This only requires Lipschitz and smoothness assumptions.\n  - The second part is to analyze the properties of the grounding hypothesis under convex assumptions.\n- The exactly-one assumption made by Theorem 2 is quite vacuous. From my understanding, with this assumption, one can derive the label supervision by using SMT solver (or correct me if not). Moreover, this assumption avoids the discussion of shortcuts problem (i.e., how to distinguish different satisfying assignments) in such neuro-symbolic learning paradigm [1, 2]. \n- Some related work is missing. For instance, [3] uses SMT solvers and incorporates MCMC sampling to support network training. Additionally, it is suggested to compare some differentiable logic methods [4, 5]. Particularly, [4] also ensures the satisfaction of symbolic constraints in inference stage. \n\n[1] Zenan Li, Zehua Liu, Yuan Yao, Jingwei Xu, Taolue Chen, Xiaoxing Ma, Jian Lu. Learning with Logical Constraints but without Shortcut Satisfaction.\n\n[2] Marconato, Emanuele, Stefano Teso, and Andrea Passerini. Neuro-Symbolic Reasoning Shortcuts: Mitigation Strategies and their Limitations.\n\n[3] Zenan Li, Yuan Yao, Taolue Chen, Jingwei Xu, Chun Cao, Xiaoxing Ma, Jian Lu. Softened Symbol Grounding for Neuro-symbolic Systems. \n\n[4] Nicholas Hoernle, Rafael Michael Karampatsis, Vaishak Belle, Kobi Gal. MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks. \n\n[5] Zhun Yang, Joohyung Lee, Chiyoun Park. Injecting Logical Constraints into Neural Networks via Straight-Through Estimators.\n\nQUESTIONS:\n- Some citations are duplicate, e.g., [31] and [32], [24] and [25]. \n- Algorithm 2 is unclear to me. For example, in Line 2, how to compute the BCE loss for $\\text{sign}(z)$ in $\\\\{-1,0,1\\\\}$. Moreover, the gradient of $\\text{sign}(z)$ still vanishes almost everywhere? \n- The authors claim that $\\ell(\\hat{y}, y^*) \\leq \\ell(y, y^*)$ (Line 211), how to derive this property?\n- How does the approach avoid the shortcut problem? For example, in the MNISTAdd task, if the image ``4+7=11`` is incorrectly recognized as ``3+8=11``, the surrogate gradient may reinforce this incorrect prediction instead of correcting it.\n\nLIMITATIONS:\nYes, the limitations of the approach have been partially presented. The work does not have negative social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes to incorporate SMT constraints in the neural network to encode domain knowledge. Specifically, it proposes an unsat core based approach, and an MaxSAT based approach for the differentiable training in the presence of SMT constraints. Empirical evaluations on several benchmark problems are presented.\n\nSTRENGTHS:\nThe proposed methods in this work come with theoretical guarantees on convergence as well as significantly better performance than several baseline approaches.\n\nWEAKNESSES:\n- The description of the proposed method especially Sec 4 is mostly in English with math hidden behind, which can be confusing for readers who want to know about the technical details. I put some of the technical questions I have in the question part.\n- The proposed method seems limited to classification task, without discussions on its generalization.\n- For the experiments, only the performance of the unsat core based algorithms is presented and there is not empirical evaluations of the MaxSMT based algorithm presented. The evaluation would be more comprehensive if both qualitative and quantitative comparisons of the two proposed approaches are discussed.\n\nMinor:\n- symbol at Line 104 is not defined.\n- double quote at Line 251 is not properly typed.\n\nQUESTIONS:\n- At Line 38, it says that the learning of the compatible representation can be done \"without requiring label supervision\". Still, this work seems to focus on supervised learning. Can the authors elaborate more on what they mean by \"without requiring label supervision\" to clear the confusion?\n- Throughout this work, the loss is always assumed to be binary cross-entropy. I wonder how much this method generalizes to other loss.\n\nMore technical questions:\n- How are the constraints in the MNIST addition problem defined? In Fig 1, it is unclear why the label is represented by five digits.\n- From Sec 4.1, there is a conversion from Boolean vectors to continuous. I wonder how is the number of bits in the vector decided and if it would affect predictive performance.\n- How is the amended output \\hat{y} computed and is it Boolean or continuous, and how is the inequality at Line 211 is guaranteed?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors implement SMTLayer, which integrates an SMT solver into a differentiable module, suitable for use with deep learning.  SMTLayer takes a vector of floating-point values as input, and produces a vector of floating-point values as output.  These input vector is  cast to boolean values, and the output of SMTLayer is a set of boolean assignments to variables such that some formula $\\phi$ is satisfiable.  The formula is not learned: it represents a constraint that is specified as part of the training task. \n\nIn order to calculate a gradient, the problem is run in reverse -- given a gradient on the outputs, SMTLayer will again solve for satisfiability, attempting to find an alternate set of inputs and outputs such that the outputs have lower loss, and $\\phi$ is satisfiable.  \n\nThe authors provide two different SMT solvers for the forward pass, and two different mechanisms for computing gradients on the backwards pass.  They then demonstrate that SMTLayer can be used to solve challenging problems (e.g. MNIST sodoku puzzles) with far greater accuracy than existing SOTA.\n\nSTRENGTHS:\nThis paper is very well written, and it addresses an important problem -- namely how to incorporate symbolic and logical computations into deep neural nets in a way that is principled, interpretable, and differentiable.  The experiments are well-designed, the results are compelling, and the authors plan to make the code available as open source.  I believe this to be an important paper with potentially high impact.\n\nWEAKNESSES:\nThere were a few areas that I did not quite understand -- see questions, below.\n\nQUESTIONS:\nOn the backwards pass, SMTLayer must take the gradient of the output $y$, and use it to construct an alternative output $\\hat{y}$ with lower loss.  Could you elaborate a bit on how this is done?  What happens if SMTLayer is embedded deep within another neural network, so that you have a gradient on $y$, but that gradient does not come from a simple cross-entropy loss function?  (E.g., there are a bunch of additional NN layers between $y$ and the loss function.)  Do you sample several alternative options for $\\hat{y}$, in order to find one that works?  Note that I am not intimately familiar with the details of SMT solvers.\n\nWhat is typical cost of running an SMT solver, compared to the rest of the neural network?  Does the solver dominate training time? \n\nCan you think of additional areas where an SMT solver could potentially be applied?\n\nLIMITATIONS:\nThe authors do not discuss negative societal impacts, but IMO, any impacts are likely to be positive -- incorporating interpretable logical constraints into models seems like an improvement over SOTA wrt. to most issues of AI alignment.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}