SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhaoyu Li $^{1,2}$ , Jinpei Guo $^{4}$ , Yuhe Jiang $^{1}$ , Xujie Si $^{1,2,3}$
<sup>1</sup>University of Toronto, <sup>2</sup>Vector Institute, <sup>3</sup>Mila, <sup>4</sup>Shanghai Jiao Tong Univeristy {zhaoyu, six}@cs.toronto.edu
Bridging logical reasoning and deep learning is crucial for advanced AI systems. In this work, we present a new framework that addresses this goal by generating interpretable and verifiable logical rules through differentiable learning, without relying on pre-specified logical structures. Our approach builds upon SATNet, a differentiable MaxSAT solver that learns the underlying rules from input-output examples. Despite its efficacy, the learned weights in SATNet are not straightforwardly interpretable, failing to produce human-readable rules. To address this, we propose a novel specification method called "maximum equality", which enables the interchangeability between the learned weights of SATNet and a set of propositional logical rules in weighted MaxSAT form. With the decoded weighted MaxSAT formula, we further introduce several effective verification techniques to validate it against the ground truth rules. Experiments on stream transformations and Sudoku problems show that our decoded rules are highly reliable: using exact solvers on them could achieve  $100\%$  accuracy, whereas the original SATNet fails to give correct solutions in many cases. Furthermore, we formally verify that our decoded logical rules are functionally equivalent to the ground truth ones.
Logical reasoning is a fundamental cognitive skill that empowers humans to organize knowledge in a structured form, reason about relationships between different pieces of information, and draw inferences based on logical rules and constraints. In recent years, there has been a growing research interest in integrating advanced deep learning systems with the ability of logical reasoning [9, 10, 23, 32]. A notable outcome in this field is SATNet [32], a differentiable MaxSAT solver based on a low-rank semidefinite programming (SDP) approach. SATNet utilizes a learnable matrix  $S$  to capture the underlying structure of logical rules and dynamically adjusts its weight from input-output examples. This innovative design enables SATNet to learn sophisticated logical rules, such as the parity function and Sudoku puzzles.
Despite the promising results of SATNet, the interpretability of its learned weights remains unclear. This limitation arises from the fact that the clause matrix  $S$  learned by SATNet is not ternary<sup>1</sup>, preventing its explicit interpretation as a set of logical rules as originally motivated [33]. Consequently, this poses a challenge to the wider adoption of SATNet in real-world applications that require transparent decision-making processes.
To address this limitation, we present a new framework that extracts a set of interpretable logical rules from the learned weights in SATNet. We first demonstrate that merely rounding  $S$  to a ternary matrix is insufficient for interpretation and then show empirical and theoretical evidence that  $S$  may fail to capture the underlying logical rules in certain cases. Instead of directly decoding  $S$ , we propose utilizing the matrix  $C = S^T S$  to derive explicit logical rules, based on the SDP formulation in
<sup>1</sup>In SATNet's original formulation, it assumes the  $S$  matrix can only consist of 1, -1, and 0 elements to represent the logical formula in MaxSAT form. However, after training,  $S$  is continuous rather than ternary.
Figure 1: An example pipeline of our framework for  $4 \times 4$  Sudoku problem. We first train SATNet to learn the Sudoku rules from data. After training, we use maximum equality specification to decode a set of interpretable rules as a weighted MaxSAT formula from the learned weight  $C$ . Given the decoded rules, we call an exact solver for inference and further verify them with ground truth rules. Optional domain knowledge can be incorporated through numerical weights or propositional rules.
SATNet. Specifically, we introduce a novel specification formulation called "maximum equality", which transforms the learned matrix  $C$  into a weighted MaxSAT formula. We rigorously prove that our proposed formulation has a powerful expressive ability, enabling  $C$  to represent any propositional logical rules under proper parameterization.
With the decoded logical rules, we can employ an exact solver (e.g., Gurobi [13]) to replace SATNet during inference, rendering the solving process more accurate and transparent. Additionally, our specification method offers flexibility in incorporating prior domain knowledge in propositional logic. This integration can be done through either the learned weights  $C$  or the weighted MaxSAT formula, bolstering the reliability of our learned or decoded rules. To further validate the correctness of the decoded logical rules, we propose several verification techniques that prove the equivalence between our decoded weighted MaxSAT formula and the human-written rules in SAT form. In particular, we establish that if the optimal solution of the weighted MaxSAT formula always corresponds to a satisfying assignment of the SAT formula for any valid partial inputs, then these two formulas function equivalently. Notably, when the space of inputs becomes excessively huge, we introduce a sufficient condition for an efficient equivalence check, which assures not only the functional equivalence but also a similar distribution of solution space. We summarize our proposed framework in Figure 1.
To evaluate the efficacy of our proposed framework, we conduct a series of experiments on various tasks, including stream transformations (i.e., the parity function, addition, and counting) and Sudoku puzzles. For stream transformations, we strictly verify that our decoded weighted MaxSAT formula is functionally equivalent to the human-written logical rules, which ensures that it accurately captures the underlying logic of these tasks. On Sudoku puzzles, we show that using exact solvers on our generated formula could achieve  $100\%$  accuracy on the testing set and further verify that it functions equivalently to the ground truth for all unique puzzle inputs. Conversely, SATNet fails to achieve perfect accuracy on Sudoku puzzles, where we certify this is because SATNet gives a sub-optimal solution to our decoded weighted MaxSAT formula.
In summary, our contributions include:
- We pinpoint the interpretability challenges inherent in the original SATNet and present both empirical and theoretical insights into SATNet's inability to capture logical rules in some cases.
- We introduce a novel interpretation method that decodes a set of logical rules as a weighted MaxSAT formula from SATNet and enables the flexible integration of domain knowledge.
- We propose multiple verification approaches to validate the correctness of our decoded rules with human-written ground truth rules.
- Experiments on stream transformations and Sudoku puzzles show that our decoded weighted MaxSAT formula exactly captures the underlying logical rules and is verifiably reliable.
In this paper, we focus on problems whose inherent logical rules can be expressed using propositional logic. For a given logical problem, we define a set of  $n_d$  problem Boolean variables  $z_1, \ldots, z_{n_d}$  to represent the potential relationships within the problem and express the ground truth logical rules as a SAT formula  $\phi(z_1, \ldots, z_{n_d})$ . The dataset  $\mathcal{D}$  consists of input-output examples  $(\mathbf{x_i}, \mathbf{y_i})$ , where each example partitions  $n_d$  variables to an input set  $Z_{\mathcal{I}}$  and an output set  $Z_{\mathcal{O}}$  and assigns a value to each variable accordingly. The task is to learn the underlying logical rules from the datasets  $\mathcal{D}$  such that the learned rules  $\phi'$  can accurately map the input variables  $z_i \in Z_{\mathcal{I}}$  to the output variables  $z_o \in Z_{\mathcal{O}}$ , similar to how the ground truth rules  $\phi$  conduct the mapping.
Let's take  $4 \times 4$  Suku as an example. In this case, we have a total of 64 variables, where each successive group of 4 variables represents a single grid in the puzzle that can hold values ranging from 1 to 4. Each input  $\mathbf{x_i}$  in the dataset  $\mathcal{D}$  corresponds to the initially provided values in a Suku puzzle, and the output  $\mathbf{y_i}$  represents the solved values of that puzzle. We transform each input-output example to construct an input set  $z_{i} \in Z_{\mathcal{I}}$  and an output set  $z_o \in Z_{\mathcal{O}}$ . For instance, if variables  $z_{1}, z_{2}, z_{3}, z_{4}$  correspond to the values of the first grid in the Suku board, and the given value is 4, then variables  $z_{1}, \ldots, z_{4} \in Z_{\mathcal{I}}$  with the configuration  $z_{1} = 0, z_{2} = 0, z_{3} = 0, z_{4} = 1$ . The task is to learn the inherent Suku rules from the dataset  $\mathcal{D}$  such that given the initial known values of a Suku puzzle, the learned logical rules can accurately determine the values of the remaining cells.
2.2 SATNet
SATNet [32] is a differentiable MaxSAT solver that models the underlying rules  $\phi$  in the data as a MaxSAT problem and solves it using SDP relaxation. It assumes that the logical rules are written in conjunctive normal form with  $n_d$  defined variables  $\tilde{Z} \in \{-1, 1\}^{n_d}$  and  $m$  clauses, which can be represented by a clause matrix  $\tilde{S} \in \{1, -1, 0\}^{m \times n_d}$ . Specifically, each element  $\tilde{s}_{ij}$  in  $\tilde{S}$  denotes the sign of variable  $\tilde{z}_i$  in clause  $j$ . The MaxSAT problem can be formulated as follows:
$$
\underset {\tilde {Z} \in \{- 1, 1 \} ^ {n _ {d}}} {\text {m a x i m i z e}} \sum_ {j = 1} ^ {m} \bigvee_ {i = 1} ^ {n _ {d}} \mathbf {1} \left\{\tilde {s} _ {i j} \tilde {z} _ {i} > 0 \right\}. \tag {1}
$$
SATNet applies semidefinite relaxation to Equation 1 by relaxing each variable  $\tilde{z}_i$  to a unit vector  $v_i\in \mathbb{R}^k$  with respect to a unit "truth direction"  $v_{\top}$  based on the randomized rounding [12]. It additionally introduces a coefficient vector  $s_\top = \{-1\} ^m$  associated with the truth vector and writes the SDP relaxation of the MaxSAT problem as:
$$
\underset {V \in \mathbb {R} ^ {k \times \left(n _ {d} + 1\right)}} {\text {m i n i m i z e}} \langle S ^ {T} S, V ^ {T} V \rangle , \quad \text {s . t .} \| v _ {i} \| = 1, i = \{T, 1, \dots , n _ {d} \}, \tag {2}
$$
where  $V = [v_{\top}, v_1, \ldots, v_{n_d}] \in \mathbb{R}^{k \times (n_d + 1)}$  and  $S = [s_{\top}, \tilde{s}_1, \ldots, \tilde{s}_n] \in \mathbb{R}^{m \times (n + 1)^2}$ .
Given the input variables  $z_{i} \in Z_{\mathcal{I}}$ , SATNet relaxes their values to unit vectors and solves Equation 2 by the mixing method [31], a fast coordinate descent approach. In this method, each output vector  $v_{o}$  is updated iteratively as follows:
$$
v _ {o} = - g _ {o} / \left\| g _ {o} \right\|, \text {w h e r e} g _ {o} = V S ^ {T} s _ {o} - \left\| s _ {o} \right\| ^ {2} v _ {o}, \text {f o r} o \in \mathcal {O}. \tag {3}
$$
The updates in Equation 3 are proven to converge to the globally optimal fixed point of the SDP [31]. After obtaining the relaxed outputs  $v_{o}$  from coordinate descent, SATNet converts each output variable vector to a probabilistic value by  $p(z_{o} = 1) = \cos^{-1}\left(-v_{i}^{T}v_{\top}\right) / \pi$  and computes the binary cross-entropy loss between the ground-truth variable assignments  $z_{o}$ . When back-propagating the loss, SATNet employs a custom algorithm that uses an efficient coordinate descent algorithm to directly compute the gradient of  $S$ , without explicitly unrolling the forward-pass computations. Such features enable SATNet to learn the  $S$  matrix efficiently from input-output examples.
It is worth noting that SATNet requires to specify the number of clauses  $m$  before learning a clause matrix  $S$ . Moreover, to improve the representational power of  $S$ , SATNet further incorporates  $n_a$  auxiliary variables alongside the  $n_d$  problem-defined variables. These auxiliary variables are considered output variables but are not involved in the loss computation.
2In the original paper of SATNet [32], each row of  $S$  is also normalized using the constant  $\sqrt{4\sum_{j}|\bar{s}_{ij}|}$ . However, in cases where  $S$  is learned (rather than provided), this normalization can be omitted for simplicity.
In this section, we first explore the issue of uninterpretability in SATNet by analyzing the clause matrix  $S$ . We then revisit the formulation of SATNet and present our specification approach that decodes a set of interpretable logical rules from the learned weights  $C = S^T S$ . After this, we show that our method also allows for the flexible incorporation of domain knowledge via additional rules in propositional logic or the additive weights of  $C$ . To validate the correctness of our decoded rules, we introduce several verification strategies to compare them with the human-written ones.
3.1 Uninterpretability of SATNet
SATNet utilizes the clause matrix  $S$  to capture the underlying structure of logical rules and learns its weight through input-output examples. However, SATNet doesn't enforce  $S$  to be ternary during training, making it hard to directly obtain a set of interpretable logical rules from  $S$ . Towards deriving human-readable rules from SATNet, we first conduct a range of experiments on the  $S$  matrix<sup>3</sup>.
Rounding  $S$  to a ternary matrix. One straightforward idea to derive a ternary matrix from  $S$  is to round its weights to the values  $\{1, -1, 0\}$ . To test this approach, we first train SATNet on the  $4 \times 4$  Sudoku dataset, which achieves  $100\%$  training accuracy and  $99.89\%$  testing accuracy. Subsequently, we apply a threshold function  $f_{\mathrm{thre}}$  to round each entry of  $S$  with a threshold value  $\epsilon > 0$ :
$$
f _ {\mathrm {t h r e}} (s _ {i j}) = \left\{ \begin{array}{l l} s _ {i j} / | s _ {i j} |, & \text {i f} | s _ {i j} | > \epsilon , \\ 0, & \text {o t h e r w i s e}. \end{array} \right.
$$
In our experiments, we exhaustively evaluate all possible threshold values and round  $S$  to a set of ternary matrices. We assess the correctness of these matrices by solving Sudoku test instances using an exact MaxSAT solver applied to the corresponding MaxSAT formulas. Our empirical result shows that none of these ternary matrices could yield the correct logical rules, as they fail to solve any Sudoku puzzles in the testing set. This finding demonstrates that simply rounding  $S$  is insufficient for interpreting the learned weights in SATNet.
Substituting ground truth rules into  $S$ . To investigate the representational capability of the clause matrix  $S$ , we conduct an experiment by injecting human-written ground truth rules of  $4 \times 4$  Sudoku puzzles into  $S$ . If  $S$  is truly capable of capturing such logical rules, we would expect SATNet to solve Sudoku test instances quite well when provided with the ground truth rules. Contrarily, SATNet achieves only  $53.95\%$  solving accuracy on the testing dataset, giving incorrect solutions for a substantial number of Sudoku puzzles. On the other hand, after training from examples, SATNet could reach near-perfect solving accuracy without the use of ground truth rules. This discrepancy highlights the difficulty in representing and interpreting the underlying logical rules using  $S$  alone.
Limitations of  $S$  in representing logical rules. We further supplement our empirical observations with theoretical evidence demonstrating the limitations of the clause matrix  $S$  in capturing certain logical rules. Let's consider the parity function as an example, which has specific ground truth rules and the corresponding (unnormized)  $S$  matrix defined as follows:
$$
\text {R u l e s i n m a t r i x f o r m :} \left( \begin{array}{c c c} 1 & 1 & - 1 \\ 1 & - 1 & 1 \\ - 1 & 1 & 1 \\ - 1 & - 1 & - 1 \end{array} \right), \quad S = \left( \begin{array}{c c c c} - 1 & 1 & 1 & - 1 \\ - 1 & 1 & - 1 & 1 \\ - 1 & - 1 & 1 & 1 \\ - 1 & - 1 & - 1 & - 1 \end{array} \right).
$$
In this example, the first two variables in the ground truth rules denote the input bits, while the third variable indicates the parity as the output bit. SATNet, however, introduces an additional truth variable and incorporates it in each clause of  $S$  with negative polarity. As a result, the matrix  $C = S^T S$  becomes a diagonal matrix  $4 \cdot I$ , where  $I$  is the identity matrix. Given the SDP formulation of SATNet, its optimization remains unaffected by the diagonal elements of  $C$ , as the calculations associated with these elements result in constants. When all the off-diagonal entries of  $C$  are zero, the SDP becomes trivial, thereby preventing SATNet from capturing any meaningful rules from data. This limitation is not unique to the parity function but also extends to other logical rules with similar symmetrical properties. Therefore, the clause matrix  $S$  defined in SATNet confronts a challenge in interpretability, as it theoretically fails to represent a range of logical rules.
3Our detailed experimental settings are provided in Section 4.
3.2 Decoding Interpretable Logical Rules from SATNet
Given the lack of interpretability in SATNet's clause matrix  $S$ , the questions of why SATNet is capable of learning the underlying logical rules and how we can decode these rules remain unanswered. To address these questions, we start by revising the optimization formulation of SATNet to a more general form. Building upon this reformulation, we introduce our novel specification approach, "maximum equality", which not only offers a viable interpretation of SATNet but also enables us to derive a set of logical rules in the form of weighted MaxSAT using the learned weights of SATNet.
Maximum equality specification. Instead of using the clause matrix  $S$ , we reformulate Equation 2 in terms of the matrix  $C = S^T S$  (with  $n_a$  auxiliary variables) and decompose it as:
$$
\underset {V \in \mathbb {R} ^ {k \times \left(n _ {d} + n _ {a} + 1\right)}} {\text {m i n i m i z e}} \sum_ {i = \top , j = \top} ^ {n _ {d} + n _ {a}} c _ {i j} \cdot v _ {i} ^ {T} v _ {j}, \quad \text {s . t .} \| v _ {i} \| = 1 \text {f o r} i \in \{\top , 1, \dots , n _ {d} + n _ {a} \}. \tag {4}
$$
In Equation 4, the objective is to minimize the weighted sum of dot products between all pairs of variable vectors. Considering that all variable vectors are unit vectors, we can view the dot product  $v_{i}^{T}v_{j}$  as a measure of similarity between  $v_{i}$  and  $v_{j}$ . From this perspective, the objective of Equation 4 is to minimize the similarity between each pair of variable vectors, with the weight  $c_{ij}$  dictates the importance of each similarity. Based on this observation, we can further view Equation 4 as a relaxation of the following objective:
$$
\underset {Z \in \{0, 1 \} ^ {n _ {d} + n _ {a} + 1}} {\text {m a x i m i z e}} \bigwedge_ {i = \top , j = \top} ^ {n _ {d} + n _ {a}} - 2 \cdot c _ {i j} \cdot \delta \left(z _ {i}, z _ {j}\right). \tag {5}
$$
Here,  $\delta(\cdot, \cdot)$  is the Kronecker delta function such that  $\delta(z_i, z_j) = 1$  when  $z_i = z_j$  and  $0$  else. Equation 5 applies the Kronecker delta function to each pair of Boolean variable assignments  $(z_i, z_j)$  and aims to maximize the overall values according to each weight  $-2 \cdot c_{ij}$ . Note that The factor of 2 is incorporated because the product of two unit vectors  $v_i^T v_j$  ranges between  $[-1, 1]$ , while the Kronecker delta function can only take on values of  $0$  or  $1$ . We refer to Equation 5 as the "maximum equality" formulation, as it maximizes the weighted equality among pairs of variable assignments.
Expressive power of maximum equality formulation. To dive deeper into the maximum equality formulation, we first prove its representational power in expressing any propositional logical rules.
Proposition 1. Our proposed maximum equality formulation can represent arbitrary logical rules within propositional logic, under proper parameterization.
Proof. It is well-established that Max2SAT is NP-Complete [11]. To demonstrate that any propositional logical formula can be reduced to a maximum equality formula, we only need to show the reduction from Max2SAT to maximum equality. Given a clause  $(a \lor b)$  in a Max2SAT formula, we can transform it into three maximum equality clauses:
$$
- w (a = b) \wedge w (a = v _ {\top}) \wedge w (b = v _ {\top}),
$$
where  $v_{\top}$  is the truth variable and  $w$  can be any weight value. Similarly, the clauses  $(a \lor \neg b), (\neg a \lor b)$  and  $(\neg a \lor \neg b)$  can be mapped to three maximum equality clauses, where each satisfying assignment of the original clause has a weight of  $2 \cdot w$  more than the unsatisfying one. Compiling these translated clauses gives us a direct transition from a Max2SAT formula to its maximum equality counterpart. While Max2SAT can be transformed to maximum equality by merely introducing a truth variable, general propositional formulas might necessitate more auxiliary variables to reduce to a Max2SAT formula. Therefore, to represent any propositional logical formula using maximum equality, we may also need to incorporate enough auxiliary variables by setting an appropriate value  $n_a$ .
Transforming maximum equality to weighted MaxSAT. For a maximum equality formula, we can also transform it into a more standard form, i.e., a weighted MaxSAT formula. Technically, for every clause  $w$  ( $a = b$ ) in a maximum equality formula, we convert it into multiple weighted MaxSAT clauses by introducing a new variable  $d$  to represent the equality condition  $a = b$  as follows:
$$
h (\neg a \lor b \lor \neg d) \wedge h (a \vee \neg b \lor \neg d) \wedge w (d), \quad \text {i f} w > 0,
$$
$$
h (\neg a \vee \neg b \vee d) \wedge h (a \vee b \vee d) \wedge - w (\neg d), \quad \text {i f} w <   0,
$$
where  $h$  denotes the hard constraint, which indicates the infinite weight. Using this transformation, we can decode the learned rules in terms of  $C$  to a set of logical rules as a weighted MaxSAT formula.
Substituting ground truth rules into  $C$ . To validate the efficacy of our proposed maximum equality specification as a viable interpretation for SATNet, we directly inject human-written rules into the  $C$  matrix, bypassing the use of the clause matrix  $S$ . This substitution process involves transforming the ground truth rules into a Max2SAT formula and then converting it to the corresponding maximum equality formula. We perform experiments specifically on the parity function, as utilizing the  $S$  matrix directly in this case can lead to a trivial SDP formulation for SATNet. As expected, when given the ground truth rules to  $C$ , SATNet perfectly solves the parity function. This result serves as compelling evidence that the maximum equality formulation could effectively capture and express the logical rules inherent in the problem.
Learning a sparse  $C$  matrix in SATNet. It is also worth noting that our specification method would result in a weighted MaxSAT formula with approximately  $3 \cdot \binom{n+n_a+1}{2}$  clauses if  $C$  is a dense matrix that contains no zero elements. For SATNet,  $C = S^T S$  is always such a dense matrix. However, solving with such a large number of clauses can be computationally expensive when invoking an exact solver for the weighted MaxSAT formula. To reduce the size of our decoded logical rules, we modify the implementation of SATNet to learn the  $C$  matrix directly and further apply the Iterative Hard Thresholding (IHT) algorithm [7, 4] to sparsify it.