Paper: Convex and Bilevel Optimization for Neuro-Symbolic Inference and Learning (ICLR-2024)

Reviewer comment (Evaluation):
Neither variant of the propsed method (CC / LF) consistently beats the ADMM baseline across all datasets. However, it is significantly faster in most cases.

Author response:
Regarding the parallelization strategies for our D-BCD inference algorithm, we would like to emphasize that connected components (CC) and lock-free (LF) D-BCD algorithms should be applied in different settings. Specifically, the CC parallelization strategy should be used when there are many independent groups of variables, i.e., many connected components of the factor graph defining the inference problem. For instance, in the MNIST-Add problem, each addition example, i.e., a set of images defining an addition, creates an independent set of variables contained in a single connected component of the inference factor graph. On the other hand, LF should be applied when the factor graph is large but connected. This is the case in the citation networks like Citeseer and Cora. Furthermore, we are happy that D-BCD often outperforms ADMM in a single round of inference, as shown in Table 2, however, we feel its real strength is shown when it is applied to learning where inference is repeated, and it can leverage warm-starts, as demonstrated in Table 3.

Reviewer comment (Experiments):
Only one baseline was considered. As neural-symbolic systems aren't my field I'll defer to other reviewers on whether this is adequate.

Author response:
In response to the comment on the experiments' baselines, we emphasize that ADMM is the state-of-the-art inference algorithm for NeuPSL. Furthermore, we have included alternative inference baselines in the extended inference runtime experiments in E.4 of the appendix. Moreover, Dual-BCD is the first inference algorithm developed for the dual LCQP formulation of NeuPSL inference presented in this work, i.e., the first algorithm to produce optimal dual variables that can be used to produce both optimal primal variables and principled learning gradients. For baselines in performance experiments, we want to underscore the primary contribution of the paper is the learning algorithm rather than the NeSy framework of NeuPSL, i.e., we are comparing our method to the incumbent training algorithm that can only minimize value-based losses. We have included other NeSy frameworks in the four datasets of Tables 5 and 6 as reference points.

Reviewer comment (Reproducibility):
Where does the value function introduced in (7) come from? Is it learned from data?

Author response:
Q2) The value-function is parameterized, and the goal is to learn parameter settings from data. The energy function defining the value-function integrates neural and symbolic models. The high-level architecture (the neural layers and the symbolic structure) is designed by a human using a framework like NeuPSL.

Reviewer comment (Writing):
Notation is not always properly defined (e.g., the prox operator).

Author response:
We have revisited and improved the exposition of the NeSy learning algorithm in section 4. Specifically, we have substituted the proximal operator with its definition, clarified function dependencies, and emphasized the connection between the original optimization problem of inference and the Moreau envelope in section 5. Moreover, we have added a brief description of how the dual/penalty variables are updated, and we point the reader to the appendix for more details.

Reviewer comment (Writing):
Dependencies are omitted without explicitly mentioning them (e.g., in the definition of the value function).

Author response:
We have revisited and improved the exposition of the NeSy learning algorithm in section 4. Specifically, we have substituted the proximal operator with its definition, clarified function dependencies, and emphasized the connection between the original optimization problem of inference and the Moreau envelope in section 5. Moreover, we have added a brief description of how the dual/penalty variables are updated, and we point the reader to the appendix for more details.

Reviewer comment (Reproducibility):
Important parts are left out of the main text (e.g., how the dual variables in equation 10 are updated or how the Moreau envelope of the energy function is computed).

Author response:
We have revisited and improved the exposition of the NeSy learning algorithm in section 4. Specifically, we have substituted the proximal operator with its definition, clarified function dependencies, and emphasized the connection between the original optimization problem of inference and the Moreau envelope in section 5. Moreover, we have added a brief description of how the dual/penalty variables are updated, and we point the reader to the appendix for more details.

Reviewer comment (Reproducibility):
If I understand correctly, Algorithm 1 requires computing both the Value function and the Moreau envelope at each iteration. Both require minimization over \( y \), so in the case of NeuPSL are these both computed using the BCD algorithm in Section 5? I.e., is the \( \epsilon \) smoothening in equation 13 the same as the \( \frac{1}{\rho} \) smoothening term in equation 8? In any case, I would recommend highlighting more explicitly how exactly Section 5 links into Section 4.

Author response:
Q1) Algorithm 1 does require computing both the value-function and the Moreau envelope of the energy function. For NeuPSL, the energy function, and thus the value-function, is regularized and its minimizer is found using the dual BCD algorithm. As previously mentioned, we have added a description of the connection between the original optimization problem of NeuPSL inference and the Moreau envelope in section 5.

Reviewer comment (Reproducibility):
At the end of Section 5.1, the authors mention that mapping primal variables to dual variables requires calculating a pseudo-inverse of the matrix \( A \). Is this at any point required in the proposed learning algorithm? If yes, is it described anywhere?

Author response:
Q2) The mapping of primal to dual variables is not necessary as long as one solves the dual LCQP formulation of inference to obtain optimal dual variables needed for learning gradients. For this reason, the pseudo inverse of the constraint matrix does not need to be computed when the dual BCD algorithm is used. We have added a discussion motivating and clarifying this point in Section 5.3.

Reviewer comment (Novelty):
Are the authors aware of previous work using Moreau envelopes in the context of neuro-symbolic or structure learning with non-differentiable settings? Such links would be a useful addition to the related work. Otherwise, if this has not been used before, the novelty of this contribution could also be highlighted more.

Author response:
Q3) We are unaware of previous work using Moreau envelopes to smooth energy functions to support gradient-based parameter learning. We have emphasized this novelty in the introduction.

Reviewer comment (Writing):
One of the mentioned key contributions is that the "dual BCD algorithm for NeuPSL inference [...] naturally produces statistics necessary for learning gradients". I don't understand this statement. How are the statistics in the BCD algorithm necessary for "learning" gradients? Do you mean they help in computing the gradients required for the NeSy learning algorithm?

Author response:
Q4) What we meant by the statement you referred to is that the dual BCD algorithm developed for the regularized LCQP formulation produces the optimal dual variables that are necessary for computing principled sub-gradients that can be used for learning. Previous inference algorithms developed for NeuPSL have only operated on the non-smooth primal formulation in equation 17 of the appendix. We edited this statement to be more direct and clear and added the original primal inference formulation back to the main body of the paper to clarify the contribution.

Reviewer comment (Writing):
The differences between implicit-differentiation-based methods and value-function approaches are not sufficiently explained, and are not clear to me even though I know exactly how to differentiate through an optimization problem using envelope theorems and similar. This is much more background than many readers will have, so if I'm confused, chances are most people will be.

Author response:
We have added to the structure and discussion in our related work (Section 2). Specifically, we have expanded on the definition and comparison between implicit differentiation and value-function approaches to bilevel optimization.  
Q1) An implicit differentiation approach derives an analytic expression for the gradient of the upper-level objective using implicit differentiation. The resulting gradient requires second-order derivatives of the lower-level objective function and constraints (if they are supported) at the lower-level minimizer. On the other hand, value-function approaches reformulate the bilevel problem as a single-level constrained program to derive gradient-based methods that only require a gradient of the optimal value of the lower-level objective function.

Reviewer comment (Writing):
The main technical sections are notation-heavy, and are quite complex to read. Theorem 5.3 in particular is somewhat hard-to-parse and takes a third of a page, in total, to state, even though its first main claim is a simply saying that a certain objective is convex-concave, and the other claims are also relatively simple.

Author response:
We have consolidated and simplified the statements of theorems 5.1 and 5.2.

Reviewer comment (Evaluation):
The evaluation is purely quantitative: the authors show better numbers on tables compared to alternatives. While this is a valid way to evaluate, it also gives a much weaker idea of what is going on compared to evaluations that are not table-based and consider factors other than performance metrics. Nothing I saw in the experimental section rules out a situation of the form "none of the algorithms work, but the authors' is slightly less broken" - I would like to see some evidence that this isn't the case.

Author response:
The prediction performance results presented in our empirical evaluation, Section 6, improve upon the performance of NeuPSL models presented in published research. Please see Appendix E.1 for details on the NeuPSL models. Moreover, we have included prominent alternative NeSy baselines for the Citeseer, Cora, and MNIST experiments in Tables 5 and 6. This gives a relative point for comparison of the performance of NeuPSL.  
Q3) We would be happy to add additional metrics to our evaluation that you feel would better represent the performance than the ones mentioned in Table 1. However, we are unsure what types of evaluation you are referring to. We included additional NeSy methods as reference points for the four datasets in Tables 5 and 6, and we want to reiterate that the results in Table 4 improve upon the performance of NeuPSL models presented in published research.

Reviewer comment (Evaluation):
I've previously seen misleading results like this in reinforcement learning papers where an agent achieved "good performance" through random actions that were slightly-more-aligned with the objective compared to baselines, but was so far from correct behavior that most people would reasonably view both the method and the baselines as equally bad, and the differences in metrics as meaningless. In total, how do we know the neuro-symbolic system is performing as it should in this setting?

Author response:
The prediction performance results presented in our empirical evaluation, Section 6, improve upon the performance of NeuPSL models presented in published research. Please see Appendix E.1 for details on the NeuPSL models. Moreover, we have included prominent alternative NeSy baselines for the Citeseer, Cora, and MNIST experiments in Tables 5 and 6. This gives a relative point for comparison of the performance of NeuPSL.  
Q3) We would be happy to add additional metrics to our evaluation that you feel would better represent the performance than the ones mentioned in Table 1. However, we are unsure what types of evaluation you are referring to. We included additional NeSy methods as reference points for the four datasets in Tables 5 and 6, and we want to reiterate that the results in Table 4 improve upon the performance of NeuPSL models presented in published research.

Reviewer comment (Novelty):
The paper is a neuro-symbolic AI approach for learning and inference with barely any mention of previous related work in the field [1, 2, 3, 4, 5, 6, ..].

Author response:
We have expanded our related work section, and included in this change is the addition of a subset of the NeSy papers you have cited. However, many of these new citations belong to a subgroup of NeSy systems that enforces constraints on the structure of the symbolic model to ensure the final prediction has an explicit expression for its gradient with respect to the model parameters. In this work, we focus on the general setting where inference is an expressive and complex optimization problem, i.e., an explicit form of the gradient of the prediction, i.e., the problem solution, with respect to the model parameters does not exist or is impractical to compute. Going into detail on a tangential research direction under the large umbrella of NeSy is distracting for the reader and we maintained the focus on systems in the more general category and on bilevel optimization.

Reviewer comment (Theory):
I'm struggling to understand what the problem being solved here is exactly. The second paragraph in the introduction mentions that "the predictions are not guaranteed to have an analytical form or be differentiable, and traditional deep learning techniques are not directly applicable", could you please say more about that? All the NeSy AI techniques that I am aware of: semantic loss, deepproblog, NeSy entropy, semantic strengthening, semantic probabilistic layers, neupsl, etc... are able to train the models end-to-end using back-propagation.'

Author response:
Q1) You are absolutely correct that a line of NeSy approaches making predictions with an explicit gradient with respect to the neural and symbolic parameters exists. This research has undoubtedly made significant strides in creating efficient and end-to-end differentiable NeSy models. However, an emerging and, in our opinion, more advanced line of NeSy research exists where predictions are solutions to complex optimization problems [1,2,3,4,5,6,7,8]. For these models, predictions are not guaranteed to have an analytic map from the inputs and parameters or even be differentiable, and traditional deep-learning techniques are not directly applicable. For instance, you cited NeuPSL as an end-to-end trainable model via backpropagation. While it is true the gradient of the neural parameters is computed using the backpropagation algorithm, the gradient propagated in the original NeuPSL publication [1] is not from a supervised loss and is not with respect to the final NeuPSL prediction. Rather the gradient is for a value-based loss. Moreover, our paper is the first to formally show the value-function of NeuPSL is sub-differentiable (Theorem 5.3) with respect to the neural network predictions. Previous research, including  [1], assumed this was the case without formal proof.

Reviewer comment (Novelty):
As a follow-up question: why should we be interested in developing this equivalent formulation as a bilevel problem? What could one possibly gain that improves upon the exact inference proposed in semantic loss, deepproblog, NeSy entropy and semantic probabilistic layers?

Author response:
Q2) NeSy-EBMs are highly expressive and are used to perform large-scale joint reasoning with loopy dependencies and to ensure the satisfaction of constraints during both learning and inference. Moreover, NeSy-EBMs often do perform exact inference, contrary to your implication, and exploring this direction of research is valuable to the community.

Reviewer comment (Evaluation):
I am struggling to understand the results obtained on MNIST-addition. How can it be that NeuPSL, which is approximate, can outperform DeepProbLog which performs exact inference?

Author response:
Q3) Variations in performance between models in the MNIST-Addition tasks are a result of several differences. In the MNIST problem, NeuPSL inference is solved to an exact solution during both learning and final evaluation. Furthermore, the BCE loss used to train NeuPSL includes both a minimizer and a value-based loss, which can lead to data efficiency and good generalization. Lastly, we used the neural model architectures provided in the original publications for LTNs and DeepProblog, and naturally, there are variations in the implementations.