Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a method  to reduce the cost of adding a final ILP stage to a NN..
strengths_and_weaknesses: The paper explains its ideas clearly, It essentially transforms a linear ILP classifier into a continuous derivable problem my moving using distances to the hyperplanes. One may argue that this is not an ILP problem any longer, but it still stems from the original formulation, The results on sudoku loook good, supporting the idea,

Weaknesses:
- How easy it is to encode problems in LP?The examples are rater limited,
- The other  results are not as supportive, namely Table 3 (what happened to the run-times?)
questions: Tab 3 should have running times?
RRNs: the extra info  is? Is it used by the RRN resukts in the paper?
Diid you consider the other benchmarks in the RRN paper?
3.2 How does your system handle misclassified data? Can you introduce slack?
3.3 and 4,1 Why 3.3, given 4.1?
3.4 what is the difference?
limitations: The main limitations addressed in this paper is scalability; it would be nice to have more application ex.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposes a novel method for end-to-end training integer linear programming. The proposed method converts the ILP problem into a satisfiability problem, in which all constraints are transformed into inequalities. During the learning stage, the parameters of the inequality constraints can be learned via gradient descent given positive examples and randomly sampled negative examples. Experiments show that the proposed approach is effective and scalable. However, I am not an expert in integer programming, so I cannot judge the novelty of this work.
strengths_and_weaknesses: Pros:
1. The proposed approach is more scalable than previous methods, and the performance is good.
2. This paper has covered extensive related works and provides a detailed discussion about them.
3. The authors also propose several effective methods for sampling negative examples.

Cons:
1. Compared to the solver-based method, the proposed approach abandons the explicit symbolic representation, so it will be difficult to generalize outside of the training examples, e.g., training on 4x4 sudoku and test on 9x9 sudoku.
2. Converting the problem to a classification problem increases the sample complexity.

questions: 1. Is it possible to recover the inequalities from the learned parameters?
2. How does your method perform when comparing to SATNet?

limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces a solver-free approach for learning the constraint and cost parameters of an integer linear program from data. This means that the method does not rely on an expensive invocation of the ILP solver on the forward pass.
Instead, it relies on the available ground-truth solutions to train a set of soft max-margin classifiers corresponding to the ILP constraints and one additional constraint representing the cost.
The classifiers are trained such that a ground truth solution is classified positively by all classifiers, and any other integer point (sampled to be difficult to classify) is classified negatively by at least one of the classifiers.

The presented method (ILP-Loss) is evaluated on three experiments, including visual sudoku, random synthetic constraints, and a real-world keypoint matching benchmark. The main competing method is CombOptNet [1], which relies on invoking the ILP solver on every forward pass.

In the visual Sudoku experiment, the authors also include an additional SOTA baseline (RRN [2]) which is based on a graph neural network. ILP-Loss outperforms the RRN baseline on the largest board instances and performs comparably on the smaller ones while training significantly faster. CombOptNet times out after a very long training time, presumably because the ILP solver invocations dramatically slow down the training.
In the random constraints experiment, ILP-Loss again outperforms CombOptNet in terms of accuracy and runtime.
Finally, in the keypoint matching experiment, ILP-Loss is shown to achieve comparable performance to CombOptNet at a similar runtime, as the computational bottleneck of this experiment is in the neural network backbone.

[1]: Anselm Paulus, Michal Rolínek, Vít Musil, Brandon Amos, and Georg Martius. Comboptnet: Fit the right np-hard problem by learning integer programming constraints. ICML 2021. 

[2]: Rasmus Berg Palm, Ulrich Paquet, and Ole Winther. Recurrent relational networks. NeurIPS 2018.
strengths_and_weaknesses: ### Originality
The authors present a method for learning the constraint and cost coefficients of an ILP as multiple soft max-margin classifiers. While this approach has already been used to learn a classifying convex polytope from data (as also mentioned by the authors in the abstract), this is to the best of my knowledge the first time that the cost terms have been learned as an additional classifier, thereby extending the setting to cover integer linear programs.

### Quality
Overall the quality of this paper is good.
However, there is one major weakness in my opinion, which is a missing evaluation of the setting in which there is no single set of constraints describing the whole dataset, but in which instead the constraints depend on the input $x$. The presentation of the method suggests that this is doable (line 157), however, none of the experiments validates this. I am particularly surprised by the missing comparison as such an experiment has already been included in [1], and while all the other experiments from [1] were repeated this is the only one missing. I would ask the authors to include a comparison on this experiment, as I believe in any case (the presented method performing good or bad), the community would gain important insights. I will raise my rating of this paper if the authors include the missing comparison.

In addition, I would like to also point out that the papers [1,3,4], which are cited as neural-ILP architectures (line 44), are in fact capable of (and designed as) neural-ILP-neural architectures. This provides important context, especially for the experimental comparison to [1], and therefore should be corrected and discussed.

### Clarity
The clarity of the paper is good, it is easy to read and understand the introduced method.

### Significance
I think this paper is significant, as it appears to be the currently best method for learning the constraint parameters of a neural-ILP architecture. The performance on Visual Sudoku outperforming the current state-of-the-art method is impressive and demonstrates the capabilities of the presented method.



[3]: Marin Vlastelica Pogancic, Anselm Paulus, Vít Musil, Georg Martius, and Michal Rolínek. Differentiation of blackbox combinatorial solvers. ICLR 2020.

[4]: Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Jean-Philippe Vert, and Francis Bach. Learning with differentiable pertubed optimizers. NeurIPS 2020.
questions: See the suggestions in the quality section.
limitations: The authors discussed the limitation that currently a neural-ILP-neural architecture is not possible with the presented method. It would be great to include a short discussion of why this is not possible, and contrast it with some of the existing work which is applicable to this setting.
review_title: null

Meta-review: metareview: 
The paper presents the first solver-free training approach for learning neural integer linear programs. The idea is to encode within the loss that the final polyhedron separates the positives (all constraints satisfied) from the negatives (negatives (at least one violated constraint or a suboptimal cost value), via a soft-margin formulation. Compared to the Neural ILP baselines, this turns out to be faster without sacrificing accuracy. All reviewers agree that this is solid work. I fully agree. 
confidence: Certain
Decision: Accept