Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a new method for the provable repair of neural networks. Both point-wise and polytope-based repairs are considered. The main idea is to synthesize a patch network and combine it with the buggy network. The patch network consists of two subcomponents: a support network that is only activated for the inputs in the linear region containing buggy inputs and an affine patch network that repairs the network in the buggy region. The experimental evaluation is performed on similar datasets and networks as prior work and shows the effectiveness of the new methodology.
main_review: Pros:
1. The paper tackles a very relevant but relatively unexplored problem. While there is substantial literature on proving robustness of neural networks, there is only limited work on providing algorithms for provable repair which is as important when considering the deployment of neural networks in the real world. I enjoyed reading the paper, it is well-written. I appreciate that the authors provide intuitive examples to explain the meanings of the different symbols and equations in their algorithmic construction.

2. The technical contributions are novel and solid. 

3. The authors provide theoretical results which shows that their algorithms provide stronger guarantees than the state-of-the-art for provable neural network repair.

4. The evaluation is performed on similar benchmarks as prior work and shows significant improvement on obtaining provable repair while minimizing changes to the network.

Cons:
1. The claim about preserving the continuity property needs to be toned down.  The first row in Table 1 says that the proposed repair preserves the continuity property however, the text in Section 3.2 mentions that this is not the case.

2. Scalability to area repairs defined over higher-dimensional polytopes is limited but that is also a limitation of all existing work.


I have a few other comments/questions:

1. The notation in eq. (3) that (c,d) \in {(c,d)...} is weird.

2. For multi-area repair, does different ordering of patch areas yield different repaired networks?.

3. What is the running time for all the tools in Table 2?  Does the timing in Table 3 also include the time to generate vertices?

4. Is it possible to obtain results on a CIFAR10 network in the same setting as the MNIST network or it would be too expensive?

5. What is the value of \gamma used in the experiments? It would be good to show the different metrics vary as a function of /gamma?

6. Similar to DDNN, is extension to arbitrary activations possible for pointwise repair?
summary_of_the_review: The paper presents a complete and sound methodology for the provable repair of neural networks while minimizing the side effects of the repair. The ideas presented in the paper are new, come with solid theoretical guarantees and advance the state-of-the-art. Based on the contributions, I would recommend acceptance of this work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a new methodology for repairing ReLU neural networks. Concretely, for a single affine region, a patch function is synthesized from an affine function that performs the repair and a support network to make sure the repair only applies to the affine region. Multiple affine regions can be fixed iteratively. The techniques are evaluated on the MNIST and HCAS benchmarks.
main_review: The presented techniques are quite interesting and are novel to my knowledge. The presentation of the technique is also clear. I like the running example in section 3. I do have some concerns and questions.
1. Point-wise repair.
To me, the main motivation of point-wise repair is to improve the overall performance (e.g., accuracy) of the network instead of simply making the network perform correctly on a small set of input points. If our goal were the latter, then instead of using the proposed techniques, we could just put the network in a big if-statement which returns the desired output on those points. In other words, in the point-wise repair scenario, we do hope the repair to have side effects, for instance, increase of test accuracy beyond those repaired test images (are the buggy points images in the test set?). It would be beneficial to demonstrate whether this is the case, as right now it looks like the increase in test accuracy results only from fixing the buggy points (with the neural network outputs minimally unchanged).
2. Affine-region repair.
Partitioning the input space into regions where the network is affine and fixing each of those regions sounds feasible for small networks like HCAS. However, it seems challenging to partition the input region into affine sub-regions for large networks or high-dimensional inputs. I do think it is important to show in the main paper 1) the application of the techniques on more complex models; 2) area repairs (for robustness properties for instance) on perception networks. 
For HCAS, how are the linear regions represented, boxes?
3. Can the authors comment on the (initial) architectures of the evaluated networks?

Overall, while I think the idea is interesting, more work needs to be done to show the benefit beyond minimal change for point-wise repair and to address the scalability issues.
summary_of_the_review: Interesting ideas for neural network repair but there are concerns regarding motivation and scalability.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper develops a method for repairing neural networks with ReLU activation
functions. The method works by augmenting a given network with a patch function that
corrects the output of the network on a given buggy input. This enables the
method to be the only one among related work that is not only sound and
complete but also guarantees (i) minimal distances between  the functions
implemented by the original and repaired networks; (ii) unaltered behaviour of
the repaired network for inputs different than the buggy one.

main_review: This is a very well written paper which was a pleasure to read.  The paper
draws a clear picture of the state-of-the-art in neural network repair which is
used to  adequately motivate the present contribution. In particular, I think
that the potential side effects that the current state-of-the-art may have
whilst repairing a network, such as introducing a new bug in a previously
non-buggy input, is an important limiting factor of the practical significance
of  neural network repair. It is nice to see an approach towards overcoming
this shortcoming.  I think that the claimed advantages of the approach over
related work are adequately evaluated on an MNIST network and a variant of the
ACAS networks. The scalability of the method is also discussed and additional
experiments on AlexNet are provided. A minor concern that I have regarding the
evaluation is the following.

I think it would be impractical to remove all buggy behaviour (outside of a
test set) from image classifiers. In light of this a metric that is potentially
more important than the ones measured in the experiments is  the robustness of
classifiers w.r.t to input transformations such as noise perturbations. Data
augmentation schemes often aim at improving  precisely this, thus I think that
data augmentation and the present method should be compared on this. So,
although the MNIST dataset was successfully used to showcase the advantages of
REASSURE over related methods, the inclusion of experimental results on
robustness would in my view strengthen the evaluation of the approach.
 
Some minor comments:

- Page 1-2. loss defined based on -> loss defined on

- Page 3. linear functions -> affine functions.

- Page 6. can solve it is -> can solve it.

- Page 6. with every entry is equal -> where every entry is equal.

- Page 7. REASSURE provide -> REASSURE provides.

- Shouldn't the distance the Theorem 4 be bounded from above?

summary_of_the_review: The paper alleviates some significant limitations in neural network repair in a
method that is adequately evaluated, thereby making for a solid ICLR
contribution.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed an approach for repairing neural networks that use ReLU activation functions. Different with existing techniques that retrain the model or change the weights, the proposed approach generates a patch in the original model, i.e., add sub-models. The LP solver is used to solve the patch function for a linear region. The experimental  results demonstrated that the approach could outperform existing methods. 
main_review: # Strengths:
1) This paper aims to address an important problem.
2) This paper is well-written and easy to follow.
3) The approach has some novelty.

# Weakness:
1) This paper only focuses on CPWL neural networks, which limit its usefulness.
2) Point-wise repair is a more practical scenario. In real world applications, there are usually a set of buggy inputs such as the misclassified images, texts or audio. Most of existing repair techniques aim to focus on this problem (e.g., retraining). However, this paper requires the linear area for repairing, which is not always provided. Although it provides a way to transform one input point to the linear region in 2.5, there are still some concerns: a) how to guarantee the correctness of the transformed linear regions, the linear region is an over-approximation of the input x?  b) if there are too many failed inputs, then many linear regions will be generated, which may cause the scalability issues. The usage of LP solving may also introduce the scalability issues.  c) In Eq.1, what do you mean $z_j^i <0$? Is it the neuron output of one input $x$?
3) Some description is over-claimed. For example, in Section 1, this paper claimed that, for existing approaches, ''there is no guarantee that the given buggy inputs are fixed and no new bugs are introduced, the Generalization in Section 2.6. It seems that the approach can also not guarantee that no new bugs are introduced  and the claimed generalization, especially for the point-wise repair. Consider the comments (2) above, how to guarantee them? Any support for this claim "the optimization-based approach cannot guarantee removal of the buggy behaviors, and the verification-based approach does not scale beyond networks of a few hundred neurons."?
4) Ensuring the locality is important, but why  Minimality is required? It seems that if we can fix the errors and do not affect the correct inputs, it should be okay. Why Minimality is so important?
5) In Section 3.2, you suppose the linear region is $\{x| a_ix\le b_i, i \in I\}$, is it general for all linear regions?
6) The evaluation is weak. It only compares the proposed method with the existing methods on MNIST for point-wise repair, which may confirm the scalability concerns? Although there are some results on AlexNet, the results are not complete and systematic. For Section 5.1, more discussions are required. For example,  how many failed inputs are fixed by each method? The performance is also not included. How many linear regions are used in different configurations? How is the generalization, i.e., can the repair fix unseen buggy inputs? 



summary_of_the_review: Overall, this paper is well-written and interesting. However, considering the limited application, the little discussion on the point-wise repair and the weak experiments, I think the paper should be still improved before it is accepted.
review_title: null

Meta-review: Reviewers were almost unanimous in favor of this paper, with scores of 5,8,6,8.
I think it's a neat idea and am inclined to accept despite some issues w/ motivation / scalability. 
Science proceeds in increments, and it's OK to propose something with scalability issues that someone else later tries to fix, etc.
Decision: Accept (Poster)