Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The goal of the paper is the provide counterfactual explanations (CFs) for neural classification models. The paper makes the point that existing methods for generating CFs are posthoc -- so they are not aware of the decision boundary of the model. As a result, the approximations taken by these models lead to subpar CFs. The posthoc search also means that the process of generating CFs is time consuming. To overcome these issues, the paper proposes to generate CFs as a part of the model forward pass. Specifically, the model architecture and the objective is modified to generate CFs. Experiments are shown comparing the performance with several existing methods.
strength_and_weaknesses: ### Strengths 

1. The topic of the paper (generating CFs) is important and timely.

### Weaknesses

Overall, the paper makes a bunch of adhoc choices when setting up the architecture and the loss function. The added value over the other methods is not very clear. Also, the proposed architecture does not support important criteria like diversity. See detailed comments below:

1. First, it is not very clear if improving the runtime is such an important aspect that requires training the model from scratch. From Table 3, it looks like Vanilla CF seems to be able to generate a CF within 1-2 seconds which seems quite reasonable. Also, adding the CF generation to model training seems to prolong the training time (due to multiple objective and alternative updates). How large was this additional training time?

2. Some of the choices could be justified better. For instance, why consider the non-standard squared loss in the training objective. The paper provides some ablation analysis but the reasoning behind moving from the standard cross entropy loss is not very clear. Similarly, why should the change loss be the squared loss (is it the L2 norm)? Why not consider the sparsity inducing distance like in Wachter et al? Similarly, should we expect the proposed method to be sparse in Section 4.1 when the objective function contains no sparsity term itself?

3. Section 3.3: Why is adversarial robustness is a special concern here? Aren’t usual NNs also prone to adversarial perturbations? I see the point that the addition of the validity loss might exacerbate the robustness. But then the paper should quantify the additional lack of robustness.

4. Section 3.3: How does the alternate update strategy solve the divergent gradient problem? Since the models two objectives are in conflict with each other, is there an equilibrium point? How do you detect convergence?

5. When measuring the manifold distance, it is not clear why the L1 metric was chosen. Why not L2 distance? Why only consider the 1-NN?

6. Recent work on CFs seems to support more criteria than validity and proximity. For instance, Mothilal et al consider diversity. Schleich et al (https://arxiv.org/pdf/2101.01292.pdf) consider a combination of distance function to cap the number of feature changes and the max. change in the feature. Can these criteria be included in CounterNet?

7. Since many of the gradient based methods highly depend on the init point (due to non-convexity of the CF objective), simply restarting with different random seeds could help with validity. It would be greatly helpful to add the restarting strategy to see how much the validity improves.

8. Different CF methods come with different hyperparameters (e.g., balancing between sparsity and validity). How were these HPs selected?
clarity,_quality,_novelty_and_reproducibility: Clarity: The paper is mostly easy to follow.

Quality: I think the quality in terms of explaining the added value and the design choices can be significantly improved. See the detailed comments and suggestions above.

Novelty: The idea of generating explanations as a part of the forward pass has been around for some time. Though I do not recall it being used in the context of CFs. Nonetheless, this seems to be a rather minor novelty.

Reproducibility: The paper links to the open source code though I have not checked it.
summary_of_the_review: Overall, it is not clear if the problem solved in the paper (improving runtime and validity of CFs) is a relevant problem to begin with. The paper also could be improved in terms of conveying the design choices which seem quite adhoc. For these reasons, I don't think the paper is ready yet.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper focuses on generating counterfactual explanations for model predictions, and then proposes an end-to-end learning framework that learns an explanation generator while training the predictive model. They also analyze the challenges in jointly training the predictor and the explanation generator and propose an iterative training procedure.
strength_and_weaknesses: Strength:
 - It proposes a new learning framework to generate counterfactual explanations(CF) for ML models, which facilitates service providers to specialize CF explanation techniques that can leverage the knowledge of their particular ML model.
- They theoretically analyze the joint objective and address two challenges in the jointly end-to-end training procedure. It may offer some insights into the reasonability of the iterative training scheme when jointly training two modules with different roles.

Weaknesses
- On the method:
    - The idea of intrinsic explainability is not new, though may have not been applied in generating CF explanations directly.
    - The so-called “block-wise coordinate descent procedure” is indeed an iterative training scheme, which is very common when handling two or more modules. 
    - While the paper emphasizes the proposed framework can achieve better cost-invalidity trade-off than the post-hot methods, it’s not clear why the framework can achieve that. The paper says “Second, in the post-hoc CF explanation paradigm, the optimization procedure that finds CF explanations is completely uninformed by the ML model training procedure (and the resulting decision boundary). Consequently, such a post-hoc procedure does not properly balance the cost-invalidity trade-off”, but it seems impossible and infeasible to obtain the decision boundary of the black-box model.
- Others:
    - A very similar paper seems to have been published on ICML2021.
    - GitHub link provided in the paper seems to leak information about the author.
    - There is a typo in the header of Table 5.

clarity,_quality,_novelty_and_reproducibility: The writing is clear and good. The reproducibility statement is also detailed. However, the novelty is limited.
summary_of_the_review: As the novelty is limited and the main argument(on cost-invalidity trade-off) is not well clarified, I am lean on the negative side.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: The work proposes a new method for generating Countefactuals for neural networks. The method consists of designing the network by having a part dedicated to the creation of counterfactuals, which is trained together with the predictive part. Through reasoned design and a special training, this new network, called CounterNet, is able to achieve accuracy on a par with other models. It can also generate counterfactuals in much less time than current post-hoc methods and achieve competitive values of widely used metrics from prior literature.
strength_and_weaknesses: # Strengths:

1. Clear and concise explanation of the Counterfactual Explainability setting.

2.  The key limitations of previous methods are clearly presented, as well as how the new method is able to overcome them. The limitations of this new method are also explicitly stated.

3. Much appreciated is the theoretical analysis of Convergence and Adversarial Examples, which gives depth to the paper.

4. Experiments seems complete, with multiple datasets, many baselines and ablation analysis about design choices.

# Weaknesses:

1. Although some are well shown, I think some limitations of the model need to be made more explicit. W1a) First of all, this method is not suitable for already trained networks, which makes it useless if you cannot/will not replace the model you have with this one.
     * Does the training run slower than that of a standard NN? If so, this should be highlighted as a (small) limitation. The time of just training the predictor could be added to Table 3.
     * I also think it should be made explicit that, considering the manifold, VCNet is still very competitive, also managing to have perfect validity, but lesser manifold on 3 of the 4 datasets.
     * While briefly stated in a footnote, there is no experimental proof that this architecture works well in domains where alternatives neuronal structures performs best. The authors try their method with a Convolutional block, but with tabular datasets; as they say "convolutional block is not well-suitable for tabular datasets”. Likewise, is not explicit if the CounterNet used for the Image dataset is a convolutional one (Appendix I). Since this is not a good enough experimental proof to state that the method works well (as CF Generator) with other types of neural networks (as you say for images), it is necessary in my opinion to make this limitation explicit in the main paper and not only in the supplementary.

2. “Note that passing px through the CF generator network is analogous to feeding information about the decision boundary to the CF example generation procedure, which leverages this knowledge to find high-quality CF examples x′” The reason why px contains boundary information is not clear. Likewise, is not theoretically explained why the CF generator needs both zx and px (apart from the experiments in the Ablation Analysis section).

3. Some claims in the Experimental evaluation section should be sustained via statistical tests.
clarity,_quality,_novelty_and_reproducibility: The work is presented extremely clearly and, to the best of my knowledge, represents a clearly novel method for the field. Reproducibility should be high, given the publication of the code (although not personally tested). I consider the quality of this work to be very high.
summary_of_the_review: The strengths of the work far outweigh the weaknesses I have identified. Regarding these, they are mostly on the clarity of certain images or statements, which I strongly encourage the authors to follow, to make the work even more polished than it is now.

## Minor remarks:
1. In Introduction, I would advise some minor specifications:
    * It should be specified that x2’ is an instance with different (or opposite) label.
    * “More generally, CF examples with low (high) cost of change imply high (low) invalidities”. While usually true, this is not always the case. I would rephrase it, including “often”, “usually” or a synonym.
    * “Next, we describe key limitations of prior CF techniques.” Seems redundant/ a repetition.
    * “First, all prior methods” I’ll add, just once and here, “to our best knowledge” or similar.
    * “Post-hoc explanation techniques are agnostic” Not every method is agnostic.
    * “Consequently, such a post-hoc procedure does not properly balance the cost-invalidity trade-off (as explained above), causing shortcomings in the quality of the generated CF explanations” This would need a citation

2. In Section 3:
    * “output is binary-valued, then yˆx and yˆx′ should take on opposite values (i.e., yˆx + yˆx′ = 1)”. Since in a binary classification task, labels can also be (+1) and (-1), I would write “output is binary-valued (0, 1)” or similar
    * While very well designed, Figure 2 hides two flaws. First, it does not include a yx’ used just for training, that is indeed cited in the text; probably adding a dotted line at the end of Predictor will fix this. Second, it should be highlighted that the connection from the Predictor to the CF Generator happens only once; otherwise, the network, in training regime, will continuously produce Counterfactuals of its own Counterfactuals in loop. Even in text, the “feedback loop” from x’ to input is mentioned, but not the px to CF Generator.
    * “generate a latent vector z ∈ Rk (s.t. k < d)”, “and up-samples” While generally true, there’s no need to specify k < d, since, to my understanding, nothing prevents the network to work even if k ≥ d, unless you have experiments that prove otherwise.
    * I think that three hyper-parameters are not actually needed for the loss (1), cause what’s important is the relative weight of the three losses. The additional parameter probably just ensure that there are multiple optimal configuration, fixing other hyperparameters (e.g. learning rate, decay rate, etc.). However, I don't think it should be changed.
    * “Proof of Lemma 3.1 and 3.2 can be found in Appendix A.” It’s strange to mention Lemma 3.2 in the Lemma 3.1 section. I would move this sentence to the previous introductory paragraph, that of subsection 3.3
    * “we update network weights θ = {θh, θf , θg} twice” This is in contrast with the sentence “we only update the weights in the CF generator θg”, cause you seem to update twice just the θg weights. But, if looking carefully at L1 loss, it doesn’t depend on θg, so each weight is actually updated just once, if you freeze the gradient updates.

3. Experimental evaluation
    * The Ablation Analysis section is really appreciated, but in my opinion needs slightly more clarity. Perhaps even just starting a new line every time a modification is mentioned would help.
    * "reaches the theoretical upper bound of robustness (i.e., the robustness of base model)". This statement is incorrect for me. First of all, I do not understand why the upper bound of robustness should be the robustness of base model. Even if it were, CounterNet exceeds it, so it cannot be an upper bound. Secondly, there is no theoretical study of the bounds of robustness, so I would avoid such a mathematically specific statement.

4. Appendix:
    * In Section G, “methods with only 2 ̃% decrease (in average).” The tilde ~ should be before the 2%
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper tackles the problem is generating counterfactual (CF) explanations for neural networks. The authors propose a model that consists of an encoder, a predictor, and a CF generator. Correspondingly these models are trained by optimizing an objective function with three loss terms, prediction loss, validity loss, and change loss. Experiments show that CounterNet can generate CF explanations with reasonable performance. 
strength_and_weaknesses: Strength 

+ Generating counterfactual explanations is an important and interesting problem in the ML community. 

+ The design of the model with three terms, prediction loss, validity loss, and change loss, makes sense. 

Weaknesses

- The proposed method has limited technical merit, especially compared to recent methods such as CounteRGAN [1]. 

- Some claims in the paper on major limitations of existing methods are not true. 

- Important and highly relevant baselines are missing.

- Lemma 3.1 is incorrect or irrelevant to Eq (1) (see below for details). 

clarity,_quality,_novelty_and_reproducibility: The paper is well written and well organized. The problem it tackles is also interesting for the ML community. 

One of my major concerns is the technical novelty, especially compared to to CounteRGAN [1], which is also highly efficient and uses a feed forward CF generator. This also begs the question why [1] is not included as a baseline to evaluate metrics such as Validity and Proximity. 

In terms of runtime, note that methods like CounteRGAN [1] only need one single feedforward pass to generate the CE and therefore should be much faster than the proposed method. Unfortunately, [1] is not included as a baseline, making it difficult to reliably evaluate the efficiency of the proposed method compared to the state of the art. 

This is also highly related to the authors’ claim that existing methods are slow because they rely on solving an optimization problem (one of the claimed major limitations). In fact [1] also contradicts the authors’ another mentioned limitation that existing methods are post-hoc and do not leverage training information. Note that [1] could also potentially be trained jointly with the predictor. 

Lemma 3.1 is problematic or disconnected with Eq (3). Note that according to Eq (3) the prediction loss and the validity loss use different $y$’s as the target. Specifically, the prediction loss use $y_i$, the original label for $x_i$, as the target, while the validity loss use $\hat{y}_{x_i}$, the predicted label, as the target. However, in Lemma 3.1, $L_1$ and $L_2$ share the same target $y$. 

Besides, the divergent gradient issue in Lemma 3.1 only happens when $x$ and $x’$ are close. However, we can almost be sure that $x$ and $x’$ are NOT close near the convergence point. Therefore, Lemma 3.1 is not related to the convergence point of the optimization problem at all. Rather, it is only related to the initialization point of the optimization problem. 

For the second issue on adversarial examples, Lemma 3.2 is incomplete. Note that in practice, $L_1$ and $L_2$ are jointly optimized. In this case, the optimization process will try to train the predictor such that the prediction loss is minimized while training the CF generator to minimize the validity loss. 

It would also be helpful to provide ablation study results on the performance when Eq (3) is optimized in the naïve way. It seems this is included as CounterNet-SingleBP in Table 4. 

Table 1 shows that CounterNet has a disadvantage in terms of predictive accuracy when compared to post-hoc approaches, since post-hoc approaches could preserve the predictive accuracy of the base models fully. 

According to Table 2, it seems CounterNet does not have consistent improvement upon baseline methods. 

The writing is inconsistent. In the abstract, the authors mentioned that existing CF methods suffer from two major limitations. In the introduction section, the number of major limitations becomes three. 

The authors argue that post-hoc explanations are a major limitation in that they do not full make use of the information during training of the ML model. I find it somewhat vague and unconvincing. 

[1] CounteRGAN: Generating Counterfactuals for Real-Time Recourse and Interpretability using Residual GANs. UAI 2022. 

summary_of_the_review: This paper tackles the problem is generating counterfactual (CF) explanations for neural networks. This is an interesting and important problem in the ML community. Overall the model design makes sense to me. My major concerns include the absence of highly related state-of-the-art baselines, several false claims or over-claims in the abstract and introduction, and the limited technical merit, especially given a related work CounteRGAN.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper aims provide counterfactual explanations for neural classification models. Specifically, it proposes to generate counterfactual explanations as a part of the model forward pass by modifying model architecture and the objective. It conducted experiments to compare with some existing methods.

While it is a very important topic, it makes several adhoc choices when setting up the architecture and the loss function. Unfortunately, authors did not explain clearly on the values added and the proposed architecture does not support important criteria like diversity. 
justification_for_why_not_higher_score: Its contributions and novelties are not clear
Decision: Reject