Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper focuses on improving the standard (clean) accuracy for certifiably
robust models.  To achieve good certified accuracy, previous works typically
make the standard accuracy much worse than naturally trained models. The
authors propose a selection mechanism to choose between a certified model with
low clean accuracy and a naturally trained model with high clean accuracy.  At
a high level, when the certified model cannot certify, there is no point to use
it for classification. A naturally trained model (which cannot be certified as
well) is selected to improve standard accuracy.

Strengths:

Most previous works on certified defense focus on improving certified accuracy,
and standard accuracy is usually sacrificed. This paper focuses on a different
and important setting where high standard accuracy is desired, which is
neglected by many previous works. I think this is a good step.

The proposed selection scheme can balance a certifiably robust model with a
naturally trained but highly accurate model. Such a combination can be helpful
in the settings where high prediction accuracy is required.

The proposed method is technically sound. Using a certified selector makes the
whole network certified when it chooses the certified network. To improve clean
accuracy, The core network is used when the certified selector chooses the core
network (i.e., the selector believes the certified network cannot make a good
prediction on this example) or cannot certify.

The paper overall is well motivated and organized.

Issues and questions:

At a high level, this certification scheme does not improve certified accuracy
(it only makes it worse); it only helps with the verified accuracy vs. clean
accuracy trade-off.  Thus, a crucial part of evaluation is to show the verified
accuracy vs.  clean accuracy tradeoff.  However, it is not well demonstrated in
the experiments. Especially, I think results Table 1 are not so useful because
we can't see how the baseline certified defense models perform and cannot see
this tradeoff. Also, the certified accuracy numbers are really low compared to
other works, and sometimes close to 0 (e.g., on ImageNet-200 only 3% accuracy).
Thus, it is important to show a tradeoff figure here.

I recommend using figures similar to Figure 2 to present the results for all
settings (CIFAR 2/255 and 8/255; downscaled ImageNet-200 at 1/255) (but be
aware Figure 2 has its own issues, see comments below). Importantly, we should
fix a well known certified model (e.g., COLT or CROWN-IBP) and then, apply ACE
with different thresholds to see how the clean accuracy improves with dropped
verified accuracy.  For CIFAR, COLT or CROWN-IBP pretrained models can be used
as the base certified model.  For Imagenet-200, I found a recent work [1]
presented certified defense models on 64*64 TinyImageNet and ImageNet datasets
which can be helpful. They reported around 15% certified accuracy and also uses
much larger model structures which should improve the results in this paper by
using their pretrained models as the base certified model for selection (I
doubt the simple CNN models in this paper are sufficient for ImageNet). Again,
the trade-off part is the most important results to see in this paper, which is
not well demonstrated.

Figure 2 made a misleading comparison because the ACE based methods are using
COLT as the base certified classifier and it is inappropriate to compare it to
CROWN-IBP with different kappas. We should either also use COLT trained with
different weights on natural loss (similar to the kappa in CROWN-IBP) to see
this tradeoff, or use CROWN-IBP as the base certified classifier in this
figure.  Especially, in the CIFAR 2/255 setting, COLT achieves better clean
accuracy than CROWN-IBP, so this gives ACE an advantage in this comparison, and
the claim that ACE achieves a better trade-off than using the tuned kappa
parameters in (CROWN-)IBP training cannot be justified.

It also seems to me that in Figure 2 the CIFAR 2/255 CROWN-IBP numbers are much
worse than the ones reported in CROWN-IBP paper (they reported 28.48% standard
error and 46.03% verified error), but in Figure 2 it is much worse (~35%
standard error and ~50% verified error). If we use the correct CROWN-IBP model,
it should start at a similar place at the ACE based methods in Figure 2, rather
than on the far left. Can you explain?

Conclusion:

I like the aim on standard accuracy and the network selection idea proposed in
this paper, but its current evaluation is partially missing or misleading and
cannot justify all claims. So I cannot recommend accepting its current version.
However I will be glad to discuss with the authors and re-evaluate the paper
based on new evaluation results from the authors. I will be happy to accept this
paper if the authors can address my issues mentioned above.

---
### After rebuttal

See my reply below for my comments after rebuttal. Overall I feel the paper still has room for improvement and there are several open issues, but it has been improved so it is marginally above acceptance threshold now.

---
Reference:

[1] Xu, Kaidi, et al. "Provable, Scalable and Automatic Perturbation Analysis on General Computational Graphs" https://arxiv.org/pdf/2002.12920
review_title: Review for Paper3751

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a new network architecture that combines 1) a state-of-the-art deep neural network with high accuracy (but potentially no robustness certificate), and 2) a small certification network with high certifiable robustness (but not necessarily very high accuracy), using a selection network that adaptively chooses between these two networks. They show that by doing so, the new architecture is able to take advantage of both networks and thus obtain good natural accuracy with better certified robustness that significantly improves upon prior benchmarks.

The main advantage of this framework is its flexibility in allowing arbitrary combinations of STOA deep networks with any networks with certified robustness and their selection mechanism is able to make good use of both.

1. I like this simple idea and I am glad to see its good performance, although I wish the author can develop more theoretical results to quantify the value of a hybrid model.

2. According to (2), the objective may not be differentiable because of the binary function $g$. Can you elaborate on how gradient-based algorithms are applied to this formulation?

3. Can you provide some interpretation of the learned selection mechanism $g_{\theta_s}$? In particular, what features of the samples make them be passed through the core or the certification network? 


review_title: Combining SOTA network with certified network using an adaptive selection mechanism

final_score: rating: 6: Marginally above acceptance threshold
review_content: # Summary of Contributions
The paper presents an approach to trade off natural accuracy and certified robustness by combining a network with high natural accuracy (the “core network”) with a second network with high certifiable robustness (the “certification network”). A selection mechanism is used to decide which network an input sample should be processed by. The selection mechanism allows the combined system to perform significantly better than a weighted average of the core and certification networks (e.g. randomly assigning input to the core network with some probability $p$) would. 

# Score Recommendation
Despite the weaknesses in experimental evidence, clarity and reproducibility identified below, I recommend an acceptance because the authors have demonstrated that the selection mechanism presented works for non-trivial problems, providing a simple way to trade off natural accuracy and certified robustness. 

ACE can consistently benefit from advances that improve the natural accuracy of the core network. In addition, as long as a selection mechanism can be found that is compatible with the certification network, it would be possible for ACE to leverage improvements in certified defenses.

While results are only presented for $l_\infty$ perturbations, I expect that the same approach can be applied to different perturbations, as long as it is possible for the selection mechanism to have a tunable selection rate while having non-trivial robustness to the perturbation of choice.

# Weaknesses
## Missing Experimental Evidence
- The paper claims in the abstract that it is the first to obtain a high natural accuracy with non-trivial certified robustness; the results (91.6% natural accuracy, 22.8% certified robustness) are compared to the prior state-of-the art (77.4% natural accuracy, 16.5% certified robustness). However, I am concerned that the comparison may not be completely fair (if the network the comparison is made to is one of the CROWN-IBP trained DM-Large networks; see the section on “Clarity” below). The paper itself acknowledges (in the first paragraph of page 7) that they did not conduct the extensive training required to obtain the performance reported in the CROWN-IBP paper.
  - The authors should either train the CROWN-IBP DM-Large network with at least as much resources as in the original paper, or clearly identify in both the abstract and introduction that the amount of compute used was constrained.

- The paper claims that “ACE produces much more favorable robustness-accuracy trade-offs than varying hyperparameters of the existing certified defenses” on the basis of comparing DM-Large CROWN-IBP to the Conv2 COLT-based ACE SelectionNet. I believe that more evidence is required to substantiate this claim, since the choice of a base network is rather arbitrary (why choose one comparable to the second DM-Large network, not the first or the third?).
  - One possible set of experiments is to use each of the 5 DM-Large networks with non-trivial certifiable accuracy as the certification network, and then showing that the resulting families of  ACE SelectionNets have a better robustness-accuracy tradeoff.

## Possible Experimental Errors
The Conv3-COLT network in Figure 2 has a performance (~75% natural accuracy, ~50% certifiable accuracy) that is significantly worse than that reported in the original COLT paper (78.4% natural accuracy, 60.5% certified robustness). What is the cause of this significant gap?

## Clarity
- The term “ConvMedBig” is used in the caption for Figure 2 and elsewhere in the paper, but is not defined in the original paper. (It appears that the authors may be referencing [the name in code](https://github.com/eth-sri/colt/blob/20f30b073558ae80e5e726515998c1f31d48b6c6/code/networks.py#L79)). The authors should provide more detail about specifically which network this is. In fact, it appears from the third paragraph of Section 7 (and the code linked above) that “ConvMedBig” matches the network Conv3 exactly. If this is the case, the same name should be used.

- The authors compare to a prior approach with 77.4% accuracy and 16.5% certified robustness but do not specify what this approach is. (It appears from Figure 2 that this may be one of the DM-Large CROWN-IBP networks)

- At the bottom of page 6, the paper states that “the smaller networks to which COLT scales lack capacity to obtain the kind of robustness-accuracy trade-off that we target”. What does this mean? A significant proportion of the results in Table 1 are presented for COLT, so I’m confused by this statement.

## Reproducibility
- Hyperparameters for the PGD attacks used are not provided, making it difficult to understand the strength of the adversarial attack being used. (If the adversarial attack is weak, the adversarial accuracy presented in Table 1 may be significantly higher than the actual robust model accuracy).
- More details should be provided about the algorithm used for certifying the networks in Table 1 (other than the Entropy-COLT-Conv2 network). The third paragraph of Section 5 states that “we only use … convex relaxation-based certification methods based on intervals and zonotopes”, but I couldn’t find any further details (for example, what zonotopes were used to verify the certification network?)

# Questions for Authors
- I’d like to better understand how the adversarial accuracy of the network was evaluated; the paper only mentions that it is “usually computed using an adversarial attack such as PGD” (see the first paragraph of Page 3). One of my concerns is that the selection mechanism (particularly where a selection network is used) may reduce the success of PGD adversarial attacks without increasing the robustness of the network, possibly via gradient masking [1].
- In paragraph 2 of Section 5, the paper specifies that an adversarially trained network was used as the core network. Given that the last paragraph of Section 4 states that “we assume that certification of the core-network always fails”, why did you choose an adversarially trained network (which presumably has slightly worse natural accuracy?)

[1]: Papernot, Nicolas, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. "Practical black-box attacks against machine learning." In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519. 2017.


# Additional Feedback
- hyperparamters → hyperparameters (4th line, third paragraph of section 5)
- Figure 2 presents many different networks but it is not clear what the point of the figure is. Compounding the issue is the fact that the corresponding discussion begins almost an entire page later. For improved clarity, the authors should consider adding more to the caption for Figure 2 or moving it closer to the discussion.
- The term “natural accuracy” and “standard accuracy” is used interchangeably; the paper should settle on one.
- Figure 7 labels the y-axis “std of input zono errors” but this term is never introduced anywhere else in the text. (Is this the standard deviation, perhaps?)

# Post-Rebuttal Comments
I've maintained my score at 6.

During the comment period, the authors made progress in improving the clarity of their presentation. As with reviewer 3, I feel that there is still room for improvement; in particular, moving some experiments in Section 5 to the appendix could make for a more focused paper with a clearer message for the reader. (Unfortunately, we did not have enough time during the comment period to get there). 

I'd also note that the paper is now at nine pages; this means that I am holding it to a higher bar.

Overall, however, I continue to recommend an acceptance as the method to trade off natural accuracy and certified robustness is simple and significantly improves on the state of the art; for me, these strengths outweight the remaining issues. 
review_title: Novel idea with good results that would improved by better experimental evidence and clarity

Meta-review: This paper proposes a selection mechanism to choose between a certified model with low clean accuracy and a naturally trained model with high accuracy, to improve the standard clean accuracy for certifiably robust models.  At a high-level, the idea behind this combined system is that when the certified model cannot certify, one should avoid using it for classification, but rather should use a naturally trained model.  A state-of-the-art naturally trained networks is used as the "core network", and a small certification network with high certifiable robustness is used as the "certification network". The major contribution is a selection network that adaptively chooses between these two networks. 

Pro
+ The idea of using two networks adaptively is novel. The proposed selection mechanism has been shown to be able to combine the merits of both networks to obtain better natural accuracy with good certified robustness. 


Con
- The experiment section still has room for improvement. Specifically, the presentation of the results were not convincingly conveying the tradeoff between the clean accuracy and the certified accuracy.  After the rebuttal, the authors made some improvements that addressed many of the concerns about the clarity and reproducibility issues. However, reviewers suggest further polishing the experiment section. 

Overall, I think the novelty of the paper combined with the promising results achieved outweigh the presentation issues. I would recommend accepting this paper. 
Decision: Accept (Poster)