Auditing AI models for Verified Deployment under Semantic Specifications

Abstract: Auditing trained deep learning (DL) models prior to deployment is vital for preventing unintended consequences. One of the biggest challenges in auditing is in understanding how we can obtain human-interpretable specifications that are directly useful to the end-user. We address this challenge through a sequence of semantically-aligned unit tests, where each unit test verifies whether a predefined specification (e.g., accuracy over 95\%) is satisfied with respect to controlled and semantically aligned variations in the input space (e.g., in face recognition, the angle relative to the camera). We perform these unit tests by directly verifying the semantically aligned variations in an interpretable latent space of a generative model by building a bridge with the DL model. Our framework, AuditAI, bridges the gap between interpretable formal verification and scalability. With evaluations on four different datasets, covering images of chest X-rays, human faces, ImageNet classes, and towers, we show how AuditAI allows us to obtain controlled variations for verification and certified training. We address the limitations of the standard approach of verifying using only pixel-space perturbations.

Reviews and author responses:

 Dear AC,

Thank you for your questions. We understand the concerns around IBP but wanted to clarify that our framework incorporates an improved version of IBP, called CROWN-IBP [Zhang et al., 2019b] that has shown competitive results. We discuss this in section 2.3 of the paper.  In addition, the AuditAI framework can be used with other bound propagation techniques as well and is not limited to IBP-based methods.   

Please find below our answers to the specific questions below. All of the questions will be clarified in the updated manuscript to prevent similar misunderstandings for future readers.    

> How is classification done exactly for ImageNet 100?  

For the ImageNet experiments, the variations in latent code correspond to different images in the same class. For example, in the class “hen” the latent variations yield different poses of "hen" in the image. The classification task is to output the correct class with such class-conditional latent perturbations.

> What kind of subset of classes has been used for the ImageNet 100 experiment? Has this set been used somewhere else? If not, what was the motivation for the selection of the classes? What kind of performance would one get if one would train a standard model, e.g. a ResNet50, directly on this set of classes in pixel space? 

- We chose 100 classes randomly, mainly for computational tractability, instead of training on the entire 1000 classes. The full dataset would require resources beyond our budget, both in terms of time and compute. Notably, tractability was the only consideration for this selection.   
- The accuracy of ResNet-50 trained in pixel-space on these classes is 84.4% (i.e. error is 15.6%), which is comparable to that of the verified error with $\epsilon=25$% in Table 1. We will mention this detail in the paper for reference.

> What is the normal classification performance of the classifier for ImageNet 100 trained with IBP (without any guarantees)? 

The nominal performance of the classifier without any perturbation guarantees in terms of error corresponding to Table 1 is $0.15\pm 0.03$. We will include this information in the results section for completeness.

> Is the guarantee in Table 1 with respect to a cube in latent space of size epsilon around every test point or is the guarantee in Table 1 for the cube defined in Section 2.4? 

The former, as is standard in the verification literature. The guarantee in Table 1 is with respect to a cube in the latent space of size epsilon around every test point. We will make include clarification on this in the paper.

> What is a verified error fraction? Why are you not just reporting usual verified errors in percentage? 

We are indeed reporting the usual verified error in the form of a fraction instead of a percentage. I.e. in the form of 0.23 instead of 23%.   
We apologize for the confusion and will make this clear in the paper. 

> Maybe I am missing something but is anywhere stated what fraction of the test points fall into the cubes defined in Section 2.4? Isn't this an important point of your paper as according to Section 2.4 this cube would be used to judge if an input is valid? 

Thanks for the question, **we had initially not reported this number because there is no ground-truth to compare this against**. Here are the details of this: The fraction of test images within the range described in section 2.4, for $\epsilon=25$% in Table 1 is $85.3$%, in Table 3 is $89.5$%, in Table 5 (FFHQ) is $86.4$%, in Table 5 (LSUN) is $90.2$%.  
However, we will include this in the revised paper with an appropriate discussion about the lack of comparison. 

> You provide classifications of your latent code like "sunlight/brightness"? Where do you get this from? How would these latent codes enter a formal auditing process? How would a user understand what the latent dimension which is certified means for him in practice? 

- We obtain the latent dimensions corresponding to semantic variations like sunlight/brightness from techniques that identify latent directions of change in GANs, which we have referenced in sections 4.3 and 4.4. 
- This mapping from latent dimension to semantics will be provided to the end-user as part of the specification sheet that shows the auditing guarantees. 

> The experiment in Section 4.5 is not clear to me. It seems that for the smaller radius eps=0.5 more images can be attacked, than for the larger radius eps=1.5 (which includes the smaller radius). 

This is a typo in section 4.5, the numbers for the two categories should indeed be reversed!  **9.6% is for eps = 1.5 and 8.5% is for eps=0.5** 
Thank you for helping us find this error.

We hope our answers have clarified the questions from the AC. kindly let us know if there is anything else we can clarify or elaborate on. 
Thank you for your time!


 Dear authors,

at the moment it is a bit difficult to judge the utility of the approach. Typically, when training models using IBP one has a trade-off between the achieved accuracy and the guarantees one can obtain. This discussion seems to be missing. Also IBP training is mainly effective for rather shallow architectures which do not allow SOTA performance e.g. for ImageNet 100 you use five fully connected layers on the latent space (Appendix A.3)

Thus my questions are:
- How is classification done exactly for ImageNet 100?
- What kind of subset of classes has been used for the ImagetNet 100 experiment? Has this set been used somewhere else?
If not, what was the motivation for the selection of the classes? What kind of performance would one get if one would train a standard model, e.g. a ResNet50, directly on this set of classes in pixel space?
- What is the normal classification performance of the classifier for ImageNet 100 trained with IBP (without any guarantees)?
- Is the guarantee in Table 1 with respect to a cube in latent space of size epsilon around every test point or is the guarantee in 
Table 1 for the cube defined in Section 2.4?
- What is a verified error fraction? Why are you not just reporting usual verified errors in percentage?
- Maybe I am missing something but is anywhere stated what fraction of the test points fall into the cubes defined in Section 2.4?
Isn't this an important point of your paper as according to Section 2.4 this cube would be used to judge if an input is valid?
- You provide classifications of your latent code like "sunlight/brightness"? Where do you get this from? How would these latent codes
enter a formal auditing process? How would a user understand what the latent dimension which is certified means for him in practice?
- The experiment in Section 4.5 is not clear to me. It seems that for the smaller radius eps=0.5 more images can be attacked, than for the larger radius eps=1.5 (which includes the smaller radius).



 Dear reviewer,

Thank you for your review once again. We had updated the paper with reviewers' suggestions and responded to the points you raised in the review. We would be grateful if you could kindly let us know if there is anything else we can clarify for a revised positive assessment of our paper. 

Thanks,
Authors

 Thank you for your time. Yes, for the example that you referred to we have the case (2), where by adding pose variations the normal accuracy doesn't drop. This is because  auditing of the model with perturbations [lb, ub] ensures that the model is robust for this range of variation in pose, and when there are pose variations in the test set, the accuracy would be same as the general test with pose variations. 

In real applications, where the accuracy 95% is specified, yes, we would compute the lb and ub bounds, and evaluate on a particular diverse dataset, corresponding to the application domain the model is going to be deployed in - we can also generate a dataset for controlled variations (in poses for example) with the generative model that is a part of the framework (trained on diverse unsupervised data), and check whether the accuracy matches 95%.  

Thank you once again for your thoughtful review and discussion points.

 Thanks for your answers.
For the example (3rd point on your rebuttal), you mentioned 95%. Can you offer more information about this? 
I have two understandings: 
(1) For example, you have a model evaluated on a general test set, you can achieved 99%. By adding pose variations, does it  drop to 95% by computing the lb and ub with IBP? 
(2) You have a test set and you can achieve 95%. By adding different pose variations, you can still keep the accuracy 95% with computing perturbations [lb, ub].

In real application, I assume the accuracy 95% is given by the customers/researchers, then you compute the lb and ub using IBP. However, how did you achieve 95%? you specify one particular dataset and evaluate on that?

 We thank the reviewer for their very helpful suggestions and positive assessment of our paper. We are glad the reviewer found our paper valuable for society. Please find below our responses to the comments and questions

- Thank you for the comment about the joint encoder. Indeed it would not be possible to combine a pretrained network for example ResNet-50 directly in the current framework, but the architecture of the encoder can be any network that is trained with appropriate disentanglement loss for latent representations. 

- Yes, we limit the results to classification,  while leaving out other  tasks like detection and segmentation. We hope that our paper would serve as a good foundation for future work to explore auditing in other tasks. 

- Regarding the example for face angles varying within 30 degrees, for a particular $(x,y)$ pair, where $x$ is the input image and $y$ is the output label, the specification $F(x,y) = -f(x)f(x_{0^{\circ}}) \leq 0$ says that the classifier output would not be changed from the corresponding output of $x_{0^{\circ}}$, the face image of $x$ without any rotation. The set of all variations is $\{e_i(x)|\;\; \forall x \;\;|e_i(x)| < 30^{\circ} \}$. Now, the 95% accuracy will hold true when for 95% of the images uniformly sampled from this range, the condition $F(x,y)\leq 0$ holds. 

- Thank you for suggesting the inclusion of papers that use model confidence and uncertainty estimates for auditing. We have included a discussion of this in the related works now, where such approaches typically use calibrated uncertainty estimates to reject out-of-distribution samples at test-time. This notion is indeed very similar to auditing. 


 We thank the reviewer for their comments and feedback on our paper. Please find our response pointwise to the comments. We hope to clarify any potential misunderstandings the reviewer might have about the paper.

- We would like to kindly point out that our paper is not a direct extension of IBP and AdvMix [1]. In particular, we would like to emphasize that our approach specifically verifies for semantic changes in the input, that cannot be quantified easily by directly manipulating the image pixels. AdvMix still targets the adversarial robustness problem with a very different strategy using manifold mixup. So, the verification for AdvMix is in the image-space, while for AuditAI, since we have an encoder for encoding images to the latent representation of a generative model, the verification occurs in the latent space. As explained in section 2, verifying in the latent space effectively allows us to obtain verification bounds on much larger image-space variations compared to approaches that verify directly in the image space.

- We would like to clarify the differences with respect to [2]. First, the goals of [2] and AuditAI are very different: [2] aims to improve robustness against adversarial image corruptions, while we aim to audit deep learning models with semantic variations. This difference in goal also leads to different technical approaches: [2] optimizes through the image space with adversarial optimization, while avoiding verification through the image space is the key to the success of our approach. Second, since [2] models perturbation sets, images with such perturbations explicitly are necessary to train the Conditional VAE (CVAE). While, for AuditAI, we identify directions of latent variations that are present in pre-trained GANs, and as such do not require explicit supervision for training the generative model, and can scale to datasets like ImageNet and CheXpert, and high dimensional images like that in FFHQ.

- Regarding the latent space variations of a GAN, recent works have shown that we can obtain a wide range of semantic variations. Other works have shown how to locate these directions of semantic variations, such that manipulation of latent codes correspond to semantic variations in the input space  [a,b,c,d] . We make use of these advances for capturing semantic variations, and as better methods for GAN are developed in the future, they will still be applicable for incorporation with the AuditAI framework. 

- We used the term unit test with analogy to software engineering - where unit tests are used to determine the accuracy of computer programs and find bugs. We wanted to distinguish from standard verification protocols that do not verify with respect to semantic variations. We would be happy to change this term if the reviewer believes that a different term would be more suited to describe the method. -
We have incorporated the edits suggested in the revised paper. 

Kindly let us know if there is anything else we can clarify for a revised assessment of our paper. 

[a] Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021). Alias-free generative adversarial networks. arXiv preprint arXiv:2106.12423, 2(8).

[b] Yüksel, O. K., Simsar, E., Er, E. G., & Yanardag, P. (2021). LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions. arXiv preprint arXiv:2104.00820.

[c] Wang, T., Zhang, Y., Fan, Y., Wang, J. and Chen, Q., 2021. High-Fidelity GAN Inversion for Image Attribute Editing. arXiv preprint arXiv:2109.06590.

[d] Shen, Y., Yang, C., Tang, X., & Zhou, B. (2020). Interfacegan: Interpreting the disentangled face representation learned by gans. IEEE transactions on pattern analysis and machine intelligence.

 We thank the reviewer for their detailed positive assessment of our paper, and the very helpful suggestions and feedback. We have incorporated the edit suggestions and typo corrections the reviewer pointed out. Please find below our responses to specific comments

- Regarding the formal foundations, we have now mentioned in the text that $\gamma$ and $\delta$ are relative weights on the loss function terms. We have switched the order of paragraphs below, such that the definitions come up before the terms are used.

- In page 5, thank you for pointing out that the section heading should not be “Interpretability.” We have changed this to “Generative Model” to better align with the discussion of the generative model in that section.

- In section 4.5, we have re-framed the section heading to better explain the analysis in the section. Here, by “image certified by AuditAI” we meant the images for which auditing has determined that the specification on the output is satisfied. Our aim of this experiment was to do a sanity check for whether those images can be attacked in the pixel-space by adversarial corruptions.

- Regarding the human study, we would like to clarify that our aim was only to determine the effectiveness of the generative model that we used, and not to evaluate the AuditAI framework as a whole. Hence we were not concerned about utility for downstream machine learning datasets. Since previous works on generative models used such human studies for determining how realistic the generated images are (and hence get an estimate of the effectiveness of the generative model) [a,b], we chose a similar human study for this. We agree about the limitations of this, and would definitely incorporate a study of similar nature but with trained clinicians. 

- For the encoding function e(), any latent variable model that can map input images to the latent space of the generative model can be used. In particular, recent advances in GAN inversion for training very accurate encoders, and advances in increasingly realistic Variational Autoencoders [c,d] can be used for the encoding function e(). 

[a] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134, 2017.

[b] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European conference on computer vision, pp. 649–666. Springer, 2016.

[c] Vahdat, Arash, and Jan Kautz. "Nvae: A deep hierarchical variational autoencoder." arXiv preprint arXiv:2007.03898 (2020).

[d] Wang, T., Zhang, Y., Fan, Y., Wang, J. and Chen, Q., 2021. High-Fidelity GAN Inversion for Image Attribute Editing. arXiv preprint arXiv:2109.06590.





 We thank the reviewer for their detailed positive assessment of our paper, and the very helpful suggestions and feedback. Please find below our responses to specific comments

- Regarding the experiment to check that the AuditAI framework “works,” we had some sanity check results in section 4.5. We first obtained a set of images certified by AuditAI such that the specification is satisfied according to AuditAI for these images. We then checked what fraction of these certified images can be attacked in the pixel-space through adversarial perturbations. The results for this experiments showed that only about 9% of the images could be attacked in the pixel-space. We would like the reviewer to kindly let us know if they had any other specific experiment in mind for the toy setting, and we would be very happy to incorporate it in the paper.

- Thank you for the point about the assumption of being able to learn a disentangled latent space for the generative model, with semantically meaningful concepts. We have now emphasized this more in section 2.3 of the paper in the sections describing the latent space and the generative model loss.

- Regarding the human study, we would like to clarify that our aim was only to determine the effectiveness of the generative model that we used, and not to evaluate the AuditAI framework as a whole. Since previous works on generative models used such human studies for determining how realistic the generated images are (and hence get an estimate of the effectiveness of the generative model) [a,b], we chose a similar human study for this. We agree about the limitations of this, and would definitely incorporate a study of similar nature but with trained clinicians. 

- Thank you for pointing out that Theorem 1 should have some regularity assumptions on $f$. For application of IBP-based methods, we need the function to have bounded derivatives. We have now added this in the description of the theorem.

- We had mentioned details about IBP in the section A.1 (Appendix) of the original paper, as part of preliminaries. We hope this would be a good reference summary for IBP’s search procedure. 

- By “the corresponding convex bound of the variations of $\mathcal{S}^0_{i,in}$ in the image space  ” we mean given a set of variations in the latent space, what are the corresponding bounds for those variations translated to the image space through the generator of the generative model. While we do not need to do this for AuditAI, in section 2.2 we describe that this could be a potential way for visualizing the bound on the range of variations in the image space. 

[a] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134, 2017.

[b] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European conference on computer vision, pp. 649–666. Springer, 2016.


This paper proposes a new and useful direction: AI model audit. The proposed solution (framework) is very interesting. The experiments show the effectiveness of the proposed method. Pros
1. This paper proposes a new direction AI model audit. I think this is very useful direction, in particular, for industry. Rejecting making a decision (e.g. reject making decision for an image of bad quliaty and ask the user to re-submit a better image) is much better than making a wrong decision. 
2. The proposed solution is convincing for me and the experiments show the efficacy of the solution.

Cons
1. The classification model and the generative model share the same encoder. You cannot directly apply the proposed method to any exsiting network. For example, the classification backbone is ResNet-50. You cannot connect an encoder (for disentanglement learning) to this ResNet-50, or first few layers of ResNet50 works as the encoder.
2. The key theoritical barries are addressed by an off-the-shelf method, IBP.
3. The method cannot be applied to other tasks, e.g. object detection, segmentation, etc. At least, no supportive experiments for other tasks.

Questions and Suggestions
1. Can you translate the face rotation angels are within 30 degree and maintain 95% accuracy to the representation with  F(x,y) <= 0 and S_{i, in}.
2. In practice, many people use the uncertainty/quality/confidence score to do AI model audit. I would suggest to compare with them in Related Work section. Despite my concerns, I think this is a VERY valuable work for the society. The paper is well written and organized.

This paper presents, AuditAI, a system for verifying that a trained model meets a particular `specification', and also for certified training. The system uses a generative model to construct embeddings for inputs and then tests whether these embedding vectors belong to a particular set. Membership in the pre-specified set is reduced to satisfying the test. The key technical insight in this work is applying the interval bound propagation technique to embedding/latent codes in order to verify semantic behaviors of trained models. This paper tackles an important problem and provides a comprehensive discussion and evaluation of the approach. Certified training and verification is quite important for developing reliable models going forward. I detail key strengths, weaknesses, and additional questions below.


### Strengths
- Extending IBP to the latent code is straightforward and simple. 
- The breadth/scope that this work tackles is impressive. The paper shows how to verify a particular behavior like brightness, etc, and also how to train in a certified manner. 
- The scope of the empirical assessment in the work is also impressive. The paper tests on domain shift, ImageNet, Chest Xrays, and FFHQ. In each case, demonstrating convincing performance of the modified IBP.


### Weaknesses
- Toy setting with known ground truth. There is a simple experiment that the authors can run to provide a sanity check that their variation of IBP 'works'. Why not train a toy model that specifically violates a unit test of interest and show that the entire scheme here recovers the violation? 
- The principal assumption in this work is that the latent code and the corresponding generator can decompose the input into semantically meaningful concept. Even further, the assumption is that each dimension of the latent code only encodes for a single concept. Contemporary evidence shows that this assumption might not exactly be true. While this issue doesn't invalidate this work, I think the authors to make this assumption/requirement more explicit.
- User study. The study is interesting, but I am not sure what to take away from this part of the work. First, I don't think Turkers are quite equipped to judge Chest XRays. However, even if we assume that they are, showing indistinguishability between real and generated images does not demonstrate the effectiveness of AuditAI.
- I am confused that Theorem 1 does not have assumptions on $f$. It seems like what IBP does (not an expert on this technique) is to use a bound on the logits to bound the output of the network. Wouldn't this require assumptions on the activations or some form of bound on the Lipschitz constant of the function? 
- It is unclear to me how the search procedure in IBP actually works in practice. Can this information be added to the appendix?
- What does, "the corresponding convex bound of the variations of $\mathcal{S}^{0}_{i, in}$ in image space" mean? I lean towards an acceptance for this work since it provides a convincing demonstration of IBP for latent embeddings, which means more interesting properties beyond adversarial robustness on the input space can be verified.

This paper presents AuditAI that can do certified training, verification under semantic specifications based on a generative model. 
The authors extended IBP to latent space variations and evaluate on a wide range of datasets. 
 Strength:
- The paper is written clearly.

Weakness:
- The paper seems to be a natural extension of IBP, Advmix [1]
- I am not sure whether the distinction with [2] is clear (in appendix A.6). I think modeling perturbation sets for images with perturbations (like adversarial corruptions) explicitly can capture the specification better. 
- For the perturbation in the semantic space of a generative model, it is not clear to me whether this is a semantic preserving perturbation. Also it is not clear whether the latent for a particular GAN model can capture all the variations. Usually, the diversity of the GAN is quite limited. 
- I don’t find the unit test arguments to be very novel as it is a standard verification protocol. 
- The name AuditAI is not formal. 
- Minor: for few references, the use of citep, citet is mixed.
- Minor: theorem 1 “(“ should be removed.

[1] https://arxiv.org/abs/1912.03192

[2] https://openreview.net/pdf?id=MIDckA56aD
 Overall, the novelty of the paper is a bit limited and I am not sure whether the perturbation on the latent space of generative models indeed captures all the possible semantic specifications. 


This paper presents a framework for auditing deep learning models in regards to
a specification. The goal is to increase the confidence in a trained model. The
auditing is performed by generating variations in input data to evaluate the
boundaries in the input for an expected output. An evaluation was performed
against a pixel-based training approach and under consideration of 4 different
datasets. ## Major Issues
* At times the formal foundation is not precisely specified so a detailed
   understanding is at times difficult, e.g.:
    * Page 4: $L = L_{task} + ɣ L_{spec} + δ L_{gen}$ -> gamma and delta not
      explained
    * Page 4: Variables used in paragraph "Task Performance" are only introduced
      in the next paragraph "Verifiablity" (e.g. $z_K$, $y_{true}$)
* Page 5: Not clear how the paragraph "Interpretability" which discusses
   $L_{gen} is related to interpretability.
* Page 8; Section 4.5: Authors are talking about "certified images"; so far the
   certification was about the training not individual images, the intend of
   this section is not clear; also is not a robustness against adversarial
   attacks also a property which can be seen as a variation which is the target
   of the framework.
* Page 8; Section 5: The authors perform a human study to evaluate whether the
   generated images are realistic for the human eye. However I am unsure whether
   this is a good metric for evaluation since realistic images for the human eye
   are known to not be equivalent to useful images for machine learning datasets
   (c.f.g. adversarial attacks). Also the participants of the study are of
   unknown expertise and the chosen dataset was that of Chest-X-ray images.

## Discussion
The paper contributes to a solution of a problem that is both relevant and in
scope of the conference. The authors motivate the paper and clearly state the
contributions as well as the followed approach. While some rework of the formal
foundation (see `Points Against: 3.) would be good to increase understandability
they are sound and allow a deep understanding of the approach and also convince
the reader that the approach is well thought out. However the usage of a
generative model to create the inputs to check boundaries does limit the
confidence in the framework since the reliability of the generative network is
now at question. Which means the reliability of the framework depends again on
another machine learning component. The evaluation of this component, the
generative network, was done by performing a human study about how realistic the
images are for the human eye and was also performed by participants of unknown
expertise. The specification of the encoding function $e()$ is not clear aside
from the used example of the change in degree and should be discussed further,
as this is also an important aspect in the framework. Without clear
specification of $e()$ and assurance that this in fact represents the variation
which needs to be audited the result of audit is questionable. Non-the-less the
contribution is worth publishing as this represents a sound foundation for
future research.

## Minor issues
* Figure 1: The mapping of the three parts of the figure to the explanation
  could be improved.
* No structural overview of the paper at the end of the introduction.
* Page 2; last paragraph: "… 25%larger …" -> space missing
* across the whole paper: "upto" -> up to
* Page 5; first paragraph: "tigher" -> tighter
* Page 7; Table 4 is referenced before Table 3 in the text, so these two tables
  should be swapped in the tex document.
* Page 8 and 9: "AuditAI" is not in monospace font
* the authors at times claim to certify, however this is in my opinion to
  strong, most of the time they speak of auditing which is the better
  description ## Points in Favor
+ Relevant problem and in scope of the conference.
+ Presentation of a framework for auditing deep learning models in respect to
   given specification.
+ Formally sound foundation of the framework and its mathematical workings.
+ Generally the introduction of the framework is given in stages with
   relaxing assumptions, so the understandability is good.
+ Experiments show the improvement with the framework.

## Points Against
- Generative Network used to create the variations: while this strengthens the
   confidence in the model it does move the reliability of the framework to
   the GAN as well.
- The framework is dependent on the encoding function e() and it is not clear
   whether this is always specifiable.
- Partly, the formal foundation is not precisely specified

Meta-review: The paper discusses the auditing of AI systems which is clearly a very important topic. The approach targets mainly image classification systems where the main idea is to do the certification not in pixel space but in a latent space of a generative model. For the certification the authors use the existing CROWN-IBP for cubes around test points in latent space. The authors report results on several datasets using existing methods for the generative models.

Most of the reviewers liked this work even though all of them also raised quite similar concerns. The positive reviews were to some extent based on the fact that auditing of AI systems is a very important problem - and I completely agree on this. On the other hand auditing of AI systems, in particular of safety critical systems as discussed with the chest x-ray application in this paper, is a serious problem where a paper has to fulfill very high standards. In particular, being transparent about its limitations is crucial which in my opinion does not hold for the current version of this paper.

My main conerns are:
- the motivating example (varying the angle from which a face is seen by less than 30 degress) is misleading as such a precise threat model is not feasible with the generative models discussed later on (see last bullet point)
- there is no discussion on the potential trade-off regarding test accuracy training the classifier i) in latent space, ii) with CROWN-IBP using a rather shallow model. This is important to judge the utility of this approach - it has to be transparent what the trade-off is regarding test accuracy and what kind of price one has to pay in terms of test accuracy for the certificates.
- the deployment check discussed in Section 2.4 is nowhere implemented as later on only boxes in latent space around individual test points are checked but not a fixed box
- the main problem connected to what I mentioned above with the motivating example is that the latent dimensions have to be interpreted by humans. I doubt that this can be made fully transparent to humans what such a certificate in latent space meand and thus to give very precise labels to the latent dimensions like "brightness" is in my point of view misleading. It suggests a much more finegrained semantic control than what is available and to use such inprecise and potentially wrong ``labels'' is highly problematic for an auditing process as e.g. this paper does not contain any guarantee against "brightness"-changes. For safety critical systems this is an important distinction.

As a minor point there is little technical novelty (certification with CROWN-IBP is done with an existing library) but this plays no role in the decision.

In total the paper is not ready yet for publication. The authors should revise their manuscript by addressing the points above, being fully transparent about what the paper can do and what it cannot do. I suggest to do less examples but provide more details on how the authors envision the auditing/certification process for the given application, in particular also with the discussion about rejection of inputs not lying in certified areas (which has not been done in the present paper).

Decision: Reject