Paper: Trustless Audits without Revealing Data or Models (ICLR-2024)

Reviewer comment (Writing):
The description of ZK-SNARKs in Section 2 is also a bit too concise for a reader that is not already familiar with this concept. A few concrete examples of what x and w could be, in particular within the context of privacy-preserving machine learning, would help.

Author response:
We have incorporated your suggestions on the presentation of ZK-SNARKs into the updated draft. Due to the limited space, we present an extended discussion in the Appendix, including four examples of using AIRs and further discussion on ZK-SNARKs.

Reviewer comment (Theory):
The limit on which functions can (or cannot) be audited with the proposed approach is not clear. For instance, it is not clear if the proposed approach could be used to prove fairness properties about the model.

Author response:
In principle, any computable function can be used as the audit function. Fairness properties can easily be checked with the same overhead as the copyright audit. We have updated the draft to clarify this point (Section 6).

Reviewer comment (Experiments):
The experiments are conducted on a limited number of datasets and architectures, and thus it is not clear how the approach would scale. Additional experiments with classical datasets such as MNIST and CIFAR and architectures such as ResNet are suggested.

Author response:
We have conducted additional experiments on CIFAR-10 and MNIST and have updated the draft to address your concerns.  
We compared our method to semi-honest MPC published this year [1]. Note that this semi-honest MPC is not secure against malicious adversaries. We computed the proving times and costs for the semi-honest MPC and ZkAudit. For the semi-honest MPC, we _only_ measured the egress fees. 

CIFAR-10 HiNet

[1]: 100,827 minutes, 5,280,375 GB, 475,234 dollars

ZkAudit: 113,311 minutes | 2,417 dollars

MNIST FFFNN

[1]: 626 minutes, 30,820 GB, 2773.8 dollars

ZkAudit: 13,226 minutes, 282 dollars

Our method is up to 196x cheaper than secure MPC and takes approximately the same amount of time to prove on CIFAR10. Although MNIST is slower, it is substantially cheaper.

[1]  https://www.usenix.org/system/files/sec23fall-prepub-212-rathee.pdf

Reviewer comment (Evaluation):
The cost should also be reported in terms of computational time in addition to the monetary cost to be more meaningful.

Author response:
We compared our method to semi-honest MPC published this year [1]. Note that this semi-honest MPC is not secure against malicious adversaries. We computed the proving times and costs for the semi-honest MPC and ZkAudit. For the semi-honest MPC, we _only_ measured the egress fees. 

CIFAR-10 HiNet

[1]: 100,827 minutes, 5,280,375 GB, 475,234 dollars

ZkAudit: 113,311 minutes | 2,417 dollars

MNIST FFFNN

[1]: 626 minutes, 30,820 GB, 2773.8 dollars

ZkAudit: 13,226 minutes, 282 dollars

Our method is up to 196x cheaper than secure MPC and takes approximately the same amount of time to prove on CIFAR10. Although MNIST is slower, it is substantially cheaper.

[1]  https://www.usenix.org/system/files/sec23fall-prepub-212-rathee.pdf

Reviewer comment (Writing):
The notion of trust should be more specifically defined in the paper, in particular as the focus of the paper is to argue that the proposed approach is « trustless ».

Author response:
In this work, trust means that a third party must assume the model provider acts according to the protocol. A malicious adversary may deviate from the protocol. Under mild cryptographic assumptions, ZkAudit does not require the model provider to act honestly as the model provider will be caught if attempting to deviate from the protocol.

Reviewer comment (Writing):
How does a traversal ordering attack work?

Author response:
A traversal ordering attack allows a trainer to change the model outputs in undetectable ways by simply changing the order the data is seen at training time [1].

[1] https://arxiv.org/abs/2204.06974

Reviewer comment (Evaluation):
What are the test accuracies obtained for the different conducted experiments?

Author response:
All accuracies reported are the _test_ accuracy after the training procedure. We have clarified this in the draft (Section 5).

Reviewer comment (Theory):
How does the use of salt to prevent dictionary attacks when hashing the weights combine with the ZK-SNARKS?

Author response:
Denote the weights of the model as W. A salt appends _random_ bits R to W. Then, under mild assumptions on hash functions, H(W || R) is not susceptible to dictionary attacks, since the padding R is random (so the output of the hash will appear random).

Reviewer comment (Experiments):
The paper shows improvements in cost and performance over existing methods, but there is no comparison to non ZK-based methods.

Author response:
_No_ prior work for ZK-SNARKs for ML can perform training, since no prior work can compute the softmax function. We are the first method to be able to perform training on modern DNNs.

To compare the cost of prior work, we removed the softmax and estimated the time for the remainder of the operations. Even without the softmax, the prior work is up to 94x costlier than our proposed method.

Reviewer comment (Experiments):
The paper could be improved by showing how far away ZkAudit is from the cost/time it would take if the auditors had access to the model and dataset or even if they were in an SMPC setting as the authors cite early in the paper.

Author response:
_Secure_ MPC is unfortunately slow and expensive. For example, work published this year on semi-honest, secure MPC, which is not even secure against malicious adversaries, takes _100,827 minutes_ and _5,280,375 gigabytes_ of communication to train _on epoch_ on CIFAR10 [1]. The data egress fees alone would cost _$475,233.75_.

We took the models in [1] and compared the total cost of _just_ data egress fees compared the cost of computing the ZK-SNARK proofs for CIFAR10.

CIFAR-10 HiNet

[1]: 100,827 minutes, 5,280,375 GB, 475,234 dollars

ZkAudit: 113,311 minutes | 2,417 dollars

MNIST FFFNN

[1]: 626 minutes, 30,820 GB, 2773.8 dollars

ZkAudit: 13,226 minutes, 282 dollars

Our method is up to 196x cheaper than secure MPC and takes approximately the same amount of time to prove on CIFAR10. Although MNIST is slower, it is substantially cheaper.

[1] https://www.usenix.org/system/files/sec23fall-prepub-212-rathee.pdf

Reviewer comment (Evaluation):
While it is expected that ZkAudit will be slower or more costly than these other scenarios, it would be useful to know by how much — for example, is it several orders of magnitude?

Author response:
_Secure_ MPC is unfortunately slow and expensive. For example, work published this year on semi-honest, secure MPC, which is not even secure against malicious adversaries, takes _100,827 minutes_ and _5,280,375 gigabytes_ of communication to train _on epoch_ on CIFAR10 [1]. The data egress fees alone would cost _$475,233.75_.

We took the models in [1] and compared the total cost of _just_ data egress fees compared the cost of computing the ZK-SNARK proofs for CIFAR10.

CIFAR-10 HiNet

[1]: 100,827 minutes, 5,280,375 GB, 475,234 dollars

ZkAudit: 113,311 minutes | 2,417 dollars

MNIST FFFNN

[1]: 626 minutes, 30,820 GB, 2773.8 dollars

ZkAudit: 13,226 minutes, 282 dollars

Our method is up to 196x cheaper than secure MPC and takes approximately the same amount of time to prove on CIFAR10. Although MNIST is slower, it is substantially cheaper.

[1] https://www.usenix.org/system/files/sec23fall-prepub-212-rathee.pdf

Reviewer comment (Novelty):
Novelty/contribution may be limited (or unclearly stated). Backwards pass work includes numerical representation adjustments and a better implementation of softmax. The structure of the paper thus makes it look like these two bits of work were the missing pieces of zk-SNARK work that prevented them from being realistically applied to SGD. These are presumed to be "optimizations" as in the related work statement:  

"All of the work we are aware of focuses on optimizing inference, so all of this work omits the softmax computation and does not optimize the backward pass. In this work, we leverage ideas from inference to optimize the forward pass but show how to compute full SGD in ZK-SNARKs and optimize the softmax."  

This suggests that while other works can perform SGD, its backwards pass, and/or softmax, but just not efficiently. The title of the work and other statements, however, suggests that this is the first work that can handle these computations at all.  

Further, the contributions in the numerical representations do not seem significant and softmax may be even omitted from model training (or inference) without significant changes to training or inference. There thus appears some incongruity with the stated contribution and methods achieving those contributions.  

Suggestion: clarify whether the work presents the first verification systems that includes data/model training and that to achieve this within the frameworks from prior works, only changes to numerical representation (and softmax) were needed. Alternatively, if there was more necessary to achieve the goal, describe them in more detail. Alternatively, it may be that prior works already achieved the goal with less performant system in which case the title of the work and contributions need to be clarified. In this last case, comparisons of accuracy and efficiency need to be provided (as in Figures 1, 2).

Author response:
## Requiring the softmax

All methods we are aware of for doing multi-class classification require performing a complex non-linearity such as a softmax or multi-class hinge. Thus, _no_ prior work is able to perform training. For example, it is unclear how to perform even the exponential function in the prior work for ZK-SNARKs for ML.

Inference can be done by removing the softmax layer and taking the argmax of the logits instead, which is what prior work does. ZkAudit-I leverages prior work. We have made these points more clear in the paper.

Reviewer comment (Writing):
Model parameters are not revealed to verifiers but model architectures are. This is noted but the abstract is unqualified; please adjust the abstract to make this clear.

Author response:
1. We have updated the abstract to mention model architectures are revealed.

Reviewer comment (Evaluation):
Some tabular results do not state which operation they are measuring and whether it is of a single instance of that operation (for inference) or multiple/entire dataset. Please include this.

Author response:
2. We updated table captions to mention if the results were for a single operation or for the entire dataset.

Reviewer comment (Experiments):
For prior techniques which do not handle softmax at all or well, consider including versions of models without softmax alongside the softmax ones. This does not mean merely using an existing softmax implementation as done in Section 5.3, but instead removing the entire softmax operation from the model.

Author response:
3. We do compare our work against prior work estimates when removing the softmax. However, training would not work without the softmax.

Reviewer comment (Presentation):
Table 3: include units in table or description.

Author response:
4. We have included units in Table 3.

Reviewer comment (Evaluation):
Some results that depend on dataset size could be better presented in terms of cost per instance (Copyright audit is one example). The currently noted "$108" is not.

Author response:
5. We have clarified how the audits scale with the size of the dataset.

Reviewer comment (Writing):
Please include a bit more on how operations are implemented via AIR in its background section.

Author response:
6. We have included more information about how operations are implemented within the AIR in the background section.

Reviewer comment (Novelty):
Question B — How does this work compare: *Efficient Representation of Numerical Optimization Problems for {SNARKs}*, Angel et al. 2022. I also see several papers in federated learning that leverage SNARKs for enhancing trust. If models involving softmax or similar steps to the presently-experimented ones were used, I imagine they would have similar accuracy problems. Are their solutions applicable?

Author response:
## Comparison to Angel et al (2022)

The “numerical optimization problems” Angel et al (2022) address are constrained optimizations via linear programming and semidefinite programming. Neither of these is applicable to DNN training, which we focus on. In fact, Angel et al do not show how to perform the forward pass or backward pass for DNNs. We have included a discussion in the related work section.

Reviewer comment (Experiments):
Question C — Can the numerical representation work described in the paper be applied to other verified computation problems beyond model training?

Author response:
## Does our work apply to other verified computation problems?

Yes, our work applies to other verified computation problems. Our work is most applicable to optimization problems that need to perform complex non-linear functions as part of their optimization (e.g., the softmax, exponentials).