Optimistic Verifiable Training by Controlling Hardware Nondeterminism
Authors: Megha Srivastava
Simran Arora
Dan Boneh
Venue: NeurIPS 2024 poster (2024)
Keywords: auditing
training
security
verification
safety
hardware
nondeterminism
Primary area: safety_in_machine_learning
TL;DR: A method for controlling GPU nondeterminism to enable exact verification of large-scale model training services.

Abstract: The increasing compute demands of AI systems has led to the emergence of services that train models on behalf of clients lacking necessary resources. However, ensuring correctness of training and guarding against potential training-time attacks, such as data poisoning and backdoors, poses challenges. Existing works on verifiable training largely fall into two classes: proof-based systems, which can be difficult to scale, and ``optimistic'' methods that consider a trusted third-party auditor who replicates the training process. A key challenge with the latter is that hardware nondeterminism between GPU types during training prevents an auditor from replicating the training process exactly, and such schemes are therefore non-robust. We propose a method that combines training in a higher precision than the target model, rounding after intermediate computation steps, and storing rounding decisions based on an adaptive thresholding procedure, to successfully control for  nondeterminism. Across three different NVIDIA GPUs (A40, Titan XP, RTX 2080 Ti), we achieve exact training replication at FP32 precision for both full-training and fine-tuning of ResNet-50 (23M) and GPT-2 (117M) models. Our  verifiable training scheme significantly decreases the storage and time costs compared to proof-based systems.

Optimistic Verifiable Training by Controlling Hardware Nondeterminism
Megha Srivastava" Department of Computer Science Stanford University edu
Simran Arora
Department of Computer Science
Stanford University s simarora@stante edu
megha@cs
stanford Seetha@cssstanforrooo
Dan Boneh
Department of Computer Science
Stanford University dabo@cs. stanford edu
Abstract
The increasing compute demands of AI systems have led to the emergence services that train models behalf of clients lacking necessary resources. How- ever, ensuring correctness of training and guarding against potential training-time attacks, such as data poisoning and backdoors, poses challenges. Existing works verifiable training largely fall into two classes: proof-based systems, which are dif- fieult scale, and "optimistic" methods that consider third-party auditor who replicate the training process and contest the trainer. A key challenge with the latter is that nondeterminism between GPU types during training prevents exact replica- tion of the training process, resulting in schemes that are non-robust. We propose a method that combines training in higher precision than the target, rounding after intermediate computations, and sharing rounding decisions based on adaptive thresholding procedure, successfully control for nondeterminism. Across three different NVIDIA GPUs (A40, Titan XP, RTX 2080 Ti), we achieve exact training replication at FP32 precision for both full-training and fine-tuning of ResNet-50 (23M) and GPT-2 (117M) models. Our verifiable training scheme significantly de- creases the storage and time costs compared to proof- based systems, and is publicly released http:///hthhhboooom/eaaaa//eeeep
training
1 Introduction
We are currently in the "large-scale era" of machine learning (ML), where the exciting capabilities of modern AI systems have required dramatic increase in training compute needs (Sevilla 2022 In turn, several model training services, such as Replicate, OpenAl's Finetuning API, Together AI, Amazon Sagemaker, MosaicML Training, and Gensyn, have been created to support clients who lack the resources to train a model themselves. However, these services require clients to place significant degree of trust in them train the model correctly. without introducing a training- -ime attack such as data poisoning or undetectable backdoors (Wan et al.) 2023 Goldwasser et al. 2022 How can we help a client, such as an individual or a small company, hold the service provider accountable in case of mishehavior during training?
Consider an education start-up that wishes to finetune the Llama-70b language model (70B parame- ters) on their own curated dataset to support student learning. This task requires significant resources,
CCoresponnence to imegaaccs...... stanford.....m steghafcs.stanfordd edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and the company might even lack the necessary expertise. Instead, they might choose to pay trainer with vast computing resources perform the training task (Figure However, what if the trainer adds data points that spread misinformation, introduces backdoor that advances political agenda for specific prompts, or tries to save work by under-training the model? If the client starts to notice suspicious model behavior, is there any action they can take? We study this problem of verifiable training, or ensuring that the training of an ML model was performed correctly.
One possibility is for the trainer to provide the client with cryptographic proof that the model was trained according to the specification. How- ever, proof- broee- systems require cryptographic techniques that can be difficult to scale to the complexity of real-world MI. systems. For in- stance, recent work based on eero-lnoo proof systems for verifiable inference, a much simpler task than training, requires more than 8 minutes to generate proofs for only 20 images [Liu et al. 202T]. Thus, practical proof-based methods for verifiable training have only been implemented for simple tasks such as logistic and linear regression [Garg et al.) 2023 Ames
[1) share data
Client
share data
Malicious Trainer
Auditor
(4) share randomness, architecture, and rounding
ITT33
bcys
[5) verification gamee
abe3
s0F4
3ab8
k13r
5d23 2poen
d23f
dfgg
cbp2
Epoch
Spoch
et al. 2022]
training time attack (e data palsoning)
auditor disputes trainer
An alternative "optimistic" approach to con- sider a third-pary auditor [F This could be a trusted 3rd party, such as a non-profit orga- nization that may not have sufficient computing resources to provide training as a service be- yond auditing, or a different provider that the client approaches and wishes to compare with the original model trainer. When client sus- pects foul play, they can ask the auditor to chal- lenge the trainer by training the model using the auditor's own compute, and demonstrate that the trainer did not train correctly. Based on the provided evidence required from the auditor (i.e. the precise timesteps model training diverged, as shown in Figure 1), the client can then choose refuse the trainer's model, pursue legal action against the trainer, or even dispute a potentially corrupt auditor if the client deems such evidence as incorrect, or another auditor disagrees. This protocol can be efficiently carried out using techniques from the literature on verifiable computing. such as the "verification game" method of Teutsch and ReitwieBner (2019). which uses an interactive binary-search procedure identify the exact intermediate computation step (c.g. training epoch) where the two parties diverged. Applying verifiable computation techniques to model training is particularly important given the increase in decentralized machine learning services like Gensyn, which seek to make MI. compute more accessible by creating network of many untrusted GPUs.
Figure 1: Overview of our scheme. After an auditor challenges a trainer, they train the model, storing weights Merkle tree, and enter a binary search procedure to identify the exact steps of the dispute. We show how control GPU nondeter- minism between auditor and trainer, expanding the set of potential auditors.
Unfortunately, the issue with such "optimistic" approaches is nondeterminism during model training: two models trained on different GPU types, even with same data order and random seed, can learn different weights (Figure 2). The auditor cannot simply compare their model weights with the trainer's. 's, and recent work has shown that protocols based on comparing model weights, such as Dia al. 2021 's "proof of learning." are not robust and can be forged due to errors from nondeterminism Thudi et al. 2022 Fang et al. 2023].
Our work addresses this limitation by asking: the trainer provide additional information the auditor that removes the effects of hardware nondeterminism? Our starting point is the observation that hardware nondeterminism occurs due to the accumulation of errors from floating point operations. For example, a matrix-vector multiply often results in different floating point values on different GPUs, since GPUs often accumulate in different orders. To address this issue, natural approach is to perform training using higher precision (e.g. FP32) than the target precision of the model weights (e.g. FP16), and periodically round back to the target precision. The hope is that all floating point errors will be confined to the higher precision bits, so that the rounded values are deterministic. However, this fails because computed values can occasionally straddle the "rounding boundary": i.c.

---

the trainer can round up while the auditor rounds down, quickly causing them to diverge. Instead, we propose solution where the trainer records the rounding direction for certain intermediate computation so that auditor can stay in syne with the trainer. As this requires the trainer to record a large number of bits, we also show how to reduce the amount of data needed to eliminate errors.
We use this strategy to adapt the verification game described by Teutsch and ReitwiefSner (2019) for verifiable training. The game's 's efficiency lies in our ability to store hashes of model checkpoints in a Merkle tree [Merkle, 1988]. To determine if training was performed according to the specification, the auditor needs to reconstruct the Merkle tree and compare the resulting Merkle root hash with the Merkle root hash provided by the trainer's 's- - if they do not match, the two parties can enter an interactive binary search procedure to identify the exact training step of the dispute. The purpose of the binary search game is to hold both parties accountable: an auditor should not be able to simply claim that model was improperly trained, but convince a third-party (e.g., the public, or a judge) by showing at what point during training the trainer mishehaved. We show our verifiable training scheme can scale to tasks such as full training of ResNet-50 (23M parameters) and finetuning of GPT-2 (1 (17MM parameters), significantly outperforming existing methods with respect to both time and storage cost, while eliminating statistical error due to non-determinism. For example, the proposal in prior work Jia et al. [20T]] would require > 140x x more storage cost than our method by comparing model weights at every step in order to achieve low (yet still non-zero) statistical error.
Concretely, our contributions include: (1) A method for two parties, training the same model on different GPU types, to achieve identical training results by logging and sharing rounding decisions; (2) A verifiable training scheme based on the verification game from Teutsch and Reitwiešner (201)). which stores model weights a Merkle tree for efficient comparison between a trainer and auditor; (3) Experiments showing the ability of our approach to scale to large models such as 5es and GPT-2 between three different NVIDIA GPU architectures (A40, Titan XP, RTX 2080 Ti); (4) Methods to reduce the storage cost of our approach via efficient encoding of rounding logs and an adaptive threshold mechanism to reduce the amount of rounding decisions logged; and (5) Comparisons with existing methods, including proof-based systems, that highlight the improved storage and time efficiency of our method.
2 Related Works
Without any verifiable training scheme in place, significant trust placed in the trainer, leaving a client vulnerable to many different attacks, such as the "poisoning" of data samples to cause
undesirable behavior (e.g.. generating unsafe text [Carlini al. 2023 Koh et al.) 2021 Wan et al..
2023)) and planting backdoors triggered by certain inputs [Goldwasser et al.) 2022 Therefore, training ML models in trusted environments has been an exciting direction explored by many researchers. One line of work consists of proof- based systems, where a proof of correctness (for a desired specification) is provided using cryptographic techniques such as succinct non -nterrttee
arguments (S)ARKs) [Micali 1994 Biitansy et al.] 2012 Lee et al. 2020 Liu et al. 2222 Garg
al., 2023, Kang et al. 2022 However, even the most recent proof based systems for verifiable
training suffer extreme latencies, such as 22 minutes for training VGG-11 1 on one batch of 16 data inputs (Abbaszadeh et al., 2024), and have therefore primarily been developed for simpler models (e.g., logistic regression) that are less likely to be delegated to others in the first place [Garg et al.)
2023 Ames et al.) 2022]. Meanwhile, an alternative solution of training models in trusted execution environment (TEE), such as NVIDIA's H100, incurs a performance penalty due to the cost of running inside a TEE Dhanuskodi et al. 2023]. Furthermore, clients lose all security guarantees if an attacker can extract the attestation key from even one GPU Niisson et al. 2020 Bulck et al.) 20018).
Our approach is most similar to proof-of-learning protocols, which consider a trusted 3rd party that compares checkpointing during the course of training with the original training sequence (Jia et al.) 2021]. However, such methods not only incur high storage cost by requiring model weights to be stored frequently, but are non-robust due to errors from training nondeterminism. Several works have shown that proof-of-learning protocols can be spoofed and fail to verify correctness in several important contexts [Fagg et al. 2023 Kong et al.) 2023 Thudi et al. 2022 Although CCoi et al. 2023 recently proposed a verification procedure that is immune to several known proof-of-learning
Our method is implemented entirely within the pytorch framework (compatible with version 2.3.1), and is available at/https:/ h/ //gi /gtgtttttuuuuaeo www.////ggaabbt///eeeeaaaaaaaaaa training
3

---

Test Input
Accuracy
90.2 %
89.9 %
90.7 %
Test Input
Perplexity
4.18 ppl
4.18 ppl
4.22 ppl
Scores
A40
TitanXP
RTX T1
This above all:
Scores
A40
TitanXP
RTX Ti
thine own self
dog
0.09
0.26
0.75
true 0.0101
0.0091
0.0096
cal
0.69
0.28
0.20
my
0.0098
0.0100
0.0083
deer
0.08
0.29
0.01
thou
0.0097
0.0095
0.0106
[a] CIFAR-1G Classification [Res.Net
(b) Shakespeare Text Finetuning (GPT-2)
Even after ensuring the same software version, random seed, and use of deterministic algorithms via library flags, training nondeterminism persists between three GPU types.
Figure 2:
attacks, their method is not only limited to supervised learning algorithms, but also based an assumption that models temporarily overfit data during training. which may not always hold true.
GPU Nondeterminism: Prior work has investigated software patches for deterministic training.
for instance by enforcing FP accumulation ordering, at a significant cost to efficiency Jooybar et al.
[2033] Defour and Collange| [2015) Chou et al. [2020), TensorFlow [202T] Zhuang al. [2021].
While these options address deterministic computation on a single GPU architecture, achieving deterministic results across multiple GPU architectures remains challenging Crane [2018a], NVIDIA
[2022]. We control hardware nondeterminism across GPUs in order to design an efficient and reliable verifiable training scheme. However, our method's impact extends beyond verifiable training, as training nondeterminism can have several negative consequences including bias, reproducibility, and downstream effects on ML pipelines (Zhuang et al. 2021 Crane 2018b
Srivastava et al. 2020].
3 Set-Up: The Verification Game
Our method for verifiable training is based on the interactive verification game proposed by Teutsch and Reitwießner [009 in the context of blockchains. The core idea is to resolve dispute between challenger, in our case the auditor, and a solver, in our case the trainer, for an expensive computation (e.g., model training). In order for the auditor to take any meaningful action (e.g., pursue legal action), they need to prove the exact source of the dispute (c.g., training time-step where an attack occurred). If we can model weights at different time steps into compact data structure such as a Merkle tree, then identifying the source of disagreement can be done efficiently using binary search [Merkle 1988). More precisely, the verification game consists of the following parties:
1. trainer, who has putatively trained model according to a client's specifications. In our example, this is service provider with sufficient compute power to train model. client, who receives a model from the trainer and approaches an auditor.
3. auditor, who officially challenges the trainer behalf of client. This is trusted 3rd- ard-pprtt that has sufficient resources but does not necessarily provide training as service. The client can choose several auditors to audit the trainer's model.
judge: Sometimes a judge may need to arbitrate a legal claim. The judge can only perform minimal computations (e.g.,
training epoch), but examine the auditor's claims and enforce penalty against either the trainor, for incorrect training, or the auditor, for a false alarm.
When the trainer is approached by auditor, they would need to share training parameters, model architecture, and randomness, as shown in Figure The auditor would then replicate the training process, storing model weights in Merkle tree at the same checkpointing interval as the trainer (every leaf node a Merkle tree is hash of the data and every non-leaf node is a hash of its children). The main loop of the verification game starts when both parties have the root of their respective Merkle trees. If training was performed correctly, then the trainer's 's root should match the auditor's s. Otherwise, binary search procedure is performed, where the auditor iteratively descends the Merkle tree until it identifies two consecutive leaf nodes, and + 1, where the hash at matches that of the trainer, but the hash at leaf i + does not. This identifies the point in the computation of the dispute.
This interactive verification game requires the cooperation of the trainer. If the trainer refuses to share the value at certain node of their Merkle tree within given time frame, they can be considered to have failed the audit. Additionally, the trainer and auditor use a Merkle tree to store model weights, requiring far less storage than prior work, if correct training produces identical weights (and

---

Example
Sum Order
F1332
Rounded to FP16
0, 6 0.1, 0.2
a+b b
0011 10000011111111111111111111111111 1001111
001 1001001 1001 1O
a + c +
0011 1001001 (001 1001 10011001111111011001011111111111 10
001 100 0D1 1001 1O
01000001 1011 1000
01001 1011 1011001
b, 10.02, 1662111188615008,
+
01000001 1011 10001 (1XXXXXXXI0
01001101110 100010111011000
Table 1:
Two examples of floating point accumulation error when rounding arithmetic performed higher precision (e.g. FP32) down to lower precision (e.g. FP16). In the second example, the error in the FP32 result transfers to the rounded FP16 result.
identical hash values). The problem that training nondeterminism leads to weight divergence, and causes this verification game to always fail. This why we seek to prevent divergence in training.
4 The Nondeterminism Challenge
Although there are user-side controls for forcing deterministic operations within a single GPU architecture, , these controls do not prevent nondeterminism between GPU architectures (e.g.. NVIDIA H100 and V100), where trained models can have similar aggregate performance (e.g., accuracy) yet
yield very different predictions, shown in Figure 2 Crane 2018a] NVIDIA 2022 There are three main sources of nondeterminism between GPU
types:
1. Floating- Point Arithmetic: Computers represent real values using integer and FP representa- tions, typically the IEEE 754 standard (Figure (5. There is a tradcoff between the approximation fidelity and the # of bits used represent the real values. The chosen precision controls the repre- sentable numerical range (e.g., 32-bit FP values can represent values between 1.17549443e 38 and 3.40082347c + 38). Because computers round to representable FP values, changing the order in which FP values are accumulated can change the resulting sum Kahan [965]. Whitehead and Fit-Florea [01 Over the course of the many operations during training, this can lead large difference in the end result between the trainer and auditor.
2. Parallel Computation: In GPU, single operation (called kernel) is executed by thousands of threads in parallel. GPUs contain a set of streaming multiprocessors (SMs), which run the thread blocks required for the kernel. At the hardware level, these blocks are divided into warps that are assigned to the available cores. Because different GPUs have a different number and size of compute units, applications partition arithmetic workloads (e.g., batch matrix multiplies) differently to achieve high performance NVIDIA [022], thus changing the order of FP operations.
The time taken for memory access by each thread depends on the physical location of the data, which can create variable delays Jooybarr et al. (0013), Defour and Collange| [2015). Chou et al. [2020]. The GPU memory hierarchy consists of large
3. Memory Hierarchy and Variable Delays:
amounts of high bandwidth memory (HBM) and small amounts of fast SRAM memory, and maintains an L1 and L2 cache to improve access times. The caches sizes and access times differ across GPU architectures (c.g. an NVIDIA A100 has 192KB /0 MB of L1/L2 cache memory, while the H100 has 256KB/ / 50MB). This affects warp scheduling, leading to changes in operation ordering resulting in nondeterminism. Finally, to compute primitives such as GEMMs (D = A + C), the workhorse of machine learning, GPUs split the work of computing the tiles of D across thread block NVIDIA 2023 resulting in nondeterminism that a robust verification method needs to control.
5 Method Overview
5.1
Accumulation Errors Start at Higher Precision Bits
Our key idea is that if nondeterminism of training between GPU types occurs due to FP operations, then any error will initially be introduced the lower bits. Suppose that both trainer and auditor train at a higher FP (e.g., byr = 64) precision than the client's target model precision (e.g., bn = 32) and then periodically round (e.g., b, = 32) after intermediate computation steps (e.g., a convolution layer). One might hope that this will "erase" the errors due to nondeterminism, and prevent them from accumulating. Unfortunately, simply rounding to the nearest FP32 after each computation during training is insufficient for determinism. The problem is due to rounding errors that straddle
5

---

1og:0
1og12
Case A
Beata2
1og:1
2og:1
putput (ofas54)
Case B
Nozt82
1ost22
GPU output
(flen664)
Log:0
Rounding
Case
16x1132
103132
Hoar32
1og=1
7
Idstermines
Figure 3: Divergence between outputs two different GPUs (in FP64) for given function and input can result in different rounding choices when rounding to the nearest FP32. We only wish to log rounding decisions for Case A, where the auditor should copy the trainer's rounding choice in order to reach the same value. This requires defining a logging region, determined by threshold T,
the rounding boundary. Consider Case A Figure 3 which shows divergence the output of computation using FP64 on two different GPUs. Because the outputs of GPU and are on different sides of the boundary, rounding to the nearest FP32 results in different values, introducing error.
What if the trainer records their rounding choice (c.g., up, down, none) for every intermediate computation? The auditor could then copy the trainer's choice, and therefore round the exact same value and successfully control for nondeterminism. However, the auditor should not copy the trainer's behavior for every output (see Cases B & C. Figure 3). If a computation output on GPU is too close to the rounded value, then it is possible that GPU is also close in distance but from the opposite direction. this case, the auditor should ignore the trainer's choice. We therefore need to introduce a threshold under which the trainer does not record their rounding choice.
Our method requires upper bounding the divergence daiv between any two different GPUs for any intermediate computation f (i.e. difference in outputs for the same input). Let es represent the distance between two FP32 values, after rounding to b, bits of the mantissa (Figure 5 and controlling for the exponent. We need to select b, and such that daie < Cb, and ddiv < 2r (Figure 3}. Because the set of possible FP numbers finite, there exist optimal bounds for b, and т. In practice, we find that b, < 32 and 0.25 E32 are sufficient for standard intermediate computations in neural network training (e.g.. convolution, layer norm) in FP64. We study different values for b, Section 6
5.2 Primitives
We assume both trainer and auditor train models using the IEEE-754 standard FP numbers (Figure Besides requiring read and write disk 1/O operations, we define the following functions:
1. rnds, (z)): rounds input x to the nearest FP up to b, bits the mantissa, as shown in Figure 5
2. log(x, by, 7, f): logs to file f logging direction c, which is either 0 (down), 1 (ignore), depending on threshold 7 and rounding amount b,, as shown in Algorithm 4
(up)
rev(x, by, c): reverses rounding of input based on logging direction c. If æ < rnds, (x) & then return & rounded to the nearest float below x with b, precision. If x > rdd, (I) & c 2, then return 2' rounded the nearest float above x with 6, precision. Otherwise, do not correct.
4. (hreshold(1, b,,, ber): identifies the optimal threshold to log rounding directions (0 or 2) instead of
1, which the rev function ignores, based on the binary search procedure in Section [5.4
5. hashsha255 (f): creates a SHA-256 hash of provided model weights 0 (in b,, precision).
tree(leaf,, !eaf,... leaf, create a Merkle tree where each leaf node is the output of hash (
for model weights at a given checkpoint, with checkpointing interval k [Merkle 1988
5.3 Training and Auditing
The traincr's task begins when a client approaches them with dataset D, training specifications (epochs E, loss function loss, etc.), and a requested model precision The trainer can then choose a training precision by > bm. rounding amount b, < bm. and a checkpointing interval & to periodically store small hashaha256 of model weights / in Merkle tree, for efficient comparison with an eventual auditor. Then, as detailed in Algorithm T the trainer can perform training as normal, but after every intermediate computation (e.g., convolution) perform the rnds, operation

---

each output. Rounding is applied to computations in both the forward and backward passes. Finally, either using fixed threshold T or a layer-specific optimal from the threshold function described in Section 5.4 the trainer applies log, which logs rounding choices only for the computations an auditor should copy. The output of the algorithm includes a rounding log file F and the root of the Merkle tree which, along with the shared randomness R and all training parameters, the trainer can share with any trusted third-pary auditor who challenges them.
After a client approaches them, the auditor initiates the verification game described in Section 3 To avoid penalty, the trainer must cooperate by sharing the rounding amount by. randomness R used in training (e.g., a pseudo-random number generator), the checkpointing interval k, and set of rounding logs F. The auditor then follows the training procedure and corrects their rounding choice (e.g., up or down) to match those logged in F using the rev operation, as detailed in Algorithm 2 (Appendix). By correcting each rounding mismatch during the course of training. the auditor is able to prevent nondeterminism errors from accumulating. Therefore, the auditor can store the has (0) of model weights in a Merkle tree at interval k, knowing that if training was done correctly, the model weights should be identical to the trainer's s at any timestep. The output of Algorithm 2 is the root of the auditor's s Merkle tree, which they can use to compare with the trainer's 's root.
5.4 Reducing storage cost
Logging rounding decisions for every neural network layer output during training incurs a large baseline storage cost, and is our main limitation. For dataset D, batch size B, training epochs E, and model layers Lg, the upper bound on the total storage cost for verifiable training with our method is:
storage cost (B) = D x Ex
OLI + Σ oLbb
(I)
where o and o represent the size of outputs of the forward pass and backward pass of layer 1. Note that the log entries do not need to be kept around in the RAM and can be written straight to the disk. Moreover, this cost is a one-time cost incurred by the trainer, who in our context is likely to be a powerful commercial provider with access to such storage capacity. Furthermore, as we later show in Section 6 for models with many lincar layers like ceanssfomer-bbase language models (c.g., GPT-2), where parameters significantly outnumber intermediate computations, this storage cost is significantly smaller than alternative approaches that require saving model weights (Jia et al. 2021]. Nevertheless, we now describe our method for reducing storage cost by (i) efficiently encoding rounding logs and (ii) adaptive selection of the threshold
reduce the storage costs.
Each log entry is a value from the set 0, 1, 2, as opposed to the FP model weights. We pack sub-sequences of five log entries into single byte via a fast GPU-based radix-3 to radix-2 conversion, yielding 1.6 bits/entry storage that is close to the best possible packing of 1.58 bits/entry, and yiclds a 77% storage reduction relative to naively storing one log entry per byte.
Efficient Encoding:
Recall that we need to select a threshold I that controls for whether the trainer logs a rounding choice, or instead logs 1 which the auditor ignores. The more one can increase T, the more 1 values are recorded, which can make rounding logs more compressible (due long sequences of 1s). Furthermore, it is possible that the divergence daie between outputs on two different GPUs, given the same input, is function -ppecific. For example, while convolution requires several matrix multiplications that might result in large FP accumulation error, normalization operations are unlikely to result in large daior and a larger T can be applied. We develop an efficient algorithm (Algorithm 3 in the Appendix) to find the optimal value for given a particular layer and data of output values that led to different rounding choices between any two GPUs (c.g., Case A in Figure B). For given rounding amount b, and training precision ber. the algorithm performs binary search between 7 = 0.25- E32 (our upper bound on the daie between two GPUs for any function) and 0.5 Eb,, (the rounding boundary). By performing this procedure for the different intermediate computations in a model, the trainer can hope to better compress the rounding log F.
Adaptive Threshold:
Merkle Tree Storage:
Storing SHA-256 hashes of model weights during training in a Merkle tree creates an efficient mechanism for the verification game described Section with negligible storage requirements. The audit ends when either the trainer withdraws, the auditor confirms that training was performed correctly, or the auditor can present paths to the two leaves of their Merkle tree where divergence starts, providing evidence to dispute the trainer.
7

---

Standard Training
Smplle Rounding
Our Method
BesNee-50
6PT.2
ResNet-50
GPT-2
Wogggn Citt.
9.40
Text Lowa Clange
45-
1-00
1012 12 Welggh
20-
0.5
0.3
J+-2
Inde
Time
20
0:5
6-22
0.3
100
350
202
65 20 G0
25
40
05
50
25
66
Steps
% Log Entrics ignored
Log Entr lgnored
Figure 4: We successfully control for nondeterminism between GPU types for both ResNet-50 (a.) and GPT-2 (b.) tasks, while standard training and simple rounding without performing rev corrections result in model divergence performance (c.), but at the cost increasing time for trainer (d.).
the course of training. Stronger rounding has minimal affect model
Table 2: Efficient encoding reduces storage requirements by 77%, and rounding to b 26 improves the compression further between 5-20% (values reported for 1 step of training). The original proof- of-learning protocol from Jia et al. 2021 requires storing 2.78 GB of model weights for GPT-2, more than 140x our storage cost, while still incurring statistical error.
ResNett-50
ResNet-50 6 26
GPT-2 b 32
GPT-2 26
Naive Encoding
456 MB
456 MB
92 MB
92 MB
Efficient Encoding
105 MB
105 MB
22 MB
22 MB
+ Zip Compression
MB
91 MB
20 MB
18 MB
Empirical Results
We evaluate our verifiable training method on the two large-scale models listed below with all possible trainer and auditor pairs across NVIDIA GPUs A40, TITAN Xp, and RTX 2080 Ti (see Appendix B for more details). In Section 6.2
we compare our method with recent proof-based systems.
1. ResNet-50: We train (from random initialization) ResNet-50 (23M) on CIFAR- 10 with dataset size 50K & batch size B=64. Test accuracy 90.7% after 100 epochs training on Titan RTX Ti.
2. GPT-2: We finetune GPT-2 (1 (117M) on a corpus of Shakespeare text with dataset size 1.1M tokens, batch size B=8, and sequence length 64. Perplexity = 4.22 after epoch training on Titan RTX Ti.
Figure 2 shows that nondeterminism due to GPU architecture exists for both tasks. While we can repeatedly obtain identical results across training runs on the same GPU architecture, training on different GPU architectures results in fundamentally different models.
6.1 Implementation and Findings
We implement our verifiable training method entirely on top of the pytorch framework, with torch version 1.13.1 and CUDA version 11.7. The intermediate computations we apply rnd, to are layers (e.g., torch. mn. Conv2D) in the model's computation graph. Rounding-related operations (rnd and rev) either using casting or FP functions (e.g., torch. nextafter) that can run on the GPU, thus having little impact on computational speed. Because we observed that the torch. randn operation used for dropout in GPT-2 non-deterministic for long inputs (even for the same seed, see Appendix D, we implement our own dropout as our method requires shared randomness R.
Successful control for non-determinism: Our method completely eliminates non-determinism between full training runs of both for both the ResNet-50 training and GPT-2 fine-tuning tasks across all possible trainer and auditor pairs between the A40, Titan XP, and RTX 2080 Ti GPUs As standard FP32 training results in an increasing divergence (12-distance of weights) between models on different GPUs over the course of training. Furthermore, we show the simple approach of training in FP64 and rounding to FP32 after every intermediate computation, but without the auditor correcting rounding decisions with rev, fails to mitigate this issue. Only our method, in which the auditor follows the rounding decisions (b, 32) made by the trainer for every intermediate computation, eliminates non-determinism and persists over time. Our implementation, which requires disk I/O during training to store the rounding decisions, results in a small increase in training time for the trainer (1.2-1.4x) and auditor (133-177x) using non-optimized, protoype implementation (Table 5) We report the storage requirements of our method in Table showing

---

Table 3: Average of rev corrections performed by auditor per training step. Even at b = 32, auditing only requires 20-25 corrections (2e-6 to 9e-6 % of samples) per training step.
ResNet-50
b = 32
b 31
b 30
b == 29
b b= 28
b = 27
b 26
Forward
1
2
3
3 3+1 1
0
0
Backward
10 0.6
6 0.6
2
0.7 = 0.7
0
0 0
0
GPT-2
b = 32
b b= 31
b = 30
b = 29
b = 28
b b= 27
b b= 26
Forward
2 0.7
2.3 0.8
2.2 0.4
0.2 0.2
0.4 1 0.2
0
040
Backward
19 + 13
0.75 + 0.3
1.2 0.4
0.2 0.2
0. = 0.0
0=
0
Table 4: Adaptive thresholds identified for different operations using Algorithm
with b = 32.
2D Convolution
Batch Norm
Linear
Layer Norm (768.1)
Dimension
256 (I.I)
(128. 128, 16, 16)
(768,768)
0.305 2 2- 23
0.199 2 23
0.165 + 2 22
0.499 * 2 -22
that our efficient encoding scheme reduces the size of the trainer's rounding logs by 77%, relative to naive logging. Because the Merkle tree stores 32-byte SHA-256 hashes, its overall size (KBs) and creation time are negligible and not reported. Finally, we show that decreasing the rounding amount b to values even low as 26 has little effect model performance (we observe change in accuracy. so report test loss), but increase training time (Figure44 4. We observe that smaller values of b do allow more log entries be ignored, improving compression of the file, which we discuss next.
Our approach outperforms (Table the storage costs of proof-of-learning protocols that save model weights for GPT-2 (2.78GB) which has many linear layers - we observe more than 140x reduction relative to the approach in Jia et al.] [222]] We further reduce the storage cost of our method by decreasing the rounding amount b and implementing the adaptive thresholding strategy (Section [.4). Table 4 reports adaptive thresholds T for four different pytorch layers at rounding amount 6, = 32. Convolutions require the lowest т, indicating larger divergence in outputs between GPU types, which is expected due to the large # of matrix multiplications. Meanwhile, T is higher for normalization layers, likely due to smaller divergences between GPU types. Because adaptive thresholding seeks to reduce the # of times rounding decisions (0 and 2) are logged and improve log file compression, we report storage cost after zip compression in Table 2] As expected, more aggressive rounding (which results in a higher T) improves the compression rate. Although the compression gains mild in comparison to our encoding step, they build-up over the course of training. Finally, we report the average # of rev corrections an auditor needs to perform for one training step in our two tasks (Table 3. These values are surprisingly small in comparison to the # of operations logged - only a maximum of 2e-6% % (ResNet-50) and 9e-6% % (GPT-2) of logged values, are actually needed by the auditor! We also observe that severe rounding (c.g.. b = 27) completely eliminated the hardware -endeeeemmnnsmm for our tasks, requiring no corrections from the auditor. This shows huge gap between the # of values currently saved by the trainer and those needed by the auditor, motivating an exciting future possibility of significantly reducing the storage cost of our method we could reliably predict when a divergence will not occur.
Compression with adaptive threshold:
6.2 Comparison with alternative approaches
Logistic Regression: Garg et al. 2023 recently proposed a zero- enooelee proof- based system for verifiable training of a logistic regression, which importantly does not leak information about the client's 's data or require a trusted third-party auditor, unlike our work. However, since verifiable training itself is motivated by a client not having sufficient resources to train the model, it is crucial to consider the implications of scale. The authors report the prover time and proof size requirements for one training pass of logistic regression on a dataset of 28 items, with 1024 dimensions and a batch size of 2014, 72 seconds (training and proof generation time) and 350 MB respectively. We replicate this training task, and find that our method significantly improves upon both storage and time requirements, requiring only 106 KB and 7 seconds (both training and auditing). Furthermore, because Garg et al. [2023] do not report the duration of "offline phase" of their method, their reported value is lower bound on the actual time required. Finally, we note that the original proof-of-learnin protocol from Jia et al. [2021], which also considers a trusted third-party, would require 9.2 MB per training step to store all model weights. Therefore, our method is at least 85x more space efficient.
VGG-11: Concurrent to this work, Abbaszadeh et al. 2024 introduce a vero-knowledge proof- of-training protocol for deep neural networks, presenting results for one batch step of training for

---

simplified version of the VGG-11 model with 10M parameters. which is less than the original VGG-11 network and ResNet-50 [Simonyan and Zisserman 2013]. While the authors do not provide architectural details, we can assume that increasing the # of parameters to the original VGG-11 I would only increase their reported proof time and size. We, therefore, compare their reported values with an implementation of our method for the same task of verifying the training of VGG-11 1 on CIFAR-10 10 with a batch size of 16. While their use of incrementally verifiable computation leads to tractable proof size (1.66MB vs. the 1.2MB per iteration cost of our method), AAbbazzadeh et al. (2024]'s s method requires 22 min. per training iteration. In comparison, our method requires training and auditing times of only 6 sec. per iteration and is significantly more efficient (factor of 220x), an important consideration for model training as a commercial service
Finally, in Appendix Section J we compare our results with an adaption of Gupta et al. 2023 s protocol for secure inference of GPT-2. Compared with our method's 's storage cost (18MB) and training time (11s for training. would introduce around a 10,000x data and 40x time overhead. While proof- proof-based systems provide strong security guarantees without third party, they do at the cost of relying on hard-to-scale cryptographic techniques, as well as approximating non-linear functions that can harm performance.
13.5s for auditing), scaling Gupta et al. [2023]'s 's protocol for training
7 Security Analysis
Our work makes I-of-n honesty assumption, i.e., long one of auditors is honest, any attack from a malicious trainer that results in diverging model weights will be detected. One consideration is the potential manipulation of the rounding logs by an adversarial trainer who could select rounding decisions that achieve a desired outcome, and which the auditor would follow. Concretely, let us define our threat model that the trainer knows an auditor's 's GPU priori. Recall that an auditor only copies the trainer's 's rounding decision in Case A in Figure 3 when both GPUs compute values close to the rounding boundary. Under this threat model, the trainer can identify the steps where the auditor is close to the boundary (as in Case A), enumerate the set of 2n different models that result from different rounding decisions, and selectively pick a model that exhibits desired property.
However, the trainer cannot use this strategy to embed an arbitrary property (e.g.. a specific backdoor). It can only select from the set of models that differ in certain rounding decisions, which all require the trainer to use the correct training specifications accepted by the client (such as exact training data & hyperparameters). Furthermore, since the expected # of divergences between the trainer and the auditor is extremely small (see Table 3 the set of possible models where an auditor would not detect an attack (c.g., many rev ops) is limited. Finally, we show in Table in the appendix that the divergence (measured both as fn-norm between model weights and output distributions) due to GPU non-determinism is significantly less than the divergence due to data ordering during training. Therefore, if a client will accept a model trained with any random ordering of the data during training, then it is unlikely that adversarial trainer that can only alter rounding decisions produce a model that the client would not accept. Nevertheless, fully understanding the model properties obtained by manipulating rounding logs adversarially is an important future direction.
could
Limitations and Future Work
Our verifiable training scheme successfully controls for hardware nondeterminism. It expands the pool of potential auditors of model training service, allowing us to envision world where client can even use two competing service providers it trusts to audit each other. Relative to proof- based systems, a limitation is the need for all parties trust the third-party auditor. If the trainer provides finetuning services on top of closed-source models (c.g., OpenAD), then our scheme will only work for the third-party auditors that the trainer is willing to share model weights with. Other limitations included the added latency of training in higher precision and the storage cost. While we have shown that our method requires significantly less storage than alternatives, the vast majority of stored rounding decisions are not used by the auditor and are therefore unnecessary (Section 6). Therefore, an exciting direction for future work is to mitigate this gap by better predicting when GPU divergence between computations occurs. Recent work has similarly argued for stronger profile of noise during training in the context of verification [Fang et al., 2003). Finally, another direction for future work includes adapting our method for distributed training [Li et al. 2020
10

---

Acknowledgements
We thank Bill Dally, Duncan Riach, Gabriel Poesia, and Chris Ré for helpful discussion and feedback. Megha Srivastava was supported by IBM PhD Fellowship and the NSF Graduate Research Fellowship Program under Grant No. DGE- 1656186. In addition, this work was funded by NSE, DARPA, the Simons Foundation, UBRI, and NTT Research. Opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA.
References
Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, and Pablo Villalobos. Compute trends across three eras of machine learning. In 2022 International Joint Conference on Neural Networks (JCNN), IEEE, July 2022. doi: 10. (LL10//jccm330004222229999999999 URL hhttp://dx.do. doi. org/10. 10. 1109/
[JCNN55064 2022. 9891914
Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. Poisoning language models during instruction tuning, 2023.
Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntamathan, and Or Zamir. Planting undetectable backdoors in machine learning models, 2022.
Tianyi Liu, Xiang Xie, and Yupeng Zhang. zkenn: Zero knowledge proofs for convolutional neural network predictions and accuracy. Cryptology ePrint Archive, Paper 2021/673, 2021. URL httpttts:///::iitt iacr org/2021/673
Sanjam Garg, Aarushi Goel, Somesh Jha, Saced Mahloujifar, Mohammad Mahmoody, Guru- Vamsi Policharla, and Mingyuan Wang. Experimenting with zero-knowledge proofs of training. Cryptology ePrint Archive, Paper 2023/1345, 2023. URL https::/epprinn iacr wwt.o/23//////.toooo. 1345 2023/1345
httpa://eprint. iacr org/
Scott Ames, Carmit Hazay, Yuval Ishai, and Muthuramakrishhan Venkitasubramanian. Ligero: Lightweight sublincar arguments without a trusted setup. Cryptology ePrint Archive, Paper 2022/1608, 2022. URI.
httpp://eprintt. iacr. wlwarroorg//2///reee 1608
htttps:/eprintt. iacr. org/2022/1600
Jason Teutsch and Christian Reitwietnerr A scalable verification solution for blockchains. CoRR, wtts//998.0444 2019. htttt:://aw...ooo/////.a/
04756
Hengrui Jia, Mohammad Yaghini, Christopher Choquette-Chon, Natalic Dullerud, Anvith Thudi, Varun Chandrasekaran, and Nicolas Paperoo. lflamiing:: Definitions and practice. CoRR, abs/2103.05633, 2021. https://::://// arwiv.iiiivv/.ii//aaaaaaaaaaaaaa org/ abs 2103, 05633
Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Papernot. On the necessity of auditable algorithmic definitions for machine unleamnigg In 3/ USENIX Securiry Symposium (USENIX Security 22), pages 4007-4022, Boston, MA, August 2022. USENIX Association. ISBN 978-1-939133-31-11 URL https www..usenix.rm org/
Congyu Fang, Hengrui Jia, Anvith Thudi, Mohammad Yaghini, Christopher A. Choquette Ciaquette-Choo, Natalie Dullerud, Varun Chandrasckaran, and Nicolas Papernot. Pronf-of-lcarning is currently more broken than you think, 2023.
Ralph C. Merkle. A digital signature based conventional encryption function. In Carl Pomerance. editor, Advances in Cryptology CRYPTO '87, pages 378, Berlin, Heidelberg. 1988. Springer Berlin Heidelberg. ISBN 778-3-540-48184-3
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson. Andreas Terzis, Kurt Thomas, and Florian Tramèr. Poisoning web-scale training datasets is practical, 2023.
Pang Wei Koh, Jacob Steinhardt, and Percy Liang. Stronger data poisoning attacks break data sanitization defenses, 2021.
Silvio Micali. CS proof' (extended abstracts). In 35th Annual Symposium on Foundations of Computer Science, Santa Fe. New Mexico, USA, 20-22 November 1994, pages 436 453, IEEE Computer Society, 1994, doi: 10.1109/SFCS 19W.9//FFSS.994..665559 1994.365746 URL httttt://///oo org/10 1109/SFCSS 1994. 365746

---

Nir Bitansky, Ran Canetti, Alessandro Chiesa. and Eran Tromer. From extractable collision resistance to succinet non-interactive arguments of knowledge, and back again. In Innovations in Theoretical Computer Science (ITCS), pages 326-349, ACM, 2012.
Seunghwa Lee, Hankyung Ko, Jihye Kim, and Hyunok Oh, venn: Verifiable convolutional neural network based on zk-snarks. Cryptology ePrint Archive, Paper 2020/584, 2020. UtI.ptttps:/epprint iacr. www...rg///............. / 584 https://eprint iacrr org/2020/584
Daniel Kang, Tatsunori Hashimoto, Ion Stoica, and Yi Sun. Scaling up trustless dnn inference with zero- knowledge proofs, 2022.
Kasra Abbaszadch, Christodoulos Pappas, Dimitrios Papadopoulos, and Jonathan Katz. Gero-knowledge proofs training for deep neural networks. Cryptology ePrint Archive, Paper 2024/162, 2024. URL
httpa://eprint. waw.riaaccoooooooo wwwarr.org//2/////0// 162 https://eprint iacp iacr owg/ org/2024/162
Gobikrishna Dhanuskodi, Sudeshna Guha, Vidhya Krishnan, Aruna Manjunatha, Rob Nertney, Michael O'Connor, and Phil Rogers. Creating the first confidential gpus. Commun. ACM, (7(1)660-67, dee 2023. ISSN 0001-0782. doi: 10.1145/3626827. httpssts:://do: org/ 10. 1145/3626827
Alexander Nilsson, Pegah Nikbakht Bideh, and Joakim Brorsson. A survey of published attacks on intel sgx, 2020.
Jo Van Bulck. Marina Minkin, Ofir Weisse, Daniel Genkin, Baris Kasikei. Frank Piessens, Mark Silberstein, Thomas F. Wenisch, Yuval Yarom, and Raoul Strackx. Foreshadow: Extracting the keys to the intel SGX kingdom with transient Out-of-Order exccution. In 27th USENIX Security Symposium (USENIX Security 18), page 991-1008, Baltimore, MD, August 2018. USENIX Association. ISBN 978-1-9399 178-1-939333-04-5 URL https: //ww: /www usenix. /tt.a//eeerattt///aaaaaeeaeaae/e
Zhifeng Kong, Amrita Roy Chowdhury, and Kamalika Chaudhuri. Can membership inferencing be refuted?? 2023.
Dami Choi, Yonadav Shavit, and David Duvenaud. Tools for verifying neural models' training data. In Neural Information Processing Systems, 2023.
Hadi Jooybar, Wilson W. L. Fung, Mike O' O'Connor, Joseph Devietti, and Tor M. Aamodt. Gpudet: deterministic gpu architecture. ASPLOS '1: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems, 2013.
David Defour and Caroline Collange. Reproducible floating-poing atomic addition in data data-parallel environment. In Proc. of the Federated Conference on Computer Science and Information Systems, 2015.
Yuan Hsi Chou, Christopher Ng, Shaylin Cattell, Jeremy Intan, Matthew D. Sinclair Joseph Devietti, Timothy G. Rogers, and Tor M. Aamodt. Deterministic atomic buffering. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2020.
TensorFlow. Tensorflow 2.8.0-roO, 2021. URI. htttss///gtthabbccc///eeerrrftot
releases/tag tttp///gg/v2...xxe//.
Donglin Zhuang, Xingyao Zhang, Shuaiwen Leon Song, and Sara Hooker. Randomness in neural network training: Characterizing the impact of tooling. In arXiv:2106. 1182206.1187.,.. 2021.
Matt Crane. Questionable answers in question answering research: Reproducibility and variability of published
results. Transactions of the Association for Computational Linguistics, 6:241-252, 2018a. doi: 10. 166/tacl a 0.000.88 URL stttps://aclantholegg www.ggg.....rgooeee arg/Q18-1018
NVIDIA. Determinism across gpu architectures, 2022.
URL attpa:///ittuu..com///
Matt Crane. Questionable answers in question answering research: Reproducibility and variability of published
results. Transactions of the Association for Computational Linguistics, 6:241-252, 2018b. doi: 10. 162/tacl_
10.0008.. URL /aljaatta///ccaaatoolloee www.lloooyoorrg.go wwwpeeeorrgr/r-11/00001 Q01811188
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, and Eric Horvitz, An empirical analysis of backward compatibility in machine learning systems, 2020.
William Kahan, Further remarks on reducing truncation errors, 1965. URL https:/dld acm. org/doi//ppe 10. 10111115333777 36370/ 363723
12

---

IEEE 754 Floating Point Standard
Exponent
Marrieaa
round (C+52)
round (2=311
Figure 5: We define rounding to b bits as rounding to the nearest 32-bit FP number that has Os in the last 32 bits of the mantissa, after accounting for the exponent.
Nathan Whitehead and Alex Fit-Florca. Precision & performance: Floating point and ieee 754 compliance for nvidia gpus, 2011. URL URL/htpss //develppeeeeeeeeeeeeee www.iiiiiiii..... htwp:ppee..iiii..cooooeeeeeeeeee htt.://a..../ataeeee//////////// /ccbaaaai/caa/////// http://WVDDAA-CCD---Laaaaaaaaaa
pdf
NVIDIA. Cuda: Hopper tuning guide. 2023.
URL http:://occ..vidaaacommm
www.niiiiii.o..m www.daa.com/uuda/pd.....f cuda/pdf//////////c /ooperr
Tuning Guide. .df
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, 2015.
Kanav Gupta, Neha Jawalkar, Ananta Mukherjee, Nishanth Chandran, Divya Gupta, Ashish Panwar, and Rahul Sharma. Sigma: Secure gpt inference with function secret sharing. Cryptology ePrint Archive, Paper 2023/1269, 2023. htttp:///////prittttt iacr. .aaarrorg//aa. 2202//2269
https://eprint. iacr. org/ /023
1269
Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala. Pytorch distributed: experiences on accelcrating data parallel training. Proc. VLDB Endow. 13(12):3005-35 3018, August 2020. ISSN 2150-8097. doi: 10. 14777/3415578.34155551
URL https:/t::///o org/ 11771/3447447777 34115500
A
IEEE Floating Point Image
See Figure 5
B
GPU Details
All experiments reported our paper are run with the following three GPUs:
NVIDIA Titan XP: 3840 Cores, 12 GB
NVIDIA RTX 2080 Ti: 4352 Cores, 11 GB
NVIDIA A40: 10752 Cores, 48 GB
We able successfully replicate training runs between all pairs of these GPUs.
C Logging Algorithm
See Algorithm 4
D Train Algorithm
See Algorithm
E Audit Algorithm
See Algorithm 2
13

---

F
Adaptive Thresholding Algorithm
See Algorithm 3
G Time Requirements
See Table 5
H
Model Divergence Comparison
See 6
I Random Number Generation
Our verifiable training scheme requires shared randomness between the trainer and auditor, which is used for deciding input data batching, weight initialization, and operations such dropout (randomly setting outputs to zero). More formally, our scheme requires sharing the same random seed and pseudo-random generator. However, in our implementation based pytorch (assuming the same software version between trainer and auditor), we chose to rely on the the torch random seed functionality. While this successfully controls for batch input ordering and weight initialization, is unfortunately not sufficient for random number generation, as operations such as torch. nn. mandn () leverage parallelism when the requested # of values is higher than a certain amount. Specifically, we found that across T40, RTX 2080 Ti. V100, A40, and A A100, given the same seed, torch randint) produces identical tensors onlt up size 40960. At size 40961, T40 (which an older GPU) deviated from the rest. Likewise, at size 69633, 2080 Ti deviated from the rest, and so on. Based on these observations, we arranged for calls to torch. randint)) in the dropout layer (which is the only operation using large random tensors in our tasks) to be replaced by generating and concatenating multiple random tensors of size 40960 or less. Specifically, a random tensor of size n > 40960 is generated by concatenating (n//40960) random tensors of size 40960 and one random tensor of size (n%40960). However, we emphasize that it is therefore important our scheme either for both parties to implement this change priori, simply use external source for pseudorandomness.
J Comparison with GPT-2 Inference
The previously discussed proof- based systems for verifiable training by-pass the need for a third- party auditor, but very few efficient systems exist in the literature. Many more works study secure inference of deep neural networks, which could be used to construct verifiable training protocols with stronger security guarantees than ours (e.g., allowing trainer keep a proprietary model's weights private), but come at a significant cost to performance and resources. To demonstrate this, we consider adapting Gupta et al. [2023]'s protocol for secure inference of GPT-2 based on multi-party computation, to our context of verifiable training. Gupta et al. [2023) show how two parties, the client with private data and the trainer, can jointly compute the forward pass of known model architecture without revealing additional information beyond the model output to each other. Because they report the the communication overhead P 0.37GB and time T 0.96 seconds for one forward pass on single data input, we can calculate 2 x P x D x E = 189 GB and 2 x T x D X E 983 seconds as estimated communication cost and time, respectively, for step of training in out GPT-2 task, where 2 considers both the forward and backward pass. Compared with our method's required storage cost (18MB) and training time (IIs for training, 13.5 seconds for auditing), scaling Gupta et al. (2023) protocol for training would introduce around 10,000x data and 40x time overhead.
14

---

Algorithm 1 train
INPUT: dataset D, epochs E, batch size B, shared randomness R, model Wo, loss function loss, rounding amount br, training precision btr, target model precision bm, checkpointing interval k OUTPUT: Merkle root Myoot, rounding log file F
1: F, Meases create empty file and leaf list
2: Wo +- init( R, ber) //initialize weights
3: T D=E
4: for 1 do
5: input + batch D. B. // get batch // forward pass
6: for layer lo E Wa.layes do
7:
output <- (o@nput))
8:
T + thresholdd(a, by, ber) //set threshold log(output, by, F)
9:
10:
output + rnds, (output)
11:
imput + output 12: end for
13: loss loss loss(output)
14: // backward pass, reversed layers
15:
poudpo <- Viess
16:
for layer lo E laylayers do
17:
praddinputt.oo i (ggradooutput)))
18:
T + -ttreshold(Vii br, ber)
[elgraddimpuute bp, F)
20: grad_ ind_inppttea +- rnds, (grad_inpul
grad output + grad_input end for
grad_output
22:
8 update update weights
24:
if mod k = 0 then
Mleaves appenddsaas///ssssshhsssss (0 in precision bm))
26: end if
end for Mroot treeree triee(Meaves) // create Merkle tree
28:
29:
return F, Mroot, and model We in target precision bm
Table 5: Training time requirements, including Merkle tree operations (at A: 5), for I step of training broken down by stage of our verifiable training process. Note that reported times are specific the particular dataset, batch size, and task, and using non-optimized prototype codebase therefore the relative increase is time is more important.
ResNet-50
GPT-2
Original (No Rounding or Disk I/O)
24s
8s
Trainer
28s
11s
Auditor
31s
13.5
Table 6: Comparison of model divergence due to data ordering versus GPU non-determinism. Reported numbers are averaged between 10 pairs of models, error bars are standard deviation.
Metrie
Data Ordering
GPU Non- den-determmnism
12 weight difference
133.2 + 9
1.1 + 0.07
12 output distance
5.3 4 0.03
0.26 + 0.02
15

---

Algorithm 2 audit
dataset D, epochs E, batch size B, shared randomness R. model We, loss function loss, rounding amount by, training precision byy, target model precision 6m. checkpointing interval k log file F from trainer
INPUT:
OUTPUT: Merkle tree root Mroot
Mieoves t- create empty leaf list
2: Wa f- init R, ber) //initialize weights
3: T' t- D+E
4: for 11.. do
5: imput + batch D. B) // get data batch
// forward pass
6:
for layer ln Wayess do outpuu & (i(ippt
7:
8:
for outpul; oulput do
9:
// M.tch trainer rounding
10:
C (-=rea(outputt, F))
11:
output; + rev(outputs, by, c)
12:
end for
13: imput + output
14: end for
15: loss t -osss (ooss()tput))
16:
// backward pass
17:
<- Viess
18:
for layer lo Wo-layers do grad_input input t- Vi, ) ouupput) for grad_inpui input; E grad_input do
19:
21:
// Match trainer rounding c read(grad_impput, F)
22:
23:
pnd_innpt, &- rev(grad inputs, by, c)
end for prnd_oupputt + grad input end for
25:
27:
O update update weights
28: if mod then
29:
Mieaves apcnesssppend/hssssssssss (0 in precision b,)))
30: end if
31: end for
32: Mroot f- ttree(Meavee // create Merkle tree
33: return Mroot
16

---

Algorithm 3 threshold
INPUT: layer I, rounding amount 6,., training precision by-
OUTPUT: threshold
1: P -- initialize empty list
2: N, T initialize large of data points and iterations
3: for i=I...N do
4:
GPU1, GPU2 select two different GPU architectures
5:
x - select random input for layer in by floating-pon precision
6: 31 (G) 1/2 t- (GPU2(I), apply layer on input z each GPU
7:
if rnde, (v1) * mdd, (92) then
8:
1/1 > rnd,, (y1) and y2 rnds, (12) then P.append (191 rnda, (81)))
9:
P.append [2/2 rnd. (y2)))
11:
end if
1) < rnd, (y1) and y/2 > rnds, then P.append y/ rnd. (V1 ))
13:
14:
P.append (/2 rnds, (y2)))
15: end if
16: end if
17: end for
18: //binary search to select threshold
19: lower, upper, + 0.25 (2 (2-23),00 0.5 * (29-h, )
bb
20: for tl....
21:
(lower + upper) //2
22: success 4- True
23: for Pt P do
24:
exp gett exponent of
p: < exp *T then
26:
success + False
end if
28:
end for
if success then
30:
lower f-
31: else
32:
upper <
33:
if
34:
end for
35:
return
Algorithm log
INPUT: value v, rounding amount b,, threshold file F
1: exp +- get exponent
2: if [ar rnds, (x)l exp *7 and rnds, (x:) then
3: write(2, F) // log rounding
4: else if [xr rnd,, (a:) exp*7 and x mdy, (x) then
5: write(0, F) // log rounding down
6: else
7: write(1, F) // log rounding ignore
8: end if
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer:[Yes)
Justification:
See Limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer:[Yes Yes]
Justification: See Experiments and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
Code will be released in public version.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuiu wwwiicccguie//eeeeeeeeeis CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Experiments and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
No experiment results have variation necessitating statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
20

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix for GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:::///eurippee httppp/s..........ttttcccceeeeea
Answer: [Yes]
Justification: Yes
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Security Analysis.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
21

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
22

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "ulnA3xTrcB", "reviewer": "Reviewer_RB9A", "summary": "This paper studies the problem of verifying the correctness of the training process of model. In particular, the user who lacks sufficient resources pays a service provider with sufficient resources to train models. Then a trusted third-party auditor will check whether the training process is legit. The proposed approach rounds after intermediate computation steps, and stores rounding decisions based on an adaptive thresholding procedure, to successfully control for nondeterminism among different GPUs and also saves the storage space. Empirical results show that the proposed approach can run more efficiently than the previous baselines while also using smaller storage space, which makes it valuable for verifying larger models.", "questions": "There is no direct outstanding questions, but I would encourage the authors to focus on the writing to highlight the contribution more explicitly.", "limitations": "Limitations are well addressed in this work.", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The problem of verifying the correctness of the model training process is an important problem and leveraging a third-party auditor can be one possible way to scale these techniques to larger models.\n2. The proposed approach outperforms the existing baselines both in terms of verification efficiency and usage of storage space.", "weaknesses": "There is a lack of description on the relation and difference of this work and Teutsch & Reitwießner (2019). I think the authors will benefit by clarifying the difference and as well as emphasizing their novelty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! Our work is the first to show how to apply Teutsch & Reitwießner (2019)’s method, which was developed specifically for blockchain verification, to a machine learning setting. As such, our method needs to address GPU nondeterminism challenges that Teutsch & Reitwießner (2019) do not need to. Concrete differences includes: \n\n1. **Novel Setting**: The motivation of Teutsch & Reitwießner (2019) is to address the “verifier’s dilemma” that occurs in blockchain protocols where verifying a transaction (which ensures the security of the blockchain) is too computationally intensive for a user, resulting in less financial incentive for blockchain “miners” to perform verification. They do not explore machine learning applications, including our goal of verifiable training, where the outsourced computation needs to take place on specialized hardware like GPUs. \n2. **Rounding to Eliminate Nondeterminism**:  Because the protocol in Teutsch & Reitwießner (2019) does not need to address nondeterminism issues, their approach of both parties independently storing intermediate computation outputs in a Merkle tree suffices.  In our work, both parties need to round intermediate computations and share rounding directions, as described in Section 5.3. This modifies the computation task and is a more interactive procedure. \n3. **Rounding Log Storage**:   Unlike Teutsch & Reitwießner (2019), our method requires parties to store rounding log information to address nondeterminism. Our paper therefore proposes a novel method for reducing the storage cost, as described in Section 5.4, that describes an adaptive threshold selection procedure unique to our setting.\n\nTo address the reviewer’s concerns, we will add the following sentences in line 66 in the Introduction: \n\n*\"We show how to adapt the verification game from Teutsch & Reitwießner (2019), which that was proposed for verifying transactions in blockchain protocols, where an efficient Merkle tree data structure stores intermediate computation outputs for computationally intensive tasks. Unfortunately, naively adapting this strategy for verifiable training, where intermediate computation outputs are model checkpoints, fails due to hardware nondeterminism, as the root hashes of the Merkle trees will not match even when all parties are honest. We therefore demonstrate a rounding-based procedure (Section 5.3), as well as a method for reducing its storage cost (Section 5.4), to successfully eliminate nondeterminism challenges. We additionally show our method outperforms baselines for verifying model training.\"*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SPJKP2inWt", "reviewer": "Reviewer_DqT4", "summary": "Machine learning is increasingly compute power consuming, and as a result, clients may delegate the training to external parties with high computation power. A challenge is how to verify the external parties is training the model as promised.\n\nThis work examines a way of auditing by having a trusted third party to verify the outcome. In particular, it investigates the impact of discrepancy in hardware to the third party's auditing outcome.", "questions": "As mentioned in the weakness box, the setting really confuses me. Please elaborate/justify the scenario in the author response. Thanks!\n\nThe technical contribution of mitigating the gap due to hardware difference is interesting. I feel such a technique should have a more impactful and realistic corresponding setting in ML.", "limitations": "Limitations are adequately addressed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The overall approach of the paper is sound and the assessments are refreshing.", "weaknesses": "My main concern of the work is the motivation. In particular, while hardware nondeterminism is a problem, is it really the bottleneck to such auditing? The existence of a trusted third-party is not very obvious to me already: if it exists, we can just delegate the training to this third party; if it doesn't have the compute power to verify all training process of the trainer, then the trainer can still acts suspiciously when not audited in the round.\n\nThe setting of the work is hardly convincing to me.\n\n============= After Author Response ==================================\n\nI appreciate the response from the authors. The response cleared some of my doubts over the setting.\nComing from a ML security/privacy background, I still don't buy the existence of a trusted third party. However, I appreciate the contribution of this auditing process as a first step in a non adversarial environment. Therefore, I've raised my score accordingly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We would like to provide further clarification on the motivation of our work, which arose from real problems faced by industry partners, as the need for robust verification schemes is increasing rapidly (e.g. Together AI, which offers fine-tuning services for open source models, has more than 45,000 registered developers, but no method for these clients to verify that training on their data was performed correctly). \n\n\nConcretely, as described in Section 7, our work makes a 1-out-of-n honesty assumption for verifiable training; i.e., as long as one of n auditors is honest, any attack from a malicious trainer will be detected. This opens two possible settings for our work: \n\n1. **No one single trusted auditor (de-centralized)**: In this case, a client would outsource model training to many service providers/GPUs, and use them to audit each other. For example, Service A can provide both the model and the corresponding rounding log to a client, perhaps at a premium cost. The client can then approach Service B with this rounding log, and use B to confirm A performed the task correctly, or decide between A and B if the models differ. Different services can now compete on how available these logs are for auditing, and clients can collectively deem services that better enable auditing as more “trustworthy”. This overall mitigates the disadvantage developers who do not have access to vast training resources currently face.\nThis problem setting does indeed arise in practice, such as with companies like Gensyn or Ora, which seek to create a large, decentralized compute cluster for AI tasks like model training. In fact, Gensyn currently uses a protocol that’s main limitation is lack of control of hardware non-determinism, which our paper improves [1]. Finally, we note that recent discussions with others in industry (e.g. Atoma Network), have shown interest in applying our method for repeated model inference (e.g. text generation), where GPU nondeterminism leads to divergent outputs. Therefore, there does exist strong, real-world motivation for controlling hardware non-determinism between two parties for machine learning tasks. \n\n2. **Trusted 3rd party auditor**: This setting motivates the existing academic literature our work is based on. For example, the proof-of-learning protocol in [2] describes a setting where a legal entity serves as a verifier, and re-performs gradient computations at specific steps to verify training. Legal entities may not have the availability to provide model training as a large-scale service, but could provide auditing services at a cost to the client. In these cases, the client may choose to pursue auditing only in serious cases where they are highly suspicious of the model they received, and wish to pursue dispute resolution. Even if auditing is not performed for all training tasks, our method would allow model training service providers to build a reputation for passing/failing trusted 3rd party audits, increasing overall trust in the rapidly growing model-training-as-a-service space.\n\nWhile we chose the framing of an “auditor” to stay close to related academic works we build on, we agree with the reviewer that the broader motivation and existence of the de-centralized setting could be made clearer. We will therefore make the following additions to the paper: \n\n1. Line 34 in Introduction: An alternative ``optimistic'' approach is to consider a 3rd party verifier. This could be a trusted 3rd party,  such as a non-profit organization that may not have sufficient computing resources to provide model training as a service beyond auditing, or a different service provider that the client approaches and wishes to compare with the original trainer. \n2. Line 44 in Introduction: Applying verifiable computation techniques to the setting of model training is particularly important given the increase in decentralized machine learning services like Gensyn, which seek to make ML compute more accessible by creating a network of many untrusted GPUs. \n3. Replace mentions of “trusted 3rd party auditor” to “3rd party (e.g. trusted auditor or other service provider)”\n\n\n[1] https://docs.gensyn.ai/litepaper \n[2] https://arxiv.org/pdf/2103.05633", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mE2KfRCOSX", "reviewer": "Reviewer_V9Xz", "summary": "The authors present a novel approach to achieving identical results in model training across different GPU types and introduce a verifiable training scheme. To achieve this, they:\n- proposed a technique for two parties training the same model on different GPU types to achieve identical results by sharing rounding decisions.\n- presented a verifiable training scheme which uses a Merkle tree to store model weights for efficient comparison between a trainer and an auditor.\n- conducted experiments demonstrating the scalability of the approach to ResNet-50 and GPT-2 across three different NVIDIA GPU architectures (A40, Titan XP, RTX 2080 Ti).\n- proposed methods to reduce the storage cost via efficient encoding of rounding logs and an adaptive threshold mechanism to minimize the amount of rounding decisions logged.\n- compared their approach with existing methods, including proof-based systems, and the results show that their approach is storage and time-efficient.", "questions": "Please attend to weaknesses above.", "limitations": "The authors present the limitation of their work in the limitation section.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper introduces a novel method for achieving identical training results across different GPU types by sharing rounding decisions, which could significantly enhance reproducibility in machine learning. The use of a verifiable training scheme based on a well-established verification game adds a layer of trust and transparency to the training process, making it more reliable for sensitive applications. The methods to reduce storage costs via efficient encoding of rounding logs and an adaptive threshold mechanism address practical concerns related to resource usage. This in itself is a huge contribution.\n- The experiments using foundation models like ResNet-50 and GPT-2 across multiple GPU architectures showcases the robustness and practicality of the proposed approach. The paper also provides thorough comparisons with existing methods, highlighting the improved storage and time efficiency of the proposed approach, which strengthens the case for its adoption.", "weaknesses": "- The authors implemented their verifiable training method entirely on top of the PyTorch framework, using torch version 1.13.1. Given that PyTorch has since released version 2.3.1, there may be compatibility issues or inaccuracies if the method is implemented on the updated version, i.e., using an older version could affect the claim that their method achieves perfect training replication of the two used models, if someone else tries to implement their approach using the updated version, or another framework.\n- While the paper includes a comparison with existing methods, the authors assume certain metrics from the baselines due to the unavailability of specific information. This may affect the fairness of the comparisons, especially as the reported improvements might only be valid for the given scenarios (and would be different for other scenarios).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We address concerns regarding pytorch version and fairness comparisons below. \n\n**PyTorch Version**\nWe have re-run our experiments with PyTorch 2.3.1, and can confirm our method achieves perfect training replications both within the new version, and between versions. This intuitively makes sense: we do not use any PyTorch methods that would likely change in a way that compromises our method’s effectiveness. Beyond standard methods used for model training, we rely on the following 2 specific PyTorch functions: \n\n1. torch.nextafter: This returns the next floating point after the given input for a given direction. Since this is a static property of the input’s data type (e.g. 32-bit float), this should not change between versions. Even if it did, this part of our code could be replaced by a hand-written function using knowledge of the IEEE floating point standard. \n2. module.register_forward_hook : This registers a global forward hook that will be called in every forward pass. This function is widely used in ML interpretability research, as it allows one to examine internal layers of a neural network. We therefore do not expect its core functionality to change in future updates. Even if it did, this part of our code could be replaced by re-writing rounding logic as a model layer. \n\nWhile we implement our method in PyTorch for convenience, because GPU nondeterminism is fundamentally due to floating point non-associativity, we do not expect updates in PyTorch to compromise our ability to replicate training between GPU devices.. \n\n\n**Fairness Comparison**\nThank you for your feedback, and we agree that some prior works omit information to provide an exact comparison. For this reason, we have been intentionally overly conservative in our comparisons, and believe the reported benefit of our method is a *lower* bound on the actual improvement. Specifically: \n\n1. Garg. et. al., 2023 do not report the time required for the offline phase of their method. Our method improves on theirs without even including this offline phase. Whether their offline phase takes 0 seconds or more, our method is more time efficient. Furthermore, their work uses techniques specific to logistic regression, so it is not possible to extend their reported time to other training settings. \n2. Abbaszadeh et. al., 2024 state that they use a simplified version of VGG-11 with fewer parameters. The cost of their method, like ours, would increase with the number of parameters, as that increases the number of computation steps. We report results for the full VGG-11 model, so matching their model would only improve the benefit our method provides.\n\nUnfortunately, there exist very few approaches that scale to large-scale training tasks. However, we strongly believe that the scale of the gains of our method over proof-based alternatives would hold across many settings, as proof-based systems are fundamentally difficult to scale.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their valuable feedback to improve our work. Reviewers found our work addresses an important problem (RB9A), our technical contribution interesting (DQT4), and that our proposed method is both robust and practical in the context of large-scale foundation models (V9XZ). 

We address each reviewer’s specific concerns in the individual responses.

Meta-review: This work proposes an algorithm for verifying the correct training of a machine learning model in a setting of this training be delegated to an external service provider. This work uses several techniques to reduce computation, control for hardware nondeterminism, and save storage cost.

Overall, there is consensus that this work provides a novel contribution that is sound and improves over a well chosen set of benchmarks. There is some limitation in the practicality of this approach in that this work makes the strong assumption of a trusted third party. However, given that this is an assumption that has been used several times before, it is not unfounded.
Decision: Accept (poster)