Optimistic Verifiable Training by Controlling Hardware Nondeterminism

Authors: megha  srivastava
simran  arora
dan  boneh
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 11:23:28

Abstract: The increasing compute demands of AI systems has led to the emergence of services that train models on behalf of clients lacking necessary resources. However, ensuring correctness of training and guarding against potential training-time attacks, such as data poisoning and backdoors, poses challenges. Existing works on verifiable training largely fall into two classes: proof-based systems, which can be difficult to scale, and ``optimistic'' methods that consider a trusted third-party auditor who replicates the training process. A key challenge with the latter is that hardware nondeterminism between GPU types during training prevents an auditor from replicating the training process exactly, and such schemes are therefore non-robust. We propose a method that combines training in a higher precision than the target model, rounding after intermediate computation steps, and storing rounding decisions based on an adaptive thresholding procedure, to successfully control for  nondeterminism. Across three different NVIDIA GPUs (A40, Titan XP, RTX 2080 Ti), we achieve exact training replication at FP32 precision for both full-training and fine-tuning of ResNet-50 (23M) and GPT-2 (117M) models. Our  verifiable training scheme significantly decreases the storage and time costs compared to proof-based systems.

Main paper: 
 introduction: 
 We are currently in the "large-scale era" of machine learning (ML), where the exciting capabilities of modern AI systems have required a dramatic increase in training compute needs [Sevilla et al., 2022] . In turn, several model training services, such as Replicate, OpenAI's Finetuning API, Together AI, Amazon Sagemaker, MosaicML Training, and Gensyn, have been created to support clients who lack the resources to train a model themselves. However, these services require clients to place a significant degree of trust in them to train the model correctly, without introducing a training-time attack such as data poisoning or undetectable backdoors [Wan et al., 2023 , Goldwasser et al., 2022] . How can we help a client, such as an individual or a small company, hold the service provider accountable in case of misbehavior during training?Consider an education start-up that wishes to finetune the Llama-70b language model (70B parameters) on their own curated dataset to support student learning. This task requires significant resources, and the company might even lack the necessary expertise. Instead, they might choose to pay a trainer with vast computing resources to perform the training task (Figure 1 ). However, what if the trainer adds data points that spread misinformation, introduces a backdoor that advances a political agenda for specific prompts, or tries to save work by under-training the model? If the client starts to notice suspicious model behavior, is there any action they can take? We study this problem of verifiable training, or ensuring that the training of an ML model was performed correctly. (4) share randomness, architecture, and rounding logs Figure 1 : Overview of our scheme. After an auditor challenges a trainer, they train the model, storing weights in a Merkle tree, and enter a binary search procedure to identify the exact steps of the dispute. We show how to control GPU nondeterminism between auditor and trainer, expanding the set of potential auditors.One possibility is for the trainer to provide the client with a cryptographic proof that the model was trained according to the specification. However, proof-based systems require cryptographic techniques that can be difficult to scale to the complexity of real-world ML systems. For instance, recent work based on zero-knowledge proof systems for verifiable inference, a much simpler task than training, requires more than 8 minutes to generate proofs for only 20 images [Liu et al., 2021] . Thus, practical proof-based methods for verifiable training have only been implemented for simple tasks such as logistic and linear regression [Garg et al., 2023 , Ames et al., 2022 ].An alternative "optimistic" approach is to consider a third-pary auditor (Figure 1 ). This could be a trusted 3rd party, such as a non-profit organization that may not have sufficient computing resources to provide training as a service beyond auditing, or a different provider that the client approaches and wishes to compare with the original model trainer. When a client suspects foul play, they can ask the auditor to challenge the trainer by training the model using the auditor's own compute, and demonstrate that the trainer did not train correctly. Based on the provided evidence required from the auditor (i.e. the precise timesteps model training diverged, as shown in Figure 1 ), the client can then choose to refuse the trainer's model, pursue legal action against the trainer, or even dispute a potentially corrupt auditor if the client deems such evidence as incorrect, or another auditor disagrees. This protocol can be efficiently carried out using techniques from the literature on verifiable computing, such as the "verification game" method of Teutsch and Reitwießner [2019] , which uses an interactive binary-search procedure to identify the exact intermediate computation step (e.g. training epoch) where the two parties diverged. Applying verifiable computation techniques to model training is particularly important given the increase in decentralized machine learning services like Gensyn, which seek to make ML compute more accessible by creating a network of many untrusted GPUs.Unfortunately, the issue with such "optimistic" approaches is nondeterminism during model training: two models trained on different GPU types, even with same data order and random seed, can learn different weights (Figure 2 ). The auditor cannot simply compare their model weights with the trainer's, and recent work has shown that protocols based on comparing model weights, such as Jia et al. [2021] 's "proof of learning," are not robust and can be forged due to errors from nondeterminism [Thudi et al., 2022 , Fang et al., 2023] .Our work addresses this limitation by asking: can the trainer provide additional information to the auditor that removes the effects of hardware nondeterminism? Our starting point is the observation that hardware nondeterminism occurs due to the accumulation of errors from floating point operations.For example, a matrix-vector multiply often results in different floating point values on different GPUs, since GPUs often accumulate in different orders. To address this issue, a natural approach is to perform training using a higher precision (e.g. FP32) than the target precision of the model weights (e.g. FP16), and periodically round back to the target precision. The hope is that all floating point errors will be confined to the higher precision bits, so that the rounded values are deterministic. However, this fails because computed values can occasionally straddle the "rounding boundary": i.e., the trainer can round up while the auditor rounds down, quickly causing them to diverge. Instead, we propose a solution where the trainer records the rounding direction for certain intermediate computation so that auditor can stay in sync with the trainer. As this requires the trainer to record a large number of bits, we also show how to reduce the amount of data needed to eliminate errors.We use this strategy to adapt the verification game described by Teutsch and Reitwießner [2019] for verifiable training. The game's efficiency lies in our ability to store hashes of model checkpoints in a Merkle tree [Merkle, 1988] . To determine if training was performed according to the specification, the auditor needs to reconstruct the Merkle tree and compare the resulting Merkle root hash with the Merkle root hash provided by the trainer's -if they do not match, the two parties can enter an interactive binary search procedure to identify the exact training step of the dispute. The purpose of the binary search game is to hold both parties accountable: an auditor should not be able to simply claim that a model was improperly trained, but convince a third-party (e.g., the public, or a judge) by showing at what point during training the trainer misbehaved. We show our verifiable training scheme can scale to tasks such as full training of ResNet-50 (23M parameters) and finetuning of GPT-2 (117M parameters), significantly outperforming existing methods with respect to both time and storage cost, while eliminating statistical error due to non-determinism. For example, the proposal in prior work Jia et al. [2021] would require > 140× more storage cost than our method by comparing model weights at every step in order to achieve low (yet still non-zero) statistical error.Concretely, our contributions include: (1) A method for two parties, training the same model on different GPU types, to achieve identical training results by logging and sharing rounding decisions;(2) A verifiable training scheme based on the verification game from Teutsch and Reitwießner [2019] , which stores model weights in a Merkle tree for efficient comparison between a trainer and auditor;(3) Experiments showing the ability of our approach to scale to large models such as ResNet-50 and GPT-2 between three different NVIDIA GPU architectures (A40, Titan XP, RTX 2080 Ti); (4) Methods to reduce the storage cost of our approach via efficient encoding of rounding logs and an adaptive threshold mechanism to reduce the amount of rounding decisions logged; and (5) Comparisons with existing methods, including proof-based systems, that highlight the improved storage and time efficiency of our method.2  
related works: 
 Without any verifiable training scheme in place, significant trust is placed in the trainer, leaving a client vulnerable to many different attacks, such as the "poisoning" of data samples to cause undesirable behavior (e.g., generating unsafe text [Carlini et al., 2023 , Koh et al., 2021 , Wan et al., 2023] ) and planting backdoors triggered by certain inputs [Goldwasser et al., 2022] . Therefore, training ML models in trusted environments has been an exciting direction explored by many researchers. One line of work consists of proof-based systems, where a proof of correctness (for a desired specification) is provided using cryptographic techniques such as succinct non-interactive arguments (SNARKs) [Micali, 1994 , Bitansky et al., 2012 , Lee et al., 2020 , Liu et al., 2021 , Garg et al., 2023 , Kang et al., 2022] . However, even the most recent proof-based systems for verifiable training suffer extreme latencies, such as 22 minutes for training VGG-11 on one batch of 16 data inputs [Abbaszadeh et al., 2024] , and have therefore primarily been developed for simpler models (e.g., logistic regression) that are less likely to be delegated to others in the first place [Garg et al., 2023 , Ames et al., 2022] . Meanwhile, an alternative solution of training models in a trusted execution environment (TEE), such as NVIDIA's H100, incurs a performance penalty due to the cost of running inside a TEE [Dhanuskodi et al., 2023] . Furthermore, clients lose all security guarantees if an attacker can extract the attestation key from even one GPU [Nilsson et al., 2020 , Bulck et al., 2018] .Our approach is most similar to proof-of-learning protocols, which consider a trusted 3rd party that compares checkpointing during the course of training with the original training sequence [Jia et al., 2021] . However, such methods not only incur high storage cost by requiring model weights to be stored frequently, but are non-robust due to errors from training nondeterminism. Several works have shown that proof-of-learning protocols can be spoofed and fail to verify correctness in several important contexts [Fang et al., 2023 , Kong et al., 2023 , Thudi et al., 2022] . Although Choi et al. [2023] recently proposed a verification procedure that is immune to several known proof-of-learning attacks, their method is not only limited to supervised learning algorithms, but also based on an assumption that models temporarily overfit data during training, which may not always hold true.GPU Nondeterminism: Prior work has investigated software patches for deterministic training, for instance by enforcing FP accumulation ordering, at a significant cost to efficiency Jooybar et al. [2013] , Defour and Collange [2015] , Chou et al. [2020] , TensorFlow [2021], Zhuang et al. [2021] . While these options address deterministic computation on a single GPU architecture, achieving deterministic results across multiple GPU architectures remains challenging Crane [2018a] , NVIDIA [2022] . We control hardware nondeterminism across GPUs in order to design an efficient and reliable verifiable training scheme. However, our method's impact extends beyond verifiable training, as training nondeterminism can have several negative consequences including bias, reproducibility, and downstream effects on ML pipelines [Zhuang et al., 2021 , Crane, 2018b , Srivastava et al., 2020] .3 Set-Up: The Verification Game Our method for verifiable training is based on the interactive verification game proposed by Teutsch and Reitwießner [2019] in the context of blockchains. The core idea is to resolve a dispute between a challenger, in our case the auditor, and a solver, in our case the trainer, for an expensive computation (e.g., model training). In order for the auditor to take any meaningful action (e.g., pursue legal action), they need to prove the exact source of the dispute (e.g., training time-step where an attack occurred). If we can save model weights at different time steps into a compact data structure such as a Merkle tree, then identifying the source of disagreement can be done efficiently using binary search [Merkle, 1988] . More precisely, the verification game consists of the following parties:1. trainer, who has putatively trained a model according to a client's specifications. In our example, this is a service provider with sufficient compute power to train a model. 2. client, who receives a model from the trainer and approaches an auditor. 3. auditor, who officially challenges the trainer on behalf of a client. This is a trusted 3rd-party that has sufficient resources but does not necessarily provide training as a service. The client can choose several auditors to audit the trainer's model. 4. judge: Sometimes a judge may need to arbitrate a legal claim. The judge can only perform minimal computations (e.g., one training epoch), but can examine the auditor's claims and enforce a penalty against either the trainer, for incorrect training, or the auditor, for a false alarm.When the trainer is approached by an auditor, they would need to share training parameters, model architecture, and randomness, as shown in Figure 1 . The auditor would then replicate the training process, storing model weights in a Merkle tree at the same checkpointing interval as the trainer (every leaf node in a Merkle tree is a hash of the data and every non-leaf node is a hash of its children).The main loop of the verification game starts when both parties have the root of their respective Merkle trees. If training was performed correctly, then the trainer's root should match the auditor's.Otherwise, a binary search procedure is performed, where the auditor iteratively descends the Merkle tree until it identifies two consecutive leaf nodes, i and i + 1, where the hash at i matches that of the trainer, but the hash at leaf i + 1 does not. This identifies the point in the computation of the dispute.This interactive verification game requires the cooperation of the trainer. If the trainer refuses to share the value at a certain node of their Merkle tree within a given time frame, they can be considered to have failed the audit. Additionally, the trainer and auditor use a Merkle tree to store model weights, requiring far less storage than prior work, if correct training produces identical weights (and Table 1 : Two examples of floating point accumulation error when rounding arithmetic performed higher precision (e.g. FP32) down to lower precision (e.g. FP16). In the second example, the error in the FP32 result transfers to the rounded FP16 result.identical hash values).The problem is that training nondeterminism leads to weight divergence, and causes this verification game to always fail. This why we seek to prevent divergence in training.In a GPU, a single operation (called a kernel) is executed by thousands of threads in parallel. GPUs contain a set of streaming multiprocessors (SMs), which run the thread blocks required for the kernel. At the hardware level, these blocks are divided into warps that are assigned to the available cores. Because different GPUs have a different number and size of compute units, applications partition arithmetic workloads (e.g., batch matrix multiplies) differently to achieve high performance NVIDIA [2022] , thus changing the order of FP operations. 
memory hierarchy and variable delays:: 
 The time taken for memory access by each thread depends on the physical location of the data, which can create variable delays Jooybar et al. [2013] , Defour and Collange [2015] , Chou et al. [2020] . The GPU memory hierarchy consists of large amounts of high bandwidth memory (HBM) and small amounts of fast SRAM memory, and maintains an L1 and L2 cache to improve access times. The caches sizes and access times differ across GPU architectures (e.g. an NVIDIA A100 has 192KB / 40 MB of L1/L2 cache memory, while the H100 has 256KB / 50MB). This affects warp scheduling, leading to changes in operation ordering resulting in nondeterminism. Finally, to compute primitives such as GEMMs (D = A • B + C), the workhorse of machine learning, GPUs split the work of computing the tiles of D across a thread block NVIDIA [2023] , resulting in nondeterminism that a robust verification method needs to control.5 Method Overview 
the nondeterminism challenge: 
 Although there are user-side controls for forcing deterministic operations within a single GPU architecture , these controls do not prevent nondeterminism between GPU architectures (e.g., NVIDIA H100 and V100), where trained models can have similar aggregate performance (e.g., accuracy) yet yield very different predictions, as shown in Figure 2 Crane [2018a], NVIDIA [2022] . There are three main sources of nondeterminism between GPU types:1. Floating-Point Arithmetic: Computers represent real values using integer and FP representations, typically the IEEE 754 standard (Figure 5 ). There is a tradeoff between the approximation fidelity and the # of bits used to represent the real values. The chosen precision controls the representable numerical range (e.g., 32-bit FP values can represent values between 1.17549435e -38 and 3.40282347e + 38). Because computers round to representable FP values, changing the order in which FP values are accumulated can change the resulting sum Kahan [1965] , Whitehead and Fit-Florea [2011] . Over the course of the many operations during training, this can lead to a large difference in the end result between the trainer and auditor. 
accumulation errors start at higher precision bits: 
 Our key idea is that if nondeterminism of training between GPU types occurs due to FP operations, then any error will initially be introduced in the lower bits. Suppose that both trainer and auditor train at a higher FP (e.g., b tr = 64) precision than the client's target model precision (e.g., b m = 32) and then periodically round (e.g., b r = 32) after intermediate computation steps (e.g., a convolution layer). One might hope that this will "erase" the errors due to nondeterminism, and prevent them from accumulating. Unfortunately, simply rounding to the nearest FP32 after each computation during training is insufficient for determinism. The problem is due to rounding errors that straddle What if the trainer records their rounding choice (e.g., up, down, none) for every intermediate computation?The auditor could then copy the trainer's choice, and therefore round to the exact same value and successfully control for nondeterminism. However, the auditor should not copy the trainer's behavior for every output (see Cases B & C, Figure 3 ). If a computation output on GPU 1 is too close to the rounded value, then it is possible that GPU 2 is also close in distance but from the opposite direction. In this case, the auditor should ignore the trainer's choice. We therefore need to introduce a threshold τ under which the trainer does not record their rounding choice.Our method requires upper bounding the divergence d div between any two different GPUs for any intermediate computation f (i.e. difference in outputs for the same input). Let ϵ b represent the distance between two FP32 values, after rounding to b r bits of the mantissa (Figure 5 ) and controlling for the exponent. We need to select b r and τ such that d div < ϵ br and d div < 2τ (Figure 3 ). Because the set of possible FP numbers is finite, there exist optimal bounds for b r and τ . In practice, we find that b r ≤ 32 and τ > 0.25 • ϵ 32 are sufficient for standard intermediate computations in neural network training (e.g., convolution, layer norm) in FP64. We study different values for b r in Section 6.We assume both trainer and auditor train models using the IEEE-754 standard FP numbers (Figure 5 ). Besides requiring read and write disk I/O operations, we define the following functions:1. rnd br (x): rounds input x to the nearest FP up to b r bits of the mantissa, as shown in Figure 5 . identifies the optimal threshold to log rounding directions (0 or 2) instead of 1, which the rev function ignores, based on the binary search procedure in Section 5.4. 5. hash sha256 (θ): creates a SHA-256 hash of provided model weights θ (in b m precision). 6. tree(leaf 1 , leaf 2 ..., leaf n ) : create a Merkle tree where each leaf node is the output of hash sha256 (θ)for model weights θ at a given checkpoint, with a checkpointing interval k [Merkle, 1988] .The trainer's task begins when a client approaches them with dataset D, training specifications (epochs E, loss function loss, etc.), and a requested model precision b m .The trainer can then choose a training precision b tr > b m , a rounding amount b r ≤ b m , and a checkpointing interval k to periodically store small hash sha256 (θ) of model weights θ in a Merkle tree, for efficient comparison with an eventual auditor. Then, as detailed in Algorithm 1, the trainer can perform training as normal, but after every intermediate computation (e.g., convolution) perform the rnd br operation on each output. Rounding is applied to computations in both the forward and backward passes. Finally, either using a fixed threshold τ or a layer-specific optimal τ from the threshold function described in Section 5.4, the trainer applies log, which logs rounding choices only for the computations an auditor should copy. The output of the algorithm includes a rounding log file F and the root of the Merkle tree which, along with the shared randomness R and all training parameters, the trainer can share with any trusted third-pary auditor who challenges them.After a client approaches them, the auditor initiates the verification game described in Section 3. To avoid penalty, the trainer must cooperate by sharing the rounding amount b r , randomness R used in training (e.g., a pseudo-random number generator), the checkpointing interval k, and set of rounding logs F . The auditor then follows the training procedure and corrects their rounding choice (e.g., up or down) to match those logged in F using the rev operation, as detailed in Algorithm 2 (Appendix). By correcting each rounding mismatch during the course of training, the auditor is able to prevent nondeterminism errors from accumulating. Therefore, the auditor can store the hash sha256 (θ) of model weights θ in a Merkle tree at interval k, knowing that if training was done correctly, the model weights should be identical to the trainer's at any timestep. The output of Algorithm 2 is the root of the auditor's Merkle tree, which they can use to compare with the trainer's root.Logging rounding decisions for every neural network layer output during training incurs a large baseline storage cost, and is our main limitation. For dataset D, batch size B, training epochs E, and model layers L θ , the upper bound on the total storage cost for verifiable training with our method is:EQUATIONwhere o l,f and o l,f represent the size of outputs of the forward pass and backward pass of layer l. Note that the log entries do not need to be kept around in the RAM and can be written straight to the disk. Moreover, this cost is a one-time cost incurred by the trainer, who in our context is likely to be a powerful commercial provider with access to such storage capacity. Furthermore, as we later show in Section 6, for models with many linear layers like Transformer-based language models (e.g., GPT-2), where parameters significantly outnumber intermediate computations, this storage cost is significantly smaller than alternative approaches that require saving model weights [Jia et al., 2021] . Nevertheless, we now describe our method for reducing storage cost by (i) efficiently encoding rounding logs and (ii) adaptive selection of the threshold τ to reduce the storage costs.Efficient Encoding: Each log entry is a value from the set 0, 1, 2, as opposed to the FP model weights.We pack sub-sequences of five log entries into a single byte via a fast GPU-based radix-3 to radix-2 conversion, yielding 1.6 bits/entry storage that is close to the best possible packing of 1.58 bits/entry, and yields a 77% storage reduction relative to naively storing one log entry per byte.Adaptive Threshold: Recall that we need to select a threshold τ that controls for whether the trainer logs a rounding choice, or instead logs 1 which the auditor ignores. The more one can increase τ , the more 1 values are recorded, which can make rounding logs more compressible (due to long sequences of 1s). Furthermore, it is possible that the divergence d div between outputs on two different GPUs, given the same input, is function-specific. For example, while convolution requires several matrix multiplications that might result in a large FP accumulation error, normalization operations are unlikely to result in large d div , and a larger τ can be applied. We develop an efficient algorithm (Algorithm 3 in the Appendix) to find the optimal value for τ given a particular layer and data of output values that led to different rounding choices between any two GPUs (e.g., Case A in Figure 3 ).  
empirical results: 
 We evaluate our verifiable training method on the two large-scale models listed below with all possible trainer and auditor pairs across NVIDIA GPUs A40, TITAN Xp, and RTX 2080 Ti (see Appendix B for more details). In Section 6.2, we compare our method with recent proof-based systems. Figure 2 shows that nondeterminism due to GPU architecture exists for both tasks. While we can repeatedly obtain identical results across training runs on the same GPU architecture, training on different GPU architectures results in fundamentally different models.We implement our verifiable training method entirely on top of the pytorch framework, with torch version 1.13.1 and CUDA version 11.7. The intermediate computations we apply rnd b to are layers (e.g., torch.nn.Conv2D) in the model's computation graph. Rounding-related operations (rnd and rev) either using casting or FP functions (e.g., torch.nextafter) that can run on the GPU, thus having little impact on computational speed. Because we observed that the torch.randn operation used for dropout in GPT-2 is non-deterministic for long inputs (even for the same seed, see Appendix I), we implement our own dropout as our method requires shared randomness R.Successful control for non-determinism: Our method completely eliminates non-determinism between full training runs of both for both the ResNet-50 training and GPT-2 fine-tuning tasks across all possible trainer and auditor pairs between the A40, Titan XP, and RTX 2080 Ti GPUs. As Figure 4 shows, standard FP32 training results in an increasing divergence (l2-distance of weights) between models on different GPUs over the course of training. Furthermore, we show the simple approach of training in FP64 and rounding to FP32 after every intermediate computation, but without the auditor correcting rounding decisions with rev, fails to mitigate this issue. Only our method, in which the auditor follows the rounding decisions (b r = 32) made by the trainer for every intermediate computation, eliminates non-determinism and persists over time. Our implementation, which requires disk I/O during training to store the rounding decisions, results in a small increase in training time for the trainer (1.2-1.4x) and auditor (1.3-1.7x) using a non-optimized, protoype implementation (Table 5 ). We report the storage requirements of our method in Table 2 , showing 4 ). We observe that smaller values of b do allow more log entries to be ignored, improving compression of the file, which we discuss next.15 ± 3 6 ± 2 3 ± 1 3 ± 1 0 0 0 Backward 10 ± 0.6 6 ± 0.6 2 ± 1 0.7 ± 0.7 0 ± 0 0 ± 0 0 ± 0 GPT-2 b = 32 b = 31 b = 30 b = 29 b = 28 b = 27 b = 26 Forward 2 ± 0.7 2.3 ± 0.8 2.2 ± 0.4 0.2 ± 0.2 0.4 ± 0.2 0 ± 0 0 ± 0 Backward 19 ± 13 0.75 ± 0.3 1.2 ± 0.4 0.2 ± 0.2 0. ± 0.0 0 ± 0 0 ± 0Compression with adaptive threshold: Our approach outperforms (Table 2 ) the storage costs of proof-of-learning protocols that save model weights for GPT-2 (2.78GB), which has many linear layers -we observe more than 140x reduction relative to the approach in Jia et al. [2021] . We further reduce the storage cost of our method by decreasing the rounding amount b and implementing the adaptive thresholding strategy (Section 5.4). Table 4 reports adaptive thresholds τ for four different pytorch layers at rounding amount b r = 32. Convolutions require the lowest τ , indicating larger divergence in outputs between GPU types, which is expected due to the large # of matrix multiplications. Meanwhile, τ is higher for normalization layers, likely due to smaller divergences between GPU types. Because adaptive thresholding seeks to reduce the # of times rounding decisions (0 and 2) are logged and improve log file compression, we report storage cost after zip compression in Table 2 . As expected, more aggressive rounding (which results in a higher τ ) improves the compression rate. Although the compression gains are mild in comparison to our encoding step, they build-up over the course of training. Finally, we report the average # of rev corrections an auditor needs to perform for one training step in our two tasks (Table 3 ). These values are surprisingly small in comparison to the # of operations logged -only a maximum of 2e-6% (ResNet-50) and 9e-6% (GPT-2) of logged values, are actually needed by the auditor! We also observe that severe rounding (e.g., b = 27) completely eliminated the hardware non-determinism for our tasks, requiring no corrections from the auditor. This shows a huge gap between the # of values currently saved by the trainer and those needed by the auditor, motivating an exciting future possibility of significantly reducing the storage cost of our method if we could reliably predict when a divergence will not occur.Logistic Regression: Garg et al. [2023] recently proposed a zero-knowledge proof-based system for verifiable training of a logistic regression, which importantly does not leak information about the client's data or require a trusted third-party auditor, unlike our work. However, since verifiable training itself is motivated by a client not having sufficient resources to train the model, it is crucial to consider the implications of scale. The authors report the prover time and proof size requirements for one training pass of logistic regression on a dataset of 2 18 items, with 1024 dimensions and a batch size of 2014, as 72 seconds (training and proof generation time) and 350 MB respectively. We replicate this training task, and find that our method significantly improves upon both storage and time requirements, requiring only 106 KB and 7 seconds (both training and auditing). Furthermore, because Garg et al. [2023] do not report the duration of "offline phase" of their method, their reported value is a lower bound on the actual time required. Finally, we note that the original proof-of-learning protocol from Jia et al. [2021] , which also considers a trusted third-party, would require 9.2 MB per training step to store all model weights. Therefore, our method is at least 85x more space efficient.  
security analysis: 
 Our work makes a 1-of-n honesty assumption, i.e., as long as one of n auditors is honest, any attack from a malicious trainer that results in diverging model weights will be detected. One consideration is the potential manipulation of the rounding logs by an adversarial trainer who could select rounding decisions that achieve a desired outcome, and which the auditor would follow. Concretely, let us define our threat model so that the trainer knows an auditor's GPU a priori. Recall that an auditor only copies the trainer's rounding decision in Case A in Figure 3 , when both GPUs compute values close to the rounding boundary. Under this threat model, the trainer can identify the n steps where the auditor is close to the boundary (as in Case A), enumerate the set of 2 n different models that result from different rounding decisions, and selectively pick a model that exhibits a desired property.However, the trainer cannot use this strategy to embed an arbitrary property (e.g., a specific backdoor).It can only select from the set of models that differ in certain rounding decisions, which all require the trainer to use the correct training specifications accepted by the client (such as exact training data & hyperparameters). Furthermore, since the expected # of divergences between the trainer and the auditor is extremely small (see Table 3 ), the set of possible models where an auditor would not detect an attack (e.g., many rev ops) is limited. Finally, we show in Table 6 in the appendix that the divergence (measured both as ℓ 2 -norm between model weights and output distributions) due to GPU non-determinism is significantly less than the divergence due to data ordering during training. Therefore, if a client will accept a model trained with any random ordering of the data during training, then it is unlikely that an adversarial trainer -that can only alter rounding decisions -could produce a model that the client would not accept. Nevertheless, fully understanding the model properties obtained by manipulating rounding logs adversarially is an important future direction. 
limitations and future work: 
 Our verifiable training scheme successfully controls for hardware nondeterminism. It expands the pool of potential auditors of a model training service, allowing us to envision a world where a client can even use two competing service providers it trusts to audit each other. Relative to proof-based systems, a limitation is the need for all parties to trust the third-party auditor. If the trainer provides finetuning services on top of closed-source models (e.g., OpenAI), then our scheme will only work for the third-party auditors that the trainer is willing to share model weights with. Other limitations included the added latency of training in higher precision and the storage cost. While we have shown that our method requires significantly less storage than alternatives, the vast majority of stored rounding decisions are not used by the auditor and are therefore unnecessary (Section 6). Therefore, an exciting direction for future work is to mitigate this gap by better predicting when GPU divergence between computations occurs. Recent work has similarly argued for a stronger profile of noise during training in the context of verification [Fang et al., 2023] . Finally, another direction for future work includes adapting our method for distributed training [Li et al., 2020] . 1 1 0 1 1 0 0 1 1 0 0 1 1 … 1 1 0 1 1 0 0 1 1 0 0 1 0 1 1 0 1 1 0 0 1 1 0 0 0 0 Exponent 8See Figure 5 .All experiments reported in our paper are run with the following three GPUs:• NVIDIA Titan XP: 3840 Cores, 12 GB • NVIDIA RTX 2080 Ti: 4352 Cores, 11 GB • NVIDIA A40: 10752 Cores, 48 GBWe are able to successfully replicate training runs between all pairs of these 3 GPUs.See Algorithm 4See Algorithm 1.See Algorithm 2.See Algorithm 3.See Table 5 .See Table 6 .Our verifiable training scheme requires shared randomness between the trainer and auditor, which is used for deciding input data batching, weight initialization, and operations such as dropout (randomly setting outputs to zero). More formally, our scheme requires sharing the same random seed and pseudo-random generator. However, in our implementation based on pytorch (assuming the same software version between trainer and auditor), we chose to rely on the the torch random seed functionality. While this successfully controls for batch input ordering and weight initialization, it is unfortunately not sufficient for random number generation, as operations such as torch.nn.randn() leverage parallelism when the requested # of values is higher than a certain amount. Specifically, we found that across T40, RTX 2080 Ti, V100, A40, and A100, given the same seed, torch.randint() produces identical tensors onlt up to size 40960. At size 40961, T40 (which is an older GPU) deviated from the rest. Likewise, at size 69633, 2080 Ti deviated from the rest, and so on. Based on these observations, we arranged for calls to torch.randint() in the dropout layer (which is the only operation using large random tensors in our tasks) to be replaced by generating and concatenating multiple random tensors of size 40960 or less. Specifically, a random tensor of size n > 40960 is generated by concatenating (n//40960) random tensors of size 40960 and one random tensor of size (n%40960). However, we emphasize that it is therefore important in our scheme either for both parties to implement this change a priori, or simply use an external source for pseudorandomness.The previously discussed proof-based systems for verifiable training by-pass the need for a thirdparty auditor, but very few efficient systems exist in the literature. Many more works study secure inference of deep neural networks, which could be used to construct verifiable training protocols with stronger security guarantees than ours (e.g., allowing a trainer to keep a proprietary model's weights private), but come at a significant cost to performance and resources. To demonstrate this, we consider adapting Gupta et al. for layer l θ ∈ W θ .layers do 7:output ← l θ (input) GP U 1, GP U 2 ← select two different GPU architectures 5:8: τ ← threshold(l θ , b r , b tr ) //x ← select random input for layer l in b tr floating-point precision 6: Guidelines:y 1 ← l GP U 1 (x), y 2 ← l GP U 2 (x),• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [NA] Justification: [NA] Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: [NA] Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.Our method is implemented entirely within the pytorch framework (compatible with version 2.3.1), and is available at https://github.com/meghabyte/verifiable-training. 
 
 Appendix:  


Decision: This work proposes an algorithm for verifying the correct training of a machine learning model in a setting of this training be delegated to an external service provider. This work uses several techniques to reduce computation, control for hardware nondeterminism, and save storage cost.

Overall, there is consensus that this work provides a novel contribution that is sound and improves over a well chosen set of benchmarks. There is some limitation in the practicality of this approach in that this work makes the strong assumption of a trusted third party. However, given that this is an assumption that has been used several times before, it is not unfounded.

Reviews: review: paper_summary: paper_summary: This paper studies the problem of verifying the correctness of the training process of model. In particular, the user who lacks sufficient resources pays a service provider with sufficient resources to train models. Then a trusted third-party auditor will check whether the training process is legit. The proposed approach rounds after intermediate computation steps, and stores rounding decisions based on an adaptive thresholding procedure, to successfully control for nondeterminism among different GPUs and also saves the storage space. Empirical results show that the proposed approach can run more efficiently than the previous baselines while also using smaller storage space, which makes it valuable for verifying larger models.
strength_weakness: strength_weakness: There is a lack of description on the relation and difference of this work and Teutsch & Reitwießner (2019). I think the authors will benefit by clarifying the difference and as well as emphasizing their novelty.
questions: questions: There is no direct outstanding questions, but I would encourage the authors to focus on the writing to highlight the contribution more explicitly.
limitations: limitations: Limitations are well addressed in this work.
paper_summary: paper_summary: Machine learning is increasingly compute power consuming, and as a result, clients may delegate the training to external parties with high computation power. A challenge is how to verify the external parties is training the model as promised.

This work examines a way of auditing by having a trusted third party to verify the outcome. In particular, it investigates the impact of discrepancy in hardware to the third party's auditing outcome.
strength_weakness: strength_weakness: My main concern of the work is the motivation. In particular, while hardware nondeterminism is a problem, is it really the bottleneck to such auditing? The existence of a trusted third-party is not very obvious to me already: if it exists, we can just delegate the training to this third party; if it doesn't have the compute power to verify all training process of the trainer, then the trainer can still acts suspiciously when not audited in the round.

The setting of the work is hardly convincing to me.

============= After Author Response ==================================

I appreciate the response from the authors. The response cleared some of my doubts over the setting.
Coming from a ML security/privacy background, I still don't buy the existence of a trusted third party. However, I appreciate the contribution of this auditing process as a first step in a non adversarial environment. Therefore, I've raised my score accordingly.
questions: questions: As mentioned in the weakness box, the setting really confuses me. Please elaborate/justify the scenario in the author response. Thanks!

The technical contribution of mitigating the gap due to hardware difference is interesting. I feel such a technique should have a more impactful and realistic corresponding setting in ML.
limitations: limitations: Limitations are adequately addressed.
paper_summary: paper_summary: The authors present a novel approach to achieving identical results in model training across different GPU types and introduce a verifiable training scheme. To achieve this, they:
- proposed a technique for two parties training the same model on different GPU types to achieve identical results by sharing rounding decisions.
- presented a verifiable training scheme which uses a Merkle tree to store model weights for efficient comparison between a trainer and an auditor.
- conducted experiments demonstrating the scalability of the approach to ResNet-50 and GPT-2 across three different NVIDIA GPU architectures (A40, Titan XP, RTX 2080 Ti).
- proposed methods to reduce the storage cost via efficient encoding of rounding logs and an adaptive threshold mechanism to minimize the amount of rounding decisions logged.
- compared their approach with existing methods, including proof-based systems, and the results show that their approach is storage and time-efficient.
strength_weakness: strength_weakness: - The authors implemented their verifiable training method entirely on top of the PyTorch framework, using torch version 1.13.1. Given that PyTorch has since released version 2.3.1, there may be compatibility issues or inaccuracies if the method is implemented on the updated version, i.e., using an older version could affect the claim that their method achieves perfect training replication of the two used models, if someone else tries to implement their approach using the updated version, or another framework.
- While the paper includes a comparison with existing methods, the authors assume certain metrics from the baselines due to the unavailability of specific information. This may affect the fairness of the comparisons, especially as the reported improvements might only be valid for the given scenarios (and would be different for other scenarios).
questions: questions: Please attend to weaknesses above.
limitations: limitations: The authors present the limitation of their work in the limitation section.

Comments: title: Thanks for the response
comment: Thank you! We will update the paper with our result of achieving training replicability with PyTorch 2.3.1. We would like to note that none of the values in our Figures changed with this update, as our method achieves 0 distance between model weights, and that result stays the same.
Thanks for the response. Please incorporate the promised changes in the final version.
I've updated my score accordingly.
The authors claimed that they re-run their experiments with PyTorch 2.3.1. I would suggest that they add the results to the paper to further show that their method is framework agnostic. I appreciate that they did further experiments to show this. Kudos!