Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: S LALOM : F AST , V ERIFIABLE AND P RIVATE E XECUTION OF N EURAL N ETWORKS IN T RUSTED H ARDWARE
Revised: S LALOM : F AST , V ERIFIABLE AND P RIVATE E XECUTION OF N EURAL N ETWORKS IN T RUSTED H ARDWARE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations.
Revised: As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations.

Original: A pragmatic solution comes from Trusted Execution Environments (TEEs), which use hardware and software protections to isolate sensitive computations from the untrusted software stack.
Revised: A pragmatic solution comes from Trusted Execution Environments (TEEs), which use hardware and software protections to isolate sensitive computations from the untrusted software stack.

Original: However, these isolation guarantees come at a price in performance, compared to untrusted alternatives.
Revised: However, these isolation guarantees come at a price in performance, compared to untrusted alternatives.

Original: This paper initiates the study of high performance execution of Deep Neural Networks (DNNs) in
Revised: This paper initiates the study of high performance execution of Deep Neural Networks (DNNs) in

Original: TEEs by efﬁciently partitioning DNN computations between trusted and untrusted devices.
Revised: TEEs by efﬁciently partitioning DNN computations between trusted and untrusted devices.

Original: Building upon an efﬁcient outsourcing scheme for matrix multiplication, we propose Slalom , a framework that securely delegates execution of all linear layers in a DNN from a TEE (e.g.,
Revised: Building upon an efﬁcient outsourcing scheme for matrix multiplication, we propose Slalom , a framework that securely delegates execution of all linear layers in a DNN from a TEE (e.g.,

Original: Intel SGX or Sanctum) to a faster, yet untrusted, co-located processor.
Revised: Intel SGX or Sanctum) to a faster, yet untrusted, co-located processor.

Original: We evaluate Slalom by executing DNNs in an Intel SGX enclave, which selectively delegates work to an untrusted GPU.
Revised: We evaluate Slalom by running DNNs in an Intel SGX enclave, which selectively delegates work to an untrusted GPU.

Original: Fortwo canonical DNNs, VGG16 and MobileNet, we obtain20 × and 6 ×  increases in throughput for veriﬁable inference, and11 × and 4 ×  for veriﬁable and private inference.
Revised: For canonical DNNs (VGG16, MobileNet and ResNet variants) we obtain 6 × to 20 × increases in throughput for veriﬁable inference, and 4 × to 11 × for veriﬁable and private inference.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Machine learning is increasingly used in sensitive decision making and security-critical settings.
Revised: Machine learning is increasingly used in sensitive decision making and security-critical settings.

Original: At the same time, the growth in both cloud offerings and software stack complexity widens the attack surface for ML applications.
Revised: At the same time, the growth in both cloud offerings and software stack complexity widens the attack surface for ML applications.

Original: This raises the question of integrity and privacy guarantees for ML computations in untrusted environments, in particular for ML tasks outsourced by a client to a remote server.
Revised: This raises the question of integrity and privacy guarantees for ML computations in untrusted environments, in particular for ML tasks outsourced by a client to a remote server.

Original: Prominent examples include cloud-based ML
Revised: Prominent examples include cloud-based ML

Original: APIs (e.g., a speech-to-text application that consumes user-provided data) or general ML-as-a-Service platforms.
Revised: APIs (e.g., a speech-to-text application that consumes user-provided data) or general ML-as-a-Service platforms.

Original: Trusted Execution Environments (TEEs), e.g, Intel SGX (McKeen et al., 2013), ARM TrustZone (Alves & Felton, 2004) or Sanctum (Costan et al., 2016) offer a pragmatic solution to this problem.
Revised: Trusted Execution Environments (TEEs), e.g, Intel SGX (McKeen et al., 2013), ARM TrustZone (Alves & Felton, 2004) or Sanctum (Costan et al., 2016) offer a pragmatic solution to this problem.

Original: TEEs use hardware and software protections to isolate sensitive code from other applications, while attesting to its correct execution.
Revised: TEEs use hardware and software protections to isolate sensitive code from other applications, while attesting to its correct execution.

Original: Running outsourced ML computations in TEEs provides remote clients with strong privacy and integrity guarantees.
Revised: Running outsourced ML computations in TEEs provides remote clients with strong privacy and integrity guarantees.

Original: For outsourced ML computations, TEEs outperform pure cryptographic approaches (e.g, (Gilad-Bachrach et al., 2016; Mohassel & Zhang, 2017; Ghodsi et al., 2017; Juvekar et al., 2018)) by multiple orders of magnitude.
Revised: For outsourced ML computations, TEEs outperform pure cryptographic approaches (e.g, (Gilad-Bachrach et al., 2016; Mohassel & Zhang, 2017; Ghodsi et al., 2017; Juvekar et al., 2018)) by multiple orders of magnitude.

Original: At the same time, the isolation guarantees of TEEs still come at a steep price in performance, compared to untrusted alternatives (i.e., running ML models on contemporary hardware with no security guarantees).
Revised: At the same time, the isolation guarantees of TEEs still come at a steep price in performance, compared to untrusted alternatives (i.e., running ML models on contemporary hardware with no security guarantees).

Original: For instance,
Revised: For instance,

Original: Intel SGX (Intel Corp., 2015) incurs signiﬁcant overhead for memory intensive tasks (Orenbach et al., 2017;
Revised: Intel SGX (Intel Corp., 2015) incurs signiﬁcant overhead for memory intensive tasks (Orenbach et al., 2017;

Original: Harnik & Tsfadia, 2017), has difﬁculties exploiting multi-threading, and is currently limited to desktop CPUs that are outmatched by untrusted alternatives (e.g., GPUs or server CPUs).
Revised: Harnik & Tsfadia, 2017), has difﬁculties exploiting multi-threading, and is currently limited to desktop CPUs that are outmatched by untrusted alternatives (e.g., GPUs or server CPUs).

Original: Thus, our thesis is that for modern ML workloads, TEEs will be at least an order of magnitude less efﬁcient than the best available untrusted hardware .
Revised: Thus, our thesis is that for modern ML workloads, TEEs will be at least an order of magnitude less efﬁcient than the best available untrusted hardware .

Original: This leads us to the main question of this paper: How can we most efﬁciently leverage TEEs for secure machine learning?
Revised: This leads us to the main question of this paper: How can we most efﬁciently leverage TEEs for secure machine learning?

Original: This was posed by Stoica et al.
Revised: This was posed by Stoica et al.

Original: (2017) as one of nine open research problems for system challenges
Revised: (2017) as one of nine open research problems for system challenges

Original: Under review as a conference paper at ICLRin AI.
Revised: Under review as a conference paper at ICLRin AI.

Original: A speciﬁc challenge they raised is that of appropriately splitting ML computations between trusted and untrusted components, to increase efﬁciency as well as security by minimizing the Trusted ComputingBase.
Revised: A speciﬁc challenge they raised is that of appropriately splitting ML computations between trusted and untrusted components, to increase efﬁciency as well as security by minimizing the Trusted Computing

Original:  This paper explores a novel approach to this challenge, wherein a Deep Neural Network (DNN) execution is partially outsourced from a TEE to a co-located, untrusted but faster device .
Revised: Base. This paper explores a novel approach to this challenge, wherein a Deep Neural Network (DNN) execution is partially outsourced from a TEE to a co-located, untrusted but faster device .

Original: Our approach, inspired by the veriﬁable ASICs of Wahby et al. , differs from cryptographic ML outsourcing.
Revised: Our approach, inspired by the veriﬁable ASICs of Wahby et al. (2016), differs from cryptographic ML outsourcing.

Original: In our case, work is delegated between two co-located parties, thus allowing for highly interactive—yet conceptually simpler— outsourcing protocols with orders-of-magnitude better efﬁciency.
Revised: In our case, work is delegated between two co-located parties, thus allowing for highly interactive—yet conceptually simpler— outsourcing protocols with orders-of-magnitude better efﬁciency.

Original: Our work also deviates from prior systems which execute DNNs fully in a TEE (Ohrimenko et al., 2016;
Revised: Our work also departs from prior systems that execute DNNs fully in a TEE Ohrimenko et al., 2016;

Original: Hunt et al., 2018; Cheng et al., 2018).
Revised: Hunt et al., 2018; Cheng et al., 2018; Hanzlik et al., 2018)

Original: The main observation that guides our approach is that matrix multiplication—the main bottleneck in DNNs— admits a concretely efﬁcient veriﬁable outsourcing scheme known as Freivalds’ algorithm (Freivalds, 1977), which can also be turned private in our setting.
Revised: The main observation that guides our approach is that matrix multiplication—the main bottleneck in DNNs— admits a concretely efﬁcient veriﬁable outsourcing scheme known as Freivalds’ algorithm (Freivalds, 1977) , which can also be turned private in our setting.

Original: Our TEE selectively outsources these CPU intensive steps to a fast untrusted co-processor (and runs the remaining steps itself) therefore achieving much better performance than running the entire computation in the enclave, without compromising security.
Revised: Our TEE selectively outsources these CPU intensive steps to a fast untrusted co-processor (and runs the remaining steps itself) therefore achieving much better performance than running the entire computation in the enclave, without compromising security

Original: Contributions.
Revised: Contributions.

Original: We propose Slalom, a framework for efﬁcient DNN inference in any trusted execution environment (e.g., SGX or Sanctum).
Revised: We propose Slalom, a framework for efﬁcient DNN inference in any trusted execution environment (e.g., SGX or Sanctum).

Original: To evaluate Slalom, we build a lightweight DNN library for Intel SGX, which may be of independent interest.
Revised: To evaluate Slalom, we build a lightweight DNN library for Intel SGX, which may be of independent interest.

Original: Our library allows for outsourcing all linear layers to an untrusted GPU without compromising integrity or privacy.
Revised: Our library allows for outsourcing all linear layers to an untrusted GPU without compromising integrity or privacy.

Original: Our code is available at https://redacted-for-submission .
Revised: Our code is available at https://redacted-for-submission .

Original: We formally prove Slalom’s security, and evaluate it on two DNNs— VGG16 (Simonyan & Zisserman, 2014) and MobileNet (Howard et al., 2017)— that lie at two extremes of the computational efﬁciency spectrum.
Revised: We formally prove Slalom’s security, and evaluate it on multiple canonical DNNs with a variety of computational costs—VGG16 (Simonyan & Zisserman, 2014), MobileNet (Howard et al., 2017), and ResNets (He et al., 2016).

Original: Compared to running all computations in SGX, outsourcing linear layers to an untrusted GPU increases throughput (and energy efﬁciency to a similar extent) by 6 - 20 × for veriﬁable inference, and by 4 - 11 × for veriﬁable and private inference.
Revised: Compared to running all computations in SGX, outsourcing linear layers to an untrusted GPU increases throughput (as well as energy efﬁciency) by 6 × to 20 × for veriﬁable inference, and by 4 × to 11 × for veriﬁable and private inference.

Original: Finally, we discuss open challenges towards efﬁcient veriﬁable training of DNNs in TEEs.
Revised: Finally, we discuss open challenges towards efﬁcient veriﬁable training of DNNs in TEEs.

Original:  B ACKGROUND 2. P ROBLEM S ETTING
Revised:  B ACKGROUND 2. P ROBLEM S ETTING

Original: We consider an outsourcing scheme between a client C and a server S , where S executes a DNN F ( x ) :
Revised: We consider an outsourcing scheme between a client C and a server S , where S executes a DNN F ( x ) :

Original: X → Y on data provided by C .
Revised: X → Y on data provided by C .

Original: The DNN can either belong to the user (e.g., as in some ML-as-a-service platforms), or to the server (e.g., as in a cloud-based ML API).
Revised: The DNN can either belong to the user (e.g., as in some ML-as-a-service platforms), or to the server (e.g., as in a cloud-based ML API).

Original: Depending on the application, this scheme should satisfy one or more of the following security properties (see Appendix B for formal deﬁnitions):
Revised: Depending on the application, this scheme should satisfy one or more of the following security properties (see Appendix B for formal deﬁnitions):

Original: • t-Integrity: For any S and input x , the probability that a user interacting with S does not abort (i
Revised: • t-Integrity: For any S and input x , the probability that a user interacting with S does not abort (i

Original: e., output ⊥ ) and outputs an incorrect value ˜ y (cid:54) =
Revised: e., output ⊥ ) and outputs an incorrect value ˜ y (cid:54) =

Original: F ( x ) is less than t .
Revised: F ( x ) is less than t .

Original: • Privacy: The server S learns no information about the user’s input x . • Model privacy: If the model F is provided by the user, S learns no information about F (beyond e.g., its approximate size).
Revised: • Privacy: The server S learns no information about the user’s input x . • Model privacy: If the model F is provided by the user, S learns no information about F (beyond e.g., its approximate size).

Original: If F belongs to the server, C learns no more about F than what is revealed by y = F ( x ) . 
Revised: If F belongs to the server, C learns no more about F than what is revealed by y = F ( x ) . T

Original: 2. T RUSTEDE XECUTION E NVIRONMENTS (TEE S ), I NTEL SGX, AND A S TRONG B ASELINE
Revised: RUSTED

Revised: E XECUTION E NVIRONMENTS (TEE S ), I NTEL SGX, AND A S TRONG B ASELINE

Original: Trusted Execution Environments (TEE) such as Intel SGX, ARM TrustZone or Sanctum (Costan et al., 2016) enable execution of programs in secure enclaves .
Revised: Trusted Execution Environments (TEE) such as Intel SGX, ARM TrustZone or Sanctum (Costan et al., 2016) enable execution of programs in secure enclaves .

Original: Hardware protections isolate computations in enclaves from all programs on the same host, including the operating system.
Revised: Hardware protections isolate computations in enclaves from all programs on the same host, including the operating system.

Original: Enclaves can produce remote attestations —digital
Revised: Enclaves can produce remote attestations —digital

Original: [Table]
Revised: [Table]

Original: signatures over an enclave’s code—that a remote party can verify using the manufacturer’s public key.
Revised: signatures over an enclave’s code—that a remote party can verify using the manufacturer’s public key.

Original: Our experiments with Slalom use hardware enclaves provided by Intel SGX (see Appendix A for details).
Revised: Our experiments with Slalom use hardware enclaves provided by Intel SGX (see Appendix A for details).

Original: TEEs offer an efﬁcient solution for ML outsourcing: The server runs anisolated enclave that establishes a secure communicationchannel with C and evaluates a model F on C ’s input data.
Revised: TEEs offer an efﬁcient solution for ML outsourcing: The server runs an enclave that initiates a secure communication with C and evaluates a model F on C ’s input data.

Original: This simple scheme (which we implemented in SGX, see Section 4) outperforms cryptographic ML outsourcing protocols by 2 - 3 orders of magnitude.
Revised: This simple scheme (which we implemented in SGX, see Section 4) outperforms cryptographic ML outsourcing protocols by 2 - 3 orders of magnitude (albeit under a different trust model).

Original: See Table 1  for a comparison to two representative works, and Appendix C for more details.
Revised: See Table 1 and Appendix C for a comparison to two representative works.

Original: Yet, SGX’s security comes at a performance cost, and there remains a large gap between TEEs and untrusted devices.
Revised: Yet, SGX’s security comes at a performance cost, and there remains a large gap between TEEs and untrusted devices.

Original: For example, current SGX CPUs are limited to 128 MB of Processor Reserved Memory (PRM) (Costan & Devadas, 2016) and incur severe paging overheads when exceeding this allowance (Orenbach et al., 2017).
Revised: For example, current SGX CPUs are limited to 128 MB of Processor Reserved Memory (PRM) (Costan & Devadas, 2016) and incur severe paging overheads when exceeding this allowance (Orenbach et al., 2017).

Original: We also failed to achieve noticeable speed ups for multi-threaded DNN evaluations in SGX enclaves (see
Revised: We also failed to achieve noticeable speed ups for multi-threaded DNN evaluations in SGX enclaves (see

Original: Appendix I).
Revised: Appendix H).

Original: For DNN computations, current SGX enclaves thus cannot compete—in terms of performance or energy efﬁciency (see Appendix C)—with contemporary untrusted hardware, such as a GPU or server CPU.
Revised: For DNN computations, current SGX enclaves thus cannot compete—in terms of performance or energy efﬁciency (see Appendix C)—with contemporary untrusted hardware, such as a GPU or server CPU.

Original: In this work, we treat the above simple (yet powerful)
Revised: In this work, we treat the above simple (yet powerful)

Original: TEE scheme as a baseline, and identify settings where we can still improve upon it.
Revised: TEE scheme as a baseline, and identify settings where we can still improve upon it.

Original: We will show that our system, Slalom, substantially outperforms this baseline when the server has access to the model F (e.g., F belongs to S as in cloud ML APIs, or F is public).
Revised: We will show that our system, Slalom, substantially outperforms this baseline when the server has access to the model F (e.g., F belongs to S as in cloud ML APIs, or F is public).

Original: Slalom performs best for veriﬁable inference (the setting considered in SafetyNets (Ghodsi et al., 2017)).
Revised: Slalom performs best for veriﬁable inference (the setting considered in SafetyNets (Ghodsi et al., 2017)).

Original: If the TEE can run some ofﬂine data-independent preprocessing (e.g., as in Gazelle (Juvekar et al., 2018)), Slalom also outperforms the baseline for private (and veriﬁable) outsourced computations in a later online phase.
Revised: If the TEE can run some ofﬂine data-independent preprocessing (e.g., as in Gazelle (Juvekar et al., 2018)), Slalom also outperforms the baseline for private (and veriﬁable) outsourced computations in a later online phase.

Original: Such a two-stage approach is viable if user data is sent at irregular intervals yet has to be processed with high throughput when available.
Revised: Such a two-stage approach is viable if user data is sent at irregular intervals yet has to be processed with high throughput when available.

Original: 2. O UTSOURCING O UTSOURCED DNN S AND F REIVALDS ’ A LGORITHM
Revised: 2. O UTSOURCING O UTSOURCED DNN S AND F REIVALDS ’ A LGORITHM

Original: Our idea for speeding up DNN inference in TEEs is to further outsource work from the TEE to a co-located faster untrusted processor .
Revised: Our idea for speeding up DNN inference in TEEs is to further outsource work from the TEE to a co-located faster untrusted processor .

Original: Improving upon the above baseline thus requires that the combined cost of doing work on the untrusted device and verifying it in the TEE be cheaper than evaluating the full DNN in the TEE.
Revised: Improving upon the above baseline thus requires that the combined cost of doing work on the untrusted device and verifying it in the TEE be cheaper than evaluating the full DNN in the TEE.

Original: Wahby et al.
Revised: Wahby et al.

Original: (2016; 2017) aim at this goal for arbitrary computations outsourced between co-located ASICs.
Revised: (2016; 2017) aim at this goal for arbitrary computations outsourced between co-located ASICs.

Original: The generic non-interactive proofs they use for integrity are similar to those used in SafetyNets (Ghodsi et al., 2017), which incur overheads that are too large to warrant outsourcing in our setting (e.g., Wahby et al.
Revised: The generic non-interactive proofs they use for integrity are similar to those used in SafetyNets (Ghodsi et al., 2017), which incur overheads that are too large to warrant outsourcing in our setting (e.g., Wahby et al.

Original: ﬁnd that the technology gap between trusted and untrusted devices needs to be of over two decades for their scheme to break even).
Revised: ﬁnd that the technology gap between trusted and untrusted devices needs to be of over two decades for their scheme to break even).

Original: Similarly for privacy, standard cryptographic outsourcing protocols (e.g., (Juvekar et al., 2018)) are unusable in our setting as simply running the computation in the TEE is much more efﬁcient (see Table 1).
Revised: Similarly for privacy, standard cryptographic outsourcing protocols (e.g., (Juvekar et al., 2018)) are unusable in our setting as simply running the computation in the TEE is much more efﬁcient (see Table 1).

Original: To overcome this barrier, we design outsourcing protocols tailored to DNNs, leveraging two insights:
Revised: To overcome this barrier, we design outsourcing protocols tailored to DNNs, leveraging two insights:

Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: In our setting, the TEE is co-located with the server’s faster untrusted processors, thus widening the design space to interactive outsourcing protocols with high communication but better efﬁciency.
Revised: In our setting, the TEE is co-located with the server’s faster untrusted processors, thus widening the design space to interactive outsourcing protocols with high communication but better efﬁciency.

Original: The TEE always has knowledge of the model and can selectively outsource part of the DNN evaluation and compute others—for which outsourcing is harder—itself.
Revised: The TEE always has knowledge of the model and can selectively outsource part of the DNN evaluation and compute others—for which outsourcing is harder—itself.

Original: DNNs are a class of functions that are particularly well suited for selective outsourcing.
Revised: DNNs are a class of functions that are particularly well suited for selective outsourcing.

Original: Indeed, non-linearities— which are hard to securely outsource (with integrity or privacy)—represent a small fraction of the computation in a DNN so we can evaluate these in the TEE (e.g., for VGG16 inference on a single CPU thread, about 1 .
Revised: Indeed, non-linearities— which are hard to securely outsource (with integrity or privacy)—represent a small fraction of the computation in a DNN so we can evaluate these in the TEE (e.g., for VGG16 inference on a single CPU thread, about 1 .

Original: 5% of the computation is spent on non-linearities).
Revised: 5% of the computation is spent on non-linearities).

Original: In contrast, linear operators—the main computational bottleneck in
Revised: In contrast, linear operators—the main computational bottleneck in

Original: DNNs—admit for a conceptually simple yet concretely efﬁcient secure delegation scheme, described below.
Revised: DNNs—admit for a conceptually simple yet concretely efﬁcient secure delegation scheme, described below.

Original: Integrity.
Revised: Integrity.

Original: We verify integrity of outsourced linear layers using variants of an algorithm by Freivalds (1977).
Revised: We verify integrity of outsourced linear layers using variants of an algorithm by Freivalds (1977).

Original: Lemma 2.1 (Freivalds) .
Revised: Lemma 2.1 (Freivalds) .

Original: Let A, B and C be n × n matrices over a ﬁeld F and let s be a uniformly random vector in S n , for S ⊆ F .
Revised: Let A, B and C be n × n matrices over a ﬁeld F and let s be a uniformly random vector in S n , for S ⊆ F .

Original: Then, Pr s R ←− S n [ Cs = A ( Bs ) | C (cid:54) =
Revised: Then, Pr[ Cs = A ( Bs ) | C (cid:54) =

Original: AB ] =
Revised: AB ] =

Original: Pr s R ←− S n [( C − AB ) s = 

Revised: Pr[( C − AB ) s =

Original: | ( C − AB ) (cid:54) = 0 ] ≤ 1 / | S | .
Revised: | ( C − AB ) (cid:54) = 0 ] ≤ 1 / | S | .

Original: The randomized check requires 3 n 2 multiplications, a signiﬁcant reduction (both in concrete terms and asymptotically) over evaluating the product directly.
Revised: The randomized check requires 3 n 2 multiplications, a signiﬁcant reduction (both in concrete terms and asymptotically) over evaluating the product directly.

Original: The algorithm has no false negatives and trivially extends to rectangular matrices.
Revised: The algorithm has no false negatives and trivially extends to rectangular matrices.

Original: Independently repeating the check k times yields soundness error 1 / | S | k .
Revised: Independently repeating the check k times yields soundness error 1 / | S | k .

Original: Privacy.
Revised: Privacy.

Original: Input privacy for outsourced linear operators could be achieved with linearly homomorphic encryption, but the overhead (see the micro-benchmarks in (Juvekar et al., 2018)) is too high to compete with our baseline (i.e., computing the function directly in the TEE would be faster than outsourcing it over encrypted data).
Revised: Input privacy for outsourced linear operators could be achieved with linearly homomorphic encryption, but the overhead (see the micro-benchmarks in (Juvekar et al., 2018)) is too high to compete with our baseline (i.e., computing the function directly in the TEE would be faster than outsourcing it over encrypted data).

Original: We instead propose a very efﬁcient two-stage approach based on symmetric cryptography, i.e., an additive stream cipher.
Revised: We instead propose a very efﬁcient two-stage approach based on symmetric cryptography, i.e., an additive stream cipher.

Original: Let f : F m → F n be a linear function over a ﬁeld F .
Revised: Let f : F m → F n be a linear function over a ﬁeld F .

Original: In an ofﬂine phase, the TEE generates a stream of one-time-use pseudorandom elements r ∈
Revised: In an ofﬂine phase, the TEE generates a stream of one-time-use pseudorandom elements r ∈

Original: F m , and pre-computes u = f ( r ) .
Revised: F m , and pre-computes u = f ( r ) .

Original: Then, in the online phase when the remote client sends an input x , the TEE computes Enc ( x ) = x + r over F m (i.e., a secure encryption of x with a stream cipher), and outsources the computation of f ( Enc ( x )) to the faster processor.
Revised: Then, in the online phase when the remote client sends an input x , the TEE computes Enc ( x ) = x + r over F m (i.e., a secure encryption of x with a stream cipher), and outsources the computation of f ( Enc ( x )) to the faster processor.

Original: Given the result f
Revised: Given the result f

Original: ( Enc ( x ))
Revised: ( Enc ( x ))

Original: =
Revised: =

Original: f ( x + r ) = f ( x ) + f ( r ) =
Revised: f ( x + r ) = f ( x ) + f ( r ) =

Original: f
Revised: f

Original: ( x ) + u , the TEE recovers f
Revised: ( x ) + u , the TEE recovers f

Original: ( x ) using the pre-computed u .
Revised: ( x ) using the pre-computed u .

Original: Communication.
Revised: Communication.

Original: Using Freivalds’ algorithm and symmetric encryption for each linear layer in a DNN incurs high interaction and communication between the TEE and untrusted co-processor (e.g., over 50 MB per inference for VGG16, see Table 3).
Revised: Using Freivalds’ algorithm and symmetric encryption for each linear layer in a DNN incurs high interaction and communication between the TEE and untrusted co-processor (e.g., over 50 MB per inference for VGG16, see Table 3).

Original: This would be prohibitive if they were not co-located.
Revised: This would be prohibitive if they were not co-located.

Original: There are protocols with lower communication than repeatedly using Freivalds’
Revised: There are protocols with lower communication than repeatedly using Freivalds’

Original: ((Fiore & Gennaro, 2012; Thaler, 2013; Ghodsi et al., 2017)).
Revised: ((Fiore & Gennaro, 2012; Thaler, 2013; Ghodsi et al., 2017)).

Original: Yet, these incur a high overhead on the prover in practice and are thus not suitable in our setting.
Revised: Yet, these incur a high overhead on the prover in practice and are thus not suitable in our setting.

Original:  S LALOM
Revised:  S LALOM

Original: We introduce Slalom, a three-step approach for outsourcing DNNs from a TEE to an untrusted but faster device: (1) Inputs and weights are quantized and embedded in a ﬁeld F ; (2) Linear layers are outsourced and veriﬁed using Freivalds’ algorithm; (3) Inputs of linear layers are encrypted with a pre-computed pseudorandom stream to guarantee privacy.
Revised: We introduce Slalom, a three-step approach for outsourcing DNNs from a TEE to an untrusted but faster device: (1) Inputs and weights are quantized and embedded in a ﬁeld F ; (2) Linear layers are outsourced and veriﬁed using Freivalds’ algorithm; (3) Inputs of linear layers are encrypted with a pre-computed pseudorandom stream to guarantee privacy.

Original: Figure 1 shows two Slalom variants, one to achieve integrity, and one to also achieve privacy.
Revised: Figure 1 shows two Slalom variants, one to achieve integrity, and one to also achieve privacy.

Original: We focus on feed-forward networks with fully connected layers, convolutions, separable convolutions, pooling layers and activations, although Slalom could be extended to other architecture (e.g., residual networks).
Revised: We focus on feed-forward networks with fully connected layers, convolutions, separable convolutions, pooling layers and activations. Slalom can be extended to other architectures (e.g., residual networks, see Section 4.3).

Original: 3. Q UANTIZATION
Revised: 3. Q UANTIZATION

Original: The techniques we use for integrity and privacy (Freivalds’ algorithm and stream ciphers) work over a ﬁeld F .
Revised: The techniques we use for integrity and privacy (Freivalds’ algorithm and stream ciphers) work over a ﬁeld F .

Original: We thus quantize all inputs and weights of a DNN to integers, and embed these integers in the ﬁeld Z p of integers modulo a prime p (where p is larger than all values computed in a DNN evaluation, so as to avoid wrap-around).
Revised: We thus quantize all inputs and weights of a DNN to integers, and embed these integers in the ﬁeld Z p of integers modulo a prime p (where p is larger than all values computed in a DNN evaluation, so as to avoid wrap-around).

Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: Slalom with integrity TEE ( F,x 1 )
Revised: Slalom with integrity TEE ( F,x 1 )

Original: S ( F ) x 1 −−→ for i ∈ [1 ,n ] do y i
Revised: S ( F ) x 1 −−→ for i ∈ [1 ,n ] do y i

Original: =
Revised: =

Original: x i W i
Revised: x i W i

Original: x i +

Revised: x i +

Original: =
Revised: =

Original: σ ( y i )
Revised: σ ( y i )

Original: y 1 ...
Revised: y 1 ...

Original: yn ←−−−−− for i ∈ [1 ,n ] do assert Freivalds ( y i ,x i ,W i ) x
Revised: yn ←−−−−− for i ∈ [1 ,n ] do assert Freivalds ( y i ,x i ,W i ) x

Original: i +

Revised: i +

Original: =
Revised: =

Original: σ ( y i ) return y n
Revised: σ ( y i ) return y n

Original: Slalom with integrity & privacy TEE ( F,x 1 )
Revised: Slalom with integrity & privacy TEE ( F,x 1 )

Original: S ( F )
Revised: S ( F )

Original: Preproc : for i ∈ [1 ,n ] do r i R ←− F mi ,u i =
Revised: Preproc : for i ∈ [1 ,n ] do r i R ←− F mi ,u i =

Original: r i W i for i ∈ [1 ,n ] do ˜ x
Revised: r i W i for i ∈ [1 ,n ] do ˜ x

Original: i = x
Revised: i = x

Original: i + r i ˜ xi
Revised: i + r i ˜ xi

Original: −−→ ˜
Revised: −−→ ˜

Original: yi
Revised: yi

Original: ←−− ˜ y
Revised: ←−− ˜ y

Original: i =
Revised: i =

Original: ˜ x
Revised: ˜ x

Original: i W i
Revised: i W i

Original: y i =
Revised: y i =

Original: ˜
Revised: ˜

Original: y
Revised: y

Original: i − u
Revised: i − u

Original: i assert Freivalds ( y i ,x i ,W i ) x
Revised: i assert Freivalds ( y i ,x i ,W i ) x

Original: i +

Revised: i +

Original: =
Revised: =

Original: σ ( y i ) return y n
Revised: σ ( y i ) return y n

Original: As in (Gupta et al., 2015), we convert ﬂoating point numbers x to a ﬁxed-point representation as ˜ x = FP ( x ; l ) :
Revised: As in (Gupta et al., 2015), we convert ﬂoating point numbers x to a ﬁxed-point representation as ˜ x = FP ( x ; l ) :

Original: =
Revised: =

Original: round (2 l · x ) .
Revised: round (2 l · x ) .

Original: For a linear layer with kernel W and bias b , we deﬁne integer parameters ˜ W =
Revised: For a linear layer with kernel W and bias b , we deﬁne integer parameters ˜ W =

Original: FP ( W, l ) , ˜ b =
Revised: FP ( W, l ) , ˜ b =

Original: FP ( b, 2 l ) .
Revised: FP ( b, 2 l ) .

Original: After applying the layer to a quantized input ˜ x , we scale the output by 2 − l and re-round to an integer.
Revised: After applying the layer to a quantized input ˜ x , we scale the output by 2 − l and re-round to an integer.

Original: For efﬁciency reasons, we perform integer arithmetic using ﬂoats (so-called fake quantization), and choose p < 2 24 to avoid loss of precision (we use p = 2 24 − 3 ).
Revised: For efﬁciency reasons, we perform integer arithmetic using ﬂoats (so-called fake quantization), and choose p < 2 24 to avoid loss of precision (we use p = 2 24 − 3 ).

Original: For the models we evaluate, setting l = 8 for all weights and inputs ensures that all DNN values are bounded by 2 24 , with less than a 0 .
Revised: For the models we evaluate, setting l = 8 for all weights and inputs ensures that all DNN values are bounded by 2 24 , with less than a 0 .

Original: 5% drop in accuracy (see Table 3).
Revised: 5% drop in accuracy (see Table 3).

Original: When performing arithmetic modulo p (e.g., for Freivalds’ algorithm or when computing on encrypted data), we use double-precision ﬂoats, to reduce the number of modular reductions required (details are in Appendix F).
Revised: When performing arithmetic modulo p (e.g., for Freivalds’ algorithm or when computing on encrypted data), we use double-precision ﬂoats, to reduce the number of modular reductions required (details are in Appendix F).

Original: 3. V ERIFYING C OMMON L INEAR O PERATORS
Revised: 3. V ERIFYING C OMMON L INEAR O PERATORS

Original: We now describe Slalom’s approach to verifying the integrity of outsourced linear layers.
Revised: We now describe Slalom’s approach to verifying the integrity of outsourced linear layers.

Original: We describe these layers in detail in Appendix D and summarize this section’s results in Table 2.
Revised: We describe these layers in detail in Appendix D and summarize this section’s results in Table 2.

Original: Freivalds’ Algorithm for Batches.
Revised: Freivalds’ Algorithm for Batches.

Original: The most direct way of applying Freivalds’ algorithm to arbitrary linear layers of a DNN is by exploiting batching.
Revised: The most direct way of applying Freivalds’ algorithm to arbitrary linear layers of a DNN is by exploiting batching.

Original: Any linear layer f ( x ) from inputs of size m to outputs of size n can be represented (with appropriate reshaping) as f ( x ) = x (cid:62) W for a (often sparse and implicit)
Revised: Any linear layer f ( x ) from inputs of size m to outputs of size n can be represented (with appropriate reshaping) as f ( x ) = x (cid:62) W for a (often sparse and implicit)

Original: m × n matrix W .
Revised: m × n matrix W .

Original: For a batch X of size B , we can outsource f ( X ) and check that the output Y satisﬁes f ( s (cid:62) X )
Revised: For a batch X of size B , we can outsource f ( X ) and check that the output Y satisﬁes f ( s (cid:62) X )

Original: =
Revised: =

Original: s (cid:62) Y , for a random vector s (we are implicitly applying Freivalds to the matrix product XW = Y ).
Revised: s (cid:62) Y , for a random vector s (we are implicitly applying Freivalds to the matrix product XW = Y ).

Original: As the batch size B grows, the cost of evaluating f is amortized and the total veriﬁcation cost is | X | + | Y | + cost f multiplications (i.e., we approach one operation per input and output).
Revised: As the batch size B grows, the cost of evaluating f is amortized and the total veriﬁcation cost is | X | + | Y | + cost f multiplications (i.e., we approach one operation per input and output).

Original: Yet, as we show in Appendix G, while batched veriﬁcation is worthwhile for processors with larger memory, it is prohibitive in SGX enclaves due to the limited PRM.
Revised: Yet, as we show in Section 4.3, while batched veriﬁcation is worthwhile for processors with larger memory, it is prohibitive in SGX enclaves due to the limited PRM.

Original: For full convolutions (and pointwise convolutions), a direct application of Freivalds’ check is worthwhile even for single-element batches.
Revised: For full convolutions (and pointwise convolutions), a direct application of Freivalds’ check is worthwhile even for single-element batches.

Original: For f ( x )
Revised: For f ( x )

Original: =
Revised: =

Original: Conv ( x, W ) and purported output y , we can sample a random vector s of dimension c out (the number of output channels), and check that Conv ( x, Ws ) =
Revised: Conv ( x, W ) and purported output y , we can sample a random vector s of dimension c out (the number of output channels), and check that Conv ( x, Ws ) =

Original: ys (with appropriate reshaping).
Revised: ys (with appropriate reshaping).

Original: For a batch of inputs X , we can also apply Freivalds’ algorithm twice to reduce both W and X .
Revised: For a batch of inputs X , we can also apply Freivalds’ algorithm twice to reduce both W and X .

Original: Preprocessing.
Revised: Preprocessing.

Original: We now show how to obtain an outsourcing scheme for linear layers that has optimal veriﬁcation complexity (i.e., | x | + | y | operations) for single-element batches and arbitrary linear operators, while at the same time compressing the DNN’s weights (a welcome property in our memory-limited TEE model).
Revised: We now show how to obtain an outsourcing scheme for linear layers that has optimal veriﬁcation complexity (i.e., | x | + | y | operations) for single-element batches and arbitrary linear operators, while at the same time compressing the DNN’s weights (a welcome property in our memory-limited TEE model).

Original: [Table]
Revised: [Table]

Original: We leverage two facts: (1) DNN weights are ﬁxed at inference time, so part of Freivalds’ check can be precomputed; (2) the TEE can keep secrets from the host S , so the random values s can be re-used across layers or inputs (if we run Freivalds’ check n times with the same secret randomness, the soundness errors grows at most by a factor n ).
Revised: We leverage two facts: (1) DNN weights are ﬁxed at inference time, so part of Freivalds’ check can be precomputed; (2) the TEE can keep secrets from the host S , so the random values s can be re-used across layers or inputs (if we run Freivalds’ check n times with the same secret randomness, the soundness errors grows at most by a factor n ).

Original: Our veriﬁcation scheme with preprocessing follows from a reformulation of Lemma (2.1):
Revised: Our veriﬁcation scheme with preprocessing follows from a reformulation of Lemma (2.1):

Original: Lemma 3.1.
Revised: Lemma 3.1.

Original: Let f : F m → F n be a linear operator, f ( x ) :
Revised: Let f : F m → F n be a linear operator, f ( x ) :

Original: = x (cid:62) W .
Revised: = x (cid:62) W .

Original: Let s be uniformly random in S n , for S ⊆ F , and let ˜ s := ∇ F x ( s ) =
Revised: Let s be uniformly random in S n , for S ⊆ F , and let ˜ s := ∇ F x ( s ) =

Original: Ws .
Revised: Ws .

Original: For any x ∈ F m , y ∈ F n , we have Pr s R ←− S n [ y (cid:62) s = x (cid:62) 
Revised: For any x ∈ F m , y ∈ F n , we have Pr (cid:2) y (cid:62) s = x (cid:62) ˜ s |

Original: ˜

Original: s | y (cid:54) =
Revised: y (cid:54) =

Original: f ( x )] ≤ 1 / | S | .
Revised: f ( x ) (cid:3) ≤ 1 / | S | .

Original: Thecost of the check is | x | + | y | multiplications, at the expense of storing s and ˜ s :
Revised: The check requires | x | + | y | multiplications, and storage for s and ˜ s :

Original: =
Revised: =

Original: Ws (of size | x | and | y | respectively).
Revised: Ws (of size | x | and | y | ).

Original: To save space, we can reuse the same  s(or slices of it) for every layer.
Revised: To save space, we can reuse the same random s for every layer.

Original: The memory footprint of a model is then equal to the size of the inputs of all its linear layers. For VGG16 for instance, the memory requirements for  the TEE are reduced from 550MB to about 36MB (see Table 3).
Revised: The memory footprint of a model is then equal to the size of the inputs of all its linear layers (e.g., for VGG16 the footprint is reduced from 550MB to 36MB, see Table 3).

Original: 3. I NPUT P RIVACY
Revised: 3. I NPUT P RIVACY

Original: To guarantee privacy of the client’s inputs, we use precomputed blinding factors for each outsourced computation, as described in Section 2.3.
Revised: To guarantee privacy of the client’s inputs, we use precomputed blinding factors for each outsourced computation, as described in Section 2.3.

Original: The TEE uses a cryptographic Pseudo Random Number Generator (PRNG) to generate blinding factors.
Revised: The TEE uses a cryptographic Pseudo Random Number Generator (PRNG) to generate blinding factors.

Original: The precomputed “unblinding factors” are encrypted and stored in untrusted memory or disk.
Revised: The precomputed “unblinding factors” are encrypted and stored in untrusted memory or disk.

Original: In the online phase, the TEE regenerates the blinding factors using the same PRNG seed, and uses the precomputed unblinding factors to decrypt the output of the outsourced linear layer.
Revised: In the online phase, the TEE regenerates the blinding factors using the same PRNG seed, and uses the precomputed unblinding factors to decrypt the output of the outsourced linear layer.

Original: This blinding process incurs several overheads: (1) the computations on the untrusted device have to be performed over Z p so we use double-precision arithmetic.
Revised: This blinding process incurs several overheads: (1) the computations on the untrusted device have to be performed over Z p so we use double-precision arithmetic.

Original: The trusted and untrusted processors exchange data in-between each layer, rather than at the end of a full inference pass.
Revised: The trusted and untrusted processors exchange data in-between each layer, rather than at the end of a full inference pass.

Original: The TEE has to efﬁciently load precomputed unblinding factors, which requires either a large amount of RAM, or a fast access to disk (e.g., a PCIe SSD).
Revised: The TEE has to efﬁciently load precomputed unblinding factors, which requires either a large amount of RAM, or a fast access to disk (e.g., a PCIe SSD).

Original: Slalom’s security is summarized by the following results.
Revised: Slalom’s security is given by the following results.

Original: Formal deﬁnitions and proofs are in Appendix B. 
Revised: Formal deﬁnitions and proofs are in Appendix B. Let negl be a negligible function (for any integer c > 0 there exists an integer N c such that for all x > N c , | negl( x ) | < 1 / x c ).

Original: Theorem 3.2.
Revised: Theorem 3.2.

Original: Let Slalom be the protocol from Figure 1 (right), where F is an n -layer DNN, and Freivalds’ algorithm is repeated k times per layer with random vectors drawn from S ⊆ F .
Revised: Let Slalom be the protocol from Figure 1 (right), where F is an n -layer DNN, and Freivalds’ algorithm is repeated k times per layer with random vectors drawn from S ⊆ F .

Original: Assume all random values are generated using a secure PRNG with security parameter λ .
Revised: Assume all random values are generated using a secure PRNG with security parameter λ .

Original: Then, Slalom is a secure outsourcing scheme for F between a TEE and an untrusted co-processor S with privacy and t -integrity for t = n / | S | k − negl( λ ) .
Revised: Then, Slalom is a secure outsourcing scheme for F between a TEE and an untrusted co-processor S with privacy and t -integrity for t = n / | S | k − negl( λ ) .

Original: Corollary 3.3.
Revised: Corollary 3.3.

Original: Assuming the TEE is secure (i.e., it acts as a trusted third party hosted by S ), Slalom is a secure outsourcing scheme between a remote client C and server S with privacy and t -integrity for t = n / | S | k − negl( λ ) .
Revised: Assuming the TEE is secure (i.e., it acts as a trusted third party hosted by S ), Slalom is a secure outsourcing scheme between a remote client C and server S with privacy and t -integrity for t = n / | S | k − negl( λ ) .

Original: If the model F is the property of S , the scheme further satisﬁes model privacy.
Revised: If the model F is the property of S , the scheme further satisﬁes model privacy.

Original:  E MPIRICAL E VALUATION
Revised:  E MPIRICAL E VALUATION

Original: We evaluate Slalom on real Intel SGX hardware, onsynthetic micro-benchmarks and two sample applications (ImageNet inference with VGG16 and MobileNet).
Revised: We evaluate Slalom on real Intel SGX hardware, on micro-benchmarks and a sample application (ImageNet inference with VGG16, MobileNet and ResNet models).

Original: Our aim is to show that, compared to a baseline that runs inference fully in the TEE, outsourcing linear layers increases performance without sacriﬁcing security.
Revised: Our aim is to show that, compared to a baseline that runs inference fully in the TEE, outsourcing linear layers increases performance without sacriﬁcing security.

Original: Under review as a conference paper
Revised: [Figure]

Original: 4. I MPLEMENTATION
Revised: 4. I MPLEMENTATION

Original: As enclaves cannot access most OS features (e.g., multi-threading, disk and driver IO), porting a large framework such as TensorFlow or Intel’s MKL-DNN to SGX is hard.
Revised: As enclaves cannot access most OS features (e.g., multi-threading, disk and driver IO), porting a large framework such as TensorFlow or Intel’s MKL-DNN to SGX is hard.

Original: Instead, we designed a lightweight C++ library for feed-forward networks based on Eigen, a linear-algebra library which TensorFlow uses as a CPU backend.
Revised: Instead, we designed a lightweight C++ library for feed-forward networks based on Eigen, a linear-algebra library which TensorFlow uses as a CPU backend.

Original: Our library implements the forward pass of DNNs, with support for dense layers, standard and separable convolutions, pooling, and activations.
Revised: Our library implements the forward pass of DNNs, with support for dense layers, standard and separable convolutions, pooling, and activations.

Original: When run on a native CPU (without SGX), its performance is comparable to TensorFlow on CPU (compiled with AVX).
Revised: When run on a native CPU (without SGX), its performance is comparable to TensorFlow on CPU (compiled with AVX).

Original: Our code is available at https://redacted-for-submission .
Revised: Our code is available at https://redacted-for-submission .

Original: Slalom performs arithmetic over Z p , for p = 2 24 − 3 .
Revised: Slalom performs arithmetic over Z p , for p = 2 24 − 3 .

Original: For integrity, we apply Freivalds’ check twice to each layer ( k = 2 ), with random values from S =
Revised: For integrity, we apply Freivalds’ check twice to each layer ( k = 2 ), with random values from S =

Original: [ − 2 19 , 2 19 ] , to achieve 40 bits of statistical soundness per layer (see
Revised: [ − 2 19 , 2 19 ] , to achieve 40 bits of statistical soundness per layer (see

Original: Appendix F for details on the selection of these parameters).
Revised: Appendix F for details on the selection of these parameters).

Original: For a 50 -layer DNN, S has a chance of less thanin 22 billion of fooling the TEE on any incorrect DNN evaluation (a slightly better guarantee than in SafetyNets).
Revised: For a 50 -layer DNN, S has a chance of less thanin 22 billion of fooling the TEE on any incorrect DNN evaluation (a slightly better guarantee than in SafetyNets).

Original: For privacy, we use AES-CTR and AES-GCM to generate, encrypt and authenticate blinding factors.
Revised: For privacy, we use AES-CTR and AES-GCM to generate, encrypt and authenticate blinding factors.

Original: 4. S ETUP
Revised: 4. S ETUP

Original: We use an Intel Core i7-6700 Skylake 3.40GHz processor with 8GB of RAM, a desktop processor with SGX support.
Revised: We use an Intel Core i7-6700 Skylake 3.40GHz processor with 8GB of RAM, a desktop processor with SGX support.

Original: The outsourced computations are performed on a co-located Nvidia TITAN XP GPU.
Revised: The outsourced computations are performed on a co-located Nvidia TITAN XP GPU.

Original: Due to a lack of native internal multi-threading in SGX, we run our TEE in a single CPU thread.
Revised: Due to a lack of native internal multi-threading in SGX, we run our TEE in a single CPU thread.

Original: We discuss challenges for efﬁcient parallelization in Appendix I. We evaluate Slalom on the following workloads:
Revised: We discuss challenges for efﬁcient parallelization in Appendix H. We evaluate Slalom on the following workloads:

Original: • Synthetic benchmarks for matrix products, convolutions and separable convolutions, where we compare the enclave’s running time for computing a linear operation to that of solely verifying the result (see Appendix G).
Revised: • Synthetic benchmarks for matrix products, convolutions and separable convolutions, where we compare the enclave’s running time for computing a linear operation to that of solely verifying the result.

Original: ImageNet
Revised: ImageNet

Original: (Deng et al ., 2009) classiﬁcation with VGG16 (Simonyan & Zisserman, 2014) and MobileNet (Howard et al., 2017) models (with fused Batch Normalization layers).
Revised: (Deng et al., 2009) classiﬁcation with VGG16 (Simonyan & Zisserman, 2014), MobileNet (Howard et al., 2017), and ResNet He et al. (2016) models (with fused Batch Normalization layers when applicable).

Original: MobileNet, a model tailored for low compute devices, serves as a worst-case benchmark for Slalom, as the model’s design aggressively minimizes the amount of computation performed per layer.
Revised: MobileNet, a model tailored for low compute devices, serves as a worst-case benchmark for Slalom, as the model’s design aggressively minimizes the amount of computation performed per layer.

Original: We also consider a “fused” variant of MobileNet with no activation between depthwise and pointwise convolutions.
Revised: We also consider a “fused” variant of MobileNet with no activation between depthwise and pointwise convolutions.

Original: Removing these activations improves convergence and accuracy (Chollet, 2017; Sheng et al., 2018), while also making the network more outsourcing-friendly (i.e., it is possible to verify a separable convolution in a single step).
Revised: Removing these activations improves convergence and accuracy (Chollet, 2017; Sheng et al., 2018), while also making the network more outsourcing-friendly (i.e., it is possible to verify a separable convolution in a single step).

Original: Our evaluation focuses on throughput (number of forward passes per second).
Revised: Our evaluation focuses on throughput (number of forward passes per second).

Original: We also discuss energy efﬁciency in Appendix C to account for hardware differences between our baseline (TEE only) and Slalom (TEE + GPU). 
Revised: We also discuss energy efﬁciency in Appendix C to account for hardware differences between our baseline (TEE only) and Slalom (TEE + GPU). Under review as a conference paper at ICLR [Figure] MobileNet

Revised: [Figure]

Original: 4. R ESULTS
Revised: 4. R ESULTS

Original: Micro-Benchmarks.
Revised: Micro-Benchmarks.

Revised: Our micro-benchmark suite consists of square matrix products of increasing dimensions, convolutional operations performed by VGG16, and separable convolutions performed by MobileNet.

Revised: In all cases, the data is pre-loaded inside an enclave, so we only measure the in-enclave execution time.

Revised: Figure 2 plots the relative speedups of various veriﬁcation strategies over the cost of computing the linear operation directly.

Revised: In all cases, the baseline computation is performed in single-precision ﬂoating point, and the veriﬁcation algorithms repeat Freivalds’ check so as to attain at least 40 bits of statistical soundness.

Original: Results of our benchmarks for verifying linear operators are in Appendix G and we summarize our main ﬁndings here: (1) Verifying outsourced matrix products and convolutions is much faster ( 3 × - 63 × ) than computing them in the TEE, especially with preprocessing as described in Section 3.2; (2) It is hard to evaluate large matrix products or to leverage batched veriﬁcation in Intel SGX, due to the limited PRM.
Revised: For square matrices of dimensions up to 2048, verifying an outsourced result is 4 × to 8 × faster than computing it. For larger matrices, we exceed the limit of SGX’s DRAM, so the enclave resorts to expensive paging which drastically reduces performance both for computation and veriﬁcation. For convolutions (standard or separable), we achieve large savings with outsourcing if Freivalds’ algorithm is applied with preprocessing. The savings get higher as the number of channels increases.

Revised: Without preprocessing,

Revised: Freivalds’ algorithm results in savings when c out is large.

Revised: Due to SGX’s small PRM, batched veriﬁcation is only effective for operators with small memory footprints.

Revised: As expected, “truly” separable convolutions (with no intermediate non-linearity) are much faster to verify, as they can be viewed as a single linear operator.

Original: Veriﬁable Inference.
Revised: Veriﬁable Inference.

Original: Figure 2 shows the throughout of end-to-end forward passes in two neural networks,
Revised: Figure 3 shows the throughout of end-to-end forward passes in two neural networks,

Original: VGG16 and MobileNet.
Revised: VGG16 and MobileNet.

Original: For integrity, we compare the secure baseline (executing the DNN fully in the enclave) to two variants of the Slalom algorithm in Figure 1.
Revised: For integrity, we compare the secure baseline (executing the DNN fully in the enclave) to two variants of the Slalom algorithm in Figure 1.

Original: The ﬁrst (in red) applies Freivalds’ algorithm “on-the-ﬂy”, while the second more efﬁcient variant (in orange) pre-computes part of Freivalds’ check as described in Section 3.2.
Revised: The ﬁrst (in red) applies Freivalds’ algorithm “on-the-ﬂy”, while the second more efﬁcient variant (in orange) pre-computes part of Freivalds’ check as described in Section 3.2.

Original: The VGG16 network is much larger (500MB) than SGX’s PRM.
Revised: The VGG16 network is much larger (500MB) than SGX’s PRM.

Original: As a result, there is a large overhead on the forward pass and veriﬁcation without preprocessing.
Revised: As a result, there is a large overhead on the forward pass and veriﬁcation without preprocessing.

Original: If the enclave securely stores preprocessed products Wr for all network weights, we drastically reduce the memory footprint and achieve up to a 20 . 3 × increase in throughput.
Revised: If the enclave securely stores preprocessed products Wr for all network weights, we drastically reduce the memory footprint and achieve up to a 20 . 3 × increase in throughput.

Original: We also ran the lower-half of the VGG16 network (without the fully connected layers), a common Under review as a conference paper at ICLR I m a g e s / s e c [Figure]
Revised: We also ran the lower-half of the VGG16 network (without the fully connected layers), a common approach for extracting features for transfer learning or object recognition (Liu et al., 2016).

Original: MobileNet

Original: [Figure]

Original: MobileNet (fused

Original: [Figure]

Original: privacy

Original: [Figure]

Original: approach for extracting features for transfer learning or object recognition (Liu et al., 2016).

Original: This part ﬁts in the
Revised: This part ﬁts in the

Original: PRM, and we thus achieve higher throughput for in-enclave forward passes and on-the-ﬂy veriﬁcation.
Revised: PRM, and we thus achieve higher throughput for in-enclave forward passes and on-the-ﬂy veriﬁcation.

Original: For MobileNet, we achieve between 3 . 6 × and 6 . 4 × speedups when using Slalom for veriﬁable inference (for the standard or “fused” model, respectively).
Revised: For MobileNet, we achieve between 3 . 6 × and 6 . 4 × speedups when using Slalom for veriﬁable inference (for the standard or “fused” model, respectively).

Original: The speedups are smaller than for VGG16, as MobileNet performs much fewer operations per layer (verifying a linear layer requires computing at least two multiplications for each input and output.
Revised: The speedups are smaller than for VGG16, as MobileNet performs much fewer operations per layer (verifying a linear layer requires computing at least two multiplications for each input and output.

Original: The closer the forward pass gets to that lower-bound, the less we can save by outsourcing).
Revised: The closer the forward pass gets to that lower-bound, the less we can save by outsourcing).

Original: Private Inference.
Revised: Private Inference.

Original: We further benchmark the cost of private DNN inference, where inputs of outsourced linear layers are additionally blinded .
Revised: We further benchmark the cost of private DNN inference, where inputs of outsourced linear layers are additionally blinded .

Original: Blinding and unblinding each layer’s inputs and outputs is costly,  especially in SGX due to the extra in-enclave memory reads and writes.
Revised: Blinding and unblinding each layer’s inputs and outputs is costly, Under review as a conference paper at ICLR [Figure] especially in SGX due to the extra in-enclave memory reads and writes.

Original: Nevertheless, for VGG16 and the fused
Revised: Nevertheless, for VGG16 and the fused

Original: MobileNet variant without intermediate activations, we achieve respective speedups of 13 . 

Revised: MobileNet variant without intermediate activations, we achieve respective speedups of 13 .

Original: × and 5 . 

Revised: × and 5 .

Original: × for private outsourcing (in black in Figure 2), and speedups of 10 .
Revised: × for private outsourcing (in black in Figure 3), and speedups of 10 .

Original: 7 × and 4 .
Revised: 7 × and 4 .

Original: 1 × when also ensuring integrity (in purple).
Revised: 1 × when also ensuring integrity (in purple).

Original: For this benchmark, the precomputed unblinding factor are stored in untrusted memory.
Revised: For this benchmark, the precomputed unblinding factor are stored in untrusted memory.

Original: We performed the same experiments on a standard CPU (i.e., without SGX) and ﬁnd that Slalom’s improvements are even higher in non-resource-constrained or multi-threaded environments (see Appendix H-I).
Revised: We performed the same experiments on a standard CPU (i.e., without SGX) and ﬁnd that Slalom’s improvements are even higher in non-resource-constrained or multi-threaded environments (see Appendix G-H).

Original: Slalom’s improvements over the baseline also hold when accounting for energy efﬁciency (see Section C).
Revised: Slalom’s improvements over the baseline also hold when accounting for energy efﬁciency (see Section C).

Revised: Extending Slalom to Deep Residual Networks.

Revised: The Slalom algorithm in Figure 1 and our evaluations above focus on feed-forward architectures.

Revised: Extending Slalom to more complex DNNs is quite simple.

Revised: To illustrate, we consider the family of ResNet models (

Original: C ONCLUSION
Revised: He et al., 2016), which use residual blocks f ( x ) = σ ( f 1 ( x ) + f 2 ( x )) that merge two feed-forward “paths” f 1 and f 2 into a ﬁnal activation σ . To verify integrity of f ( x ) , the TEE simply veriﬁes all linear layers in f 1 and f 2 and computes σ directly.

Revised: For privacy, the TEE applies the interactive Slalom protocol in Figure 1 (right) in turn to f 1 and f 2 , and then computes σ .

Revised: The results for the privacy-preserving

Revised: Slalom variant in Figure 4 use a preliminary implementation that performs all required operations—and thus provides meaningful performance numbers—but without properly constructed unblinding factors.

Revised: We use the ResNet implementation from Keras Chollet et al.

Revised: (2015), which contains a pre-trained 50 -layer variant.

Revised: For this model, we ﬁnd that our quantization scheme results in less than a 0 .

Revised: 5% decrease in accuracy (see

Revised: Table 3).

Revised: For other variants (i.e., with 18 , 34 , 101 and 152 layers) we compute throughput on untrained models.

Revised: Figure 4 shows benchmarks for different ResNet variants when executed fully in the enclave (our baseline) as well as secure outsourcing with integrity or privacy and integrity.

Revised: For all models, we achieve 6 .

Revised: 6 × to 14 .

Revised: 4 × speedups for veriﬁable inference and 4 .

Original: {+4 × to 9 . 
+}
Revised: {+4 × to 9 . 
+}

Revised: × speedups when adding privacy.

Revised: Comparing results for different models is illustrative of how Slalom’s savings scale with model size and architectural design choices.

Revised: The 18 and 34 -layer ResNets use convolutions with 3 × 3 kernels, whereas the larger models mainly use pointwise convolutions.

Revised: As shown in Table 2 verifying a convolution is about a factor k 2 · c out than computing it, which explains the higher savings for models that use convolutions with large kernel windows.

Revised: When adding more layers to a model, we expect Slalom’s speedup over the baseline to remain constant (e.g., if we duplicate each layer, the baseline computation and the veriﬁcation should both take twice as long).

Revised: Yet we ﬁnd that Slalom’s speedups usually increase as layers get added to the ResNet architecture.

Revised: This is because the deeper ResNet variants are obtained by duplicating layers towards the end of the pipeline, which have the largest number of channels and for which Slalom achieves the highest savings.

Revised:  C HALLENGES FOR V ERIFIABLE AND P RIVATE T RAINING

Revised: Our techniques for secure outsourcing of DNN inference might also apply to DNN training.

Revised: Indeed, a backward pass consists of similar linear operators as a forward pass, and can thus be veriﬁed with Freivalds’ algorithm.

Revised: Yet, applying Slalom to DNN training is challenging, as described below, and we leave this problem open.

Revised: Under review as a conference paper at ICLR

Revised: • Quantizing DNNs for training is harder than for inference, due to large changes in weight magnitudes (Micikevicius et al., 2018).

Revised: Thus, a more ﬂexible quantization scheme than the one we used would be necessary.

Revised: • Because the DNN’s weights change during training, the same preprocessed random vectors for Freivalds’ check cannot be re-used indeﬁnitely.

Revised: The most efﬁcient approach would presumably be to train with very large batches than can then be veriﬁed simultaneously.

Revised: • Finally, the pre-computation techniques we employ for protecting input privacy do not apply for training, as the weights change after every processed batch.

Revised: Moreover, Slalom does not try to hide the model weights from the untrusted processor, which might be a requirement for private training.

Revised:  C ONCLUSION

Original: This paper has studied the efﬁciency of evaluating a DNN in a Trusted Execution Environment (TEE) to provide strong integrity and privacy guarantees.
Revised: This paper has studied the efﬁciency of evaluating a DNN in a Trusted Execution Environment (TEE) to provide strong integrity and privacy guarantees.

Original: We explored new approaches for segmenting a DNN evaluation to securely outsource work from a trusted environment to a faster co-located but untrusted processor.
Revised: We explored new approaches for segmenting a DNN evaluation to securely outsource work from a trusted environment to a faster co-located but untrusted processor.

Original: We designed Slalom, a framework for efﬁcient DNN evaluation that outsources all linear layers from a TEE to a GPU.
Revised: We designed Slalom, a framework for efﬁcient DNN evaluation that outsources all linear layers from a TEE to a GPU.

Original: Slalom leverage Freivalds’ algorithm for verifying correctness of linear operators, and additionally encrypts inputs with precomputed blinding factors to preserve privacy.
Revised: Slalom leverage Freivalds’ algorithm for verifying correctness of linear operators, and additionally encrypts inputs with precomputed blinding factors to preserve privacy.

Original: Slalom can work with any TEE and we evaluated its performance using Intel SGX on various workloads.
Revised: Slalom can work with any TEE and we evaluated its performance using Intel SGX on various workloads.

Original: Fortwo canonical DNNs (VGG16 and
Revised: For canonical DNNs (VGG16, MobileNet and

Original: MobileNet), we have shown that Slalom boosts inference throughput without compromising security.
Revised: ResNet variants), we have shown that Slalom boosts inference throughput without compromising security.

Original: Securely outsourcing matrix products from a TEE has applications in ML beyond DNNs (e.g., non negative matrix factorization, dimensionality reduction, etc.)
Revised: Securely outsourcing matrix products from a TEE has applications in ML beyond DNNs (e.g., non negative matrix factorization, dimensionality reduction, etc.)

Original: In Appendix J, we also explore avenues and challenges towards applying similar techniques to DNN training, an interesting direction for future work.
Revised: We have also explored avenues and challenges towards applying similar techniques to DNN training, an interesting direction for future work.

Original: Finally, our general approach of outsourcing work from a TEE to a faster co-processor could be applied to other problems which have fast veriﬁcation algorithms, e.g., those considered in (McConnell et al., 2011; Zhang et al., 2014).
Revised: Finally, our general approach of outsourcing work from a TEE to a faster co-processor could be applied to other problems which have fast veriﬁcation algorithms, e.g., those considered in (McConnell et al., 2011; Zhang et al., 2014)

Original: A D ETAILS ON I NTEL SGX S ECURITY
Revised: A D ETAILS ON I NTEL SGX S ECURITY

Original: SGX enclaves isolate execution of a program from all other processes on a same host, including a potentially malicious OS.
Revised: SGX enclaves isolate execution of a program from all other processes on a same host, including a potentially malicious OS.

Original: In particular, enclave memory is fully encrypted and authenticated.
Revised: In particular, enclave memory is fully encrypted and authenticated.

Original: When a word is read from memory into a CPU register, a Memory Management Engine handles the decryption (Costan & Devadas, 2016).
Revised: When a word is read from memory into a CPU register, a Memory Management Engine handles the decryption (Costan & Devadas, 2016).

Original: While SGX covers many software and hardware attack vectors, there is a large and prominent class of sidechannel attacks that it explicitly does not address (Costan & Devadas, 2016; Tramèr et al., 2017).
Revised: While SGX covers many software and hardware attack vectors, there is a large and prominent class of sidechannel attacks that it explicitly does not address (Costan & Devadas, 2016; Tramèr et al., 2017).

Original: In the past years, many attacks have been proposed, with the goal of undermining privacy of enclave computations (Xu et al., 2015; Brasser et al., 2017; Moghimi et al., 2017; Götzfried et al., 2017; Van Bulck et al., 2017; Lee et al., 2017).
Revised: In the past years, many attacks have been proposed, with the goal of undermining privacy of enclave computations (Xu et al., 2015; Brasser et al., 2017; Moghimi et al., 2017; Götzfried et al., 2017; Van Bulck et al., 2017; Lee et al., 2017).

Original: Most of these attacks rely on data dependent code behavior in an enclave (e.g., branching or memory access) that can be partially observed by other processes running on the same host.
Revised: Most of these attacks rely on data dependent code behavior in an enclave (e.g., branching or memory access) that can be partially observed by other processes running on the same host.

Original: These side-channels are a minor concern for the DNN computations considered in this paper, as the standard computations in a DNN are data-oblivious (i.e., the same operations are applied regardless of the input data)
Revised: These side-channels are a minor concern for the DNN computations considered in this paper, as the standard computations in a DNN are data-oblivious (i.e., the same operations are applied regardless of the input data)

Original: (Ohrimenko et al., 2016).
Revised: (Ohrimenko et al., 2016).

Original: The recent Spectre attacks on speculative execution (Kocher et al., 2018) also prove damaging to SGX (as well as to most other processors), as recently shown (Chen et al., 2018; Dall et al., 2018; Van Bulck et al., 2018).
Revised: The recent Spectre attacks on speculative execution (Kocher et al., 2018) also prove damaging to SGX (as well as to most other processors), as recently shown (Chen et al., 2018; Dall et al., 2018; Van Bulck et al., 2018).

Original: Mitigations for these side-channel attacks are being developed (Shinde et al., 2016; Shih et al., 2017; Chen et al., 2017; Intel Corp., 2018) but a truly secure solution might require some architectural changes, e.g., as in the proposed Sanctum processor (Costan et al., 2016).
Revised: Mitigations for these side-channel attacks are being developed (Shinde et al., 2016; Shih et al., 2017; Chen et al., 2017; Intel Corp., 2018) but a truly secure solution might require some architectural changes, e.g., as in the proposed Sanctum processor (Costan et al., 2016).

Original: We refrain from formally modeling SGX’s (or other TEE’s) security in this paper, as Slalom is mostly concerned with outsourcing protocols wherein the TEE acts as a client.
Revised: We refrain from formally modeling SGX’s (or other TEE’s) security in this paper, as Slalom is mostly concerned with outsourcing protocols wherein the TEE acts as a client.

Original: We refer the interested reader to (Pass et al., 2017;
Revised: We refer the interested reader to (Pass et al., 2017;

Original: Fisch et al., 2017; Subramanyan et al., 2017) for different attempts at such formalisms.
Revised: Fisch et al., 2017; Subramanyan et al., 2017) for different attempts at such formalisms.

Revised: Under review as a conference paper at ICLR

Original: B F ORMAL S ECURITY D EFINITIONS AND P ROOFS
Revised: B F ORMAL S ECURITY D EFINITIONS AND P ROOFS

Original: We deﬁne a secure outsourcing scheme, between a client C and a server S , for a DNN F ( x ) :
Revised: We deﬁne a secure outsourcing scheme, between a client C and a server S , for a DNN F ( x ) :

Original: X → Y from some family F (e.g., all DNNs of a given size).
Revised: X → Y from some family F (e.g., all DNNs of a given size).

Original: We ﬁrst assume that the model F is known to both C and S :
Revised: We ﬁrst assume that the model F is known to both C and S :

Original: Deﬁnition B.1 (Secure Outsourcing Schemes) .
Revised: Deﬁnition B.1 (Secure Outsourcing Schemes) .

Original: A secure outsourcing scheme consists of an ofﬂine preprocessing algorithm Preproc , as well as an interactive online protocol Outsource (cid:104)C , S(cid:105) , deﬁned as follows:
Revised: A secure outsourcing scheme consists of an ofﬂine preprocessing algorithm Preproc , as well as an interactive online protocol Outsource (cid:104)C , S(cid:105) , deﬁned as follows:

Original: • st ← Preproc ( F, 1 λ ) : The preprocessing algorithm is run by C and generates some data-independent state st (e.g., cryptographic keys or precomputed values to accelerate the online outsourcing protocol.)
Revised: • st ← Preproc ( F, 1 λ ) : The preprocessing algorithm is run by C and generates some data-independent state st (e.g., cryptographic keys or precomputed values to accelerate the online outsourcing protocol.)

Original: • Y ∪ {⊥} ← Outsource (cid:104)C ( F, x, st ) , S ( F ) (cid:105) :
Revised: • Y ∪ {⊥} ← Outsource (cid:104)C ( F, x, st ) , S ( F ) (cid:105) :

Original: The online outsourcing protocol is initiated by C with inputs ( F, x, st ) .
Revised: The online outsourcing protocol is initiated by C with inputs ( F, x, st ) .

Original: At the end of the protocol, C either outputs a value y ∈ Y or aborts (i
Revised: At the end of the protocol, C either outputs a value y ∈ Y or aborts (i

Original: e., C outputs ⊥ ).
Revised: e., C outputs ⊥ ).

Original: The properties that we may require from a secure outsourcing scheme are:
Revised: The properties that we may require from a secure outsourcing scheme are:

Original: • Correctness: For any F ∈ F and x ∈ X , running st ← Preproc ( F, 1 λ ) and y ← Outsource (cid:104)C ( F, x, st ) , S ( F ) (cid:105) yields y = F ( x ) .
Revised: • Correctness: For any F ∈ F and x ∈ X , running st ← Preproc ( F, 1 λ ) and y ← Outsource (cid:104)C ( F, x, st ) , S ( F ) (cid:105) yields y = F ( x ) .

Original: • t -Integrity:
Revised: • t -Integrity:

Original: For any F ∈ F , input x ∈ X and probabilistic polynomial-time adversary S ∗ , the probability that ˜ y
Revised: For any F ∈ F , input x ∈ X and probabilistic polynomial-time adversary S ∗ , the probability that ˜ y

Original: =
Revised: =

Original: Outsource (cid:104)C ( F, x, st ) , S ∗ ( F ) (cid:105) and ˜ y / ∈ { F ( x ) , ⊥} is less than t .
Revised: Outsource (cid:104)C ( F, x, st ) , S ∗ ( F ) (cid:105) and ˜ y / ∈ { F ( x ) , ⊥} is less than t .

Original: • Input privacy: For any F ∈ F , inputs x, x (cid:48) ∈ X and probabilistic polynomial-time adversary S ∗ , the views of S ∗ in Outsource (cid:104)C ( F, x, st ) , S ∗ ( F ) (cid:105) and Outsource (cid:104)C ( F, x (cid:48) , st ) , S ∗ ( F ) (cid:105) are computationally indistinguishable.
Revised: • Input privacy: For any F ∈ F , inputs x, x (cid:48) ∈ X and probabilistic poly-time adversary S ∗ , the views of S ∗ in Outsource (cid:104)C ( F, x, st ) , S ∗ ( F ) (cid:105) and Outsource (cid:104)C ( F, x (cid:48) , st ) , S ∗ ( F ) (cid:105) are computationally indistinguishable.

Original: Efﬁciency: The  computation performed by C inthe online Outsourceprotocol should be less than the cost for C to evaluate F ∈ F .
Revised: Efﬁciency: The online computation of C in Outsource should be less than the cost for C to evaluate F ∈ F .

Original: Under review as a conference paper at ICLR

Original: Model Privacy.
Revised: Model Privacy.

Original: In some applications a secure outsourcing scheme may also require to hide the model F from either S or C (in which case that party would obviously not take F as input in the above scheme).
Revised: In some applications a secure outsourcing scheme may also require to hide the model F from either S or C (in which case that party would obviously not take F as input in the above scheme).

Original: Privacy with respect to an adversarial server S ∗ (which Slalom does not provide), is deﬁned as the indistinguishability of S ∗ ’s views in Outsource (cid:104)C ( F, x, st ) , S ∗ (cid:105) and Outsource (cid:104)C ( F (cid:48) , x, st ) , S ∗ (cid:105) for any F, F (cid:48) ∈ F .
Revised: Privacy with respect to an adversarial server S ∗ (which Slalom does not provide), is deﬁned as the indistinguishability of S ∗ ’s views in Outsource (cid:104)C ( F, x, st ) , S ∗ (cid:105) and Outsource (cid:104)C ( F (cid:48) , x, st ) , S ∗ (cid:105) for any F, F (cid:48) ∈ F .

Original: As noted in Section 2.1, a meaningful model-privacy guarantee with respect to C requires that
Revised: As noted in Section 2.1, a meaningful model-privacy guarantee with respect to C requires that

Original: S ﬁrst commit to a speciﬁc DNN F , and then convinces C that her outputs were produced with the same model as all other clients’.
Revised: S ﬁrst commit to a speciﬁc DNN F , and then convinces C that her outputs were produced with the same model as all other clients’.

Original: We refer the reader to Canetti et al.
Revised: We refer the reader to Canetti et al.

Original: for formal deﬁnitions for such commit-and-prove schemes, and to Tramèr et al.
Revised: for formal deﬁnitions for such commit-and-prove schemes, and to Tramèr et al.

Original: (2017) who show how to trivially instantiate them using a TEE.
Revised: (2017) who show how to trivially instantiate them using a TEE.

Original: Proof of Theorem 3.2.
Revised: Proof of Theorem 3.2.

Original: Let st ← Preproc and Outsource (cid:104) TEE( F, x, st ) , S(cid:105) be the outsourcing scheme deﬁned in Figure 1 (right).
Revised: Let st ← Preproc and Outsource (cid:104) TEE( F, x, st ) , S(cid:105) be the outsourcing scheme deﬁned in Figure 1 (right).

Original: We assume that all random values sampled by the TEE are produced by a secure cryptographically secure pseudorandom number generator (PRNG) (with elements in S ⊆ F for the integritycheck vectors s used in Freivalds’ algorithm, and in F for the blinding vectors r i ).
Revised: We assume that all random values sampled by the TEE are produced by a secure cryptographically secure pseudorandom number generator (PRNG) (with elements in S ⊆ F for the integritycheck vectors s used in Freivalds’ algorithm, and in F for the blinding vectors r i ).

Original: We ﬁrst consider integrity.
Revised: We ﬁrst consider integrity.

Original: Assume that the scheme is run with input x 1 and that the TEE outputs y n .
Revised: Assume that the scheme is run with input x 1 and that the TEE outputs y n .

Original: We will bound Pr[ y n (cid:54) =
Revised: We will bound Pr[ y n (cid:54) =

Original: F ( x 1 ) | y n (cid:54) =
Revised: F ( x 1 ) | y n (cid:54) =

Original: ⊥ ] .
Revised: ⊥ ] .

Original: By the security of the PRNG, we can replace the vectors s used in Freivalds’ algorithm by truly uniformly random values in S ⊆ F , via a simple hybrid argument.
Revised: By the security of the PRNG, we can replace the vectors s used in Freivalds’ algorithm by truly uniformly random values in S ⊆ F , via a simple hybrid argument.

Original: For the i -th linear layer, with operator W i , input x i and purported output y i , we then have that y i (cid:54) =
Revised: For the i -th linear layer, with operator W i , input x i and purported output y i , we then have that y i (cid:54) =

Original: x i W i with probability at most 1  | S | k .
Revised: x i W i with probability at most 1 / | S | k .

Original: By a simple union bound, we thus have that Pr[ y n (cid:54) =
Revised: By a simple union bound, we thus have that Pr[ y n (cid:54) =

Original: F ( x 1 )] ≤ n  | S | k − negl( λ ) .
Revised: F ( x 1 )] ≤ n / | S | k − negl( λ ) .

Original: Note that this bound holds even if the same (secret) random values s are re-used across layers.
Revised: Note that this bound holds even if the same (secret) random values s are re-used across layers.

Original: For privacy, consider the views of an adversary S ∗ when Slalom is run with inputs x 1 and x (cid:48) 1 .
Revised: For privacy, consider the views of an adversary S ∗ when Slalom is run with inputs x 1 and x (cid:48) 1 .

Original: Again, by the security of the PRNG, we consider a hybrid protocol where we replace the pre-computed blinding vectors r i by truly uniformly random values in F .
Revised: Again, by the security of the PRNG, we consider a hybrid protocol where we replace the pre-computed blinding vectors r i by truly uniformly random values in F .

Original: In this hybrid protocol, ˜ x i = x
Revised: In this hybrid protocol, ˜ x i = x

Original: i + r
Revised: i + r

Original: i is simply a “one-time-pad” encryption of x i over the ﬁeld F , so S ∗ ’s views in both executions of the hybrid protocol are equal (information theoretically).
Revised: i is simply a “one-time-pad” encryption of x i over the ﬁeld F , so S ∗ ’s views in both executions of the hybrid protocol are equal (information theoretically).

Original: Thus, S ∗ ’s views in both executions of the original protocol are computationally indistinguishable.
Revised: Thus, S ∗ ’s views in both executions of the original protocol are computationally indistinguishable.

Revised: Under review as a conference paper at ICLR

Original: Proof of Corollary 3.3.
Revised: Proof of Corollary 3.3.

Original: The outsourcing protocol between the remote client C and server S hosting the TEE is simply deﬁned as follows (we assume the model belongs to S ):
Revised: The outsourcing protocol between the remote client C and server S hosting the TEE is simply deﬁned as follows (we assume the model belongs to S ):

Original: • st ← Preproc () :
Revised: • st ← Preproc () :

Original: C and the TEE setup a secure authenticated communication channel, using the TEE’s remote attestation property.
Revised: C and the TEE setup a secure authenticated communication channel, using the TEE’s remote attestation property.

Original: The TEE receives the model F from S and initializes the Slalom protocol.
Revised: The TEE receives the model F from S and initializes the Slalom protocol.

Original: • Outsource (cid:104)C ( x, st ) , S ( F ) (cid:105) :
Revised: • Outsource (cid:104)C ( x, st ) , S ( F ) (cid:105) :

Original: – C sends x to the TEE over the secure channel.
Revised: – C sends x to the TEE over the secure channel.

Original: – The TEE securely computes y = F ( x ) using Slalom.
Revised: – The TEE securely computes y = F ( x ) using Slalom.

Original: – The TEE sends y (and a publicly veriﬁable commitment to F ) to C over the secure channel.
Revised: – The TEE sends y (and a publicly veriﬁable commitment to F ) to C over the secure channel.

Original: It is easy to see that if the TEE is secure (i.e., it acts as a trusted third party hosted by S ), then the result follows.
Revised: If the TEE is secure (i.e., it acts as a trusted third party hosted by S ), then the result follows.

Original: C P ERFORMANCE C OMPARISON OF DNN O UTSOURCING S CHEMES
Revised: C P ERFORMANCE C OMPARISON OF DNN O UTSOURCING S CHEMES

Original: We provide a brief overview of the outsourcing approaches compared in Table 1.
Revised: We provide a brief overview of the outsourcing approaches compared in Table 1.

Original: Our baseline runs a DNN in a
Revised: Our baseline runs a DNN in a

Original: TEE (a single-threaded Intel SGX enclave) and can provide all the security guarantees of an ML outsourcing scheme.
Revised: TEE (a single-threaded Intel SGX enclave) and can provide all the security guarantees of an ML outsourcing scheme.

Original: On a high-end GPU (an Nvidia TITAN XP), we achieve over 50 × higher throughput but no security.
Revised: On a high-end GPU (an Nvidia TITAN XP), we achieve over 50 × higher throughput but no security.

Original: For example, for MobileNet, the enclave evaluates 16 images/sec and the GPU 900 images/sec ( 56 × higher).
Revised: For example, for MobileNet, the enclave evaluates 16 images/sec and the GPU 900 images/sec ( 56 × higher).

Original: Under review as a conference paper at ICLR

Original: SafetyNets (Ghodsi et al., 2017) and Gazelle (Juvekar et al., 2018) are two representative works that achieve respectively integrity and privacy using purely cryptographic approaches (without a TEE).
Revised: SafetyNets (Ghodsi et al., 2017) and Gazelle (Juvekar et al., 2018) are two representative works that achieve respectively integrity and privacy using purely cryptographic approaches (without a TEE).

Original: SafetyNets does not hide the model from either party, while Gazelle leaks some architectural details to the client.
Revised: SafetyNets does not hide the model from either party, while Gazelle leaks some architectural details to the client.

Original: The cryptographic techniques used by these systems incur large computation and communication overheads in practice.
Revised: The cryptographic techniques used by these systems incur large computation and communication overheads in practice.

Original: The largest model evaluated by SafetyNets is a 4-layer TIMIT model with quadratic activations which runs at aboutimages/sec (on a notebook CPU).
Revised: The largest model evaluated by SafetyNets is a 4-layer TIMIT model with quadratic activations which runs at aboutimages/sec (on a notebook CPU).

Original: In our baseline enclave, the same model runs at over 3 , 500 images/sec.
Revised: In our baseline enclave, the same model runs at over 3 , 500 images/sec.

Original: The largest model evaluated by Gazelle is an 8 -layer CIFAR10 model.
Revised: The largest model evaluated by Gazelle is an 8 -layer CIFAR10 model.

Original: In the enclave, we can evaluate 450 images/sec whereas Gazelle evaluates a single image in 3 .
Revised: In the enclave, we can evaluate 450 images/sec whereas Gazelle evaluates a single image in 3 .

Original: 5 sec with 300 MB of communication between client and server.
Revised: 5 sec with 300 MB of communication between client and server.

Original: A Note on Energy Efﬁciency.
Revised: A Note on Energy Efﬁciency.

Original: When comparing approaches with different hardware (e.g., our single-core CPU baseline versus Slalom which also uses a GPU), throughput alone is not the fairest metric.
Revised: When comparing approaches with different hardware (e.g., our single-core CPU baseline versus Slalom which also uses a GPU), throughput alone is not the fairest metric.

Original: E.g., the baseline’s throughput could also be increased by adding more SGX CPUs.
Revised: E.g., the baseline’s throughput could also be increased by adding more SGX CPUs.

Original: A more accurate comparison considers the energy efﬁciency of a particular approach, a more direct measure of the recurrent costs to the server S .
Revised: A more accurate comparison considers the energy efﬁciency of a particular approach, a more direct measure of the recurrent costs to the server S .

Original: For example, when evaluating MobileNet or VGG16, our GPU draws 85 W of power, whereas our baseline SGX
Revised: For example, when evaluating MobileNet or VGG16, our GPU draws 85 W of power, whereas our baseline SGX

Original: CPU draws 30 W. As noted above, the GPU also achieves more than 50 × higher throughput, and thus is at least 18 × more energy efﬁcient (e.g., measured in Joules per image) than the enclave.
Revised: CPU draws 30 W. As noted above, the GPU also achieves more than 50 × higher throughput, and thus is at least 18 × more energy efﬁcient (e.g., measured in Joules per image) than the enclave.

Original: For Slalom, we must take into account the cost of running both the enclave and GPU.
Revised: For Slalom, we must consider the cost of running both the enclave and GPU.

Original: In our evaluations, the outsourced computations on the GPU account for at most 10% of the total running time of Slalom (i.e., the integrity checks and data encryption/decryption in the enclave are the main bottleneck).
Revised: In our evaluations, the outsourced computations on the GPU account for at most 10% of the total running time of Slalom (i.e., the integrity checks and data encryption/decryption in the enclave are the main bottleneck).

Original: Thus, the power consumption attributed to Slalom3 is roughly 10% · 85W + 90% · 30W = 35 .
Revised: Thus, the power consumption attributed to Slalom is roughly 10% · 85W + 90% · 30W = 35 .

Original: 5W .
Revised: 5W .

Revised: Note that when not being in use by Slalom,

Revised: the trusted

Revised: CPU or untrusted GPU can be used by other tasks running on the server.

Original: As Slalom achieves 4 × - 20 × higher throughput than our baseline for the tasks we evaluate, it is also about 3 .
Revised: As Slalom achieves 4 × - 20 × higher throughput than our baseline for the tasks we evaluate, it is also about 3 .

Original: 4 × - 17 .
Revised: 4 × - 17 .

Original: 1 × more energy efﬁcient.
Revised: 1 × more energy efﬁcient.

Original: D N OTATION FOR S TANDARD L INEAR O PERATORS
Revised: D N OTATION FOR S TANDARD L INEAR O PERATORS

Original: Below we describe some common linear operators used in deep neural networks.
Revised: Below we describe some common linear operators used in deep neural networks.

Original: For simplicity, we omit additive bias terms, and assume that convolutional operators preserve the spatial height and width of their inputs.
Revised: For simplicity, we omit additive bias terms, and assume that convolutional operators preserve the spatial height and width of their inputs.

Original: Our techniques easily extend to convolutions with arbitrary strides, paddings, and window sizes.
Revised: Our techniques easily extend to convolutions with arbitrary strides, paddings, and window sizes.

Revised: Under review as a conference paper at ICLR

Original: For a fully-connected layer f FC , the kernel W has dimension ( h in × h out ) .
Revised: For a fully-connected layer f FC , the kernel W has dimension ( h in × h out ) .

Original: For an input x of dimension h in , we have f FC ( x ) =
Revised: For an input x of dimension h in , we have f FC ( x ) =

Original: x (cid:62) W .
Revised: x (cid:62) W .

Original: The cost of the layer is h in · h out multiplications.
Revised: The cost of the layer is h in · h out multiplications.

Original: A convolutional layer has kernel W of size ( k × k × c in × c out ) .
Revised: A convolutional layer has kernel W of size ( k × k × c in × c out ) .

Original: On input x of size ( h × w × c in ) , f conv ( x ) =
Revised: On input x of size ( h × w × c in ) , f conv ( x ) =

Original: Conv ( x ; W ) produces an output of size ( h × w × c out ) .
Revised: Conv ( x ; W ) produces an output of size ( h × w × c out ) .

Original: A convolution can be seen as the combination of two linear operators: a “patch-extraction” process that transforms the input x into an intermediate input x (cid:48) of dimension ( h · w, k 2 · c in ) by extracting k × k patches, followed by a matrix multiplication with W .
Revised: A convolution can be seen as the combination of two linear operators: a “patch-extraction” process that transforms the input x into an intermediate input x (cid:48) of dimension ( h · w, k 2 · c in ) by extracting k × k patches, followed by a matrix multiplication with W .

Original: The cost of this layer is thus k 2 · h · w · c in · c out multiplications.
Revised: The cost of this layer is thus k 2 · h · w · c in · c out multiplications.

Original: A separable convolution has two kernels, W 1 of size ( k × k × c in ) and W 2 of size ( c in × c out ) .
Revised: A separable convolution has two kernels, W 1 of size ( k × k × c in ) and W 2 of size ( c in × c out ) .

Original: On input x of size ( h × w × c in ) , f sep-conv ( x ) produces an output of size ( h × w × c out ) , by applying a depthwise convolution f dp-conv ( x ) with kernel W 1 followed by a pointwise convolution f pt-conv ( x ) with kernel W 2 .
Revised: On input x of size ( h × w × c in ) , f sep-conv ( x ) produces an output of size ( h × w × c out ) , by applying a depthwise convolution f dp-conv ( x ) with kernel W 1 followed by a pointwise convolution f pt-conv ( x ) with kernel W 2 .

Original: The depthwise convolution consists of c in independent convolutions with ﬁlters of size k × k × 1 × 1 , applied to a single input channel, which requires k 2 · h · w · c in multiplications.
Revised: The depthwise convolution consists of c in independent convolutions with ﬁlters of size k × k × 1 × 1 , applied to a single input channel, which requires k 2 · h · w · c in multiplications.

Original: A pointwise convolution is simply a matrix product with an input of size ( h · w ) × c in , and thus requires h · w · c in · c out multiplications.
Revised: A pointwise convolution is simply a matrix product with an input of size ( h · w ) × c in , and thus requires h · w · c in · c out multiplications.

Original: Under review as a conference paper at ICLR

Original: E N EURAL N ETWORK D ETAILS
Revised: E N EURAL N ETWORK D ETAILS

Original: Table 3 provides details about the two DNNs we use in our evaluation, VGG16 and MobileNet.
Revised: Table 3 provides details about the two DNNs we use in our evaluation (all pre-trained models are taken from

Revised: Keras Chollet et al.

Original: We report top 1 and top 5 accuracy on ImageNet with and without the simple quantization scheme described in Section 3.1.
Revised: We report top 1 and top 5 accuracy on ImageNet with and without the simple quantization scheme described in Section 3.1.

Original: Quantization results in at most a 0 .
Revised: Quantization results in at most a 0 .

Original: 5% drop in top 1 and top 5 accuracy.
Revised: 5% drop in top 1 and top 5 accuracy.

Original: More elaborate quantization schemes exist (e.g., Micikevicius et al.
Revised: More elaborate quantization schemes exist (e.g., Micikevicius et al.

Original: that we have not experimented with in this work.
Revised: that we have not experimented with in this work.

Original: We report the number of model parameters, which is relevant to the memory constraints of TEEs such as Intel
Revised: We report the number of model parameters, which is relevant to the memory constraints of TEEs such as Intel

Original: SGX.
Revised: SGX.

Original: We also list the total size of the inputs and outputs of all the model’s linear layers, which impact the amount of communication between trusted and untrusted co-processors in Slalom, as well as the amount of data stored in the TEE when using Freivalds’ algorithm with preprocessing.
Revised: We also list the total size of the inputs and outputs of all the model’s linear layers, which impact the amount of communication between trusted and untrusted co-processors in Slalom, as well as the amount of data stored in the TEE when using Freivalds’ algorithm with preprocessing.

Original: [Table]
Revised: [Table]

Original: F M ODULAR A RITHMETIC WITH F LOATING P OINT O PERATIONS
Revised: F M ODULAR A RITHMETIC WITH F LOATING P OINT O PERATIONS

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we brieﬂy describe how Slalom performs modular arithmetic over a ﬁeld Z p in the TEE, while leveraging standard ﬂoating point operations to maximize computational efﬁciency.
Revised: we brieﬂy describe how Slalom performs modular arithmetic over a ﬁeld Z p in the TEE, while leveraging standard ﬂoating point operations to maximize computational efﬁciency.

Original: The main computations in the TEE are inner products over Z p for Freivalds’ check (a matrix product is itself a set of inner products).
Revised: The main computations in the TEE are inner products over Z p for Freivalds’ check (a matrix product is itself a set of inner products).

Original: Our quantization scheme (see Section 3.1) ensures that all DNN values can be represented in Z p , for p (cid:47) 2 24 , which ﬁts in a standard ﬂoat.
Revised: Our quantization scheme (see Section 3.1) ensures that all DNN values can be represented in Z p , for p (cid:47) 2 24 , which ﬁts in a standard ﬂoat.

Original: To compute inner products, we ﬁrst cast elements to doubles (as a single multiplication in Z p would exceed the range of integers exactly representable as ﬂoats). 
Revised: To compute inner products, we ﬁrst cast elements to doubles (as a single multiplication in Z p would exceed the range of integers exactly representable as ﬂoats). Single or double

Revised: Under review as a conference paper at ICLRprecision ﬂoats are preferable to integer types on Intel architectures due to the availability of much more efﬁcient

Revised: SIMD instructions, at a minor reduction in the range of exactly representable integers.

Original: In our evaluation, we target a soundness error of 2 − 40 for each layer.
Revised: In our evaluation, we target a soundness error of 2 − 40 for each layer.

Original: This leads to a tradeoff between the number of repetitions k of Freivalds’ check, and the size of the set S from which we draw random values.
Revised: This leads to a tradeoff between the number of repetitions k of Freivalds’ check, and the size of the set S from which we draw random values.

Original: One check with | S | = 2 40 is problematic, as multiplying elements in Z p and S can exceed the range of integers exactly representable as doubles ( 2 53 ).
Revised: One check with | S | = 2 40 is problematic, as multiplying elements in Z p and S can exceed the range of integers exactly representable as doubles ( 2 53 ).

Original: With k = 2 repetitions, we can set S =
Revised: With k = 2 repetitions, we can set S =

Original: [ − 2 19 , 2 19 ] .
Revised: [ − 2 19 , 2 19 ] .

Original: Multiplications are then bounded by 2 24+19 = 2 43 , and we can accumulate 2 10 terms in the inner-product before needing a modular reduction.
Revised: Multiplications are then bounded by 2 24+19 = 2 43 , and we can accumulate 2 10 terms in the inner-product before needing a modular reduction.

Original: In practice, we ﬁnd that increasing k further (and thus reducing | S | ) is not worthwhile, as the cost of performing more inner products trumps the savings from reducing the number of modulos.
Revised: In practice, we ﬁnd that increasing k further (and thus reducing | S | ) is not worthwhile, as the cost of performing more inner products trumps the savings from reducing the number of modulos.

Original: G M ICRO -B ENCHMARKS
Revised: G R ESULTS ON A S TANDARD CPU

Original: Our micro-benchmark suite consists of (square) matrix products of increasing dimensions, convolutional operations performed by VGG16, and separable convolutions performed by MobileNet. In all cases, the data is pre-loaded inside an enclave, so we only measure the in-enclave execution time. [Figure]

Original: [Figure]

Original: Figure 3 plots the relative speedups of various veriﬁcation strategies over the cost of computing the linear operation directly.

Original: In all cases, the baseline computation is performed in single-precision ﬂoating point, and the veriﬁcation algorithms repeat Freivalds’ check so as to attain at least 40 bits of statistical soundness.

Original: The same benchmarks run outside of the trusted enclave are in Figure 4 below.

Original: For square matrices of dimensions up to 2048, verifying an outsourced result is 4 × to 8 × faster than computing it.

Original: For larger matrices, we exceed the limit of SGX’s DRAM, so the enclave resorts to expensive paging which drastically reduces performance both for computation and veriﬁcation.

Original: For convolutions (standard or separable), we achieve large savings with outsourcing if Freivalds’ algorithm is applied with preprocessing.

Original: As expected, the savings get higher as the number of channels increases.

Original: Applying Freivalds’ algorithm “on-the-ﬂy” (i.e., without preprocessing) also results in savings if c out is large. Due to  SGX’s small PRM, batched veriﬁcation is only effective for operations with small memory footprints. As expected, “truly” separable convolutions (with no intermediate non-linearity) are much faster to verify, as they can be viewed as a single linear operator.
Revised: For completeness, and to asses how our outsourcing scheme fairs in an environment devoid of Intel SGX’s performance quirks, we rerun the evaluations in Section 4 on the same CPU but outside of SGX’s enclave mode.

Original: Under review as a conference paper at ICLR

Original: I m a g e s / s e c 2.2 5.4 9.

Original: VGG    18.7 40.2 49.

Original: MobileNet   18.7 40.2 49.

Original: MobileNet (fused)

Original: [Figure]
Revised: [Figure]

Original: ) verify verify batched verify w. preproc privacy privacy + integrity

Original: H R ESULTS ON A S TANDARD CPU

Original: For completeness, and to evaluate how our outsourcing scheme fairs in an environment devoid of Intel SGXs performance quirks, we run the same evaluations as in Section 4 on the same CPU but outside of SGX’s enclave mode.

Original: Figure 4 show the results of the micro-benchmarks for matrix multiplication, convolution and separbale convolutions.
Revised: Figure 5 show the results of the micro-benchmarks for matrix multiplication, convolution and separable convolutions.

Original: In all cases, verifying a computation becomes 1 - 2 orders of magnitude faster than computing it as the outer dimension grows.
Revised: In all cases, verifying a computation becomes 1 - 2 orders of magnitude faster than computing it as the outer dimension grows.

Original: Compared to the SGX benchmarks, we also see a much better viability of batched veriﬁcation (we haven’t optimized batched veriﬁcations much, as they are inherently slow on SGX.
Revised: Compared to the SGX benchmarks, we also see a much better viability of batched veriﬁcation (we haven’t optimized batched veriﬁcations much [Figure] as they are inherently slow on SGX.

Original: It is likely that these numbers could be improved signiﬁcantly, to approach those of veriﬁcation with preprocessing).
Revised: It is likely that these numbers could be improved signiﬁcantly, to approach those of veriﬁcation with preprocessing)

Original:  Figure 5 shows the throughput achieved by VGG16 and MobileNet on a single core with either direct computation or various veriﬁcation strategies (with and without privacy).
Revised: [Figure] [Figure] Under review as a conference paper at ICLR Figure 6 shows benchmarks for VGG16 and MobileNet on a single core with either direct computation or various secure outsourcing strategies.

Original: For integrity alone, we achieve savings up to 8 .
Revised: For integrity alone, we achieve savings up to 8 .

Original: 9 × and 20 .[-
-]
Revised: 9 × and 19 .[-
-]

Original:  × for MobileNet and VGG16 respectively.
Revised: 5 × for MobileNet and VGG16 respectively.

Original: Even without storing any secrets in the enclave, we obtain good speedups using batched veriﬁcation.
Revised: Even without storing any secrets in the enclave, we obtain good speedups using batched veriﬁcation.

Original: As noted above, it is likely that the batched results could be further improved.
Revised: As noted above, it is likely that the batched results could be further improved.

Original: With additional blinding to preserve privacy, we achieve speedups of 3 . 9 × and 8 . 2 × for MobileNet and VGG16 respectively.
Revised: With additional blinding to preserve privacy, we achieve speedups of 3 . 9 × and 8 . 1 × for MobileNet and VGG16 respectively.

Original: I P ARALLELIZATION
Revised: H P ARALLELIZATION

Original: Our experiments on SGX in Section 4 where performed using a single execution thread, as SGX enclaves do not have the ability to create threads.
Revised: Our experiments on SGX in Section 4 where performed using a single execution thread, as SGX enclaves do not have the ability to create threads.

Original: We have also experimented with techniques for achieving parallelism in SGX, both for standard computations and outsourced ones, but with little success.
Revised: We have also experimented with techniques for achieving parallelism in SGX, both for standard computations and outsourced ones, but with little success.

Original: To optimize for throughput, the simplest approach for DNN inference would be to run multiple forward passes simultaneously.
Revised: To optimize for throughput, a simple approach is to run multiple forward passes simultaneously.

Original: On a standard CPU, this form of “outer-parallelism” achieves close to linear scaling as we increase the number of threads from 1 to 4 on our quad-core machine.
Revised: On a standard CPU, this form of “outer-parallelism” achieves close to linear scaling as we increase the number of threads from 1 to 4 on our quad-core machine.

Original: With SGX however, we did not manage to achieve any parallel speedup for VGG16—whether for direct computation or verifying outsourced results— presumably because each independent thread requires extra memory that quickly exceeds the PRM limit.
Revised: With SGX however, we did not manage to achieve any parallel speedup for VGG16—whether for direct computation or verifying outsourced results—presumably because each independent thread requires extra memory that quickly exceeds the PRM limit.

Original: For the smaller MobileNet model, we get less than a 1 . 5 × speedup using up to 4 threads, for direct computation or outsourced veriﬁcation alike.
Revised: For the smaller MobileNet model, we get less than a 1 . 5 × speedup using up to 4 threads, for direct computation or outsourced veriﬁcation alike.

Original: DNNs typically also make use of intra-operation parallelism, i.e., computing the output of a given layer using multiple threads.
Revised: DNNs typically also make use of intra-operation parallelism, i.e., computing the output of a given layer using multiple threads.

Original: Our DNN library currently does not support intra-operation parallelism, but implementing[Figure] a dedicated thread pool for SGX could be an interesting extension for future work.
Revised: Our DNN library currently does not support intra-operation parallelism, but implementing a dedicated thread pool for SGX could be an interesting extension for future work.

Original: Instead, we evaluate the potential beneﬁts of intra-op parallelism on a standard untrusted CPU, for our matrix-product and convolution benchmarks.
Revised: Instead, we evaluate the potential beneﬁts of intra-op parallelism on a standard untrusted CPU, for our matrix-product and convolution benchmarks.

Original: We make use of Eigen’s internal multi-threading support to speed up these operations [Figure] and custom
Revised: We make use of Eigen’s internal multi-threading support to speed up these operations, and custom

Original: OpenMP code to parallelize dot products, as Eigen does not do this on its own.
Revised: OpenMP code to parallelize dot products, as Eigen does not do this on its own.

Original:  Figure 6 shows the results using 4 threads.
Revised: [Figure] Figure 7 shows the results using 4 threads.

Original: For convolutions, we have currently only implemented multi-threading for the veriﬁcation with preprocessing (which requires only standard dot products).
Revised: For convolutions, we have currently only implemented multi-threading for the veriﬁcation with preprocessing (which requires only standard dot products).

Original: Surprisingly maybe, we ﬁnd that multi-threading increases the gap between direct and veriﬁed computations of matrix products, probably because dot products are extremely easy to parallelize efﬁciently (compared to full convolutions).
Revised: Surprisingly maybe, we ﬁnd that multi-threading increases the gap between direct and veriﬁed computations of matrix products, probably because dot products are extremely easy to parallelize efﬁciently (compared to full convolutions).

Original: We also obtain close to linear speedups for veriﬁable separable convolutions, but omit the results as we currently do not have an implementation of multi-threaded direct computation for depthwise convolutions, which renders the comparison unfair.
Revised: We also obtain close to linear speedups for veriﬁable separable convolutions, but omit the results as we currently do not have an implementation of multi-threaded direct computation for depthwise convolutions, which renders the comparison unfair.

Original: Due to the various memory-access overheads in SGX, it is unclear whether similar speedups could be obtained by using intra-op parallelism in an enclave, but this is an avenue worth exploring.
Revised: Due to the various memory-access overheads in SGX, it is unclear whether similar speedups could be obtained by using intra-op parallelism in an enclave

Original: The techniques described in this paper for veriﬁable outsourcing of DNN forward operations could also be applied to DNN training. Indeed, the backward pass in a DNN essentially makes use of similar linear operators as in the forward pass, and could thus be veriﬁed using Freivalds’ algorithm. Yet, extending Slalom to DNN training is not without challenges:
Revised: [Figure]

Original: • Quantization at training time is a difﬁcult and important problem with ongoing research. For instance, a more ﬂexible quantization scale than the ﬁxed one we use for inference would be necessary, to account for large changes in weight magnitudes during training (Micikevicius et al., 2018). • Because the weights change over time, we cannot use the same preprocessed random vectors for Freivalds’ check throughout. The most efﬁcient approach would be to verify a whole batch of forward-backward passes simultaneously.
Revised: but this is an avenue worth exploring

Original: Of course, due to the current memory size limits in SGX, operating over large batches is difﬁcult, which is problematic for training with or without outsourcing. • Finally, the precomputation techniques we employ for protecting input privacy do not apply for training, as the weights change after every processed batch. Moreover, Slalom does not try to hide the model weights from the untrusted processor, which might be a requirement for private training.
Revised: [Figure]