Original: Iron : Private Inference on Transformers
Revised: Iron : Private Inference on Transformers

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We initiate the study of private inference on Transformer-based models in theclient-server setting, where clients have private inputs and servers hold proprietarymodels.
Revised: We initiate the study of private inference on Transformer-based models in theclient-server setting, where clients have private inputs and servers hold proprietarymodels.

Original: Our main contribution is to provide several new secure protocols for matrixmultiplication and complex non-linear functions like Softmax, GELU activations,and LayerNorm, which are critical components of Transformers.
Revised: Our main contribution is to provide several new secure protocols for matrixmultiplication and complex non-linear functions like Softmax, GELU activations,and LayerNorm, which are critical components of Transformers.

Original: Specifically, wefirst propose a customized homomorphic encryption-based protocol for matrixmultiplication that crucially relies on a novel compact packing technique.
Revised: Specifically, wefirst propose a customized homomorphic encryption-based protocol for matrixmultiplication that crucially relies on a novel compact packing technique.

Original: Thisdesign achieves √ m × less communication ( m is the number of rows of the outputmatrix) and at least 3 × performance improvement over the state-of-the-art.
Revised: Thisdesign achieves √ m × less communication ( m is the number of rows of the outputmatrix) and at least 3 × performance improvement over the state-of-the-art.

Original: Second,we design efficient protocols for three non-linear functions via integrating advancedunderlying protocols and specialized optimizations.
Revised: Second, we design efficient protocols for three non-linear functions via integratingadvanced underlying protocols and specialized optimizations.

Original: Compared to the most efficientprotocols, our recipes reduce about half of the communication and computationoverhead.
Revised: Compared to themost efficient protocols, our recipes reduce about half of the communication andcomputation overhead.

Original: Furthermore, all protocols are numerically precise, which preserve themodel accuracy of plaintext.
Revised: Furthermore, all protocols are numerically precise, whichpreserve the model accuracy of plaintext.

Original: These techniques together allow us to implement Iron ,the first Transformer-based private inference framework.
Revised: These techniques together allow usto implement Iron , an efficient Transformer-based private inference framework.

Original: Experiments conductedon several real-world datasets and models demonstrate that Iron achieves 3 ∼ 14 ×less communication and 3 ∼ 11 × less runtime compared to the prior art.
Revised: Experiments conducted on several real-world datasets and models demonstrate that Iron achieves 3 ∼ 14 × less communication and 3 ∼ 11 × less runtime comparedto the prior art.

Original:  Introduction
Revised:  Introduction

Original: Transformer-based models [1–5] have achieved tremendous success in the fields of natural languageprocessing (NLP) and computer vision (CV) due to their strong representation capabilities.
Revised: Transformer-based models [1–5] have achieved tremendous success in the fields of natural languageprocessing (NLP) and computer vision (CV) due to their strong representation capabilities.

Original: As anew neural network architecture, the Transformer [1] mainly utilizes the self-attention mechanismto compute representations without sequence-aligned recurrence or convolution.
Revised: As anew neural network architecture, the Transformer [1] mainly utilizes the self-attention mechanismto compute representations without sequence-aligned recurrence or convolution.

Original: Following thiswork, a number of Transformer variants, such as BERT [2] and GPT [3] in NLP, ViT [4] and Swin
Revised: Following thiswork, a number of Transformer variants, such as BERT [2] and GPT [3] in NLP, ViT [4] and Swin

Original: Transformer [5] in CV, have achieved state-of-the-art performance on lots of real-world tasks.
Revised: Transformer [5] in CV, have achieved state-of-the-art performance on lots of real-world tasks.

Original: The success of Transformers and other big models facilitates emerging inference services andapplications [6, 7].
Revised: The success of Transformers and other big models facilitates emerging inference services andapplications [6, 7].

Original: In particular, a service provider trains a complex model based on the Transformer,and deploys it as a paid inference service, e.g., machine translation, question answering.
Revised: In particular, a service provider trains a complex model based on the Transformer,and deploys it as a paid inference service, e.g., machine translation, question answering.

Original: A client queries this service with his input samples and obtains the desired responses.
Revised: A clientqueries this service with his input samples and obtains the desired responses.

Original: Unfortunately, current inference systems suffer from serious privacy concerns [8].
Revised: Unfortunately, currentinference systems suffer from serious privacy concerns [8].

Original: On the one hand, clients need to sendconfidential inputs to the service provider, which could compromise the data privacy of these clients if the provider is untrusted.
Revised: On the one hand, clients need to sendconfidential inputs to the service provider, which could compromise the data privacy of these clientsif the provider is untrusted.

Original: On the other hand, it is undesirable for the provider to distribute the proprietary Transformer-based model to clients, since he needs a large amount of data and computation resources to construct the model [9].
Revised: On the other hand, it is undesirable for the provider to distribute theproprietary Transformer-based model to clients, since he needs a large amount of data and computationresources to construct the model [9].

Original: Therefore, there exists a gap between unprecedented performance and privacy constraints, which motivates our study of private Transformer inference
Revised: Therefore, there exists a gap between unprecedented performanceand privacy constraints, which motivates our study of private Transformer inference.

Original: Private inference aims to protect server’s model weights from clients, while guaranteeing that the server learns no information about clients’ private inputs.
Revised: Private inference aims to protect server’s model weights from clients, while guaranteeing that theserver learns no information about clients’ private inputs.

Original: Recently, private inference on traditional neural networks (e.g., convolutional neural networks) have been approached by using secure 2-party computation (2PC) techniques [10–14].
Revised: Recently, private inference on traditionalneural networks (e.g., convolutional neural networks) have been approached by using secure 2-party computation (2PC) techniques [10–14].

Original: However, due to essentially different structures, private
Revised: However, due to essentially different structures, private

Original: Transformer inference brings several new challenges.
Revised: Transformer inference brings several new challenges.

Original: First, Transformer-based models use lots ofhigh-dimensional matrix multiplications , rather than matrix-vector multiplications widely studied inprior works.
Revised: First, Transformer-based models use lots ofhigh-dimensional matrix multiplications , rather than matrix-vector multiplications widely studied inprior works.

Original: While we can straightforwardly extend prior matrix-vector multiplication protocols toour setting, unfortunately, even the most efficient design [14] incurs heavy communication because offrequently interacting ciphertexts.
Revised: While we can straightforwardly extend prior matrix-vector multiplication protocols toour setting, unfortunately, even the most efficient design [14] incurs heavy communication because offrequently interacting ciphertexts.

Original: Second, Transformer-based models use complex math functions like
Revised: Second, Transformer-based models use complex math functions like

Original: Softmax, GELU activations [15], and LayerNorm, in each block , rather than crypto-friendly non-linearfunctions such as ReLU and Maxpool.
Revised: Softmax, GELU activations [15], and LayerNorm, in each block , rather than crypto-friendly non-linearfunctions such as ReLU and Maxpool.

Original: Existing methods either use precision-impaired high-orderpolynomial approximations [16, 11] or only support limited math functions for specific scenarios[17].
Revised: Existing methods either use precision-impaired high-orderpolynomial approximations [16, 11] or only support limited math functions for specific scenarios[17].

Original: Even worse, all of these approaches are computationally intensive and often require a largeamount of communication (for more related works, please refer to Appendix A.5).
Revised: Even worse, all of these approaches are computationally intensive and often require a largeamount of communication (for more related works, please refer to Appendix A.5).

Original: To facilitate thewidespread adoption of Transformer-based inference services in privacy-critical scenarios, designingefficient protocols for the above complex operations is of paramount importance.
Revised: To facilitate thewidespread adoption of Transformer-based inference services in privacy-critical scenarios, designingefficient protocols for the above complex operations is of paramount importance.

Original: In this paper, we design Iron , an efficient hybrid cryptographic framework for private Transformer inference without revealing any sensitive information about the server’s model weights or clients’ inputs.
Revised: In this paper, we design Iron , an efficient hybrid cryptographic framework for private Transformer inference without revealing any sensitive information about the server’s model weights or clients’ inputs.

Original: Iron contributes several new specialized protocols for the complicated operations in Transformersto alleviate the performance overhead.
Revised: Iron contributes several new specialized protocols for the complicated operations in Transformersto alleviate the performance overhead.

Original: Specifically, we first propose a customized homomorphicencryption-based protocol for matrix multiplications.
Revised: Specifically, we first propose a customized homomorphicencryption-based protocol for matrix multiplications.

Original: Our insight is to pack more plaintext inputs intoa single ciphertext by devising a compact packing method, while preserving the functionality of matrixmultiplication.
Revised: Our insight is to pack more plaintext inputs intoa single ciphertext by devising a compact packing method, while preserving the functionality of matrixmultiplication.

Original: Compared to the state-of-the-art matrix-vector multiplication solution implemented in
Revised: Compared to the state-of-the-art matrix-vector multiplication solution implemented in

Original: Cheetah [14], we can achieve √ m × ( m is the number of rows of the output matrix) improvementin terms of communication overhead, which is about 8 × reduction for various Transformer models.
Revised: Cheetah [14], we can achieve √ m × ( m is the number of rows of the output matrix) improvementin terms of communication overhead, which is about 8 × reduction for various Transformer models.

Original: Second, we carefully design efficient protocols for Softmax, GELU, and LayerNorm.
Revised: Second, we carefully design efficient protocols for Softmax, GELU, and LayerNorm.

Original: These protocolsare built upon SIRNN [17], the state-of-the-art cryptographic framework for private inference onrecurrent neural networks, and are never supported previously.
Revised: These protocolsare built upon SIRNN [17], the state-of-the-art cryptographic framework for private inference onrecurrent neural networks, and are never supported previously.

Original: We also make several customizedoptimizations, such as reducing the overhead of exponentiation in Softmax and simplifying GELUand LayerNorm.
Revised: We also make several customizedoptimizations, such as reducing the overhead of exponentiation in Softmax and simplifying GELUand LayerNorm.

Original: These optimizations achieve 1 .
Revised: These optimizations achieve 1 .

Original: 3 ∼ 1 . 8 × less runtime and 1 .
Revised: 3 ∼ 1 . 8 × less runtime and 1 .

Original: 4 ∼ 1 . 8 × less communication on three non-linear functions.
Revised: 4 ∼ 1 . 8 × less communication on three non-linear functions.

Original: Furthermore, these protocols are numerically precise, whichpreserve the model accuracy of plaintext.
Revised: Furthermore, these protocols are numerically precise, whichpreserve the model accuracy of plaintext.

Original: We also give a formal security proof for our designedprotocols to demonstrate the security guarantee.
Revised: We also give a formal security proof for our designedprotocols to demonstrate the security guarantee.

Original: Based on the above efficient components, we implement the first private Transformer inferenceframework, Iron , and conduct end-to-end experiments with various BERT architectures [2] (BERTTiny, BERT-Medium, BERT-Base, and BERT-Large) on GLUE benchmarks [18].
Revised: Based on the above efficient components, we implement a private Transformer inference framework, Iron , and conduct end-to-end experiments with various BERT architectures [2] (BERT-Tiny, BERTMedium, BERT-Base, and BERT-Large) on GLUE benchmarks [18].

Original: Note that Ironis readily extended to other Transformer-based models (e.g., ViT) since they share very similararchitectures and same operations.
Revised: Note that Iron is readilyextended to other Transformer-based models (e.g., ViT) since they share very similar architecturesand same operations.

Original: Experimental results show that Iron achieves 3 ∼ 14 × lesscommunication and 3 ∼ 11 × less runtime costs over SIRNN on four BERT models.
Revised: Experimental results show that Iron achieves 3 ∼ 14 × less communicationand 3 ∼ 11 × less runtime costs over SIRNN on four BERT models.

Original: Moreover,compared with the general-purpose state-of-the-art framework MP-SPDZ [16], Iron has up to twoorders of magnitude improvement in terms of both communication and computation efficiency.
Revised: Moreover, compared with thegeneral-purpose state-of-the-art framework MP-SPDZ [16], Iron has up to two orders of magnitudeimprovement in terms of both communication and computation efficiency.

Original:  Preliminaries2.1 Threat Model
Revised:  Preliminaries2.1 Threat Model

Original: As shown in the left part of Figure 1, Iron works in a general private inference scenario, where theserver P 0 holds a Transformer-based model M with private weights w , while the client P 1 holds aprivate input x .
Revised: As shown in the left part of Figure 1, Iron works in a general private inference scenario, where theserver P 0 holds a Transformer-based model M with private weights w , while the client P 1 holds aprivate input x .

Original: Our framework enables the client to query the server’s inference service and learnthe output of the model on its input, i.e., M ( w, x ) .
Revised: Our framework enables the client to query the server’s inference service and learnthe output of the model on its input, i.e., M ( w, x ) .

Original: Same as prior works [14, 17], we consider anhonest-but-curious adversary that passively corrupts either the server or the client, but not both.
Revised: Same as prior works [14, 17], we consider anhonest-but-curious adversary that passively corrupts either the server or the client, but not both.

Original: Suchan adversary follows the protocol specification exactly, but may try to learn more information 1 thanallowed (e.g., the model’s weights or inference inputs) via analyzing the data it receives.
Revised: Suchan adversary follows the protocol specification exactly, but may try to learn more information 1 thanallowed (e.g., the model’s weights or inference inputs) via analyzing the data it receives.

Original: In Appendix
Revised: In Appendix

Original: A.1.2, we give a more formal description of the threat model for security analysis.
Revised: A.1.2, we give a more formal description of the threat model for security analysis.

Original: 2.2 Cryptographic Primitives

Original: All our protocols are built on the 2-out-of-2 additive secret sharing (ASS) technique [20, 21] overthe ring Z 2 ℓ , in which an ℓ -bit input x is split into two random shares ⟨ x ⟩ 0 , ⟨ x ⟩ 1 , held by P 0 and P 1 ,

Original: Server
Revised: Server

Original: Proprietary model
Revised: Proprietary model

Original: [Figure]
Revised: [Figure]

Original: Inputs Embedding Pos&segEncoding MatMul
Revised: 2.2 Cryptographic Primitives

Original: Softmax

Original: MatMul LayerNorm

Original: LayerNorm Linear Outputs Linear GELU 𝑄𝑄 𝐾𝐾 𝑉𝑉
Revised: All our protocols are built on the 2-out-of-2 additive secret sharing (ASS) technique [20, 21] overthe ring Z 2 ℓ , in which an ℓ -bit input x is split into two random shares ⟨ x ⟩ 0 , ⟨ x ⟩ 1 , held by P 0 and P 1 ,respectively, such that x = ⟨ x ⟩ 0 + ⟨ x ⟩ 1 mod Z 2 ℓ .

Original: Attention

Original: FeedForward Π MatMul Π MatMul Π Softmax Π LayerNorm Π GELU [Figure] respectively, such that x = ⟨ x ⟩ 0 + ⟨ x ⟩ 1 mod Z 2 ℓ . When ℓ = 1 , i.e., over Z 2 , we use ⟨ x ⟩ B to denoteboolean shares.
Revised: When ℓ = 1 , i.e., over Z 2 , we use ⟨ x ⟩ B to denoteboolean shares.

Original: In our protocols, we use ⟨ x ⟩ to denote that P b holds ⟨ x ⟩ b for b ∈ { 0 , 1 } .
Revised: In our protocols, we use ⟨ x ⟩ to denote that P b holds ⟨ x ⟩ b for b ∈ { 0 , 1 } .

Original: The security[20] guarantees that given a share ⟨ x ⟩ 0 or ⟨ x ⟩ 1 , the value of x is perfectly hidden.
Revised: The security[20] guarantees that given a share ⟨ x ⟩ 0 or ⟨ x ⟩ 1 , the value of x is perfectly hidden.

Original: This secret-sharingproperty is maintained throughout our private inference scheme.
Revised: This secret-sharingproperty is maintained throughout our private inference scheme.

Original: ASS naturally supports linearoperations without communication.
Revised: ASS naturally supports linearoperations without communication.

Original: For instance, to compute cx + y with the constant c and secretshares ⟨ x ⟩ and ⟨ y ⟩ , P b can locally compute ⟨ z ⟩ b = c ⟨ x ⟩ b + ⟨ y ⟩ b , where ⟨ z ⟩ 0 + ⟨ z ⟩ 1 = cx
Revised: For instance, to compute cx + y with the constant c and secretshares ⟨ x ⟩ and ⟨ y ⟩ , P b can locally compute ⟨ z ⟩ b = c ⟨ x ⟩ b + ⟨ y ⟩ b , where ⟨ z ⟩ 0 + ⟨ z ⟩ 1 = cx

Original: +
Revised: +

Original: y mod 2 ℓ .
Revised: y mod 2 ℓ .

Original: To achieve more functionalities under shared inputs, we require to invoke advanced homomorphicencryption and oblivious transfer techniques, which are illustrated as follows.
Revised: To achieve more functionalities under shared inputs, we require to invoke advanced homomorphicencryption and oblivious transfer techniques, which are illustrated as follows.

Original: Notations.
Revised: Notations.

Revised: Let x ∤ y means x is not a divisor of y .

Original: We use bold lower-case letters (e.g., x ) to represent vectors, and bold upper-case letters(e.g., X ) to denote matrices.
Revised: We use bold lower-case letters (e.g., x ) to representvectors, and bold upper-case letters (e.g., X ) to denote matrices.

Original: Like prior works [17, 14], we encode inputs with the fixed-pointrepresentation denoted by Fix (refer to Appendix A.1.1 for details).
Revised: Like prior works [17, 14], we encodeinputs with the fixed-point representation denoted by Fix (refer to Appendix A.1.1 for details).

Original: Let  A N, 2 ℓ denote the set ofinteger polynomials A N, 2 ℓ = Z 2 ℓ [ x ] / (cid:0) x
Revised: Let 100 A N, 2 ℓ denote the set of integer polynomials A N, 2 ℓ = Z 2 ℓ [ x ] / (cid:0) x

Original: N + 1 (cid:1) .
Revised: N + 1 (cid:1) .

Original: We use the circumflex of lower-case letters (e.g., ˆ a )to represent a polynomial, and ˆ a [ i ] to denote the i -th coefficient of ˆ a .
Revised: We use the circumflex oflower-case letters (e.g., ˆ a ) to represent a polynomial, and ˆ a [ i ] to denote the i -th coefficient of ˆ a .

Original: Given polynomials ˆ x, ˆ y ∈
Revised: Givenpolynomials ˆ x, ˆ y ∈

Original: A N, 2 ℓ ,the product ˆ z = ˆ x · ˆ y over A N, 2 ℓ is defined as
Revised: A N, 2 ℓ , the product ˆ z = ˆ x · ˆ y over A N, 2 ℓ is defined as

Original: [Equation]
Revised: [Equation]

Original: Linearly Homomorphic Encryption (LHE)
Revised: Linearly Homomorphic Encryption (LHE)

Original: This encryption scheme additionally enableslinearly homomorphic operations on the ciphertexts.
Revised: This encryption scheme additionally enableslinearly homomorphic operations on the ciphertexts.

Original: An LHE scheme is a tuple of algorithms
Revised: An LHE scheme is a tuple of algorithms

Original: LHE =
Revised: LHE =

Original: ( KeyGen ; Enc ; Dec ; Eval ) with the parameter { N, q, p = 2 ℓ } and the following syntax: 1)
Revised: ( KeyGen ; Enc ; Dec ; Eval ) with the parameter { N, q, p = 2 ℓ } and the following syntax: 1)

Original: KeyGen (cid:0) 1 k (cid:1) → ( pk, sk ) : on input a security parameter κ , KeyGen is a randomized algorithm thatoutputs a public key
Revised: KeyGen (cid:0) 1 k (cid:1) → ( pk, sk ) : on input a security parameter κ , KeyGen is a randomized algorithm thatoutputs a public key

Original: pk ∈
Revised: pk ∈

Original: A N,q and a secret key sk ∈ A N,q . 2)
Revised: A N,q and a secret key sk ∈ A N,q . 2)

Original: Enc ( pk, ˆ m )
Revised: Enc ( pk, ˆ m )

Original: → ˆ c : the encryptionalgorithm Enc takes a plaintext polynomial ˆ m ∈
Revised: → ˆ c : the encryptionalgorithm Enc takes a plaintext polynomial ˆ m ∈

Original: A N,p and encrypts it using pk into a ciphertextpolynomial ˆ c ∈
Revised: A N,p and encrypts it using pk into a ciphertextpolynomial ˆ c ∈

Original: A N,q . 3)
Revised: A N,q . 3)

Original: Dec ( sk, ˆ c )
Revised: Dec ( sk, ˆ c )

Original: → ˆ m : on input sk and a ciphertext ˆ c , the (deterministic)decryption algorithm
Revised: → ˆ m : on input sk and a ciphertext ˆ c , the (deterministic)decryption algorithm

Original: Dec recovers the plaintext message ˆ m . 4) Eval ( pk, ˆ c 1 , ˆ c 2 , func )
Revised: Dec recovers the plaintext message ˆ m . 4) Eval ( pk, ˆ c 1 , ˆ c 2 , func )

Original: → ˆ c : oninput pk , two ciphertexts ˆ c 1 , ˆ c 2 containing ˆ m 1 , ˆ m 2 , and a linear function func , Eval outputs a newciphertext ˆ c encrypting func ( ˆ m 1 , ˆ m 2 ) .
Revised: → ˆ c : oninput pk , two ciphertexts ˆ c 1 , ˆ c 2 containing ˆ m 1 , ˆ m 2 , and a linear function func , Eval outputs a newciphertext ˆ c encrypting func ( ˆ m 1 , ˆ m 2 ) .

Original: Note that ⊞ , ⊠ ∈ func denote homomorphic addition andhomomorphic multiplication with a constant polynomial, respectively.
Revised: Note that ⊞ , ⊠ ∈ func denote homomorphic addition andhomomorphic multiplication with a constant polynomial, respectively.

Original: Iron builds the matrixmultiplication protocol on the Brakerski-Fan-Vercauteren (BFV) scheme [24, 25], which is one ofthe state-of-the-art lattice-based LHE solutions.
Revised: Iron builds the matrixmultiplication protocol on the Brakerski-Fan-Vercauteren (BFV) scheme [24, 25], which is one ofthe state-of-the-art lattice-based LHE solutions.

Original: Oblivious Transfer.
Revised: Oblivious Transfer.

Original: The 1-out-of- k oblivious transfer (OT) [26] is denoted by k - OT ℓ , where oneparty is the sender with k messages x 0 , . . .
Revised: The 1-out-of- k oblivious transfer (OT) [26] is denoted by k - OT ℓ , where oneparty is the sender with k messages x 0 , . . .

Original: , x k − 1 ∈ { 0 , 1 } ℓ and the other party is the receiver withan index i ∈ [ k ] .
Revised: , x k − 1 ∈ { 0 , 1 } ℓ and the other party is the receiver withan index i ∈ [ k ] .

Original: The receiver learns x i as the output, and the sender learns nothing.
Revised: The receiver learns x i as the output, and the sender learns nothing.

Original: Additionally,we also use the 1-out-of-2 correlated OT, denoted by 2 - COT ℓ [27], which is defined as follows: thesender inputs a correlation x ∈ Z 2 ℓ , the receiver inputs a choice bit i ∈ { 0 , 1 } , and the protocoloutputs a random element r ∈
Revised: Additionally,we also use the 1-out-of-2 correlated OT, denoted by 2 - COT ℓ [27], which is defined as follows: thesender inputs a correlation x ∈ Z 2 ℓ , the receiver inputs a choice bit i ∈ { 0 , 1 } , and the protocoloutputs a random element r ∈

Original: Z 2 ℓ to the sender and r + i · x to the receiver.
Revised: Z 2 ℓ to the sender and r + i · x to the receiver.

Original: k - OT ℓ and 2 - COT ℓrequire 2 λ
Revised: k - OT ℓ and 2 - COT ℓrequire 2 λ

Original: + kℓ and λ + ℓ bits of communication, respectively, and are executed in 2 rounds.
Revised: + kℓ and λ + ℓ bits of communication, respectively, and are executed in 2 rounds.

Original: The OTprotocols are widely used to build the underlying protocols [13, 17] in Figure 3 we rely on.
Revised: The OTprotocols are widely used to build the underlying protocols [13, 17] in Figure 3 we rely on.

Original:  Overview
Revised:  Overview

Original: [Figure]

Original: 3.1 Transformer Architecture
Revised: 3.1 Transformer Architecture

Original: [Figure]

Original: The Transformer is an encoder-decoder architecture, where both parts have a similar structure.
Revised: The Transformer is an encoder-decoder architecture, where both parts have a similar structure.

Original: Hence,we mainly focus on the encoder below.
Revised: Hence,we mainly focus on the encoder below.

Original: The encoder is composed of a stack of identical blocks,each with two sub-layers, i.e., a multi-head self-attention mechanism and a feed-forward network, as shown in Figure 1.
Revised: The encoder is composed of a stack of identical blocks, each with two sub-layers, i.e., a multi-head self-attention mechanism and a feed-forward network, asshown in Figure 1.

Original: Besides, residual connection and layer normalization (LayerNorm) are employedaround each of the two sub-layers.
Revised: Besides, residual connection and layer normalization (LayerNorm) are employedaround each of the two sub-layers.

Original: We describe a block of the encoder as follows.
Revised: We describe a block of the encoder as follows.

Original: Attention layer.
Revised: Attention layer.

Original: An attention function can be described as mapping a query X Q and a set of keyvalue pairs ( X K , X V ) to a weighted sum of the values, where the weight is computed by a metric ofthe query with the corresponding key [1].
Revised: An attention function can be described as mapping a query X Q and a set of keyvalue pairs ( X K , X V ) to a weighted sum of the values, where the weight is computed by a metric ofthe query with the corresponding key [1].

Original: This function can be formalized as below:
Revised: This function can be formalized as below:

Original: [Equation]
Revised: [Equation]

Original: where X Q , X K , X V are different linear projections of the input X , i.e., X Q = XW Q , X K =
Revised: where X Q , X K , X V are different linear projections of the input X , i.e., X Q = XW Q , X K =

Original: XW K , X V = XW V .
Revised: XW K , X V = XW V .

Original: Multi-head attention extends the above mechanism to H parallel attentionlayers and is illustrated in Appendix A.1.3.
Revised: Multi-head attention extends the above mechanism to H parallel attentionlayers and is illustrated in Appendix A.1.3.

Original: Feed-forward layer.
Revised: Feed-forward layer.

Original: A fully connected feed-forward layer consists of two linear transformationswith a GELU activation in between, where GELU is the Gaussian Error Linear Unit function [15].
Revised: A fully connected feed-forward layer consists of two linear transformationswith a GELU activation in between, where GELU is the Gaussian Error Linear Unit function [15].

Original: This layer can be represented as follows:
Revised: This layer can be represented as follows:

Original: [Equation]
Revised: [Equation]

Original: In addition to the above encoder-decoder blocks, an embedding layer is employed at the beginningof the model to convert input tokens X input to continuous feature vector representations.
Revised: In addition to the above encoder-decoder blocks, an embedding layer is employed at the beginningof the model to convert input tokens X input to continuous feature vector representations.

Original: This isformulated as X = X input W E , where W E is the embedding lookup table.
Revised: This isformulated as X = X input W E , where W E is the embedding lookup table.

Original: 3.2 Private Transformer Inference
Revised: 3.2 Private Transformer Inference

Original: According to the required crypographic operations, the layers of Transformers can be broken intotwo categories - linear and non-linear .1)
Revised: According to the required crypographic operations, the layers of Transformers can be broken intotwo categories - linear and non-linear .1)

Original: Linear layers : these include embedding, matrix multiplication in Attention, and fully-connectedlayer.
Revised: Linear layers : these include embedding, matrix multiplication in Attention, and fully-connectedlayer.

Original: All protocols of these operations rely crucially on the matrix multiplication protocol Π MatMulin Section 4.1.
Revised: All protocols of these operations rely crucially on the matrix multiplication protocol Π MatMulin Section 4.1.

Original: It lies in the setting where P 0 and P 1 take as inputs the matrices X and Y , and learnthe sharings ⟨ Z ⟩ 0 and ⟨ Z ⟩ 1 , respectively, such that Z = XY .2)
Revised: It lies in the setting where P 0 and P 1 take as inputs the matrices X and Y , and learn the sharings ⟨ Z ⟩ 0 and ⟨ Z ⟩ 1 , respectively, such that Z = XY .

Original:  Non-linear layers : these consist of Softmax, GELU and LayerNorm.
Revised: 2) Non-linear layers : these consist of Softmax, GELU and LayerNorm.

Original: The non-linear operationsare directly evaluated by exploiting our proposed protocols in Section 4.2, i.e., Π GELU , Π Softmax ,Π LayerNorm .
Revised: The non-linear operations are directly evaluated by exploiting our proposed protocols in Section 4.2, i.e., Π GELU , Π Softmax ,Π LayerNorm .

Original: All of these take the additive shares ⟨ X ⟩ as inputs, and output the additive shares⟨ Y ⟩ = Π Func ( ⟨ X ⟩ ) , where Func ∈ { GELU , Softmax , LayerNorm } .
Revised: All of these take the additive shares ⟨ X ⟩ as inputs, and output the additive shares ⟨ Y ⟩ = Π Func ( ⟨ X ⟩ ) , where Func ∈ { GELU , Softmax , LayerNorm } .

Original: Overview of private inference.
Revised: Overview of private inference.

Original: In the entire private inference process, we maintain the followinginvariant: P 0 and P 1 begin with additive shares of the input to the layer, and end with additive shares(over the same ring Z 2 ℓ ) of the output of the layer after the protocol.
Revised: In the entire private inference process, we maintain the followinginvariant: P 0 and P 1 begin with additive shares of the input to the layer, and end with additive shares(over the same ring Z 2 ℓ ) of the output of the layer after the protocol.

Original: This allows us to stitch protocolsfor arbitrary layers sequentially to obtain a secure computation scheme for any Transformer-basedmodels.
Revised: This allows us to stitch protocolsfor arbitrary layers sequentially to obtain a secure computation scheme for any Transformer-basedmodels.

Original: To clearly understand our scheme, we take an example of private inference between P 0and P 1 on the embedding layer and the first encoder block, as shown in the right part of Figure 1.
Revised: To clearly understand our scheme, we take an example of private inference between P 0and P 1 on the embedding layer and the first encoder block, as shown in the right part of Figure 1.

Original: Here, our example uses the designed protocols as a black box, i.e., Π MatMul , Π GELU , Π Softmax , andΠ LayerNorm , and we discuss how to efficiently implement these protocols in Section 4.
Revised: Here, our example uses the designed protocols as a black box, i.e., Π MatMul , Π GELU , Π Softmax , andΠ LayerNorm , and we discuss how to efficiently implement these protocols in Section 4.

Original: In the embedding layer, P 0 and P 1 take as inputs W E and X input , and invoke the matrix multiplication protocol Π MatMul to compute the input embedding X .
Revised: In the embedding layer, P 0 and P 1 take as inputs W E and X input , and invoke the matrix multiplication protocol Π MatMul to compute the input embedding X .

Original: This is represented as ⟨ X
Revised: This is represented as ⟨ X

Original: ⟩ =Π
Revised: ⟩ =Π

Original: MatMul ( X input , W E ) .
Revised: MatMul ( X input , W E ) .

Original: In the attention layer, Π MatMul is also invoked to generate the query, keyand value matrices.
Revised: In the attention layer, Π MatMul is also invoked to generate the query, keyand value matrices.

Original: For instance, ⟨ X Q ⟩ = Π MatMul ( ⟨ X ⟩ 1 , W Q ) + ⟨ X ⟩ 0 W Q , where ⟨ X ⟩ 0 W Qcan be computed locally by P 0 .
Revised: For instance, ⟨ X Q ⟩ = Π MatMul ( ⟨ X ⟩ 1 , W Q ) + ⟨ X ⟩ 0 W Q , where ⟨ X ⟩ 0 W Qcan be computed locally by P 0 .

Original: The same idea is used in the generation of ⟨ X K ⟩ and ⟨ X V ⟩ .
Revised: The same idea is used in the generation of ⟨ X K ⟩ and ⟨ X V ⟩ .

Original: Then, we compute X QK =
Revised: Then, we compute X QK =

Original: X Q X TK that requires two invocations of Π MatMul .
Revised: X Q X TK that requires two invocations of Π MatMul .

Original: The formulation is⟨ X
Revised: The formulation is⟨ X

Original: QK ⟩ = Π MatMul ( ⟨ X Q ⟩ 0 , ⟨ X K ⟩ 1 ) + Π MatMul ( ⟨ X Q ⟩ 1 , ⟨ X K ⟩ 0 ) + ⟨ X Q ⟩ 0 ⟨ X K ⟩ T 0 + ⟨ X Q ⟩ 1 ⟨ X K ⟩ T 1 ,where the last two terms can be evaluated by P 0 and P 1 locally.
Revised: QK ⟩ = Π MatMul ( ⟨ X Q ⟩ 0 , ⟨ X K ⟩ 1 ) + Π MatMul ( ⟨ X Q ⟩ 1 , ⟨ X K ⟩ 0 ) + ⟨ X Q ⟩ 0 ⟨ X K ⟩ T 0 + ⟨ X Q ⟩ 1 ⟨ X K ⟩ T 1 ,where the last two terms can be evaluated by P 0 and P 1 locally.

Original: After that, we evaluate⟨ (cid:101)
Revised: After that, we evaluate⟨ (cid:101)

Original: X QK ⟩ = Π Softmax ( ⟨ X QK ⟩ / √ d ) , followed by computing ⟨ (cid:101) Z ⟩ = ⟨ (cid:101) X
Revised: X QK ⟩ = Π Softmax ( ⟨ X QK ⟩ / √ d ) , followed by computing ⟨ (cid:101) Z ⟩ = ⟨ (cid:101) X

Original: QK X V ⟩ by two invocations ofΠ MatMul .
Revised: QK X V ⟩ by two invocations ofΠ MatMul .

Original: Then, the two parties invoke the LayerNorm protocol to evaluate ⟨ Z ⟩ = Π LayerNorm ( ⟨ (cid:101) Z ⟩ ) .
Revised: Then, the two parties invoke the LayerNorm protocol to evaluate ⟨ Z ⟩ = Π LayerNorm ( ⟨ (cid:101) Z ⟩ ) .

Original: In the feed-forward layer, they compute ⟨ (cid:101) Y 1 ⟩ = Π MatMul ( ⟨ Z ⟩ 1 , W 1 ) + ⟨ Z ⟩ 0 W 1 , followed by⟨ (cid:101)
Revised: In the feed-forward layer, they compute ⟨ (cid:101) Y 1 ⟩ = Π MatMul ( ⟨ Z ⟩ 1 , W 1 ) + ⟨ Z ⟩ 0 W 1 , followed by⟨ (cid:101)

Original: Y ⟩ = Π GELU ( ⟨ (cid:101)
Revised: Y ⟩ = Π GELU ( ⟨ (cid:101)

Original: Y 1 ⟩ ) .
Revised: Y 1 ⟩ ) .

Original: After that, they compute ⟨ (cid:101) Y 2 ⟩ = Π MatMul ( ⟨ (cid:101)
Revised: After that, they compute ⟨ (cid:101) Y 2 ⟩ = Π MatMul ( ⟨ (cid:101)

Original: Y ⟩ 1 , W 1 )
Revised: Y ⟩ 1 , W 1 )

Original: + ⟨ (cid:101)
Revised: + ⟨ (cid:101)

Original: Y ⟩ 0 W 1 , where⟨ (cid:101)
Revised: Y ⟩ 0 W 1 , where⟨ (cid:101)

Original: Y ⟩ 0 W 1 can be computed locally by P 0 .
Revised: Y ⟩ 0 W 1 can be computed locally by P 0 .

Original: Finally, the evaluation of the encoder completes aftercomputing ⟨ Y ⟩ = Π LayerNorm ( ⟨ (cid:101) Y 2 ⟩ ) .
Revised: Finally, the evaluation of the encoder completes aftercomputing ⟨ Y ⟩ = Π LayerNorm ( ⟨ (cid:101) Y 2 ⟩ ) .

Original: 𝐗𝐗 = 1 2 3𝐘𝐘 = 5 7 9 6 8𝒁𝒁 = 𝐗𝐗 � 𝐘𝐘 ≡ 17 23 29 7 21 3 mod 2�𝑦𝑦 = 5𝑥𝑥 

Revised: 𝐗𝐗 = 1 2 3𝐘𝐘 = 5 7 9 6 8𝒁𝒁 = 𝐗𝐗 � 𝐘𝐘 ≡ 17 23 29 7 21 3 mod 2�𝑦𝑦 = 5𝑥𝑥

Original: + 6𝑥𝑥 1 + 7𝑥𝑥 2 + 8𝑥𝑥 3 + 9𝑥𝑥 4 + 10𝑥𝑥 5 �𝑥𝑥 = 2𝑥𝑥 0 + 1𝑥𝑥 1 + 4𝑥𝑥 6 + 3𝑥𝑥̂𝑧𝑧 ≡ 10𝑥𝑥 0 +17 𝑥𝑥 1 + 20𝑥𝑥 2 + 23 𝑥𝑥 3 + 26𝑥𝑥 4 + 29 𝑥𝑥 5 + 30𝑥𝑥+ 7 𝑥𝑥 

Revised: + 6𝑥𝑥 1 + 7𝑥𝑥 2 + 8𝑥𝑥 3 + 9𝑥𝑥 4 + 10𝑥𝑥 5 �𝑥𝑥 = 2𝑥𝑥 0 + 1𝑥𝑥 1 + 4𝑥𝑥 6 + 3𝑥𝑥̂𝑧𝑧 ≡ 10𝑥𝑥 0 +17 𝑥𝑥 1 + 20𝑥𝑥 2 + 23 𝑥𝑥 3 + 26𝑥𝑥 4 + 29 𝑥𝑥 5 + 30𝑥𝑥+ 7 𝑥𝑥

Original: + 14𝑥𝑥 8 + 21 𝑥𝑥 9 + 26𝑥𝑥
Revised: + 14𝑥𝑥 8 + 21 𝑥𝑥 9 + 26𝑥𝑥

Original: + 3 𝑥𝑥 11 + 30𝑥𝑥 12 mod ( 𝑥𝑥 16 + 1,2 5 )
Revised: + 3 𝑥𝑥 11 + 30𝑥𝑥 12 mod ( 𝑥𝑥 16 + 1,2 5 )

Original: An example over ℤ 2�𝑥𝑥 � �𝑦𝑦 mod( 𝑥𝑥 16 +1,2 5 )
Revised: An example over ℤ 2�𝑥𝑥 � �𝑦𝑦 mod( 𝑥𝑥 16 +1,2 5 )

Original: Evaluation in Π MatMul
Revised: Evaluation in Π MatMul

Original:  Supporting Protocols
Revised:  Supporting Protocols

Original: 4.1 Protocol for Matrix Multiplication
Revised: 4.1 Protocol for Matrix Multiplication

Original: As shown in Section 1, naively extending well-studied matrix-vector multiplication protocols [28, 13,14] to our matrix multiplication setting results in a significant communication overhead, mainly due tofrequently interacting ciphertexts.
Revised: As shown in Section 1, naively extending well-studied matrix-vector multiplication protocols [28, 13,14] to our matrix multiplication setting results in a significant communication overhead, mainly due tofrequently interacting ciphertexts.

Original: In this work, we build a specialized matrix multiplication protocolon top of the state-of-the-art protocol, Cheetah [14].
Revised: In this work, we build a specialized matrix multiplication protocolon top of the state-of-the-art protocol, Cheetah [14].

Original: Recall that the plaintext of LHE schemes is apolynomial, which can pack a large number of inputs to amortize the overhead [28, 13].
Revised: Recall that the plaintext of LHE schemes is apolynomial, which can pack a large number of inputs to amortize the overhead [28, 13].

Original: The keycontribution of our proposed protocol is a more compact input packing approach.
Revised: The keycontribution of our proposed protocol is a more compact input packing approach.

Original: Our starting point is that polynomial multiplication implies vector inner product, if we arrange thecoefficients properly [14].
Revised: Our starting point is that polynomial multiplication implies vector inner product, if we arrange thecoefficients properly [14].

Original: As shown in Equation 1, when multiplying two polynomials of degree- N ,the ( N - 1 )-th coefficient of the resulting polynomial is the inner product of the two coefficient vectorsin opposite orders.
Revised: As shown in Equation 1, when multiplying two polynomials of degree- N ,the ( N - 1 )-th coefficient of the resulting polynomial is the inner product of the two coefficient vectorsin opposite orders.

Original: By using an appropriate arrangement of coefficients, this idea can be extendedto our matrix multiplications, since they consist of a set of inner products.
Revised: By using an appropriate arrangement of coefficients, this idea can be extendedto our matrix multiplications, since they consist of a set of inner products.

Original: To this end, we give thedefinitions of two input packing functions π L : Z m × n 2 ℓ → A N, 2 ℓ and π R : Z n × k 2 ℓ → A N, 2 ℓ as follows:
Revised: To this end, we give thedefinitions of two input packing functions π L : Z m × n 2 ℓ → A N, 2 ℓ and π R : Z n × k 2 ℓ → A N, 2 ℓ as follows:

Original: [Equation]
Revised: [Equation]

Original: ] , for i ∈ [ m ] , j ∈ [ n ] ˆ y = π R ( Y ) , s.t. , ˆ y [ j · n + i ]
Revised: ] , for i ∈ [ m ] , j ∈ [ n ] ˆ y = π R ( Y ) , s.t. , ˆ y [ j · n + i ]

Original: =
Revised: =

Original: Y [ i, j ] , for i ∈ [ n ] , j ∈ [ k ] where all other coefficients of ˆ x and ˆ y are set to 0.
Revised: Y [ i, j ] , for i ∈ [ n ] , j ∈ [ k ] where all other coefficients of ˆ x and ˆ y are set to 0.

Original: Multiplication of polynomials ˆ z
Revised: Multiplication of polynomials ˆ z

Original: = ˆ x · ˆ y directlygives the result of matrix multiplication Z = XY mod 2 ℓ in some of ˆ z ’s coefficients.
Revised: = ˆ x · ˆ y directlygives the result of matrix multiplication Z = XY mod 2 ℓ in some of ˆ z ’s coefficients.

Original: We formalizethis process as below, and give a toy example in Figure 2.
Revised: We formalizethis process as below, and give a toy example in Figure 2.

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Assuming mnk ≤ N and given two polynomials ˆ x = π L ( X ) and
Revised: Assuming mnk ≤ N and given two polynomials ˆ x = π L ( X ) and

Original: ˆ y = π R ( Y ) , thematrix multiplication Z = XY mod 2 ℓ can be evaluated via the product ˆ z
Revised: ˆ y = π R ( Y ) , thematrix multiplication Z = XY mod 2 ℓ can be evaluated via the product ˆ z

Original: = ˆ x · ˆ y , where Z [ i, j ] iscomputed in ˆ z
Revised: = ˆ x · ˆ y , where Z [ i, j ] iscomputed in ˆ z

Original: [ i · n · k + ( j + 1) · n − 1] for i ∈ [ m ] and j ∈ [ k ] .
Revised: [ i · n · k + ( j + 1) · n − 1] for i ∈ [ m ] and j ∈ [ k ] .

Original: We defer the proof to Appendix A.2.1.
Revised: We defer the proof to Appendix A.2.1.

Original: When mnk > N , we first partition the matrices X , Yinto sub-matrices of m w × n w and n w × k w elements, respectively, such that m w n w k w ≤ N .
Revised: When mnk > N , we first partition the matrices X , Yinto sub-matrices of m w × n w and n w × k w elements, respectively, such that m w n w k w ≤ N .

Original: Zero-padding is required when m w ∤ m , n w ∤ n or k w ∤ k .
Revised: Zero-padding is required when m w ∤ m , n w ∤ n or k w ∤ k .

Original: The protocol is shown in Algorithm 1.
Revised: The protocol is shown in Algorithm 1.

Original: Complexity and security analysis .
Revised: Complexity and security analysis .

Original: For complexity, totally, two parties interact kk w ( mm w + nn w )ciphertexts, and operate with O ( mnk/N ) homomorphic additions and multiplications.
Revised: For complexity, totally, two parties interact kk w ( mm w + nn w )ciphertexts, and operate with O ( mnk/N ) homomorphic additions and multiplications.

Original: We formalizethe selection of parameters m w , n w , k w as an optimization problem, to minimize the communicationcost.
Revised: We formalizethe selection of parameters m w , n w , k w as an optimization problem, to minimize the communicationcost.

Original: Through our analysis in Appendix A.2.2, in general scenarios, our method theoretically achieves 202 √ m communication improvement over Cheetah [14].
Revised: Through our analysis in Appendix A.2.2, in general scenarios, our method theoretically achieves 203 √ m communication improvement over Cheetah [14].

Original: Notably, when mnk ≤ N , we reduce thecommuncation cost by a factor of m , since Cheetah encodes each row of the matrix into a ciphertext.
Revised: Notably, when mnk ≤ N , we reduce thecommuncation cost by a factor of m , since Cheetah encodes each row of the matrix into a ciphertext.

Original: Besides, the security proof is shown in Appendix A.2.3.
Revised: Besides, the security proof is shown in Appendix A.2.3.

Original: 4.2 Protocols for Non-linear Functions
Revised: 4.2 Protocols for Non-linear Functions

Original: Our non-linear protocols rely on several underlying protocols from the state-of-the-art works [13, 17].
Revised: Our non-linear protocols rely on several underlying protocols from the state-of-the-art works [13, 17].

Original: In Figure 3, we enumerate the inputs and outputs of these protocols and then use them as a black box(See Appendix A.3.2 for details).
Revised: In Figure 3, we enumerate the inputs and outputs of these protocols and then use them as a black box(See Appendix A.3.2 for details).

Original: On this basis, we provide efficient protocols for Softmax, GELU,and LayerNorm with specific optimizations.
Revised: On this basis, we provide efficient protocols for Softmax, GELU,and LayerNorm with specific optimizations.

Original: Additional details and security analysis are shown in
Revised: Additional details and security analysis are shown in

Original: Appendices A.3.3 and A.3.4.
Revised: Appendices A.3.3 and A.3.4.

Original: 4.2.1 Softmax
Revised: 4.2.1 Softmax

Original: To evaluate attention layers, we need an efficient protocol to compute Softmax on secret-sharedvalues.
Revised: To evaluate attention layers, we need an efficient protocol to compute Softmax on secret-sharedvalues.

Original: In particular, for a vector x ∈ Z d 2 ℓ , the Softmax function is denoted as Softmax i ( x ) =  share ⟨ Z ⟩ 0 .
Revised: In particular, for a vector x ∈ Z d 2 ℓ , the Softmax function is denoted as Softmax i ( x ) = the share ⟨ Z ⟩ 0 .

Original: On receiving the ciphertexts { CT  α,γ } from P 0 , P 1 computes ⟨ ˆ z β,γ ⟩ 1 =
Revised: On receiving the ciphertexts (cid:8) CT ′ α,γ (cid:9) from P 0 , P 1 computes ⟨ ˆ z α,γ ⟩ 1 =

Original: Dec (CT  α,γ ) that are decoded to ⟨ Z ⟩ 1 using the method in Theorem 4.1.
Revised: Dec (CT ′ α,γ ) that are decoded to ⟨ Z ⟩ 1 using the method in Theorem 4.1.

Original: Multiply Π Mul OT Input :
Revised: Multiply Π Mul OT Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 s.t.
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 s.t.

Original: 𝒛𝒛 = 𝒙𝒙𝒙𝒙 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 , 𝑦𝑦 𝑏𝑏
Revised: 𝒛𝒛 = 𝒙𝒙𝒙𝒙 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 , 𝑦𝑦 𝑏𝑏

Original: Compare Π CMP s.t.
Revised: Compare Π CMP s.t.

Original: 𝒛𝒛 = 𝟏𝟏 { 𝒙𝒙 > 𝟎𝟎 }
Revised: 𝒛𝒛 = 𝟏𝟏 { 𝒙𝒙 > 𝟎𝟎 }

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏𝐵𝐵 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏𝐵𝐵 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: NegExp Π nExp s.t.
Revised: NegExp Π nExp s.t.

Original: 𝒛𝒛 = 𝒆𝒆 𝒙𝒙 , 𝒙𝒙 <
Revised: 𝒛𝒛 = 𝒆𝒆 𝒙𝒙 , 𝒙𝒙 <

Original: RecipSqrt Π rSqrt s.t .
Revised: RecipSqrt Π rSqrt s.t .

Original: 𝒛𝒛 = 𝟏𝟏 𝒙𝒙 , 𝒙𝒙 > 𝝐𝝐
Revised: 𝒛𝒛 = 𝟏𝟏 𝒙𝒙 , 𝒙𝒙 > 𝝐𝝐

Original: Recip Π Recip s.t.
Revised: Recip Π Recip s.t.

Original: 𝒛𝒛 = 𝟏𝟏 / 𝒙𝒙
Revised: 𝒛𝒛 = 𝟏𝟏 / 𝒙𝒙

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 e x i / (cid:80) j ∈
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 e x i / (cid:80) j ∈

Original: [ d ] e x j for i ∈
Revised: [ d ] e x j for i ∈

Original: [ d ] .
Revised: [ d ] .

Original: The main challenge is to efficiently compute the underlying exponentialfunction.
Revised: The main challenge is to efficiently compute the underlying exponentialfunction.

Original: Following the idea from [11, 29], we first normalize the input vector by x − max i ∈
Revised: Following the idea from [11, 29], we first normalize the input vector by x − max i ∈

Original: [ d ] x i ,which is always negative, and then invoke the existing exponential protocol Π nExp in Figure 3 that onlyevaluates exponentiation on negative inputs.
Revised: [ d ] x i ,which is always negative, and then invoke the existing exponential protocol Π nExp in Figure 3 that onlyevaluates exponentiation on negative inputs.

Original: A simple analysis shows softmax ( x − max i ∈
Revised: A simple analysis shows softmax ( x − max i ∈

Original: [ d ] x i ) isequal to softmax ( x ) .
Revised: [ d ] x i ) isequal to softmax ( x ) .

Original: We evaluate max using a tree-reduction protocol, denoted by Π max .
Revised: We evaluate max using a tree-reduction protocol, denoted by Π max .

Original: Specifically,we arrange the vector x ∈ Z d 2 ℓ into a 2-ary tree with the depth of log d , and evaluate the tree in atop-down fashion [30].
Revised: Specifically,we arrange the vector x ∈ Z d 2 ℓ into a 2-ary tree with the depth of log d , and evaluate the tree in atop-down fashion [30].

Original: In each comparison of two secret-shared elements x i and x j , we reduce itto the invocations of Π CMP and Π Mul OT , i.e., max( x i , x j ) =
Revised: In each comparison of two secret-shared elements x i and x j , we reduce itto the invocations of Π CMP and Π Mul OT , i.e., max( x i , x j ) =

Original: Π
Revised: Π

Original: Mul OT ( x i − x j , Π CMP
Revised: Mul OT ( x i − x j , Π CMP

Original: ( x i − x j ))
Revised: ( x i − x j ))

Original: + x j .
Revised: + x j .

Original: With the above insight, the Softmax protocol is detailed in Algorithm 2.
Revised: With the above insight, the Softmax protocol is detailed in Algorithm 2.

Original: SIRNN [17] also provides a solution for the generic exponential protocol by extending Π nExp .
Revised: SIRNN [17] also provides a solution for the generic exponential protocol by extending Π nExp .

Original: Theidea is that the exponential of x equals 1 nExp ( − x ) if x ≥ 0 , and nExp( x ) otherwise, where nExpis the same as the exponential function except for negative inputs.
Revised: Theidea is that the exponential of x equals 1 nExp ( − x ) if x ≥ 0 , and nExp( x ) otherwise, where nExpis the same as the exponential function except for negative inputs.

Original: Compared with our solution, itadditionally requires d calls to the reciprocal protocol Π Recip and multiplication protocol Π MUT OT insoftmax.
Revised: Compared with our solution, itadditionally requires d calls to the reciprocal protocol Π Recip and multiplication protocol Π MUT OT insoftmax.

Original: Besides, compared with the generic library MP-SPDZ [16] that provides native support forexponentiation, our protocols achieve orders of magnitude improvement as shown in Section 5.2.
Revised: Besides, compared with the generic library MP-SPDZ [16] that provides native support forexponentiation, our protocols achieve orders of magnitude improvement as shown in Section 5.2.

Original: Algorithm 2 Secure Softmax Protocol
Revised: Algorithm 2 Secure Softmax Protocol

Original: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈
Revised: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈

Original: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.
Revised: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.

Original: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈
Revised: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈

Original: Z d 2 ℓ , ⟨ y ⟩ 1 ∈ Z d 2 ℓ , respectively, where y =
Revised: Z d 2 ℓ , ⟨ y ⟩ 1 ∈ Z d 2 ℓ , respectively, where y =

Original: Softmax ( x ) .
Revised: Softmax ( x ) .

Original: 1: P 0 , P 1 invoke Π max ( x ) to compute ⟨ max ( x ) ⟩ , where max ( x ) = max i ∈
Revised: 1: P 0 , P 1 invoke Π max ( x ) to compute ⟨ max ( x ) ⟩ , where max ( x ) = max i ∈

Original: [ d ] x i .
Revised: [ d ] x i .

Original: For i ∈ [ d ] , P 0 , P 1 invoke Π nExp on input ⟨ ¯ x i ⟩ , and learn ⟨ e ¯ x i ⟩ , where ¯ x i =
Revised: For i ∈ [ d ] , P 0 , P 1 invoke Π nExp on input ⟨ ¯ x i ⟩ , and learn ⟨ e ¯ x i ⟩ , where ¯ x i =

Original: x i − max ( x ) .
Revised: x i − max ( x ) .

Original: 3: P 0 , P 1 invoke Π Recip with inputs ⟨ (cid:80) i ∈ [ d ] e ¯ x i ⟩ and learn ⟨ 1 / (cid:80) i ∈
Revised: 3: P 0 , P 1 invoke Π Recip with inputs ⟨ (cid:80) i ∈ [ d ] e ¯ x i ⟩ and learn ⟨ 1 / (cid:80) i ∈

Original: [ d ] e ¯ x i ⟩ . 4:
Revised: [ d ] e ¯ x i ⟩ . 4:

Original: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 / (cid:80) i ∈ [ d ] e ¯ x i ⟩ and ⟨ e ¯ x i ⟩ , and set outputs as ⟨ y i⟩ .
Revised: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 / (cid:80) i ∈ [ d ] e ¯ x i ⟩ and ⟨ e ¯ x i ⟩ , and set outputs as ⟨ y i

Original:  4.2.2 GELU
Revised: . 4.2.2 GELU

Original: Rather than crypto-friendly ReLU [13], Transformer-based models utilize GELU activations [15],which can be represented as GELU ( x ) = 0 .
Revised: Rather than crypto-friendly ReLU [13], Transformer-based models utilize GELU activations [15],which can be represented as GELU ( x ) = 0 .

Original: 5 x (cid:16) 1 + Tanh (cid:104)(cid:112) 2 /π (cid:0) x + 0 .
Revised: 5 x (cid:16) 1 + Tanh (cid:104)(cid:112) 2 /π (cid:0) x + 0 .

Original: 044715 x 3 (cid:1)(cid:105)(cid:17) .
Revised: 044715 x 3 (cid:1)(cid:105)(cid:17) .

Original: The com- plete protocol is shown in Algorithm 3, in which we provide two insights to reduce the cost of GELU .
Revised: The com- plete protocol is shown in Algorithm 3, in which we provide two insights to reduce the cost of GELU .

Original: First, we present an optimized protocol for the square of a secret-shared input ⟨ x ⟩ .
Revised: First, we present an optimized protocol for the square of a secret-shared input ⟨ x ⟩ .

Original: This relies on theobservation: x 2 = ⟨ x ⟩ 20 + ⟨ x ⟩ 21 + 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , where the first two terms can be locally computed by
Revised: This relies on theobservation: x 2 = ⟨ x ⟩ 20 + ⟨ x ⟩ 21 + 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , where the first two terms can be locally computed by

Original: P 0 and P 1 .
Revised: P 0 and P 1 .

Original: We only invoke OT protocols to compute the cross term 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , and the optimizedoverhead is half that of the multiplication protocol Π Mul OT .
Revised: We only invoke OT protocols to compute the cross term 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , and the optimizedoverhead is half that of the multiplication protocol Π Mul OT .

Original: Second, we further optimize the evaluation of Tanh ( x ) =
Revised: Second, we further optimize the evaluation of Tanh ( x ) =

Original: e 2 x − 1 e 2 x +1 .
Revised: e 2 x − 1 e 2 x +1 .

Original: We observe that the sign of x isequal to the sign of Tanh ( x ) , which allows us to leverage the negative-only exponential protocolΠ nExp almost for free.
Revised: We observe that the sign of x isequal to the sign of Tanh ( x ) , which allows us to leverage the negative-only exponential protocolΠ nExp almost for free.

Original: At a high level, our Tanh protocol first learns the sign of the input x , and thenevaluates Tanh on the negative input
Revised: At a high level, our Tanh protocol first learns the sign of the input x , and thenevaluates Tanh on the negative input

Original: ¯ x with the constraint | ¯ x | = | x | .
Revised: ¯ x with the constraint | ¯ x | = | x | .

Original: Finally, the protocol generatesthe real output Tanh ( x ) that equals Tanh (¯ x ) if x ≤ 0 , and − Tanh (¯ x ) otherwise.
Revised: Finally, the protocol generatesthe real output Tanh ( x ) that equals Tanh (¯ x ) if x ≤ 0 , and − Tanh (¯ x ) otherwise.

Original: Our leaner Tanhprotocol is given in Algorithm 5 of Appendix A.3.1.
Revised: Our leaner Tanhprotocol is given in Algorithm 5 of Appendix A.3.1.

Original: SIRNN [17] recently proposed the most efficient
Revised: SIRNN [17] recently proposed the most efficient

Original: Tanh protocol with the insight of Tanh ( x ) =
Revised: Tanh protocol with the insight of Tanh ( x ) =

Original: Sigmoid (2 x ) − 1 , where Sigmoid ( x ) = 1 1+ e − x .
Revised: Sigmoid (2 x ) − 1 , where Sigmoid ( x ) = 1 1+ e − x .

Original: However, the evaluation of Sigmoid uses the same idea as the general exponential protocol, as shownin Section 4.2.1.
Revised: However, the evaluation of Sigmoid uses the same idea as the general exponential protocol, as shownin Section 4.2.1.

Original: Compared with the protocol in SIRNN, our recipe for Tanh saves one invocation ofthe reciprocal protocol Π Recip and multiplication protocol Π Mul OT .
Revised: Compared with the protocol in SIRNN, our recipe for Tanh saves one invocation ofthe reciprocal protocol Π Recip and multiplication protocol Π Mul OT .

Original: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈
Revised: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈

Original: Z 2 ℓ , ⟨ x ⟩ 1 ∈ Z 2 ℓ , respectively.
Revised: Z 2 ℓ , ⟨ x ⟩ 1 ∈ Z 2 ℓ , respectively.

Original: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈
Revised: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈

Original: Z 2 ℓ , ⟨ y ⟩ 1 ∈ Z 2 ℓ , respectively, where y = GELU ( x ) .
Revised: Z 2 ℓ , ⟨ y ⟩ 1 ∈ Z 2 ℓ , respectively, where y = GELU ( x ) .

Original: 1: P
Revised: 1: P

Original: , P 1 invoke Π Mul OT with inputs ⟨ x ⟩ , and set their outputs as ⟨ z ⟩
Revised: , P 1 invoke Π Mul OT with inputs ⟨ x ⟩ , and set their outputs as ⟨ z ⟩

Original: =
Revised: =

Original: Fix ( (cid:112) 2 /π )( ⟨ x ⟩ + Fix (0 . 044715) ⟨ x ⟩ 3 ) 2: P 0 , P 1 invoke Π Tanh with inputs ⟨ z ⟩ , and set their outputs as ⟨ Tanh ( z ) ⟩ .
Revised: Fix ( (cid:112) 2 /π )( ⟨ x ⟩ + Fix (0 . 044715) ⟨ x ⟩ 3 ) 2: P 0 , P 1 invoke Π Tanh with inputs ⟨ z ⟩ , and set their outputs as ⟨ Tanh ( z ) ⟩ .

Original: 3: P 0 , P 1 invoke Π Mul OT with inputs ⟨ Fix (0 . 5) x ⟩ and ⟨ Fix (1) + Tanh ( z ) ⟩ , and output ⟨ y ⟩ .
Revised: 3: P 0 , P 1 invoke Π Mul OT with inputs ⟨ Fix (0 . 5) x ⟩ and ⟨ Fix (1) + Tanh ( z ) ⟩ , and output ⟨ y ⟩ .

Original: 4.2.3 LayerNorm
Revised: 4.2.3 LayerNorm

Original: For a vector x ∈ Z d 2 ℓ , the LayerNorm function is denoted by LayerNorm
Revised: For a vector x ∈ Z d 2 ℓ , the LayerNorm function is denoted by LayerNorm

Original: i ( x ) = γ ( x i − µ ) /σ
Revised: i ( x ) = γ ( x i − µ ) /σ

Original: + βfor i ∈
Revised: + βfor i ∈

Original: [ d ] , where µ = (cid:80) i ∈ [ d ] x i /d and σ = (cid:113)(cid:80) i ∈
Revised: [ d ] , where µ = (cid:80) i ∈ [ d ] x i /d and σ = (cid:113)(cid:80) i ∈

Original: [ d ] ( x i − µ ) 2 .
Revised: [ d ] ( x i − µ ) 2 .

Original: In contrast to batch normalization(BN) in CNNs that is evaluated for free [14], LayerNorm requires multiplication and reciprocalsquare root operations.
Revised: In contrast to batch normalization(BN) in CNNs that is evaluated for free [14], LayerNorm requires multiplication and reciprocalsquare root operations.

Original: In our implementation, we observe that the multiplication dominates theoverhead of LayerNorm.
Revised: In our implementation, we observe that the multiplication dominates theoverhead of LayerNorm.

Original: To address this issue, we adopt the same idea in GELU to compute thesquare of x i − µ , which saves half of the communication and computation costs.
Revised: To address this issue, we adopt the same idea in GELU to compute thesquare of x i − µ , which saves half of the communication and computation costs.

Original: Besides, inspired by the optimization in BN [14], we apply a LayerNorm merge technique to furtherreduce the overhead.
Revised: Besides, inspired by the optimization in BN [14], we apply a LayerNorm merge technique to furtherreduce the overhead.

Original: Specifically, we make the observation that the weights γ and β of the LayerNormlayer are already known by P 0 .
Revised: Specifically, we make the observation that the weights γ and β of the LayerNormlayer are already known by P 0 .

Original: As a result, P 0 first multiplies the scale factor γ with the weights ofthe next linear layer.
Revised: As a result, P 0 first multiplies the scale factor γ with the weights ofthe next linear layer.

Original: After invoking the linear layer protocol on the scaled weights, P 0 adds the shiftweight σ to his additive share.
Revised: After invoking the linear layer protocol on the scaled weights, P 0 adds the shiftweight σ to his additive share.

Original: The optimized protocol for LayerNorm is presented in Algorithm 4.
Revised: The optimized protocol for LayerNorm is presented in Algorithm 4.

Original: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈
Revised: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈

Original: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.
Revised: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.

Original: Output: P 0 , P 1 get ⟨ y ⟩ 0 , ⟨ y ⟩ 1 , respectively, where y = LayerNorm ( x ) .
Revised: Output: P 0 , P 1 get ⟨ y ⟩ 0 , ⟨ y ⟩ 1 , respectively, where y = LayerNorm ( x ) .

Original: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT to compute ⟨ ( x i − µ ) 2 ⟩ , where µ = (cid:80) i ∈ [ d ] x i /d .
Revised: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT to compute ⟨ ( x i − µ ) 2 ⟩ , where µ = (cid:80) i ∈ [ d ] x i /d .

Original: 2: P 0 , P 1 invoke Π rSqrt with inputs ⟨ (cid:80) i ∈
Revised: 2: P 0 , P 1 invoke Π rSqrt with inputs ⟨ (cid:80) i ∈

Original: [ d ] ( x i − µ ) 2 ⟩ to learn output ⟨ 1 σ ⟩ .
Revised: [ d ] ( x i − µ ) 2 ⟩ to learn output ⟨ 1 σ ⟩ .

Original: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 σ ⟩ and ⟨ x i − µ ⟩ , and set outputs as ⟨ y ⟩ .
Revised: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 σ ⟩ and ⟨ x i − µ ⟩ , and set outputs as ⟨ y ⟩ .

Original:  Evaluation
Revised:  Evaluation

Original: 5.1 Experimental Setup
Revised: 5.1 Experimental Setup

Original: Implementation.
Revised: Implementation.

Original: Iron is built on top of the SEAL library [31] and the EMP toolkit [32] in C++.
Revised: Iron is built on top of the SEAL library [31] and the EMP toolkit [32] in C++.

Original: Wealso use the EzPC framework [33].
Revised: Wealso use the EzPC framework [33].

Original: This framework compiles high-level TensorFlow code to securecomputation protocols, which are then executed by our designed cryptographic backends.
Revised: This framework compiles high-level TensorFlow code to securecomputation protocols, which are then executed by our designed cryptographic backends.

Original: Like [17],
Revised: Like [17],

Original: [Table]
Revised: [Table]

Original: we simulate a LAN network setting, where the bandwidth is 377 MBps and the echo latency is 0.8ms.
Revised: we simulate a LAN network setting, where the bandwidth is 377 MBps and the echo latency is 0.8ms.

Original: All the following experiments are performed on AWS c5.9xlarge instances with Intel Xeon 8000series CPUs at 3.6GHz.
Revised: All the following experiments are performed on AWS c5.9xlarge instances with Intel Xeon 8000series CPUs at 3.6GHz.

Original: Datasets and Models.
Revised: Datasets and Models.

Original: We evaluate Iron on four NLP models from [34, 35]: BERT-Tiny, BERTMedium, BERT-Base and BERT-Large.
Revised: We evaluate Iron on four NLP models from [34, 35]: BERT-Tiny, BERTMedium, BERT-Base and BERT-Large.

Original: These models are parameterized by three hyper-parameters:the number of blocks L , the dimension of representations H and the number of input tokens N (referto Appendix A.4.1 for the hyper-parameters of these models).
Revised: These models are parameterized by three hyper-parameters:the number of blocks L , the dimension of representations H and the number of input tokens N (referto Appendix A.4.1 for the hyper-parameters of these models).

Original: We train the models for four NLP tasksover the datasets of MNLI, MRPC, SST-2, and QNLI from GLUE benchmarks [18].
Revised: We train the models for four NLP tasksover the datasets of MNLI, MRPC, SST-2, and QNLI from GLUE benchmarks [18].

Original: We report theaccuracy of private inference in Appendix A.4.2, which is comparable to plaintext accuracy due toour numerically precise protocols [17].
Revised: We report theaccuracy of private inference in Appendix A.4.2, which is comparable to plaintext accuracy due toour numerically precise protocols [17].

Original: 5.2 Microbenchmark Evaluation
Revised: 5.2 Microbenchmark Evaluation

Original: Matrix multiplication .
Revised: Matrix multiplication .

Original: In the left part of Table 1, we compare the performance of the proposedmatrix multiplication protocol with the state-of-the-art counterparts of Cheetah [14] and SIRNN [17].
Revised: In the left part of Table 1, we compare the performance of the proposedmatrix multiplication protocol with the state-of-the-art counterparts of Cheetah [14] and SIRNN [17].

Original: Compared with Cheetah, our rumtime is 3 ∼ 26 × faster, and our communication cost is 8 ∼ 25 ×lower, depending on the input size.
Revised: Compared with Cheetah, our rumtime is 3 ∼ 26 × faster, and our communication cost is 8 ∼ 25 ×lower, depending on the input size.

Original: Notably, for small-size matrices (e.g., ones with 32 × 8 and 8 × 16elements), our protocol only requires 0.11 MB communication and 6 ms runtime, while Cheetahachieves the computation with 2.79 MB and 160 ms.
Revised: Notably, for small-size matrices (e.g., ones with 32 × 8 and 8 × 16elements), our protocol only requires 0.11 MB communication and 6 ms runtime, while Cheetahachieves the computation with 2.79 MB and 160 ms.

Original: The reason is that our protocol encrypts thewhole matrix into a single ciphertext, while Cheetah could only encrypt each row into a ciphertext,totally m ciphertexts ( m is the number of rows of the output matrix).
Revised: The reason is that our protocol encrypts thewhole matrix into a single ciphertext, while Cheetah could only encrypt each row into a ciphertext,totally m ciphertexts ( m is the number of rows of the output matrix).

Original: Moreover, compared with
Revised: Moreover, compared with

Original: SIRNN that implements the most efficient OT-based matrix multiplication protocol, our protocolincurs up to two orders of magnitude less communication and an order of magnitude shorter runtime.
Revised: SIRNN that implements the most efficient OT-based matrix multiplication protocol, our protocolincurs up to two orders of magnitude less communication and an order of magnitude shorter runtime.

Original: Non-linear functions .
Revised: Non-linear functions .

Original: The right part of Table 1 shows the comparison of our Softmax, GELU and
Revised: The right part of Table 1 shows the comparison of our Softmax, GELU and

Original: LayerNorm protocols with the generic MP-SPDZ framework [16] and the state-of-the-art SIRNNlibrary [17] for math functions.
Revised: LayerNorm protocols with the generic MP-SPDZ framework [16] and the state-of-the-art SIRNNlibrary [17] for math functions.

Original: It is worth noting that we implement some functions that theseframeworks did not provide before.
Revised: It is worth noting that we implement some functions that theseframeworks did not provide before.

Original: In particular, GELU and LayerNorm are implemented in MPSPDZ by calling its built-in functions for tanh, square root, and reciprocal, while we add Softmaxand GELU protocols in SIRNN utilizing sigmoid and exponent functions.
Revised: In particular, GELU and LayerNorm are implemented in MPSPDZ by calling its built-in functions for tanh, square root, and reciprocal, while we add Softmaxand GELU protocols in SIRNN utilizing sigmoid and exponent functions.

Original: As shown in this table,our protocols are orders of magnitude better than those of MP-SPDZ, in terms of both runtimeand communication.
Revised: As shown in this table,our protocols are orders of magnitude better than those of MP-SPDZ, in terms of both runtimeand communication.

Original: In particular, for the communication cost, our protocols achieve 785 ∼ 993 ×improvement.
Revised: In particular, for the communication cost, our protocols achieve 785 ∼ 993 ×improvement.

Original: Moreover, while our protocols are built upon the underlying protocols from SIRNN, wealso achieve 1 .
Revised: Moreover, while our protocols are built upon the underlying protocols from SIRNN, wealso achieve 1 .

Original: 3 ∼ 1 . 8 × lower runtime and 1 .
Revised: 3 ∼ 1 . 8 × lower runtime and 1 .

Original: 4 ∼ 1 . 8 × lower communication due to our customizedoptimizations.
Revised: 4 ∼ 1 . 8 × lower communication due to our customizedoptimizations.

Original: Such an improvement gives us significant savings in communication complexity, sincethe non-linear layer protocols dominate the overall overhead, as described below.
Revised: Such an improvement gives us significant savings in communication complexity, sincethe non-linear layer protocols dominate the overall overhead, as described below.

Original: 5.3 End-to-end Inference Evaluation
Revised: 5.3 End-to-end Inference Evaluation

Original: Comparison with prior methods.
Revised: Comparison with prior methods.

Original: In Figure 4, we evaluate our protocols on 4 BERT modelscompared with SIRNN [17].
Revised: In Figure 4, we evaluate our protocols on 4 BERT modelscompared with SIRNN [17].

Original: It is observed that our rumtime is 3 .
Revised: It is observed that our rumtime is 3 .

Original: 3 ∼ 11 .
Revised: 3 ∼ 11 .

Original: 83 × faster than that of
Revised: 83 × faster than that of

Original: SIRNN, and our communication cost is 3 .
Revised: SIRNN, and our communication cost is 3 .

Original: 47 ∼ 14 .
Revised: 47 ∼ 14 .

Original: 11 × lower over four models.
Revised: 11 × lower over four models.

Original: Moreover, ourperformance gains scale up as the model size grows.
Revised: Moreover, ourperformance gains scale up as the model size grows.

Original: This is because our protocols achieve betteramortized overhead when processing large-scale evaluations.
Revised: This is because our protocols achieve betteramortized overhead when processing large-scale evaluations.

Original: We also compare the end-to-end privateinference with MP-SPDZ in Appendix A.4.2.
Revised: We also compare the end-to-end privateinference with MP-SPDZ in Appendix A.4.2.

Original: The results show that our protocols are orders ofmagnitude better, in terms of both time and communication costs.
Revised: The results show that our protocols are orders ofmagnitude better, in terms of both time and communication costs.

Original: This is in line with the observationin SIRNN that specialized protocols are more communication efficient than generic alternatives.
Revised: This is in line with the observationin SIRNN that specialized protocols are more communication efficient than generic alternatives.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Performance breakdown.
Revised: Performance breakdown.

Original: In Figure 5, we present the runtime and communication breakdownof Iron on four BERT models.
Revised: In Figure 5, we present the runtime and communication breakdownof Iron on four BERT models.

Original: For clarity, we just report the result of one encoder.
Revised: For clarity, we just report the result of one encoder.

Original: Recall thatour private inference can be divided into linear protocols and non-linear protocols.
Revised: Recall thatour private inference can be divided into linear protocols and non-linear protocols.

Original: For linearprotocols, we observe as the model size increases, the proportion of communication overhead remainsapproximately constant, accounting for 12 ∼ 13% .
Revised: For linearprotocols, we observe as the model size increases, the proportion of communication overhead remainsapproximately constant, accounting for 12 ∼ 13% .

Original: This shows that our compact ciphertext encodingmethod effectively reduces the size of communication.
Revised: This shows that our compact ciphertext encodingmethod effectively reduces the size of communication.

Original: For non-linear functions, as the model sizeincreases, the proportion of computation overhead gradually decreases, from 84% in BERT-Tinyto 76% in BERT-Large.
Revised: For non-linear functions, as the model sizeincreases, the proportion of computation overhead gradually decreases, from 84% in BERT-Tinyto 76% in BERT-Large.

Original: The savings come from amortizing the communication and computationcosts by packing data from large tensors.
Revised: The savings come from amortizing the communication and computationcosts by packing data from large tensors.

Original: Despite such advantages, the main bottleneck of our workis the communication overhead of non-linear layers.
Revised: Despite such advantages, the main bottleneck of our workis the communication overhead of non-linear layers.

Original: However, it is an open problem to solve thecommunication issue while maintaining the model accuracy.
Revised: However, it is an open problem to solve thecommunication issue while maintaining the model accuracy.

Original:  Discussion
Revised:  Discussion

Original: Pushing expensive operations into an offline phase.
Revised: Pushing expensive operations into an offline phase.

Original: Our framework is readily extended to the preprocessing model, including an offline client-input independent phase and an online input-dependentphase.
Revised: Our framework is readily extended to the preprocessing model, including an offline client-input independent phase and an online input-dependentphase.

Original: This paradigm has been instantiated in private CNN inference [36, 8], and the results show thatabout 99% of the cryptographic overhead can be moved to the offline phase.
Revised: This paradigm has been instantiated in private CNN inference [36, 8], and the results show thatabout 99% of the cryptographic overhead can be moved to the offline phase.

Original: We briefly outline howto build our protocols with this paradigm.
Revised: We briefly outline howto build our protocols with this paradigm.

Original: For linear operations, we can generate in advance Beaver’striple in the matrix form [21, 36] using our matrix multiplication protocol.
Revised: For linear operations, we can generate in advance Beaver’striple in the matrix form [21, 36] using our matrix multiplication protocol.

Original: In the online phase,these triples are consumed, and it additionally requires non-cryptographic operations and plaintextcommunication.
Revised: In the online phase,these triples are consumed, and it additionally requires non-cryptographic operations and plaintextcommunication.

Original: For non-linear layers, almost all operations crucially rely on the OT protocols, whichcan be pre-generated in the offline phase [37, 14].
Revised: For non-linear layers, almost all operations crucially rely on the OT protocols, whichcan be pre-generated in the offline phase [37, 14].

Original: Like linear protocols, the online phase is cheap.
Revised: Like linear protocols, the online phase is cheap.

Original: Using mixed-bitwidths in private inference.
Revised: Using mixed-bitwidths in private inference.

Original: Iron works with a uniform bitwidth, which is requiredto be large enough to accommodate all intermediate values.
Revised: Iron works with a uniform bitwidth, which is requiredto be large enough to accommodate all intermediate values.

Original: While effectively avoiding integeroverflows, our protocols may cause intensive communication, since the performance of non-linearoperators depends critically on bitwidths.
Revised: While effectively avoiding integeroverflows, our protocols may cause intensive communication, since the performance of non-linearoperators depends critically on bitwidths.

Original: Taking inspiration from [17, 38], one possible optimizationis to employ non-uniform (mixed) bitwidths, operate in low bitwidths and switch to high bitwidthsonly when necessary.
Revised: Taking inspiration from [17, 38], one possible optimizationis to employ non-uniform (mixed) bitwidths, operate in low bitwidths and switch to high bitwidthsonly when necessary.

Original: To this end, the mixed-precision model should be generated with propercompilers by using techiniques like quantization [39, 40].
Revised: To this end, the mixed-precision model should be generated with propercompilers by using techiniques like quantization [39, 40].

Original: With mixed-bitwidths fixed-point models,our protocols can be integrated seamlessly.
Revised: With mixed-bitwidths fixed-point models,our protocols can be integrated seamlessly.

Original: Applying orthogonal model optimizations.
Revised: Applying orthogonal model optimizations.

Original: We can improve performance by simplifying the modelarchitecture such as model pruning and advanced neural architecture search [41, 42], which havebeen commonly adopted in private CNN inference [12, 43, 44].
Revised: We can improve performance by simplifying the modelarchitecture such as model pruning and advanced neural architecture search [41, 42], which havebeen commonly adopted in private CNN inference [12, 43, 44].

Original: Like prior works [12, 43, 44] thatfind and tailor models to the requirements of private inference, we can define a proper search spacewith the goal of decreasing the overhead, e.g., substituting costly Softmax and GELU functions, orreducing the matrices’ dimension in matrix multiplication.
Revised: Like prior works [12, 43, 44] thatfind and tailor models to the requirements of private inference, we can define a proper search spacewith the goal of decreasing the overhead, e.g., substituting costly Softmax and GELU functions, orreducing the matrices’ dimension in matrix multiplication.

Original:  Conclusion
Revised:  Conclusion

Original: We propose Iron , an efficient cryptographic framework for private Tramsformer inference.
Revised: We propose Iron , an efficient cryptographic framework for private Tramsformer inference.

Original: Specifically, we carefully design a new encoding method for optimizing homomorphic encryption-basedmatrix multiplication.
Revised: Specifically, we carefully design a new encoding method for optimizing homomorphic encryption-basedmatrix multiplication.

Original: Further, we devise several communication-efficient non-linear protocols like
Revised: Further, we devise several communication-efficient non-linear protocols like

Original: Softmax, LayerNorm and GELU by integrating advanced secret-sharing primitives.
Revised: Softmax, LayerNorm and GELU by integrating advanced secret-sharing primitives.

Original: Experimentalresults show that Iron outperforms prior art by up to one order of magnitude in terms of computationand communication overheads.
Revised: Experimentalresults show that Iron outperforms prior art by up to one order of magnitude in terms of computationand communication overheads.

Original: We believe that our novel protocols would help advance the practicalinstantiations of private Transformer inference.
Revised: We believe that our novel protocols would help advance the practicalinstantiations of private Transformer inference.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No]
Revised: [No]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [No] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: [No](b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes](c)

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?

Original: [No]
Revised: [No]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No]
Revised: [No]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: [Table] a) Did you include the full text of instructions given to participants and screenshots, if[Table]
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: ) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] ) Did you include the estimated hourly wage paid to participants and the total amount[Table]
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]