Original: Iron : Private Inference on Transformers
Revised: Iron : Private Inference on Transformers

Original: Abstract
Revised: Abstract

Original: We initiate the study of private inference on Transformer-based models in the client-server setting, where clients have private inputs and servers hold proprietary models.
Revised: We initiate the study of private inference on Transformer-based models in the client-server setting, where clients have private inputs and servers hold proprietary models.

Original: Our main contribution is to provide several new secure protocols for matrix multiplication and complex non-linear functions like Softmax, GELU activations, and LayerNorm, which are critical components of Transformers.
Revised: Our main contribution is to provide several new secure protocols for matrix multiplication and complex non-linear functions like Softmax, GELU activations, and LayerNorm, which are critical components of Transformers.

Original: Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplication that crucially relies on a novel compact packing technique.
Revised: Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplication that crucially relies on a novel compact packing technique.

Original: This design achieves √ m × less communication ( m is the number of rows of the output matrix) over the most efficient work.
Revised: This design achieves √ m × less communication ( m is the number of rows of the output matrix) over the most efficient work.

Original: Second, we design efficient protocols for three non-linear functions via integrating advanced underlying protocols and specialized optimizations.
Revised: Second, we design efficient protocols for three non-linear functions via integrating advanced underlying protocols and specialized optimizations.

Original: Compared to the state-of-the-art protocols, our recipes reduce about half of the communication and computation overhead.
Revised: Compared to the state-of-the-art protocols, our recipes reduce about half of the communication and computation overhead.

Original: Furthermore, all protocols are numerically precise, which preserve the model accuracy of plaintext.
Revised: Furthermore, all protocols are numerically precise, which preserve the model accuracy of plaintext.

Original: These techniques together allow us to implement Iron , an efficient Transformer-based private inference framework.
Revised: These techniques together allow us to implement Iron , an efficient Transformer-based private inference framework.

Original: Experiments conducted on several real-world datasets and models demonstrate that Iron achieves 3 ∼ 14 × less communication and 3 ∼ 11 × less runtime compared to the prior art.
Revised: Experiments conducted on several real-world datasets and models demonstrate that Iron achieves 3 ∼ 14 × less communication and 3 ∼ 11 × less runtime compared to the prior art.

Original:  Introduction
Revised:  Introduction

Original: Transformer-based models [1–5] have realized tremendous success in the fields of natural language processing (NLP) and computer vision (CV) due to their strong representation capabilities.
Revised: Transformer-based models [1–5] have realized tremendous success in the fields of natural language processing (NLP) and computer vision (CV) due to their strong representation capabilities.

Original: As a new neural network architecture, the Transformer [1] mainly utilizes the self-attention mechanism to compute representations without sequence-aligned recurrence or convolution.
Revised: As a new neural network architecture, the Transformer [1] mainly utilizes the self-attention mechanism to compute representations without sequence-aligned recurrence or convolution.

Original: Following this work, a number of Transformer variants, such as BERT [2] and GPT [3] in NLP, ViT [4] and Swin
Revised: Following this work, a number of Transformer variants, such as BERT [2] and GPT [3] in NLP, ViT [4] and Swin

Original: Transformer [5] in CV, have achieved state-of-the-art performance on lots of real-world tasks.
Revised: Transformer [5] in CV, have achieved state-of-the-art performance on lots of real-world tasks.

Original: The success of Transformers and other big models facilitates emerging inference services and applications [6, 7].
Revised: The success of Transformers and other big models facilitates emerging inference services and applications [6, 7].

Original: In particular, a service provider trains a complex model based on the Transformer, and deploys it as a paid inference service, e.g., machine translation and question answering.
Revised: In particular, a service provider trains a complex model based on the Transformer, and deploys it as a paid inference service, e.g., machine translation and question answering.

Original: A client queries this service with his input samples and obtains the desired responses.
Revised: A client queries this service with his input samples and obtains the desired responses.

Original: Unfortunately, current inference systems suffer from serious privacy concerns [8].
Revised: Unfortunately, current inference systems suffer from serious privacy concerns [8].

Original: On the one hand, clients need to send confidential inputs to the service provider, which could compromise the data privacy of these clients if the provider is untrusted.
Revised: On the one hand, clients need to send confidential inputs to the service provider, which could compromise the data privacy of these clients if the provider is untrusted.

Original: On the other hand, it is undesirable for the provider to distribute the proprietary Transformer-based model to clients, since he needs a large amount of data and computation resources to construct the model [9].
Revised: On the other hand, it is undesirable for the provider to distribute the proprietary Transformer-based model to clients, since he needs a large amount of data and computation resources to construct the model [9].

Original: Therefore, there exists a gap between unprecedented performance and privacy constraints, which motivates our study of private Transformer inference.
Revised: Therefore, there exists a gap between unprecedented performance and privacy constraints, which motivates our study of private Transformer inference.

Original: Private inference aims to protect server’s model weights from clients, while guaranteeing that the server learns no information about clients’ private inputs.
Revised: Private inference aims to protect server’s model weights from clients, while guaranteeing that the server learns no information about clients’ private inputs.

Original: Recently, private inference on traditional neural networks (e.g., convolutional neural networks) have been approached by using secure 2-party computation (2PC) techniques [10–14].
Revised: Recently, private inference on traditional neural networks (e.g., convolutional neural networks) have been approached by using secure 2-party computation (2PC) techniques [10–14].

Original: However, due to essentially different structures, private
Revised: However, due to essentially different structures, private

Original: Transformer inference brings several new challenges.
Revised: Transformer inference brings several new challenges.

Original: First, Transformer-based models use lots of high-dimensional matrix multiplications , rather than matrix-vector multiplications widely studied in prior works.
Revised: First, Transformer-based models use lots of high-dimensional matrix multiplications , rather than matrix-vector multiplications widely studied in prior works.

Original: While we can straightforwardly extend prior matrix-vector multiplication protocols to our setting, unfortunately, even the most efficient design [14] incurs heavy communication because of interacting a large amount of ciphertexts.
Revised: While we can straightforwardly extend prior matrix-vector multiplication protocols to our setting, unfortunately, even the most efficient design [14] incurs heavy communication because of interacting a large amount of ciphertexts.

Original: Second, Transformer-based models use complex math functions like Softmax, GELU activations [15], and LayerNorm, in each block , rather than cryptofriendly non-linear functions such as ReLU and Maxpool.
Revised: Second, Transformer-based models use complex math functions like Softmax, GELU activations [15], and LayerNorm, in each block , rather than cryptofriendly non-linear functions such as ReLU and Maxpool.

Original: Existing methods either use precisionimpaired high-order polynomial approximations [16, 11] or only support limited math functions for specific scenarios [17].
Revised: Existing methods either use precisionimpaired high-order polynomial approximations [16, 11] or only support limited math functions for specific scenarios [17].

Original: Even worse, all of these approaches are computationally intensive and often require a large amount of communication (for more related works, please refer to Appendix A.5).
Revised: Even worse, all of these approaches are computationally intensive and often require a large amount of communication (for more related works, please refer to Appendix A.5).

Original: To facilitate the widespread adoption of Transformer-based inference services in privacy-critical scenarios, designing efficient protocols for the above complex operations is of paramount importance.
Revised: To facilitate the widespread adoption of Transformer-based inference services in privacy-critical scenarios, designing efficient protocols for the above complex operations is of paramount importance.

Original: In this paper, we design Iron , an efficient hybrid cryptographic framework for private Transformer inference without revealing any sensitive information about the server’s model weights or clients’ inputs.
Revised: In this paper, we design Iron , an efficient hybrid cryptographic framework for private Transformer inference without revealing any sensitive information about the server’s model weights or clients’ inputs.

Original: Iron contributes several new specialized protocols for the complicated operations in Transformers to alleviate the performance overhead.
Revised: Iron contributes several new specialized protocols for the complicated operations in Transformers to alleviate the performance overhead.

Original: Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplications.
Revised: Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplications.

Original: Our insight is to pack more plaintext inputs into a single ciphertext by devising a compact packing method, while preserving the functionality of matrix multiplication.
Revised: Our insight is to pack more plaintext inputs into a single ciphertext by devising a compact packing method, while preserving the functionality of matrix multiplication.

Original: Compared to the most efficient matrix-vector multiplication solution implemented in Cheetah [14], we can achieve √ m × ( m is the number of rows of the output matrix) improvement in terms of communication overhead, which is about 8 × reduction for various Transformer models.
Revised: Compared to the most efficient matrix-vector multiplication solution implemented in Cheetah [14], we can achieve √ m × ( m is the number of rows of the output matrix) improvement in terms of communication overhead, which is about 8 × reduction for various Transformer models.

Original: Second, we carefully design efficient protocols for Softmax, GELU, and LayerNorm.
Revised: Second, we carefully design efficient protocols for Softmax, GELU, and LayerNorm.

Original: These protocols are built upon SIRNN [17], the state-of-the-art cryptographic framework for private inference on recurrent neural networks, and make several customized optimizations, such as reducing the overhead of exponentiation in Softmax and simplifying GELU and LayerNorm.
Revised: These protocols are built upon SIRNN [17], the state-of-the-art cryptographic framework for private inference on recurrent neural networks, and make several customized optimizations, such as reducing the overhead of exponentiation in Softmax and simplifying GELU and LayerNorm.

Original: These optimizations achieve 1 .
Revised: These optimizations achieve 1 .

Original: 3 ∼ 1 . 8 × less runtime and 1 .
Revised: 3 ∼ 1 . 8 × less runtime and 1 .

Original: 4 ∼ 1 . 8 × less communication on three non-linear functions.
Revised: 4 ∼ 1 . 8 × less communication on three non-linear functions.

Original: Furthermore, these protocols are numerically precise, which preserve the model accuracy of plaintext.
Revised: Furthermore, these protocols are numerically precise, which preserve the model accuracy of plaintext.

Original: We also give a formal security proof for our designed protocols to demonstrate the security guarantee.
Revised: We also give a formal security proof for our designed protocols to demonstrate the security guarantee.

Original: Based on the above efficient components, we implement a private Transformer inference framework,
Revised: Based on the above efficient components, we implement a private Transformer inference framework,

Original: Iron , and conduct end-to-end experiments with various BERT architectures [2] (BERT-Tiny, BERTMedium, BERT-Base, and BERT-Large) on GLUE benchmarks [18].
Revised: Iron , and conduct end-to-end experiments with various BERT architectures [2] (BERT-Tiny, BERTMedium, BERT-Base, and BERT-Large) on GLUE benchmarks [18].

Original: Note that Iron is readily extended to other Transformer-based models (e.g., ViT) since they share very similar architectures and same operations.
Revised: Note that Iron is readily extended to other Transformer-based models (e.g., ViT) since they share very similar architectures and same operations.

Original: Experimental results show that Iron achieves 3 ∼ 14 × less communication and 3 ∼ 11 × less runtime costs over SIRNN on four BERT models.
Revised: Experimental results show that Iron achieves 3 ∼ 14 × less communication and 3 ∼ 11 × less runtime costs over SIRNN on four BERT models.

Original: Moreover, compared with the general-purpose state-of-the-art framework MP-SPDZ [16], Iron has up to two orders of magnitude improvement in terms of both communication and computation efficiency.
Revised: Moreover, compared with the general-purpose state-of-the-art framework MP-SPDZ [16], Iron has up to two orders of magnitude improvement in terms of both communication and computation efficiency.

Original: A concurrent work [19] proposed a privacy-preserving Transformer inference with homomorphic encryption, called THE-X. Below, we illustrate some important differences in terms of protocol design and security.
Revised: A concurrent work [19] proposed a privacy-preserving Transformer inference with homomorphic encryption, called THE-X. Below, we illustrate some important differences in terms of protocol design and security.

Original: (1) Protocol design.
Revised: (1) Protocol design.

Original: Our work aims to design new efficient protocols for the complex operations of Transformer-based models, while orthogonal to ours, THE-X replaces them with crypto-friendly operations.
Revised: Our work aims to design new efficient protocols for the complex operations of Transformer-based models, while orthogonal to ours, THE-X replaces them with crypto-friendly operations.

Original: For example, THE-X replaces GELUs with simpler operations, i.e., ReLUs, and Softmax with the combination of ReLU and polynomials.
Revised: For example, THE-X replaces GELUs with simpler operations, i.e., ReLUs, and Softmax with the combination of ReLU and polynomials.

Original: (2) Security.
Revised: (2) Security.

Original: Our work achieves more rigorous privacy protection than THE-X. Specifically, our work uses homomorphic encryption and secret sharing techniques to hide private information (including intermediate results) of all layers.
Revised: Our work achieves more rigorous privacy protection than THE-X. Specifically, our work uses homomorphic encryption and secret sharing techniques to hide private information (including intermediate results) of all layers.

Original: Such rigorous privacy guarantee is in line with recent state-of-the-art private inference works [14, 17].
Revised: Such rigorous privacy guarantee is in line with recent state-of-the-art private inference works [14, 17].

Original: However, in THE-X, the inputs of each non-linear layer are leaked to the client, which may causes severe privacy leakages in real-world applications [13].
Revised: However, in THE-X, the inputs of each non-linear layer are leaked to the client, which may causes severe privacy leakages in real-world applications [13].

Original: Therefore, our work may be used to enhance the security of THE-X.
Revised: Therefore, our work may be used to enhance the security of THE-X.

Original:  Preliminaries 2.1 Threat Model
Revised:  Preliminaries 2.1 Threat Model

Original: As shown in the left part of Figure 1, Iron works in a general private inference scenario, where the server P 0 holds a Transformer-based model M with private weights w , while the client P 1 holds a private input x .
Revised: As shown in the left part of Figure 1, Iron works in a general private inference scenario, where the server P 0 holds a Transformer-based model M with private weights w , while the client P 1 holds a private input x .

Original: Our framework enables the client to query the server’s inference service and learn the output of the model on its input, i.e., M ( w, x ) .
Revised: Our framework enables the client to query the server’s inference service and learn the output of the model on its input, i.e., M ( w, x ) .

Original: Same as prior works [14, 17], we consider an honest-but-curious adversary that passively corrupts either the server or the client, but not both.
Revised: Same as prior works [14, 17], we consider an honest-but-curious adversary that passively corrupts either the server or the client, but not both.

Original: Such an adversary follows the protocol specification exactly, but may try to learn more information 3 than allowed (e.g., the model’s weights or inference inputs) via analyzing the data it receives.
Revised: Such an adversary follows the protocol specification exactly, but may try to learn more information 3 than allowed (e.g., the model’s weights or inference inputs) via analyzing the data it receives.

Original: In Appendix
Revised: In Appendix

Original: A.1.2, we give a more formal description of the threat model for security analysis.
Revised: A.1.2, we give a more formal description of the threat model for security analysis.

Original: 2.2 Cryptographic Primitives
Revised: 2.2 Cryptographic Primitives

Original: All our protocols are built on the 2-out-of-2 additive secret sharing (ASS) technique [21, 22] over the ring Z 2 ℓ , in which an ℓ -bit input x is split into two random shares ⟨ x ⟩ 0 , ⟨ x ⟩ 1 , held by P 0 and P 1 , respectively, such that x = ⟨ x ⟩ 0 + ⟨ x ⟩ 1 mod Z 2 ℓ .
Revised: All our protocols are built on the 2-out-of-2 additive secret sharing (ASS) technique [21, 22] over the ring Z 2 ℓ , in which an ℓ -bit input x is split into two random shares ⟨ x ⟩ 0 , ⟨ x ⟩ 1 , held by P 0 and P 1 , respectively, such that x = ⟨ x ⟩ 0 + ⟨ x ⟩ 1 mod Z 2 ℓ .

Original: When ℓ = 1 , i.e., over Z 2 , we use ⟨ x ⟩ B to denote boolean shares.
Revised: When ℓ = 1 , i.e., over Z 2 , we use ⟨ x ⟩ B to denote boolean shares.

Original: In our protocols, we use ⟨ x ⟩ to denote that P b holds ⟨ x ⟩ b for b ∈ { 0 , 1 } .
Revised: In our protocols, we use ⟨ x ⟩ to denote that P b holds ⟨ x ⟩ b for b ∈ { 0 , 1 } .

Original: The security [21] guarantees that given a share ⟨ x ⟩ 0 or ⟨ x ⟩ 1 , the value of x is perfectly hidden.
Revised: The security [21] guarantees that given a share ⟨ x ⟩ 0 or ⟨ x ⟩ 1 , the value of x is perfectly hidden.

Original: This secret-sharing property is maintained throughout our private inference scheme.
Revised: This secret-sharing property is maintained throughout our private inference scheme.

Original: ASS naturally supports linear operations without communication.
Revised: ASS naturally supports linear operations without communication.

Original: For instance, to compute cx + y with the constant c and secret shares ⟨ x ⟩ and ⟨ y ⟩ , P b can locally compute ⟨ z ⟩ b = c ⟨ x ⟩ b + ⟨ y ⟩ b , where ⟨ z ⟩ 0 + ⟨ z ⟩ 1 = cx
Revised: For instance, to compute cx + y with the constant c and secret shares ⟨ x ⟩ and ⟨ y ⟩ , P b can locally compute ⟨ z ⟩ b = c ⟨ x ⟩ b + ⟨ y ⟩ b , where ⟨ z ⟩ 0 + ⟨ z ⟩ 1 = cx

Original: +
Revised: +

Original: y mod 2 ℓ .
Revised: y mod 2 ℓ .

Original: To achieve more functionalities under shared inputs, we require to invoke advanced homomorphic encryption or oblivious transfer techniques [22].
Revised: To achieve more functionalities under shared inputs, we require to invoke advanced homomorphic encryption or oblivious transfer techniques [22].

Original: Notations.
Revised: Notations.

Original: Let x ∤ y means x is not a divisor of y .
Revised: Let x ∤ y means x is not a divisor of y .

Original: We use bold lower-case letters (e.g., x ) to represent vectors, and bold upper-case letters (e.g., X ) to denote matrices.
Revised: We use bold lower-case letters (e.g., x ) to represent vectors, and bold upper-case letters (e.g., X ) to denote matrices.

Original: Like prior works [17, 14], we encode inputs with the fixed-point representation denoted by Fix (refer to Appendix A.1.1 for details).
Revised: Like prior works [17, 14], we encode inputs with the fixed-point representation denoted by Fix (refer to Appendix A.1.1 for details).

Original: Let A
Revised: Let A

Original: N,p denote the set of integer polynomials A N,p =
Revised: N,p denote the set of integer polynomials A N,p =

Original: Z p [ x ] / (cid:0) x N + 1 (cid:1) .
Revised: Z p [ x ] / (cid:0) x N + 1 (cid:1) .

Original: We use the circumflex of lower-case letters (e.g., ˆ a ) to represent a polynomial, and ˆ a [ i ] to denote the i -th coefficient of ˆ a .
Revised: We use the circumflex of lower-case letters (e.g., ˆ a ) to represent a polynomial, and ˆ a [ i ] to denote the i -th coefficient of ˆ a .

Original: Given polynomials ˆ x, ˆ y ∈
Revised: Given polynomials ˆ x, ˆ y ∈

Original: A N,p , the product ˆ z
Revised: A N,p , the product ˆ z

Original: = ˆ x · ˆ y over A N,p is defined as
Revised: = ˆ x · ˆ y over A N,p is defined as

Original: [Equation]
Revised: [Equation]

Original: Additively Homomorphic Encryption (AHE) [23, 24].
Revised: Additively Homomorphic Encryption (AHE) [23, 24].

Original: This encryption scheme additionally enables linearly homomorphic operations on ciphertexts.
Revised: This encryption scheme additionally enables linearly homomorphic operations on ciphertexts.

Original: Specifically, an AHE scheme is a tuple of algorithms AHE =
Revised: Specifically, an AHE scheme is a tuple of algorithms AHE =

Original: ( KeyGen ; Enc ; Dec ; Eval ) with the parameter { N, q, p } and the following syntax: 1) KeyGen (cid:0) 1 k (cid:1) → ( pk, sk ) : on input a security parameter κ , KeyGen is a randomized algorithm that outputs a public key
Revised: ( KeyGen ; Enc ; Dec ; Eval ) with the parameter { N, q, p } and the following syntax: 1) KeyGen (cid:0) 1 k (cid:1) → ( pk, sk ) : on input a security parameter κ , KeyGen is a randomized algorithm that outputs a public key

Original: pk ∈
Revised: pk ∈

Original: A N,q and a secret key sk ∈ A N,q . 2)
Revised: A N,q and a secret key sk ∈ A N,q . 2)

Original: Enc ( pk, ˆ m )
Revised: Enc ( pk, ˆ m )

Original: → ˆ c : the encryption algorithm Enc takes a plaintext polynomial ˆ m ∈
Revised: → ˆ c : the encryption algorithm Enc takes a plaintext polynomial ˆ m ∈

Original: A N,p and encrypts it using pk into a ciphertext polynomial ˆ c ∈
Revised: A N,p and encrypts it using pk into a ciphertext polynomial ˆ c ∈

Original: A N,q . 3)
Revised: A N,q . 3)

Original: Dec ( sk, ˆ c )
Revised: Dec ( sk, ˆ c )

Original: → ˆ m : on input sk and a ciphertext ˆ c , the (deterministic) decryption algorithm
Revised: → ˆ m : on input sk and a ciphertext ˆ c , the (deterministic) decryption algorithm

Original: Dec recovers the plaintext message ˆ m . 4) Eval ( pk, ˆ c 1 , ˆ c 2 , func )
Revised: Dec recovers the plaintext message ˆ m . 4) Eval ( pk, ˆ c 1 , ˆ c 2 , func )

Original: → ˆ c : on input pk , two ciphertexts ˆ c 1 , ˆ c 2 containing ˆ m 1 , ˆ m 2 , and a linear function func , Eval outputs a new ciphertext ˆ c encrypting func ( ˆ m 1 , ˆ m 2 ) .
Revised: → ˆ c : on input pk , two ciphertexts ˆ c 1 , ˆ c 2 containing ˆ m 1 , ˆ m 2 , and a linear function func , Eval outputs a new ciphertext ˆ c encrypting func ( ˆ m 1 , ˆ m 2 ) .

Original: Let ⊞ , ⊟ , ⊠ denote homomorphic addition, homomorphic subtraction and homomorphic multiplication with a plaintext, respectively.
Revised: Let ⊞ , ⊟ , ⊠ denote homomorphic addition, homomorphic subtraction and homomorphic multiplication with a plaintext, respectively.

Original: Iron builds the matrix multiplication protocol on the Brakerski-Fan-Vercauteren (BFV) scheme [25, 26], which is one of the state-of-the-art lattice-based homomorphic encryption solutions.
Revised: Iron builds the matrix multiplication protocol on the Brakerski-Fan-Vercauteren (BFV) scheme [25, 26], which is one of the state-of-the-art lattice-based homomorphic encryption solutions.

Original: Oblivious Transfer.
Revised: Oblivious Transfer.

Original: The 1-out-of- k oblivious transfer (OT) [27] is denoted by k - OT ℓ , where one party is the sender with k messages x 0 , . . .
Revised: The 1-out-of- k oblivious transfer (OT) [27] is denoted by k - OT ℓ , where one party is the sender with k messages x 0 , . . .

Original: , x k − 1 ∈ { 0 , 1 } ℓ and the other party is the receiver with an index i ∈ [ k ] .
Revised: , x k − 1 ∈ { 0 , 1 } ℓ and the other party is the receiver with an index i ∈ [ k ] .

Original: The receiver learns x i as the output, and the sender learns nothing.
Revised: The receiver learns x i as the output, and the sender learns nothing.

Original: Additionally, we also use the 1-out-of-2 correlated OT, denoted by 2 - COT ℓ [28], which is defined as follows: the sender inputs a correlation x ∈ Z 2 ℓ , the receiver inputs a choice bit i ∈ { 0 , 1 } , and the protocol outputs a random element r ∈
Revised: Additionally, we also use the 1-out-of-2 correlated OT, denoted by 2 - COT ℓ [28], which is defined as follows: the sender inputs a correlation x ∈ Z 2 ℓ , the receiver inputs a choice bit i ∈ { 0 , 1 } , and the protocol outputs a random element r ∈

Original: Z 2 ℓ to the sender and r + i · x to the receiver.
Revised: Z 2 ℓ to the sender and r + i · x to the receiver.

Original: k - OT ℓ and 2 - COT ℓ require 2 λ + kℓ and λ + ℓ bits of communication, respectively, and are executed in 2 rounds.
Revised: k - OT ℓ and 2 - COT ℓ require 2 λ + kℓ and λ + ℓ bits of communication, respectively, and are executed in 2 rounds.

Original: The OT protocols are widely used to build the underlying protocols [13, 17] in Figure 3 we rely on.
Revised: The OT protocols are widely used to build the underlying protocols [13, 17] in Figure 3 we rely on.

Original: Server
Revised: Server

Original: Proprietary model
Revised: Proprietary model

Original: Client
Revised: Client

Original: Transformer
Revised: Transformer

Original: [Figure]
Revised: [Figure]

Original: Inputs
Revised: Inputs

Original: [Figure]
Revised: [Figure]

Original: Encoding
Revised: Encoding

Original: [Figure]
Revised: [Figure]

Original: LayerNorm LayerNorm
Revised: LayerNorm LayerNorm

Original: [Figure]
Revised: [Figure]

Original:  Overview 3.1 Transformer Architecture
Revised:  Overview 3.1 Transformer Architecture

Original: The Transformer is an encoder-decoder architecture, where both parts have a similar structure.
Revised: The Transformer is an encoder-decoder architecture, where both parts have a similar structure.

Original: Hence, we mainly focus on the encoder below.
Revised: Hence, we mainly focus on the encoder below.

Original: The encoder is composed of a stack of identical blocks, each with two sub-layers, i.e., a multi-head self-attention mechanism and a feed-forward network, as shown in Figure 1.
Revised: The encoder is composed of a stack of identical blocks, each with two sub-layers, i.e., a multi-head self-attention mechanism and a feed-forward network, as shown in Figure 1.

Original: Besides, residual connection and layer normalization (LayerNorm) are employed around each of the two sub-layers.
Revised: Besides, residual connection and layer normalization (LayerNorm) are employed around each of the two sub-layers.

Original: We describe a block of the encoder as follows.
Revised: We describe a block of the encoder as follows.

Original: Attention layer.
Revised: Attention layer.

Original: An attention function can be described as mapping a query X Q and a set of keyvalue pairs ( X K , X V ) to a weighted sum of the values, where the weight is computed by a metric of the query with the corresponding key [1].
Revised: An attention function can be described as mapping a query X Q and a set of keyvalue pairs ( X K , X V ) to a weighted sum of the values, where the weight is computed by a metric of the query with the corresponding key [1].

Original: This function can be formalized as below:
Revised: This function can be formalized as below:

Original: [Equation]
Revised: [Equation]

Original: where X Q , X K , X V are different linear projections of the input X , i.e., X Q = XW Q , X K =
Revised: where X Q , X K , X V are different linear projections of the input X , i.e., X Q = XW Q , X K =

Original: XW K , X V = XW V , and d is the dimension of representations.
Revised: XW K , X V = XW V , and d is the dimension of representations.

Original: Multi-head attention extends the above mechanism to H parallel attention layers and is illustrated in Appendix A.1.3.
Revised: Multi-head attention extends the above mechanism to H parallel attention layers and is illustrated in Appendix A.1.3.

Original: Feed-forward layer.
Revised: Feed-forward layer.

Original: A fully connected feed-forward layer consists of two linear transformations with a GELU activation in between, where GELU is the Gaussian Error Linear Unit function [15].
Revised: A fully connected feed-forward layer consists of two linear transformations with a GELU activation in between, where GELU is the Gaussian Error Linear Unit function [15].

Original: This layer can be represented as follows:
Revised: This layer can be represented as follows:

Original: [Equation]
Revised: [Equation]

Original: In addition to the above encoder-decoder blocks, an embedding layer is employed at the beginning of the model to convert input tokens X input to continuous feature vector representations.
Revised: In addition to the above encoder-decoder blocks, an embedding layer is employed at the beginning of the model to convert input tokens X input to continuous feature vector representations.

Original: This is formulated as X = X input W E , where W E is the embedding lookup table.
Revised: This is formulated as X = X input W E , where W E is the embedding lookup table.

Original: 3.2 Private Transformer Inference
Revised: 3.2 Private Transformer Inference

Original: According to the required crypographic operations, the layers of Transformers can be broken into two categories - linear and non-linear .
Revised: According to the required crypographic operations, the layers of Transformers can be broken into two categories - linear and non-linear .

Original: 1) Linear layers : these include embedding, matrix multiplication in Attention, and fully-connected layer.
Revised: 1) Linear layers : these include embedding, matrix multiplication in Attention, and fully-connected layer.

Original: All protocols of these operations rely crucially on the matrix multiplication protocol Π MatMul in Section 4.1.
Revised: All protocols of these operations rely crucially on the matrix multiplication protocol Π MatMul in Section 4.1.

Original: It lies in the setting where P 0 and P 1 take as input the matrices X and Y , and learn the sharings ⟨ Z ⟩ 0 and ⟨ Z ⟩ 1 , respectively, such that Z = XY .
Revised: It lies in the setting where P 0 and P 1 take as input the matrices X and Y , and learn the sharings ⟨ Z ⟩ 0 and ⟨ Z ⟩ 1 , respectively, such that Z = XY .

Original: 2) Non-linear layers : these consist of Softmax, GELU and LayerNorm.
Revised: 2) Non-linear layers : these consist of Softmax, GELU and LayerNorm.

Original: The non-linear operations are directly evaluated by exploiting our proposed protocols in Section 4.2,
Revised: The non-linear operations are directly evaluated by exploiting our proposed protocols in Section 4.2,

Original: i
Revised: i

Original: e., Π GELU , Π Softmax , Π LayerNorm .
Revised: e., Π GELU , Π Softmax , Π LayerNorm .

Original: All of these take the additive shares ⟨ X ⟩ as input
Revised: All of these take the additive shares ⟨ X ⟩ as input

Original: and output the additive shares ⟨ Y ⟩ = Π Func ( ⟨ X ⟩ ) , where Func ∈ { GELU , Softmax , LayerNorm } .
Revised: and output the additive shares ⟨ Y ⟩ = Π Func ( ⟨ X ⟩ ) , where Func ∈ { GELU , Softmax , LayerNorm } .

Original: Overview of private inference.
Revised: Overview of private inference.

Original: In the entire private inference process, we maintain the following invariant: P 0 and P 1 begin with additive shares of the input to the layer, and end with additive shares (over the same ring Z 2 ℓ ) of the output of the layer after the protocol.
Revised: In the entire private inference process, we maintain the following invariant: P 0 and P 1 begin with additive shares of the input to the layer, and end with additive shares (over the same ring Z 2 ℓ ) of the output of the layer after the protocol.

Original: This allows us to stitch protocols for arbitrary layers sequentially to obtain a secure computation scheme for any Transformer-based models.
Revised: This allows us to stitch protocols for arbitrary layers sequentially to obtain a secure computation scheme for any Transformer-based models.

Original: To clearly understand our scheme, we take an example of private inference between Pand P 1 on the embedding layer and the first encoder block, as shown in the right part of Figure 1.
Revised: To clearly understand our scheme, we take an example of private inference between Pand P 1 on the embedding layer and the first encoder block, as shown in the right part of Figure 1.

Original: Here, our example uses the designed protocols as a black box, i.e., Π MatMul , Π GELU , Π Softmax , and Π LayerNorm , and we discuss how to efficiently implement these protocols in Section 4.[Figure] [Figure] [Figure]
Revised: Here, our example uses the designed protocols as a black box, i.e., Π MatMul , Π GELU , Π Softmax , and Π LayerNorm , and we discuss how to efficiently implement these protocols in Section 4.

Original: mod
Revised: 𝐗𝐗 =

Original: [Figure]
Revised: [Figure]

Original: �𝑦𝑦 = 5𝑥𝑥 

Revised: 𝐘𝐘 = 5 7 9 6 8𝒁𝒁 = 𝐗𝐗 � 𝐘𝐘 ≡ 17 23 29 7 21 3 mod 2�𝑦𝑦 = 5𝑥𝑥

Original: + 6𝑥𝑥 1 + 7𝑥𝑥 2 + 8𝑥𝑥 3 + 9𝑥𝑥 4 + 10𝑥𝑥 5 �𝑥𝑥 = 2𝑥𝑥 0 + 1𝑥𝑥 1 + 4𝑥𝑥 6 + 3𝑥𝑥̂𝑧𝑧 ≡ 10𝑥𝑥 0 +17 𝑥𝑥 1 + 20𝑥𝑥 2 + 23 𝑥𝑥 3 + 26𝑥𝑥 4 + 29 𝑥𝑥 5 + 30𝑥𝑥+ 7 𝑥𝑥 

Revised: + 6𝑥𝑥 1 + 7𝑥𝑥 2 + 8𝑥𝑥 3 + 9𝑥𝑥 4 + 10𝑥𝑥 5 �𝑥𝑥 = 2𝑥𝑥 0 + 1𝑥𝑥 1 + 4𝑥𝑥 6 + 3𝑥𝑥̂𝑧𝑧 ≡ 10𝑥𝑥 0 +17 𝑥𝑥 1 + 20𝑥𝑥 2 + 23 𝑥𝑥 3 + 26𝑥𝑥 4 + 29 𝑥𝑥 5 + 30𝑥𝑥+ 7 𝑥𝑥

Original: + 14𝑥𝑥 8 + 21 𝑥𝑥 9 + 28 𝑥𝑥 

Revised: + 14𝑥𝑥 8 + 21 𝑥𝑥 9 + 28 𝑥𝑥

Original: + 3 𝑥𝑥 11 + 30𝑥𝑥 12 mod ( 𝑥𝑥 16 + 1,2 5 )
Revised: + 3 𝑥𝑥 11 + 30𝑥𝑥 12 mod ( 𝑥𝑥 16 + 1,2 5 )

Original: An example over ℤ [Figure] �𝑥𝑥 � �𝑦𝑦 mod( 𝑥𝑥 16 +1,2 5 )
Revised: An example over ℤ 2�𝑥𝑥 � �𝑦𝑦 mod( 𝑥𝑥 16 +1,2 5 )

Original: Evaluation in Π MatMul
Revised: Evaluation in Π MatMul

Original: In the embedding layer, P 0 and P 1 take as inputs W E and X input , and invoke the matrix multiplication protocol Π MatMul to compute the input embedding X .
Revised: In the embedding layer, P 0 and P 1 take as inputs W E and X input , and invoke the matrix multiplication protocol Π MatMul to compute the input embedding X .

Original: This is represented as ⟨ X
Revised: This is represented as ⟨ X

Original: ⟩ = Π
Revised: ⟩ = Π

Original: MatMul ( X input , W E ) .
Revised: MatMul ( X input , W E ) .

Original: In the attention layer, Π MatMul is also invoked to generate the query, key and value matrices.
Revised: In the attention layer, Π MatMul is also invoked to generate the query, key and value matrices.

Original: For instance, ⟨ X Q ⟩ = Π MatMul ( ⟨ X ⟩ 1 , W Q ) + ⟨ X ⟩ 0 W Q , where ⟨ X ⟩ 0 W Q can be computed locally by P 0 .
Revised: For instance, ⟨ X Q ⟩ = Π MatMul ( ⟨ X ⟩ 1 , W Q ) + ⟨ X ⟩ 0 W Q , where ⟨ X ⟩ 0 W Q can be computed locally by P 0 .

Original: The same idea is used in the generation of ⟨ X K ⟩ and ⟨ X V ⟩ .
Revised: The same idea is used in the generation of ⟨ X K ⟩ and ⟨ X V ⟩ .

Original: Then, we compute X QK =
Revised: Then, we compute X QK =

Original: X Q X TK that requires two invocations of Π MatMul .
Revised: X Q X TK that requires two invocations of Π MatMul .

Original: The formulation is ⟨ X QK ⟩ = Π MatMul ( ⟨ X Q ⟩ 0 , ⟨ X K ⟩ 1 ) + Π MatMul ( ⟨ X Q ⟩ 1 , ⟨ X K ⟩ 0 ) + ⟨ X Q ⟩ 0 ⟨ X K ⟩ T 0 + ⟨ X Q ⟩ 1 ⟨ X K ⟩ T 1 , where the last two terms can be evaluated by P 0 and P 1 locally.
Revised: The formulation is ⟨ X QK ⟩ = Π MatMul ( ⟨ X Q ⟩ 0 , ⟨ X K ⟩ 1 ) + Π MatMul ( ⟨ X Q ⟩ 1 , ⟨ X K ⟩ 0 ) + ⟨ X Q ⟩ 0 ⟨ X K ⟩ T 0 + ⟨ X Q ⟩ 1 ⟨ X K ⟩ T 1 , where the last two terms can be evaluated by P 0 and P 1 locally.

Original: After that, we evaluate ⟨ (cid:101)
Revised: After that, we evaluate ⟨ (cid:101)

Original: X QK ⟩ = Π Softmax ( ⟨ X QK ⟩ / √ d ) , followed by computing ⟨ (cid:101) Z ⟩ = ⟨ (cid:101) X
Revised: X QK ⟩ = Π Softmax ( ⟨ X QK ⟩ / √ d ) , followed by computing ⟨ (cid:101) Z ⟩ = ⟨ (cid:101) X

Original: QK X V ⟩ by two invocations of Π MatMul .
Revised: QK X V ⟩ by two invocations of Π MatMul .

Original: Then, the two parties invoke the LayerNorm protocol to evaluate ⟨ Z ⟩ = Π LayerNorm ( ⟨ (cid:101) Z ⟩ ) .
Revised: Then, the two parties invoke the LayerNorm protocol to evaluate ⟨ Z ⟩ = Π LayerNorm ( ⟨ (cid:101) Z ⟩ ) .

Original: In the feed-forward layer, they compute ⟨ (cid:101) Y 1 ⟩ = Π MatMul ( ⟨ Z ⟩ 1 , W 1 ) + ⟨ Z ⟩ 0 W 1 , followed by ⟨ (cid:101)
Revised: In the feed-forward layer, they compute ⟨ (cid:101) Y 1 ⟩ = Π MatMul ( ⟨ Z ⟩ 1 , W 1 ) + ⟨ Z ⟩ 0 W 1 , followed by ⟨ (cid:101)

Original: Y ⟩ = Π GELU ( ⟨ (cid:101)
Revised: Y ⟩ = Π GELU ( ⟨ (cid:101)

Original: Y 1 ⟩ ) .
Revised: Y 1 ⟩ ) .

Original: After that, they compute ⟨ (cid:101) Y 2 ⟩ = Π MatMul ( ⟨ (cid:101)
Revised: After that, they compute ⟨ (cid:101) Y 2 ⟩ = Π MatMul ( ⟨ (cid:101)

Original: Y ⟩ 1 , W 1 )
Revised: Y ⟩ 1 , W 1 )

Original: + ⟨ (cid:101)
Revised: + ⟨ (cid:101)

Original: Y ⟩ 0 W 1 , where ⟨ (cid:101)
Revised: Y ⟩ 0 W 1 , where ⟨ (cid:101)

Original: Y ⟩ 0 W 1 can be computed locally by P 0 .
Revised: Y ⟩ 0 W 1 can be computed locally by P 0 .

Original: Finally, the evaluation of the encoder completes after computing ⟨ Y ⟩ = Π LayerNorm ( ⟨ (cid:101) Y 2 ⟩ ) .
Revised: Finally, the evaluation of the encoder completes after computing ⟨ Y ⟩ = Π LayerNorm ( ⟨ (cid:101) Y 2 ⟩ ) .

Original:  Supporting Protocols 4.1 Protocol for Matrix Multiplication
Revised:  Supporting Protocols 4.1 Protocol for Matrix Multiplication

Original: As shown in Section 1, naively extending well-studied matrix-vector multiplication protocols [29, 13, 14] to our matrix multiplication setting results in a significant communication overhead, mainly due to frequently interacting ciphertexts.
Revised: As shown in Section 1, naively extending well-studied matrix-vector multiplication protocols [29, 13, 14] to our matrix multiplication setting results in a significant communication overhead, mainly due to frequently interacting ciphertexts.

Original: In this work, we build a specialized matrix multiplication protocol on top of the most efficient protocol, Cheetah [14].
Revised: In this work, we build a specialized matrix multiplication protocol on top of the most efficient protocol, Cheetah [14].

Original: Recall that the plaintext of an AHE scheme is a polynomial, which can pack a large number of inputs to amortize the overhead [29, 13].
Revised: Recall that the plaintext of an AHE scheme is a polynomial, which can pack a large number of inputs to amortize the overhead [29, 13].

Original: The key contribution of our proposed protocol is a more compact input packing approach.
Revised: The key contribution of our proposed protocol is a more compact input packing approach.

Original: Our starting point is that polynomial multiplication implies vector inner product, if we arrange the coefficients properly [14].
Revised: Our starting point is that polynomial multiplication implies vector inner product, if we arrange the coefficients properly [14].

Original: As shown in Equation 1, when multiplying two polynomials of degree- ( N - 1 ), the ( N - 1 )-th coefficient of the resulting polynomial is the inner product of the two coefficient vectors in opposite orders.
Revised: As shown in Equation 1, when multiplying two polynomials of degree-( N - 1 ), the ( N - 1 )-th coefficient of the resulting polynomial is the inner product of the two coefficient vectors in opposite orders.

Original: By using an appropriate arrangement of coefficients, this idea can be extended toour matrix multiplications, since they consist of a set of inner products.
Revised: By using an appropriate arrangement of coefficients, this idea can be extended to matrix multiplications, since they consist of a set of inner products.

Original: To this end, we give the definitions of two input packing functions π L : Z m × n 2 ℓ → A N, 2 ℓ and π R : Z n × k 2 ℓ → A N, 2 ℓ as follows:
Revised: We give the definitions of two input packing functions π L : Z m × n 2 ℓ → A N, 2 ℓ and π R : Z n × k 2 ℓ → A N, 2 ℓ as follows:

Original: [Equation]
Revised: [Equation]

Original: · k + ( n − 1) − j ] = 
Revised: − 1) − j ] = X [ i, j ] ,

Original: X
Revised: for

Original: [ i, j ] , for i ∈ [ m ] , j ∈ [ n ] ˆ y =
Revised: i ∈ [ m ] , j ∈ [ n ] ˆ y =

Original: π R ( Y ) , s.t. ,
Revised: π R ( Y ) , s.t. ,

Original: ˆ y [ j · n + i ]
Revised: ˆ y [ j · n + i ]

Original: =
Revised: =

Original: Y [ i, j ] , for i ∈ [ n ] , j ∈ [ k ] where all other coefficients of ˆ x and ˆ y are set to 0.
Revised: Y [ i, j ] , for i ∈ [ n ] , j ∈ [ k ] where all other coefficients of ˆ x and ˆ y are set to 0.

Original: Multiplication of polynomials ˆ z
Revised: Multiplication of polynomials ˆ z

Original: = ˆ x · ˆ y directly gives the result of matrix multiplication Z = XY mod 2 ℓ in some of ˆ z ’s coefficients.
Revised: = ˆ x · ˆ y directly gives the result of matrix multiplication Z = XY mod 2 ℓ in some of ˆ z ’s coefficients.

Original: We formalize this process as below, and give a toy example in Figure 2.
Revised: We formalize this process as below, and give a toy example in Figure 2.

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Assuming mnk ≤ N and given two polynomials ˆ x = π L ( X ) and
Revised: Assuming mnk ≤ N and given two polynomials ˆ x = π L ( X ) and

Original: ˆ y = π R ( Y ) , the matrix multiplication Z = XY mod 2 ℓ can be evaluated via the product ˆ z
Revised: ˆ y = π R ( Y ) , the matrix multiplication Z = XY mod 2 ℓ can be evaluated via the product ˆ z

Original: = ˆ x · ˆ y , where Z [ i, j ] is computed in ˆ z
Revised: = ˆ x · ˆ y , where Z [ i, j ] is computed in ˆ z

Original: [ i · n · k + ( j + 1) · n − 1] for i ∈ [ m ] and j ∈ [ k ] .
Revised: [ i · n · k + ( j + 1) · n − 1] for i ∈ [ m ] and j ∈ [ k ] .

Original: We defer the proof to Appendix A.2.1.
Revised: We defer the proof to Appendix A.2.1.

Original: When mnk > N , we first partition the matrices X , Y into sub-matrices of m w × n w and n w × k w elements, respectively, such that m w n w k w ≤ N .
Revised: When mnk > N , we first partition the matrices X , Y into sub-matrices of m w × n w and n w × k w elements, respectively, such that m w n w k w ≤ N .

Original: Zero-padding is required when m w ∤ m , n w ∤ n or k w ∤ k .
Revised: Zero-padding is required when m w ∤ m , n w ∤ n or k w ∤ k .

Original: The protocol is shown in Algorithm 1.
Revised: The protocol is shown in Algorithm 1.

Original: Complexity and security analysis .
Revised: Complexity and security analysis .

Original: For complexity, totally, two parties interact kk w ( mm w + nn w ) ciphertexts, and operate with O ( mnk/N ) homomorphic additions and multiplications.
Revised: For complexity, totally, two parties interact kk w ( mm w + nn w ) ciphertexts, and operate with O ( mnk/N ) homomorphic additions and multiplications.

Original: We formalize the selection of parameters m w , n w , k w as an optimization problem, to minimize the communication cost.
Revised: We formalize the selection of parameters m w , n w , k w as an optimization problem, to minimize the communication cost.

Original: Through our analysis in Appendix A.2.2, in general scenarios, our method theoretically achieves √ m communication improvement over Cheetah [14].
Revised: Through our analysis in Appendix A.2.2, in general scenarios, our method theoretically achieves √ m communication improvement over Cheetah [14].

Original: Notably, when mnk ≤ N , we reduce the communcation cost by a factor of m , since Cheetah encodes each row of the matrix into a ciphertext.
Revised: Notably, when mnk ≤ N , we reduce the communcation cost by a factor of m , since Cheetah encodes each row of the matrix into a ciphertext.

Original: Besides, the security proof is shown in Appendix A.2.3.
Revised: Besides, the security proof is shown in Appendix A.2.3.

Original: Algorithm Input: P 0 holds X ∈ Z m × n 2 ℓ , and P 1 holds Y ∈ Z n × k 2 ℓ . Output: P 0 and P 1 get ⟨ Z ⟩ 0 , ⟨ Z ⟩ 1 ∈Z m × k
Revised: Algorithm random samples plaintext polynomials m γ ∈

Original: , respectively, where Z = XY . 1: P 0 , P 1 compute the partition window size 0 < m w ≤ m, 0 < n w ≤ n and 0 < k w ≤ k such that m w n w k w ≤ N , and set n ′ = ⌈ n/n w ⌉ , m ′ = ⌈ m/m w ⌉ , and k′ =
Revised: [ k

Original: ⌈ k/k w ⌉ . 2: P 1 partitions the matrix Y into block matrices Y β,γ ∈ Z n w × k w 2 ℓ for β ∈ [ n ′ ] and γ ∈ [ k ′ ] . 3: P 1 encodes the matrices to polynomials ˆ y β,γ = π R ( Y β,γ ) for β ∈ [ n ′ ] and γ ∈ [ k ′ ] .
Revised: P decodes these polynomials a random mask R m according to Theorem 4.1 4 .

Original: Then Psends to P 0 the ciphertexts { CT β,γ= Enc (ˆ y β,γ ) } 4: P0 partitions the matrix X into block matrices X α,β ∈ Z m w × n w 2 ℓ .
Revised: receiving the ciphertexts { CT β,γ } from P

Original: P 0 samples R ∈

Original: Z m × k 2 ℓ uniformly at random and partitions the matrix R into R α,γ ∈ Z m w × k w 2 ℓ . 5: P 0 encodes the two block matrices to polynomials: 1) ˆ x α,β = π L ( X α,β ) for α ∈ [ m ′ ] and β ∈ [ n ′ ] ,and 2) for α ∈ [ m ′ ] and γ ∈ [ k ′ ] , encoding R α,γ into ˆ r α,γ according to Theorem 4.1. 6: On receiving the ciphertexts { CT β,γ } from P1 , P 0 operates CT ′ α,γ = ⊞ β ∈ [ m ′ ]
Revised: , P operates CT ′ α,γ = ⊞ β ∈ [ n ′ ]

Original: (ˆ x α,β ⊠ CT β,γ ) ⊟
Revised: (ˆ x α,β ⊠ CT β,γ ) ⊟

Original: ˆ r α,γ for α ∈ [ m ′ ] and γ ∈ [ k ′ ] .
Revised: ˆ r α,γ for α ∈ [ m ′ ] and γ ∈ [ k ′ ] .

Original: Then P 0 sends to P 1 the ciphertexts (cid:8) CT ′ α,γ (cid:9) .
Revised: Then P 0 sends to P 1 the ciphertexts (cid:8) CT ′ α,γ (cid:9) .

Original: 7: P 0 outputs R mod 2 ℓ as the share ⟨ Z ⟩ 0 .
Revised: 7: P 0 outputs R mod 2 ℓ as the share ⟨ Z ⟩ 0 .

Original: On receiving the ciphertexts (cid:8) CT ′ α,γ (cid:9) from P 0 , P 1 computes ⟨ ˆ z α,γ ⟩ 1 =
Revised: On receiving the ciphertexts (cid:8) CT ′ α,γ (cid:9) from P 0 , P 1 computes ⟨ ˆ z α,γ ⟩ 1 =

Original: Dec (CT ′ α,γ ) that are decoded to ⟨ Z ⟩ 1 using the method in Theorem 4.1.
Revised: Dec (CT ′ α,γ ) that are decoded to ⟨ Z ⟩ 1 using the method in Theorem 4.1.

Original: 4.2 Protocols for Non-linear Functions
Revised: 4.2 Protocols for Non-linear Functions

Original: Our non-linear protocols rely on several underlying protocols from the state-of-the-art works [13, 17].
Revised: Our non-linear protocols rely on several underlying protocols from the state-of-the-art works [13, 17].

Original: In Figure 3, we enumerate the inputs and outputs of these protocols and then use them as a black box (See Appendix A.3.2 for details).
Revised: In Figure 3, we enumerate the inputs and outputs of these protocols and then use them as a black box (See Appendix A.3.2 for details).

Original: On this basis, we provide efficient protocols for Softmax, GELU, and LayerNorm with specific optimizations.
Revised: On this basis, we provide efficient protocols for Softmax, GELU, and LayerNorm with specific optimizations.

Original: Additional details and security analysis are shown in
Revised: Additional details and security analysis are shown in

Original: Appendices A.3.3 and A.3.4.
Revised: Appendices A.3.3 and A.3.4.

Original: Multiply Π Mul OT Input :
Revised: Multiply Π Mul OT Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 s.t.
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 s.t.

Original: 𝒛𝒛 = 𝒙𝒙𝒙𝒙 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 , 𝑦𝑦 𝑏𝑏
Revised: 𝒛𝒛 = 𝒙𝒙𝒙𝒙 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏 , 𝑦𝑦 𝑏𝑏

Original: Compare Π CMP s.t.
Revised: Compare Π CMP s.t.

Original: 𝒛𝒛 = 𝟏𝟏 { 𝒙𝒙 > 𝟎𝟎 }
Revised: 𝒛𝒛 = 𝟏𝟏 { 𝒙𝒙 > 𝟎𝟎 }

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏𝐵𝐵 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏𝐵𝐵 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: NegExp Π nExp s.t.
Revised: NegExp Π nExp s.t.

Original: 𝒛𝒛 = 𝒆𝒆 𝒙𝒙 , 𝒙𝒙 <
Revised: 𝒛𝒛 = 𝒆𝒆 𝒙𝒙 , 𝒙𝒙 <

Original: RecipSqrt Π rSqrt s.t .
Revised: RecipSqrt Π rSqrt s.t .

Original: 𝒛𝒛 = 𝟏𝟏 𝒙𝒙 , 𝒙𝒙 > 𝝐𝝐
Revised: 𝒛𝒛 = 𝟏𝟏 𝒙𝒙 , 𝒙𝒙 > 𝝐𝝐

Original: Recip Π Recip s.t.
Revised: Recip Π Recip s.t.

Original: 𝒛𝒛 = 𝟏𝟏 / 𝒙𝒙
Revised: 𝒛𝒛 = 𝟏𝟏 / 𝒙𝒙

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: Input :
Revised: Input :

Original: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏
Revised: Output : • 𝑃𝑃 𝑏𝑏 : 𝑧𝑧 𝑏𝑏 • 𝑃𝑃 𝑏𝑏 : 𝑥𝑥 𝑏𝑏

Original: To evaluate attention layers, we need an efficient protocol to compute Softmax on secret-shared values.
Revised: To evaluate attention layers, we need an efficient protocol to compute Softmax on secret-shared values.

Original: In particular, for a vector x ∈ Z d 2 ℓ , the Softmax function is denoted as Softmax i ( x ) =
Revised: In particular, for a vector x ∈ Z d 2 ℓ , the Softmax function is denoted as Softmax i ( x ) =

Original: e x i / (cid:80) j ∈ [ d ] e x j for i ∈
Revised: e x i / (cid:80) j ∈ [ d ] e x j for i ∈

Original: [ d ] .
Revised: [ d ] .

Original: The main challenge is to efficiently compute the underlying exponential function.
Revised: The main challenge is to efficiently compute the underlying exponential function.

Original: Following the idea from [11, 30], we first normalize the input vector by x − max i ∈
Revised: Following the idea from [11, 30], we first normalize the input vector by x − max i ∈

Original: [ d ] x i , which is always negative, and then invoke the existing exponential protocol Π nExp in Figure 3 that only evaluates exponentiation on negative inputs.
Revised: [ d ] x i , which is always negative, and then invoke the existing exponential protocol Π nExp in Figure 3 that only evaluates exponentiation on negative inputs.

Original: A simple analysis shows softmax ( x − max i ∈
Revised: A simple analysis shows softmax ( x − max i ∈

Original: [ d ] x i ) is equal to softmax ( x ) .
Revised: [ d ] x i ) is equal to softmax ( x ) .

Original: We evaluate max using a tree-reduction protocol, denoted by Π max .
Revised: We evaluate max using a tree-reduction protocol, denoted by Π max .

Original: Specifically, we arrange the vector x ∈ Z d 2 ℓ into a 2-ary tree with the depth of log d , and evaluate the tree in a top-down fashion [31].
Revised: Specifically, we arrange the vector x ∈ Z d 2 ℓ into a 2-ary tree with the depth of log d , and evaluate the tree in a top-down fashion [31].

Original: In each comparison of two secret-shared elements x i and x j , we reduce it to the invocations of Π CMP and Π Mul OT , i.e., max( x i , x j ) =
Revised: In each comparison of two secret-shared elements x i and x j , we reduce it to the invocations of Π CMP and Π Mul OT , i.e., max( x i , x j ) =

Original: Π
Revised: Π

Original: Mul OT ( x i − x j , Π CMP
Revised: Mul OT ( x i − x j , Π CMP

Original: ( x i − x j ))
Revised: ( x i − x j ))

Original: + x j .
Revised: + x j .

Original: With the above insight, the Softmax protocol is detailed in Algorithm 2.
Revised: With the above insight, the Softmax protocol is detailed in Algorithm 2.

Original: SIRNN [17] also provides a solution for the generic exponential protocol by extending Π nExp .
Revised: SIRNN [17] also provides a solution for the generic exponential protocol by extending Π nExp .

Original: The idea is that the exponential of x equals 1 nExp ( − x ) if x ≥ 0 , and nExp( x ) otherwise, where nExp is the same as the exponential function except for negative inputs.
Revised: The idea is that the exponential of x equals 1 nExp ( − x ) if x ≥ 0 , and nExp( x ) otherwise, where nExp is the same as the exponential function except for negative inputs.

Original: Compared to our solution, realizing the softmax function with this generic exponential protocol additionally requires d calls to the reciprocal protocol Π Recip and multiplication protocol Π MUT OT .
Revised: Compared to our solution, realizing the softmax function with this generic exponential protocol additionally requires d calls to the reciprocal protocol Π Recip and multiplication protocol Π MUT OT .

Original: Besides, compared with the generic library
Revised: Besides, compared with the generic library

Original: MP-SPDZ [16] that provides native support for exponentiation, our protocols achieve orders of magnitude improvement as shown in Section 5.2.
Revised: MP-SPDZ [16] that provides native support for exponentiation, our protocols achieve orders of magnitude improvement as shown in Section 5.2.

Original: Algorithm 2 Secure Softmax Protocol
Revised: Algorithm 2 Secure Softmax Protocol

Original: .2 GELU
Revised: .2 GELU

Original: Rather than crypto-friendly ReLU [13], Transformer-based models utilize GELU activations [15], which can be represented as GELU ( x ) = 0 .
Revised: Rather than crypto-friendly ReLU [13], Transformer-based models utilize GELU activations [15], which can be represented as GELU ( x ) = 0 .

Original: 5 x (cid:16) 1 + Tanh (cid:104) (cid:112) 2 /π (cid:0) x
Revised: 5 x (cid:16) 1 + Tanh (cid:104) (cid:112) 2 /π (cid:0) x

Original: + 0 .
Revised: + 0 .

Original: 044715 x 3 (cid:1)(cid:105)(cid:17) .
Revised: 044715 x 3 (cid:1)(cid:105)(cid:17) .

Original: The complete protocol is shown in Algorithm 3, in which we provide two insights to reduce the cost of GELU .
Revised: The complete protocol is shown in Algorithm 3, in which we provide two insights to reduce the cost of GELU .

Original: First, we present an optimized protocol for the square of a secret-shared input ⟨ x ⟩ .
Revised: First, we present an optimized protocol for the square of a secret-shared input ⟨ x ⟩ .

Original: This relies on the observation: x 2 = ⟨ x ⟩ 20 + ⟨ x ⟩ 21 + 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , where the first two terms can be locally computed by
Revised: This relies on the observation: x 2 = ⟨ x ⟩ 20 + ⟨ x ⟩ 21 + 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , where the first two terms can be locally computed by

Original: P 0 and P 1 .
Revised: P 0 and P 1 .

Original: We only invoke OT protocols to compute the cross term 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , and the optimized overhead is half that of the multiplication protocol Π Mul OT .
Revised: We only invoke OT protocols to compute the cross term 2 ⟨ x ⟩ 0 ⟨ x ⟩ 1 , and the optimized overhead is half that of the multiplication protocol Π Mul OT .

Original: Second, we further optimize the evaluation of Tanh ( x ) =
Revised: Second, we further optimize the evaluation of Tanh ( x ) =

Original: e 2 x − 1 e 2 x +1 .
Revised: e 2 x − 1 e 2 x +1 .

Original: We observe that the sign of x is equal to the sign of Tanh ( x ) , which allows us to leverage the negative exponential protocol Π nExp almost for free.
Revised: We observe that the sign of x is equal to the sign of Tanh ( x ) , which allows us to leverage the negative exponential protocol Π nExp almost for free.

Original: At a high level, our Tanh protocol first learns the sign of the input x , and then evaluates Tanh on the negative input
Revised: At a high level, our Tanh protocol first learns the sign of the input x , and then evaluates Tanh on the negative input

Original: ¯ x with the constraint | ¯ x | = | x | .
Revised: ¯ x with the constraint | ¯ x | = | x | .

Original: Finally, the protocol generates the real output Tanh ( x ) that equals Tanh (¯ x ) if x ≤ 0 , and − Tanh (¯ x ) otherwise.
Revised: Finally, the protocol generates the real output Tanh ( x ) that equals Tanh (¯ x ) if x ≤ 0 , and − Tanh (¯ x ) otherwise.

Original: Our leaner Tanh protocol is given in Algorithm 5 of Appendix A.3.1.
Revised: Our leaner Tanh protocol is given in Algorithm 5 of Appendix A.3.1.

Original: SIRNN [17] recently proposed the most efficient
Revised: SIRNN [17] recently proposed the most efficient

Original: Tanh protocol with the insight of Tanh ( x ) =
Revised: Tanh protocol with the insight of Tanh ( x ) =

Original: Sigmoid (2 x ) − 1 , where Sigmoid ( x ) = 1 1+ e − x .
Revised: Sigmoid (2 x ) − 1 , where Sigmoid ( x ) = 1 1+ e − x .

Original: However, the evaluation of Sigmoid uses the same idea as the general exponential protocol, as shown in Section 4.2.1.
Revised: However, the evaluation of Sigmoid uses the same idea as the general exponential protocol, as shown in Section 4.2.1.

Original: Compared with the protocol in SIRNN, our recipe for Tanh saves one invocation of the reciprocal protocol Π Recip and multiplication protocol Π Mul OT .
Revised: Compared with the protocol in SIRNN, our recipe for Tanh saves one invocation of the reciprocal protocol Π Recip and multiplication protocol Π Mul OT .

Original: Algorithm
Revised: Algorithm

Original: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈
Revised: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈

Original: Z 2 ℓ , ⟨ x ⟩ 1 ∈ Z 2 ℓ , respectively.
Revised: Z 2 ℓ , ⟨ x ⟩ 1 ∈ Z 2 ℓ , respectively.

Original: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈
Revised: Output: P 0 , P 1 get ⟨ y ⟩ 0 ∈

Original: Z 2 ℓ , ⟨ y ⟩ 1 ∈ Z 2 ℓ , respectively, where y = GELU ( x ) .
Revised: Z 2 ℓ , ⟨ y ⟩ 1 ∈ Z 2 ℓ , respectively, where y = GELU ( x ) .

Original: 1: P 0 , P 1 invoke Π Mul OT with inputs ⟨ x ⟩ , and set their outputs as ⟨ z ⟩
Revised: 1: P 0 , P 1 invoke Π Mul OT with inputs ⟨ x ⟩ , and set their outputs as ⟨ z ⟩

Original: =
Revised: =

Original: Fix ( (cid:112) 2 /π )( ⟨ x ⟩ +
Revised: Fix ( (cid:112) 2 /π )( ⟨ x ⟩ +

Original: Fix (0 . 044715)
Revised: Fix (0 . 044715)

Original: ⟨ x ⟩
Revised: ⟨ x ⟩

Original: 3 ) 2: P 0 , P 1 invoke Π Tanh with inputs ⟨ z ⟩ , and set their outputs as ⟨ Tanh ( z ) ⟩ . 3: P 0 , P 1 invoke Π Mul OT with inputs ⟨ Fix (0 . 5) x ⟩ and ⟨ Fix (1) + Tanh ( z ) ⟩ , and output ⟨ y ⟩ .
Revised: 3 ) 2: P 0 , P 1 invoke Π Tanh with inputs ⟨ z ⟩ , and set their outputs as ⟨ Tanh ( z ) ⟩ . 3: P 0 , P 1 invoke Π Mul OT with inputs ⟨ Fix (0 . 5) x ⟩ and ⟨ Fix (1) + Tanh ( z ) ⟩ , and output ⟨ y ⟩ .

Original: 4.2.3 LayerNorm
Revised: 4.2.3 LayerNorm

Original: For a vector x ∈ Z d 2 ℓ , the LayerNorm function is denoted by LayerNorm
Revised: For a vector x ∈ Z d 2 ℓ , the LayerNorm function is denoted by LayerNorm

Original: i ( x ) = γ ( x i − µ ) /σ
Revised: i ( x ) = γ ( x i − µ ) /σ

Original: + β for i ∈
Revised: + β for i ∈

Original: [ d ] , where µ = (cid:80) i ∈ [ d ] x i /d and σ = (cid:113)(cid:80) i ∈
Revised: [ d ] , where µ = (cid:80) i ∈ [ d ] x i /d and σ = (cid:113)(cid:80) i ∈

Original: [ d ] ( x i − µ ) 2 .
Revised: [ d ] ( x i − µ ) 2 .

Original: In contrast to batch normalization (BN) in CNNs that is evaluated for free [14], LayerNorm requires multiplication and reciprocal square root operations.
Revised: In contrast to batch normalization (BN) in CNNs that is evaluated for free [14], LayerNorm requires multiplication and reciprocal square root operations.

Original: In our implementation, we observe that the multiplication dominates the overhead of LayerNorm.
Revised: In our implementation, we observe that the multiplication dominates the overhead of LayerNorm.

Original: To address this issue, we adopt the same idea in GELU to compute the square of x i − µ , which saves half of the communication and computation costs.
Revised: To address this issue, we adopt the same idea in GELU to compute the square of x i − µ , which saves half of the communication and computation costs.

Original: Besides, inspired by the optimization in BN [14], we apply a LayerNorm merge technique to further reduce the overhead.
Revised: Besides, inspired by the optimization in BN [14], we apply a LayerNorm merge technique to further reduce the overhead.

Original: Specifically, we make the observation that the weights γ and β of the LayerNorm layer are already known by P 0 .
Revised: Specifically, we make the observation that the weights γ and β of the LayerNorm layer are already known by P 0 .

Original: As a result, P 0 first multiplies the scale factor γ with the weights of
Revised: As a result, P 0 first multiplies the scale factor γ with the weights of

Original: [Table]
Revised: [Table]

Original: the next linear layer.
Revised: the next linear layer.

Original: After invoking the linear layer protocol on the scaled weights, P 0 adds the shift weight σ to his additive share.
Revised: After invoking the linear layer protocol on the scaled weights, P 0 adds the shift weight σ to his additive share.

Original: The optimized protocol for LayerNorm is presented in Algorithm
Revised: The optimized protocol for LayerNorm is presented in Algorithm

Original: [Table]
Revised: [Table]

Original: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈
Revised: Input: P 0 , P 1 hold ⟨ x ⟩ 0 ∈

Original: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.
Revised: Z d 2 ℓ , ⟨ x ⟩ 1 ∈ Z d 2 ℓ , respectively.

Original: Output: P 0 , P 1 get ⟨ y ⟩ 0 , ⟨ y ⟩ 1 , respectively, where y = LayerNorm ( x ) .
Revised: Output: P 0 , P 1 get ⟨ y ⟩ 0 , ⟨ y ⟩ 1 , respectively, where y = LayerNorm ( x ) .

Original: For i
Revised: For i

Original: ∈ [ d ] , P 0 , P 1 invoke Π Mul OT to compute ⟨ ( x i − µ ) 2 ⟩ , where µ = (cid:80) i ∈ [ d ] x i /d .
Revised: ∈ [ d ] , P 0 , P 1 invoke Π Mul OT to compute ⟨ ( x i − µ ) 2 ⟩ , where µ = (cid:80) i ∈ [ d ] x i /d .

Original: 2: P 0 , P 1 invoke Π rSqrt with inputs ⟨ (cid:80) i ∈
Revised: 2: P 0 , P 1 invoke Π rSqrt with inputs ⟨ (cid:80) i ∈

Original: [ d ] ( x i − µ ) 2 ⟩ to learn output ⟨ 1 σ ⟩ .
Revised: [ d ] ( x i − µ ) 2 ⟩ to learn output ⟨ 1 σ ⟩ .

Original: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 σ ⟩ and ⟨ x i − µ ⟩ , and set outputs as ⟨ y i ⟩ .
Revised: For i ∈ [ d ] , P 0 , P 1 invoke Π Mul OT with inputs ⟨ 1 σ ⟩ and ⟨ x i − µ ⟩ , and set outputs as ⟨ y i ⟩ .

Original:  Evaluation 5.1 Experimental Setup
Revised:  Evaluation 5.1 Experimental Setup

Original: Implementation.
Revised: Implementation.

Original: Iron is built on top of the SEAL library [32] and the EMP toolkit [33] in C++.
Revised: Iron is built on top of the SEAL library [32] and the EMP toolkit [33] in C++.

Original: We also use the EzPC framework [34].
Revised: We also use the EzPC framework [34].

Original: This framework compiles a high-level TensorFlow code to secure computation protocols, which are then executed by our designed cryptographic backends.
Revised: This framework compiles a high-level TensorFlow code to secure computation protocols, which are then executed by our designed cryptographic backends.

Original: Like [17], we simulate a LAN network setting, where the bandwidth is 377 MBps and the echo latency is 0.8ms.
Revised: Like [17], we simulate a LAN network setting, where the bandwidth is 377 MBps and the echo latency is 0.8ms.

Original: All the following experiments are performed on AWS c5.9xlarge instances with Intel Xeonseries CPUs at 3.6GHz.
Revised: All the following experiments are performed on AWS c5.9xlarge instances with Intel Xeonseries CPUs at 3.6GHz.

Original: Datasets and Models.
Revised: Datasets and Models.

Original: We evaluate Iron on four NLP models from [35, 36]: BERT-Tiny, BERTMedium, BERT-Base and BERT-Large.
Revised: We evaluate Iron on four NLP models from [35, 36]: BERT-Tiny, BERTMedium, BERT-Base and BERT-Large.

Original: These models are parameterized by three hyper-parameters: the number of blocks, the dimension of representations and the number of input tokens (refer to
Revised: These models are parameterized by three hyper-parameters: the number of blocks, the dimension of representations and the number of input tokens (refer to

Original: Appendix A.4.1 for the hyper-parameters of these models).
Revised: Appendix A.4.1 for the hyper-parameters of these models).

Original: We train the models for four NLP tasks over the datasets of the Stanford Sentiment Treebank (SST-2), the Microsoft Research Paraphrase
Revised: We train the models for four NLP tasks over the datasets of the Stanford Sentiment Treebank (SST-2), the Microsoft Research Paraphrase

Original: Corpus (MRPC), the Multi-Genre Natural Language Inference Corpus (MNLI) and the Stanford
Revised: Corpus (MRPC), the Multi-Genre Natural Language Inference Corpus (MNLI) and the Stanford

Original: Question Answering Dataset (QNLI) from GLUE benchmarks [18].
Revised: Question Answering Dataset (QNLI) from GLUE benchmarks [18].

Original: 5.2 Microbenchmark Evaluation
Revised: 5.2 Microbenchmark Evaluation

Original: Matrix multiplication .
Revised: Matrix multiplication .

Original: In the left part of Table 1, we compare the performance of the proposed matrix multiplication protocol with the state-of-the-art counterparts of Cheetah [14] and SIRNN [17].
Revised: In the left part of Table 1, we compare the performance of the proposed matrix multiplication protocol with the state-of-the-art counterparts of Cheetah [14] and SIRNN [17].

Original: For fairness, we follow Cheetah for the parameter setup of homomorphic encryption.
Revised: For fairness, we follow Cheetah for the parameter setup of homomorphic encryption.

Original: Compared with
Revised: Compared with

Original: Cheetah, our rumtime is 3 ∼ 26 × faster, and our communication cost is 8 ∼ 25 × lower, depending on the input size.
Revised: Cheetah, our rumtime is 3 ∼ 26 × faster, and our communication cost is 8 ∼ 25 × lower, depending on the input size.

Original: Notably, for small-size matrices (e.g., ones with 32 × 8 and 8 × 16 elements), our protocol only requires 0.11 MB communication and 6 ms runtime, while Cheetah achieves the computation with 2.79 MB and 160 ms.
Revised: Notably, for small-size matrices (e.g., ones with 32 × 8 and 8 × 16 elements), our protocol only requires 0.11 MB communication and 6 ms runtime, while Cheetah achieves the computation with 2.79 MB and 160 ms.

Original: The reason is that our protocol encrypts the whole matrix into a single ciphertext, while Cheetah could only encrypt each row into a ciphertext, totally m ciphertexts ( m is the number of rows of the output matrix).
Revised: The reason is that our protocol encrypts the whole matrix into a single ciphertext, while Cheetah could only encrypt each row into a ciphertext, totally m ciphertexts ( m is the number of rows of the output matrix).

Original: Moreover, compared with SIRNN that implements the most efficient OT-based matrix multiplication protocol, our protocol incurs up to two orders of magnitude less communication and an order of magnitude less runtime.
Revised: Moreover, compared with SIRNN that implements the most efficient OT-based matrix multiplication protocol, our protocol incurs up to two orders of magnitude less communication and an order of magnitude less runtime.

Original: Non-linear functions .
Revised: Non-linear functions .

Original: The right part of Table 1 shows the comparison of our Softmax, GELU and
Revised: The right part of Table 1 shows the comparison of our Softmax, GELU and

Original: LayerNorm protocols with the generic MP-SPDZ framework [16] and the state-of-the-art SIRNN library [17] for math functions.
Revised: LayerNorm protocols with the generic MP-SPDZ framework [16] and the state-of-the-art SIRNN library [17] for math functions.

Original: It is worth noting that we implement some functions that these
Revised: It is worth noting that we implement some functions that these

Original: [Figure]
Revised: [Figure]

Original: frameworks did not provide before.
Revised: frameworks did not provide before.

Original: In particular, GELU and LayerNorm are implemented in MPSPDZ by calling its built-in functions for tanh, square root, and reciprocal, while we add Softmax and GELU protocols in SIRNN utilizing sigmoid and exponent functions.
Revised: In particular, GELU and LayerNorm are implemented in MPSPDZ by calling its built-in functions for tanh, square root, and reciprocal, while we add Softmax and GELU protocols in SIRNN utilizing sigmoid and exponent functions.

Original: As shown in this table, our protocols are orders of magnitude better than those of MP-SPDZ, in terms of both runtime and communication.
Revised: As shown in this table, our protocols are orders of magnitude better than those of MP-SPDZ, in terms of both runtime and communication.

Original: In particular, for the communication cost, our protocols achieve 785 ∼ 993 × improvement.
Revised: In particular, for the communication cost, our protocols achieve 785 ∼ 993 × improvement.

Original: Moreover, while our protocols are built upon the underlying protocols from SIRNN, we also achieve 1 .
Revised: Moreover, while our protocols are built upon the underlying protocols from SIRNN, we also achieve 1 .

Original: 3 ∼ 1 . 8 × lower runtime and 1 .
Revised: 3 ∼ 1 . 8 × lower runtime and 1 .

Original: 4 ∼ 1 . 8 × lower communication due to our customized optimizations.
Revised: 4 ∼ 1 . 8 × lower communication due to our customized optimizations.

Original: Such an improvement gives us significant savings in communication complexity, since the non-linear layer protocols dominate the overall overhead, as described below.
Revised: Such an improvement gives us significant savings in communication complexity, since the non-linear layer protocols dominate the overall overhead, as described below.

Original: 5.3 End-to-end Inference Evaluation
Revised: 5.3 End-to-end Inference Evaluation

Original: Comparison with prior methods.
Revised: Comparison with prior methods.

Original: In the left part of Figure 4, we evaluate our protocols on 4 BERT models compared with SIRNN [17].
Revised: In the left part of Figure 4, we evaluate our protocols on 4 BERT models compared with SIRNN [17].

Original: It is observed that our rumtime is 3 .
Revised: It is observed that our rumtime is 3 .

Original: 3 ∼ 11 .
Revised: 3 ∼ 11 .

Original: 83 × faster than that of SIRNN, and our communication cost is 3 .
Revised: 83 × faster than that of SIRNN, and our communication cost is 3 .

Original: 47 ∼ 14 .
Revised: 47 ∼ 14 .

Original: 11 × lower over four models.
Revised: 11 × lower over four models.

Original: Moreover, our performance gains scale up as the model size grows.
Revised: Moreover, our performance gains scale up as the model size grows.

Original: This is because our protocols achieve better amortized overhead when processing large-scale evaluations.
Revised: This is because our protocols achieve better amortized overhead when processing large-scale evaluations.

Original: We also compare the end-to-end private inference with MP-SPDZ in Figure 4.
Revised: We also compare the end-to-end private inference with MP-SPDZ in Figure 4.

Original: The results show that our protocols are orders of magnitude better, in terms of both time and communication costs.
Revised: The results show that our protocols are orders of magnitude better, in terms of both time and communication costs.

Original: This is because specialized protocols are more communication efficient than generic alternatives, which is also observed by SIRNN.
Revised: This is because specialized protocols are more communication efficient than generic alternatives, which is also observed by SIRNN.

Original: Performance breakdown.
Revised: Performance breakdown.

Original: In Figure 5, we present the runtime and communication breakdown of
Revised: In Figure 5, we present the runtime and communication breakdown of

Original: Iron on four BERT models.
Revised: Iron on four BERT models.

Original: For clarity, we just report the result of one encoder.
Revised: For clarity, we just report the result of one encoder.

Original: Recall that our private inference can be divided into linear protocols and non-linear protocols.
Revised: Recall that our private inference can be divided into linear protocols and non-linear protocols.

Original: For linear protocols, we observe that as the model size increases, the proportion of communication overhead remains approximately constant, accounting for 12 ∼ 13% .
Revised: For linear protocols, we observe that as the model size increases, the proportion of communication overhead remains approximately constant, accounting for 12 ∼ 13% .

Original: This shows that our compact ciphertext encoding method effectively reduces the size of communication.
Revised: This shows that our compact ciphertext encoding method effectively reduces the size of communication.

Original: For non-linear functions, as the model size increases, the proportion of computation overhead gradually decreases, from 84% in BERT-Tiny to 76% in BERT-Large.
Revised: For non-linear functions, as the model size increases, the proportion of computation overhead gradually decreases, from 84% in BERT-Tiny to 76% in BERT-Large.

Original: The savings come from amortizing the communication and computation costs by packing data from large tensors.
Revised: The savings come from amortizing the communication and computation costs by packing data from large tensors.

Original: Despite such advantages, the main bottleneck of our work is the communication overhead of non-linear layers.
Revised: Despite such advantages, the main bottleneck of our work is the communication overhead of non-linear layers.

Original: However, it is an open problem to solve the communication issue while maintaining the model accuracy.
Revised: However, it is an open problem to solve the communication issue while maintaining the model accuracy.

Original: Accuracy comparison with plaintext.
Revised: Accuracy comparison with plaintext.

Original: In the left part of Figure 6, we show the accuacy of plaintext (float-point) and Iron (fixed-point) on the BERT-Tiny model.
Revised: In the left part of Figure 6, we show the accuacy of plaintext (float-point) and Iron (fixed-point) on the BERT-Tiny model.

Original: We observe that the accuracy achieved by Iron matches the accuracy of the plaintext TensorFlow code.
Revised: We observe that the accuracy achieved by Iron matches the accuracy of the plaintext TensorFlow code.

Original: Specifically, the accuracy loss does not exceed 0 .
Revised: Specifically, the accuracy loss does not exceed 0 .

Original: 3% over all datasets, and surprisingly, Iron exceeds the plaintext baseline on MNLI by 0 .
Revised: 3% over all datasets, and surprisingly, Iron exceeds the plaintext baseline on MNLI by 0 .

Original: Similar results also appear in private CNN inference [13].
Revised: Similar results also appear in private CNN inference [13].

Original: Such accuracy advantages experimentally validate that our protocols are numerically precise.
Revised: Such accuracy advantages experimentally validate that our protocols are numerically precise.

Original: Moreover, in the right part of
Revised: Moreover, in the right part of

Original: Figure 6, we also compare the accuracy with the plaintext baseline, as the fractional scale varies on MRPC.
Revised: Figure 6, we also compare the accuracy with the plaintext baseline, as the fractional scale varies on MRPC.

Original: We observe that Iron with the scale of 12 exactly matches the accuracy of the plaintext model.
Revised: We observe that Iron with the scale of 12 exactly matches the accuracy of the plaintext model.

Original: The accuracy loss is lower than 1% when the scale ≥ 6 .
Revised: The accuracy loss is lower than 1% when the scale ≥ 6 .

Original: This conclusion is in line with the prior work [13], namely that neural networks can tolerate stochastic fault behavior [37].
Revised: This conclusion is in line with the prior work [13], namely that neural networks can tolerate stochastic fault behavior [37].

Original:  Discussion
Revised:  Discussion

Original: We discuss possible solutions to further improve the efficiency of private inference on Transformers.
Revised: We discuss possible solutions to further improve the efficiency of private inference on Transformers.

Original: They are compatible with our proposed protocols and hence can be directly integrated into our framework.
Revised: They are compatible with our proposed protocols and hence can be directly integrated into our framework.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Pushing expensive operations into an offline phase.
Revised: Pushing expensive operations into an offline phase.

Original: Our framework is readily extended to the preprocessing model, including an offline client-input independent phase and an online input-dependent phase.
Revised: Our framework is readily extended to the preprocessing model, including an offline client-input independent phase and an online input-dependent phase.

Original: This paradigm has been instantiated in private CNN inference [38, 8], and the results show that about 99% of the cryptographic overhead can be moved to the offline phase.
Revised: This paradigm has been instantiated in private CNN inference [38, 8], and the results show that about 99% of the cryptographic overhead can be moved to the offline phase.

Original: We briefly outline how to build our protocols with this paradigm.
Revised: We briefly outline how to build our protocols with this paradigm.

Original: For linear operations, we can generate in advance Beaver’s triple in the matrix form [22, 38] using our matrix multiplication protocol.
Revised: For linear operations, we can generate in advance Beaver’s triple in the matrix form [22, 38] using our matrix multiplication protocol.

Original: In the online phase, these triples are consumed, and it additionally requires non-cryptographic operations and plaintext communication.
Revised: In the online phase, these triples are consumed, and it additionally requires non-cryptographic operations and plaintext communication.

Original: For non-linear layers, almost all operations crucially rely on the OT protocols, which can be pre-generated in the offline phase [39, 14].
Revised: For non-linear layers, almost all operations crucially rely on the OT protocols, which can be pre-generated in the offline phase [39, 14].

Original: Like linear protocols, the online phase is cheap.
Revised: Like linear protocols, the online phase is cheap.

Original: Using mixed-bitwidths in private inference.
Revised: Using mixed-bitwidths in private inference.

Original: Iron works with a uniform bitwidth, which is required to be large enough to accommodate all intermediate values.
Revised: Iron works with a uniform bitwidth, which is required to be large enough to accommodate all intermediate values.

Original: While effectively avoiding integer overflows, our protocols may cause intensive communication, since the performance of non-linear operators depends critically on bitwidths.
Revised: While effectively avoiding integer overflows, our protocols may cause intensive communication, since the performance of non-linear operators depends critically on bitwidths.

Original: Taking inspiration from [17, 40], one possible optimization is to employ non-uniform (mixed) bitwidths, operate in low bitwidths and switch to high bitwidths only when necessary.
Revised: Taking inspiration from [17, 40], one possible optimization is to employ non-uniform (mixed) bitwidths, operate in low bitwidths and switch to high bitwidths only when necessary.

Original: To this end, the mixed-precision model should be generated with proper compilers by using techiniques like quantization [41, 42].
Revised: To this end, the mixed-precision model should be generated with proper compilers by using techiniques like quantization [41, 42].

Original: With mixed-bitwidths fixed-point models, our protocols can be integrated seamlessly.
Revised: With mixed-bitwidths fixed-point models, our protocols can be integrated seamlessly.

Original: Applying orthogonal model optimizations.
Revised: Applying orthogonal model optimizations.

Original: We can improve performance by simplifying the model architecture such as model pruning and advanced neural architecture search [43, 44], which have been commonly adopted in private CNN inference [12, 45, 37].
Revised: We can improve performance by simplifying the model architecture such as model pruning and advanced neural architecture search [43, 44], which have been commonly adopted in private CNN inference [12, 45, 37].

Original: Like prior works [12, 45, 37] that find and tailor models to the requirements of private inference, we can define a proper search space with the goal of decreasing the overhead, e.g., substituting costly Softmax and GELU functions, or reducing the matrices’ dimension in matrix multiplication.
Revised: Like prior works [12, 45, 37] that find and tailor models to the requirements of private inference, we can define a proper search space with the goal of decreasing the overhead, e.g., substituting costly Softmax and GELU functions, or reducing the matrices’ dimension in matrix multiplication.

Original: As mentioned in Section 1, the concurrent work, THE-X [19], has explored replacing complex math functions with HE-friendly alternatives while achieving comparable accuracy.
Revised: As mentioned in Section 1, the concurrent work, THE-X [19], has explored replacing complex math functions with HE-friendly alternatives while achieving comparable accuracy.

Original:  Conclusion
Revised:  Conclusion

Original: We propose Iron , an efficient cryptographic framework for private Tramsformer inference.
Revised: We propose Iron , an efficient cryptographic framework for private Tramsformer inference.

Original: Specifically, we carefully design a new encoding method for optimizing homomorphic encryption-based matrix multiplication.
Revised: Specifically, we carefully design a new encoding method for optimizing homomorphic encryption-based matrix multiplication.

Original: Further, we devise several communication-efficient non-linear protocols like
Revised: Further, we devise several communication-efficient non-linear protocols like

Original: Softmax, LayerNorm and GELU by integrating advanced secret-sharing primitives.
Revised: Softmax, LayerNorm and GELU by integrating advanced secret-sharing primitives.

Original: Experimental results show that Iron outperforms prior art by up to one order of magnitude in terms of computation and communication overheads.
Revised: Experimental results show that Iron outperforms prior art by up to one order of magnitude in terms of computation and communication overheads.

Original: We believe that our novel protocols would help advance the practical instantiations of private Transformer inference.
Revised: We believe that our novel protocols would help advance the practical instantiations of private Transformer inference.

Original: Acknowledgment
Revised: Acknowledgment

Original: The authors would like to thank the anonymous reviewers for their insightful comments.
Revised: The authors would like to thank the anonymous reviewers for their insightful comments.

Original: This work is supported by the Key-Area Research and Development Program of Guangdong Province under
Revised: This work is supported by the Key-Area Research and Development Program of Guangdong Province under

Original: Grant 2020B0101360001, National Natural Science Foundation of China under Grants 62020106013, 61972454, and 61802051, Sichuan Science and Technology Program under Grants 2020JDTD and 2020YFG0298, the Fundamental Research Funds for Chinese Central Universities under GrantZYGX2020ZB027, and Singapore Ministry of Education [Table] Checklist
Revised: Grant 2020B0101360001, National Natural Science Foundation of China under Grants 62020106013, 61972454, and 61802051, Sichuan Science and Technology Program under Grants 2020JDTD and 2020YFG0298, the Fundamental Research Funds for Chinese Central Universities under Grant

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [No] (d) Did you include the total amount of compute and the type of resources used (e

Original: g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?

Original: [No] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]

Original: A Appendix A.1 More Details on Preliminaries A.1.1 Fixed-Point Encoding

Original: Same as other neural networks, Transformer-based models use floating-point arithmetic, however cryptographic protocols operate on integers.

Original: Therefore, we require a float-to-integer conversion [46, 30, 17] to represent a floating-point number x ∈ Q into the ring Z 2 ℓ .

Original: Specifically, we first encode it as a fixed-point number, which is parametrized by a scale parameter s that determines the fractional precision.

Original: Then, we embed the fixed-point representation into the ring with 2’s complement representation.

Original: The formulation is a =

Original: ⌊ 2 s × x ⌋ ∈

Original: Z 2 ℓ if x is a non-negative number, and a = 2 ℓ − ⌊ 2 s ×

Original: | x |⌋ ∈ Z 2 ℓ if x is a negative number, where s is the length of the (binary) fractional bits and ℓ is the bitwidth of the secret sharing ring.

Original: Unless otherwise stated, similar as prior works [14], we set the bitwidth as 37 and the scale as 12 in the fixed-point encoding.

Original: Because of the use of the above fixed-point encoding, after multiplication, the scale of the output is 2 s .

Original: Therefore, a truncation operation is required to reduce scale.

Original: We use the Π Trunc protocol proposed in [13] and improved by [17], which leads to faithful implementation of fixed-point arithmetic.

Original: For simplicity, we omit this operation in our protocol description.

Original: The overhead of truncation in Iron will be reported in Table 4.

Original: A.1.2 Formal Description of the Threat Model

Original: Same as prior private inference works [13, 14, 17], the security of Iron is provably provided in the simulation paradigm against static honest-but-curious probabilistic polynomial-time (PPT) adversaries.

Original: Namely, a PPT adversary

Original: A passively corrupts either the server or the client at the beginning of the protocol and honestly follows the protocol specification.

Original: In the simulation paradigm, two worlds are defined: a real world where the server and the client perform the protocol according to the specification in the presence of A , and an ideal world where the parties send their inputs to a trusted dealer (also called functionality) that executes the evaluation faithfully.

Original: The executions in both worlds are coordinated by the environment Env , which chooses the inputs to the parties and plays the role of a distinguisher between the real and ideal executions.

Original: It is required that for any adversary, the real-world distribution is computationally indistinguishable to the ideal-world distribution.

Original: Some of our protocols invoke sub-protocols

Original: and we describe them using the hybrid model.

Original: This is similar to a real execution, except that sub-protocols are replaced by the invocations of the corresponding functionality instances.

Original: We recap the definition of a private inference protocol in [38] [14] as below.

Original: Definition A.1.

Original: A protocol Π PI between the server having as input a model M with weights w and the client having as input a sample x is a private inference protocol against honest-but-curious adversaries if it satisfies the following guarantees: 1) Correctness : on every model weights w and every input sample x , the output of the client at the end of the protocol is the correct inference

Original: M ( w, x ) .

Original: 2) Security : For a corrupted client, there exists an efficient simulator Sim C such that

Original: View PI C is computationally indistinguishable Sim C (output) , where View PI C is the view of the client in the execution of Π PI and output denotes the output of the inference.

Original: Similarly, for a corrupted server, there exists an efficient simulator Sim S such that View PI S is computationally indistinguishable Sim S .

Original: Notice that the honest-but-curious security proof of Iron according to the above definition will follow trivially from sequential composibility of individual sub-protocols [47, 13, 30].

Original: Hence, we require to provide a security proof for our matrix multiplication and non-linear protocols.

Original: We refer to Section

Original: A.2.3 and A.3.4 for the sub-protocols’ security analysis.

Original: A.1.3 Multi-Head Attention

Original: Instead of performing a single attention function, existing Transformer-based models [1, 2] follow a multi-head attention variant, which can be represented as

Original: [Equation]

Original: where H is the number of heads, and X Q,

Original: [Equation]

Original: for j ∈ [ H ] .

Original: The main intuition is that multi-head attention allows the model to jointly attend to information from different representation subspaces at different positions [1].

Original: A.2 More Details on the Matrix Multiplication Protocol A.2.1 Correction Proof of the Matrix Multiplication Protocol

Original: Proof.

Original: For each i ∈ [ m ] and j ∈ [ k ] , we write ϵ i,j = i · n · k + ( j + 1) · n − 1 for simplicity.

Original: Based on the description of Section 4.1, for ϵ i,j ≥ nk , ˆ y

Original: [ ϵ i,j ] = 0 holds.

Original: Therefore, given the definition of

Original: Equation 1, we have ˆ z

Original: [-[ ϵ i,j ] = (cid:80) 
-]
Revised: [-[ ϵ i,j ] = (cid:80) 
-]

Original: [-≤ µ<n ˆ x [ i · n · k +( n − 1) − µ ]ˆ y [ j · n + µ ] = (cid:80) 
-]
Revised: [-≤ µ<n ˆ x [ i · n · k +( n − 1) − µ ]ˆ y [ j · n + µ ] = (cid:80) 
-]

Original: ≤ µ<n X

Original: [ i, µ ] Y [ µ, j ] , which is exactly Z

Original: [ i ][ j ] .

Original: A.2.2 Optimal Parameters Selection in the Matrix Multiplication Protocol

Original: As shown in Section 4.1, the matrix multiplication X m × n · Y n × k requires the communication of mm w ( nn w + kk w ) ciphertexts.

Original: To minimize the ciphertext communication cost, we formalize the selection of the parameters m w , n w , k w as an optimization problem, i.e., min { m w ,n w ,k w } mm w ( nn w + kk w ) , s.t., m w n w k w ≤ N , where m, n, k, N are constants.

Original: Given the difficulty of solving the above multivariate optimization, we figure out a sub-optimal solution.

Original: To this end, we first fix m w = m 4 , and hence the constraint is transformed to n w k w ≤ Nm .

Original: Correspondingly, the new optimization problem is min { n w ,k w } nn w + kk w , s.t., n w k w ≤ Nm .

Original: Then, the following holds:

Original: [Equation]

Original: where the first inequality is due to 1 a + 1 b ≥ 2 √ ab , and the second inequality comes from n w k w ≤ Nm .

Original: As a result, assuming m w = m , the optimal communication size is 2 √ mnk √ N ciphertexts.

Original: Notably, it may not have the optimal analytical solution, because the variables must be discrete positive integers, rather than real number.

Original: To achieve minimal communication in our implementation, like Cheetah [14], we use the exhaustive testing approach on all the results satisfying the constraint to find the optimal matrix partitioning strategy.

Original: Note that this is quite fast due to the small search range.

Original: Communication comparison with Cheetah.

Original: As shown in Section 3.1 of [14], the Cheetah’s communication overhead is m ( nn w + kk w ) ciphertexts with the constraint n w k w ≤ N .

Original: By using the similar analysis as above, we can obtain its optimal solution, i.e., 2 m √ nk √ N .

Original: Therefore, we obtain a √ m × communication improvement in an ideal situation (i.e., the optimal analytical integer solution exists).

Original: A.2.3 Security Proof of the Matrix Multiplication Protocol

Original: Theorem A.2.

Original: In presence of an honest-but-curious adversary, the protocol Π MatMul in Algorithmrealizes the matrix multiplication functionality, in which P 0 and P 1 take as inputs the matrices X and Y , and learn the secret shares ⟨ Z ⟩ 0 and ⟨ Z ⟩ 1 , respectively, such that Z = XY .

Original: Proof.

Original: The correctness of Theorem A.2 is directly derived from Theorem 4.1.

Original: We below focus on the protocol’s security when the server or the client is corrupted.

Original: Our security proof follows the simulation paradigm defined in Section A.1.2.

Original: In this paradigm we need show that the real-world distribution is computationally indistinguishable to the simulated distribution by the simulator Sim in the ideal world.

Original: Proof of indistinguishability with the corrupted server.

Original: The server’s view of View MatMul S consists of ciphertexts CT β,γ .

Original: The simulator Sim S for this view can be constructed as follows:

Original: Given the access to public parameters, Sim S outputs ciphertexts (cid:102) CT β,γ = Enc (0) to the server.

Original: The security against the corrupted server is directly reduced to the semantic security of the underlying homomorphic encryption scheme.

Original: Thus we have that the simulated view View MatMul S in the ideal world is computationally indistinguishable from the real-world distribution of the protocol.

Original: Proof of indistinguishability with corrupted clients.

Original: The client’s view of View MatMul C consists of ciphertexts CT ′ α,γ , and the decryption of these ciphertexts, i.e., ⟨ Z ⟩ 1 .

Original: The simulator Sim C for this view can be constructed as follows:

Original: On receiving the ciphertexts CT α,γ from the client, Sim C samples uniformly random polynomials ˆ r α,γ ∈ A N, 2 ℓ , and computes (cid:102) CT ′ α,γ =

Original: Enc

Original: (ˆ r α,γ ) .

Original: Given the access to the output, Sim C outputs (cid:102) CT ′ α,γ to the client.

Original: Similarly, the ciphertexts (cid:102) CT ′ α,γ are computationally indistinguishable from (cid:102) CT α,γ due to the semantic security.

Original: Besides, the values of ⟨ Z ⟩ 1 distribute uniformly in Z 2 ℓ , which is exactly the same distribution of values in ˆ r α,γ .

Original: Thus we have that View MatMul C is computationally indistinguishable from the real-world distribution of the protocol.

Original: A.3 More Details on Non-linear Protocols A.3.1 Tanh

Original: Input: P P 1 hold ⟨ x ⟩ 0 ∈ Z ℓ ,
Revised: ZYGX2020ZB027, and Singapore Ministry of Education

Original: ⟨ x ⟩

Original: ∈ Z 2 ℓ ,

Original: respectively.

Original: Output:

Original: P 0 , P 1 get ⟨ y ⟩ 0 ∈

Original: Z 2 ℓ , ⟨ y ⟩ 1 ∈ Z 2 ℓ , respectively, where y =

Original: Tanh ( x ) .

Original: [-1: P 0 , P 1 parse ⟨ x ⟩ 0 = msb 
-]
Revised: [-1: P 0 , P 1 parse ⟨ x ⟩ 0 = msb 
-]

Original: ∥ a 0 and ⟨ x ⟩ 1 = msb 1 ∥ a 1 , and invoke Π CMP to learn ⟨ carry ⟩ Bb , where carry = 1 { a 0 + a 1 > 2 ℓ − 1 − 1 } , and the inputs are 2 ℓ − 1 − a 0 − 1 and a 1 from P 0 and P 1 , respectively.

Original: For b ∈ { 0 , 1 } , P b outputs ⟨ MSB ( x ) ⟩ Bb = ⟨ carry ⟩ Bb ⊕ msb b . 2: P 0 , P 1 invoke Π Mul OT with inputs ⟨ 2 x ⟩ and ⟨ MSB ( x ) ⟩ B , and set outputs as ⟨ ¯ x ⟩ , where ¯ x = 2 x · MSB ( x ) − x that is always negative with the constraint of |

Original: ¯ x | = | x | .

Original: 3: P 0 , P 1 invoke Π nExp with negative inputs ⟨ 2¯ x ⟩ and learn ⟨ e 2¯ x ⟩ . 4: P 0 , P 1 invoke Π Recip with inputs ⟨ e 2¯ x ⟩ , and learns ⟨ ¯ y ⟩ where ¯ y

Original: =

Original: −

Original: e

Original: 2¯

Original: x +1 .

Original: 5: P 0 , P 1 invoke an instance of Π Mul OT on input ⟨ MSB ( x ) ⟩ B and ⟨ ¯ y ⟩ , and learn ⟨ y ⟩ , where y

Original: = ¯ y

Original: + MSB ( x ) · ( − 2¯ y ) .

Original: We present an optimized protocol for Tanh in Algorithm 5, which builds over the protocol used in [17].

Original: Our optimization relies on the observation: when evaluating on x , the sign of Tanh ( x ) is the same as that of x .

Original: Compared with the protocol in [17], our alternative saves one invocation of the reciprocal protocol Π Recip and multiplication protocol Π Mul OT .

Original: A.3.2 Underlying Protocols from [17, 13]

Original: We outline the underlying protocols from existing works [17, 13], and the detailed implementation could be found in the corresponding papers.

Original: • Multiplication

Original: (Mul OT ):

Original: The OT-based multiplication protocol Π Mul OT takes as input ⟨ x ⟩ ∈ { 0 , 1 } ℓ and ⟨ y ⟩ ∈ { 0 , 1 } ℓ and returns ⟨ z ⟩ such that z = xy .

Original: The well-known technique is proposed by ABY [22] and optimized in [13].

Original: Currently, the optimal solution invokes 2- COT i for i ∈ { 1 , . .

Original: ℓ } requiring communication ℓ ( λ + ℓ +12 ) bits with 2 rounds that is equivalent to ℓ instances of COT ℓ +12 .

Original: A variant of multiplication is multiplexer 5 , which takes as input ⟨ x ⟩ B ∈ { 0 , 1 } and ⟨ y ⟩ ∈ { 0 , 1 } ℓ and outputs ⟨ z ⟩ ∈ { 0 , 1 } ℓ such that

Original: z

Original: =

Original: y

Original: if x = 1 and 0 otherwise.

Original: The multiplexer protocol Π Mux can be realized by 2 parallel calls of 2- COT ℓ with communication 2( λ + ℓ ) bits and 2 rounds.

Original: Comparison (CMP): The comparison protocol Π CMP takes as input ⟨ x ⟩ ∈ { 0 , 1 } ℓ , and returns ⟨ z ⟩ such that z = 1 { x ≥ 0 } .

Original: Recently, [13] gave an efficient protocol for Π CMP with communication less than λℓ + 14 ℓ bits with log ℓ rounds.

Original: • Exponential on negative inputs (nExp): The exponential protocol Π nExp takes as input x ∈ { 0 , 1 } ℓ , where x ≤ 0 , and returns ⟨ z ⟩ such that z = e x .

Original: The protocol is proposed by [17], which invokes digit decomposition to generate small-length inputs and integrates the OT-based lookup table technique to compute exponential on the small-length inputs.

Original: • Reciprocal of Square Root (rSqrt): The protocol of square root’s reciprocal, Π rSqrt , takes as input x ∈ { 0 , 1 } ℓ and returns ⟨ z ⟩ ∈ { 0 , 1 } ℓ such that z = 1 √ x .

Original: [17] proposed the state-of-the-art OT-based protocol, which relies on the Goldschmidt’s algorithm [48] that iterates on an initial approximation.

Original: • Reciprocal (Recip): The reciprocal protocol Π Recip takes as input x ∈ { 0 , 1 } ℓ and returns ⟨ z ⟩ ∈ { 0 , 1 } ℓ such that z = 1 /x .

Original: The most efficient implementation is proposed in [17] with a similar idea as the protocol Π rSqrt .

Original: A.3.3 Extra Optimization

Original: MSB-known protocol optimization.

Original: As pointed out by [17], 2PC protocols could be designed in a far more efficient way when the MSB of the inputs are known.

Original: In particular, we optimize truncation and OT-based multiplication in this case.

Original: For example, the MSB-known truncation protocol requires

Original: O ( λ ( s + 3)) communication, instead of O ( λ ( ℓ + 3)) , where λ is the security parameter, ℓ is the bit length of the secret sharing ring and s is the fractional scale.

Original: We elaborate the optimization for the GELU protocol in Algorithm 3, and the same idea can also be used in the Softmax and LayerNorm protocols.

Original: For GELU, we fist compute the shares of MSB ( x ) , instead of calculating it in the latter Tanh protocol, and then in the following sub-process, we use this knowledge to reduce overhead.

Original: Moreover, we further observe that the GELU protocol implies more

Original: MSB-known operations if proper computation order is considered.

Original: We rewrite the GELU formulation as below:

Original: [Equation]

Original: We observe that 1 + 0 .

Original: 044715 x 2 and x Tanh (cid:104)(cid:112)

Original: [Equation]

Original: /πx (cid:0) 1 + 0 .

Original: 044715 x 2 (cid:1)(cid:105) are always non-negative, where the latter holds because the sign x equals to that of Tanh (cid:104)(cid:112) 2 /πx (cid:0) 1 + 0 .

Original: 044715 x 2 (cid:1)(cid:105) .

Original: A.3.4 Security Proof of Non-linear Protocols

Original: Similar as the security of protocols in [13, 17], our protocols directly follow in the hybrid model.

Original: In particular, the security of the Softmax and GELU protocols are easy to see in ( CMP , nExp , Recip , MUL OT ) -hybrid.

Original: Besides, the security of the LayerNorm protocol follows in ( rSqrt , MUL OT ) -hybrid.

Original: A.4 More Details on Experimental Evaluation A.4.1 Additional Experimental Setup

Original: [Table][Table]
Revised: [Table]

Original: We evaluation Iron on 4 widely used pre-trained BERT models with different hyper-parameters, as shown in Table 2.

Original: We denote the number of blocks as b , the dimension of representations as d , and the number of input tokens as t .

Original: We always fix the number of self-attention heads to d/and the size of feed-forward features to 4 d .

Original: The end-task models are obtained by stacking a linear classifier on top of the Transformer architectures with fine-tuning.

Original: We follow the default fine-tuning hyper-parameters in [35], e.g., batch size 32, learning rate 2 × 10 − 5 and epoch 3.

Original: Notice that any hyper-parameters optimization during the training phase is compatible with our scheme.

Original: Besides, we use 4 datasets for different tasks from GLUE [18], which include the Stanford Sentiment Treebank

Original: [Table]

Original: (SST-2), the Microsoft Research Paraphrase Corpus (MRPC), the Multi-Genre Natural Language

Original: Inference Corpus (MNLI) and the Stanford Question Answering Dataset (QNLI).

Original: Table 3 shows the datasets’ details.

Original: A.4.2 Additional Experimental Results

Original: Detailed performance breakdown on BERT.

Original: In Table 4, we show the detailed performance breakdown including the communication and computation costs of matrix multiplication, truncation,

Original: GELU, softmax and layer normalization.

Original: The most expensive non-linear operation is GELU due to its huge number.

Original: For example, for each layer of BERT-Base, the number of GELU is 393,216.

Original: We also observe that the our linear operation is lightweight in terms of communication

Original: [Table]

Original: A.5 Related Works

Original: Recently a quantity of works have designed customized protocols for performing private inference on neural networks, especially convolutional neural networks.

Original: These special-purpose protocols improve the computation and communication costs and generally fall into two categories: linear protocols and non-linear protocols.

Original: We briefly discuss the progress as below.

Original: Linear protocols.

Original: Gazelle [29] proposed an optimized AHE-based linear algebra kernels, which support matrix-vector multiplication and convolution operations.

Original: The main innovation is a new packing method to minimize the expensive rotation operations, which is the critical component for the linear algebra.

Original: After that, CrypTFlow2 [13] proposed a comprehensive implementation for linear layers, based on both AHE-based and OT-based solution 6 .

Original: For the AHE-based solution, they use the protocol from Gazelle, and employ several optimizations such as parallelization and reducing ciphertext size.

Original: They observe the AHE-based solution performs better than the OT-based counterpart, especially for large-scale models.

Original: More recently, Huang et al. presented Cheetah [14], the most efficient AHE-based linear layer protocols, including matrix-vector multiplication and convolution operations.

Original: The improvement comes from a novel input packing technique, which is rotation-free and hence efficient.

Original: Moreover, the packing method is compatible with secret sharing in a ring.

Original: This support further benefits the subsequent non-linear operations [14].

Original: However, existing protocols are only optimized for matrix-vector multiplication, rather than general matrix multiplication Iron relies on.

Original: As mentioned earlier, directly extending the most efficient matrix-vector multiplication protocol still causes prohibitively high communication overhead.

Original: Therefore, to approach such communication issue, we propose a special-purpose protocol for matrix multiplication, based on the state-of-the-art protocol in Cheetah.

Original: Notice that different from the setting of 2PC private inference, [49] proposed a private outsourced inference scheme, which stands for encrypted data and encrypted model.

Original: To this end, [49] designs a homomorphic matrix multiplication protocol for multiplying two encrypted matrices, which is fundamentally different our homomorphic multiplication with a plaintext.

Original: As a result, it requires to invoke costly homomorphic multiplication and rotation operations, which are about 2 ∼ 20 × more expensive than the underlying operations of our protocol (refer to Table 9 of [50]).

Original: Non-linear protocols.

Original: Although earlier works [29, 38] implemented non-linear function evaluation with garbled circuits (GC), CrypTFlow2 [13] found that these GC-based solutions result in high communication overhead.

Original: Therefore, the authors designed optimized OT-based protocols, such as truncation and comparison.

Original: These protocols achieve state-of-the-art performance, and can be seen as general underlying building blocks for the design of advanced protocols [17].

Original: Despite the efficiency advantage for truncation and comparison, these protocols can not support complex functions, like exponent in Transformers.

Original: Actually, the state-of-the-art general-purpose framework, MP-SPDZ, provides comprehensive protocols.

Original: However, as shown in SIRNN [17], the protocols implemented with MP-SPDZ are communication-heavy and computation-intensive.

Original: Therefore, SIRNN [17] proposed special-purpose protocols for exponent on negative inputs, sigmoid and reciprocal of square root, which achieve orders of magnitude improvement over MP-SPDZ, both in terms of runtime and communication.

Original: However, these functions are still insufficient to implement a private inference framework on Transformers.

Original: Therefore, on the basis of the building blocks in [13, 17], we propose new protocols for three non-linear functions that are critical components for Transformers, and make several specialized optimizations.

Original: Note that, [11] also proposed a softmax protocol but in an unrealistic setting, where except the client and the server, a trusted third party (TTP) exists and assists to generate correlated randomness to accelerate protocol evaluation.

Original: However, in a practical application, it is difficult to have a completely TTP [13, 14].

Original: In contrast to [11], our setting lies in a practical client-server setting, without any unrealistic assumptions.