SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xin-Ran Xie\*, Man-Jie Yuan\*, Xue-Tong Bai, Wei Gao, Zhi-Hua Zhou  
National Key Laboratory for Novel Software Technology, Nanjing University, China  
School of Artificial Intelligence, Nanjing University, China  
{xiexr, yuanmj, baixt, gaow, zhouzh}@lamda.nju.edu.cn
Random forests have been one of the successful ensemble algorithms in machine learning. Various techniques have been utilized to preserve the privacy of random forests, such as anonymization, differential privacy, homomorphic encryption, etc. This work takes one step towards data encryption by incorporating some crucial ingredients of learning algorithm. Specifically, we develop a new encryption to preserve data's Gini impurity, which plays an important role during the construction of random forests. The basic idea is to modify the structure of binary search tree to store several examples in each node, and encrypt the data features by incorporating label and order information. Theoretically, our scheme is proven to preserve the minimum Gini impurity in ciphertexts without decrypting, and we also present the security guarantee for encryption. For random forests, we encrypt data features based on our Gini-impurity-preserving scheme, and take the homomorphic encryption scheme CKKS to encrypt data labels owing to their importance and privacy. We finally present extensive empirical studies to validate the effectiveness, efficiency and security of our proposed method.
From the pioneer work [1], random forests have been one successful ensemble algorithm [2-4], with diverse applications such as ecology [5], computational biology [6], objection recognition [7], remote sensing [8], computer vision [9], etc. The basic idea is to construct a large number of random trees individually and make prediction based on an average of their predictions. Numerous variants of random forests have been developed to improve performance under different settings [10-22], as well as theoretical understandings on the success of random forests [21, 23-27]. The splitting criterion, such as Gini impurity and information gain, has been one of the most important ingredient during the construction of random forests [1, 28].
Various techniques have been adopted to preserve the privacy of random forests, especially for sensitive tasks such as medical diagnosis, financial predictions, and so on. For example, differential privacy [29] has been successfully applied to preserve the privacy of random forests [30, 31] and decision trees [32-34], by adding certain noise perturbations. Another relevant approach is the secure multi-party computation for random forests and decision tree [35-39], where the privacy is preserved by multi-party joint computation over respective data inputs without leakage.
Homomorphic encryption [40-43] has been one of the most important cryptosystems in privacy-preserving computing [44-47]. Based on such scheme, various algorithms have been developed to train privacy random forests and decision trees [48-52], while some other methods only considered inference without training due to computational costs [53-58]. In addition, LeFevre et al. [59] took
*These authors contribute equally.
Table 1: Comparisons of communications and complexities for different privacy-preserving decision trees. Here,  $n$  is the number of examples in training data, and  $\tau$  is the cardinality of label space. Let  $h$  and  $\kappa$  be the height and number of leaves of decision tree  $\left( {h < \kappa }\right)$  ,respectively. Denote by  $\bar{y}$  the average number of possible splitting features and positions in the construction of decision trees,and  $p$  is the number of clients for secure multi-party computation. ‘-’ means the corresponding methods focusing only on inference without training.
Scheme	Training communication	Training comp. complexity	Predictive communication	Predictive comp. complexity	Privacy of model
Rounds	Bandwidth	Client	Server	Rounds	Bandwidth	Client	Server
SMCDT [61]	O(κ)	O(¯τn)	O(κ¯τn)	O(κ¯τ)	O(1)	O(1)	O(1)	O(h)	✗
PPID3 [36]	O(κ)	O(p2¯τn)	O(κp2¯τn)	O(κp2¯τn)	O(1)	O(1)	O(1)	O(h)	✗
SID3 [37]	O(hp)	O(κ¯τ)	O(κ¯τn)	O(κ¯τ)	O(1)	O(1)	O(1)	O(h)	✗
OPPC4.5 [39]	O(κp)	O(p¯τ)	O(κ¯τ(n + p))	O(m¯τp)	O(1)	O(1)	O(1)	O(h)	✗
PivotRFs [62]	O(κp)	O(¯τ + τn)	O(κ¯τn)	O(κ¯τ)	O(p)	O(κ)	O(κ)	O(κ)	✓
MulPRFs [63]	O(h)	O(log n + log d)	O(hdn log n)	O(hdn log n)	O(h)	O(1)	O(h)	O(h)	✗
PPD-ERTs [64]	O(hp)	O(κ¯τ)	O(κ¯τn)	O(κ¯τ)	O(1)	O(1)	O(1)	O(h)	✗
HEldpRFs [51]	O(h)	O(κ¯τ)	O(κ¯τ)	O(κ¯τn)	O(1)	O(κ)	O(κ)	O(κ)	✓
SecureDT [65]	-	-	-	-	O(1)	O(1)	O(1)	O(κ)	✓
PrivateDT [66]	-	-	-	-	O(1)	O(1)	O(1)	O(κ)	✓
Our work	O(h)	O(κ¯)	O(κ)	O(κ¯τn)	O(1)	O(1)	O(1)	O(h)	✓
the anonymization [60] for random forests by grouping similar attributes so as to hardly identify specific individual information.
This work takes one step towards data encryption by incorporating some crucial ingredients of learning algorithm, and main contributions can be summarized as follows:
- We present a new encryption to preserve data's Gini impurity, and the basic idea is to modify the structure of binary search trees to maintain several samples on each node, and encrypt data's features by incorporating label and order information. Our scheme could change the data frequencies, which is also beneficial for data security.
- Theoretically, we prove the preservation of minimum Gini impurity in ciphertexts without decryption, which plays an important role on the construction of random forests. Our scheme also satisfies the security against Gini-impurity-preserving chosen plaintext attack.
- We focus on the privacy random forests in the popular client-server protocol, and take our Gini-impurity-preserving encryption for data features. We adopt homomorphic encryption CKKS to encrypt data labels. Our encrypted decision tree takes smaller communication and computational complexities, as shown in Table 1.
- Extensive experiments show that our encrypted random forests take significantly better performance than prior privacy random forests via encryption, anonymization and differential privacy, and are comparable to original (plaintexts) random forests without encryption. Our encrypted random forests make a good balance between computational cost and data security.
The rest of this work is constructed as follows: Section 2 introduces relevant work. Section 3 presents an encryption on data's Gini impurity. Section 4 proposes the encrypted random forests. Section 5 conducts extensive experiments. Section 6 concludes with future work.
Homomorphic Encryption (HE) is a cryptosystem, which allows operations on encrypted data without access to a secret key [40]. We can perform some mathematical operations such as addition and multiplication operations on encrypted data without revealing sensitive information. Given an encryption function  $E(\cdot)$  and a decryption function  $D(\cdot)$ , the HE scheme provides two operators  $\oplus$  and  $\otimes$  such that, for every pair of plaintexts  $x_{1}$  and  $x_{2}$ ,
$$
D \left(E \left(x _ {1}\right) \oplus E \left(x _ {2}\right)\right) = x _ {1} + x _ {2} \quad \text {a n d} \quad D \left(E \left(x _ {1}\right) \otimes E \left(x _ {2}\right)\right) = x _ {1} \times x _ {2},
$$
where  $+$  and  $\times$  denote standard addition and multiplication operations, respectively.
Various HE schemes have been developed during the past years, e.g., ElGamal [67], Paillier [68], CKKS [42] encryption, etc. Relevant techniques have been successfully applied to machine learning tasks such as regression problem [69, 70], neural network [71-75], collaborative filtering [76], etc. Generally, HE schemes are accompanied with high computational costs, and one main challenge is to maintain a good trade-off among security, effectiveness and computational cost in real applications.
Figure 1: A simple illustration for our encryption: each plaintext is encrypted into a ciphertext vector  $(c_{i},e_{i,j})$ . Here, random numbers  $c_{1} < c_{2} < \dots < c_{s}$  are introduced to preserve the Gini impurity for random forests, and we take homomorphic encryption scheme for  $e_{i,j} = \operatorname{Enc}(k_{\mathrm{pub}},j)$  in Eqn. (5), which is helpful for decryption.
Secure Multi-Party Computation (SMC) [77] is another cryptographic technique to jointly compute a function from multiple private inputs with confidential, which has been used for machine learning to protect privacy data, such as neural network [78-80],  $k$ -means clustering [81-83], random forests and decision trees [35-39], etc. Differential privacy is introduced to preserve individual privacy by taking statistically inconsequential changes to data [84], and relevant techniques have been utilized in neural network [85-87], random forests [30, 31] and decision trees [32-34].
We introduce some notations used in this work. Write  $[\tau] = \{1, 2, \dots, \tau\}$  for integer  $\tau \geq 2$ . Let  $\mathcal{X} \subset \mathbb{R}^d$  and  $\mathcal{Y} = [\tau]$  denote the feature and label space, respectively. A training sample is given by  $S_n = \{(\pmb{x}_1, y_1), (\pmb{x}_2, y_2), \dots, (\pmb{x}_n, y_n)\}$ . Let  $|A|$  be the cardinality of set  $A$ , and  $[\cdot, \cdot]$  denotes the corresponding encrypted value. Let  $\mathcal{N}(\mu, \sigma^2)$  be a normal distribution of mean  $\mu$  and variance  $\sigma^2$ .
3 An Encryption for Gini Impurity
This section presents the first encryption to preserve the minimum Gini impurity over encrypted data without decryption. For simplicity, we give the detailed encryption on one-dimensional feature by incorporating label information, and make similar considerations for other dimensions.
3.1 Theoretical Analysis for Gini Impurity
Let  $A = \{(a_{1},y_{1}),\dots ,(a_{n},y_{n})\}$  be a dataset with labels  $y_{i}\in [\tau ]$  , and define the Gini value as
$$
\operatorname {G i n i} (A) = 1 - \sum_ {y \in [ \tau ]} p _ {y} ^ {2},
$$
where  $p_y$  denotes the proportion of the label  $y$ . Let  $A_a^l = \{(a_i, y_i) : a_i \leq a, (a_i, y_i) \in A\}$  and  $A_a^r = \{(a_i, y_i) : a_i > a, (a_i, y_i) \in A\}$  be the left and right subsets of  $A$  w.r.t. a splitting point  $a$ , respectively. We define the Gini impurity w.r.t. dataset  $A$  and splitting point  $a$  as
$$
I _ {G} (A, a) = w _ {l} \cdot \operatorname {G i n i} \left(A _ {a} ^ {l}\right) + w _ {r} \cdot \operatorname {G i n i} \left(A _ {a} ^ {r}\right), \tag {1}
$$
where  $w_{l} = |A_{a}^{l}| / n$  and  $w_{r} = |A_{a}^{r}| / n$ . Let  $I_G^* (A)$  be the minimum Gini impurity of dataset  $A$ , i.e.,
$$
I _ {G} ^ {*} (A) = \min  _ {a \in \mathbb {R}} \left\{I _ {G} (A, a) \right\}. \tag {2}
$$
The minimum Gini impurity plays a crucial role on nodes splitting during the construction of random forests. We re-sort dataset  $A$  with a non-decreasing order for  $a_1, a_2, \dots, a_n$  as follows:
$$
A = \left\{\left(a _ {\langle 1 \rangle}, y _ {\langle 1 \rangle}\right), \left(a _ {\langle 2 \rangle}, y _ {\langle 2 \rangle}\right), \dots , \left(a _ {\langle n \rangle}, y _ {\langle n \rangle}\right) \right\} ， \tag {3}
$$
where  $a_{\langle 1\rangle}\leq a_{\langle 2\rangle}\leq \dots \leq a_{\langle n\rangle}$ , and  $y_{\langle 1\rangle},y_{\langle 2\rangle},\dots ,y_{\langle n\rangle}$  denote their corresponding labels. By incorporating label information, we partition dataset  $A$  into several datasets  $\mathcal{I}_1,\mathcal{I}_2,\dots ,\mathcal{I}_s$  as follows:
$$
\begin{array}{l} \mathcal {I} _ {1} = \left\{\left(a _ {\langle 1 \rangle}, y _ {\langle 1 \rangle}\right), \dots , \left(a _ {\langle k _ {1} \rangle}, y _ {\langle k _ {1} \rangle}\right) \right\}, \\ \mathcal {I} _ {2} = \left\{\left(a _ {\langle k _ {1} + 1 \rangle}, y _ {\langle k _ {1} + 1 \rangle}\right),, \dots , \left(a _ {\langle k _ {1} + k _ {2} \rangle}, y _ {\langle k _ {1} + k _ {2} \rangle}\right) \right\}, \tag {4} \\ \cdot \cdot \cdot \\ \end{array}
$$
$$
\mathcal {I} _ {s} = \left\{(a _ {\langle k _ {1} + k _ {2} + \dots + k _ {s - 1} + 1 \rangle}, y _ {\langle k _ {1} + k _ {2} + \dots + k _ {s - 1} + 1 \rangle}), \dots , (a _ {\langle n \rangle}, y _ {\langle n \rangle}) \right\}.
$$
Here, any two adjacent datasets have different labels, and all samples have an identical label in one dataset  $\mathcal{I}_j$ , i.e.,  $y_{\langle i\rangle} = y_{\langle i^{\prime}\rangle}$  for every  $(a_{\langle i\rangle},y_{\langle i\rangle})\in \mathcal{I}_j$  and  $(a_{\langle i^{\prime}\rangle},y_{\langle i^{\prime}\rangle})\in \mathcal{I}_j$ .
Algorithm 1 The Gini-impurity-preserving encryption
Input: Dataset  $A = \{(a_{1},y_{1}),\dots ,(a_{n},y_{n})\}$    
Output: Binary search tree  $\mathcal{BT}$  , ciphertexts  $\{\llbracket a_1\rrbracket ,\dots ,\llbracket a_n\rrbracket \}$    
Initialize: Tree  $\mathcal{BT} = \emptyset$  with its.  $c h i p e r_{1} = c_{\mathrm{max}} / 2$  , where  $c_{\mathrm{max}} = 2^{\lambda \log_2n}$    
for  $i = 1,\dots ,n$  do   
 $\% \%$  Step-I: Search a node for sample  $(a_i,y_i)$  in binary search tree  $\mathcal{BT}$    
Set  $t =$  root of  $\mathcal{BT}$ $t_\mathrm{min} = 0,t_\mathrm{max} = c_\mathrm{max}$  and index  $= 1$    
while  $t$  is an internal node and index  $= = 1$  do index  $= 0$  if t.left  $\neq \emptyset$  and  $a_i <   \max \{a_j:(a_j,y_j)\in t.$  left_samples} then  $t = t.$  left,  $t_\mathrm{max} = t.$  cipher1, index  $= 1$  else if t.right  $\neq \emptyset$  and  $a_i > \min \{a_j:(a_j,y_j)\in t.$  right_samples} then  $t = t.$  right,  $t_\mathrm{min} = t.$  cipher1, index  $= 1$  end if end while Update  $t = t.$  left if Eqn. (6) is true, and update  $t = t.$  right if Eqn. (7) is true   
 $\% \%$  Step-II: Update the binary search tree  $\mathcal{BT}$    
if  $y_{i}\neq y_{j}$  for some  $(a_j,y_j)\in t$  samples then Split node  $t$  by Algorithm 2 with inputs of  $(a_i,y_i)$  and the corresponding interval  $[t_{\mathrm{min}},t_{\mathrm{max}}]$  end if Append example  $(a_i,y_i)$  into t_samples and update tcipher2  $=$  Enc(kpub,|t_samples|) Encrypt  $\llbracket a_i\rrbracket = (\textit{t.cipher}_1,\textit{t.cipher}_2)$    
end for
We consider two important factors in encryption: i) preservation of the minimum Gini impurity  $I_G^*(A)$  over the encrypted data, and ii) a cryptosystem for encoding and decoding data. Based on such recognition, we introduce the following encryption, for every example  $(a_{\langle i \rangle}, y_{\langle i \rangle}) \in \mathcal{I}_j$ ,
$$
\llbracket a _ {\langle i \rangle} \rrbracket = \left(\llbracket a _ {\langle i \rangle} \rrbracket_ {1}, \llbracket a _ {\langle i \rangle} \rrbracket_ {2}\right) = \left\{ \begin{array}{l l} \left(c _ {1}, \operatorname {E n c} \left(k _ {\mathrm {p u b}}, i\right)\right) & \text {f o r} j = 1, \\ \left(c _ {j}, \operatorname {E n c} \left(k _ {\mathrm {p u b}}, i - k _ {1} - \dots - k _ {j - 1}\right)\right) & \text {f o r} 2 \leq j \leq s. \end{array} \right. \tag {5}
$$
Here,  $c_{1}, c_{2}, \dots, c_{s}$  are random numbers s.t.  $c_{1} < c_{2} < \dots < c_{s}$ , which aim to preserve the minimum Gini impurity. We take the homomorphic encryption scheme CKKS with a public key  $k_{\mathrm{pub}}$  for  $\left[[a_{\langle i\rangle}]\right]_2 = \operatorname{Enc}(k_{\mathrm{pub}}, i - k_1 - \dots - k_{j-1})$  in Eqn. (5), and it is useful for decryption. Figure 1 presents a simple illustration for our encryption, and the detailed decryption is given in Appendix A.
We now present our main theorem as follows:
Theorem 1. We have  $I_G^*(A) = I_G^*(A')$ , for re-sort dataset  $A$  by Eqn. (3) and for the corresponding encrypted dataset  $A' = \{([a_{\langle 1 \rangle}]_1, y_{\langle 1 \rangle}), \dots, ([a_{\langle n \rangle}]_1, y_{\langle n \rangle})\}$  from Eqns. (4)-(5).
This theorem shows that our encryption could preserve the minimum Gini impurity over encrypted data. The detailed proof is presented in Appendix B, which involves the proof of piecewise monotonicity of  $I_G(A,a)$  w.r.t. splitting point  $a$ , and then solves the minimum splitting point on plaintexts, as well as the corresponding point on encrypted data.
3.2 Binary Search Tree for Encryption
We now present new binary search tree to encrypt  $a_1, \dots, a_n$  dynamically, especially for un-ordered dataset  $A = \{(a_1, y_1), \dots, (a_n, y_n)\}$ , or when example  $(a_i, y_i)$  arrives in a streaming data. We begin with an alternative structure for binary search tree to maintain several samples on a node from Eqns. (4)-(5), rather than previous only one sample [88, 89]. Our new structure is given by
Struct Tree {Plaintext samples; Ciphertext  $\text{cipher}_1$ ,  $\text{cipher}_2$ ; Tree left, right}.
The samples stores one or multiple samples from  $A$ , and  $\text{cipher}_1$  and  $\text{cipher}_2$  are the first and second ciphertext in Eqn. (5), and left and right denote left and right child of the current node, respectively.
We initialize an empty tree  $\mathcal{BT} = \emptyset$  and set its  $c_{\text{ipher}_1} = c_{\max} / 2$  with  $c_{\max} = 2^{\lambda \log_2 n}$ , and then we construct binary search tree iteratively. We maintain an interval  $[t_{\min}, t_{\max}]$  in each iteration so as
Algorithm 2 Splitting a node for encryption
Input: Example  $(a_{i},y_{i})$  , node  $t$  of binary search tree  $\mathcal{BT}$  , and interval  $[t_{\mathrm{min}},t_{\mathrm{max}}]$    
Output: Updated node  $t$    
Initialize an empty node  $l$  with  $l$  .samples  $\equiv \{(a_j,y_j)\in t$  .samples:  $a_{j} <   a_{i}\}$    
if  $l$  .samples  $\neq \emptyset$  then if  $t$  .left  $\neq \emptyset$  then Set  $l$  .cipher1 according to Eqn. (8), and update  $l$  .left  $= t$  .left,  $t$  .left  $= l$  else Set  $l$  .cipher1 according to Eqn. (9), and update  $t$  .left  $= l$  end if   
end if   
Initialize an empty node  $r$  with  $r$  .samples  $\equiv \{(a_j,y_j)\in t$  .samples:  $a_{j} > a_{i}\}$    
if  $r$  samples  $\neq \emptyset$  then if  $t$  right  $\neq \emptyset$  then Set  $r$  .cipher1 according to Eqn. (10), and update  $r$  .right  $= t$  .right,  $t$  .right  $= r$  else Set  $r$  .cipher1 according to Eqn. (11), and update  $t$  .right  $= r$  end if   
end if   
Update  $t$  samples  $= t$  samples  $\backslash l$  samples  $\backslash r$  samples
to keep the increasing order of ciphertexts  $c_{1}, c_{2}, \dots, c_{s}$  in Eqn. (5). During the  $i$ -th iteration, we receive a sample  $(a_{i}, y_{i})$ , and then take two steps as follows:
Step-I: Search a node for sample  $(a_i, y_i)$  in binary search tree  $\mathcal{BT}$
Let  $t$  be a node pointer with the initialization of the root of  $\mathcal{BT}$ . We search a path downward in  $\mathcal{BT}$  by comparing with  $a_i$ , and the search will terminate when  $t$  is a leaf node or an empty node.
For an internal node  $t$ , the search continues to its left child and updates  $t_{\max} = t.cipher_1$  if
the left child  $t.$  left  $\neq \emptyset$  and  $a_{i} < \max \{a_{j}:(a_{j},y_{j})\in t.$  left_samples  $\}$
and the search continues to its right child and updates  $t_{\mathrm{min}} = t \cdot \text{cipher}_1$  if
the right child  $t$ .right ≠ ∅ and  $a_i > \min \{a_j : (a_j, y_j) \in t \text{.right_samples}\}$ ;
otherwise, the search terminates. This procedure can be easily implemented with a while loop.
It is necessary to consider two special cases after the above search. We update  $t = t.left$  if
$t.\text{left} \neq \emptyset, a_{i} < \min \{a_{j} : (a_{j}, y_{j}) \in t.\text{samples}\}$  and  $y_{i} = y_{j}$  for all  $(a_{j}, y_{j}) \in t.\text{left.sample}$ . (6)
In a similar manner, we update  $t = t$ .right if
$t.\text{right} \neq \emptyset$ ,  $a_i > \max \{a_j : (a_j, y_j) \in t.\text{samples}\}$  and  $y_i = y_j$  for all  $(a_j, y_j) \in t.\text{right.sample}$ . (7)
Step-II: Update the binary search tree  $\mathcal{BT}$
After Step-I, we could find a node  $t$  for sample  $(a_i, y_i)$  and the corresponding interval  $[t_{\min}, t_{\max}]$ . We directly append the example  $(a_i, y_i)$  into  $t$ . samples if  $y_i = y_j$  for every  $(a_j, y_j) \in t$ . samples; otherwise, it is necessary to split the node  $t$  according to  $a_i$ .
We initialize an empty node  $l$  with  $l$ .samples  $= \{(a_{j},y_{j})\in t$ .samples:  $a_{j} < a_{i}\}$ , and it is sufficient to consider  $l$ .samples  $\neq \emptyset$ . If  $t$ .left  $\neq \emptyset$ , then we set
$l.\mathrm{c�pher}_1 = (t.\mathrm{left.c�pher}_1 + t.\mathrm{c�pher}_1) / 2 + \xi$  s.t.  $t.\mathrm{left.cipher}_1 <   l.\mathrm{cipher}_1 <   t.\mathrm{cipher}_1$  ，(8)
and update  $l \cdot left = t \cdot left, t \cdot left = l$ ; otherwise, we set
$l.\mathrm{c�pher}_1 = (t_{\mathrm{min}} + t.\mathrm{c�pher}_1) / 2 + \xi$  s.t.  $l.\mathrm{c�pher}_1\in (t_{\mathrm{min}},t.\mathrm{c�pher}_1)$  ，(9)
and update  $t$ .left = l. Here,  $\xi$  is a random number sampled from  $\mathcal{N}(0,1)$ , and notice that we may randomly sample  $\xi$  multiple times so that the condition holds in Eqns (8)-(9), respectively.
Algorithm 3 Finding the best splitting feature and position
Input: Encrypted datasets  $\llbracket S_n^t\rrbracket$  , available splitting feature and position  $\llbracket s\rrbracket_{i = 1}^{j}$  , and secret key  $k_{\mathrm{sec}}$    
Output: index  $i^{*}$ $\% \%$  Server: for  $i\in [j]$  do Calculate Gini impurity  $I_G(\llbracket S_n^t\rrbracket ,\llbracket s\rrbracket_i)$  from Eqn. (12) w.r.t splitting feature and position  $\llbracket s\rrbracket_{i}$  end for Send ciphertexts  $\{I_G(\llbracket S_n^t\rrbracket ,\llbracket s\rrbracket_i)\}_{i\in [j]}$  to the client   
 $\% \%$  Client: Get the decrypted  $\{\mathrm{Dec}(k_{\mathrm{sec}},I_G(\llbracket S_n^t\rrbracket ,\llbracket s\rrbracket_i))\}_{i\in [j]}$  Set  $i^{*} = -1$  if  $\mathrm{Dec}(k_{\mathrm{sec}},I_G(\llbracket S_n^t\rrbracket ,\llbracket s\rrbracket_i)) = 0$  for every  $i\in [j]$  ; otherwise, set  $i^{*}$  by Eqn. (13) Send  $i^{*}$  to the server
We make similar update for the right child of node  $t$ : initialize an empty node  $r$  with  $r$ .samples =  $\{(a_j, y_j) \in t$ .samples:  $a_j > a_i\}$ , and consider  $r$ .samples ≠ ∅. If  $t$ .right ≠ ∅, then we set
$r.\mathrm{cipher}_1 = (t.\mathrm{cipher}_1 + t.\mathrm{right.cipher}_1) / 2 + \xi$  s.t.  $t.\mathrm{cipher}_1 < r.\mathrm{cipher}_1 < t.\mathrm{right.cipher}_1$ , (10) and update  $r.\mathrm{right} = t.\mathrm{right}$ ,  $t.\mathrm{right} = r$ ; otherwise, we set
$$
r. \text {c i p h e r} _ {1} = \left(t. \text {c i p h e r} _ {1} + t _ {\max }\right) / 2 + \xi \quad \text {s . t .} \quad r. \text {c i p h e r} _ {1} \in \left(t. \text {c i p h e r} _ {1}, t _ {\max }\right), \tag {11}
$$
and update  $t$ .right = r. Algorithm 2 presents the detailed descriptions on the splitting of node  $t$ .
Algorithm 1 presents an overview of our Gini-impurity-preserving encryption, and the decryption is given in Appendix A. Our scheme does not only keep the minimum Gini impurity, but also change frequencies to prevent decryption from frequencies, which is also beneficial for encryption [90]. Our scheme takes an average of  $O(n \log n)$  computational complexity, since it requires  $O(\log n)$  and  $O(1)$  computational complexities to search and update a node in each iteration, respectively. Finally, the average and worst space complexities are  $O(\log n)$  and  $O(n)$  for our encryption, respectively.
3.3 Security Analysis
For ciphertext vector  $\llbracket a\rrbracket = (\llbracket a\rrbracket_1,\llbracket a\rrbracket_2)$  in Eqn. (5), it suffices to discuss the first ciphertext  $\llbracket a\rrbracket_{1}$  since the security of  $\llbracket a\rrbracket_{2}$  has been analyzed in homomorphic encryption CKKS [42]. Following semantic security against chosen plaintext attacks [89, 91], we define a security game GameGIPCPA:
- An adversary chooses two sequences with distinct plaintexts  $\{a_1^0, \dots, a_n^0\}$  and  $\{a_1^1, \dots, a_n^1\}$ , and sends them to a challenger;
- The challenger flips an unbiased coin  $b \in \{0,1\}$  to select  $\{a_1^b, \dots, a_n^b\}$ , and randomly sets their corresponding labels  $\{y_1^b, \dots, y_n^b\}$  with each  $y_i^b$  drawn independently and uniformly over  $[\tau]$ . The challenger encrypts  $\{a_1^b, \dots, a_n^b\}$  by Eqns. (4) and (5), and sends the ciphertexts to the adversary;
- The adversary outputs a guess of  $b$ , i.e., which sequence is selected for encryption.