SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We then introduce the security against Gini-impurity-preserving chosen plaintext attack as follows.
Definition 2. A scheme is said to be indistinguishable under Gini-impurity-preserving chosen plaintext attack if the probability of outputs with the correct guess  $b$  is negligible for the adversary  $\mathcal{A}$  in GameGIPCPA, that is,
$$
\Pr [ A (\text {G a m e} _ {\mathrm {G I P C P A}}) = b ] <   1 / 2 + \text {s m a l l c o n s t a n t}.
$$
The following theorem shows that our encrypted plaintexts sequences are indistinguishable.
Theorem 3. Our scheme for the first ciphertexts  $\llbracket a_1\rrbracket_1,\llbracket a_2\rrbracket_1,\dots ,\llbracket a_n\rrbracket_1$  in Section 3.2 is security against Gini-impurity-preserving chosen plaintext attack.
The detailed proof is presented in Appendix C, and the basic idea is inspired from [88]. We take induction on  $n$  to show that data point  $(a_{i + 1}^b, y_{i + 1})$  affects the constructed binary search trees with the same probability as  $b = 0$  and  $b = 1$ , and then the ciphertexts of data points  $(a_{i + 1}^b, y_{i + 1})$  also follow the same distribution, i.e.,
$$
P \left(\llbracket a _ {1} ^ {0} \rrbracket , \dots , \llbracket a _ {i + 1} ^ {0} \rrbracket | a _ {1} ^ {0}, \dots , a _ {i + 1} ^ {0}\right) = P \left(\llbracket a _ {1} ^ {1} \rrbracket , \dots , \llbracket a _ {i + 1} ^ {1} \rrbracket | a _ {1} ^ {1}, \dots , a _ {i + 1} ^ {1}\right).
$$
Table 2: Datasets
Datasets	#Inst	#Feat	Datasets	#Inst	#Feat	Datasets	#Inst	#Feat	Datasets	#Inst	#Feat
wdbc	569	30	adver	3,279	1,558	ailerons	13,750	41	adult	48,842	14
cancer	569	31	bibtex	7,396	1,836	house	22,784	16	mnist	70,000	780
breast	699	9	phpB0	7,797	617	a9a	32,563	123	miniboone	72,998	51
diabetes	768	8	pendigits	10,992	16	amazon	32,769	9	runwalk	88,588	6
german	1,000	24	phish	11,055	30	bank	45,211	17	covtype	581,012	54
4 Encrypted Random Forests
For encrypted random forests, we follow the popular client-server protocols [51, 65, 66, 88]. A client encrypts training and testing data, and transfers encrypted data to an honest-but-curious server. The server trains random forests from the encrypted data with the aid of client, and finally returns predictions on encrypted testing data.
Encryption for training and testing datasets
Recall training data  $S_{n} = \{(x_{1},y_{1}),\dots ,(x_{n},y_{n})\}$  with  $\pmb {x}_i = (x_{i,1},\dots ,x_{i,d})$ . The client constructs  $d$  binary search trees  $\mathcal{BT}_1,\mathcal{BT}_2,\dots ,\mathcal{BT}_d$  according to Algorithm 1 over different dimensional features and labels in  $S_{n}$ , where  $\mathcal{BT}_j$  is used to encrypt features  $\{x_{1,j},\dots ,x_{n,j}\}$  for  $j\in [d]$ .
We take the homomorphic encryption CKKS [42] to encrypt training labels  $y_{1}, \dots, y_{n}$ . Each label  $y_{i}$  is encoded with a vector of size  $\tau$  by one-hot method, and we encrypt the vector by homomorphic encryption CKKS with a public key  $k_{pub}$ . The ciphertexts  $[[y_{i}]] = [[[y_{i,1}], \dots, [[y_{i,\tau}]]]$  is given by
$$
[   [ y _ {i, j} ]   ] = \left\{ \begin{array}{l l} \operatorname {E n c} (k _ {\mathrm {p u b}}, 1) & \text {f o r} j = y _ {i}, \\ \operatorname {E n c} (k _ {\mathrm {p u b}}, 0) & \text {o t h e r w i s e}. \end{array} \right.
$$
We obtain the final training data  $\llbracket S_n\rrbracket = \{([\pmb {x}_1],[\pmb {y}_1]),\dots ,([\pmb {x}_n],[\pmb {y}_n])\}$
Let  $\tilde{S}_{n'} = \{\tilde{x}_1, \dots, \tilde{x}_{n'}\}$  be a testing data with instance  $\tilde{x}_i = (\tilde{x}_{i,1}, \dots, \tilde{x}_{i,d})$ . For every plaintext  $\tilde{x}_{i,j}$  with  $i \in [n']$  and  $j \in [d]$ , we search a node  $t$  in the binary search tree  $\mathcal{BT}_j$ , similarly to the node search (Step-I) in Section 3.2, and obtain its ciphertext  $[\tilde{x}_{i,j}]] = [t.cipher, Enc(k_{pub}, i)]$ . We have the encrypted testing data  $[\tilde{S}_{n'}] = \{[[\tilde{x}_1]], \dots, [[\tilde{x}_{n'}]]\}$ .
Construction on encrypted random forests
Encrypted random forests consist of individual decision trees  $\mathcal{DT}_1, \dots, \mathcal{DT}_m$ , where each tree  $\mathcal{DT}_i$  is constructed as follows. We first take a bootstrap sample  $\llbracket S_n'\rrbracket$  from  $\llbracket S_n\rrbracket$ , and initialize  $\mathcal{DT}_i$  with one node of data  $\llbracket S_n'\rrbracket$ . We repeat the following procedure recursively for each leaf node, until the number of training samples is smaller than  $\alpha$ , or all instances have the same label in the leaf node:
- Select a  $k$ -subset B from  $d$  available features randomly without replacement;
- Find the best splitting feature in B and position by Gini impurity from the encrypted data;
- Split the current node into left and right children via the best splitting position and feature.
Such construction is essentially similar to original random forests [1], whereas we require a different way to find the best splitting feature and position based on Gini impurity from the encrypted data.
Let  $t$  be the current leaf node for further splitting with the encrypted training data  $[[S_n^t]] \subseteq [[S_n]]$ , and  $[[s]]_1, \dots, [[s]]_j$  denote all possible splitting features and positions in the scope of the corresponding feature subset  $B$  from  $[[S_n^t]]$ . Here, the information of feature and position can be derived from the corresponding index  $i \in [j]$  and subset  $B$ .
For each  $i \in [\jmath]$ , the server partitions the current encrypted training data  $\llbracket S_n^t\rrbracket$  into left and right subsets, i.e.,  $\llbracket S_n^t\rrbracket_i^l$  and  $\llbracket S_n^t\rrbracket_i^r$ , according to the splitting feature and position  $\llbracket s\rrbracket_i$ . Let  $n_l$  and  $n_r$  be the number of training examples in  $\llbracket S_n^t\rrbracket_i^l$  and  $\llbracket S_n^t\rrbracket_i^r$ , respectively, and denote by
$$
\llbracket S _ {n} ^ {t} \rrbracket_ {i} ^ {l} = \left\{(\llbracket \boldsymbol {x} _ {1} ^ {l} \rrbracket , \llbracket y _ {1} ^ {l} \rrbracket), \dots , (\llbracket \boldsymbol {x} _ {n _ {l}} ^ {l} \rrbracket , \llbracket y _ {n _ {l}} ^ {l} \rrbracket) \right\} \quad \text {a n d} \quad \llbracket S _ {n} ^ {t} \rrbracket_ {i} ^ {r} = \left\{(\llbracket \boldsymbol {x} _ {1} ^ {r} \rrbracket , \llbracket y _ {1} ^ {r} \rrbracket), \dots , (\llbracket \boldsymbol {x} _ {n _ {r}} ^ {r} \rrbracket , \llbracket y _ {n _ {r}} ^ {r} \rrbracket) \right\}.
$$
From Eqn. (1), we have Gini impurity
$$
I _ {G} \left(\llbracket S _ {n} ^ {t} \rrbracket , \llbracket s \rrbracket_ {i}\right) = \left[ \frac {n _ {l}}{n _ {l} + n _ {r}} \otimes I _ {G} \left(\llbracket S _ {n} ^ {t} \rrbracket_ {i} ^ {l}\right) \right] \oplus \left[ \frac {n _ {r}}{n _ {l} + n _ {r}} \otimes I _ {G} \left(\llbracket S _ {n} ^ {t} \rrbracket_ {i} ^ {r}\right) \right], \tag {12}
$$
Table 3: Comparisons of prediction accuracies (mean±std).  $\bullet / \circ$  indicates that our encrypted random forests are significantly better/worse than other compared random forests (pairwise  $t$ -tests at  $95\%$  significance level). 'NA' means that no results were obtained after running out  $10^{6}$  seconds (about 11.6 days).
Dataset	Our encrypted RFs	Original RFs	AnonyRFs	DiffPrivRFs	PPD-ERTs	PivotRFs	MulPRFs	HEldpRFs
wdbc	.9525±.0141	.9617±.0018	.9091±.0205●	.8998±.0024●	.9222±.0037●	.9609±.0101	.9510±.0114	.9195±.0029●
cancer	.9766±.0082	.9824±.0143	.9271±.0016●	.9034±.0578●	.9600±.0022●	.9510±.0130●	.9656±.0102	.9823±.0024
breast	.9855±.0012	.9881±.0011	.9657±.0021●	.9271±.0515●	.9678±.0129●	.9806±.0086	.9769±.0107	.9275±.0023●
german	.7939±.0124	.8033±.0205	.7300±.0214●	.7400±.0141●	.7610±.0168●	.7533±.0122●	.7823±.0154	.7043±.0027●
diabetes	.7641±.0093	.7677±.0309	.7193±.0023●	.7328±.0124●	.7448±.0193	.7419±.0061●	.7611±.0035	.7478±.0193●
ader	.9851±.0011	.9888±.0014	.9278±.0018●	.9390±.0051●	NA	.9664±.0043●	NA	NA
bibtex	.7907±.0054	.7749±.0027●	.7425±.0009●	.7200±.0130●	NA	.7461±.0193●	NA	NA
phpB0	.9380±.0024	.9585±.0043○	.8641±.0009●	.8920±.0031●	NA	NA	NA	NA
pendigits	.9917±.0024	.9906±.0016	.9072±.0104●	.9154±.0126●	.9639±.0048●	.9070±.0130●	NA	NA
phish	.9798±.0026	.9716±.0018	.9032±.0014●	.9318±.0089●	.9555±.0125●	.9454±.0067●	.9401±.0102●	NA
ailerons	.8795±.0027	.8819±.0015	.8104±.0105●	.8322±.0091●	.8589±.0043●	.8571±.0082●	.8766±.0025	NA
house	.8794±.0007	.8913±.0039○	.8255±.0011●	.8475±.0025●	.8541±.0149●	.8508±.0016●	.8742±.0023	NA
a9a	.8321±.0011	.8303±.0012	.8046±.0027●	.7909±.0084●	.8345±.0144	.8314±.0071	.8051±.0102●	NA
amazon	.9491±.0109	.9478±.0060	.9193±.0024●	.9104±.0035●	.9221±.0024●	.9401±.0128	.9400±.0032	NA
bank	.8992±.0118	.9029±.0104	.8499±.0089●	.8517±.0064●	.8940±.0147	.8940±.0091	.8827±.0108	NA
adult	.8663±.0019	.8691±.0018	.8206±.0032●	.8355±.0053●	.8452±.0106●	.8243±.0076●	.8594±.0103	NA
mnist	.9674±.0105	.9763±.0101	.9362±.0006●	.9059±.0157●	NA	NA	NA	NA
miniboone	.9497±.0018	.9518±.0013	.8977±.0101●	.9111±.0104●	.9301±.00021●	.9501±.0011	NA	NA
runwalk	.9784±.0014	.9798±.0032	.9523±.0024●	.9401±.0040●	.9572±.0074●	.9511±.0071●	NA	NA
covtype	.9787±.0042	.9650±.0104●	.9112±.0015●	.9407±.0018●	.9569±.0134●	NA	NA	NA
win/tie/loss	2/16/2	20/0/0	20/0/0	17/3/0	14/6/0	10/10/0	19/1/0
where  $I_G([\mathbb{S}_n^t ]_i^l) = 1\ominus p_l\odot p_l$  and  $I_{G}([\mathbb{S}_{n}^{t}]_{i}^{r}) = 1\ominus p_{r}\odot p_{r}$ , with
$$
p _ {l} = \left(1 / n _ {l}\right) \otimes \left(\llbracket y _ {1} ^ {l} \rrbracket \oplus , \dots , \oplus \llbracket y _ {n _ {l}} ^ {l} \rrbracket\right) \text {a n d} p _ {r} = \left(1 / n _ {r}\right) \otimes \left(\llbracket y _ {1} ^ {r} \rrbracket \oplus , \dots , \oplus \llbracket y _ {n _ {r}} ^ {r} \rrbracket\right).
$$
Here,  $\otimes$ ,  $\odot$ ,  $\oplus$  and  $\ominus$  denote the CKKS element-wise homomorphic multiplication, dot, addition and subtraction functions, respectively, as in the work of [42].
The client gets plaintexts  $\{\mathrm{Dec}(k_{\mathrm{sec}},I_G([\mathbb{S}_n^t ]),[[s]]_i))\}_{i = 1}^J$  by decrypting with the secret key  $k_{\mathrm{sec}}$ , when the server sends ciphertexts  $\{I_G([\mathbb{S}_n^t ]),[[s]]_i)\}_{i = 1}^J$ . If all instances have the same label in  $[\mathbb{S}_n^t ]$ , then we have  $\mathrm{Dec}(k_{\mathrm{sec}},I_G([\mathbb{S}_n^t ]),[[s]]_i)) = 0$  for each  $i\in [j]$ , and we set  $i^{*} = -1$ ; otherwise, we set  $i^{*}$  as
$$
i ^ {*} \in \arg \min  _ {i \in [ j ]} \left\{\operatorname {D e c} \left(k _ {\sec}, I _ {G} \left(\llbracket S _ {n} ^ {t} \rrbracket , \llbracket s \rrbracket_ {i}\right)\right) \right\}. \tag {13}
$$
The client sends index  $i^*$  to the server for further splitting. Algorithm 3 presents the detailed descriptions on finding the best splitting feature and position.
For encrypted decision tree, the client requires the  $O(\kappa)$  computational complexity with  $\kappa$  leaves nodes, since the client performs constant basic operations for each node. The server takes the  $O(\kappa\overline{j}\tau n)$  computational complexity for Eqn. (12), where  $\overline{j}$  is an average of number of possible splitting features and positions, and  $\tau$  and  $n$  are the number of labels and training examples, respectively.
Our method takes  $O(h)$  communication rounds of  $O(\kappa \bar{j})$  communication bandwidth to train an encrypted decision tree of height  $h$ . This is because we consider the breadth-first search and aggregate all nodes in the same height and send to the client with a single message at one time.
We do not require bootstrapping for homomorphic encryption in 3-depth homomorphic multiplicative, since we independently compute the splitting feature and position for each node from Eqn. (12). This is different from previous encrypted decision trees [51, 62], which could take expensive computational complexity for bootstrapping [40, 92].
Prediction on encrypted testing dataset
After getting decision trees  $\mathcal{DT}_1, \dots, \mathcal{DT}_m$ , we predict label  $[\tilde{y}_i] = \mathcal{DT}_1([\tilde{x}_i]) \oplus \dots \oplus \mathcal{DT}_m([\tilde{x}_i])$  for test instance  $[\tilde{x}_i] \in [\tilde{S}_{n'}]$ . The server sends ciphertexts  $\{\tilde{y}_1, \dots, \tilde{y}_{n'}\}$  to the client, and the client decrypts those ciphertexts, and gets the final plaintext label by  $\tilde{y}_i = \arg \max_{j \in [\tau]} \{\mathrm{Dec}[\tilde{y}_{i,j}]\}$ .
During such prediction process, the server requires the  $O(h)$  computational complexity, since we search from the root to leaf node of tree. The client takes  $O(1)$  rounds of communication and communication bandwidth to transfer the testing data and predicting ciphertext without interaction.
Figure 2: Comparisons of training running time on different random forests. Notice that the y-axis is in log-scale, and full black columns imply that no result was obtained after running out  $10^{6}$  seconds (about 11.6 days).
We conduct experiments on 20 datasets<sup>2</sup> as summarized in Table 2. Most datasets have been well-studied in previous random forests. In addition to the original (plaintexts) random forests [1], we compare with six state-of-the-art privacy-preserving random forests in recent years.
- AnonyRFs: random forests based on anonymization with a top-down greedy search [59];
- DiffPrivRFs: random forests based on differential privacy [93];
- PPD-ERTs: extremely randomized trees from distributed structured data [64];
- PivotRFs: random forests based on a hybrid of threshold partially homomorphic encryption and secure multiparty computation techniques [62];
- MulPRFs: random forests based on the secure multiparty computation [94];
- HEldpRFs: random forests with fully homomorphic encryption and low-degree polynomial approximations [51].
For all random forests, we train 100 individual decision trees, and randomly select  $\lfloor \sqrt{d} \rfloor$  candidate features during node splitting. We set  $\alpha = 10$  for datasets of size smaller than 20,000 for our encrypted random forests; otherwise, set  $\alpha = 100$ , following [95]. For multi-class datasets, we take the one-vs-all method for MulPRFs, since it is limited to binary classification. Other parameters are set according to their respective references, and more details can be found in Appendix D.
Experimental comparisons
The performance is evaluated by five trials of 5-fold cross validation, and final prediction accuracies are obtained by averaging over these 25 runs, as summarized in Table 3. It is evident that our encrypted random forests take comparable performance with original random forests [1] on plaintexts, which nicely supports our Theorem 1 on the preservation of minimum Gini impurity in the construction of random forests. Our encrypted random forests are also comparable to MulPRFs if they can obtain results within  $10^{6}$  seconds (about 11.6 days), since MulPRFs are essentially similar to original random forests, yet with different implementation of secure multi-party computation.
As can be seen from Table 3, our random forests take significantly better performance than AnonyRFs and DiffPrivRFs, since the win/tie/loss counts show that our random forests win for most times and never lose. This is because AnonyRFs combine features by anonymization, while DiffPrivRFs add perturbations to features via differential privacy, therefore, both of them cause information lost in privacy process. Our random forests also achieve better performance than PivotRFs, since PivotRFs have to limit trees' depth for random forests due to heavy computations for HE and communications for secure multi-party computation.
Our random forests also outperform PPD-ERTs and HEldpRFs if results are obtained in  $10^{6}$  seconds, since PPD-ERTs adopt completely-random splitting, rather than selecting the minimum Gini impurity, while HEldpRFs take homomorphic encryption on features and employ low-degree polynomial approximation. Those approaches have modified the structures of original random forests.
2Downloaded from www.openml.org
Figure 3: Security comparisons for different schemes: the more red the area, the higher the security.
All experiments are performed by  $c++$  on the Ubuntu with 256GB main memory (AMD Ryzen Threadripper 3970X). We compare the training running time of our encrypted random forests and others, and the average CPU time (in seconds) is shown in Figure 2.
As expected, original random forests take the least running time over raw datasets without privacy preservation. Our encrypted random forests take larger running time than AnonyRFs and DiffPrivRFs because they are essentially similar to original random forests, yet with some simple modifications or perturbations on features. Our encrypted random forests take better performance and higher security.
Our encrypted random forests take smaller running time than PPD-ERTs, PivotRFs, MulPRFs and HEldpRFs, in particular for large datasets or high-dimensional datasets, where no results are obtained even after running out  $10^{6}$  seconds (almost 11.6 days). Because PPD-ERTs, PivotRFs and MulPRFs require expensive communication cost for multi-parity computation, while PivotRFs and HEldpRFs take heavy computation costs on HE scheme.
We present security analysis for the first ciphertext  $\llbracket a\rrbracket_1$  in ciphertext vector  $\llbracket a\rrbracket = (\llbracket a\rrbracket_1,\llbracket a\rrbracket_2)$ , and the second ciphertext  $\llbracket a\rrbracket_2$  can be ensured by HE scheme. We compare with four state-of-the-art encryptions: differential privacy [93], anonymization [59], order-preserving scheme [96] and HE scheme [42]. Here, we present results of six datasets and randomly selecting one feature, and trends are similar on other dimensions and datasets. More results can be found in Appendix D.
Figure 3 shows the comparison results, and we take the bitwise leakage matrices to measure the security as in [97]: the more red the area, the higher the security. As expected, HE scheme presents the highest security, yet with heavy computational costs, for example, no results are obtained for datasets of size exceeding 3000 even after running out  $10^{6}$  seconds. It is also observed that our scheme presents higher security than the other three schemes, since those schemes simply present perturbations, compression or preserve the entire order information regardless of learning ingredients. In comparison, our scheme could make a good balance between security and computational cost.
This work takes one step on data encryption from some crucial ingredients of learning algorithm. We present a new encryption to preserve data's Gini impurity, which plays a crucial role during the construction of random forests. For random forests, we encrypt data features based on our Gini-impurity-preserving scheme, and take the homomorphic encryption scheme CKKS to encrypt data labels. Both theoretically and empirically, we validate the effectiveness, efficiency and security of our proposed method. An interesting work is to exploit other learning ingredients, such as gini index and information gain, for data encryption in the future.
The authors want to thank the reviewers for their helpful comments and suggestions. This research was supported by National Key R&D Program of China (2021ZD0112802), NSFC (61921006, 62376119), CAAI-Huawei MindSpore Open Fund, and Fundamental Research Funds for the Central Universities (2023300246). W. Gao is the corresponding author of this paper.
Algorithm 4 Decryption   
Input: Tree node t of  $\mathcal{BT}$  , ciphertext  $[a_i]]$    
Output: plaintext  $a_{i}$  while  $[a_i]_1\neq t.$  cipher1 do if  $[a_i]_1 > t.$  cipher1 then  $t = t$  .right else if  $[a_i]_1 <   t.$  cipher1 then  $t = t$  .left end if end while   
Return  $a_{i} = t$  samples[Dec  $(k_{\mathrm{sec}},[[a_i]]_2])$
A Detailed Decryption for Our Encryption Method
A.1 Decryption for Our Encryption in Section 3.1
We present the decryption for ciphertext  $\llbracket a_i\rrbracket = (\llbracket a_i\rrbracket_1,\llbracket a_i\rrbracket_2)$  in Eqn. (5) by the following steps:
Find the partition  $\mathcal{I}_j$  according to  $[a_i]_1$ ;
- Decrypt ciphertext  $\llbracket a_i\rrbracket_2$  by the CKKS secret key  $k_{\mathrm{sec}}$ , and get index  $\tau = \mathrm{Dec}(k_{\mathrm{sec}},\llbracket a_i\rrbracket_2)$  in partition  $\mathcal{I}_j$ .
- Obtain the plaintext  $a_i$  as the  $\tau$ -th sample in partition  $\mathcal{I}_j$ .
A.2 Decryption for Our Encryption of Binary Search Tree in Section 3.2
We decrypt a ciphertext  $\llbracket a_i\rrbracket = (\llbracket a_i\rrbracket_1,\llbracket a_i\rrbracket_2)$  based on binary search tree  $\mathcal{BT}$  (in Section 3.2) and the CKKS secret key  $k_{\mathrm{sec}}$  by the following two steps, and Algorithm 4 presents the details of decryption:
- Let  $t$  be a node pointer with the initialization of the root of binary search tree  $\mathcal{BT}$ . We then search a path downward in  $\mathcal{BT}$  by comparing with  $[[a_i]]_1$ . The search continues to its left child if  $[[a_i]]_1 < t.cipher_1$  and update  $t = t.left$ ; the search continues to its right child if  $[[a_i]]_1 > t.cipher_1$  and update  $t = t.right$  until  $[[a_i]]_1 = t.cipher_1$ .
- Decrypt ciphertext  $\llbracket a_i\rrbracket_2$  by the CKKS secret key  $k_{\mathrm{sec}}$ , and get index  $\tau = \mathrm{Dec}(k_{\mathrm{sec}},[[a_i]]_2)$  in t_samples. Then we use the index  $\tau$  to get the plaintext  $a_i = t.\text{samples}[\tau]$ .
A.3 Formal Definition of Our Gini-impurity-preserving Encryption
We present a formal definition of our Gini-impurity preserving encryption as follows:
-  $S \gets \operatorname{KeyGen}(t_{\max})$ : Generate the secret state  $S$  by initializing binary search tree  $\mathcal{BT} = \emptyset$ , and a security parameter  $c_{\max}$ , which is a random number with  $c_{\max} > n$ . We maintain an interval  $[t_{\min}, t_{\max}]$  in each secret state  $S$  with  $t_{\min} = 0$  and  $t_{\max} = c_{\max}$  in the initial stage, so as to keep the order of ciphertexts  $c_1, c_2, \dots, c_s$  in Eqn. (5). In this way, the ciphertexts are random numbers with semi-order of plaintexts, and we have different ciphertext even for the same plaintexts.
-  $S', \llbracket a_i \rrbracket \gets \text{Encrypt}(S, a_i)$ : Encrypt  $a_i$  and update the secret state to  $S'$  as for receiving a sample  $(a_i, y_i)$  as follows:
- Search a node for sample  $(a_i, y_i)$  in binary search tree  $\mathcal{BT}$  as shown in Algorithm 1. Let  $t$  be a node pointer with the initialization of the root of  $\mathcal{BT}$ . We search a path downward in  $\mathcal{BT}$  by comparing with  $a_i$ . The search will terminate when  $t$  is a leaf or an empty node.
- Update the binary search tree  $\mathcal{BT}$ . We directly append the example  $(a_i, y_i)$  into  $t$ .samples if  $y_i = y_j$  for every  $(a_j, y_j) \in t$ .samples; otherwise, it is necessary to split the node  $t$  according to  $a_i$ . Algorithm 2 presents the detailed descriptions on the splitting of node  $t$ .
- Compute ciphertext  $\llbracket a_i\rrbracket$  and update the state from  $S$  to  $S^{\prime}$ . Append example  $(a_{i},y_{i})$  into  $t$  samples and update  $t.\mathrm{cipher}_2 = \mathrm{Enc}(k_{\mathrm{pub}},|t$  samples|). We then compute the ciphertext  $\llbracket a_i\rrbracket = (t.\mathrm{cipher}_1,t.\mathrm{cipher}_2)$ , and update the state from  $S$  to  $S^{\prime}$  through our  $\mathcal{BT}$ .
-  $a_i \gets \text{Decrypt}(S', [[a_i]])$ : Solve plaintext  $a_i$  for ciphertext  $[[a_i]]$  based on state  $S'$  with binary search tree  $\mathcal{BT}$  and the CKKS secret key  $k_{\mathrm{sec}}$  as follows:
- Let  $t$  be a node pointer with initialing the root of binary search tree  $\mathcal{BT}$ . We then search a path downward in binary search tree  $\mathcal{BT}$  by comparing with  $[a_i]_1$ . The search continues to its left child if  $[a_i]_1 < t.\text{cipher}_1$  and update  $t = t.left$ ; the search continues to its right child if  $[a_i]_1 > t.\text{cipher}_1$  and update  $t = t.right$  until  $[a_i]_1 = t.\text{cipher}_1$ .
- Decrypt ciphertext  $\llbracket a_i\rrbracket_2$  by CKKS secret key  $k_{\mathrm{sec}}$ , and get index  $\tau = \mathrm{Dec}(k_{\mathrm{sec}},[[a_i]]_2)$  in t_samples. Then we use the index  $\tau$  to get the plaintext  $a_{i} = t$  samples[τ].