SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 4. For dataset  $A = \{(a_{1},y_{1}),\dots ,(a_{n},y_{n})\}$ , let  $\mathcal{I}_1,\mathcal{I}_2,\dots ,\mathcal{I}_s$  be the corresponding partitions as defined by Eqn. (4). There exists a splitting point  $a^*$  such that  $I_G(A,a^*) = I_G^* (A)$  and
$$
a ^ {*} \in \bigcup_ {i \in [ s - 1 ]} \left\{\max  \left\{a _ {k} \colon \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i} \right\} / 2 + \min  \left\{a _ {k} \colon \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i + 1} \right\} / 2 \right\},
$$
where  $I_G(A, a^*)$  and  $I_G^*(A)$  are defined by Eqns. (1) and (2), respectively.
Proof. Without loss of generality, we assume that  $a_1, a_2, \dots, a_n$  are distinct elements. Our goal is to solve the optimal splitting point  $a^* \in \arg \min_{a \in \mathbb{R}} \{I_G(A, a)\}$ , and we begin with some notations used in our proof. For every label  $j \in [\tau]$ , we denote by
$$
\nu_ {j} = \left| \left\{i \in [ n ] \colon y _ {i} = j \right\} \right|,
$$
i.e., the number of the label  $j$  in dataset  $A$ . Let  $a$  be a splitting point, which splits  $A$  into left and right datasets  $A_{a}^{l}$  and  $A_{a}^{r}$ , that is,
$$
A _ {a} ^ {l} = \left\{\left(a _ {i}, y _ {i}\right): a _ {i} \leq a, \left(a _ {i}, y _ {i}\right) \in A \right\},
$$
$$
A _ {a} ^ {r} = \left\{\left(a _ {i}, y _ {i}\right): a _ {i} > a, \left(a _ {i}, y _ {i}\right) \in A \right\}.
$$
For any given  $a \in \mathbb{R}$  and  $j \in [\tau]$ , we further denote by
$$
\nu_ {j} ^ {l} = \left| \left\{i \in [ n ] \colon y _ {i} = j, a _ {i} \leq a \right\} \right|,
$$
i.e., the number of label  $j$  in subsets  $A_{a}^{l}$ . This follows that
$$
I _ {G} (A, a) = w _ {l} - w _ {l} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} ^ {l}) ^ {2}}{| A _ {a} ^ {l} | ^ {2}} + w _ {r} - w _ {r} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} - \nu_ {j} ^ {l}) ^ {2}}{(n - | A _ {a} ^ {l} |) ^ {2}},
$$
where  $w_{l} = |A_{a}^{l}| / n$ , and  $w_{r} = 1 - w_{l}$ . In the following, we will explore the monotonicity of function  $I_{G}(A,a)$  when
$$
a \geq \max  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i - 1} \right\} / 2 + \min  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i} \right\} / 2
$$
$$
a \leq \max  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i} \right\} / 2 + \min  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i + 1} \right\} / 2,
$$
for  $i = 2,3,\dots ,s - 1$ . It is easy to observe that  $\nu_{j}$  and  $\nu_{j}^{l}$  keep constants except for  $\nu_{j*}^{l}$ , where  $j_*$  denotes the label of instances in  $\mathcal{I}_i$ . It remains to discuss the variable  $\nu_{j*}^{l}$ , and we have
$$
\begin{array}{l} n ^ {2} \frac {\partial I _ {G} (A , a)}{\partial \nu_ {j _ {*}} ^ {l}} = \frac {1}{n} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} ^ {l}) ^ {2}}{(w _ {l}) ^ {2}} - 2 \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}} - \frac {1}{n} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} - \nu_ {j} ^ {l}) ^ {2}}{(w _ {r}) ^ {2}} + 2 \frac {(\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l})}{w _ {r}} \\ = \frac {1}{n} \sum_ {j \in [ \tau ]} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) + 2 \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) \\ = \frac {1}{n} \sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) \\ + \frac {1}{n} \left(\left(\frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}}\right) ^ {2}\right) + 2 \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) \\ = \frac {1}{n} \sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\frac {\left(\nu_ {j} ^ {l}\right) ^ {2}}{\left(w _ {l}\right) ^ {2}} - \frac {\left(\nu_ {j} - \nu_ {j} ^ {l}\right) ^ {2}}{\left(w _ {r}\right) ^ {2}}\right) + \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) \left(2 - \frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{n w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{n w _ {l}}\right). \\ \end{array}
$$
It is easy to observe that
$$
0 \leq \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}} \leq n \quad \text {a n d} \quad 0 \leq \frac {\nu_ {j} ^ {l}}{w _ {l}} \leq n \text {f o r e a c h} j \in [ \tau ]. \tag {14}
$$
It is sufficient to consider two cases as follows:
We consider the first case
$$
\sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) \geq 0,
$$
$$
\begin{array}{l} 0 \leq \sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) \\ = \sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\frac {\nu_ {j} ^ {l}}{w _ {l}} + \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) \left(\frac {\nu_ {j} ^ {l}}{w _ {l}} - \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) \\ \leq \sum_ {j \in [ \tau ], j \neq j _ {*}} 2 n \left(\frac {\nu_ {j} ^ {l}}{w _ {l}} - \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) = 2 n \sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\frac {\nu_ {j} ^ {l}}{w _ {l}} - \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right). \\ \end{array}
$$
$$
n - \sum_ {j \in [ \tau ], j \neq j _ {*}} \frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}} \geq n - \sum_ {j \in [ \tau ], j \neq j _ {*}} \frac {\nu_ {j} ^ {l}}{w _ {l}}, \tag {15}
$$
and it holds that
$$
\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} \geq \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}. \tag {16}
$$
Combining with Eqns. (14)-(16), we have
$$
\frac {\partial I _ {G} (A , a)}{\partial \nu_ {j ^ {*}} ^ {l}} \geq 0  ,
$$
which proves the increasing function of  $I_G(A, a)$ .
- We now consider the second case
$$
\sum_ {j \in [ \tau ], j \neq j _ {*}} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) <   0,
$$
$$
\begin{array}{l} \sum_ {j \in [ \tau ]} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) <   \left(\frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}}\right) ^ {2} \\ { = } { \left( \frac { \nu _ { j _ { * } } ^ { l } } { w _ { l } } + \frac { \nu _ { j _ { * } } - \nu _ { j _ { * } } ^ { l } } { w _ { r } } \right) \left( \frac { \nu _ { j _ { * } } ^ { l } } { w _ { l } } - \frac { \nu _ { j _ { * } } - \nu _ { j _ { * } } ^ { l } } { w _ { r } } \right) <   2 n \left( \frac { \nu _ { j _ { * } } ^ { l } } { w _ { l } } - \frac { \nu _ { j _ { * } } - \nu _ { j _ { * } } ^ { l } } { w _ { r } } \right) . } \\ \end{array}
$$
$$
\begin{array}{l} n ^ {2} \frac {\partial I _ {G} (A , a)}{\partial \nu_ {j _ {*}} ^ {l}} = \frac {1}{n} \sum_ {j \in [ \tau ]} \left(\left(\frac {\nu_ {j} ^ {l}}{w _ {l}}\right) ^ {2} - \left(\frac {\nu_ {j} - \nu_ {j} ^ {l}}{w _ {r}}\right) ^ {2}\right) + 2 \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) \\ <   2 \left(\frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}} - \frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}}\right) + 2 \left(\frac {\nu_ {j _ {*}} - \nu_ {j _ {*}} ^ {l}}{w _ {r}} - \frac {\nu_ {j _ {*}} ^ {l}}{w _ {l}}\right) = 0, \\ \end{array}
$$
which proves the decreasing function of  $I_G(A, a)$ .
In a summary, we prove the piecewise monotonicity of  $I_G(A,a)$  for
$$
\begin{array}{l} a \geq \max  \left\{a _ {k} \colon \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i - 1} \right\} / 2 + \min  \left\{a _ {k} \colon \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i} \right\} / 2 \\ a \leq \max  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i} \right\} / 2 + \min  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {i + 1} \right\} / 2, \\ \end{array}
$$
with  $i = 2,3,\dots ,s - 1$ . Moreover, it is easy to observe the monotonicity of  $I_G(A,a)$  from  $\nu_{j}^{l} = 0(j\neq j_{*})$  when
$$
a \in (- \infty , (\max  \{a _ {k}: (a _ {k}, y _ {k}) \in \mathcal {I} _ {1} \} + \min  \{a _ {k}: (a _ {k}, y _ {k}) \in \mathcal {I} _ {2} \}) / 2 ];
$$
and from  $\nu_{j} - \nu_{j}^{l} = 0$  ( $j \neq j_{*}$ ) when
$$
a \in \left[ \left(\max  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {s - 1} \right\} + \min  \left\{a _ {k}: \left(a _ {k}, y _ {k}\right) \in \mathcal {I} _ {s} \right\}\right) / 2, + \infty\right).
$$
It is not necessary to consider the splitting point  $a^* > \max \{a_k : (a_k, y_k) \in \mathcal{I}_s\}$  with  $|A_a^r| = 0$ , as well as the splitting point  $a^* < \min \{a_k : (a_k, y_k) \in \mathcal{I}_1\}$  with  $|A_a^l| = 0$ , i.e., without splitting dataset  $A$ . This completes the proof.
According to Lemma 6, we could find an optimal splitting point  $a^*$  such that
$$
a ^ {*} \in \bigcup_ {i \in [ s - 1 ]} \left\{\frac {\max  \left\{a _ {k} : \left(a _ {k} , y _ {k}\right) \in \mathcal {I} _ {i} \right\} + \min  \left\{a _ {k} : \left(a _ {k} , y _ {k}\right) \in \mathcal {I} _ {i + 1} \right\}}{2} \right\}.
$$
It is easy to observe that, for  $i \in [s - 1]$
$$
I _ {G} (A, (\max  \{a _ {k}: (a _ {k}, y _ {k}) \in \mathcal {I} _ {i} \} + \min  \{a _ {k}: (a _ {k}, y _ {k}) \in \mathcal {I} _ {i + 1} \} / 2) = I _ {G} (A, (c _ {i} + c _ {i + 1}) / 2),
$$
where  $c_{i}$  is the identical ciphertext for those elements in  $\in \mathcal{I}_i$ , and we complete the proof.
Based on Theorem 1, our encryption with binary search trees (Algorithm 1) can also preserve the minimum Gini impurity over encrypted data, which can be shown by the following theorem:
Theorem 5. We have  $I_G^*(A) = I_G^*(\hat{A})$ , for re-sort dataset  $A$  by Eqn. (3) and for the corresponding encrypted dataset  $\hat{A} = \{(\llbracket a_{\langle 1\rangle}\rrbracket_1, y_{\langle 1\rangle}), \dots, (\llbracket a_{\langle n\rangle}\rrbracket_1, y_{\langle n\rangle})\}$  from Algorithm 1.
Proof. Our constructed binary search tree  $\mathcal{BT}$  (Algorithm 1) maintains several samples on a node. For each node  $t$ , we have  $t.\mathrm{cipher}_1 < t.\mathrm{right.cipher}_1$  and  $t.\mathrm{cipher}_1 > t.\mathrm{left.cipher}_1$ . In this way, we can obtain a monotone increasing sequence  $\mathcal{I}_1,\mathcal{I}_2,\dots ,\mathcal{I}_s$  by inorder traversing the built Tree  $\mathcal{BT}$  in Algorithm 1. Each  $\mathcal{I}_i$  for  $j\in [s]$  contains several samples as follows:
$$
\mathcal {I} _ {1} = \left\{\left(a _ {\langle 1 \rangle}, y _ {\langle 1 \rangle}\right), \dots , \left(a _ {\langle k _ {1} \rangle}, y _ {\langle k _ {1} \rangle}\right) \right\}
$$
$$
\begin{array}{r c l} \mathcal {I} _ {2} & = & \left\{\left(a _ {\langle k _ {1} + 1 \rangle}, y _ {\langle k _ {1} + 1 \rangle}\right),, \dots , \left(a _ {\langle k _ {2} \rangle}, y _ {\langle k _ {2} \rangle}\right) \right\} \\ & \dots \end{array} \tag {17}
$$
$$
\mathcal {I} _ {s} = \left\{\left(a _ {\langle k _ {s - 1} + 1 \rangle}, y _ {\langle k _ {s - 1} + 1 \rangle}\right), \dots , \left(a _ {\langle n \rangle}, y _ {\langle n \rangle}\right) \right\},
$$
where  $a_{\langle i'\rangle} < a_{\langle j'\rangle}$  for  $(a_{\langle i'\rangle}, y_{\langle i'\rangle}) \in \mathcal{I}_i, (a_{\langle j'\rangle}, y_{\langle j'\rangle}) \in \mathcal{I}_j$  and  $i < j$ .
For each  $\mathcal{I}_i$ , if there is only one identical label, i.e.,  $y_{\langle i\rangle} = y_{\langle i^{\prime}\rangle}$  for every  $(a_{\langle i\rangle},y_{\langle i\rangle}),(a_{\langle i^{\prime}\rangle},y_{\langle i^{\prime}\rangle})\in$ $\mathcal{I}_j$ , then we have  $I_G^* (A) = I_G^* (\hat{A})$  from Theorem 1. On the other hand, if the values are the same for all samples in  $\mathcal{I}_j$ $(j\in [s])$ , i.e.,  $a_{\langle i\rangle} = a_{\langle i^{\prime}\rangle}$  for every  $(a_{\langle i\rangle},y_{\langle i\rangle}),(a_{\langle i^{\prime}\rangle},y_{\langle i^{\prime}\rangle})\in \mathcal{I}_j$ , then this splitting value is preserved without changing the minimum Gini-impurity of random forests. Hence, we also have  $I_G^* (A) = I_G^* (\hat{A})$ , and this completes the proof.
Given two sequences of distinct plaintext  $A^0 = \{a_1^0, a_2^0, \dots, a_n^0\}$  and  $A^1 = \{a_1^1, a_2^1, \dots, a_n^1\}$ , their corresponding labels are randomly set as follows:
- Sort  $A^b = \{a_{\langle 1\rangle}^b, a_{\langle 2\rangle}^b, \dots, a_{\langle n\rangle}^b\}$  in ascending order, i.e.,  $a_{\langle 1\rangle}^b < a_{\langle 2\rangle}^b < \dots < a_{\langle n\rangle}^b$  for  $b \in \{0,1\}$ .
- Set the corresponding labels  $\{y_{\langle 1\rangle},y_{\langle 2\rangle},\dots ,y_{\langle n\rangle}\}$  randomly and independently from a uniform distribution on  $[\tau ]$
Lemma 6. For  $a_1^b < a_2^b < \dots < a_n^b$  with  $b = \{0,1\}$ , we have the same Gini impurity for two sequences  $A^0 = \{(a_1^0,y_1),(a_2^0,y_2),\dots ,(a_n^0,y_n)\}$  and  $A^{1} = \{(a_{1}^{1},y_{1}),(a_{2}^{1},y_{2}),\dots ,(a_{n}^{1},y_{n})\}$ .
Proof. Let  $a_i^b$  be a splitting point for  $b \in \{0,1\}$  and  $i \in [n]$ , and we split  $A^b$  into left and right datasets  $A_{a_i^b}^{l,b}$  and  $A_{a_i^b}^{r,b}$  according to  $a_i^b$ , that is,
$$
A _ {a _ {i} ^ {b}} ^ {l, b} = \left\{\left(a _ {1} ^ {b}, y _ {1}\right), \left(a _ {2} ^ {b}, y _ {2}\right), \dots , \left(a _ {i} ^ {b}, y _ {i}\right) \right\} \text {a n d} A _ {a _ {i} ^ {b}} ^ {r, b} = \left\{\left(a _ {i + 1} ^ {b}, y _ {i + 1}\right), \left(a _ {i + 2} ^ {b}, y _ {i + 2}\right), \dots , \left(a _ {n} ^ {b}, y _ {n}\right) \right\}.
$$
For  $j \in [\tau]$ , denote by  $\nu_j^{l,b}$  and  $\nu_j^{r,b}$  the cardinalities of subsets  $A_{a_i^b}^{l,b}$  and  $A_{a_i^b}^{r,b}$  with label  $j$ , respectively. Then, the Gini impurity of dataset  $A^b$  and splitting point  $a_i^b$  is given by
$$
I _ {G} (A ^ {b}, a _ {i} ^ {b}) = \frac {i}{n} - \frac {i}{n} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} ^ {l , b}) ^ {2}}{(i) ^ {2}} + \frac {n - i}{n} - \frac {n - i}{n} \sum_ {j \in [ \tau ]} \frac {(\nu_ {j} ^ {r , b}) ^ {2}}{(n - i) ^ {2}}.
$$
For  $b \in \{0,1\}$ ,  $A_{a_i^0}^{l,0}$  and  $A_{a_i^1}^{l,1}$  have the same labels  $\{y_1,y_2,\dots ,y_i\}$ , and we have
$$
\sum_ {j \in [ \tau ]} \left(\nu_ {j} ^ {l, 0}\right) ^ {2} = \sum_ {j \in [ \tau ]} \left(\nu_ {j} ^ {l, 1}\right) ^ {2}.
$$
Similarly, we have  $\sum_{j\in [\tau]}(\nu_j^{r,0})^2 = \sum_{j\in [\tau]}(\nu_j^{r,1})^2$ , and this completes the proof.
We can show that adversary can not distinguish the ciphertext of  $\{(a_{\langle 1\rangle}^{0},y_{\langle 1\rangle}),\dots ,(a_{\langle n\rangle}^{0},y_{\langle n\rangle})\}$  from that of  $\{(a_{\langle 1\rangle}^{1},y_{\langle 1\rangle}),\dots ,(a_{\langle n\rangle}^{1},y_{\langle n\rangle})\}$  in a probabilistic perspective, i.e.,
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle n \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle n \rangle} ^ {0}, y _ {\langle n \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket , \dots , \llbracket a _ {\langle n \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle n \rangle} ^ {1}, y _ {\langle n \rangle})\right). \tag {18} \\ \end{array}
$$
We will prove Eqn. (18) by induction on  $n$ . We first have
For  $n = 1$ , we have  $\llbracket a_{\langle 1\rangle}^{0}\rrbracket = c_{max}^{0} / 2$  and  $\llbracket a_{\langle 1\rangle}^{1}\rrbracket = c_{max}^{1} / 2$  with  $c_{max}^{0} = c_{max}^{1} = 2^{\lambda \log_2n}$ , according to the initialization in Algorithm 1. This follows that
$$
\operatorname * {P r} \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle})\right) = \operatorname * {P r} \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket\right) = \operatorname * {P r} \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket\right) = \operatorname * {P r} \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle})\right).
$$
We assume that Eqn. (18) holds for  $n = i$  ( $i > 1$ ), that is,
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle})\right). \tag {19} \\ \end{array}
$$
Let us consider the case  $n = i + 1$ , and we add the sample  $(a_{\langle i + 1\rangle}^{b},y_{\langle i + 1\rangle})$  in binary search tree  $\mathcal{BT}^b$  (Algorithm 1). It is sufficient to consider two cases as follows:
- If we do not need to split a node for sample  $(a_{\langle i+1\rangle}^{b}, y_{\langle i+1\rangle})$  in Algorithm 1, then we have  $\llbracket a_{\langle i+1\rangle}^{1}\rrbracket = \llbracket a_{\langle i+1\rangle}^{0}\rrbracket$ . This is because  $a_{\langle 1\rangle}^{b} < a_{\langle 2\rangle}^{b} < \dots < a_{\langle i+1\rangle}^{b}$  for  $b = 0$  and  $b = 1$ , along with the same labels  $\{y_1, \dots, y_{i+1}\}$ . Hence, we obtain the same ciphertext for  $a_{\langle i+1\rangle}^{0}$  and  $a_{\langle i+1\rangle}^{1}$ , i.e.,  $t^0.\mathrm{cipher}_1 = t^1.\mathrm{cipher}_1$ . This follows that
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket , \llbracket a _ {\langle i + 1 \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle})\right), \\ \end{array}
$$
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {1} \rrbracket , \llbracket a _ {\langle i + 1 \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {1}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle})\right). \\ \end{array}
$$
By induction assumption in Eqn. (19), we have
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket , \llbracket a _ {\langle i + 1 \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {1} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {1} \rrbracket , \llbracket a _ {\langle i + 1 \rangle} ^ {1} \rrbracket | (a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {1}, y _ {\langle i + 1 \rangle})\right). \\ \end{array}
$$
- If we need to split the node for  $(a_{\langle i+1\rangle}^{b}, y_{\langle i+1\rangle})$  in Algorithm 1, then we assume that  $t^{0}$  and  $t^{1}$  are the corresponding splitting nodes. We firstly initialize the empty node  $l^{b}$  and  $r^{b}$ , and update the ciphertext  $l^{b}$ .  $\mathsf{cylinder}_{1}$  and  $r^{b}$ .  $\mathsf{cylinder}_{1}$  by Eqns (8)-(11), respectively. Notice that the random number  $\xi$  in Eqns (8)-(11) is sampled from  $\mathcal{N}(0,1)$ , and thus  $l^{0}$ .  $\mathsf{cylinder}_{1}$  and  $l^{1}$ .  $\mathsf{cylinder}_{1}$  are sampled from the same distribution. We have
$$
\begin{array}{l} \Pr \left(l ^ {0}. c i p h e r _ {1} | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(l ^ {1}. c i p h e r _ {1} | \left(a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}\right), \dots , \left(a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle}\right), \left(a _ {\langle i + 1 \rangle} ^ {1}, y _ {\langle i + 1 \rangle}\right)\right). \\ \end{array}
$$
Similarly,  $r^0$ .cipher $_1$  and  $r^1$ .cipher $_1$  are sampled from the same distribution, and we have
$$
\begin{array}{l} \Pr \left(r ^ {0}. c i p h e r _ {1} | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(r ^ {1}. c i p h e r _ {1} | \left(a _ {\langle 1 \rangle} ^ {1}, y _ {\langle 1 \rangle}\right), \dots , \left(a _ {\langle i \rangle} ^ {1}, y _ {\langle i \rangle}\right), \left(a _ {\langle i + 1 \rangle} ^ {1}, y _ {\langle i + 1 \rangle}\right)\right). \\ \end{array}
$$
For the  $(i + 1)$ -th iteration in Algorithm 1, we have
$$
\begin{array}{l} \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket , \llbracket a _ {\langle i + 1 \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right) \\ = \Pr \left(\llbracket a _ {\langle 1 \rangle} ^ {0} \rrbracket , \dots , \llbracket a _ {\langle i \rangle} ^ {0} \rrbracket | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle})\right) \\ \times \Pr \left(l ^ {0}. c i p h e r _ {1} | \left(a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}\right), \dots , \left(a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}\right), \left(a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle}\right)\right) \\ \times \Pr \left(r ^ {0}. c i p h e r _ {1} | (a _ {\langle 1 \rangle} ^ {0}, y _ {\langle 1 \rangle}), \dots , (a _ {\langle i \rangle} ^ {0}, y _ {\langle i \rangle}), (a _ {\langle i + 1 \rangle} ^ {0}, y _ {\langle i + 1 \rangle})\right), \\ \end{array}
$$
Table 4: Hyperparameter settings for tree ensemble models in experiments.  $-$  means that the parameter is not exist in the corresponding method, and 'max_bin' denotes the maximum splitting point of each feature.
Parameter	Our Work	PPD-ERTs	HEldpRFs	PivotRFs	MulPRFs	AnonyRFs	DiffPrivRFs	Original RFs
max_depth	None	None	5	4	None	None	None	None
n_estimators	100	100	100	100	100	100	100	100
max_features	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)	\(\lfloor \sqrt{d}\rfloor\)
differentia privacy level ε	-	-	-	-	-	-	1	-
anonymization parameter k	-	-	-	-	-	10	-	-
multi-party size p	2	2	2	2	2	-	-	-
max_bin	-	-	-	16	-	-	-	-