SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Besides these advantages, we also discuss the benefit of public data for removing the range parameter [10, 9]. Consider the example from [33], where the convergence rate is given by  $\left(\frac{r_n^{2d}}{n\varepsilon^2}\right)^{\frac{\alpha}{\alpha + d}}$ . When the set  $\mathcal{X} = \times_{j=1}^{d}[a^j, b^j]$  is unknown, it becomes necessary to create a histogram partition over  $\times_{j=1}^{d}[-r_n, r_n]$ , introducing an additional factor of  $r_n^{2d}$ . However, with publicly available data, we can approximate the range of the  $j$ -th dimension using  $\widehat{a^j} = \min_i X_i^j$  and  $\widehat{b^j} = \max_i X_i^j$ . Subsequently, we can perform min-max scaling on each data point from  $\times_{j=1}^{d}[\widehat{a^j}, \widehat{b^j}]$  to map it into  $\times_{j=1}^{d}[0,1]$ , and then train an LPDT on  $\times_{j=1}^{d}[0,1]$ . Any  $x$  that falls outside the range  $\times_{j=1}^{d}[0,1]$  is predicted as 0. We demonstrate that by this approach, Theorem 3.4 holds with a probability of at least  $1 - d / n_q^2$ . See derivations in Section C.6 in the appendix.
3.3 Complexity analysis
We demonstrate that LPDT is an efficient method. We first consider the average computation complexity of LPDT. The training stage consists of two parts. The partition procedure takes  $\mathcal{O}(sn_qd)$  time and the computation of (5) takes  $\mathcal{O}(sn)$  time. From the proof of Theorem 3.4, we know that  $2^s \asymp \left(n\varepsilon^2 / \log n\right)^{-\frac{d}{2\alpha + 2d}}$ . Thus the training stage complexity is around  $\mathcal{O}(n\log n\varepsilon^2 + n_qd\log n\varepsilon^2)$ . Since each prediction of the decision tree takes  $\mathcal{O}(s)$  time, the test time for each test instance is around  $\mathcal{O}(\log n\varepsilon^2)$ . As for storage complexity, since LPDT only requires the storage of the tree structure and the prediction value at each node, the space complexity of LPDT is  $\mathcal{O}\left(\left(n\varepsilon^2 / \log n\right)^{-\frac{d}{2\alpha + 2d}}\right)$ . In short, LPDT is an efficient method with a small number of parameters.
Table 1: Comparison of complexities of LDP regression methods.
LPDT	PHIST [9]	DECONV [29]
Training Time Complexity	O(n log nε2 + nq d log nε2)	O(nd log nε2)	-
Testing Time Complexity	O(log nε2)	O(log nε2)	O(nd)
Space Complexity	O((nε2 / log n)2α+2d)	O((nε2 / log n)2α+2d)	O(nd)
We compare the complexities of LPDT with other LDP regression methods in Table 1. Notably, [29] is inefficient due to its unacceptable test and space complexity. Also, the dominant term of training complexity of [9] is  $\mathcal{O}(nd\log n\varepsilon^2)$ . When  $d$  is large, we can choose a small  $n_q$  such that LPDT yields a strictly lower complexity than [9]. In addition, although [9] enjoys the same order of space complexity as LPDT, the memory of histogram-based methods suffers from the curse of dimensionality in practice. Since the storage of  $\mathcal{O}(h^{-d})$  values is required, even  $h = 1/2$  requires allocating an array of size  $2^d$ , which is problematic for large  $d$ . In contrast, LPDT can resist the curse of dimensionality by only splitting along the relevant features and keeping a small number of nodes.
In the experiments, we first validate our theoretical findings with synthetic data in Section 4.1. Then, in Section 4.2, we show the superior performance of LPDT on real-world datasets with identically distributed public data. In Section 4.3, we apply LPDT to Chicago taxi data to show that LPDT
performs well even with considerable differences between private and public data. Also, we analyze the influence of the distribution shift between private and public data on LPDT.
Splitting rule Note that most tree methods design their partition rules based on the information gained from the data. To boost the performance of LPDT, we also incorporate the variance reduction scheme from the original CART [11] to the tree construction. We denote the LPDT estimator using the max-edge partition rule with variance reduction in Section 2.3 as LPDT-M and denote the estimator using the standard variance reduction rule in [11] as LPDT-V. Since Theorem 3.1 holds for any partition, LPDT-V is also  $\varepsilon$ -LDP.
Experiment setup We choose the privacy budget  $\varepsilon \in [0.5, 8]$ , covering commonly seen magnitudes of privacy budgets from low to high privacy regimes. We compare LPDT-M and LPDT-V with the following methods: (i) Private Histogram (PHIST) [9]. (ii) Adjusted Private Histogram (APHIST) [33]. (iii) Deconvolution Kernel (DECONV) [29]. Introduction to the methods and all implementation details are presented in Appendix D.1. We employ 5-fold cross-validation for parameter selection, and techniques for tuning parameters under LDP are discussed in Section D.2. The evaluation metric is the mean squared error (MSE).
4.1 Synthetic experiments
Necessity of public data To demonstrate intuitively why public data is essential for LPDT, we first visualize its estimation on a synthetic model,  $Y = \sin (16X) + \epsilon$  where  $X \sim \mathcal{N}(0.5, 0.025)$  and  $\varepsilon \sim \mathcal{N}(0, 1)$ . In this case, the marginal distribution is highly imbalanced with the majority of samples located in the middle part of [0, 1] and a few samples on the sides. For  $\epsilon = 8$ , we fit two LPDT models: one with 500 public data and 7,000 private data, and another with 8,000 private data.
As shown in Figure 1(a), without public data, LPDT struggles with the imbalanced marginal. The grids on the side produce unstable predictions since only a few samples fall into them. As a result, LPDT tends to decrease depth  $s$  to stabilize its estimation. This leads to underfitting in the middle so that the predicted curve fails to capture the variation of the ground truth. In contrast, with the aid of public data, LPDT solves the issue as shown in Figure 1(b). For the mid
(a) Without public data
(b) With public data
Figure 1: The estimated regression curve of LPDT with and without public data. 1,000 samples are displayed in green.
dle zone where samples are abundant, LPDT creates small grids to enlarge approximation capacity. Meanwhile, it prunes the grids on the sides to ensure stability. Even with fewer private data, the MSE of LPDT is reduced from 1.19 to 1.08 thanks to the additional public data. In summary, the experiment provides empirical evidence supporting the theoretical findings in Theorems 3.4 and 3.5, which highlight the necessity of public data for the effective performance of LPDT.
Parameter analysis of depth  $s$  We conduct experiments to investigate the selection of partition depth  $s$  in terms of MSE. We generate 6,000 training samples, 2,000 test samples, and 2,000 public samples following the synthetic model described above. We pick  $\varepsilon \in \{3,4,6,8\}$  and  $s \in \{2,3,4,5,6\}$ . For each pair of  $s$  and  $\varepsilon$ , we plot the 20 times averaged MSE versus  $s$ . The result is displayed in Figure 2(a). Apparently, for each  $\varepsilon$ , as  $s$  increases, MSE first decreases until  $s$  reaches a certain value. Then MSE begins to increase as  $s$  grows. This further confirms the trade-off observed in Theorem 3.4. Moreover, the depth  $s$  at which the test error is minimized increases as  $\varepsilon$  increases. This is compatible with theory since the optimal choice of  $s \asymp \log n\varepsilon^2$  is monotonically increasing with respect to  $\varepsilon$ .
Parameter analysis of minimum leaf sample size  $n_l$  We conduct experiments to investigate the choice of  $n_l$  in terms of MSE. Following the same generating scheme, we choose  $\varepsilon \in \{3, 4, 6, 8\}$  and plot MSE of LPDT versus  $n_l$  for  $n_l \in \{15, 25, \dots, 135\}$ . In Figure 2(b), the relation between MSE and  $n_l$  is U-shaped under each  $\varepsilon$ , which indicates that a properly chosen  $n_l$  is necessary as stated in
(a)  $s$  -MSE
(b)  $n_l$  -MSE
(c)  $\varepsilon$  -MSE
Figure 2: Different parameters versus MSE.
Theorem 3.4. Furthermore, LPDT achieves the best MSE for  $n_l \in [35, 75]$  when  $\varepsilon = 4, 6, 8$ , while the minimum MSE occurred at  $n_l = 125$  when  $\varepsilon = 3$ . This finding is compatible with Theorem 3.4 which states that the optimal choice of  $n_l$  is monotonically decreasing with respect to  $\varepsilon$ .
Our parameter analyses indicate that decreasing  $\varepsilon$  favors smaller values of  $s$  and larger values of  $n_l$ . These choices lead to a decision tree partition with fewer grids. In summary, when facing higher levels of privacy demand, LPDT cuts down the number of grids to stabilize its estimation.
Privacy utility trade-off We analyze how privacy budget  $\varepsilon$  influences the quality of prediction. Under the same setup, we evaluate LPDT and other methods for  $\varepsilon \in \{0.5, 1, 2, 3, \dots, 10\}$  with 50 repetitions. The results are displayed in Figure 2(c). To show significance, we plot the  $(0.1, 0.9)$  quantiles as confidence intervals. When  $\varepsilon$  increases, the MSE of LPDT decreases much faster than the other methods. Note that the MSE of both PHIST and APHIST remains high, suggesting that their performances are limited by the histogram instead of the privacy mechanism.
4.2 Real data comparison with identically distributed public data
Experiment setup We conduct experiments on 12 real datasets, each repeated 50 times with a ratio of 1:7:2 for public data, training data, and testing data in each trial. The dataset details and pre-processing steps are summarized in Appendix D.3. To ensure significance, we adopt the Wilcoxon signed-rank test [58] with a significance level of 0.05 to check if a result is significantly better. For better comparison, we also train a decision tree (denoted as DT) on the original training data with no privacy protection, whose result will serve as a lower bound.
Table 2: Average MSE over real data sets for LDP regression methods. The best results are bolded and the second best results are underlined. The marked results with significance towards the rest results are marked with \*. Due to memory limitation, PHIST and APHIST are corrupted on two datasets which are marked with - .
DT	ε = 2	ε = 6
LPDT-M	LPDT-V	APHIST	PHIST	DECONV	LPDT-M	LPDT-V	APHIST	PHIST	DECONV
ABA	5.67e+0	1.01e+1	1.01e+1	1.89e+1	1.06e+1	1.01e+7	8.38e+0*	7.34e+0*	2.05e+1	1.05e+1	1.09e+1
AIR	2.26e+1	4.80e+1*	4.69e+1*	1.31e+3	6.80e+1	3.00e+2	4.49e+1*	3.60e+1*	1.60e+3	4.98e+1	4.72e+1
ALG	2.12e-2	2.57e-1	2.43e-1	2.52e-1	2.52e-1	9.26e+4	2.44e-1	2.46e-1	2.63e-1	2.47e-1	3.14e-1
AQU	1.92e+0	2.99e+0*	2.99e+0*	4.01e+0	2.93e+0*	5.74e+3	2.73e+0*	2.67e+0*	4.75e+0	2.83e+0	2.96e+0
BUI	1.75e+5	1.50e+6*	1.64e+6*	-	-	1.20e+9	1.44e+6*	1.31e+6*	-	-	2.04e+7
CBM	4.08e-27	2.12e+0*	1.65e+0*	9.53e+0	6.97e+0	2.37e+3	7.62e-1*	1.23e-1*	4.94e+0	3.21e+0	1.23e+5
CCP	2.19e+1	1.50e+2*	1.06e+2*	2.07e+4	3.64e+2	3.03e+2	8.42e+1*	5.18e+1*	2.24e+4	3.28e+2	2.56e+2
CON	9.38e+1	2.94e+2*	2.89e+2*	3.81e+2	3.00e+2	2.24e+7	2.44e+2*	2.13e+2*	4.16e+2	2.96e+2	3.13e+2
CPU	2.15e+1	3.41e+2	9.00e+1*	9.26e+2	3.42e+2	2.15e+5	3.02e+2*	6.15e+1*	9.98e+2	3.40e+2	3.98e+2
FIS	1.07e+0	2.15e+0*	2.14e+0*	3.14e+0	2.22e+0	3.47e+3	1.65e+0*	1.76e+0*	3.60e+0	2.16e+0	2.21e+0
HOU	2.11e+1	8.10e+1*	8.22e+1*	1.06e+2	8.52e+1	1.92e+4	7.43e+1*	7.10e+1*	1.23e+2	8.21e+1	2.44e+2
MUS	3.00e+2	3.47e+2*	3.46e+2*	-	-	9.50e+3	3.27e+2*	3.27e+2*	-	-	8.09e+3
RED	4.76e-1	7.08e-1*	7.03e-1*	3.18e+0	7.57e-1	1.23e+8	6.75e-1*	6.12e-1*	3.80e+0	7.12e-1	8.66e-1
WHI	5.77e-1	8.30e-1	8.42e-1	4.01e+0	8.15e-1	1.64e+7	7.03e-1*	6.61e-1*	4.45e+0	8.03e-1	1.47e+0
Performance of accuracy and running time The representative results for  $\varepsilon = 2,6$  are displayed in Table 2. Results of  $\varepsilon = 0.5,1,4,8$  is in Appendix D.4. It can be seen that LPDT-M and LPDT-V both significantly outperform their competitors. All methods achieve a higher MSE than DT, while the results for LPDT-M and LPDT-V are reasonably close to DT. Due to memory limitations, PHIST and APHIST fail on two datasets. We also compare the total running time in Table 6 in Appendix D.4. In general, both LPDT-M and LPDT-V achieve less running time than PHIST and APHIST, and are significantly faster than DECONV.
Table 3: Average MSE and standard deviation over Chicago taxi data.
DT	LPDT-M	PHIST	APHIST
Public	Private	ε = 0.5	ε = 1	ε = 2	ε = 4	ε = 6	ε = 8	ε = 2	ε = 8	ε = 2	ε = 8
3.71	0.80	113.45	15.74	4.89	3.35	2.86	2.70	24.72	17.22	38.22	35.5
(14.23)	(2.20)	(0.54)	(0.33)	(0.10)	(0.10)	(0.02)	(0.00)	(0.01)	(0.01)
With the private and public distributions being identical, another alternative is to use solely public data to fit a decision tree. We train such decision trees on public data  $(n_q = 0.1n)$ . It is compared with the decision tree trained on private data and LPDT with  $\varepsilon = 6$  in Table 7 in Appendix. The results show that when  $n_q \ll n$  and  $\varepsilon$  is large, training a decision tree on public data is worse than using LPDT on most of the datasets.
The power of public data We conduct experiments to show the utility gain brought by public data. We take red wine dataset as an example. With  $n_q = 100$  public data, we run LPDT (with  $D^{\mathrm{P}}$ ) and PHIST (without  $D^{\mathrm{P}}$ ). For  $\delta \in \{0.2, 0.4, 0.6, 0.8, 1\}$ ,
we train each model on  $\delta \cdot n$  samples with 20 repetitions. The result is in Figure 3. PHIST with 1,100 samples achieves the same MSE as LPDT with 660 samples. The result shows that, with a small amount of public data, LPDT achieves the same performance with much fewer samples.
Figure 3: Power of public data.
4.3 Real data comparison with non-identically distributed public data
We apply LPDT to the Chicago Taxi Dataset, a collection of taxi trips in Chicago provided by the Differential Privacy Temporal Map Challenge and contains sensitive information [21]. We use the fare of each trip as labels and other information as features. Then we regard trips paid by PR card and credit card as public data and private data, respectively. In Appendix D.5, we show the two parts are distributed differently. After preprocessing, the dataset has 101 features with 2,150,565 samples in private data and 24,436 samples in public data.
Performance We report the averaged MSE of LPDT over 20 repetitions for  $\varepsilon = 0.5, 1, 2, 4, 6$ , and 8. As a comparison to LPDT, we train two decision trees separately using the public and private datasets with no privacy protection. As for the comparison methods, DECONV fails due to the large sample size while PHIST and APHIST fail due to the dimensionality. However, after reducing the dimensionality by retaining only the continuous features, we are able to apply PHIST and APHIST. The results are displayed in Table 3. A first observation is that the decision tree trained on public data yields considerably worse results than the decision tree trained on private data. This suggests that relying solely on public data leads to biased predictions. Learning solely from public data achieves an MSE higher than LPDT for all three values of  $\varepsilon$ . Even for  $\varepsilon = 2$ , LPDT significantly outperforms histogram-based methods with  $\varepsilon = 8$ . This indicates that LPDT remains effective even when substantial disparities exist between the distributions of public and private data.
How does non-identically distributed public data help? It is counterintuitive that non-identically distributed public data can benefit private estimation. We show the logic behind using such public data by investigating the first split feature in the tree partition. LPDT identifies whether a trip ends in Zone 32 as an important feature for predicting fare and initiates the recursive partitioning process by splitting along this feature. Figure 4 illustrates that trips ending in this zone generally have lower fares for both public and private data, although the actual fares differ significantly between the two datasets. This observation demonstrates that despite having different distributions, public and private data may exhibit similar patterns, thereby allowing the partition created on public data to still be effective on private data. Following this line of reasoning, to determine whether public data is suitable for a specific private task, we can examine whether the qualitative relationships between labels and features remain consistent across both datasets. Whether a dataset can be utilized as public data can be of independent interest as in [32].
Analysis of distribution shift In Table 3, the MSE of LPDT reduces as  $\varepsilon$  increases but remains higher than the MSE achieved by training a decision tree directly on private data. The performance
Figure 4: Box plot for fares of trips dropping in and out of Zone 32.
(a) Wine data
Figure 5: Portion of public data versus MSE of LPDT.
(b) Chicago taxi data
gap can be attributed to the distribution shift between private data and public data. In the following, we investigate how the difference between the two datasets affects the performance of LPDT. Besides the Chicago taxi data, we also adopt White Wine and Red Wine data in Section 4.2 as private and public data, respectively. The datasets contain the same variables but are distributed differently, as is investigated in transfer learning literature [48]. On both datasets, we combine part of the private samples with public data and perform the partition procedure on the combined dataset, with the portion of public data denoted as  $\delta$ . When  $\delta$  is small, there is less difference between data used for partition and training. We report the average MSE of LPDT for  $\delta \in \{0.1, 0.3, 0.5, 0.7, 0.9\}$  after 20 repetitions. Figure 5 shows that a small  $\delta$  leads to a lower MSE for both datasets and all values of  $\varepsilon$ . Thus, LPDT is more powerful when the public and private data are distributed similarly, which also justifies the necessity of Assumption 3.3.
This paper addresses the challenge of effectively performing LDP regression given both public data and private data by introducing the locally private decision tree. Due to the novel idea of leveraging public data, LPDT is accurate, efficient, and interpretable. Theoretically, we establish the privacy guarantee and optimal convergence rate of LPDT. In experiments, we show the superior performance of LPDT regardless of the disparities between private and public data.
6 Limitations and broader impact
LPDT addresses the challenge of effectively performing LDP regression given both public data and private data. Future work may explore methodologies for incorporating public data in private learning using other models, such as linear models, and investigate the theoretical advantages of using public data in private learning. One limitation of this paper is that the measure of similarity between the public and private distribution considers only the marginal distribution. One may consider the similarities with respect to the regression function, potentially following transfer learning literature [13]. Moreover, in practice, public data may have a different distribution than private data, which is damaging to the learning process. The drawback yields the significance of public data selection or public data quality test such as [32, 60]. In addition, the theoretical analysis deal with  $n_q \gtrsim n^{\frac{d}{2\alpha + 2d}}$  and  $n_q = 0$ , while the intermediate zone for  $n_q$  remains unexplored.
The authors would like to thank the reviewers for their constructive comments, which led to a significant improvement in this work. The project is supported by the Special Funds of the National Natural Science Foundation of China (Grant No. 72342010) and was supported in part by the National Key R&D Program of China (2022YFB2702401). This research was supported by Public Computing Cloud, Renmin University of China.
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC
[58] Frank Wilcoxon. Individual comparisons by ranking methods. In *Breakthroughs in statistics*, pages 196–202. Springer, 1992.
[59] Xiaotong Wu, Lianyong Qi, Jiaquan Gao, Genlin Ji, and Xiaolong Xu. An ensemble of random decision trees with local differential privacy in edge computing. Neurocomputing, 485:181-195, 2022.
[60] Da Yu, Sivakanth Gopi, Janardhan Kulkarni, Zinan Lin, Saurabh Naik, Tomasz Lukasz Religa, Jian Yin, and Huishuai Zhang. Selective pre-training for private fine-tuning. arXiv preprint arXiv:2305.13865, 2023.
[61] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. Differentially private fine-tuning of language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
[62] Da Yu, Huishuai Zhang, Wei Chen, and Tie-Yan Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. In International Conference on Learning Representations, 2021.
[63] Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu. Large scale private learning via low-rank reparametrization. In International Conference on Machine Learning, pages 12208-12218. PMLR, 2021.
[64] Fang Zhou, Q Claire, and Ross D King. Predicting the geographical origin of music. In 2014 IEEE International Conference on Data Mining, pages 1115-1120. IEEE, 2014.
[65] Yingxue Zhou, Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Private sgd with gradient subspace identification. In International Conference on Learning Representations, 2021.
In this appendix, we provide the detailed results for the additional methodology (Appendix A), the error analysis of the main theoretical results (Appendix B), the full proof of all theoretical results (Appendix C), and details as well as additional results of experiments (Appendix D).
A Methodology of locally differentially private decision tree
A.1 Algorithm of max-edge partition with variance reduction rule
Algorithm 2: Max-edge Partition Rule with Variance Reduction
end Output: Partition  $\pi_s$
Input: Public data  $D^{pub}$ , depth  $s$ , minimum leaf sample size  $n_l$ . Initialization:  $\pi_0 = [0,1]^d$ .  
for  $i = 1$  to  $s$  do  
 $\pi_i = \emptyset$   
for  $j$  in  $\mathcal{I}_{i-1}$  do  
Select  $\ell$  as in (6).  
if  $\sum_{i=1}^{n} \mathbf{1}\{X_i^{pub} \in A_{i-1}^{j,k}(\ell)\} \leq n_l$  for  $k = 0,1$  then  
 $\pi_i = \pi_i \cup A_{i-1}^j$   
end  
else  
 $\pi_i = \pi_i \cup \{A_{i-1}^{j,0}(\ell), A_{i-1}^{j,1}(\ell)\}$   
end
B Error analysis
We rely on the following decomposition.
$$
\begin{array}{l} \mathcal {R} _ {L, \mathrm {P}} \left(f _ {\pi} ^ {\mathrm {D P}}\right) - \mathcal {R} _ {L, \mathrm {P}} \left(f _ {\pi}\right) = \underbrace {\mathcal {R} _ {L , \mathrm {P}} \left(f _ {\pi} ^ {\mathrm {D P}}\right) - \mathcal {R} _ {L , \mathrm {P}} \left(f _ {\pi}\right)} _ {\text {P r i v a t i z e d E r r o r}} + \underbrace {\mathcal {R} _ {L , \mathrm {P}} \left(f _ {\pi}\right) - \mathcal {R} _ {L , \mathrm {P}} \left(\bar {f} _ {\pi}\right)} _ {\text {S a m p l e E r r o r}} \\ + \underbrace {\mathcal {R} _ {L , \mathrm {P}} \left(\bar {f} _ {\pi}\right) - \mathcal {R} _ {L , \mathrm {P}} \left(f ^ {*}\right)} _ {\text {A p p r o x i m a t i o n E r r o r}}. \tag {7} \\ \end{array}
$$
where  $\overline{f}_{\pi}$ ,  $f_{\pi}$  and  $f_{\pi}^{\mathrm{DP}}$  are defined in (1), (2), and (5), respectively. Loosely speaking, the first error term quantifies the deprivation brought by adding privacy noises to the estimator, which we call the privatized error. The second term corresponds to the expected estimation error brought by the randomness of the data, which we call the sample error. The last term is called approximation error, which arises due to the limited approximation capacity of piecewise constant functions. The following three lemmas provide bounds for each of the three errors.