SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When the Gaussian assumption does not hold, one can recur to (slower) Bayesian approaches to estimate the uncertainty of unrevealed features  $X_{U}$  given the set of revealed features  $X_{R} = x_{R}$ . A common approach involves treating  $X_{U}$  as the target variable and employing a neural network to establish the mapping  $X_{U} = g_{w}(X_{R})$ . Utilizing Bayesian techniques [17, 18], the posterior of the network's parameter  $p(w|D) = p(w)p(D|w)$  can be computed initially. Based on the posterior distribution of the model's parameters  $w$ , the posterior of unrevealed features can be calculated as  $\operatorname{Pr}(X_U = x_U | X_R = x_R) = \operatorname{Pr}_{w \sim p(w|D)}(g_w(X_R) = x_U)$ . However, implementing such a Bayesian network not only significantly increases training time but also inference time. Since it is necessary to compute  $\operatorname{Pr}(X_U | X_R = x_R)$  for all possible choices of  $U \in S$ , the number of Bayesian neural network regressors scales exponentially with  $|S|$ .
Importantly, in our evaluation, the data minimization method that operates under the Gaussian assumption maintains similar decision-making and produces comparable outcomes to the Bayesian approach, even in cases where the Gaussian assumption is not applicable in practical settings. Figure 3 illustrates this comparison, showcasing the performance of the proposed mechanism on a real dataset (Credit dataset with
Figure 3: Comparison between Bayesian NN vs Gaussian in term of accuracy and data Lekage.
$|S| = 5$  ) concerning accuracy (higher is better) and data leakage (lower is better) across various failure probability  $\delta$  values. Notice how similar is the performance of the mechanisms that either leverage the Gaussian assumption or operate without it (Bayesian NN). Importantly, the assumption of a Gaussian distribution is not overly restrictive or uncommon. In fact, it is a cornerstone in many areas of machine learning, including Gaussian Processes [31], Bayesian optimization [27], and Gaussian Graphical models [24].
4.2 Testing a core feature set
The proposed iterative algorithm terminates once it determines whether a subset  $R$  of the sensitive feature set  $S$  constitutes a core feature set. This validation process falls into two scenarios:
1. When  $\delta = 0$ : To confirm that  $R$  is a pure core feature set, it is sufficient to verify that  $f_{\theta}(X_U,X_R = x_R)$  remains constant for all possible realizations of  $X_U$ . As we will show in Section 5, linear classifiers can perform this check in linear time without making any specific assumptions about the input distribution.
2. When  $\delta > 0$ : In this case, the property above is no longer valid. As per Definition 1, to confirm a core feature set, it is essential to estimate the distribution of  $\operatorname{Pr}(\tilde{f}_{\theta}(X_U, X_R = x_R))$ . In Section 5, we demonstrate how to analytically estimate this distribution for linear classifiers. Furthermore, in Section 6, we illustrate how to locally approximate this distribution for nonlinear classifiers and derive a simple yet effective estimator that can be readily implemented in practice.
5 MinDRel for linear classifiers
This section will devote to estimating the distribution  $\operatorname*{Pr}(\tilde{f}_{\theta}(X_{j} = z,X_{U\setminus \{j\}},X_{R} = x_{R}))$  , simply expressed as  $\operatorname *{Pr}(\tilde{f}_{\theta}(X_U,X_R = x_R))$  and provides an instantiation of MinDRel for linear classifiers.
In particular, it shows that both the estimation of the conditional distributions required to compute the scoring function  $F(X_{j})$  and the termination condition to test whether a set of revealed features is a core feature set, can be computed efficiently. This is an important property for the developed algorithms, which are designed to be online and interactive.
5.1 Efficiently Estimating  $\operatorname*{Pr}(\tilde{f}_{\theta}(X_U,X_R = x_R))$
For a linear classifier  $\tilde{f}_{\theta} = \theta^{\top}x$ , notice that when the input features are jointly Gaussian, the model predictions  $\tilde{f}_{\theta}(x)$  are also Gaussian, as highlighted by the following result.
Proposition 4. The model soft prediction,  $\tilde{f}_{\theta}(X_U,X_R = x_R) = \theta_U X_U + \theta_R x_R$  is a random variable following a Gaussian distribution  $\mathcal{N}\big(m_f,\sigma_f^2\big)$ , with
$$
m _ {f} = \theta_ {R} x _ {R} + \theta_ {U} ^ {\top} \left(\mu_ {U} + \Sigma_ {U R} \Sigma_ {R R} ^ {- 1} \left(x _ {R} - \mu_ {R}\right)\right) \tag {5}
$$
$$
\sigma_ {f} ^ {2} = \theta_ {U} ^ {\top} \left(\Sigma_ {U U} - \Sigma_ {U R} \Sigma_ {R R} ^ {- 1} \Sigma_ {R U}\right) \theta_ {U}, \tag {6}
$$
where  $\theta_U$  is the sub-vector of parameters  $\theta$  corresponding to the unrevealed features  $U$ .
The result above is used to assist in calculating the conditional distribution of the model hard predictions  $f_{\theta}(x)$ , following thresholding. This is a random variable that adheres to a Bernoulli distribution, as shown next, and will be used to compute the entropy of the model predictions, as well as to determine if a subset of features constitutes a core set.
Proposition 5. Let the soft model predictions  $f_{\theta}(X_U, X_R = x_R)$  be a random variable following a Gaussian distribution  $\mathcal{N}(m_f, \sigma_f^2)$ . Then, the model prediction following thresholding  $f_{\theta}(X_U, X_R = x_R)$  is a random variable following a Bernoulli distribution  $\mathcal{B}(p)$  with  $p = \Phi(\frac{m_f}{\sigma_f})$ , where  $\Phi(\cdot)$  is the CDF of the standard Normal distribution, and  $m_f$  and  $\sigma_f$ , are given in Eqs (5) and (6), respectively.
5.2 Testing pure core feature sets
In this subsection, we outline the methods for determining if a subset  $U$  is a pure core feature set, and, if so, identifying its representative label. As per Definition 1,  $U$  is a pure core feature set if  $f_{\theta}(X_U,X_R = x_R) = \tilde{y}$  for all  $X_U$ . This implies that  $\tilde{f}_{\theta}(X_U,X_R = x_R) = \theta_U^\top X_U + \theta_R^\top x_R$  must have the same sign for all  $X_U$  in the range of  $[-1,1]^{|U|}$ . Given the box constraint  $X_U\in [-1,1]^{|U|}$ , the linearity of the model considered allows us to directly compute the maximum and minimum values of  $\tilde{f}_{\theta}(X_U,X_R = x_R)$ , rather than enumerating all possible values. Specifically, we have:
$$
\max  _ {X _ {U}} \theta_ {U} ^ {\top} X _ {U} + \theta_ {R} ^ {\top} x _ {R} = \| \theta_ {U} \| _ {1} + \theta_ {R} ^ {\top} x _ {R}
$$
$$
\min _ {X _ {U}} \theta_ {U} ^ {\top} X _ {U} + \theta_ {R} ^ {\top} x _ {R} = - \| \theta_ {U} \| _ {1} + \theta_ {R} ^ {\top} x _ {R}.
$$
Thus, if both these maximum and minimum values are negative (non-negative), then  $U$  is considered a pure core feature set with representative label  $\tilde{y} = 0$  ( $\tilde{y} = 1$ ). If not,  $U$  is not a pure core feature set.
Importantly, determining whether a subset  $R$  of sensitive features  $S$  constitutes a pure core feature set can be accomplished in linear time with respect to the number of features.
Proposition 6. Assume  $f_{\theta}$  is a linear classifier. Then, determining if a subset  $U$  of sensitive features  $S$  is a pure core feature set can be performed in  $O(|P| + |S|)$  time.
5.3 MinDRel-linear Algorithm and Evaluation
A pseudo-code of MinDRel specialized for linear classifiers is reported in Algorithm 1. At inference time, the algorithm takes as input a sample  $x$  (which only exposes the set of public features  $x_{P}$ ) and uses the training data  $D$  to estimate the mean and covariance matrix needed to compute the condi-
nal distribution of the model predictions given the unrevealed features (line 1), as discussed above. After initializing empty the set of revealed features (line 2), it iteratively releases a feature at a time until a core feature set (and its associated representative label) are determined, as discussed in detail in Section 5.2. The released feature  $X_{j^{*}}$  is the one, among the unrevealed features  $U$ , that maximizes the scoring function  $F$  (line 12). Computing such a scoring function entails estimating the conditional
distribution  $\operatorname*{Pr}(X_j|X_R = x_R)$  (line 8), constructing a sample set  $Z$  from such distribution (line 9), and approximating the distribution over soft model predictions through Monte Carlo sampling to compute (line 10). Finally, after each iteration, the algorithm updates the set of the revealed and unrevealed features (line 13).
Notice that MinDRel relies on estimating the mean vector and covariance matrix from the training data, which is considered public, for the scope of this paper. If the training data is private, various techniques exist to release DP mean, and variance [22, 7] and can be readily adopted. Nonetheless, the protection of training data is beyond the focus of this work.
Figure 4 reports the cumulative count of users (y-axis) against the minimum number of features they need to disclose (x-axis) to ensure confident predictions, for various failure probabilities  $\delta$ . The model adopted is a Logistic Regression classifier trained on the Bank dataset [9]. The data minimization achieved by MinDRel is clearly apparent. For each
test sample, MinDRel identifies core feature sets that are significantly smaller than the total sensitive feature set size  $|S| = 7$ . Interestingly, when  $\delta > 0$ , it discovers core feature sets smaller than 2 for the majority of users. This implies that most users would only need to reveal a small portion of their sensitive data to achieve accurate model predictions with either absolute certainty or high confidence.
Algorithm 1: MinDRel for linear classifiers
input :A test sample  $x$  training data  $\overline{D}$
output:A core feature set  $R$  and its representative label  $\tilde{y}$
$$
1 (\mu , \Sigma) \leftarrow \left(\frac {1}{| D |} \sum_ {(x, y) \in D} x, \frac {1}{| D |} \sum_ {(x, y) \in D} (x - \mu) (x - \mu) ^ {\top}\right)
$$
2 Initialize  $R = \emptyset$
3 while True do
if  $R$  is a core feature set with repr. label  $\tilde{y}$  then return  $(R,\tilde{y})$    
else   
foreach  $j\in U$  do Compute  $\operatorname *{Pr}(X_j|X_R = x_R)$  (using Prop. 3) Z  $\leftarrow$  sample  $(\mathrm{Pr}(X_j|X_R = x_R))$  T times   
10 Compute  $\operatorname *{Pr}\left(f_{\theta}(X_{j} = z,X_{U\setminus \{j\}}X_{R} = x_{R})\right)$  (using Prop. 4 and 5)   
11 Compute  $F(X_{j})$  (using Eq. (4))
12  $j^{*}\gets \operatorname {argmax}_{j}F(X_{j})$
13  $(R,U)\gets R\cup \{j^{*}\} ,U\setminus \{j^{*}\}$
6 MinDRel for non-linear classifiers
Next, the paper focuses on computing the estimate  $\operatorname*{Pr}(\tilde{f}_{\theta}(X_U,X_R = x_R))$  and determining core feature sets when  $f_{\theta}$  is a nonlinear classifier. Then, the section presents results that illustrate the practical benefits of MinDRel for data minimization at inference time on neural networks.
Figure 4: Cumulative count of users against their core feature set size for various  $\delta$ .
6.1 Efficiently estimating  $Pr(\tilde{f}_{\theta}(X_U, X_R = x_R))$
First notice that even if the input features  $x$  are jointly Gaussian, the outputs  $f_{\theta}(x)$  will no longer adhere to a Gaussian distribution after a non-linear transformation.
This complicates estimating the distribution  $\operatorname{Pr}(\tilde{f}_{\theta}(X_U, X_R = x_R)$ . To tackle this challenge, the paper takes a local approximation of the model predictions  $\tilde{f}_{\theta}$  using a Gaussian distribution, as demonstrated in the following result.
Theorem 1. The distribution of the random variable  $\tilde{f}_{\theta} = \tilde{f}_{\theta}(X_U, X_R = x_R)$  where  $X_U \sim \mathcal{N}\left(\mu_U^{pos}, \Sigma_U^{pos}\right)$  can be approximated by a Normal distribution as
$$
\tilde {f} _ {\theta} \sim \mathcal {N} \left(\tilde {f} _ {\theta} \left(X _ {U} = \mu_ {U} ^ {p o s}, X _ {R} = x _ {R}\right), g _ {U} ^ {\top} \Sigma_ {U} ^ {p o s} g _ {U}\right), \tag {7}
$$
where  $g_{U} = \nabla_{X_{U}}\tilde{f}_{\theta}\big(X_{U} = \mu_{U}^{pos},X_{R} = x_{R}\big)$  is the gradient of model prediction at  $X_{U} = \mu_{U}^{pos}$ .
Therein, the mean vector  $\mu_U^{\mathrm{pos}}$  and covariance matrix  $\Sigma_U^{\mathrm{pos}}$  of  $Pr(X_U|X_R = x_R)$  are derived from Proposition 3. This result leverages a first-order Taylor approximation of model  $f_{\theta}$  around its mean.
6.2 Testing pure core feature sets
Unlike linear classifiers, the case for nonlinear models lacks an exact and efficient method to determine whether a set is a core feature set. This is primarily due to the non-convex nature of the adopted models, which poses challenges in finding a global optimum. This section thus proposes an approximate testing routine and demonstrates its practical ability to significantly minimize data leakage during testing while maintaining high accuracy.
To determine if a subset  $U$  of the sensitive features  $S$  is a pure core feature set, we consider a set  $Q$  of input points  $(X_U, x_R)$ . The entries corresponding to the revealed features are set to the value  $x_R$ , while the entries corresponding to the unrevealed features are sampled from the distribution  $\operatorname{Pr}(X_U | X_R = x_R)$ . The test verifies if the model predictions  $f_\theta(x)$  remain constant for all  $x$  in  $Q$ . In the next section, we will show that even considering arbitrary classifiers (e.g., we use standard neural networks), MinDRel can reduce data leakage dramatically when compared to standard approaches. The MinDRel algorithm for nonlinear classifiers differs from Algorithm 1 primarily in the method used to compute the estimates for the distribution  $\operatorname{Pr}(f_\theta(X_j = z, X_{U \setminus j}, X_R = x_R))$  of the soft model predictions (line 11). In this case, this estimate is computed by leveraging the results from Theorem 1 and Proposition 5. Moreover, the termination test of the algorithm is based on the discussion in the previous section. Appendix B reports a description of the algorithm's pseudocode.
Datasets and settings. This section evaluates MinDRel's effectiveness in limiting data exposure during inference. The experiments are conducted on six standard UCI datasets [9]. To further emphasize the benefits of MinDRel, we compare it to two baselines: the optimal model, which employs a brute force method to find the smallest core feature set and its representative label and assumes all sensitive features are known, and the all-features model, which simply adopts the original classifiers using all the data features for each test sample. The performances are displayed for a varying number of sensitive attributes  $|S| \in [2,5]$ , while we delegate a study for larger  $|S|$  to the Appendix (which excludes the time-consuming baseline optimal, as intractable for large  $|S|$ ). For each choice of  $|S|$ , we randomly select  $|S|$  features from the entire feature set and designate them as sensitive attributes. The remaining attributes are considered public. The average accuracy and data leakage are then reported based on 100 random sensitive attribute selections. Due to page limit, we present a selection of the results and discuss their trends on Bank dataset. A comprehensive overview, additional analysis, and experiments are available in the Appendix D.
7.1 Linear classifiers
Figure 5 depicts performance results in accuracy (top subplots/higher is better) and relative data leakage (bottom subplots/ lower is better) with varying number of relevable sensitive features  $|S|$ . The comparison includes three MinDRel versions:  $F$ -Score, which utilizes the scoring function elaborated in Section 4.1 to select the next feature to disclose (left); Importance, which employs a feature importance criterion leveraging the model  $f_{\theta}$  parameters to rank features (middle), detailed in the Appendix; and Random, which arbitrarily selects the next feature to reveal (right). All three versions use the same test procedure to validate whether a set qualifies as a
Figure 5: Comparison of F-Score, Importance and Random method for different  $\delta$  with two baseline methods: Optimal and All features in term of Accuracy and Data Leakage. The underlying classifier is a logistic regression classifier.
Firstly, notice that MinDRel achieves equal or better accuracy than the optimal mechanism and baseline that utilizes all features during testing. This suggests that data minimization can be accomplished under linear models without compromising accuracy! Next, observe that an increase in  $\delta$
aids in safeguarding data minimization, as evidenced by the drop in relative data leakage (note the logarithmic scale of the y-axis). This is attributable to the influence of  $\delta$  on the test for identifying a core feature set, thereby reducing its size.
Finally, the proposed scoring function outperforms other versions in terms of data leakage minimization, allowing users to disclose substantially fewer sensitive features. The Appendix also includes experiments with larger quantities of sensitive features, presenting analogous trends, where, however, a comparison against an optimal baseline was not possible in due to its exponential time complexity.
7.2 Non-linear classifiers
To assess MinDRel's performance in reducing data leakage when employing standard nonlinear classifiers, we use a neural network with two hidden layers and ReLU activation functions and train the models using stochastic gradient descent (see Appendix D for additional details). The evaluation criteria, baselines, and benchmarks adhere to the same parameters set in Section 5.3.
Figure 6 showcases the results in terms of accuracy (top subplots) and data leakage (bottom subplots). Unlike linear classifiers, nonlinear models using MinDRel with a failure probability of  $\delta = 0$  cannot guarantee the
Figure 6: Comparison of F-Score, Importance and Random method for different  $\delta$  with two baseline methods: Optimal and All features in term of Accuracy and Data Leakage. The underlying classifier is a neural network.
same level of accuracy as the "all features" baseline model. However, this accuracy difference is minimal. Notably, a failure probability of  $\delta = 0$  enables users to disclose less than half, and up to  $90\%$  fewer sensitive features across different datasets, while achieving accuracies similar to those of conventional classifiers. Next, similarly to as observed in the previous section, MinDRel with the proposed F-score selector significantly outperforms the other variants in terms of data leakage minimization. Furthermore, when considering higher failure probabilities, data leakage decreases significantly. For instance, with  $\delta \leq 0.1$ , users need to disclose only  $5\%$  of their sensitive features while maintaining accuracies comparable to the baseline models (the largest accuracy difference reported was  $0.005\%$ ). These results are significant: They show that the introduced notion of privacy leakage and the proposed algorithm can become a valuable tool to protect individuals' data privacy at test time, without significantly compromising accuracy..
This paper introduced the concept of data minimization at test time whose goal is to minimize the number of features that individuals need to disclose during model inference while maintaining accurate predictions from the model. The motivations of this notion are grounded in the privacy risks imposed by the adoption of learning models in consequential domains, and align with the data minimization principle. The paper then discusses an iterative and personalized algorithm that selects the features each individual should release with the goal of minimizing data leakage while retaining exact (in the case of linear classifiers) or high (for non-linear classifiers) accuracy. Experiments over a range of benchmarks and datasets indicate that individuals may be able to release as little as  $10\%$  of their information without compromising the accuracy of the model, providing a strong argument for the effectiveness of this approach in protecting privacy while preserving the accuracy of the model.
While this study is the first attempt at defining data minimization during inference, it also opens up avenues for further research. First, establishing bounds on the data leakage provided by our proposed method compared to an optimal procedure presents an interesting and open challenge. Second, exploring the relationship between data minimization principles and their consequent disparate impacts presents another open direction. Lastly, developing effective algorithms to provably construct core feature sets for non-linear classifiers is another important area of investigation.
This research was partially supported by NSF grant 2133169, a Google Research Scholar Award and an Amazon Research Award.
[1] California privacy rights act. https://cppa.ca.gov/. Accessed: 2023-10-23.
[2] General data protection regulation. https://gdpr-info.eu/. Accessed: 2023-10-23.
[3] General personal data protection law.
[4] Personal information protection act. https://elaw.klri.re.kr/engservice/lawView.do?hseq=53044&lang=ENG. Accessed: 2023-10-23.
[5] Protection of personal information act. https://popia.co.za/. Accessed: 2023-10-23.
[6] Abadi and et al. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016.
[7] K. Amin, T. Dick, A. Kulesza, A. Munoz, and S. Vassilvitskii. Differentially private covariance estimation. Advances in Neural Information Processing Systems, 32, 2019.
[8] C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
[9] C. Blake and C. Merz. Uci repository of machine learning databases, 1988.
[10] G. Chandrashekar and F. Sahin. A survey on feature selection methods. Computers & Electrical Engineering, 40(1):16-28, 2014.
[11] C.-H. Chang, M. Mai, and A. Goldenberg. Dynamic measurement scheduling for event forecasting using deep rl. pages 951-960, 2019.
[12] K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 2011.
[13] G. Contardo, L. Denoyer, and T. Artières. Sequential cost-sensitive feature acquisition. pages 284-294, 2016.
[14] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265-284. Springer, 2006.
[15] Y. Fu, X. Zhu, and B. Li. A survey on instance selection for active learning. Knowledge and information systems, 35(2):249-283, 2013.
[16] A. Krause and C. Guestrin. A note on the budgeted maximization of submodular functions. Citeseer, 2005.
[17] R. Krishnan, P. Esposito, and M. Subedar. Bayesian-torch: Bayesian neural network layers for uncertainty estimation, Jan. 2022.
[18] R. Krishnan and O. Tickoo. Improving model calibration with accuracy versus uncertainty optimization. In Advances in Neural Information Processing Systems, volume 33, pages 18237-18248, 2020.
[19] Larxel. Children fetal health dataset. https://www.kaggle.com/datasets/andrewmvd/fetal-health-classification, 2021.
[20] J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang, and H. Liu. Feature selection: A data perspective. ACM computing surveys (CSUR), 50(6):1-45, 2017.
[21] Y. Li and J. Oliva. Active feature acquisition with generative surrogate models. pages 6450-6459, 2021.