Title: Data Minimization at Inference Time
Venue: NeurIPS 2023
Accepted: yes
Keywords: Privacy; data minimization
Abstract: In high-stakes domains such as legal, banking, hiring, and healthcare, learning models frequently rely on sensitive user information for inference, necessitating the complete set of features. This not only poses significant privacy risks for individuals but also demands substantial human effort from organizations to verify information accuracy. 
This study asks whether it is necessary to use all input features for accurate predictions at inference time. The paper demonstrates that, in a personalized setting, individuals may only need to disclose a small subset of features without compromising decision-making accuracy. The paper also provides an efficient sequential algorithm to determine the appropriate attributes for each individual to provide. Evaluations across various learning tasks show that individuals can potentially report as little as 10\% of their information while maintaining the same accuracy level as a model that employs the full set of user information.

Data Minimization at Inference Time

Abstract

In domains with high stakes such as law, recruitment, and healthcare, learning models frequently rely on sensitive user data for inference, necessitating the complete set of features. This not only poses significant privacy risks for individuals but also demands substantial human effort from organizations to verify information accuracy. This paper asks whether it is necessary to use all input features for accurate predictions at inference time. The paper demonstrates that, in a personalized setting, individuals may only need to disclose a small subset of their features without compromising decision-making accuracy. The paper also provides an efficient sequential algorithm to determine the appropriate attributes for each individual to provide. Evaluations across various learning tasks show that individuals can potentially report as little as 10% of their information while maintaining the same accuracy level as a model that employs the full set of user information.

1 Introduction

The remarkable success of machine learning (ML) models also brought with it pressing challenges at the interface of privacy and decision-making,especially when deployed in consequential domains such as legal processes, banking, hiring, and healthcare [23]. A particularly intriguing aspect is the conventional requirement for users to disclose their entire set of features during inference, thereby creating a gateway for potential data breaches, as exemplified in recent instances where millions of individuals' data were compromised []. Concurrently, this practice also places a burden on companies and organizations to ensure the accuracy and legal compliance of the disclosed information, as often observed in financial operations, mandated by legislations like the Corporate Transparency Act[].

Significantly, this conventional approach of disclosing the enture feature set during inference might also also violate the data minimization principle, a cornerstone of several global privacy regulations including the Eurpean General Data Protection Regulation [2], the California Privacy Rights Act [1], and the Brazilian General Data Protection Law [3], among others. Through this lens, the discourse on enhancing privacy and reducing the verification onus in ML systems, particularly in crucial decision-making domains, gains a nuanced dimension, warranting a thorough examination.

This paper challenges this setting and asks whether it is necessary to require all input features for a model to produce accurate or nearly accurate predictions during inference. We refer to this question as the data minimization for inference problem. This unique question bears profound implications for privacy in model personalization, which often necessitates the disclosure of substantial user data. We show that, under a personalized setting, each individual may only need to release a small subset of their features to achieve the same prediction errors as those obtained when all features are disclosed. The overall framework is depicted in Figure 1. Following this result, we also provide an efficient sequential algorithm to identify the minimal set of attributes that each individual should reveal. Evaluations across various learning tasks indicate that individuals may be able to report as little as 10% of their information while maintaining the same accuracy level as a model using the complete set of user information.

At inference time

x

1

x

2

x

n

F

1

F

2

F

p

…

…

x

1

x

2

x

n

F

1

F

2

F

p

…

…

data

minimization

Core feature set (R)

˜

y

f

θ

representative

label

x

R

x

In summary, the paper makes the following contributions: (1) it initiates a study to analyze the optimal subset of data features that each individual should disclose at inference time in order to achieve the same accuracy as if all features were disclosed; (2) it links this analysis to a new concept of data minimization for inference in relation to privacy, (3) it proposes theoretically motivated and efficient algorithms for determining the minimal set of attributes each individual should provide to minimize their data; and (4) it conducts a comprehensive evaluation illustrating the effectiveness of the proposed methods in preserving privacy at no or small costs in accuracy.

Related work. To the best of our knowledge, this is the first work studying the connection between data minimization and accuracy at inference time. This work was motivated by the core principle in data minimization which states that 'all collected data shall be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed'. This principle is adopted as a privacy cornestone in different data protection regulations, including the General Data Protection Regulation [2] in Europe, the California Privacy Rigths Act in the US [1], the General Personal Data Protection Law in Brasil [3], the Protection of Personal Information Act, in South Africa [5], and the Personal Information Protection Act, in South Korea [4].

While we are not aware of studies on data minimization for inference problems, we draw connections with differential privacy, feature selection, and active learning. Differential Privacy (DP) [14] is a strong privacy notion that determines and bounds the risk of disclosing sensitive information of individuals participating in a computation. In the context of machine learning, DP ensures that algorithms can learn the relations between data and predictions while preventing them from memorizing sensitive information about any specific individual in the training data. In such a context, DP is primarily adopted to protect training data [6, 12, 32] and thus the setting contrasts with that studied in this work, which focuses on identifying the superfluous features revealed by users at test time to attain high accuracy. Furthermore, achieving tight constraints in differential privacy often comes at the cost of sacrificing accuracy, while the proposed privacy framework can reduce privacy loss without sacrificing accuracy under the assumption of linear classifiers.

Feature selection [10] is the process of identifying a relevant subset of features from a larger set for use in model construction, with the goal of improving performance by reducing the complexity and dimensionality of the data. The problem studied in this work can be considered a specialized form of feature selection with the added consideration of personalized levels, where each individual may use a different subset of features. This contrasts standard feature selection [20], which selects the same subset of features for each data sample. Additionally, and unlike traditional feature selection, which is performed during training and independent of the deployed classifier [10], the proposed framework performs feature selection at deployment time and is inherently dependent on the deployed classifier.

The framework proposed in this paper shares some similarities with active learning [15, 26, 33], whose goal is to iteratively select samples for experts to label in order to construct an accurate classifier with the least number of labeled samples. Similarly, the proposed framework iteratively asks individuals to reveal one attribute given their released features so far, with the goal of minimizing the uncertainty in model predictions. Finally, our study share some connections with active feature acquisition [28, 13, 11, 21] which can be categorized as one branch in active learning. However, these priors focus mostly on minimizing the total number of users features at training step. This is in sharp contrast with ours work which concentrate on inferece time.

<latexi sh

1_b

64="VOv f+CAnEJXwD

Hdu

z

0

>

c

Z

LS

M

FI

K

p

g

2

G

W

9o

5

k

m

j

P

y

U

T

rq

N

7

Y

Q8

3

R

<latexi sh

1_b

64="yQg cY

qjB

K

/

+

L

XPm

TI0

>A

3

VC7S

N

F

rM8Z

J

W

d

p

z

E

wD

u

5

O

U

n

R

v

G

o

9

k

f

H

2

<latexi sh

1_b

D

JA

qF

8k

V

rL

>

c

C7SgN

0

XzG+o

Y2

v

R

WUE

OT

I

yM

u

K

n

Z

H

5

f

Q

m

<latexi sh

1_b

64="N7CHwWR

j

Qz

50

/E

BF9A

>

3

c

V

L

rM8ZX

Jm

Y

d

p

D

u

T

O

+

P

U

K

n

v

y

G

o

k

I

q

2

f

<latexi sh

1_b

64="Mp

7

Ef

XJcvdgu/L

OzTD

5w

>A

B+n

V

S

F

W

q

I

y

j

r

Q

Z

kG

3

0

2

m

8

K

H

U

R

Y

No

C

P

9

<latexi sh

1_b

64="wzA/WZOn

Ug

IVvEQR

8jX+J

>

B

c

DLS

M

F

T

7

q

d

0

H

om

k

u

9

3P

Y

2

r

G

5

y

K

p

N

f

C

<latexi sh

1_b

64="koQY

fj u2

H

RX/

8m

NpI

>A

B+n

c

VDLSgM

F

T

7W

q

d

v

y

r

Z

J

G

3

0

w

z

E

O

5

K

U

C

P

9

<latexi sh

1_b

64="yQg cY

qjB

K

/

+

L

XPm

TI0

>A

3

VC7S

N

F

rM8Z

J

W

d

p

z

E

wD

u

5

O

U

n

R

v

G

o

9

k

f

H

2

<latexi sh

1_b

D

JA

qF

8k

V

rL

>

c

C7SgN

0

XzG+o

Y2

v

R

WUE

OT

I

yM

u

K

n

Z

H

5

f

Q

m

<latexi sh

1_b

64="N7CHwWR

j

Qz

50

/E

BF9A

>

3

c

V

L

rM8ZX

Jm

Y

d

p

D

u

T

O

+

P

U

K

n

v

y

G

o

k

I

q

2

f

<latexi sh

1_b

64="Mp

7

Ef

XJcvdgu/L

OzTD

5w

>A

B+n

V

S

F

W

q

I

y

j

r

Q

Z

kG

3

0

2

m

8

K

H

U

R

Y

No

C

P

9

<latexi sh

1_b

64="wzA/WZOn

Ug

IVvEQR

8jX+J

>

B

c

DLS

M

F

T

7

q

d

0

H

om

k

u

9

3P

Y

2

r

G

5

y

K

p

N

f

C

<latexi sh

1_b

64="koQY

fj u2

H

RX/

8m

NpI

>A

B+n

c

VDLSgM

F

T

7W

q

d

v

y

r

Z

J

G

3

0

w

z

E

O

5

K

U

C

P

9

<latexi sh

1_b

64="I

OW2

fPDLJrNn

YEBQ

U

K

>A

c

V

X

Cj

d

8R

Mo

T

Z

09

z

/y5

Hu

v

w

m

p

7

k

F

+

G

q

3

<latexi sh

1_b

64="ZR3m

CUV

nFw uHQBTEX

/

0

>A

7

c

DLSgN

Oy

r

f

Y9

oP

K

2

8

I5

J

d

p

v

k

G

jq

W

+

M

z

<latexi sh

1_b

64="mJr cKO

E

3uZ

G

Qkf0d7

>A

B8H

VDLS

N

F

U

2vq

g

w

R

y

n

P

T

X+

/

o

M

9

W

z

j

Y

C

5

p

I

2 Settings and objectives

We consider a dataset D consisting of samples ( x, y ) drawn from an unknown distribution Π . Here, x ∈ X is a feature vector and y ∈ Y = [ L ] is a label with L classes. The features in x are categorized into public x P and sensitive x S features, with their respective indexes in vector x denoted as P and S , respectively. We consider classifiers f θ : X → Y , which are trained on a public dataset from the same data distribution Π above. The classifier produces a score ˜ f θ ( x ) ∈ R L over the classes and a final output class, f θ ( x ) ∈ [ L ] , given input x . The model's outputs f θ ( x ) and ˜ f θ ( x ) are also often referred to as hard and soft predictions, respectively.

Without loss of generality, we assume that all features in X lie within the range [ -1 , 1] . In this setting, we are given a trained model f θ and, at inference time, we have access to the public features x P . These features might be revealed through user queries or collected by the provider during previous interactions. Our focus is on the setting where | S | ≪| D | , and for simplicity, the following considers binary classifiers, where Y = { 0 , 1 } and ˜ f θ ∈ R . Multi-class settings are addressed in Appendix C.

In this paper, the term data leakage of a model, refers to the percentage of sensitive features that are revealed unnecessarily, meaning that their exclusion would not significantly impact the model's output. Our goal is to design algorithms that accurately predict the output of the model using the smallest possible number of sensitive features , thus minimizing the data leakage at inference time . This objective reflects our desire for privacy.

To clarify key points discussed in the paper, let us consider a loan approval task where individual features are represented by the set { Job , Loc(action) , Inc(ome) } . In this example, the Job feature ∈ x P is public, whereas Loc and Inc ∈ x S are sensitive. We also consider a trained linear model f θ = 1 . 0 Job -0 . 5 Loc +0 . 5 Inc ≥ 0 and look at a scenario where user (A) has a public feature Job = 1 . 0 , and user (B) has a public feature Job = -0 . 9 . Both users' sensitive feature values are unknown. However, for user A, the outcome can be conclusively determined without revealing any additional information since all features are bounded within [ -1 , 1] . In contrast, for user B, the outcome cannot be determined solely based on the public feature, but revealing the sensitive feature Loc = 1 . 0 is enough to confirm the classifier outcome.

This example highlights two important observations that motivate our study: (1) not all sensitive attributes may be required for decision-making during inference , and (2) different individuals may need to disclose different amounts and types of sensitive information for decision-making .

3 Core feature sets

With these considerations in mind, this section introduces the notion of core feature set , the first contribution of the paper , which will be used to quantify data minimization. The paper presents the key findings and defers all proofs in Appendix A.

Throughout the paper, the symbols R and U are used to represent the sets of indices for revealed and unrevealed features of the sensitive attribute S , respectively. Given a vector x and an index set I , we use x I to denote the vector containing entries indexed by I and X I to represent the corresponding random variable. Finally, we write f θ ( X U , X R = x R ) as a shorthand for f θ ( X U , X R = x R , X P = x P ) to denote the prediction made by the model when the features in U are unrevealed.

Our objective is to develop algorithms that can identify the smallest subset of sensitive features to disclose , ensuring that the model's output is accurate (with high probability) irrespective of the values of the undisclosed features. We refer to this subset as the core feature set .

Definition 1 (Core feature set) . Consider a subset R of sensitive features S , and let U = S \ R be the unrevealed features. The set R is a core feature set if, for some ˜ y ∈ Y ,

where δ ∈ [0 , 1] is a failure probability.

When δ = 0 the core feature set is called pure . Additionally, the label ˜ y satisfying Equation (1) is called the representative label for the core feature set R . The concept of the representative label ˜ y is crucial for the algorithms that will be discussed later. These algorithms use limited information to

make predictions and when predictions are made using a set of unrevealed features, the representative label ˜ y will be used in place of the model's prediction.

In identifying core feature sets to minimize data leakage, it's crucial to consider model uncertainty, which refers to the unknown values of unrevealed features. The following result links core feature sets with model entropy, which measures uncertainty and is used by this work to minimize data leakage.

Proposition 1. Let R ⊆ S be a core feature set with failure probability δ < 0 . 5 . Then, there exists a monotonic decreasing function ϵ : R + → R + with ϵ (1) = 0 such that:

where H [ Z ] = -∑ z ∈ [ L ] Pr( Z = z ) log Pr( Z = z ) is the entropy of the random variable Z .

This property highlights the relationship between core feature sets and entropy associated with a model using incomplete information . As the δ value decreases, the model's predictions become more certain. When δ equals zero (or when R represents a pure core feature set), the model's predictions can be fully understood without observing x U , resulting in entropy of 0 .

It is worth noticing that enhancing prediction accuracy necessitates revealing additional information, as illustrated by the previous result and the renowned information theoretical proposition below:

Proposition 2. Given two subsets R and R ′ of sensitive features S , with R ⊆ R ′ ,

Thus, the parameter δ plays a crucial role in balancing the trade-off between privacy loss and model prediction's uncertainty . It determines the amount of sensitive information that must be disclosed to have enough confidence on model predictions. As δ increases, fewer sensitive features need to be revealed, resulting in reduced data leakage but also less certainty on model predictions, and vice versa. Note that disclosing more sensitive attributes does not necessarily improve the accuracy of predictions for a specific individual , as highlighted in previous research [30]. Although it's chal-

0

2

4

6

Min pure core feat set size

0.0

0.5

1.0

Cumulative frequency

lenging to rigorously analyze the relationship between privacy loss and predictive accuracy across various contexts, our empirical studies across multiple datasets and learning algorithms do suggest that exposing more sensitive features generally enhances prediction accuracy. Thus, the parameter δ appears to govern the trade-off between sacrificing privacy and gaining predictive performance.

As highlighted in the previous example, the core feature set is not uniform for all users. This is further exemplified in Figure 2, using the Credit dataset [9] with a logistic regression classifier. The figure reports the cumulative count of users against the minimum number of features they need to disclose to ensure confident predictions. It demonstrates that many individuals need to disclose no additional information to attain accurate predictions (corresponding to a pure feature of set size 0 ), and most individuals can achieve accurate predictions by disclosing only ≤ 2 sensitive features. These insights, together with the previous observations linking core feature sets to entropy, motivate the proposed online algorithm, the second contribution of the paper.

4 MinDRel: An algorithm to minimize data release at inference time

The goal of the proposed algorithm, called Minimize Data Reveal (MinDRel), is to uphold privacy during inference by revealing sensitive features one at a time based on their released feature values. This section provides a high-level description of the algorithm and outlines its challenges. Next, Section 5, applies MinDRel to linear classifiers and discusses its performance on several datasets and benchmarks. Further, Section 6, extends MinDRel to non-linear classifiers and considers an evaluation over a range of standard datasets.

Overview of MinDRel. MinDRel operates fundamentally on two critical actions: 1. determining the next feature to reveal for each user and 2. verifying whether the disclosed features make up a core feature set for that user. These two operations will be discussed in sections 4.1 and 4.2, respectively.

The algorithm determines which feature to disclose for a specific user by inspecting the posterior probabilities Pr( X j | X R = x R , X P = x P ) for each unrevealed feature j ∈ U , taking into account the disclosed sensitive features x R and public features x P . Given the current set of disclosed features x R and unrevealed features x U , MinDRel chooses the subsequent feature j ∈ U as follows:

where F is a scoring function that evaluates the amount of information that can be acquired about the model's predictions when feature X j is disclosed. As suggested in previous sections, it's desirable to reveal the feature that provides the most insight into the model prediction upon disclosure. MinDRel uses Shannon entropy for this purpose as it offers a natural method for quantifying information. Once feature X j is disclosed with a value of x j , the algorithm updates the posterior probabilities for all remaining unrevealed features. The process concludes either when all sensitive features have been disclosed or when a core feature set has been identified. It should also be noted that, within this framework, there is no need to perform data imputation when some features are missing. Unrevealed features are treated as random variables and are integrated during the prediction process.

Both the computation of the scoring function F and the verification of whether a set of disclosed features constitute a core feature set present two significant challenges for the algorithm. The rest of the section delves into these difficulties.

4.1 Computing the scoring function F

Designing a scoring function F that measures how confident a model's prediction is when a user discloses an additional feature X j brings up two key challenges. First , the value of X j is unknown until the decision to reveal it is made , which complicates the computation of the entropy function. Second , even if the value of X j were known, determining the entropy of model predictions in an efficient manner is another difficulty. We next discuss how to overcome these challenges.

Dealing with unknown values. To address the first challenge, we exploit the information encoded in the disclosed features to infer X j 's value and compute the posterior probability Pr( X j | X R = x R ) of the unrevealed feature X j given the values of the revealed ones. The scoring function, abbreviated as F ( X j ) , can thus be modeled as the expected negative entropy given the randomness of X j ,

where z ∈ X j is a value in the support of X j .

Efficient entropy computation. The second difficulty relates to how to estimate this scoring function efficiently. Indeed this is challenged by two key components. The first (A) is the entropy of the model's prediction given a specific unrevealed feature value, X j = z . This prediction is a function of the random variable X U \{ j } , and, due to Proposition 1, its estimation is linked to the conditional densities Pr( X U \{ j } | X R = x R , X j = z ) . The second (B) is the conditional probability Pr ( X j = z | X R = x R ) . Efficient computation of these conditional densities is discussed next.

First, we discuss a result relying on the joint Gaussian assumption of the input features. This result will be useful in providing a computationally efficient method to estimate such conditional density functions. In the following, Σ IJ represents a sub-matrix of size | I | × | J | of a matrix Σ formed by selecting rows indexed by I and columns indexed by J .

Proposition 3. The conditional distribution of any subset of unrevealed features U ′ ⊆ U , given the the values of released features X R = x R is given by:

where Σ is the covariance matrix.

Note that Equation (3) considers U ′ = { j } , and thus, component (B) can be computed efficiently exploiting the result above. To complete Equation (3), we need to estimate the

entropy H [ f θ ( X j = z, X U \ j , X R = x R )] (component A) for a specific instance z drawn from Pr( X j | X R = x R ) . This poses a challenge due to the non-linearity of the hard model predictions f θ adopted. To tackle this computational challenge, we first estimate component A using soft labels ˜ f θ and then apply a thresholding operator. More specifically, we first estimate Pr( ˜ f θ ( X j = z, X U \{ j } , X R = x R )) and, based on this distribution, we subsequentially estimate f θ as 1 { ˜ f θ ≥ 0 } , where 1 is the indicator function. In the following sections, we will show how to assess this estimate for linear and non-linear classifiers. Finally, by approximating the distribution over soft model predictions through Monte Carlo sampling, the score function in F ( X j ) can be computed as

where Z is a set of random samples drawn from Pr( X j | X R = x R ) and estimated through Proposition 3, which thus can be computed efficiently.

When the Gaussian assumption does not hold, one can recur to (slower) Bayesian approaches to estimate the uncertainty of unrevealed features X U given the set of revealed features X R = x R . A common approach involves treating X U as the target variable and employing a neural network to establish the mapping X U = g w ( X R ) . Utilizing Bayesian techniques [17, 18], the posterior of the network's parameter p ( w | D ) = p ( w ) p ( D | w ) can be computed initially. Based on the posterior distribution of the model's parameters w , the posterior of unrevealed features can be calculated as Pr( X U = x U | X R = x R ) = Pr w ∼ p ( w | D ) ( g w ( X R ) = x U ) . However, implementing such a Bayesian network not only significantly increases training time but also inference time. Since it is necessary to compute Pr( X U | X R = x R ) for all possible choices of U ∈ S , the number of Bayesian neural network regressors scales exponentially with | S | . Accuracy Data Leakage

Importantly, in our evaluation, the data minimization method that operates under the Gaussian assumption maintains similar decisionmaking and produces comparable outcomes to the Bayesian approach, even in cases where the Gaussian assumption is not applicable in practical settings. Figure 3 illustrates this comparison, showcasing the performance of the proposed mechanism on a real dataset (Credit dataset with

=0

=0.1

=0.2

0.800

0.805

0.810

0.815

0.820

0.825

0.830

Bayesian NN

=0

=0.1

=0.2

0.0

0.2

0.4

0.6

Bayesian NN

| S | = 5 ) concerning accuracy (higher is better) and data leakage (lower is better) across various failure probability δ values. Notice how similar is the performance of the mechanisms that either leverage the Gaussian assumption or operate without it (Bayesian NN). Importantly, the assumption of a Gaussian distribution is not overly restrictive or uncommon. In fact, it is a cornerstone in many areas of machine learning, including Gaussian Processes [31], Bayesian optimization [27], and Gaussian Graphical models [24].

4.2 Testing a core feature set

The proposed iterative algorithm terminates once it determines whether a subset R of the sensitive feature set S constitutes a core feature set. This validation process falls into two scenarios:

When δ =0 : To confirm that R is a pure core feature set, it is sufficient to verify that f θ ( X U , X R = x R ) remains constant for all possible realizations of X U . As we will show in Section 5, linear classifiers can perform this check in linear time without making any specific assumptions about the input distribution.

When δ > 0 : In this case, the property above is no longer valid. As per Definition 1, to confirm a core feature set, it is essential to estimate the distribution of Pr( ˜ f θ ( X U , X R = x R )) . In Section 5, we demonstrate how to analytically estimate this distribution for linear classifiers. Furthermore, in Section 6, we illustrate how to locally approximate this distribution for nonlinear classifiers and derive a simple yet effective estimator that can be readily implemented in practice.

5 MinDRel for linear classifiers

This section will devote to estimating the distribution Pr( ˜ f θ ( X j = z, X U \{ j } , X R = x R )) , simply expressed as Pr( ˜ f θ ( X U , X R = x R )) and provides an instantiation of MinDRel for linear classifiers.

In particular, it shows that both the estimation of the conditional distributions required to compute the scoring function F ( X j ) and the termination condition to test whether a set of revealed features is a core feature set, can be computed efficiently. This is an important property for the developed algorithms, which are designed to be online and interactive.

5.1 Efficiently Estimating Pr( ˜ f θ ( X U , X R = x R ))

For a linear classifier ˜ f θ = θ ⊤ x , notice that when the input features are jointly Gaussian, the model predictions ˜ f θ ( x ) are also Gaussian, as highlighted by the following result.

Proposition 4. The model soft prediction, ˜ f θ ( X U , X R = x R ) = θ U X U + θ R x R is a random variable following a Gaussian distribution N ( m f , σ 2 f ) , with

where θ U is the sub-vector of parameters θ corresponding to the unrevealed features U .

The result above is used to assist in calculating the conditional distribution of the model hard predictions f θ ( x ) , following thresholding. This is a random variable that adheres to a Bernoulli distribution, as shown next, and will be used to compute the entropy of the model predictions, as well as to determine if a subset of features constitutes a core set.

Proposition 5. Let the soft model predictions ˜ f θ ( X U , X R = x R ) be a random variable following a Gaussian distribution N ( m f , σ 2 f ) . Then, the model prediction following thresholding f θ ( X U , X R = x R ) is a random variable following a Bernoulli distribution B ( p ) with p = Φ( m f σ f ) , where Φ( · ) is the CDF of the standard Normal distribution, and m f and σ f , are given in Eqs (5) and (6) , respectively.

5.2 Testing pure core feature sets

In this subsection, we outline the methods for determining if a subset U is a pure core feature set, and, if so, identifying its representative label. As per Definition 1, U is a pure core feature set if f θ ( X U , X R = x R ) = ˜ y for all X U . This implies that ˜ f θ ( X U , X R = x R ) = θ ⊤ U X U + θ ⊤ R x R must have the same sign for all X U in the range of [ -1 , 1] | U | . Given the box constraint X U ∈ [ -1 , 1] | U | , the linearity of the model considered allows us to directly compute the maximum and minimum values of ˜ f θ ( X U , X R = x R ) , rather than enumerating all possible values. Specifically, we have:

Thus, if both these maximum and minimum values are negative (non-negative), then U is considered a pure core feature set with representative label ˜ y = 0 ( ˜ y = 1 ). If not, U is not a pure core feature set.

Importantly, determining whether a subset R of sensitive features S constitutes a pure core feature set can be accomplished in linear time with respect to the number of features.

Proposition 6. Assume f θ is a linear classifier. Then, determining if a subset U of sensitive features S is a pure core feature set can be performed in O ( | P | + | S | ) time.

5.3 MinDRel-linear Algorithm and Evaluation

A pseudo-code of MinDRel specialized for linear classifiers is reported in Algorithm 1. At inference time, the algorithm takes as input a sample x (which only exposes the set of public features x P ) and uses the training data D to estimate the mean and covariance matrix needed to compute the conditio- nal distribution of the model predictions given the unrevealed features (line 1), as discussed above. After initializing empty the set of revealed features (line 2), it iteratively releases a feature at a time until a core feature set (and its associated representative label) are determined, as discussed in detail in Section 5.2. The released feature X j ∗ is the one, among the unrevealed features U , that maximizes the scoring function F (line 12). Computing such a scoring function entails estimating the conditional

distribution Pr( X j | X R = x R ) (line 8), constructing a sample set Z from such distribution (line 9), and approximating the distribution over soft model predictions through Monte Carlo sampling to compute (line 10). Finally, after each iteration, the algorithm updates the set of the revealed and unrevealed features (line 13).

Notice that MinDRel relies on estimating the mean vector and covariance matrix from the training data, which is considered public, for the scope of this paper. If the training data is private, various techniques exist to release DP mean, and variance [22, 7] and can be readily adopted. Nonetheless, the protection of training data is beyond the focus of this work.

Figure 4 reports the cumulative count of users (y-axis) against the minimum number of features they need to disclose (x-axis) to ensure confident predictions, for various failure probabilities δ . The model adopted is a Logistic Regression classifier trained on the Bank dataset [9]. The data minimization achieved by MinDRel is clearly apparent. For each

Algorithm 1: MinDRel for linear classifiers

input : A test sample x ; training data D output : A core feature set R and its representative label ˜ y

1

(

µ,

Σ)

(

1

|

D

|

∑

(

x,y

)

∈

D

x,

1

|

D

|

∑

(

x,y

)

∈

D

(

x

-

µ

)(

x

-

µ

)

⊤

2

Initialize

R

=

∅

3

while

True

do

4

if

R

is a core feature set with repr. label

˜

y

then

5

return

(

R,

˜

y

)

6

else

7

foreach

j

∈

U

do

8

Compute

Pr(

X

j

|

X

R

=

x

R

)

(using Prop. 3)

9

Z

sample

(Pr(

X

j

|

X

R

=

x

R

))

T times

10

Compute

Pr

(

f

θ

(

X

j

=

z, X

U

\{

j

}

X

R

=

x

R

)

)

(using Prop. 4 and 5)

11

Compute

F

(

X

j

)

(using Eq. (4))

12

j

∗

argmax

j

F

(

X

j

)

13

(

R,U

)

R

∪ {

j

∗

}

, U

\ {

j

∗

}

test sample, MinDRel identifies core feature sets that are significantly smaller than the total sensitive feature set size | S | = 7 . Interestingly, when δ > 0 , it discovers core feature sets smaller than 2 for the majority of users. This implies that most users would only need to reveal a small portion of their sensitive data to achieve accurate model predictions with either absolute certainty or high confidence .

6 MinDRel for non-linear classifiers

Next, the paper focuses on computing the estimate Pr( ˜ f θ ( X U , X R = x R )) and determining core feature sets when f θ is a nonlinear classifier. Then, the section presents results that illustrate the practical benefits of MinDRel for data minimization at inference time on neural networks.

6.1 Efficiently estimating Pr ( ˜ f θ ( X U , X R = x R ))

First notice that even if the input features x are jointly Gaussian, the outputs f θ ( x ) will no longer adhere to a Gaussian distribution after a non-linear transformation.

0

2

4

6

Core feature set size

0.0

0.5

1.0

Cumulative frequency

MinDRel

=0

=0.1

=0.2

This complicates estimating the distribution Pr( ˜ f θ ( X U , X R = x R ) . To tackle this challenge, the paper takes a local approximation of the model predictions ˜ f θ using a Gaussian distribution, as demonstrated in the following result.

Theorem 1. The distribution of the random variable ˜ f θ = ˜ f θ ( X U , X R = x R ) where X U ∼ N ( µ pos U , Σ pos U ) can be approximated by a Normal distribution as

where g U = ∇ X U ˜ f θ ( X U = µ pos U , X R = x R ) is the gradient of model prediction at X U = µ pos U .

Therein, the mean vector µ pos U and covariance matrix Σ pos U of Pr ( X U | X R = x R ) are derived from Proposition 3. This result leverages a first-order Taylor approximation of model f θ around its mean.

)

6.2 Testing pure core feature sets

Unlike linear classifiers, the case for nonlinear models lacks an exact and efficient method to determine whether a set is a core feature set. This is primarily due to the non-convex nature of the adopted models, which poses challenges in finding a global optimum. This section thus proposes an approximate testing routine and demonstrates its practical ability to significantly minimize data leakage during testing while maintaining high accuracy.

To determine if a subset U of the sensitive features S is a pure core feature set, we consider a set Q of input points ( X U , x R ) . The entries corresponding to the revealed features are set to the value x R , while the entries corresponding to the unrevealed features are sampled from the distribution Pr( X U | X R = x R ) . The test verifies if the model predictions f θ ( x ) remain constant for all x in Q . In the next section, we will show that even considering arbitrary classifiers (e.g., we use standard neural networks), MinDRel can reduce data leakage dramatically when compared to standard approaches. The MinDRel algorithm for nonlinear classifiers differs from Algorithm 1 primarily in the method used to compute the estimates for the distribution Pr( f θ ( X j = z, X U \ j , X R = x R )) of the soft model predictions (line 11). In this case, this estimate is computed by leveraging the results from Theorem 1 and Proposition 5. Moreover, the termination test of the algorithm is based on the discussion in the previous section. Appendix B reports a description of the algorithm's pseudocode.

7 Experiments

Datasets and settings. This section evaluates MinDRel's effectiveness in limiting data exposure during inference. The experiments are conducted on six standard UCI datasets [9]. To further emphasize the benefits of MinDRel, we compare it to two baselines: the optimal model, which employs a brute force method to find the smallest core feature set and its representative label and assumes all sensitive features are known, and the all-features model, which simply adopts the original classifiers using all the data features for each test sample. The performances are displayed for a varying number of sensitive attributes | S | ∈ [2 , 5] , while we delegate a study for larger | S | to the Appendix (which excludes the time-consuming baseline optimal , as intractable for large | S | ). For each choice of | S | , we randomly select | S | features from the entire feature set and designate them as sensitive attributes. The remaining attributes are considered public. The average accuracy and data leakage are then reported based on 100 random sensitive attribute selections. Due to page limit, we present a selection of the results and discuss their trends on Bank dataset. A comprehensive overview, additional analysis, and experiments are available in the Appendix D.

7.1 Linear classifiers

Figure 5 depicts performance results in accuracy (top subplots/higher is better) and relative data leakage (bottom subplots/lower is better) with varying number of revealable sensitive features | S | . The comparison includes three MinDRel versions: F-Score , which utilizes the scoring function elaborated in Section 4.1 to select the next feature to disclose (left); Importance , which employs a feature importance criterion leveraging the model f θ parameters to rank features (middle), detailed in the Appendix; and Random , which arbitrarily selects the next feature to reveal (right). All three versions use the same test procedure

0.78

0.79

0.80

F-Score

Importance

Random

2

4

# Sensitive Features

10

2

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

to validate whether a set qualifies as a core feature set.

Firstly, notice that MinDRel achieves equal or better accuracy than the optimal mechanism and baseline that utilizes all features during testing. This suggests that data minimization can be accomplished under linear models without compromising accuracy! Next, observe that an increase in δ

aids in safeguarding data minimization, as evidenced by the drop in relative data leakage (note the logarithmic scale of the y-axis). This is attributable to the influence of δ on the test for identifying a core feature set, thereby reducing its size.

Finally, the proposed scoring function outperforms other versions in terms of data leakage minimization, allowing users to disclose substantially fewer sensitive features. The Appendix also includes experiments with larger quantities of sensitive features, presenting analogous trends, where, however, a comparison against an optimal baseline was not possible in due to its exponential time complexity.

7.2 Non-linear classifiers

To assess MinDRel's performance in reducing data leakage when employing standard nonlinear classifiers, we use a neural network with two hidden layers and ReLU activation functions and train the models using stochastic gradient descent (see Appendix D for additional details). The evaluation criteria, baselines, and benchmarks adhere to the same parameters set in Section 5.3.

Figure 6 showcases the results in terms of accuracy (top subplots) and data leakage (bottom subplots). Unlike linear classifiers, nonlinear models using MinDRel with a failure probability of δ = 0 cannot guarantee the

0.81

0.82

0.83

0.84

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

same level of accuracy as the ' all features ' baseline model. However, this accuracy difference is minimal. Notably, a failure probability of δ = 0 enables users to disclose less than half, and up to 90% fewer sensitive features across different datasets, while achieving accuracies similar to those of conventional classifiers. Next, similarly to as observed in the previous section, MinDRel with the proposed F-score selector significantly outperforms the other variants in terms of data leakage minimization. Furthermore, when considering higher failure probabilities, data leakage decreases significantly. For instance, with δ ≤ 0 . 1 , users need to disclose only 5% of their sensitive features while maintaining accuracies comparable to the baseline models (the largest accuracy difference reported was 0.005%). These results are significant: They show that the introduced notion of privacy leakage and the proposed algorithm can become a valuable tool to protect individuals' data privacy at test time, without significantly compromising accuracy. .

8 Conclusion and Future Work

This paper introduced the concept of data minimization at test time whose goal is to minimize the number of features that individuals need to disclose during model inference while maintaining accurate predictions from the model. The motivations of this notion are grounded in the privacy risks imposed by the adoption of learning models in consequential domains, and align with the data minimization principle. The paper then discusses an iterative and personalized algorithm that selects the features each individual should release with the goal of minimizing data leakage while retaining exact (in the case of linear classifiers) or high (for non-linear classifiers) accuracy. Experiments over a range of benchmarks and datasets indicate that individuals may be able to release as little as 10% of their information without compromising the accuracy of the model, providing a strong argument for the effectiveness of this approach in protecting privacy while preserving the accuracy of the model.

While this study is the first attempt at defining data minimization during inference, it also opens up avenues for further research. First, establishing bounds on the data leakage provided by our proposed method compared to an optimal procedure presents an interesting and open challenge. Second, exploring the relationship between data minimization principles and their consequent disparate impacts presents another open direction. Lastly, developing effective algorithms to provably construct core feature sets for non-linear classifiers is another important area of investigation.

References

California privacy rights act. https://cppa.ca.gov/ . Accessed: 2023-10-23.

General data protection regulation. https://gdpr-info.eu/ . Accessed: 2023-10-23.

General personal data protection law.

Personal information protection act. https://elaw.klri.re.kr/eng_service/lawView. do?hseq=53044&lang=ENG . Accessed: 2023-10-23.

Protection of personal information act. https://popia.co.za/ . Accessed: 2023-10-23.

Abadi and et al. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security , 2016.

K. Amin, T. Dick, A. Kulesza, A. Munoz, and S. Vassilvitskii. Differentially private covariance estimation. Advances in Neural Information Processing Systems , 32, 2019.

C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning , volume 4. Springer, 2006.

C. Blake and C. Merz. Uci repository of machine learning databases, 1988.

G. Chandrashekar and F. Sahin. A survey on feature selection methods. Computers & Electrical Engineering , 40(1):16-28, 2014.

C.-H. Chang, M. Mai, and A. Goldenberg. Dynamic measurement scheduling for event forecasting using deep rl. pages 951-960, 2019.

K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research , 2011.

G. Contardo, L. Denoyer, and T. Artières. Sequential cost-sensitive feature acquisition. pages 284-294, 2016.

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference , pages 265-284. Springer, 2006.

Y. Fu, X. Zhu, and B. Li. A survey on instance selection for active learning. Knowledge and information systems , 35(2):249-283, 2013.

A. Krause and C. Guestrin. A note on the budgeted maximization of submodular functions . Citeseer, 2005.

R. Krishnan, P. Esposito, and M. Subedar. Bayesian-torch: Bayesian neural network layers for uncertainty estimation, Jan. 2022.

R. Krishnan and O. Tickoo. Improving model calibration with accuracy versus uncertainty optimization. In Advances in Neural Information Processing Systems , volume 33, pages 18237-18248, 2020.

Larxel. Children fetal health dataset. https://www.kaggle.com/datasets/andrewmvd/ fetal-health-classification , 2021.

J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang, and H. Liu. Feature selection: A data perspective. ACM computing surveys (CSUR) , 50(6):1-45, 2017.

Y. Li and J. Oliva. Active feature acquisition with generative surrogate models. pages 64506459, 2021.

X. Liu, W. Kong, S. Kakade, and S. Oh. Robust and differentially private mean estimation. Advances in neural information processing systems , 34:3887-3901, 2021.

N. Papernot, P. McDaniel, A. Sinha, and M. Wellman. Towards the science of security and privacy in machine learning. arXiv preprint arXiv:1611.03814 , 2016.

C. Peterson, F. C. Stingo, and M. Vannucci. Bayesian inference of multiple gaussian graphical models. Journal of the American Statistical Association , 110(509):159-174, 2015.

S. Roy. Car insurance data. https://www.kaggle.com/datasets/sagnik1511/ car-insurance-data , 2021.

B. Settles. Active learning literature survey. 2009.

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE , 104(1):148-175, 2015.

H. Shim, S. J. Hwang, and E. Yang. Why pay more when you can pay less: A joint learning framework for active feature acquisition and classification. arXiv preprint arXiv:1709.05964 , 2017.

A. Sudarshan. Customer segmentation data. https://www.kaggle.com/datasets/ abisheksudarshan/customer-segmentation , 2021.

B. Ustun, Y. Liu, and D. Parkes. Fairness without harm: Decoupled classifiers with preference guarantees. In International Conference on Machine Learning , pages 6373-6382. PMLR, 2019.

C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA, 2006.

L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou. Differentially private generative adversarial network. arXiv preprint arXiv:1802.06739 , 2018.

S. Yu, B. Krishnapuram, R. Rosales, and R. B. Rao. Active sensing. pages 639-646, 16-18 Apr 2009.

A Missing proofs

Proposition 1. Given a core feature set R ⊆ S with failure probability δ < 0 . 5 , then there exists a function ϵ : R → R that is monotonic decreasing function with ϵ (1) = 0 such that:

where H [ Z ] = -∑ z ∈ [ L ] Pr( Z = z ) log Pr( Z = z ) is the entropy of the random variable Z .

Proof. In this proof, we demonstrate the binary classification case. The extension to a multi-class scenario can be achieved through a similar process.

By the definition of the core feature set, there exists a representative label, denoted as ˜ y ∈ { 0 , 1 } such that the probability of P ( f θ ( X U , X R = x R ) = ˜ y ) is greater than or equal to 1 -δ . Without loss of generality, we assume that the representative label is ˜ y = 1 . Therefore, if we denote Z as the probability of Pr ( f θ ( X U , X R = x R ) = 1) , then the probability of Pr ( f θ ( X U , X R = x R ) = 0) = 1 -Z . Additionally, we have Z ≥ 1 -δ > 0 . 5 due to the definition of core feature set and by the assumption that δ < 0 . 5 . The entropy of the model's prediction can be represented as: H [ f θ ( X U , X R = x R ) ] = -Z log Z -(1 -Z ) log(1 -Z ) .

Choose ϵ ( Z ) = -Z log Z -(1 -Z ) log(1 -Z ) . The derivative of ϵ ( Z ) is given by dϵ ( Z ) dZ = log 1 -Z Z < 0 , as Z > 0 . 5 . As a result, ϵ ( Z ) is a monotonically decreasing function, so ϵ ( Z ) ≤ ϵ (1 -δ )

When δ = 0 , we have Z = 1 , and by the property of the entropy H [ f θ ( X U , X R = x R ) ] = 0 .

Proposition 2. Given two subsets R and R ′ of sensitive features S , with R ⊆ R ′ ,

where U = S \ R and U ′ = S \ R ′ .

Proof. This is due to the property that conditioning reduces the uncertainty, or the well-known information never hurts theorem in information theory [16].

Proposition 3. The conditional distribution of any subset of unrevealed features U ′ ∈ U , given the the values of released features X R = x R is given by:

where Σ is the covariance matrix

Proof. This is a well-known property of the Gaussian distribution and we refer the reader to Chapter 2.3.2 of the textbook [8] for further details.

Proposition 4. The model predictions before thresholding, ˜ f θ ( X U , X R = x R ) = θ U X U + θ R x R is a random variable with a Gaussian distribution N ( m f , σ f ) , where

where θ U is the sub-vector of parameters θ corresponding to the unrevealed features U .

Proof. The proof of this statement is straightforward due to the property that a linear combination of Gaussian variables X U is also Gaussian. Additionally, the posterior distribution of X U is already provided in Proposition 3.

Proposition 5. Let the model predictions prior thresholding ˜ f θ ( X U , X R = x R ) , be a random variable following a Gaussian distribution N ( m f , σ 2 f ) . Then, the model prediction following thresholding f θ ( X U , X R = x R ) is a random variable following a Bernoulli distribution Bern ( p ) with p = Φ( m f σ f ) , where Φ( · ) refers to the CDF of the standard normal distribution, and m f and σ f , are given in Equations (5) and (6) , respectively.

Proof. In the case of a binary classifier, we have f θ ( x ) = 1 { ˜ f θ ( x ) ≥ 0 } . If ˜ f follows a normal distribution, denoted as ˜ f ∼ N ( m f , σ 2 f ) , then by the properties of the normal distribution, fθ follows a Bernoulli distribution, denoted as f θ ∼ Bern ( p ) , with parameter p = Φ( m f σ f ) , where Φ( · ) is the cumulative density function of the standard normal distribution.

Proposition 6. Assume f θ is a linear classifier. Then, determining if a subset U of sensitive features S is a pure core feature set can be performed in O ( | P | + | S | ) time.

Proof. As discussed in the main text, to test if a subset U is a core feature set or not, we need to check if the following two terms have the same sign (either negative or non-negative):

These can be solved in time O ( | P | + | S | ) due to the property of the linear equality above.

Theorem 1. The distribution of the random variable ˜ f θ = ˜ f θ ( X U , X R = x R ) where X U ∼ N ( µ pos U , Σ pos U ) can be approximated by a Normal distribution as

where g U = ∇ X U ˜ f θ ( X U = µ pos U , X R = x R ) is the gradient of model prediction at X U = µ pos U .

Proof. The proof relies on the first Taylor approximation of classifier ˜ f around its mean:

Since X U ∼ N ( µ pos U , Σ pos U ) hence X U -µ pos U ∼ N ( 0 , Σ pos U ) . By the properties of normal distribution, the right-hand side of Equation (12) is a linear combination of Gaussian variables, and it is also Gaussian.

B Algorithms Pseudocode

The pseudocode for MinDRel for non-linear classifiers is presented in Algorithm 2. There are two main differences between this algorithm and the case of linear classifiers. Firstly, unlike linear classifiers, the procedure of pure core feature testing on line 5 does not require the guarantee (see again Section 6.2). The accuracy of the testing procedures depends on the number of random samples that we evaluate. The greater the number of drawn samples, the more likely the testing procedure is to be accurate. During experiments, we draw 10 5 samples to perform the testing. Additionally, we use Theorem 1 to estimate the distribution of the soft prediction as seen on line 11, as the exact distribution cannot be computed analytically as in the case of linear classifiers.

C Extension from binary to multiclass classification

In the main text, we provide the implementation of MinDRel for binary classification problems. In this section, we extend the method to the multiclass classification problem.

C.1 Estimating P ( f θ ( X U , X R = x R ))

In order to achieve our goals of determining if a subset is a core feature set for a given δ > 0 , and computing the entropy in the scoring function, we need to estimate the distribution of f θ ( X U , X R = x R ) . In this section, we first discuss the method of computing the distribution of ˜ f θ ( X U , X R = x R ) for both linear and non-linear models. Once this is done, we then address the challenge of estimating the hard label distribution P ( fθ ( X U , X R = x R )) .

It is important to note that, under the assumption that the input features X are normally distributed with mean µ and covariance matrix Σ , the linear classifier ˜ f θ = θ ⊤ x will also have a multivariate

Algorithm 2: MinDRel for non-linear classifiers

For non-linear classifiers, the output f θ ( X U , X R = x R ) is not a Gaussian distribution due to the non-linear transformation. To approximate it, we use Theorem 1 which states that the non-linear function ˜ f θ ( X U , X R = x R ) can be approximated as a multivariate Gaussian distribution.

Challenges when estimating P ( f θ ( X U , X R = x R )) . For multi-class classification problems, the hard label f θ ( X U , X R = x R ) is obtained by selecting the class with the highest score, which is given by argmax i ∈ [ L ] ˜ f i θ ( X U , X R = x R ) . However, due to the non-analytical nature of the argmax function, even when ˜ f θ ( X U , X R = x R ) follows a Gaussian distribution, the distribution of f θ ( X U , X R = x R ) cannot be computed analytically. To estimate this distribution, we resort to Monte Carlo sampling. Specifically, we draw a number of samples from P ( ˜ f θ ( X U , X R = x R )) , and for each class y ∈ Y we approximate the probability P ( f θ ( X U , X R = x R ) = y ) as the proportion of samples that fall in that class y .

We provide experiments of MinDRel for multi-class classification cases in Section D.5.

D Experiments details

Datasets information. To show the advantages of the suggested MinDRel technique for safeguarding feature-level privacy, we employ benchmark datasets in our experiments. These datasets include both binary and multi-class classification datasets. The proposed method was evaluated on the following datasets for binary classification tasks:

Bank dataset [9]. The objective of this task is to predict whether a customer will subscribe to a term deposit using data from various features, including but not limited to call duration and age. There are a total of 16 features available for this analysis.

Adult income dataset [9]. The goal of this task is to predict whether an individual earns more than $50,000 annually. After preprocessing the data, there are a total of 40 features available for analysis, including but not limited to occupation, gender, race, and age.

Credit card default dataset [9]. The objective of this task is to predict whether a customer will default on a loan. The data used for this analysis includes 22 different features, such as the customer's age, marital status, and payment history.

Car insurance dataset [25]. The task at hand is to predict whether a customer has filed a claim with their car insurance company. The dataset for this analysis is provided by the insurance company and includes 16 features related to the customer, such as their gender, driving experience, age, and credit score.

Furthermore, we also evaluate the proposed method on two additional multi-class classification datasets:

Customer segmentation dataset [29]. The task at hand is to classify a customer into one of four distinct categories: A, B, C, and D. The dataset used for this task contains 9 different features, including profession, gender, and working experience, among others.

Children fetal health dataset [19]. The task at hand is to classify the health of a fetus into one of three categories: normal, suspect, or pathological, using data from CTG (cardiotocography) recordings. The data includes approximately 21 different features, such as heart rate and the number of uterine contractions.

Settings. For each dataset, 70% of the data will be used for training the classifiers, while the remaining 30% will be used for testing. The number of sensitive features, denoted as | S | , will be chosen randomly from the set of all features. The remaining features will be considered public. 100 repetition experiments will be performed for each choice of | S | , under different random seeds, and the results will be averaged. All methods that require Monte Carlo sampling will use 100 random samples. The performance of different methods will be evaluated based on accuracy and data leakage. Two different classifiers will be considered.

Linear classifiers: We use Logistic Regression as the base classifier.

Nonlinear classifiers: The nonlinear classifiers used in this study consist of a neural network with two hidden layers, using the ReLU activation function. The number of nodes in each hidden layer is set to 10. The network is trained using stochastic gradient descent (SGD) with a batch size of 32 and a learning rate of 0.001 for 300 epochs.

For Bayesian NN, we employ the package bayesian-torch [17] with the default settings. The base regressor is a neural network with one hidden layer that has 10 hidden nodes and a ReLU activation function. We train the network in 300 epochs with a learning rate of 0.001.

Baseline models. We compare our proposed algorithms with the following baseline models:

All features : This refers to the usage of the original classifier which asks users to reveal all sensitive features.

Optimal : This method involves evaluating all possible subsets of sensitive features ( 2 | S | in total) in order to identify the minimum pure core feature set. For each subset, the verification algorithm is used to determine whether it is a pure core feature set. The minimum pure core feature set that is found is then selected. It should be noted that as all possible subsets are evaluated, all sensitive feature values must be revealed. Therefore, this approach is not practical in real-world scenarios. However, it does provide a lower bound on data leakage for MinDRel (when δ = 0 ).

MinDRel models. In MinDRel there are two important steps: (1) core feature set verification and (2) selection next feature to reveal. As additional baselines, we keep the core feature set verification and vary the selection process. We consider the following three feature selection methods:

F-Score : We choose the feature based on the amount of information on model prediction we gain after revealing one feature as provided in Equation 3.

Importance : We reveal the unknown sensitive features based on the descending order of feature importance until we find a core feature set. The feature importance is determined as follows. We firstly fit a Logistic Regression f θ ( x ) = 1 { θ T x ≥ 0 } on the training dataset D using all features (public included). The importance of one sensitive feature i ∈ S is determined by ∥ θ i ∥ 2 .

Random : We reveal the unrevealed sensitive feature in random order until the revealed set is a core feature set.

0.78

0.79

0.80

F-Score

Importance

Random

2

4

# Sensitive Features

10

2

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(a) Bank dataset

0.843

0.844

0.845

F-Score

Importance

Random

2

4

# Sensitive Features

10

2

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(b) Income dataset

0.8165

0.8170

0.8175

F-Score

Importance

Random

2

4

# Sensitive Features

10

2

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(c) Credit dataset

0.84

0.85

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(d) Insurance dataset

Metrics. We compare all different algorithms in terms of accuracy and data leakage:

Accuracy. For algorithms that are based on the core feature set, such as our MinDRel and Optimal, the representative label is used as the model's prediction. Again, the representative label for δ = 0 can be identified by using testing pure core feature set procedures. For δ > 0 , the representative label is given by ˜ y = argmax y ∈Y ∫ P ( f θ ( X U = x U , X R = x R ) = y ) P ( X U | X R = x R ) dx u . The accuracy is then determined by comparing this representative label to the ground truth.

Data leakage. We compute the percentage of the number of sensitive features that users need to provide on the test set. Small data leakage is considered better.

D.1 Additional comparison between using Gaussian assumption and Bayesian NN

We first show empirically the benefits of our proposed Gaussian assumption compared to using Bayesian NN which allows more flexibility in modeling the conditional distribution P ( X U | X R = x R ) . We report both training and inference time between Bayesian NN and our Gaussian assumption on various datasets when the number of sensitive features | S | = 5 in Table 1 and Table 2. When | S | = 5 the number of possible subsets U ∈ S is 2 5 = 32 which requires training 32 Bayesian NN models. This will be especially slow for datasets with a large number of training samples (e.g., Income with 50K samples). In contrast, using Gaussian assumption we just need to precompute 32 inverse matrices Σ -1 R,R which is pretty fast for data that have a small number of features (less than 50 in our experiments). It is noted again that in this paper we focus on the case when the number of training samples is much more than the number of features. Likewise, during inference time, with Gaussian assumption, we can compute the distribution of model prediction in a closed form by simple matrix multiplication which takes O ( d 2 ) . Instead, using Bayesian NN, it requires expensive Monte Carlo sampling, especially when | U | is large to obtain an accurate estimation of P ( X U | X R = x R ) .

0.81

0.82

0.83

0.84

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

0.818

0.820

F-Score

Importance

Random

2

4

# Sensitive Features

10

2

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

0.840

0.842

0.844

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(a) Income dataset

0.830

0.835

0.840

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

Optimal

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(b) Insurance dataset

We also report the performance in terms of accuracy and data leakage between using Gaussian assumption and Bayesian NN in Figure 9. We see no significant difference in terms of accuracy and data leakage between the two choices of modeling P ( X U | X R = x R ) . In addition, as indicated above using the Gaussian assumption reduces significantly the training and inference time, in the subsequent experiments we will use the Gaussian assumption in MinRDel with F-Score selection.

D.2 Additional experiments on linear binary classifiers

Additional experiments were conducted to compare the performance of MinDRel to that of the baseline methods using linear classifiers on the Bank, Adult income, Credit, and Insurance datasets, as shown in Figure 7. As in the main text, a consistent trend in terms of performance is observed.

0.78

0.79

Gaussian assumption

Bayesian NN

2

3

4

5

# Sensitive Features

10

2

10

1

Data Leakage

2

3

4

5

# Sensitive Features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(a) Bank dataset

0.840

0.845

Gaussian assumption

Bayesian NN

2

3

4

5

# Sensitive Features

10

2

10

1

Data Leakage

2

3

4

5

# Sensitive Features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(b) Income dataset

0.8100

0.8125

0.8150

0.8175

Gaussian assumption

Bayesian NN

2

3

4

5

# Sensitive Features

10

3

10

2

10

1

Data Leakage

2

3

4

5

# Sensitive Features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(c) Credit dataset

0.82

0.84

0.86

Gaussian assumption

Bayesian NN

2

3

4

5

# Sensitive Features

10

1

Data Leakage

2

3

4

5

# Sensitive Features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(d) Insurance dataset

As the number of sensitive attributes, | S | , increases, the data leakage introduced by MinDRel with various values of δ increases at a slower rate. With different choices of | S | , MinDRel (with δ = 0 ) requires the revelation of at most 50% of sensitive information. To significantly reduce the data leakage of MinDRel, the value of δ can be relaxed. As mentioned in the main text, δ controls the trade-off between accuracy and data leakage here. The larger δ is, the greater uncertainty the model prediction has, which implies the fewer sensitive features users need to reveal and the lower accuracy of the model prediction. By choosing an appropriate value for the failure probability, such as δ = 0 . 1 , only minimal accuracy is sacrificed (at most 0.002%), while the data leakage can be reduced to as low as 5% of the total number of sensitive attributes.

D.3 Additional experiments on non-linear binary classifiers

Additional experiments were conducted to compare the performance of MinDRel to that of the baseline methods using non-linear classifiers on the Bank, Adult income, Credit, and Insurance datasets, as shown in Figure 8. As seen, while the baseline All features method requires the revelation of all sensitive attributes, MinDRel with different values of δ only requires the revelation of a much smaller number of sensitive attributes. The accuracy difference between the Baseline method and MinDRel is also minimal (at most 2%). These results demonstrate the effectiveness of MinDRel in protecting privacy while maintaining a good prediction performance for test data.

D.4 Sclability of MinDRel for large | S |

We demonstrate the performance of MinDRel when we have a large number of sensitive features | S | . Note that to reduce the runtime we did not run the Optimal method which performs an exponential search over all possible choices of the subset of S .

We first report the accuracy and data leakage of MinDRel when using F-Score or using either two heuristic rules Importance and Random in the case of logistic regression classifiers in Figure 10.

0.78

0.79

0.80

F-Score

Importance

Random

5

10

# Sensitive Features

10

1

10 0

Data Leakage

5

10

# Sensitive Features

5

10

# Sensitive Features

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(a) Bank dataset

0.840

0.845

F-Score

Importance

Random

5

10

15

# Sensitive Features

10

2

10

1

10 0

Data Leakage

5

10

15

# Sensitive Features

5

10

15

# Sensitive Features

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(b) Income dataset

0.805

0.810

0.815

F-Score

Importance

Random

5

10

15

# Sensitive Features

10

2

10

1

10 0

Data Leakage

5

10

15

# Sensitive Features

5

10

15

# Sensitive Features

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(c) Credit dataset

0.82

0.84

F-Score

Importance

Random

5

10

# Sensitive Features

10

1

10 0

Data Leakage

5

10

# Sensitive Features

5

10

# Sensitive Features

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

(d) Insurance dataset

10

15

# Sensitive Features

10

4

10

3

10

2

10

1

10 0

Income

Random

Importance

F-Score

7.5

10.0

12.5

# Sensitive Features

10

4

10

2

10 0

Bank

Random

Importance

F-Score

6

8

10

12

# Sensitive Features

10

4

10

2

10 0

Insurance

Random

Importance

F-Score

10

15

# Sensitive Features

10

3

10

2

10

1

10 0

10 1

Credit

Random

Importance

F-Score

Finally, we report the average testing time (in seconds) to get the model prediction per user of MinDRel in Figure 11. It is noted that in this case, we assume the time taken by users to release sensitive features is negligible. It is evident that when | S | > 15 , our proposed MinDRel with F-Score can take slightly more than 1 second to get the model prediction per user. This demonstrates the applicability of the models in practice.

D.5 Evaluation of MinDRel on multi-class classifiers

Linear classifiers Wealso provide a comparison of accuracy and data leakage between our proposed MinDRel and the baseline models for linear classifiers. These metrics are reported for the Customer and Children Fetal Health datasets in Figures 12a and 12b, respectively. The figures clearly show

0.48

0.49

0.50

F-Score

Importance

Random

2

4

|S|

10

1

10 0

Data Leakage

2

4

|S|

2

4

|S|

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

Optimal

(a) Customer dataset

0.89

0.90

0.91

F-Score

Importance

Random

2

4

|S|

10

1

10 0

Data Leakage

2

4

|S|

2

4

|S|

All features

MinDRel( =0)

MinDRel(

=0.1)

MinDRel(

=0.2)

Optimal

(b) Children fetal health dataset

0.49

0.50

0.51

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

MinDRel( =0)

All features

MinDRel(

=0.1)

MinDRel(

=0.2)

Optimal

(a) Customer dataset

0.89

0.90

0.91

F-Score

Importance

Random

2

4

# Sensitive Features

10

1

10 0

Data Leakage

2

4

# Sensitive Features

2

4

# Sensitive Features

MinDRel( =0)

All features

MinDRel(

=0.1)

MinDRel(

=0.2)

Optimal

(b) Children fetal health dataset

the benefits of MinDRel in reducing data leakage while maintaining a comparable accuracy to the baseline models.

Nonlinear classifiers Similarly, we present a comparison of our proposed algorithms with the baseline methods when using non-linear classifiers. These metrics are reported for the Customer and Children Fetal Health datasets in Figures 13a and 13b, respectively. The results show that using MinDRel with a value of δ = 0 results in a minimal decrease in accuracy, but significantly reduces the amount of data leakage compared to the Baseline method.

E Exetended Motivation

Finally, we delve deeper into the motivational aspects underscoring our research. Our study primarily concerns tabular datasets where the total number of features is significantly outnumbered by the training samples. We concentrate on applications necessitating user-provided personal information for decision-making, such as lending, online insurance services, and healthcare services. The prevailing weaknesses of these applications, which our work attempts to address, are outlined as follows:

Privacy Concerns: The primary issue stems from the need for users to disclose sensitive information. For instance, in the context of online healthcare services, patients are required to share an array of sensitive health data-weight, height, smoking habits, etc.-via a website or mobile application. This exposes users to potential privacy threats.

User and Organizational Expenditure of Time and Effort: Numerous applications, such as lending, involve the time-consuming and effort-intensive process of gathering sensitive information and its supporting evidence. Users, for example, are required to validate their income through payslips or employment contracts in lending applications. Similarly, the organization must invest time and resources to verify the authenticity of submitted documents.

Legal Constraints: According to the EU General Data Protection Regulation's principle of data minimization, the collection of excessive personal information by companies and organizations is restricted. Our primary text illustrates that it is not imperative to report all features to preserve the model's accuracy.

Inference Time. We also emphasize the importance of low inference time from both user and business perspectives. From the user's viewpoint, applications such as online car insurance require answering a series of questions to determine the insurance plan. Naturally, users prefer answering fewer questions in the least amount of time. From a business standpoint, prolonged inference time may lead to customer dissatisfaction, potentially resulting in contract termination. This serves as the basis for our algorithmic choices, in lieu of more complex conditional distribution modeling methods, which can significantly increase inference time.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper questions the necessity of soliciting all user information at inference time, with an eye towards privacy-sensitive domains such as law, recruitment, and healthcare, where learning models typically require access to extensive and sensitive user data for accurate prediction. The authors propose in some settings, individuals might only need to provide a small subset of their information to obtain accurate predictions, thereby protecting their privacy and easing the burden on organizations verifying the accuracy of disclosed information. To that end, the authors introduce a framework for simultaneously considering model certainty and data minimization. Their setup includes considering subsets of features that individuals can disclose in order to receive comparable prediction quality while providing fewer features. The authors propose algorithms for choosing the appropriate subset and present theoretical justification and empirical findings. Initial evaluations reveal that individuals may only need to disclose about 10% of their information to achieve the same accuracy level as a model that uses all user information.\n\nSTRENGTHS:\nThe primary strength of this paper is that it thoughtfully addresses a neglected and high-impact research question. As the authors note, there is strong evidence that \"most users would only need to reveal a small portion of their sensitive data to achieve accurate model predictions with either absolute certainty or high confidence.\" They include a strong motivating example in Section 2, Figure 1, to make it clear that it's possible that a label could be obtained without requesting all sensitive features. Similarly, they underscore the importance of this work by noting the real-world impacts on privacy and the burden of verifying the provided information.\n\nOther strengths:\n- This paper includes a thoughtful framework for studying this problem, quantifying the tradeoff between model certainty (though not model performance, as claimed) and privacy loss. \n- The proposed methods include an efficient algorithm for a joint Gaussian setting as well as a less efficient Bayesian modeling alternative\n- The experiments appear well-designed, including considering random subsets of \"private\" features using publicly available datasets\n\nWEAKNESSES:\nThe primary weakness of this work is the use of entropy as a proxy for model performance. While the authors state that initially that their goal is the \"produce accurate or nearly accurate predictions during inference\" using fewer features, the goalposts subtly shift to \"accurately predict[ing] the **output of the model** using the smallest possible number of sensitive features.\" This shift motivates the use of prediction entropy, rather than predictive performance, as a metric for the impact of obtaining additional features. This is significant because while entropy obeys the \"information never hurts\" principle, predictive performance does not (see, e.g., https://proceedings.mlr.press/v97/ustun19a.html). Considering predictive performance (instead of entropy) likely makes this problem more difficult, and it's possible that using entropy is a reasonable proxy in some cases. However, this choice merits at minimum discussion and ideally theoretical and experimental justification. \n\nOther weaknesses:\n\nUnclear/imprecise notation and language:\n- \"data leakage\" is used inconsistently throughout the paper. It is introduced as \"the percentage of sensitive features that are revealed unnecessarily, meaning that their exclusion would not significantly impact the model’s output,\" but the authors later state that \"[increasing $\\delta$ yields] reduced data leakage but also less precise model predictions\" which appears contradictory. This term is later used as a metric in the experiments, but without a precise definition and with axes that seem to contradict prior use.\n- on line 155-156, the authors state that the imputation does not occur, and the \"unrevealed features are treated as random variables and are integrated during the prediction process.\" This process is not further explained but appears to be implicit in Equation 5 with a Schur complement (and arguably as a form of imputation). \n- \"public\" is used both to describe a subset of features \"public x_P and sensitive x_S features\" and a subset of samples \"trained on a public dataset\"\n- The hyperparameter T appears on line 9 of Algorithm 1 but is not discussed from what I could tell, either theoretically or empirically\n- In line 13 of Algorithm 1, it is unclear if feature j* is actually obtained here or if it is added to the set of features to be obtained. I assume the former. \n- The assumption that the cost of obtaining each feature is uniform should be explicitly stated. \n\n\nOther:\n- The authors claim throughout the paper that they are the first to study this type of work (line 47, line 420) despite the presence of several other publications on this topic (e.g., https://arxiv.org/abs/1602.03600 or https://proceedings.mlr.press/v5/yu09a.html)\n- Some words were used improperly (e.g., \"contrast\" on line 9, \"recur\" on 196, \"valid\" (sufficient?) on line 227). Overall, could use another pass to make the writing more clear.\n\nQUESTIONS:\n- Can you explain or justify your use of model entropy as a proxy for model performance?\n- The thresholding explanation on line 190 was a bit confusing -- can you explain this process in more detail?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address the problem of data minimization at inference time, which poses a real-world challenge in real-world applications, where users might want to hide sensitive or personal attributes. They provide an efficient algorithm to sequentially determine the appropriate attributes for an individual, with the goal to maintain the original predictive accuracy (based on all attributes).\n\nSTRENGTHS:\n1. The considered problem is important and timely.\n2. The proposed theoretical framework introduces some interesting novel concepts.\n3. The paper is well motivated and well-written.\n\nWEAKNESSES:\n1. The addressed problem is inherently a privacy problem; if some feature values are not explicitly available at inference time, it does not mean that they can not be inferred. Actually, in practice if the predictive quality does not change after hiding some feature values, it is a strong indication that these values can be inferred from the other values.\n\n2. The overall objective of maintaining the original predictive quality with the minimum amount of private features is never explicitly formalized/defined.\n\n3. In many real-world applications that use personal data, accuracy is the worst measure for predictive quality, so focusing on and reporting this measure is not really meaningful (e.g., when the goal is good model calibration on imbalanced classes).\n\n4. Relevant related work has not been considered: I Prefer not to Say: Operationalizing Fair and User-guided Data Minimization\nT Leemann, M Pawelczyk, CT Eberle, G Kasneci\n\nQUESTIONS:\nCan you comment on the weakness above and limitations below?\n\nLIMITATIONS:\nThe novelty of the work is rather limited, as relevant work on this topic has already addressed similar challenges.\nThe formalization of the approach and the theoretical framework is limited as it mainly aims for maintaining the high accuracy, and it does not consider the true privacy of the features that users wish to hide.\nThe evaluation is also focused on accuracy, which is not a meaningful measure in real-world ML applications based on tabular user data.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of data minimization at inference time. Consider a set of features X which consists of public features Xp, and private features X\\Xp. A model has been trained with all the features X, i.e., f(X). Now, the goal is to allow for inference revealing only a subset of the private features Xr and keep some of them unrevealed Xu.\nIn essence, X=Xp U Xr U Xu.\n\nSTRENGTHS:\nThe paper introduces a new and interesting problem, i.e., data minimization during inference time.\n\nThey provide entropy-based measures to decide whether to include a feature or not. \n\nThey also provide theoretical guarantees and experimental results.\n\nWEAKNESSES:\nWhile the authors say this is the \"first\" work to do so, I think there are substantial similarities with the problem of feature selection or feature engineering as well as inference under missing data. The differences should be made clear with references.\n\nThe use of the term leakage can be a bit confusing since leakage is used here for the number of features revealed. But of course, the features can just be random noise and hence not leak anything, whereas one feature can also be much informative of everything and cause leakage. I think a different terminology should be used here since leakage sounds more like \"information\" rather than \"number of features\".\n\nMy most important concern: How does this approach compare with applying a local explanation framework and dropping the least important private features? For example, for each user, if you apply SHAP or LIME or even feature attributions, and then drop the private features with the least contribution for that local user. What performance would you get and how would that compare to your method?\nI would be happy to increase my rating based on a discussion/comparison with just applying existing local explanations for this problem statement.\n\nQUESTIONS:\nSee Weaknesses\nCOMMENT: After rebuttal, I increased my score by 2 points.\n\nLIMITATIONS:\nSee Weaknesses (last point on my major concern)", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose that in a large number of application of machine learning reasonable model accuracy can be realized without the model having access to the entire feature set. This has implications for privacy and data-sharing, as a model that adaptively selects features to solicit would retain model performance even when minimal sensitive data is revealed to the model.\n\nSTRENGTHS:\n* The motivation for the paper is clear and important. The ability to reduce the amount of disclosed data is appealing and a thoughtful, timely contribution. \n\n* The authors attempt to address obvious questions that may arise, specifically around the relationship between the input features being non-linear.\n\nWEAKNESSES:\n* the empirical evaluation of the model is somewhat weak. The authors do present results in terms of accuracy, however, there is no discussion as to what is the source of performance degradation, if any. It would be nice to see a more holistic evaluation that takes into account other aspects of the classifier performance, including ROC performance and calibration. \n\n* For linear classifiers, computing the argmax over the estimated entropy is easy as it factorizes over the individual dimensions of the core set. However, this may not be the case when the relationship between features is non-linear. Can the authors comment on this. Specifically can the presented algorithm still be tractable if higher-order interactions between core-set of features considered? \n\n* The current evaluation pipeline randomly assign attributes as un disclosed at random. I would recommend pre-specifying the disclosure rate based on certain notion of harm revelation of that feature would cause.\n\nQUESTIONS:\nThe authors proposed a feature sampling strategy based on greedy selection on the Exected Information Gain. Such a strategy has shown benefit in other areas of machine learning especially in active learning. There could be potentially many other well motivated acquisition functions in the context of the problem the authors present. Have the authors considered other such possibilities? If yes, can the authors comment as to why Expected Information Gain (Entropy) was selected as the most reasonable choice. \n\n\nThe presented algorithm in its current stage does a one step-greedy lookahead which would work when the relationship between features is linear. Can the authors comment on tractability when looking at the power set of all potentia features that make the coreset?\n\n\nI am willing to tweak my scores based on engagement with the authors and other reviewers in the discussion phase.\n\nLIMITATIONS:\nThere are issues with model safety and trust when a model adaptively solicits information from a user. Specifically, can the authors comment on how such an approach might effect users trust in the system when the model seeks disparate information from disparate users? \n\nI am not looking for a theoretical argument here, but more of a value judgement based discussion of potential implications of disparate treatment under such a model.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-31"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work presents a method to increase privacy of ML predictions at test time by asking users to reveal fewer features. The proposed method asks for features that are maximally informative of the prediction outcome given the features seen thus far and stops when the prediction outcome can be decided. The method addresses the case of linear as well as non-linear thresholded classifiers assuming Gaussian-distributed features and a local Taylor approximation of predicted label probability. Experiments on multiple real datasets show reduction in data required for similar accuracy as baselines.\n\nSTRENGTHS:\nWriting is clear. The method is explained clearly step-by-step. I like the presentation of the results in the plots.\n\nMethod is simple and is shown to work on real datasets. The idea of selecting maximally informative features until decision is reached is natural. The idea is executed well.\n\nProblem of reducing data required for a prediction is important to increase user privacy, so the method has practical significance.\n\nWEAKNESSES:\nOn the writing, some algorithmic details can be improved like mentioning the entropy calculation and calculating the core features sets for \\delta>0. Core feature sets can be defined more rigorously by mentioning the sources of randomness in the probability expression. Results from datasets other than Credit, which are presented in Appendix, can be summarised in the main text.\n\nReasons for the effectiveness of the method is not clear to me and are not sufficiently explored in experiments. This is needed given the success of simplifying assumptions on Gaussian distribution of features. Is this because of the dataset or model characteristics, or is due to the algorithm? Baselines such as removing all sensitive features might help check if dataset characteristics are the main contributor. An analysis of the examples which are predicted correctly without revealing features can shed more light. \n\nSome related work is missing. See detailed remarks. The technical contributions in light of this work is unclear.\n\nTheoretical analysis leaves many questions open e.g. the impact of approximations, what does optimal procedures look like, how much training data is needed to get significant privacy gains, how to extend the method to non-linear classifiers and high-dimensional data.\n\nQUESTIONS:\nNotation in Definition 1 is unclear. What does the probability of prediction f_\\theta mean for unobserved variables X_U? Do we take expectation over different values of X_U? An example of the meaning of this probability would help me.\nSimilarly, how are the unrevealed variables in entropy calculations, e.g. in term A of equation (3), handled?\n\nSuppose we remove all 5 of the sensitive features, does this have significantly lower accuracy? That is, does choosing the sensitive features matter at all for prediction.\n\nHow is entropy computed in equation 4? Please describe the Bayesian model used for the data.\n\nConsider adding the related work on following topics and discuss whether these are applicable to the problem setting. \nActive measurement of features e.g. Li and Oliva 2021 ‘Active Feature Acquisition with Generative Surrogate Models’ http://proceedings.mlr.press/v139/li21p.html\nDynamic measurement of features in time e.g. Chang et al. 2019 ‘Dynamic Measurement Scheduling for Event Forecasting using Deep RL’ https://proceedings.mlr.press/v97/chang19a.html\nFeature pruning for causal effect estimation e.g. Makar et al. 2019 ‘A Distillation Approach to Data Efficient Individual Treatment Effect Estimation’ https://ojs.aaai.org/index.php/AAAI/article/view/4375\n\n---\nAfter the rebuttal\n\nMy remaining concern relates to the second question above. Reasons for the success of the method are not clear from the experiments and how they are presented. For instance, seeing the accuracy of a baseline which removes all sensitive features (minimum data leakage) will help contextualize all line plots (e.g. percentage improvement from such a baseline can be the y-axis). This helps answer how much of the success is due to the method's feature selection versus predictability of the public features in the datasets.\nFurther, a detailed analysis of the examples that are predicted correctly with < 2 sensitive features will be instructive (e.g. looking at whether these are the same set of features or personalized to the data point).\nThis said I appreciate baselines included by authors to check the selection criteria and experiments to check effect of linearity, which are also required for a newly proposed method.\n\nLIMITATIONS:\nLimitations on modeling assumptions and theoretical analysis are acknowledged.\n\n\n## Minor comments, no response is aspected\n\nLine 4 of the Algorithm 1 to check core features should be explained in detail for the case of \\delta>0.\n\nPlease specify the assumption that input features are jointly Gaussian more prominently. e.g. in an Assumption environment, in the lines 177-180. \n\nPlease cite information processing inequality in Proposition 2 in main text.\n\nPropositions 2 to 5 and Theorem 1 are known results so they can be mentioned in text or denoted as lemmas with references.\n\nThe statement in line 174 does not require pointing to Proposition 1. It holds because of the definition of entropy.\n\nPlease provide guidelines on how to use the Gaussian approximation for categorical features.\n\nIntroduction is nicely written. However, the goal / objective of the paper is repeated multiple times which can be removed to be concise.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-08-01"}