Original: INVASE: I NSTANCE - WISE V ARIABLE S ELECTION USING N EURAL N ETWORKS
Revised: INVASE: I NSTANCE - WISE V ARIABLE S ELECTION USING N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The advent of big data brings with it data with more and more dimensions and thus a growing need to be able to efﬁciently select which features to use for a variety of problems.
Revised: The advent of big data brings with it data with more and more dimensions and thus a growing need to be able to efﬁciently select which features to use for a variety of problems.

Original: While global feature selection has been a well-studied problem for quite some time, only recently has the paradigm of instance-wise feature selection been developed.
Revised: While global feature selection has been a well-studied problem for quite some time, only recently has the paradigm of instance-wise feature selection been developed.

Original: In this paper, we propose a new instance-wise feature selection method, which we term INVASE.
Revised: In this paper, we propose a new instance-wise feature selection method, which we term INVASE.

Original: INVASE consists of 3 neural networks, a selector network, a predictor network and a baseline network which are used to train the selector network using the actor-critic methodology.
Revised: INVASE consists of 3 neural networks, a selector network, a predictor network and a baseline network which are used to train the selector network using the actor-critic methodology.

Original: Using this methodology,
Revised: Using this methodology,

Original: INVASE is capable of ﬂexibly discovering feature subsets of a different size for each instance, which is a key limitation of existing state-of-the-art methods.
Revised: INVASE is capable of ﬂexibly discovering feature subsets of a different size for each instance, which is a key limitation of existing state-of-the-art methods.

Original: We demonstrate through a mixture of synthetic and real data experiments that INVASE signiﬁcantly outperforms state-of-the-art benchmarks.
Revised: We demonstrate through a mixture of synthetic and real data experiments that INVASE signiﬁcantly outperforms state-of-the-art benchmarks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: High-dimensional data is becoming more readily available, and it brings with it a growing need to be able to efﬁciently select which features to use for a variety of problems.
Revised: High-dimensional data is becoming more readily available, and it brings with it a growing need to be able to efﬁciently select which features to use for a variety of problems.

Original: When doing predictions, it is well known that using too many variables with too few samples can lead to overﬁtting, which can signiﬁcantly hinder the performance of predictive models.
Revised: When doing predictions, it is well known that using too many variables with too few samples can lead to overﬁtting, which can signiﬁcantly hinder the performance of predictive models.

Original: In the realm of interpretability, the large dimensionality of the data is often too much information to present to a human who may be using the machine learning model as a support system.
Revised: In the realm of interpretability, the large dimensionality of the data is often too much information to present to a human who may be using the machine learning model as a support system.

Original: Understanding which features are most relevant to an outcome or to a model output is an important ﬁrst step in improving predictions and interpretability and many works exist that tackle feature selection on a global level.
Revised: Understanding which features are most relevant to an outcome or to a model output is an important ﬁrst step in improving predictions and interpretability and many works exist that tackle feature selection on a global level.

Original: However, in the heterogeneous data we typically encounter, the prediction made by a model (and indeed the true label) may rely on a different subset of the features for different subgroups within the data [13].
Revised: However, in the heterogeneous data we typically encounter, the prediction made by a model (and indeed the true label) may rely on a different subset of the features for different subgroups within the data [14].

Original: In this paper we propose a novel instance-wise feature selection method, INVASE (INstance-wise VAriable SElection), which attempts to learn which subset of the features is relevant for each sample, allowing us to display the minimal information required to explain each prediction and also to reduce overﬁtting of predictive models.
Revised: In this paper we propose a novel instance-wise feature selection method, INVASE (INstance-wise VAriable SElection), which attempts to learn which subset of the features is relevant for each sample, allowing us to display the minimal information required to explain each prediction and also to reduce overﬁtting of predictive models.

Original: Discovering a global subset of relevant features for a particular task is a well-studied problem and there are several existing methods for solving it such as Sequential Correlation Feature Selection [10], Mutual Information Feature Selection [19], Knockoff models [3], and more [9; 15].
Revised: Discovering a global subset of relevant features for a particular task is a well-studied problem and there are several existing methods for solving it such as Sequential Correlation Feature Selection [11], Mutual Information Feature Selection [21], Knockoff models [3], and more [10; 16].

Original: However, global feature selection suffers from a key limitation - the features discovered by global feature selection are the same for all samples.
Revised: However, global feature selection suffers from a key limitation - the features discovered by global feature selection are the same for all samples.

Original: In many cases, in particular when populations are highly heterogeneous, the relevant features may differ across samples.
Revised: In many cases, in particular when populations are highly heterogeneous, the relevant features may differ across samples.

Original: For instance, different patient subgroups have different relevant features for predicting heart failure [13].
Revised: For instance, different patient subgroups have different relevant features for predicting heart failure [14].

Original: Instance-wise feature selection methods such as [4; 24] instead try to discover the features that are relevant for each sample .
Revised: Instance-wise feature selection methods such as [4; 27] instead try to discover the features that are relevant for each sample .

Original: When the goal is to provide an interpretable explanation of the predictions made, a key challenge is in ensuring that we do not over-explain by providing too much information (i.e. choosing too many features).
Revised: When the goal is to provide an interpretable explanation of the predictions made, a key challenge is in ensuring that we do not over-explain by providing too much information (i.e. choosing too many features).

Original: Naturally, by performing feature selection on an individualized level we are able to select features that are more relevant to each sample, rather than having to choose the top k features globally, which may not explain the predictions for some samples very well, but simply perform well on average across all samples.
Revised: Naturally, by performing feature selection on an individualized level we are able to select features that are more relevant to each sample, rather than having to choose the top k features globally, which may not explain the predictions for some samples very well, but simply perform well on average across all samples.

Original: In this paper, we propose a novel instance-wise feature selection method which we term INVASE.
Revised: In this paper, we propose a novel instance-wise feature selection method which we term INVASE.

Original: We draw inﬂuence from actor-critic models [20] to solve the problem of backpropagating through subset sampling.
Revised: We draw inﬂuence from actor-critic models [22] to solve the problem of backpropagating through subset sampling.

Original: Our model consists of 3 neural networks: a selector network, a predictor network and a baseline network.
Revised: Our model consists of 3 neural networks: a selector network, a predictor network and a baseline network.

Original: During training, each of these are trained iteratively, with the selector network being trained to minimize a Kullback-Leibler (KL) divergence between the full conditional distribution and the selected-features-only conditional distribution of the outcome.
Revised: During training, each of these are trained iteratively, with the selector network being trained to minimize a Kullback-Leibler (KL) divergence between the full conditional distribution and the selected-features-only conditional distribution of the outcome.

Original: Our model is capable of discovering a different number of relevant variables for each sample which is a key limitation in existing instance-wise approaches (such as [4]).
Revised: Our model is capable of discovering a different number of relevant variables for each sample which is a key limitation in existing instance-wise approaches (such as [4]).

Original: We show signiﬁcant improvements over the state-of-the-art in both synthetic data and real-word data in terms of true positive rates, false discovery rates, and show better predictive performance with respect to several prediction metrics.
Revised: We show signiﬁcant improvements over the state-of-the-art in both synthetic data and real-word data in terms of true positive rates, false discovery rates, and show better predictive performance with respect to several prediction metrics.

Original: Our model can also be easily extended to handle both continuous and discrete outputs and time-series inputs (see the Appendix for details).
Revised: Our model can also be easily extended to handle both continuous and discrete outputs and time-series inputs (see the Appendix for details).

Original: 1. R ELATED W ORKS
Revised: 1. R ELATED W ORKS

Original: There are many existing works on global variable selection (see [9] for a good summary paper).
Revised: There are many existing works on global variable selection (see [10] for a good summary paper).

Original: [19] and [10] use max-dependency min-redundancy criteria [16] with mutual information and Pearson correlation, respectively.
Revised: [21] and [11] use max-dependency min-redundancy criteria [17] with mutual information and Pearson correlation, respectively.

Original: [3] uses multiple hypothesis testing for global variable selection.
Revised: [3] uses multiple hypothesis testing for global variable selection.

Original: As noted above, these global selection methods are not capable of learning sample-speciﬁc relevance.
Revised: As noted above, these global selection methods are not capable of learning sample-speciﬁc relevance.

Original: Instance-wise variable selection is also closely related to model interpretation methods.
Revised: Instance-wise variable selection is also closely related to model interpretation methods.

Original: Some previous works are based on backpropagation from the output of the predictive model to the input variables [25].
Revised: Some previous works are based on backpropagation from the output of the predictive model to the input variables [29].

Original: DeepLIFT [24] decomposes the output of the neural network on a reference input to compute the contribution of each input variable.
Revised: DeepLIFT [27] decomposes the output of the neural network on a reference input to compute the contribution of each input variable.

Original: However, both methods need white-box access to the pre-trained predictive models to compute the gradient and decomposition.
Revised: However, both methods need white-box access to the pre-trained predictive models to compute the gradient and decomposition.

Original: [2] approximates the predictive models using a Parzen window approximator when there is only black-box access to the predictive models.
Revised: [2] approximates the predictive models using a Parzen window approximator when there is only black-box access to the predictive models.

Original: Some other works are based on input perturbation such as [1], [14], [26] and [5].
Revised: Some other works are based on input perturbation such as [1], [15], [30] and [5].

Original: [17] uses Shapley values to compute the variable importance, and [22] uses locally linear models to explain the linear dependency for each sample.
Revised: [18] uses Shapley values to compute the variable importance, and [24] uses locally linear models to explain the linear dependency for each sample.

Original: [18] tries to interpret tree ensemble models using
Revised: [19] tries to interpret tree ensemble models using

Original: Shapley values but cannot generalize to other predictive models such as neural networks.
Revised: Shapley values but cannot generalize to other predictive models such as neural networks.

Original: Our work is most closely related to L2X (Learning to Explain) [4].
Revised: Our work is most closely related to L2X (Learning to Explain) [4].

Original: However, there are 3 key differences between our work and theirs.
Revised: However, there are 3 key differences between our work and theirs.

Original: In L2X, they try to maximize a lower bound of the mutual information between the target Y and the selected input variables X S .
Revised: In L2X, they try to maximize a lower bound of the mutual information between the target Y and the selected input variables X S .

Original: In contrast, we try to minimize the KL divergence between the conditional distributions Y | X and Y | X S .
Revised: In contrast, we try to minimize the KL divergence between the conditional distributions Y | X and Y | X S .

Original: In order to be able to backpropagate through subset sampling, L2X use the Gumbel-softmax trick [12] to approximately discretize the continuous outputs of the neural network.
Revised: In order to be able to backpropagate through subset sampling, L2X use the Gumbel-softmax trick [13] to approximately discretize the continuous outputs of the neural network.

Original: In our work, we use methods from actorcritic models [20] to bypass backpropagation through the sampling and instead use the predictor network to provide a reward to the selector network.
Revised: In our work, we use methods from actorcritic models [22] to bypass backpropagation through the sampling and instead use the predictor network to provide a reward to the selector network.

Original: Finally, due to the Gumbel-softmax used in
Revised: Finally, due to the Gumbel-softmax used in

Original: L2X, the number of variables to be detected must be ﬁxed in advance and is necessarily the same for every sample.
Revised: L2X, the number of variables to be detected must be ﬁxed in advance and is necessarily the same for every sample.

Original: The actor-critic methodology used in our model has no such limitations and so we are able to ﬂexibly select a different number of relevant variables for each sample and instead induce sparsity using an l 0 penalty term.
Revised: The actor-critic methodology used in our model has no such limitations and so we are able to ﬂexibly select a different number of relevant variables for each sample and instead induce sparsity using an l 0 penalty term.

Original: In fact, using the actor-critic methodology allows us to directly use the l 0 penalty term (which is not differentiable and therefore not practical to use in general).
Revised: In fact, using the actor-critic methodology allows us to directly use the l 0 penalty term (which is not differentiable and therefore not practical to use in general).

Original: A summary table highlighting the key features of all of the related works can be found in the Appendix.
Revised: A summary table highlighting the key features of all of the related works can be found in the Appendix.

Original:  P ROBLEM F ORMULATION
Revised:  P ROBLEM F ORMULATION

Original: Let X =
Revised: Let X =

Original: X 1 × ...
Revised: X 1 × ...

Original: ×
Revised: ×

Original: X d be a d -dimensional feature space and Y = { 1 , ..., c } be a discrete label space 1 .
Revised: X d be a d -dimensional feature space and Y = { 1 , ..., c } be a discrete label space 1 .

Original: Let X =
Revised: Let X =

Original: ( X 1 , ..., X d ) ∈ X and Y ∈ Y be random variables with joint density (or mass) p and marginal densities (or masses) p X and p Y respectively.
Revised: ( X 1 , ..., X d ) ∈ X and Y ∈ Y be random variables with joint density (or mass) p and marginal densities (or masses) p X and p Y respectively.

Original: We will refer to s ∈ { 0 , 1 } d as the selection vector, where s i = 1 will indicate that variable i is selected, and s i = 0 will indicate that variable i is not selected.
Revised: We will refer to s ∈ { 0 , 1 } d as the selection vector, where s i = 1 will indicate that variable i is selected, and s i = 0 will indicate that variable i is not selected.

Original: Let ∗ be any point not in any of the spaces X 1 , ..., X d and deﬁne
Revised: Let ∗ be any point not in any of the spaces X 1 , ..., X d and deﬁne

Original: X ∗ i
Revised: X ∗ i

Original: =
Revised: =

Original: X i ∪ {∗} and X ∗ =
Revised: X i ∪ {∗} and X ∗ =

Original: X ∗ 1 × ...
Revised: X ∗ 1 × ...

Original: × X ∗ d .
Revised: × X ∗ d .

Original: Given x ∈ X we will write x ( s ) to denote the suppressed feature vector deﬁned by
Revised: Given x ∈ X we will write x ( s ) to denote the suppressed feature vector deﬁned by

Original: [Equation]
Revised: [Equation]

Original: so that ∗ represents that a feature is not selected.
Revised: so that ∗ represents that a feature is not selected.

Original: In the global feature selection literature, the goal is to ﬁnd the smallest s (i.e. the one with fewest 1 s) such that E ( Y | X ( s ) )
Revised: In the global feature selection literature, the goal is to ﬁnd the smallest s (i.e. the one with fewest 1 s) such that E ( Y | X ( s ) )

Original: =
Revised: =

Original: E ( Y | X ) , or equivalently such that the conditional distribution of Y given
Revised: E ( Y | X ) , or equivalently such that the conditional distribution of Y given

Original: X ( s ) is the same as Y given all of X .
Revised: X ( s ) is the same as Y given all of X .

Original: Note that this deﬁnition is given fully in terms of random variables, rather than realizations of those random variables.
Revised: Note that this deﬁnition is given fully in terms of random variables, rather than realizations of those random variables.

Original: In contrast, our problem necessarily needs to be deﬁned in terms of realizations since we are aiming to select features for a given realization .
Revised: In contrast, our problem necessarily needs to be deﬁned in terms of realizations since we are aiming to select features for a given realization .

Original: We will write x to denote realizations of the random variable X .
Revised: We will write x to denote realizations of the random variable X .

Original: Then we formalize our problem as one of ﬁnding a selector function, S :
Revised: Then we formalize our problem as one of ﬁnding a selector function, S :

Original: X → { 0 , 1 } d such that for almost every x ∈ X (w.r.t.
Revised: X → { 0 , 1 } d such that for almost every x ∈ X (w.r.t.

Original: p X ) we have
Revised: p X ) we have

Original: [Equation]
Revised: [Equation]

Original: where d. = denotes equality in distribution and S ( x ) is minimal (i.e. fewest 1 s) such that (1) holds.
Revised: where d. = denotes equality in distribution and S ( x ) is minimal (i.e. fewest 1 s) such that (1) holds.

Original: We suppose that we have a dataset D
Revised: We suppose that we have a dataset D

Original: = { ( x j , y j ) } nj =1 consisting of n i.i.d.
Revised: = { ( x j , y j ) } nj =1 consisting of n i.i.d.

Original: realizations of the pair ( X , Y ) .
Revised: realizations of the pair ( X , Y ) .

Original: 2 Note that Y can be viewed as having either come from a dataset, in which case the problem is of selecting predictive features, or as having come from a predictive model, in which case the problem is of explaining the model’s predictions.
Revised: 2 Note that Y can be viewed as having either come from a dataset, in which case the problem is of selecting predictive features, or as having come from a predictive model, in which case the problem is of explaining the model’s predictions.

Original: 2. O PTIMIZATION P ROBLEM
Revised: 2. O PTIMIZATION P ROBLEM

Original: In order to learn a suitable selector function, we transform the constraint (1) into a soft constraint using the Kullback-Leibler (KL) divergence which, for random variables W and V with densities p W and p V is deﬁned as
Revised: In order to learn a suitable selector function, we transform the constraint (1) into a soft constraint using the Kullback-Leibler (KL) divergence which, for random variables W and V with densities p W and p V is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: We deﬁne the following loss for our selector function S
Revised: We deﬁne the following loss for our selector function S

Original: [Equation]
Revised: [Equation]

Original: where || · || simply denotes the number of non-zero entries of a vector (or equivalently in this case, the number of 1 s) and λ is a hyper-parameter that trades off between the constraint in (1) and the number of selected features.
Revised: where || · || simply denotes the number of non-zero entries of a vector (or equivalently in this case, the number of 1 s) and λ is a hyper-parameter that trades off between the constraint in (1) and the number of selected features.

Original: The KL divergence in (2) can be rewritten as
Revised: The KL divergence in (2) can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: where p Y ( ·|· ) denotes the appropriate conditional densities of Y .
Revised: where p Y ( ·|· ) denotes the appropriate conditional densities of Y .

Original: We will write
Revised: We will write

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: so that our ﬁnal loss can be written as
Revised: so that our ﬁnal loss can be written as

Original: [Equation]
Revised: [Equation]

Original: where || · || denotes the l 0 (pseudo-)norm.
Revised: where || · || denotes the l 0 (pseudo-)norm.

Original:  P ROPOSED M ODEL
Revised:  P ROPOSED M ODEL

Original: There are two main challenges in minimizing the loss in (4).
Revised: There are two main challenges in minimizing the loss in (4).

Original: First, the output space of the selector function ( { 0 , 1 } d ) is large - its size increases exponentially with the dimension of the feature space; thus a complete search is impractical in high dimensional settings (and it should be noted that it is in high dimensional settings where feature selection is most necessary).
Revised: First, the output space of the selector function ( { 0 , 1 } d ) is large - its size increases exponentially with the dimension of the feature space; thus a complete search is impractical in high dimensional settings (and it should be noted that it is in high dimensional settings where feature selection is most necessary).

Original: Second, we do not have access to the densities p Y ( ·| x ( S ( x )) ) and p Y
Revised: Second, we do not have access to the densities p Y ( ·| x ( S ( x )) ) and p Y

Original: ( y | x ) required to compute (4).
Revised: ( y | x ) required to compute (4).

Original: 3. L OSS E STIMATION
Revised: 3. L OSS E STIMATION

Original: To approximate the densities in (3), we introduce a pair of functions f φ : X ∗ × { 0 , 1 } d → [0 , 1] c parametrized by φ and f γ : X → [0 , 1] c parametrized by γ that will estimate p Y ( ·| x ( S ( x )) ) and p Y ( ·| x ) respectively.
Revised: To approximate the densities in (3), we introduce a pair of functions f φ : X ∗ × { 0 , 1 } d → [0 , 1] c parametrized by φ and f γ : X → [0 , 1] c parametrized by γ that will estimate p Y ( ·| x ( S ( x )) ) and p Y ( ·| x ) respectively.

Original: 3.1. P REDICTOR N ETWORK
Revised: 3.1. P REDICTOR N ETWORK

Original: We refer to f φ as the predictor network.
Revised: We refer to f φ as the predictor network.

Original: This will take as input a suppressed 3 feature vector x ( s ) and its corresponding selection vector s and will output a probability distribution (using a softmax layer) over the c -dimensional output space.
Revised: This will take as input a suppressed 3 feature vector x ( s ) and its corresponding selection vector s and will output a probability distribution (using a softmax layer) over the c -dimensional output space.

Original: f φ is trained to minimize the cross entropy loss given by
Revised: f φ is trained to minimize the cross entropy loss given by

Original: [Equation]
Revised: [Equation]

Original: where y i is the i th component of the one-hot encoding of y and π θ is the distribution induced by our selector network which will be deﬁned in the following section.
Revised: where y i is the i th component of the one-hot encoding of y and π θ is the distribution induced by our selector network which will be deﬁned in the following section.

Original: f φ is implemented as a fully connected neural network 4 .
Revised: f φ is implemented as a fully connected neural network 4 .

Original: 3.1. B ASELINE N ETWORK
Revised: 3.1. B ASELINE N ETWORK

Original: We refer to f γ as the baseline network, which is standard in the actor-critic literature for variance reduction.
Revised: We refer to f γ as the baseline network, which is standard in the actor-critic literature for variance reduction.

Original: f γ is implemented as a fully connected neural network and is trained to minimize
Revised: f γ is implemented as a fully connected neural network and is trained to minimize

Original: [Equation]
Revised: [Equation]

Original: For ﬁxed φ, γ we deﬁne our loss estimator, ˆ l , by
Revised: For ﬁxed φ, γ we deﬁne our loss estimator, ˆ l , by

Original: [Equation]
Revised: [Equation]

Original: 3. S ELECTOR FUNCTION OPTIMIZATION
Revised: 3. S ELECTOR FUNCTION OPTIMIZATION

Original: We approximate the selector function S :
Revised: We approximate the selector function S :

Original: X → { 0 , 1 } d by using a single neural network, ˆ S θ : X → [0 , 1] d parameterized by weights θ , that outputs a probability for selecting each feature (i
Revised: X → { 0 , 1 } d by using a single neural network, ˆ S θ : X → [0 , 1] d parameterized by weights θ , that outputs a probability for selecting each feature (i

Original: [Equation]
Revised: [Equation]

Original: e. the i th component of ˆ S θ ( x ) will denote the probability with which we select the i th feature).
Revised: e. the i th component of ˆ S θ ( x ) will denote the probability with which we select the i th feature).

Original: The selector network induces a probability distribution over the selection space ( { 0 , 1 } d ), with the probability of a given joint selection vector s ∈ { 0 , 1 } d being given by
Revised: The selector network induces a probability distribution over the selection space ( { 0 , 1 } d ), with the probability of a given joint selection vector s ∈ { 0 , 1 } d being given by

Original: [Equation]
Revised: [Equation]

Original: Using this
Revised: Using this

Original: [Equation]
Revised: [Equation]

Original: we deﬁne the following loss for our selector network
Revised: we deﬁne the following loss for our selector network

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: Taking the gradient of this loss with respect to θ gives us
Revised: Taking the gradient of this loss with respect to θ gives us

Original: [Equation]
Revised: [Equation]

Original: We update each of ˆ S θ , f φ and f γ iteratively using stochastic gradient descent.
Revised: We update each of ˆ S θ , f φ and f γ iteratively using stochastic gradient descent.

Original: Pseudo-code of
Revised: Pseudo-code of

Original: INVASE is given in Algorithm 1 and a block representation of INVASE can be found in Fig.
Revised: INVASE is given in Algorithm 1 and a block representation of INVASE can be found in Fig.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we quantitatively evaluate INVASE against various state-of-the-art benchmarks on both synthetic and real-world datasets.
Revised: In this section, we quantitatively evaluate INVASE against various state-of-the-art benchmarks on both synthetic and real-world datasets.

Original: We evaluate our performance both at identifying ground truth relevance and at enhancing predictions.
Revised: We evaluate our performance both at identifying ground truth relevance and at enhancing predictions.

Original: We compare our model with 4 global variable selection models: Knockoffs [3], Tree Ensembles (Tree) [6], Sequential Correlation Feature Selection (SCFS) [10], and LASSO regularized linear model; and 3 instance-wise feature selection methods: L2X [4], LIME [22], and Shapley [17].
Revised: We compare our model with 4 global variable selection models: Knockoffs [3], Tree Ensembles (Tree) [7], Sequential Correlation Feature Selection (SCFS) [11], and LASSO regularized linear model; and 3 instance-wise feature selection methods: L2X [4], LIME [24], and Shapley [18].

Original: The details of benchmark implementation can be found in the appendix.
Revised: The details of benchmark implementation can be found in the appendix.

Original: Implementation of INVASE
Revised: Implementation of INVASE

Original: 1: Inputs: learning rates α, β > 0 , mini-batch size n mb > 0 , dataset D 2: Initialize parameters θ, φ, γ 3: while Converge do 4:
Revised: 1: Inputs: learning rates α, β > 0 , mini-batch size n mb > 0 , dataset D 2: Initialize parameters θ, φ, γ 3: while Converge do 4:

Original: Sample a mini-batch from the dataset ( x j , y j ) n mb j =1 ∼ D 5: for j = 1 , ..., n mb do 6:
Revised: Sample a mini-batch from the dataset ( x j , y j ) n mb j =1 ∼ D 5: for j = 1 , ..., n mb do 6:

Original: Calculate selection probabilities ( p j 1 , ..., p jd ) ← ˆ S θ ( x j ) 7:
Revised: Calculate selection probabilities ( p j 1 , ..., p jd ) ← ˆ S θ ( x j ) 7:

Original: Sample selection vector 8: for i = 1 , ..., d do s ji ∼ Ber ( p ji ) 9:
Revised: Sample selection vector 8: for i = 1 , ..., d do s ji ∼ Ber ( p ji ) 9:

Original: Calculate loss ˆ l j ( x j , s j ) ← − (cid:34) c (cid:88) i
Revised: Calculate loss ˆ l j ( x j , s j ) ← − (cid:34) c (cid:88) i

Original: =
Revised: =

Original: y ji log( f φi ( x ( s j ) j , s j ))
Revised: y ji log( f φi ( x ( s j ) j , s j ))

Original: − c (cid:88) i =
Revised: − c (cid:88) i =

Original: y ji log( f γi ( x j )) (cid:35) 10:
Revised: y ji log( f γi ( x j )) (cid:35) 10:

Original: Update the selector network parameters θ θ ← θ − α 1 n mb n mb (cid:88)
Revised: Update the selector network parameters θ θ ← θ − α 1 n mb n mb (cid:88)

Original: j = (cid:16) ˆ l j ( x j , s j )
Revised: j = (cid:16) ˆ l j ( x j , s j )

Original: +
Revised: +

Original: λ
Revised: λ

Original: || s j || (cid:17) ∇ θ log π θ ( x j , s j ) 11:
Revised: || s j || (cid:17) ∇ θ log π θ ( x j , s j ) 11:

Original: Update the predictor network parameters φ φ ← φ − β 1 n mb n mb (cid:88)
Revised: Update the predictor network parameters φ φ ← φ − β 1 n mb n mb (cid:88)

Original: j = c (cid:88) i =
Revised: j = c (cid:88) i =

Original: y ji × ∇ φ log( f φi ( x ( s j ) j , s j )) 12:
Revised: y ji × ∇ φ log( f φi ( x ( s j ) j , s j )) 12:

Original: Update the baseline network parameters γ γ ← γ − β 1 n mb n mb (cid:88)
Revised: Update the baseline network parameters γ γ ← γ − β 1 n mb n mb (cid:88)

Original: j = c (cid:88) i =
Revised: j = c (cid:88) i =

Original: y ji × ∇ γ log( f γi ( x j ))
Revised: y ji × ∇ γ log( f γi ( x j ))

Original: 4. S YNTHETIC DATA EXPERIMENTS 4.1. E XPERIMENTAL SETTINGS
Revised: 4. S YNTHETIC DATA EXPERIMENTS 4.1. E XPERIMENTAL SETTINGS

Original: For our ﬁrst set of experiments, we use the same synthetic data generation models as in L2X [4].
Revised: For our ﬁrst set of experiments, we use the same synthetic data generation models as in L2X [4].

Original: The input features are generated from an 11-dimensional 67 Gaussian distribution with no correlations across the features ( X ∼ N ( 0 , I ) ).
Revised: The input features are generated from an 11-dimensional 67 Gaussian distribution with no correlations across the features ( X ∼ N ( 0 , I ) ).

Original: The label Y is sampled as a Bernoulli random variable with P ( Y = 1 | X ) = 1 1+ logit ( X ) , where logit ( X ) is varied to create 3 different synthetic datasets:
Revised: The label Y is sampled as a Bernoulli random variable with P ( Y = 1 | X ) = 1 1+ logit ( X ) , where logit ( X ) is varied to create 3 different synthetic datasets:

Original: • Syn1: exp( X 1 X 2 ) • Syn2: exp( (cid:80) 6 i =

Revised: • Syn1: exp( X 1 X 2 ) • Syn2: exp( (cid:80) 6 i =

Original: X 2 i − 4) • Syn3: − 10 × sin 2 X 7 + 2 | X 8 |
Revised: X 2 i − 4) • Syn3: − 10 × sin 2 X 7 + 2 | X 8 |

Original: + X 9 + exp( − X 10 )
Revised: + X 9 + exp( − X 10 )

Original: In each of these datasets, the label depends on the same subset of features for every sample.
Revised: In each of these datasets, the label depends on the same subset of features for every sample.

Original: To highlight the capability of INVASE to detect instance-wise dependence, we generate 3 further synthetic datasets as follows:
Revised: To highlight the capability of INVASE to detect instance-wise dependence, we generate 3 further synthetic datasets as follows:

Original: • Syn4:
Revised: • Syn4:

Original: If X 11 < 0 , logit follows Syn1 , otherwise, logit follows Syn2 .
Revised: If X 11 < 0 , logit follows Syn1 , otherwise, logit follows Syn2 .

Original: • Syn5:
Revised: • Syn5:

Original: If X 11 < 0 , logit follows Syn1 , otherwise, logit follows Syn3 .
Revised: If X 11 < 0 , logit follows Syn1 , otherwise, logit follows Syn3 .

Original: • Syn6:
Revised: • Syn6:

Original: If X 11 < 0 , logit follows Syn2 , otherwise, logit follows Syn3 .
Revised: If X 11 < 0 , logit follows Syn2 , otherwise, logit follows Syn3 .

Original: Note that in Syn4 and Syn5 , the number of relevant features is different for different samples.
Revised: Note that in Syn4 and Syn5 , the number of relevant features is different for different samples.

Original: For each of Syn1 to Syn6 we draw 20,000 samples from the data generation model and separate each into training ( D train =
Revised: For each of Syn1 to Syn6 we draw 20,000 samples from the data generation model and separate each into training ( D train =

Original: ( x i , y i ) 10000 i =1 ) and testing ( D test =
Revised: ( x i , y i ) 10000 i =1 ) and testing ( D test =

Original: ( x j , y j ) 

Revised: ( x j , y j )

Original: j =1 ) sets.
Revised: j =1 ) sets.

Original: For each method we try to ﬁnd the top k relevant features for each sample (we set k = 4 for Syn1, Syn2, Syn3,
Revised: For each method we try to ﬁnd the top k relevant features for each sample (we set k = 4 for Syn1, Syn2, Syn3,

Original: Syn4, Syn5 and k = 5 for Syn6 ), note, however, that k is not given as an input to INVASE (but is necessary for other methods).
Revised: Syn4, Syn5 and k = 5 for Syn6 ), note, however, that k is not given as an input to INVASE (but is necessary for other methods).

Original: The performance metrics we use are the true positive rate (TPR) (higher is better) and false discovery rate (FDR) 8 (lower is better) to measure the performance of the methods when the focus is on discovery (i.e. discovering which features are relevant) and we use Area Under the Receiver Operating Characteristic Curve (AUROC), Area Under the Precision
Revised: The performance metrics we use are the true positive rate (TPR) (higher is better) and false discovery rate (FDR) 8 (lower is better) to measure the performance of the methods when the focus is on discovery (i.e. discovering which features are relevant) and we use Area Under the Receiver Operating Characteristic Curve (AUROC), Area Under the Precision

Original: Recall Curve (AUPRC) and accuracy when the focus is on predictions.
Revised: Recall Curve (AUPRC) and accuracy when the focus is on predictions.

Original: 4.1. D ISCOVERY
Revised: 4.1. D ISCOVERY

Original: [Table]
Revised: [Table]

Original: As demonstrated by Table
Revised: As demonstrated by Table

Original: [Table]
Revised: [Table]

Original: our method is capable of detecting relevant features on a global level ( Syn1 , Syn2 and Syn3 ) as well as on an instance-wise level ( Syn4 , Syn5 and Syn6 ) outperforming all other methods in both cases (both global and instance-wise methods).
Revised: our method is capable of detecting relevant features on a global level ( Syn1 , Syn2 and Syn3 ) as well as on an instance-wise level ( Syn4 , Syn5 and Syn6 ) outperforming all other methods in both cases (both global and instance-wise methods).

Original: The particularly poor performance of some global feature selection methods in Syn1 , Syn2 and Syn3 (where there is no instance-wise relevance) is due to the non-linearity of the relationship between features and labels, further details can be found in the Appendix.
Revised: The particularly poor performance of some global feature selection methods in Syn1 , Syn2 and Syn3 (where there is no instance-wise relevance) is due to the non-linearity of the relationship between features and labels, further details can be found in the Appendix.

Original: The results for Syn4 , Syn5 and Syn6 demonstrate that INVASE is capable of detecting a different number of relevant features for each sample when necessary - the performance improvement over
Revised: The results for Syn4 , Syn5 and Syn6 demonstrate that INVASE is capable of detecting a different number of relevant features for each sample when necessary - the performance improvement over

Original: L2X is greater in Syn4 and Syn5 than Syn6 .
Revised: L2X is greater in Syn4 and Syn5 than Syn6 .

Original: In particular, in Syn4 , L2X is forced to overselect features when X 11 < 0 and underselect when X 11 ≥ 0 thus resulting in higher FDR and lower
Revised: In particular, in Syn4 , L2X is forced to overselect features when X 11 < 0 and underselect when X 11 ≥ 0 thus resulting in higher FDR and lower

Original: TPR, respectively.
Revised: TPR, respectively.

Original: To highlight this, in Table 2 we report the group speciﬁc FDR and TPR on Syn and Syn5 when setting k = 3 , 4 , 5 , where Group 1 refers to samples with X 11 < 0 and Group 2 to samples with X 11 ≥ 0 .
Revised: To highlight this, in Table 2 we report the group speciﬁc FDR and TPR on Syn and Syn5 when setting k = 3 , 4 , 5 , where Group 1 refers to samples with X 11 < 0 and Group 2 to samples with X 11 ≥ 0 .

Original: For k = 3 in Syn4 , we see that INVASE and L2X have comparable FDR in Group 1, since the total number of relevant features for each sample is 3 ( X 1 , X 2 , X 11 ).
Revised: For k = 3 in Syn4 , we see that INVASE and L2X have comparable FDR in Group 1, since the total number of relevant features for each sample is 3 ( X 1 , X 2 , X 11 ).

Original: However, when we increase k , we see that the FDR increases for L2X as it is forced to select more than 3 features, which necessarily means that the FDR must be at least 40% even if L2X was ﬁnding the relevant features perfectly.
Revised: However, when we increase k , we see that the FDR increases for L2X as it is forced to select more than 3 features, which necessarily means that the FDR must be at least 40% even if L2X was ﬁnding the relevant features perfectly.

Original: On the other hand, for Group 2 we see that the TPR is low for k = 3 since necessarily, L2X cannot possibly select all of the 5 relevant features.
Revised: On the other hand, for Group 2 we see that the TPR is low for k = 3 since necessarily, L2X cannot possibly select all of the 5 relevant features.

Original: INVASE, however, is able to select the correct number in both and hence enjoys low FDR and high TPR.
Revised: INVASE, however, is able to select the correct number in both and hence enjoys low FDR and high TPR.

Original: Syn5 reinforces the conclusions we drew for L2X in Syn4 .
Revised: Syn5 reinforces the conclusions we drew for L2X in Syn4 .

Original: Interestingly, though, for INVASE, we found that X 11 was almost never selected for Group 1 in Syn5 .
Revised: Interestingly, though, for INVASE, we found that X 11 was almost never selected for Group 1 in Syn5 .

Original: We believe this is because the lack of overlap between the relevant features for each group means that the predictor network can essentially learn two separate networks - one for each group.
Revised: We believe this is because the lack of overlap between the relevant features for each group means that the predictor network can essentially learn two separate networks - one for each group.

Original: This is because it is possible to create two subnetworks with non-overlapping weights that each take as input the features of a given group.
Revised: This is because it is possible to create two subnetworks with non-overlapping weights that each take as input the features of a given group.

Original: X 11 is therefore unnecessary for prediction.
Revised: X 11 is therefore unnecessary for prediction.

Original: Note, however, that X 11 is highly relevant for the selector network in deciding which features to pass on and so it is not true that X 11 isn’t relevant, but simply that the selector network does not need to “pass on” its relevance to the predictor network.
Revised: Note, however, that X 11 is highly relevant for the selector network in deciding which features to pass on and so it is not true that X 11 isn’t relevant, but simply that the selector network does not need to “pass on” its relevance to the predictor network.

Original: To investigate this further, results for settings where the features overlap between groups (and so it is not possible to disentangle the networks) can be found in the Appendix
Revised: To investigate this further, results for settings where the features overlap between groups (and so it is not possible to disentangle the networks) can be found in the Appendix

Original: [Table]
Revised: [Table]

Original: P REDICTION
Revised: P REDICTION

Original: In this experiment we analyze the effect of using feature selection as a pre-processing step for prediction.
Revised: In this experiment we analyze the effect of using feature selection as a pre-processing step for prediction.

Original: We ﬁrst perform feature selection (either instance-wise or global) and then train a 3-layer fully connected network with Batch Normalization [11] in every layer (to avoid overﬁtting) to perform predictions on top of the (feature-selected) data.
Revised: We ﬁrst perform feature selection (either instance-wise or global) and then train a 3-layer fully connected network with Batch Normalization [12] in every layer (to avoid overﬁtting) to perform predictions on top of the (feature-selected) data.

Original: In this setting we compare the two global feature selection methods (LASSO and Tree) and one instance-wise feature selection method (L2X).
Revised: In this setting we compare the two global feature selection methods (LASSO and Tree) and one instance-wise feature selection method (L2X).

Original: Furthermore, we also compare with the predictive model without any feature selections (w/o FS) and the predictive model with ground truth globally relevant features 9 (with Global).
Revised: Furthermore, we also compare with the predictive model without any feature selections (w/o FS) and the predictive model with ground truth globally relevant features 9 (with Global).

Original: In particular, this allows us to demonstrate that the improvements in prediction performance are not just because the global feature selection performed implicitly by INVASE is better than the other global feature selection methods but are also due to the fact that we select features on an instance-wise level.
Revised: In particular, this allows us to demonstrate that the improvements in prediction performance are not just because the global feature selection performed implicitly by INVASE is better than the other global feature selection methods but are also due to the fact that we select features on an instance-wise level.

Original: Experiments here are conducted on synthetic data with 100 features but the same labelling procedures as above.
Revised: Experiments here are conducted on synthetic data with 100 features but the same labelling procedures as above.

Original: As can be seen in Table 3, there is a signiﬁcant performance improvement when discarding all of the irrelevant features ( with Global ).
Revised: As can be seen in Table 3, there is a signiﬁcant performance improvement when discarding all of the irrelevant features ( with Global ).

Original: However, neither of the global feature selection methods ( Tree and Lasso ) are capable of achieving this improvement.
Revised: However, neither of the global feature selection methods ( Tree and Lasso ) are capable of achieving this improvement.

Original: On the other hand, INVASE is capable of achieving (and beating - in Syn4 and Syn6 ) this improvement, demonstrating its capability both at selecting features globally better than existing methods but also at improving on global selection with instance-wise selection (where relevant), to provide further improvements.
Revised: On the other hand, INVASE is capable of achieving (and beating - in Syn4 and Syn6 ) this improvement, demonstrating its capability both at selecting features globally better than existing methods but also at improving on global selection with instance-wise selection (where relevant), to provide further improvements.

Original: On the other hand,
Revised: On the other hand,

Original: L2X performs worse than the global methods in Syn1-3 , demonstrating an inability to perform even global feature selection in this higher dimensional setting (this is supported by the high dimensional discovery results in the Appendix), and in Syn4-6 is performing worse than with Global (which now is not even optimal).
Revised: L2X performs worse than the global methods in Syn1-3 , demonstrating an inability to perform even global feature selection in this higher dimensional setting (this is supported by the high dimensional discovery results in the Appendix), and in Syn4-6 is performing worse than with Global (which now is not even optimal).

Original: Furthermore, even though we include Batch Normalization to avoid overﬁtting, with a small number of samples and high number of dimensions, the 3-layer fully connected network still suffers from overﬁtting as demonstrated by the signiﬁcant difference in performance between w/o FS and with
Revised: Furthermore, even though we include Batch Normalization to avoid overﬁtting, with a small number of samples and high number of dimensions, the 3-layer fully connected network still suffers from overﬁtting as demonstrated by the signiﬁcant difference in performance between w/o FS and with

Original: Global .
Revised: Global .

Original: This demonstrates the necessity of feature selection as a pre-processing step.
Revised: This demonstrates the necessity of feature selection as a pre-processing step.

Original: Lastly, in comparison to with Global , with INVASE achieves performance gains in Syn4 and Syn6 .
Revised: Lastly, in comparison to with Global , with INVASE achieves performance gains in Syn4 and Syn6 .

Original: It quantitatively shows that instance-wise feature selection can further improves the predictive model from ground truth global feature selection.
Revised: It quantitatively shows that instance-wise feature selection can further improves the predictive model from ground truth global feature selection.

Original: [Table]
Revised: [Table]

Original: 4. R EAL - WORLD DATA EXPERIMENTS 4.2. D ATA D ESCRIPTION
Revised: 4. R EAL - WORLD DATA EXPERIMENTS 4.2. D ATA D ESCRIPTION

Original: In this section we use two real-world datasets to perform a series of further experiments.
Revised: In this section we use two real-world datasets to perform a series of further experiments.

Original: The ﬁrst
Revised: The ﬁrst, the

Original: [Table]

Original: the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) dataset [21] [Table] has 40,409 patients each with 31 measured features.
Revised: Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) dataset [23], has 40,409 patients each with 31 measured features.

Original: The label is all-cause mortality.
Revised: The label is all-cause mortality.

Original: The second, the Prostate, Lung,
Revised: The second, the Prostate, Lung,

Original: Colorectal and Ovarian (PLCO) Cancer Screening Trial in the US and the European Randomized
Revised: Colorectal and Ovarian (PLCO) Cancer Screening Trial in the US and the European Randomized

Original: Study of Screening for Prostate Cancer (ERSPC) dataset [7; 23] contains 38,001 each withmeasured features.
Revised: Study of Screening for Prostate Cancer (ERSPC) dataset [8; 26] contains 38,001 each withmeasured features.

Original: The label in this dataset is mortality due to prostate cancer.
Revised: The label in this dataset is mortality due to prostate cancer.

Original: We refer to this as the PLCO dataset.
Revised: We refer to this as the PLCO dataset.

Original: The ﬁrst experiment we carried out was to create semi-synthetic datasets by using the labelling procedures Syn1-6 from above but with the features now coming from real data (instead of being i.i.d.
Revised: The ﬁrst experiment we carried out was to create semi-synthetic datasets by using the labelling procedures Syn1-6 from above but with the features now coming from real data (instead of being i.i.d.

Original: Gaussian).
Revised: Gaussian).

Original: The results of this experiment can be found in the Appendix.
Revised: The results of this experiment can be found in the Appendix.

Original: 4.2. T HE DISCOVERED FEATURE IMPORTANCE IN MAGGIC DATASET
Revised: 4.2. T HE DISCOVERED FEATURE IMPORTANCE IN MAGGIC DATASET

Original: In this next experiment
Revised: In this next experiment

Original: [Table]
Revised: [Table]

Original: we visualize the ability of INVASE to select features on an individualized level.
Revised: we visualize the ability of INVASE to select features on an individualized level.

Original: Fig.
Revised: Fig.

Original: 2(left) shows the selection probability (given by INVASE) of each feature for 20 randomly selected patients in the MAGGIC dataset.
Revised: 2(left) shows the selection probability (given by INVASE) of each feature for 20 randomly selected patients in the MAGGIC dataset.

Original: Fig. 2(right) shows the selection probability of each feature averaged over different binary splits of the data (i
Revised: Fig. 2(right) shows the selection probability of each feature averaged over different binary splits of the data (i

Original: [Table]
Revised: [Table]

Original: e. when split into Male and Female).
Revised: e. when split into Male and Female).

Original: In
Revised: In

Original: Table 4, we also report the mean and variance of the number of selected features in each subgroup.
Revised: Table 4, we also report the mean and variance of the number of selected features in each subgroup.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: As can be seen, INVASE discovers signiﬁcantly different features for both individuals and for different subgroups of the dataset.
Revised: As can be seen [Table] INVASE discovers signiﬁcantly different features for both individuals and for different subgroups of the dataset.

Original: 4.2. R ESULTS: P REDICTION USING REAL DATA VARIABLES WITH REAL LABEL
Revised: ESULTS

Original: Evaluating the performance of feature selection methods on real data is difﬁcult, since ground truth relevance is often not known.
Revised: Evaluating the performance of feature selection methods on real data is difﬁcult, since ground truth relevance is often not known.

Original: We therefore cannot use TPR and FDR to evaluate the performance on real data.
Revised: We therefore cannot use TPR and FDR to evaluate the performance on real data.

Original: In our ﬁnal experiment, therefore, we instead focus on prediction performance exactly as in 4.1.3 (except now both the features and label come from real data).
Revised: In our ﬁnal experiment, therefore, we instead focus on prediction performance exactly as in 4.1.3 (except now both the features and label come from real data).

Original: [Table]
Revised: [Table]

Original: As can be seen in Table 5, INVASE consistently improves prediction performance in each of the two settings (different time horizons) in each dataset.
Revised: As can be seen in Table [Table] INVASE consistently improves prediction performance in each of the two settings (different time horizons) in each dataset.

Original:  F UTURE W ORK
Revised:  F UTURE W ORK

Original: While this paper has focused on discovering relevant features in the static setting, this could also be extended to apply in the temporal setting.
Revised: While this paper has focused on discovering relevant features in the static setting, this could also be extended to apply in the temporal setting.

Original: One such avenue of exploration for this would be to replace each of the networks with an RNN.
Revised: One such avenue of exploration for this would be to replace each of the networks with an RNN.

Original: Particular care will need to be taken in deﬁning the problem, though; do we treat each stream as a feature or each time point of each stream?
Revised: Particular care will need to be taken in deﬁning the problem, though; do we treat each stream as a feature or each time point of each stream?

Original: We leave this investigation to future work.
Revised: We leave this investigation to future work.

Revised: Learning important features

Original: A PPENDIX S UMMARY OF R ELATED W ORKS
Revised: A PPENDIX S UMMARY OF R ELATED W ORKS

Original: Key ideas
Revised: Key ideas

Original: Experiments
Revised: Experiments

Original: Global/
Revised: Global/

Original: Model # of relevant shown
Revised: Model # of relevant shown

Original: Instance-wise agnostic features
Revised: Instance-wise agnostic features

Original: SCFS
Revised: SCFS

Original: Max-dependency min-redundancy
Revised: Max-dependency min-redundancy

Original: Feature selection
Revised: Feature selection

Original: Global
Revised: Global

Original: Yes
Revised: Yes

Original: Not needed [10] criteria with Pearson correlations
Revised: Not needed [11] criteria with Pearson correlations

Original: MIFS
Revised: MIFS

Original: Max-dependency min-redundancy
Revised: Max-dependency min-redundancy

Original: Feature selection
Revised: Feature selection

Original: Global
Revised: Global

Original: Yes
Revised: Yes

Original: Not needed [19] criteria with Mutual Information
Revised: Not needed [21] criteria with Mutual Information

Original: LASSO
Revised: LASSO

Original: Linear regression
Revised: Linear regression

Original: Feature selection
Revised: Feature selection

Original: Global
Revised: Global

Original: Yes
Revised: Yes

Original: Not needed [27] with l 1 -norm penalty
Revised: Not needed [31] with l 1 -norm penalty

Original: Prediction
Revised: Prediction

Original: Knock-off
Revised: Knock-off

Original: Comparison between knock-off
Revised: Comparison between knock-off

Original: Feature selection
Revised: Feature selection

Original: Global
Revised: Global

Original: Yes
Revised: Yes

Original: Not needed [3] variables and real variables
Revised: Not needed [3] variables and real variables

Original: Hypothesis test
Revised: Hypothesis test

Original: L2X
Revised: L2X

Original: Mutual Information maximization
Revised: Mutual Information maximization

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: Yes
Revised: Yes

Original: Should be [4] with Gumbel-softmax given
Revised: Should be [4] with Gumbel-softmax given

Original: LIME
Revised: LIME

Original: Locally linear
Revised: Locally linear

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: Yes
Revised: Yes

Original: Should be [22] approximation given
Revised: Should be [24] approximation given

Original: Shapley
Revised: Shapley

Original: Shapley value estimation
Revised: Shapley value estimation

Original: Feature selection
Revised: Feature selection

Original: Instance-wise
Revised: Instance-wise

Original: Yes
Revised: Yes

Original: Should be [17] to quantify feature importance given
Revised: Should be [18] to quantify feature importance given

Original: DeepLIFT
Revised: DeepLIFT

Original: Decompose the output of
Revised: Decompose the output of

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: No
Revised: No

Original: Should be [24]
Revised: Should be [27]

Original: NN on a reference input given
Revised: NN on a reference input given

Original: Saliency
Revised: Saliency

Original: Backpropagation from the
Revised: Backpropagation from the

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: No
Revised: No

Original: Should be [25] output of the NN to the input given
Revised: Should be [29] output of the NN to the input given

Original: Tree SHAP
Revised: Tree SHAP

Original: Shapley value estimation
Revised: Shapley value estimation

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: No
Revised: No

Original: Should be [18] only for tree-ensemble models given
Revised: Should be [19] only for tree-ensemble models given

Original: Pixel-wise
Revised: Pixel-wise

Original: Measuring the effects on
Revised: Measuring the effects on

Original: Interpretation
Revised: Interpretation

Original: Instance-wise
Revised: Instance-wise

Original: No
Revised: No

Original: Should be [1] the output using input perturbation given
Revised: Should be [1] the output using input perturbation given

Original: INVASE
Revised: INVASE

Original: Minimize KL divergence using
Revised: Minimize KL divergence using

Original: Feature selection
Revised: Feature selection

Original: Instance-wise
Revised: Instance-wise

Original: Yes
Revised: Yes

Original: Not needed (Ours) deep NN inﬂuenced by
Revised: Not needed (Ours) deep NN inﬂuenced by

Original: Interpretation actor-critic models
Revised: Interpretation actor-critic models

Original: Prediction
Revised: Prediction

Original: E XTENDING INVASE TO REGRESSION
Revised: E XTENDING INVASE TO REGRESSION

Original: To extend our model to the setting where Y is continuous (regression problem), we replace the estimated loss with the reconstruction error as follows.
Revised: To extend our model to the setting where Y is continuous (regression problem), we replace the estimated loss with the reconstruction error as follows.

Original: [Equation]
Revised: [Equation]

Original: where f φ : X → R is now the (continuous) predictor function trained to minimize the (cid:96) 2 -norm between its outputs and the real labels.
Revised: where f φ : X → R is now the (continuous) predictor function trained to minimize the (cid:96) 2 -norm between its outputs and the real labels.

Original: As noted in [8], when the distribution of Y given X is
Revised: As noted in [9], when the distribution of Y given X is

Original: Gaussian, minimizing the l 2 -norm is equivalent to minimizing the KL divergence.
Revised: Gaussian, minimizing the l 2 -norm is equivalent to minimizing the KL divergence.

Original: D ETAILS OF INVASE
Revised: D ETAILS OF INVASE

Original: In the experiments, the depth of the selector, predictor, and baseline networks is set to 3.
Revised: In the experiments, the depth of the selector, predictor, and baseline networks is set to 3.

Original: The number of hidden nodes in each layer is d and 2 d , respectively.
Revised: The number of hidden nodes in each layer is d and 2 d , respectively.

Original: We use either ReLu or SeLu as the activation functions of each layer except for the output layer where we use the sigmoid activation function for the selector network and softmax activation function for the predictor and baseline networks.
Revised: We use either ReLu or SeLu as the activation functions of each layer except for the output layer where we use the sigmoid activation function for the selector network and softmax activation function for the predictor and baseline networks.

Original: The number of samples in each mini-batch is 1000 for the selector, predictor, and baseline networks.
Revised: The number of samples in each mini-batch is 1000 for the selector, predictor, and baseline networks.

Original: We use cross-validation to select λ among { 0 . 1 , 0 . 3 , 0 . 5 , 1 , 2 , 5 , 10 } .
Revised: We use cross-validation to select λ among { 0 . 1 , 0 . 3 , 0 . 5 , 1 , 2 , 5 , 10 } .

Original: We use tensorﬂow to implement INVASE.
Revised: We use tensorﬂow to implement INVASE.

Original: The source-code can be found at https://github.com/iclr2018invase/
Revised: The source-code can be found at https://github.com/iclr2018invase/

Original: INVASE/ .
Revised: INVASE/ .

Original: D ETAILS OF B ENCHMARKS
Revised: D ETAILS OF B ENCHMARKS

Original: We use the following links for the implementations of 7 benchmarks.
Revised: We use the following links for the implementations of 7 benchmarks.

Original: • L2X: https://github.com/Jianbo-Lab/L2X • LIME:
Revised: • L2X: https://github.com/Jianbo-Lab/L2X • LIME:

Original: https://github.com/marcotcr/lime • Shapley: https://github.com/slundberg/shap • Knock-off: http://web.stanford.edu/group/candes/knockoffs/ software/knockoff/ •
Revised: https://github.com/marcotcr/lime • Shapley: https://github.com/slundberg/shap • Knock-off: http://web.stanford.edu/group/candes/knockoffs/ software/knockoff/ •

Original: Tree: http://scikit-learn.org/stable/modules/generated/ sklearn.ensemble.ExtraTreesClassifier.html • LASSO: http://scikit-learn.org/stable/modules/linear_model.
Revised: Tree: http://scikit-learn.org/stable/modules/generated/ sklearn.ensemble.ExtraTreesClassifier.html • LASSO: http://scikit-learn.org/stable/modules/linear_model.

Original: html#lasso
Revised: html#lasso

Original: For L2X, we use the same network settings used in INVASE for fair comparisons.
Revised: For L2X, we use the same network settings used in INVASE for fair comparisons.

Original: For SCFS, we explicitly implement from the reference ([10]).
Revised: For SCFS, we explicitly implement from the reference ([11]).

Original: H IGH DIMENSIONAL DISCOVERY
Revised: H IGH DIMENSIONAL DISCOVERY

Original: To demonstrate the scalability of our method
Revised: To demonstrate the scalability of our method

Original: [Table]
Revised: [Table]

Original: we run an experiment in which we increase the total number of features to 100.
Revised: we run an experiment in which we increase the total number of features to 100.

Original: The features are generated as a 100-dimensional Gaussian with no correlations ( N ( 0 , I ) ) and the relationships between features and label remains as in Table 1 in the main manucript (i
Revised: The features are generated as a 100-dimensional Gaussian with no correlations ( N ( 0 , I ) ) and the relationships between features and label remains as in Table 1 in the main manucript (i

Original: [Table]
Revised: [Table]

Original: e. we are adding 89 additional noisy signals that have no effect on the label)
Revised: e. we are adding 89 additional noisy signals that have no effect on the label)

Original: [Table]
Revised: [Table]

Original: As can be seen in Table
Revised: As can be seen in Table

Original: [Table]
Revised: [Table]

Original: INVASE also works consistently better than all other benchmarks in allsynthetic datasets in this setting.
Revised: INVASE also works consistently better than all other benchmarks in allsynthetic datasets in this setting.

Original: In fact, we see a signiﬁcant reduction in performance (compared to the 11 feature setting) for L2X in Syn1 , with the TPR dropping more than 90% leading to an almost complete failure of the method to detect any relevant features.
Revised: In fact, we see a signiﬁcant reduction in performance (compared to the 11 feature setting) for L2X in Syn1 , with the TPR dropping more than 90% leading to an almost complete failure of the method to detect any relevant features.

Original: In particular, we see that L2X does not scale as well as INVASE with the dimensionality of the data, which is particularly limiting for a feature selection method.
Revised: In particular, we see that L2X does not scale as well as INVASE with the dimensionality of the data, which is particularly limiting for a feature selection method.

Original: We also compare the CPU times of the algorithm for training and testing with other instance-wise feature selection benchmarks to show the scalability in terms of computational complexity.
Revised: We also compare the CPU times of the algorithm for training and testing with other instance-wise feature selection benchmarks to show the scalability in terms of computational complexity.

Original: As can be seen in Table 8, INVASE is much faster (10 times) than LIME and Shapley methods and comparable with L2X; we see that INVASE takes approximately 50% longer to run than L2X, which can be accounted for by the addition of a 3rd network (the baseline network) in INVASE that is not present in L2X. Note, however, that this baseline network can be trained in parallel with the predictor network and we believe that doing so would lead to both INVASE and L2X having the same run-time. 
Revised: As can be seen in Table 8, INVASE is much faster (10 times) than LIME and Shapley methods and comparable with L2X; we see that INVASE takes approximately 50% longer to run than L2X, which can be accounted for by the addition of a 3rd network (the baseline network) in INVASE that is not present in L2X. Note, however, that this baseline network can be trained in parallel with the predictor network and we believe that doing so would lead to both INVASE and L2X having the same run-time. [Table] H YPER - PARAMETER A NALYSIS In the following experiment

Original: [Table]
Revised: [Table]

Revised: we provide results for various values of the hyper-parameter, λ , in the

Revised: Syn4, Syn

Revised: [Table]

Revised: and Syn6 100-dimensional setting.

Revised: Table 9 gives the results in terms of TPR and FDR.

Revised: Note that in the other experiments, we select the hyper-parameter λ which maximizes the predictor accuracy in terms of AUROC.

Revised: [Table]

Original: A DDITIONAL R ESULTS ON C OMPLEX S YNTHETIC D ATASETS
Revised: A DDITIONAL R ESULTS ON C OMPLEX S YNTHETIC D ATASETS

Original: In the main paper, the relevant subset for a sample in each of our variable synthetic datasets ( Syn46 ) depended on X 11 only, which was unused in the rest of the model (i.e. X 11 determined only the relevant subset, and was otherwise unused as a predictive variable).
Revised: In the main paper, the relevant subset for a sample in each of our variable synthetic datasets ( Syn46 ) depended on X 11 only, which was unused in the rest of the model (i.e. X 11 determined only the relevant subset, and was otherwise unused as a predictive variable).

Original: In this set of experiments, we investigate the effect of having the subset relevance depend on a variable that is also used in the model itself ( Syn4A, Syn5A, Syn6A ).
Revised: In this set of experiments, we investigate the effect of having the subset relevance depend on a variable that is also used in the model itself ( Syn4A, Syn5A, Syn6A ).

Original: We then investigate the effect of having more than one variable being used to determine subset relevance ( Syn4B, Syn5B, Syn6B, Syn7 ).
Revised: We then investigate the effect of having more than one variable being used to determine subset relevance ( Syn4B, Syn5B, Syn6B, Syn7 ).

Original: The results for these are reported in Tables 9 and 10, respectively.
Revised: The results for these are reported in Tables 10 and 11, respectively.

Original: The input features are generated from a 100-dimensional Gaussian distribution with no correlations across the features ( X ∼ N ( 0 , I ) ).
Revised: The input features are generated from a 100-dimensional Gaussian distribution with no correlations across the features ( X ∼ N ( 0 , I ) ).

Original: Y is generated according to P ( Y = 1 | X ) = 1 1+ logit ( X ) with the logit value for each synthetic dataset now deﬁned as follows:
Revised: Y is generated according to P ( Y = 1 | X ) = 1 1+ logit ( X ) with the logit value for each synthetic dataset now deﬁned as follows:

Original: • Syn4A: If X 1 < 0 , logit =
Revised: • Syn4A: If X 1 < 0 , logit =

Original: exp( X 1 X 2 ) , otherwise, logit =
Revised: exp( X 1 X 2 ) , otherwise, logit =

Original: exp( (cid:80) 6 i =

Revised: exp( (cid:80) 6 i =

Original: X 2 i − 4) .
Revised: X 2 i − 4) .

Original: • Syn5A:
Revised: • Syn5A:

Original: If X 1 < 0 , logit =
Revised: If X 1 < 0 , logit =

Original: exp( X 1 X 2 ) , otherwise, logit = − 10 × sin 2 X 7 + 

Revised: exp( X 1 X 2 ) , otherwise, logit = − 10 × sin 2 X 7 +

Original: |
Revised: |

Original: X 8 | + X 9 + exp( − X 10 ) .
Revised: X 8 | + X 9 + exp( − X 10 ) .

Original: • Syn6A: If X 7 < 0 , logit =
Revised: • Syn6A: If X 7 < 0 , logit =

Original: exp( (cid:80) 6 i =

Revised: exp( (cid:80) 6 i =

Original: X 2 i − 4) , otherwise, logit = 
Revised: X 2 i − 4) , otherwise, logit = − 10 × sin 2 X 7 +2 | X 8 | + X 9 + exp( − X 10 ) . Dataset Syn4A

Revised: Syn5A

Revised: Syn6A

Original: [Table]
Revised: Metrics (%) TPR FDR TPR

Revised: FDR

Revised: TPR

Revised: FDR

Revised: INVASE+ 77.

Revised: L2X 65.

Revised: [Table]

Original: • Syn4B: If X 1 X 3 < 0 , logit =
Revised: • Syn4B: If X 1 X 3 < 0 , logit =

Original: exp( X 1 X 2 ) , otherwise, logit =
Revised: exp( X 1 X 2 ) , otherwise, logit =

Original: exp( (cid:80) 6 i =

Revised: exp( (cid:80) 6 i =

Original: X 2 i − 4) .
Revised: X 2 i − 4) .

Original: Syn5B: If X 1 X 7 < 0 , logit =
Revised: Syn5B: If X 1 X 7 < 0 , logit =

Original: exp( X 1 X 2 ) , otherwise, logit = − 10 × sin 2 X 7 + 

Revised: exp( X 1 X 2 ) , otherwise, logit = − 10 × sin 2 X 7 +

Original: |
Revised: |

Original: X 8 | + X 9 + exp( − X 10 ) .
Revised: X 8 | + X 9 + exp( − X 10 ) .

Original: • Syn6B:
Revised: • Syn6B:

Original: If X 3 X 7 < 0 , logit =
Revised: If X 3 X 7 < 0 , logit =

Original: exp( (cid:80) 6 i =

Revised: exp( (cid:80) 6 i =

Original: X 2 i − 4) , otherwise, logit = − 10 × sin 2 X 7 + 

Revised: X 2 i − 4) , otherwise, logit = − 10 × sin 2 X 7 +

Original: |
Revised: |

Original: X 8 | + X 9 + exp( − X 10 ) .
Revised: X 8 | + X 9 + exp( − X 10 ) .

Original: • Syn7:
Revised: • Syn7:

Original: – If X 1 < 0 , X 2 < 0 , logit =
Revised: – If X 1 < 0 , X 2 < 0 , logit =

Original: exp( X 1 X 2 ) – If X 1 < 0 , X 2 ≥ 0 , logit =
Revised: exp( X 1 X 2 ) – If X 1 < 0 , X 2 ≥ 0 , logit =

Original: exp( (cid:80) 6 i =

Revised: exp( (cid:80) 6 i =

Original: X 2 i − 4) .
Revised: X 2 i − 4) .

Original: – If X 1 ≥ 0 , X 2 < 0 , logit = − 10 × sin 2 X 7 + 2 | X 8 |
Revised: – If X 1 ≥ 0 , X 2 < 0 , logit = − 10 × sin 2 X 7 + 2 | X 8 |

Original: + X 9 + exp( − X 10 ) .
Revised: + X 9 + exp( − X 10 ) .

Original: – If X 1 ≥ 0 , X 2 ≥ 0 , logit = 0 .
Revised: – If X 1 ≥ 0 , X 2 ≥ 0 , logit = 0 .

Original: 5 × exp( X 1 X 2 ) + 0 .
Revised: 5 × exp( X 1 X 2 ) + 0 .

Original: 5 × exp( (cid:80) 4 i =

Revised: 5 × exp( (cid:80) 4 i =

Original: X 2 i − 2) .
Revised: X 2 i − 2) .

Original: [Table]

Original: R ESULTS ON SEMI - SYNTHETIC DATASETS
Revised: R ESULTS ON SEMI - SYNTHETIC DATASETS

Original: In this experiment, we use real features (which have correlation across features) but generate the labels as in the synthetic experiments from the main paper, using Syn1 - Syn6 .
Revised: In this experiment, we use real features (which have correlation across features) but generate the labels as in the synthetic experiments from the main paper, using Syn1 - Syn6 .

Original: This allows us to know the ground truth relevance of the features, and calculate TPR and FDR, while using unknown and correlated feature distributions (instead of the unrealistic setting of i [Table]
Revised: This allows us to know the ground truth relevance of the features, and calculate TPR and FDR, while using unknown and correlated feature distributions (instead of the unrealistic setting of i.i.d.

Original: i.d. Gaussian used in the fully synthetic experiment).
Revised: Gaussian used in the fully synthetic experiment).

Original: The results for the MAGGIC and PLCO datasets are given below.
Revised: The results for the MAGGIC and PLCO datasets are given below.

Original: [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: As demonstrated in Tables 12 and

Original: [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: INVASE outperforms all other methods across all 6 of the synthetic-label settings using real features.

Original: [Table] [Table] [Table] [Table]
Revised: This also demonstrates the capability of INVASE in settings where there are unknown correlation structures in the features.

Original: [Table] [Table] [Table] [Table]
Revised: 5. P REDICTIVE P ERFORMANCE C OMPARISON ON REAL - WORLD DATASETS

Original: [Table] [Table] [Table] [Table]
Revised: In this experiment, we evaluate the predictive performance gains of using each feature selection method as a pre-processing step on the two real datasets, MAGGIC and PLCO (as was done for synthetic data in Section 4.1.3).

Original: [Table] [Table] [Table] [Table]
Revised: For each method, we ﬁrst perform feature selection and then train a predictive model on top of the feature-selected data, where the model has the same architecture as the INVASE predictor network (to create a fair comparison of methods).

Original: [Table] [Table] [Table] As demonstrated in Tables 11 and 12, INVASE outperforms all other methods across all 6 of the synthetic-label settings using real features.
Revised: As can be seen in Table 14, INVASE signiﬁcantly outperform the other approaches.

Original: This also demonstrates the capability of INVASE in settings where there are unknown correlation structures in the features.
Revised: [Table]

Original:  C ORRELATIONS BETWEEN FEATURES AND LABELS IN THE SYNTHETIC AND SEMI - SYNTHETIC EXPERIMENTS
Revised: [Table] [Table] [Table] C ORRELATIONS BETWEEN FEATURES AND LABELS IN THE SYNTHETIC AND SEMI - SYNTHETIC EXPERIMENTS

Original: [Table]
Revised: [Table]

Original: As can be seen in Table [Table] among 33 relevant features, only 9 features have more than 0.05 (linear) correlation with the label.
Revised: As can be seen in Table 15, among 33 relevant features, only 9 features have more than 0.05 (linear) correlation with the label.

Original: In particular, using a linear model, it is very hard to discover the relevant features.
Revised: In particular, using a linear model, it is very hard to discover the relevant features.

Original: However, Knock-off (based on LASSO and linear correlations), LASSO, and SCFS are linear models, resulting in a poor performance in our experiments.
Revised: However, Knock-off (based on LASSO and linear correlations), LASSO, and SCFS are linear models, resulting in a poor performance in our experiments.

Original: The above table results are directly reﬂected in the results given in the main manuscript.
Revised: The above table results are directly reﬂected in the results given in the main manuscript.

Original: Variables

Original: [Table]
Revised: [Table]

Original: We do the same analysis for the MAGGIC dataset; results are given in Table 14.
Revised: We do the same analysis for the MAGGIC dataset; results are given in Table 16.

Original: We see that here the linear correlation with the label is stronger and this is reﬂected in Tables 11 and 12, where all of the linear models performed better than in the fully-synthetic settings.
Revised: We see that here the linear correlation with the label is stronger and this is reﬂected in Tables 12 and 13, where all of the linear models performed better than in the fully-synthetic settings.

Original: However, we note that although they had a better performance, in most cases it was still not comparable with INVASE.
Revised: However, we note that although they had a better performance, in most cases it was still not comparable with INVASE.

Original: D EFINITION OF TPR AND FDR
Revised: D EFINITION OF TPR AND FDR

Original: [Table]
Revised: [Table]

Revised: C OMPUTER V ISION

Revised: Another natural application of INVASE is in computer vision.

Revised: To brieﬂy demonstrate the applicability and capability of INVASE to computer vision, we conduct two experiments using the Kaggle

Revised: Dogs vs. Cats dataset ( https://www.kaggle.com/c/dogs-vs-cats ) [6] and the Oxford

Revised: Pet dataset ( http://www.robots.ox.ac.uk/˜vgg/data/pets/ ) [20].

Revised: The goal is to select a set 16 x 16 patches of each image that maximize the predictive capability of a model.

Revised: In order to apply INVASE to this problem, we simply treat each 16 x 16 patch as a feature.

Revised: We use the U-Net [25] architecture for the selector network and the VGG network [28] architecture for the predictor and baseline networks.

Revised: Below we give qualitative results of INVASE applied to these datasets, where we see that INVASE successfully identiﬁes patches of each image in which the animal’s face is visible.

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]