Year: 2020

Abstract:
In the past few years, various approaches have been developed to explain and interpret deep neural network (DNN) representations, but it has been pointed out that these representations are sometimes unstable and not reproducible. In this paper, we interpret these representations as hypotheses driven by DNN (called DNN-driven hypotheses) and propose a method to quantify the reliability of these hypotheses in statistical hypothesis testing framework. To this end, we introduce Selective Inference (SI) framework, which has received much attention in the past few years as a new statistical inference framework for data-driven hypotheses. The basic idea of SI is to make conditional inferences on the selected hypotheses under the condition that they are selected. In order to use SI framework for DNN representations, we develop a new SI algorithm based on homotopy method which enables us to derive the exact (non-asymptotic) conditional sampling distribution of the DNN-driven hypotheses. In this paper, we demonstrate the proposed method in computer vision tasks as practical examples. We conduct experiments on both synthetic and real-world datasets, through which we offer evidence that our proposed method can successfully control the false positive rate, has decent performance in terms of computational efficiency, and provides good results in practical applications.

heading: 1 INTRODUCTION
text: The remarkable predictive performance of deep neural networks (DNNs) stems from their ability to learn appropriate representations from data. In order to understand the decision-making process of DNNs, it is thus important to be able to explain and interpret DNN representations. For example, in image classification tasks, knowing the attention region from DNN representation allows us to understand the reason for the classification. In the past few years, several methods have been developed to explain and interpret DNN representations (Ribeiro et al., 2016; Bach et al., 2015; Doshi-Velez & Kim, 2017; Lundberg & Lee, 2017; Zhou et al., 2016; Selvaraju et al., 2017); however, some of them have turned out to be unstable and not reproducible (Kindermans et al., 2017; Ghorbani et al., 2019; Melis & Jaakkola, 2018; Zhang et al., 2020; Dombrowski et al., 2019; Heo et al., 2019). Therefore, it is crucially important to develop a method to quantify the reliability of DNN representations.
In this paper, we interpret these representations as hypotheses that are driven by DNN (called DNNdriven hypotheses) and employ statistical hypothesis testing framework to quantify the reliability of DNN representations. For example, in an image classification task, the reliability of an attention region can be quantified based on the statistical significance of the difference between the attention region and the rest of the image. Unfortunately, however, traditional statistical test cannot be applied to this problem because the hypothesis (attention region in the above example) itself is selected by the data. Traditional statistical test is valid only when the hypothesis is non-random. Roughly speaking, if a hypothesis is selected by the data, the hypothesis will over-fit to the data and the bias needs to be corrected when assessing the reliability of the hypothesis.
Our main contribution in this paper is to introduce Selective Inference (SI) approach for testing the reliability of DNN representations. The basic idea of SI is to perform statistical inference under the condition that the hypothesis is selected. SI approach has been demonstrated to be effective
in the context of feature selections such as Lasso. In this paper, in order to introduce SI for DNN representations, we develop a novel SI algorithm based on homotopy method, which enables us to derive the exact (non-asymptotic) conditional sampling distribution of the DNN-driven hypothesis. We use p-value as a criterion to quantify the reliability of DNN representation. In the literature, pvalues are often misinterpreted and there are various source of mis-interpretation has been discussed (Wasserstein & Lazar, 2016). In this paper, by using SI, we address one of the sources of misinterpreted p-values; the p-values are biased when the hypothesis is selected after looking at the data (often called double-dipping or data dredging). We believe our approach is a first significant step to provide valid p-values for assessing the reliability of DNN representations. Figure 1 shows an example that illustrates the importance of our method.
Related works. Several recent approaches have been developed to visualize and understand a trained DNN. Many of these post-hoc approaches (Mahendran & Vedaldi, 2015; Zeiler & Fergus, 2014; Dosovitskiy & Brox, 2016; Simonyan et al., 2013) have focused on developing visualization tools for the activation maps and/or the filter weights within trained networks. Others have aimed to identify the discriminative regions in an input image, given a trained network (Selvaraju et al., 2017; Fong & Vedaldi, 2017; Zhou et al., 2016; Lundberg & Lee, 2017). In parallel, some recent studies have showed that many popular methods for explanation and interpretation are not stable with respect to the perturbation or the adversarial attack on the input data and the model (Kindermans et al., 2017; Ghorbani et al., 2019; Melis & Jaakkola, 2018; Zhang et al., 2020; Dombrowski et al., 2019; Heo et al., 2019). However, there are no previous studies that quantitatively evaluate the stability and reproducibility of DNN representations with a rigorous statistical inference framework.
In the past few years, SI has been actively studied for inference on the features of linear models selected by several feature selection methods, e.g., Lasso (Lee et al., 2016; Liu et al., 2018; Duy & Takeuchi, 2020). The basic idea of SI is to make inference conditional on the selection event, which allows us to derive the exact (non-asymptotic) sampling distribution of the test statistic. Besides, SI has also been applied to various problems (Bachoc et al., 2014; Fithian et al., 2015; Choi et al., 2017; Tian et al., 2018; Chen & Bien, 2019; Hyun et al., 2018; Bachoc et al., 2018; Loftus & Taylor, 2014; Loftus, 2015; Panigrahi et al., 2016; Tibshirani et al., 2016; Yang et al., 2016; Suzumura et al., 2017; Duy et al., 2020). However, to the best of our knowledge, there is no existing study that provides SI for DNNs, which is technically challenging. This study is partly motivated by Tanizaki et al. (2020) where the authors provide a framework to compute p-values for image segmentation results provided by graph cut and threshold-based segmentation algorithms. As we demonstrate in this paper, our method can be also used to assess the reliability of DNN-based segmentation results.
Contribution. To our knowledge, this is the first study that provides an exact (non-asymptotic) inference method for statistically quantifying the reliability of data-driven hypotheses that are discovered from DNN representation. We propose a novel SI homotopy method, inspired by Duy & Takeuchi (2020), for conducting powerful and efficient SI for DNN representations. We conduct experiments on both synthetic and real-world datasets, through which we offer evidence that our proposed method can successfully control the false positive rate, has decent performance in terms of computational efficiency, and provides good results in practical applications. We provide our implementation in the supplementary document and it will be released when this paper is published.

heading: 2 PROBLEM STATEMENT
text: To formulate the problem, we denote an image with n pixels corrupted with Gaussian noise as X = (X1, ..., Xn)
> = µ+ ε, ε ∼ N(0,Σ), (1) where µ ∈ Rn is an unknown mean pixel intensity vector and ε ∈ Rn is a vector of Normally distributed noise with the covariance matrix Σ that is known or able to be estimated from external data. We note that we do not assume that the pixel intensities in an image follow Normal distribution in Equation (1). Instead, we only assume that the vector of noises added to the true pixel values follows a multivariate Normal distribution. For an image X and a trained DNN, the main target is to identify an attention region (discriminative/informative region) in the input image X based on a DNN representation. A pixel is assigned to the attention region if its corresponding value in the representation layer is greater than a pre-defined threshold. We denote the set of pixels ofX divided into attention region and non-attention region as C+X and C − X , respectively. Definition 1. We define A(X) as the event that the result of dividing pixels of image X into two sets of pixels C+X and C − X is obtained by applying a DNN onX , i.e.,
A(X) = {C+X , C − X}. (2)
Quantifying the statistical significance of DNN-driven hypotheses. Given an observed image xobs ∈ Rn sampled from the model (1), we can obtain C+
xobs and C− xobs by applying DNN on xobs.
Let us consider a score ∆ that represents the degree to which the attention region differs from the non-attention region. In general, we can define any score as long as it is written in the form ∆ = η>xobs. For example, we can define ∆ as the difference in average pixel values between the attention region and the non-attention region, i.e.,
∆ = mC+ xobs −mC− xobs =
1 |C+ xobs | ∑
i∈C+ xobs
xobsi − 1 |C− xobs | ∑
i∈C− xobs
xobsi = η >xobs,
where η = 1|C+ xobs |1 n C+ xobs − 1|C− xobs |1 n C− xobs , and 1nC ∈ Rn is a vector whose elements belonging to a set C are 1, and 0 otherwise. If the value of |∆| is sufficiently large, the difference between C+
xobs and C− xobs is significant and
the attention region is reliable. To quantify the statistical significance, we consider a statistical hypothesis testing with the following null hypothesis H0 and alternative hypothesis H1:
H0 : µC+ xobs = µC− xobs vs. H1 : µC+ xobs 6= µC− xobs , (3)
where µC+ xobs and µC− xobs are the true means of the pixel values in the attention region and nonattention region, respectively. Given a significance level α (e.g., 0.05), we reject H0 if the p-value is smaller than α, which indicates the attention region differs from the non-attention region. Otherwise, we cannot say that the difference is significant.
In a standard (naive) statistical test, the hypotheses in (3) are assumed to be fixed, i.e., non-random. Then, the naive (two-sided) p-value is simply given as
pnaive = PH0 ( |η>X| ≥ |∆| ) = PH0 ( |η>X| ≥ |η>xobs| ) . (4)
However, since the hypotheses in (3) are actually not fixed in advance, the naive p-value is not valid in the sense that, if we reject H0 with a significance level α, the false detection rate (type-I error) cannot be controlled at level α, which indicates that pnaive is unreliable. This is due to the fact that the hypotheses (the attention region) in (3) are selected by looking at the data (the input image), and thus selection bias exists. This selection bias is sometimes called data dredging, data snooping or p-hacking (Ioannidis, 2005; Head et al., 2015).
Selective inference (SI) for computing valid p-values. The basic idea of SI is to make inference conditional on the selection event, which allows us to derive the exact (non-asymptotic) sampling distribution of the test statistic η>X in an attempt to avoid the selection bias. Thus, we employ the following conditional p-value
pselective = PH0 ( |η>X| ≥ |η>xobs| | A(X) = A(xobs), q(X) = q(xobs) ) , (5)
where q(X) = (In − cη>)X with c = Ση(η>Ση)−1. The first condition A(X) = A(xobs) indicates the event that the result of dividing pixels into an attention region and non-attention region for a random image X is the same as that of the observed image xobs, i.e., C+X = C + xobs
and C−X = C − xobs
. The second condition q(X) = q(xobs) indicates the component that is independent of the test statistic forX is the same as the one for xobs. The q(X) corresponds to the component z in the seminal SI paper of Lee et al. (2016) (Sec 5, Eq 5.2 and Theorem 5.2). The p-value in (5), which is called selective type I error or selective p-values in the SI literature (Fithian et al., 2014), is valid in the sense that PH0(pselective < α) = α,∀α ∈ [0, 1], i.e., the false detection rate is theoretically controlled at level α indicating the selective p-value is reliable.
To calculate the selective p-value in (5), we need to identify the conditional data space. Let us define the set of x ∈ Rn that satisfies the conditions in (5) as
X = {x ∈ Rn | A(x) = A(xobs), q(x) = q(xobs)}. (6) According to the second condition, the data in X are restricted to a line (Sec 6 in Liu et al. (2018), and Fithian et al. (2014)). Therefore, the set X can be re-written, using a scalar parameter z ∈ R, as
X = {x(z) = a+ bz | z ∈ Z}, (7) where a = q(xobs), b = Ση`(η>` Ση`) −1, and
Z = { z ∈ R | A(x(z)) = A(xobs) } . (8)
Now, let us consider a random variable Z ∈ R and its observation zobs ∈ R that satisfyX = a+bZ and xobs = a+ bzobs. Then, the selective p-value in (5) is re-written as
pselective = PH0 ( |η>X| ≥ |η>xobs| |X ∈ X ) = PH0 ( |Z| ≥ |zobs| | Z ∈ Z ) . (9)
Since the variable Z ∼ N(0,η>Ση) under the null hypothesis, the law of Z | Z ∈ Z follows a truncated Normal distribution. Once the truncation region Z is identified, the selective p-value (9) can be computed as
pselective = F Z 0,η>Ση(−|z obs|) + 1− FZ0,η>Ση(|z obs|), (10)
where F Em,s2 is the c.d.f. of the truncated normal distribution with mean m, variance s 2 and truncation region E . Therefore, the most important task is to identify Z .
Extension of the problem setup to hypothesis driven from DNN-based image segmentation. We interpret the hypothesis driven from image segmentation result as the one obtained from the representation at output layer instead of internal representation. Our problem setup is general and can be directly applied to this case. For example, we can consider the attention region as the object region and the non-attention region as the background region. Then, we can conduct SI to quantify the significance of the difference between object and background regions. We note that we consider the case where the image is segmented into two regions—object and background—to simplify the problem and notations. The extension to more than two regions is straightforward.

heading: 3 PROPOSED METHOD
text: As we discussed in §2, to calculate the selective p-value, the truncation region Z in Equation (8) must be identified. To constructZ , we have to 1) computeA(x(z)) for all z ∈ R, and 2) identify the set of intervals of z on whichA(x(z)) = A(xobs). However, it seems intractable to obtainA(x(z)) for infinitely many values of z ∈ R. Our first idea to develop SI for DNN is that we additionally condition on some extra event to make the problem tractable. We now focus on a class of DNNs whose activation functions (AFs) are piecewise-linear, e.g., ReLU, Leaky ReLU (the extension to general AFs is discussed later). Then, we consider additionally conditioning on the selected piece of each piecewise-linear AF in the DNN.
Definition 2. Let sj(x) be “the selected piece” of a piecewise-linear AF at the j-th unit in a DNN for a given input image x, and let s(x) be the set of sj(x) for all the nodes in a DNN .
For example, for a ReLU activation function, sj(x) takes either 0 or 1 depending on whether the input to the j-th unit is located at the flat part (inactive) or the linear part (active) of the ReLU function. Using the notion of selected pieces s(x), instead of computing the selective p-value in (9), we consider the following over-conditioning (oc) conditional p-value
pocselective = PH0 ( |Z| ≥ |zobs| | Z ∈ Zoc ) , (11)
where Zoc = { z ∈ R | A(x(z)) = A(xobs), s(x(z)) = s(xobs) } . However, such an overconditioning in SI leads to the loss of statistical power (Lee et al., 2016).
Our second idea is to develop a homotopy method to resolve the over-conditioning problem, i.e., remove the conditioning of s(x(z)) = s(xobs). With the homotopy method, we can efficiently compute A(x(z)) in a finite number of operations without the need of considering infinitely many values of z ∈ R, which is subsequently used to obtain truncation region Z in (8). The main idea is to compute a finite number of breakpoints at which one node of the network is going to change its status from active to inactive or vice versa. This concept is similar to the regularization path of Lasso where we can compute a finite number of breakpoints at which the active set changes.
To this end, we introduce a two-step iterative approach generally described as follows (see Fig. 2):
• Step 1 (over-conditioning step). Considering over-conditioning case by additionally conditioning on the selected pieces of all the hidden nodes in the DNN.
• Step 2 (homotopy step). Combining multiple over-conditioning cases by homotopy method to obtain A(x(z)) for all z ∈ R.

heading: 3.1 STEP1: OVER-CONDITIONING STEP
text: We now show that by conditioning on the selected pieces s(xobs) of all the hidden nodes, we can write the selection event of the DNN as a set of linear inequalities.
Lemma 1. Consider a class of DNN which consists of affine operations and piecewise-linear AFs. Then, the over-conditioning region is written as
Zoc = {z ∈ R | Θ(s(x obs))x(z) ≤ ψ(s(x obs))}
for a matrix Θ(s(x obs)) and a vector ψ(s(x obs)) which depend only on the selected pieces s(xobs).
Proof. For the class of DNN, by fixing the selected pieces of all the piecewise-linear AFs, the input to each AF is represented by an affine function of an image x. Therefore, the condition for selecting
a piece in a piecewise-linear AF, sj(x(z)) = sj(xobs), is written as a linear inequality w.r.t. x(z). Similarly, the value of each unit in the representation layer is also written as an affine function of x(z). Since the attention region is selected if the value is greater than a threshold, the choice of attention region A(x(z)) = A(xobs) is characterized by a set of linear inequalities w.r.t. x(z).
Furthermore, let us consider max-operation, an operation to select the max one from a finite number of candidates. A max-operation is characterized by a set of comparison operators, i.e., inequalities. Let us consider a DNN which contains max-operators, and denote s̃(x) be the set of selected candidates of all the max-operators for an input image x.
Corollary 1. Consider a class of DNN which consists of affine operations, max-operations and piecewise-linear AFs. Then, a region Z̃oc defined as Z̃oc := {z ∈ Zoc | s̃(x(z)) = s̃(xobs)} is characterized by a set of linear inequalities w.r.t. x(z).
The proof is shown in Appendix A.1.
Remark 1. In this work, we mainly focus on the trained DNN where the activation functions used at hidden layers are piecewise linear, e.g., ReLU, Leaky ReLU, which is commonly used in CNN. Otherwise, if there is any specific demand to use non-piecewise linear functions such as sigmoid or tanh at hidden layers, we can apply some piecewise-linear approximation approach to these functions. We provided examples about the approximation for this case in Appendix A.5.
Remark 2. Most of the basic operations in a trained neural network are written as affine operations. In the traditional neural network, the multiplication results between the weight matrix and the output of the previous layer and its summation with bias vector is affine operation. In a CNN, the main convolution operation is obviously an affine operation. Upsampling operation is also affine.
Remark 3. Although the max-pooling operation is not an affine operation, it can be written as a set of linear inequalities. For instance, v1 = max{v1, v2, v3} can be written as a set {e>1 v ≤ e>2 v, e > 1 v ≤ e>3 v}, where v = (v1, v2, v3)> and ei is a standard basis vector with a 1 at position i.
Remark 4. In Remark 1, we mentioned that we need to perform piecewise linear approximation for non-piecewise linear activations. However, if these functions are used at output layer, we do not need to perform the approximation task because we can define the set of linear inequalities based on the values before doing activation. See the next example for the case of sigmoid function.
Example 1. Let us consider a 3-layer neural network with n input nodes, h hidden nodes and n ouput nodes. Let W (1) ∈ Rh×n and w(1) ∈ Rh respectively be the weight matrix and bias vector between input layer and hidden layer, and W (2) ∈ Rn×h and w(2) ∈ Rn respectively be the weight matrix and bias vector between hidden layer and output layer. The activation function at hidden layer is ReLU, and we use sigmoid function at output layer. At the hidden layer, for any node j ∈ [h], the selection event is written as{
W (1) j,: x+ w (1) j ≥ 0, if the output of ReLU function at jth node ≥ 0, W (1) j,: x+ w (1) j < 0, otherwise.
Let a(1) ∈ Rh and s(1) ∈ Rh be the vectors in which a(1)j∈[h] = 1, s (1) j∈[h] = 1 if the output of ReLU function at the jth node≥ 0, and a(1)j = 0, s (1) j = −1 otherwise. Then we have the linear inequality system Θ1x ≤ ψ1 where Θ1 = (−s(1)1 W (1) 1,: , ...,−s (1) h W (1) h,: ) > and ψ1 = (s (1) 1 w (1) 1 , ..., s (1) h w (1) h ) >. Next, for any output node o ∈ [n], the selection event—a linear inequality—is written as{ W (2) o,: ((W (1)x+w(1)) ◦ a(1)) + w(2)o ≥ 0, if the output of sigmoid function at oth node ≥ 0.5,
W (2) o,: ((W (1)x+w(1)) ◦ a(1)) + w(2)o < 0, otherwise,
where ◦ is the element-wise product. Similar to the hidden layer, we can also construct the linear inequality system Θ2x ≤ ψ2 at the output layer. Finally, the whole linear inequality system is written as
Θx ≤ ψ = (Θ1 Θ2)> x ≤ (ψ1 ψ2)> . (12)
Algorithm 1 compute solution path Input: a, b, [zmin, zmax] 1: Initialization: t = 1, zt = zmin, T = zt 2: while zt < zmax do 3: Obtain A(x(zt)) by applying a trained DNN to x(zt) = a+ bzt 4: Compute the next breakpoint zt+1← Equation (13). Then assign T = T ∪ {zt+1}, and t = t+ 1 5: end while
Output: {A(x(zt)}zt∈T

heading: 3.2 STEP 2: HOMOTOPY STEP
text: We now introduce a homotopy method to compute A(x(z)) based on over-conditioning step. Lemma 2. Consider a real value zt. By applying a trained DNN to x(zt), we obtain a set of linear inequalities Θ(s(x(zt)))x(zt) ≤ ψ(s(x(zt))). Then, the next breakpoint zt+1 > zt at which the status of one node is going to be changed from active to inactive or vice versa, i.e., the sign of one linear inequality is going to be changed, is calculated by
zt+1 = min k:(Θ(s(x(zt)))b)k>0
ψ (s(x(zt))) k − (Θ(s(x(zt)))a)k
(Θ(s(x(zt)))b)k . (13)
The proof is shown in Appendix A.2. Algorithm 1 shows our solution to efficiently identify A(x(z)). In this algorithm, multiple breakpoints z1 < z2 < ... < z|T | are computed one by one. Each breakpoint zt, t ∈ [|T |], indicates a point at which the sign of one linear inequality is changed, i.e., the status of one node in the network is going to change from active to inactive or vice versa. By identifying all these breakpoints {zt}t∈[|T |], the solution path is given by A(x(z)) = A(x(zt)) if z ∈ [zt, zt+1], t ∈ [|T |]. For the choice of [zmin, zmax], see Appendix A.3.

heading: 4 EXPERIMENT
text: We highlight the main results. Several additional results and details can be found in Appendix A.6.
Numerical Experiments. We demonstrate the performances of two versions of the proposed method: proposed-method (homotopy) and proposed-method-oc. The p-values in these two versions were computed by (5) and (11), respectively. Besides, we also compared the proposed methods with the naive p-value in (4) and the permutation test. The details of permutation test procedure is described in Appendix A.6. To test the FPR control, we generated 120 null images x = (x1, ..., xn) in which xi ∈ [n] ∼ N(0, 1) for each n ∈ {64, 256, 1024, 4096}. To test the power, we generated images x = (x1, ..., xn) with n = 256 for each true average difference in the underlying model µC+x − µC−x = ∆µ ∈ {0.5, 1.0, 1.5, 2.0}. For each case, we ran 120 trials. We chose the significance level α = 0.05. For more information about the setup as well as the the structure of a neural network, see the experimental setup paragraph in Appendix A.6. The results of FPR control are shown in the first part of Fig. 3. The proposed methods could successfully control the FPR under α = 0.05 while the naive method can not. Since the naive method fails to control FPR, we did not consider the power anymore. In the second part of Fig. 3, we see that the over-conditioning option has lower power than the homotopy method. It is because the truncation region in proposed-methodoc is shorter than the one in proposed-method (homotopy), which is demonstrated in the third part of Fig. 3. The last part of Fig. 3 shows the reason why the proposed homotopy method is efficient. With the homotopy method, we only need to consider the number of encountered intervals on the line along the direction of test statistic which is almost linearly increasing in practice.
Real-data examples. We performed comparison on real-world brain image dataset, which includes 939 images with tumor and 941 images without tumor. We first compared our method with permutation test in terms of FPR control. The results are shown in Table 1. Since the permutation test could not control the FPR properly, we did not compare the power. The comparisons between naive p-value and selective p-value are shown in Figs. 4, 5, 6 and 7. The naive p-value was still small
even when the image has no tumor region, which indicates that the naive p-values cannot be used for quantifying the reliability of DNN-driven hypotheses. The proposed method could successfully identify false positive detections as well as true positive detections.

heading: 5 CONCLUSION
text: We proposed a novel method to conduct statistical inference on the significance of the data-driven hypotheses driven from neural network representation based on the concept of selective inference. In the context of explainable DNN or interpretable DNN, we are primarily interested in the reliability of the trained network when given new inputs (not training inputs). Therefore, the validity of our proposed method does not depend on how the DNN is trained.
In regard of the generality, the proposed method can be applied to any kind of network as long as the network operation is characterized by a set of linear inequalities (or approximated by piecewiselinear functions) because all the algorithms and theories in §2 and §3 only depend on the property of each component and does not depend on the entire structure of the network.
We believe that this paper provides a significant step toward reliable artificial intelligence (AI) and open several directions for statistically evaluating the reliability of DNN representation-driven hypotheses. Although it is not necessary to account the impact of training in this paper because the validity of our proposed method does not depend on how the DNN is trained, defining a new problem setup and providing solution for the case in which the training process needs to be considered is a potential direction. Moreover, widening the practical applicability of the proposed method in other fields such as NLP and signal processing would also represent a valuable contribution.

Review summaries:
This paper proposed a novel method which to quantify the reliability of DNN-driven hypotheses in a statistical hypothesis testing framework. Naive statistical testings are not appropriate for the DNN-driven hypotheses, where the hypotheses are selected by looking at the data(i.e. The selection bias exists). To address this problem, the authors developed a novel homotopy method under the Selective-Inference(SI) framework, which can derive the exact sampling distribution of the DNN-driven hypotheses. In this paper,  the authors mainly focus on DNNs which consist of affine operations, max-operations, and piecewise-linear activation. As described by Lee et al. (2016), the main idea of SI is to make the inference conditional on the selection event. Specifically to the DNN-driven hypotheses, the authors proposed a novel method that consists of two steps, 1) Adding extra conditioning to make the problem traceable. 2) Combining multiple over-conditioning cases by homotopy method to solve the over-conditioning problem. The experimental results on both synthetic and real-world datasets illustrate the proposed method can successfully control the FP error rate.