Original: Quantifying Statistical Signiﬁcance of Neural Network-based Image Segmentation by Selective Inference
Revised: Quantifying Statistical Signiﬁcance of Neural Network-based Image Segmentation by Selective Inference

Original: Abstract
Revised: Abstract

Original: Although a vast body of literature relates to image segmentation methods thatuse deep neural networks (DNNs), less attention has been paid to assessing thestatistical reliability of segmentation results.
Revised: Although a vast body of literature relates to image segmentation methods that use deep neural networks (DNNs), less attention has been paid to assessing the statistical reliability of segmentation results.

Original: In this study, we interpret the segmentation results as hypotheses driven by DNN (called DNN-driven hypotheses) andpropose a method to quantify the reliability of these hypotheses within a statisticalhypothesis testing framework.
Revised: In this study, we interpret the segmentation results as hypotheses driven by DNN (called DNN-driven hypotheses) and propose a method to quantify the reliability of these hypotheses within a statistical hypothesis testing framework.

Original: To this end, we introduce a conditional selectiveinference (SI) framework—a new statistical inference framework for data-drivenhypotheses that has recently received considerable attention—to compute exact(non-asymptotic) valid p -values for the segmentation results.
Revised: To this end, we introduce a conditional selective inference (SI) framework—a new statistical inference framework for data-driven hypotheses that has recently received considerable attention—to compute exact (non-asymptotic) valid p -values for the segmentation results.

Original: To use the conditional
Revised: To use the conditional

Original: SI framework for DNN-based segmentation, we develop a new SI algorithm basedon the homotopy method, which enables us to derive the exact (non-asymptotic)sampling distribution of DNN-driven hypothesis.
Revised: SI framework for DNN-based segmentation, we develop a new SI algorithm based on the homotopy method, which enables us to derive the exact (non-asymptotic) sampling distribution of DNN-driven hypothesis.

Original: We conduct several experimentsto demonstrate the performance of the proposed method.
Revised: We conduct several experiments to demonstrate the performance of the proposed method.

Original:  Introduction
Revised:  Introduction

Original: Image segmentation is a fundamental task in computer vision, and it has been widely applied inmany areas.
Revised: Image segmentation is a fundamental task in computer vision, and it has been widely applied in many areas.

Original: The goal of image segmentation is to assign a label to every pixel in an observed image,such that pixels with the same label share certain characteristics.
Revised: The goal of image segmentation is to assign a label to every pixel in an observed image, such that pixels with the same label share certain characteristics.

Original: In the literature, numerous imagesegmentation algorithms have been developed based on thresholding [19], region-growing [18] orgraph cuts [2, 3].
Revised: In the literature, numerous image segmentation algorithms have been developed based on thresholding [18], region-growing [17] or graph cuts [2, 3].

Original: However, over the past few years, image segmentation using deep neural networks(DNNs) has become a popular model that exhibits remarkable performance improvements [17, 1, 20].
Revised: However, over the past few years, image segmentation using deep neural networks (DNNs) has become a popular model that exhibits remarkable performance improvements [16, 1, 19].

Original: Although a vast body of literature relates to DNN-based segmentation methods, less attention has beenpaid to evaluating the statistical reliability of the segmentation results.
Revised: Although a vast body of literature relates to DNN-based segmentation methods, less attention has been paid to evaluating the statistical reliability of the segmentation results.

Original: In the absence of statisticalreliability, it is difﬁcult to manage the risk of obtaining incorrect segmentation results, which might beharmful when they are used in high-stakes decision-making, such as medical diagnoses or automaticdriving.
Revised: In the absence of statistical reliability, it is difﬁcult to manage the risk of obtaining incorrect segmentation results, which might be harmful when they are used in high-stakes decision-making, such as medical diagnoses or automatic driving.

Original: Therefore, it is necessary to develop a valid statistical inference method for data-driven
Revised: Therefore, it is necessary to develop a valid statistical inference method for data-driven

Original: DNN-based segmentation results that can properly control the risk of obtaining false positives.
Revised: DNN-based segmentation results that can properly control the risk of obtaining false positives.

Original: Valid statistical inference on the DNN-based segmentation results is intrinsically difﬁcult because theobserved data is used twice: once for segmentation and once again for inference.
Revised: Valid statistical inference on the DNN-based segmentation results is intrinsically difﬁcult because the observed data is used twice: once for segmentation and once again for inference.

Original: This is often referredto as double dipping [13].
Revised: This is often referred to as double dipping [12].

Original: In statistics, it has been recognized that naively computing p -values indouble dipping is highly biased and correcting this bias is challenging, especially when the hypothesesare selected using complicated procedures such as DNN.
Revised: In statistics, it has been recognized that naively computing p -values in double dipping is highly biased and correcting this bias is challenging, especially when the hypotheses are selected using complicated procedures such as DNN.

Original: Our idea is to introduce the conditional
Revised: Our idea is to introduce the conditional

Original: Selective Inference (SI) framework for resolving this challenge.
Revised: Selective Inference (SI) framework for resolving this challenge.

Original: The basic idea behind conditional SIis to perform statistical inference conditional on the selection event (i.e., segmentation results), whichallows us to derive the exact (non-asymptotic) sampling distribution of the test statistic for makingthe valid statistical inference.
Revised: The basic idea behind conditional SI is to perform statistical inference conditional on the selection event (i.e., segmentation results), which allows us to derive the exact (non-asymptotic) sampling distribution of the test statistic for making the valid statistical inference.

Original: Related works.
Revised: Related works.

Original: In the past few years, conditional SI has been recognized as a new promisingapproach by which to evaluate the statistical reliability of data-driven hypotheses, and it has beenactively studied for inference on the features of linear models selected by several feature selectionmethods—for example, Lasso [15, 16, 14].
Revised: In the past few years, conditional SI has been recognized as a new promising approach by which to evaluate the statistical reliability of data-driven hypotheses, and it has been actively studied for inference on the features of linear models selected by several feature selection methods—for example, Lasso [14, 15, 13].

Original: The basic idea of SI is to make inferences conditional onthe selection event, which allows us to derive the exact sampling distribution of the test statistic.
Revised: The basic idea of SI is to make inferences conditional on the selection event, which allows us to derive the exact sampling distribution of the test statistic.

Original: Inaddition, SI has been applied to various problems such as change point detection [12, 9, 22], outlierdetection [5, 26], generalized Lasso [11, 8], sequential feature selection [25, 21], and interactionmodel [23, 6].
Revised: In addition, SI has been applied to various problems such as change point detection [11, 8, 21], outlier detection [5, 25], generalized Lasso [10, 7], sequential feature selection [24, 20], and interaction model [22, 6].

Original: However, to our knowledge, no study to date provides conditional SI for DNNs.
Revised: However, to our knowledge, no study to date provides conditional SI for DNNs.

Original: The most closely related work (and the motivation for this study) is [24], where the authors provide aframework to compute valid p -values for image segmentation results obtained by threshold (TH)-based and graph cut (GC)-based segmentation algorithms.
Revised: The most closely related work (and the motivation for this study) is [23], where the authors provide a framework to compute valid p -values for image segmentation results obtained by threshold (TH)- based and graph cut (GC)-based segmentation algorithms.

Original: The novel idea of [24] is to consider thesampling distribution of the test statistic conditional on the selection event, in which the segmentationresult is obtained.
Revised: The novel idea of [23] is to consider the sampling distribution of the test statistic conditional on the selection event, in which the segmentation result is obtained.

Original: In that study, since the authors consider simple segmentation algorithms, theselection event is simply characterized by a set of linear or quadratic inequalities which is representedas a single polytope in the data space.
Revised: In that study, since the authors consider simple segmentation algorithms, the selection event is simply characterized by a set of linear or quadratic inequalities which is represented as a single polytope in the data space.

Original: Then, they can directly use the approach in the seminalconditional SI paper [15] to conduct the inference.
Revised: Then, they can directly use the approach in the seminal conditional SI paper [14] to conduct the inference.

Original: However, it is not the case of DNN because theselection event of the DNN is much more complicated than that of TH and GC.
Revised: However, it is not the case of DNN because the selection event of the DNN is much more complicated than that of TH and GC.

Original: Therefore, the methodin [24] is not applicable in the case of DNN-based segmentation.
Revised: Therefore, the method in [23] is not applicable in the case of DNN-based segmentation.

Original: There are several types of problem setting with various network structures for DNN-based segmentation.
Revised: There are several types of problem setting with various network structures for DNN-based segmentation.

Original: In this study, we focus on the most standard one, the supervised segmentation setting with asimple structure (see §3 and §4).
Revised: In this study, we focus on the most standard one, the supervised segmentation setting with a simple structure (see §3 and §4).

Original: As will be described in §2, our target is to conduct the statisticalinference in the test phase (not in the training phase).
Revised: As will be described in §2, our target is to conduct the statistical inference in the test phase (not in the training phase).

Original: Namely, our goal is to quantify the reliabilityof the segmentation result when a new test image is given to a DNN that has been trained in advance.
Revised: Namely, our goal is to quantify the reliability of the segmentation result when a new test image is given to a DNN that has been trained in advance.

Original: Contributions.
Revised: Contributions.

Original: Our main contributions are as follows:
Revised: Our main contributions are as follows:

Original: • To the best of our knowledge, this is the ﬁrst study to provide an exact (non-asymptotic) inference method for statistically quantifying the reliability of image segmentation results obtained from DNNs.
Revised: • To the best of our knowledge, this is the ﬁrst study to provide an exact (non-asymptotic) inference method for statistically quantifying the reliability of image segmentation results obtained from DNNs.

Original: • We propose a homotopy method to conduct a powerful and efﬁcient conditional SI for DNN-basedsegmentation tasks.
Revised: • We propose a homotopy method to conduct a powerful and efﬁcient conditional SI for DNN-based segmentation tasks.

Original: In this study, we mainly focus on a standard convolutional neural network (CNN)as a working example and show that all basic operations in the standard CNN (e.g., convolution, maxpooling) can be incorporated in our proposed homotopy-based conditional SI method.• We undertake experiments on both synthetic and real-world datasets, through which we offer
Revised: In this study, we mainly focus on a standard convolutional neural network (CNN) as a working example and show that all basic operations in the standard CNN (e g., convolution, max pooling) can be incorporated in our proposed homotopy-based conditional SI method.

Original:  evidence that our proposed method can successfully control the False Positive Rate (FPR), has good 
Revised: • We undertake experiments on both synthetic and real-world datasets, through which we offer evidence that our proposed method can successfully control the False Positive Rate (FPR), has good performance in terms of computational efﬁciency, and provides good results in practical applications. Our implementation is available at https://github.com/vonguyenleduy/dnn

Original: performance in terms of computational efﬁciency, and provides good results in practical applications.
Revised: [Equation] segmentation [Equation] selective_inference

Original:  Preliminary
Revised:  Preliminary

Original: In this section, we ﬁrst review the conditional SI method for TH-based and GC-based segmentationalgorithms in [24].
Revised: In this section, we ﬁrst review the conditional SI method for TH-based and GC-based segmentation algorithms in [23].

Original: Then, we clarify the challenges of introducing the conditional SI for DNN-basedsegmentation tasks and present our idea to resolve the difﬁculties.
Revised: Then, we clarify the challenges of introducing the conditional SI for DNN-based segmentation tasks and present our idea to resolve the difﬁculties.

Original: 2.1 Conditional SI for TH-based and GC-based Image Segmentation [24]
Revised: 2.1 Conditional SI for TH-based and GC-based Image Segmentation [23]

Original: Consider an image with n pixels corrupted by Gaussian noise as
Revised: Consider an image with n pixels corrupted by Gaussian noise as

Original: [Equation]
Revised: [Equation]

Original: [Equation] where µ 2 R n is an unknown mean pixel intensity vector, and " 2 R n is a vector of normallydistributed noise with covariance matrix ⌃ , which is known or can be estimated from external data.
Revised: where µ 2 R n is an unknown mean pixel intensity vector, and " 2 R n is a vector of normally distributed noise with covariance matrix ⌃ , which is known or can be estimated from external data.

Original: We note that the assumption in Equation (1) does not mean that the pixel intensities in an imagefollow a multivariate normal distribution.
Revised: We note that the assumption in Equation (1) does not mean that the pixel intensities in an image follow a multivariate normal distribution.

Original: Instead, the vector of noise added to the true pixel valuesare assumed to follows a multivariate normal distribution.
Revised: Instead, the vector of noise added to the true pixel values are assumed to follows a multivariate normal distribution.

Original: For simplicity, we consider segmentationproblems in which the image is divided into two regions: object and background regions 1 .
Revised: For simplicity, we consider segmentation problems in which the image is divided into two regions: object and background regions 2 .

Original: We denotethe sets of pixels in the object and background regions as O X and B X , respectively.
Revised: We denote the sets of pixels in the object and background regions as O X and B X , respectively.

Original: Statistical inference.
Revised: Statistical inference.

Original: To quantify the statistical signiﬁcance of the segmentation result, Tanizaki etal.
Revised: To quantify the statistical signiﬁcance of the segmentation result, Tanizaki et al.

Original: [24] consider the following statistical test:
Revised: [23] consider the following statistical test:

Original: [Equation]
Revised: [Equation]

Original: where µ O X and µ B X are the true means of the pixel values in the object and background regions.
Revised: where µ O X and µ B X are the true means of the pixel values in the object and background regions.

Original: Areasonable choice of the test statistic is the difference in the average pixel values between the objectand background regions
Revised: A reasonable choice of the test statistic is the difference in the average pixel values between the object and background regions

Original: [Equation]
Revised: [Equation]

Original: where ⌘ =

Original: [Equation]

Original:  O [Equation]  1 |B X | 1 n B X is the vector indicating the test-statistic direction, and 1 n C 2 R n isa vector whose elements belonging to a set C are set to 1, and
Revised: where ⌘ = |O X | 1 n O X  1 |B X | 1 n B X is the vector indicating the test-statistic direction, and 1 n C 2 R n is a vector whose elements belonging to a set C are set to 1, and

Original: 0 otherwise.
Revised: 0 otherwise.

Original: We would like to note thatthe hypotheses in (2) and the test-statistic direction ⌘ in (3) are data-driven because they depend on
Revised: We would like to note that the hypotheses in (2) and the test-statistic direction ⌘ in (3) are data-driven because they depend on

Original: X .
Revised: X .

Original: Given a signiﬁcance level ↵ 2 [0 , 1] (e.g., 0.05), we reject the null hypothesis H 

Revised: Given a signiﬁcance level ↵ 2 [0 , 1] (e.g., 0.05), we reject the null hypothesis H

Original: if the p -value issmaller than ↵ , which indicates that the object region differs from the background region.
Revised: if the p -value is smaller than ↵ , which indicates that the object region differs from the background region.

Original: Otherwise,we cannot say that a difference is signiﬁcant.
Revised: Otherwise, we cannot say that a difference is signiﬁcant.

Original: Conditional SI for computing valid p -values.
Revised: Conditional SI for computing valid p -values.

Original: Let us deﬁne A ( X ) as the event in which theresult of dividing the pixels of image X into the object region O X and the background region B X isobtained by applying a segmentation
Revised: Let us deﬁne A ( X ) as the event in which the result of dividing the pixels of image X into the object region O X and the background region B X is obtained by applying a segmentation

Original: algorithm A (i.e., TH-based or GC-based image segmentationalgorithm) on X —that is,
Revised: algorithm A (i.e., TH-based or GC-based image segmentation algorithm) on X —that is,

Original: [Equation]
Revised: [Equation]

Original: To conduct a valid inference based on the concept of conditional SI, [24] proposes to consider thesampling distribution of the test statistic ⌘ > X conditional on the segmentation results, that is,
Revised: To conduct a valid inference based on the concept of conditional SI, [23] proposes to consider the sampling distribution of the test statistic ⌘ > X conditional on the segmentation results, that is,

Original: [Equation]
Revised: [Equation]

Original: where x obs is an observation (realization) of the random image X .
Revised: where x obs is an observation (realization) of the random image X .

Original: Then, to test the statisticalsigniﬁcance of the segmentation result, the authors introduced the selective p -value , which satisﬁesthe following sampling property:
Revised: Then, to test the statistical signiﬁcance of the segmentation result, the authors introduced the selective p -value , which satisﬁes the following sampling property:

Original: [Equation]
Revised: [Equation]

Original: i.e., p selective follows a uniform distribution under the null hypothesis.
Revised: i.e., p selective follows a uniform distribution under the null hypothesis.

Original: The p selective is deﬁned as
Revised: The p selective is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where q ( X ) is the sufﬁcient statistic of the nuisance parameter that needs to be conditioned on inorder to tractably conduct the inference and it is deﬁned as q ( X )
Revised: where q ( X ) is the sufﬁcient statistic of the nuisance parameter that needs to be conditioned on in order to tractably conduct the inference and it is deﬁned as q ( X )

Original: =
Revised: =

Original: ( I n  c ⌘ > ) X with c =⌃ ⌘ ( ⌘ > ⌃ ⌘ )  1 .
Revised: ( I n  c ⌘ > ) X with c = ⌃ ⌘ ( ⌘ > ⌃ ⌘ )  1 .

Original: Here, we note that the selective p -value depends on q ( X ) , but the sampling propertyin (6) continues to be satisﬁed without this additional condition because we can marginalize over allvalues of q ( X ) .
Revised: Here, we note that the selective p -value depends on q ( X ) , but the sampling property in (6) continues to be satisﬁed without this additional condition because we can marginalize over all values of q ( X ) .

Original: The q ( X ) corresponds to the component z in the seminal conditional SI paper [15](see Sec. 5, Eq.
Revised: The q ( X ) corresponds to the component z in the seminal conditional SI paper [14] (see Sec. 5, Eq.

Original: 5.2 and Theorem 5.2).
Revised: 5.2 and Theorem 5.2).

Original: We note that additional conditioning on q ( X ) is a standardapproach in the conditional SI literature and is used in almost all the conditional SI-related studies.
Revised: We note that additional conditioning on q ( X ) is a standard approach in the conditional SI literature and is used in almost all the conditional SI-related studies.

Original: Example 1.
Revised: Example 1.

Original: (Selection event of TH-based segmentation and the conditional inference)
Revised: (Selection event of TH-based segmentation and the conditional inference)

Original: Given athreshold ⌧ and an observed data x obs , the selection event of TH-based segmentation is written as
Revised: Given a threshold ⌧ and an observed data x obs , the selection event of TH-based segmentation is written as

Original: [Equation]
Revised: [Equation]

Original: which is simply a single polytope in the data space.
Revised: which is simply a single polytope in the data space.

Original: Then, the conditional inference in (5) for thecase of TH-based segmentation is explicitly deﬁned as
Revised: Then, the conditional inference in (5) for the case of TH-based segmentation is explicitly deﬁned as

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Revised: in the background (Class 1) and object (Class 2), as well as the naive p -value and the proposed selective p -value.

Revised: Even from an image containing no object, the naive p -value is very small, indicating that it cannot be used to evaluate the reliability of the segmentation result.

Revised: By applying the proposed method, we can successfully identify the false segmentation result.

Original: Similarly, [24] showed that the selection event of GC-based segmentation can be approximated bya set of quadratic inequalities which is also represented by a polytope.
Revised: Similarly, [23] showed that the selection event of GC-based segmentation can be approximated by a set of quadratic inequalities which is also represented by a polytope.

Original: When the selection event isrepresented by a single polytope, we can directly apply the original conditional SI method by [15]. [Figure] [Figure]
Revised: When the selection event is represented by a single polytope, we can directly apply the original conditional SI method by [14].

Original: 2.2 Challenges of Introducing Conditional SI for DNN-based Image Segmentation
Revised: 2.2 Challenges of Introducing Conditional SI for DNN-based Image Segmentation

Original: In this paper, we consider the same problem setup as described in [24] (i.e., from (2) to (7)).
Revised: In this paper, we consider the same problem setup as described in [23] (i.e., from (2) to (7)).

Original: The maindifference is that the segmentation event A ( X ) =
Revised: The main difference is that the segmentation event A ( X ) =

Original: {O X , B X } in (4) is now obtained from a DNN119 that has already been trained in advance for the segmentation task.
Revised: {O X , B X } in (4) is now obtained from a DNN that has already been trained in advance for the segmentation task.

Original: The discussion thus far indicatesthat we can conduct an exact statistical inference for the segmentation result if we can computethe selective p -value in (7).
Revised: The discussion thus far indicates that we can conduct an exact statistical inference for the segmentation result if we can compute the selective p -value in (7).

Original: However, to compute p selective , the major challenge is to characterizeselection event {A ( X ) =
Revised: However, to compute p selective , the major challenge is to characterize selection event {A ( X ) =

Original: A ( x obs ) } of DNN-based segmentation, which is much more complicatedand cannot be simply represented by an intersection of linear or quadratic inequalities as in the caseof TH-based and GC-based segmentations.
Revised: A ( x obs ) } of DNN-based segmentation, which is much more complicated and cannot be simply represented by an intersection of linear or quadratic inequalities as in the case of TH-based and GC-based segmentations.

Original: In the next section, we propose a novel method thataddresses all the aforementioned challenges by utilizing the concept of a parametrized line search(i.e. , the homotopy method).
Revised: In the next section, we propose a novel method that addresses all the aforementioned challenges by utilizing the concept of a parametrized line search (i.e., the homotopy method).

Original: Figure 1 illustrates the selection bias in naive statistical inference andthe importance of the proposed method.
Revised: Figure 1 illustrates the selection bias in naive statistical inference and the importance of the proposed method.

Original:  Proposed Method
Revised:  Proposed Method

Original: In this section, we propose a method for computing the selective p -values in (7).
Revised: In this section, we propose a method for computing the selective p -values in (7).

Original: The main task is toidentify the following set of x 2 R n that satisﬁes the condition part:
Revised: The main task is to identify the following set of x 2 R n that satisﬁes the condition part:

Original: [Equation]
Revised: [Equation]

Original: To identify X , we ﬁrst reformulate the problem of identifying this data space as a problem of searchingthe data on a parametrized line.
Revised: To identify X , we ﬁrst reformulate the problem of identifying this data space as a problem of searching the data on a parametrized line.

Original: Then, we introduce a homotopy method to conduct a parametrizedline search illustrated in Fig.
Revised: Then, we introduce a homotopy method to conduct a parametrized line search illustrated in Fig.

Original: 3.1 Characterization of the Conditional Data 
Revised: 3.1 Characterization of the Conditional Data Space X

Original: Space X

Original: In (8), according to the second condition on the sufﬁcient statistic of the nuisance parameter q ( x ) ,the data in X is restricted to a line in R n , as stated in the following lemma.
Revised: In (8), according to the second condition on the sufﬁcient statistic of the nuisance parameter q ( x ) , the data in X is restricted to a line in R n , as stated in the following lemma.

Original: Lemma 1.
Revised: Lemma 1.

Original: Let us deﬁne
Revised: Let us deﬁne

Original: [Equation]

Original: and b

Original: [Equation]

Original: ⌃

Original: [Equation]
Revised: [Equation]

Original: Then, the conditional data space X can be rewritten using a scalar parameter z 2 R as[Equation] [Equation] =
Revised: Then, the conditional data space X can be rewritten using a scalar parameter z 2 R as

Original: [Equation]

Original: + b z | z 2 Z} , where Z =

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The proof is deferred to Appendix A.
Revised: The proof is deferred to Appendix A.

Original: ⌅
Revised: ⌅

Revised: Data Space  n x obs

Revised: Observed Segmentation

Revised: Result

Original: We note that whereas the seminal conditional SI work [15] already implicitly considers the data onthe line, this was explicitly discussed for the ﬁrst time in Section 6 of [16].
Revised: DNN  ( x obs ) Proposed Method  ( z 1 ) x   ( z 1 )  ( z 2 ) x   ( z 2 )  ( z 3 ) x   ( z 3 )  ( z 4 ) x

Revised:   ( z 4 )  ( z 5 ) x

Revised:   ( z 5 )

Revised: Parametrized Line x ( z ) =

Revised: a + b z

Revised: Conditional Data Space

Revised: [Equation]

Revised: We note that whereas the seminal conditional SI work [14] already implicitly considers the data on the line, this was explicitly discussed for the ﬁrst time in Section 6 of [15].

Original: Lemma 1 indicates that weneed not consider the n -dimensional data space.
Revised: Lemma 1 indicates that we need not consider the n -dimensional data space.

Original: Instead, we need only consider the one-dimensionalprojected data space Z in (10).
Revised: Instead, we need only consider the one-dimensional projected data space Z in (10).

Original: Now, let us consider a random variable Z 2 R and its observation144 z obs 2 R that satisﬁes
Revised: Now, let us consider a random variable Z 2 R and its observation z obs 2 R that satisﬁes

Original: X = a + b Z and x obs = a + b z obs .
Revised: X = a + b Z and x obs = a + b z obs .

Original: The selective p -value (7) is rewritten asp selective = P H
Revised: The selective p -value (7) is rewritten as

Original: [Equation]
Revised: [Equation]

Original: Because the variable Z ⇠ N (0 , ⌘ > ⌃ ⌘ ) under the null hypothesis, Z | Z 2 Z follows a truncated146 normal distribution.
Revised: Because the variable Z ⇠ N (0 , ⌘ > ⌃ ⌘ ) under the null hypothesis, Z | Z 2 Z follows a truncated normal distribution.

Original: Once the truncation region
Revised: Once the truncation region

Original: Z is identiﬁed, computation of the selective p -valuein (11) is straightforward.
Revised: Z is identiﬁed, computation of the selective p -value in (11) is straightforward.

Original: Therefore, the remaining task is to identify Z .
Revised: Therefore, the remaining task is to identify Z .

Original: 3.2 Identiﬁcation of Truncation Region Z
Revised: 3.2 Identiﬁcation of Truncation Region Z

Original: As discussed in §3.1, to calculate the selective p -value (11), we must identify the truncation region
Revised: As discussed in §3.1, to calculate the selective p -value (11), we must identify the truncation region

Original: Z in (10).
Revised: Z in (10).

Original: To construct Z , we must (a) compute A ( x ( z )) for all z 2 R , and (b) identify the set ofintervals of z on which A ( x ( z ))
Revised: To construct Z , we must (a) compute A ( x ( z )) for all z 2 R , and (b) identify the set of intervals of z on which A ( x ( z ))

Original: =
Revised: =

Original: A ( x obs ) .
Revised: A ( x obs ) .

Original: However, it seems intractable to obtain A ( x ( z )) forinﬁnitely many values of z 2 R .
Revised: However, it seems intractable to obtain A ( x ( z )) for inﬁnitely many values of z 2 R .

Original: Our ﬁrst idea of introducing conditional SI for DNN is that we additionally condition on someextra events to make the problem tractable.
Revised: Our ﬁrst idea of introducing conditional SI for DNN is that we additionally condition on some extra events to make the problem tractable.

Original: We now focus on a class of DNNs whose activationfunctions (AFs) are piecewise-linear—for example, ReLU, Leaky ReLU.
Revised: We now focus on a class of DNNs whose activation functions (AFs) are piecewise-linear—for example, ReLU, Leaky ReLU.

Original: Then, we consider additionalconditioning on the selected piece of each piecewise-linear AF.
Revised: Then, we consider additional conditioning on the selected piece of each piecewise-linear AF.

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: Let s j ( x ) be “the selected piece” of a piecewise-linear AF at the j -th unit in a DNNfor a given input image x , and let s ( x ) be the set of s j ( x ) for all the nodes in a DNN.
Revised: Let s j ( x ) be “the selected piece” of a piecewise-linear AF at the j -th unit in a DNN for a given input image x , and let s ( x ) be the set of s j ( x ) for all the nodes in a DNN.

Original: For example, given a ReLU AF, s j ( x ) takes either 0 or 1, depending on whether the input to the j -thunit is located at the ﬂat part (inactive) or the linear part (active) of the ReLU function.
Revised: For example, given a ReLU AF, s j ( x ) takes either 0 or 1, depending on whether the input to the j -th unit is located at the ﬂat part (inactive) or the linear part (active) of the ReLU function.

Original: Using thenotion of selected pieces s ( x ) , instead of computing the selective p -value in (11), we consider thefollowing over-conditioning (oc) conditional p -value:
Revised: Using the notion of selected pieces s ( x ) , instead of computing the selective p -value in (11), we consider the following over-conditioning (oc) conditional p -value:

Original: [Equation]
Revised: [Equation]

Original:  whereZ oc =  z 2 R | A ( x ( z )) = A ( x obs ) , s ( x ( z ))
Revised: 12) where

Original: = s ( x obs ) .
Revised: [Equation]

Original: However, such an overconditioning in SI leads to a loss of statistical power [15, 10].
Revised: However, such an over-conditioning in SI leads to a loss of statistical power [14, 9].

Original: Our second idea is to develop a homotopy method to resolve the over-conditioning problem—thatis, remove the conditioning of s ( x ( z ))
Revised: Our second idea is to develop a homotopy method to resolve the over-conditioning problem—that is, remove the conditioning of s ( x ( z ))

Original: =
Revised: =

Original: s ( x obs ) .
Revised: s ( x obs ) .

Original: Using the homotopy method, we can efﬁcientlycompute A ( x ( z )) in a ﬁnite number of operations without the need to consider inﬁnitely many valuesof z 2 R , which is subsequently used to obtain the truncation region Z in (10).
Revised: Using the homotopy method, we can efﬁciently compute A ( x ( z )) in a ﬁnite number of operations without the need to consider inﬁnitely many values of z 2 R , which is subsequently used to obtain the truncation region Z in (10).

Original: The main idea istocompute a ﬁnite number of breakpoints at which one node of the network changes its status fromactive toinactive or vice versa.
Revised: The main idea is to

Original: This concept is similar to the regularization path of Lasso, where wecan compute a ﬁnite number of breakpoints at which the active set changes. To this end, we introduce a two-step iterative approach, generally described as follows (see Fig. 2): • Step 1 (over-conditioning step). Considering the over-conditioning case by additionally condi-
Revised: [Figure]

Original: tioning on the selected pieces of all hidden nodes in the DNN. • Step 2 (homotopy step). Combining multiple over-conditioning cases using the homotopy method to obtain A ( x ( z )) for all z 2 R .
Revised: conv

Original: [Figure]
Revised: [Figure]

Original: < 0 or > > threshold or 
Revised: < 0 or > > threshold or <

Original: [Figure]

Original: threshold
Revised: threshold

Original: [Figure]
Revised: [Figure]

Original: Input Image !
Revised: Input Image !

Original: × ! × !
Revised: × ! × !

Original: conv maxpool ! "
Revised: conv maxpool ! "

Original: × ! " !
Revised: × ! " !

Original: × !
Revised: × !

Original: upsample conv
Revised: upsample conv

Original: Output SegmentationResult !
Revised: Output SegmentationResult !

Original: × !
Revised: × !

Original: [Figure]
Revised: [Figure]

Revised: compute a ﬁnite number of breakpoints at which one node of the network changes its status from active to inactive or vice versa.

Revised: This concept is similar to the regularization path of Lasso, where we can compute a ﬁnite number of breakpoints at which the active set changes.

Revised: To this end, we introduce a two-step iterative approach, generally described as follows (see Fig. 2):

Revised: • Step 1 (over-conditioning step).

Revised: Considering the over-conditioning case by additionally conditioning on the selected pieces of all hidden nodes in the DNN.

Revised: • Step 2 (homotopy step).

Revised: Combining multiple over-conditioning cases using the homotopy method to obtain A ( x ( z )) for all z 2 R .

Original: 3.3 Step1: Over-Conditioning Step
Revised: 3.3 Step

Revised: Over-Conditioning Step

Original: We show that by additionally conditioning on the selected pieces s ( x obs ) of all the hidden nodes, theselection event of DNN is written as a set of linear inequalities.
Revised: We show that by additionally conditioning on the selected pieces s ( x obs ) of all the hidden nodes, the selection event of DNN is written as a set of linear inequalities.

Original: The illustration is shown in Fig. 3a.
Revised: The illustration is shown in Fig. 3a.

Original: Lemma 2.
Revised: Lemma 2.

Original: Consider a class of DNN that consists of afﬁne operations and piecewise-linear AFs.
Revised: Consider a class of DNN that consists of afﬁne operations and piecewise-linear AFs.

Original: Then, the overconditioning region is written as
Revised: Then, the overconditioning region is written as

Original: [Equation]
Revised: [Equation]

Original: } using a matrix ⇥ ( s ( x obs )) and a vector ( s ( x obs )) that depend only on the selected pieces s ( x obs ) .
Revised: } using a matrix ⇥ ( s ( x obs )) and a vector ( s ( x obs )) that depend only on the selected pieces s ( x obs ) .

Original: Proof.
Revised: Proof.

Original: For the class of DNN that consists of afﬁne operations and piecewise-linear AFs, by ﬁxingthe selected pieces of all the piecewise-linear AFs, the input to each AF is represented by an afﬁnefunction of an image x .
Revised: For the class of DNN that consists of afﬁne operations and piecewise-linear AFs, by ﬁxing the selected pieces of all the piecewise-linear AFs, the input to each AF is represented by an afﬁne function of an image x .

Original: Therefore, the condition for selecting a piece in a piecewise-linear AF,s j ( x ( z )) =
Revised: Therefore, the condition for selecting a piece in a piecewise-linear AF, s j ( x ( z )) =

Original: s j ( x obs ) , is written as a linear inequality with respect to x ( z ) .
Revised: s j ( x obs ) , is written as a linear inequality with respect to x ( z ) .

Original: ⌅
Revised: ⌅

Original: Remark 1.
Revised: Remark 1.

Original: (Activation functions in the hidden layers)
Revised: (Activation functions in the hidden layers)

Original: In this work, we focus on a trained DNNwhere the AFs used at hidden layers are piecewise linear —for example, ReLU, Leaky ReLU, whichis commonly used in a CNN.
Revised: In this work, we focus on a trained DNN where the AFs used at hidden layers are piecewise linear —for example, ReLU, Leaky ReLU, which is commonly used in a CNN.

Original: Otherwise, if there is any speciﬁc demand to use non-piecewise linearfunctions (such as sigmoid or hyperbolic tangent), we can apply a piecewise-linear approximationapproach to these functions.
Revised: Otherwise, if there is any speciﬁc demand to use non-piecewise linear functions (such as sigmoid or hyperbolic tangent), we can apply a piecewise-linear approximation approach to these functions.

Original: We provide examples of such approximation in Appendix D.
Revised: We provide examples of such approximation in Appendix D.

Original: Remark 2.
Revised: Remark 2.

Original: (Operations in a trained neural network)
Revised: (Operations in a trained neural network)

Original: Most basic operations in a trained neural networkare written as afﬁne operations.
Revised: Most basic operations in a trained neural network are written as afﬁne operations.

Original: In a traditional neural network, the multiplication results between theweight matrix and the output of the previous layer and its summation with the bias vector is an afﬁneoperation.
Revised: In a traditional neural network, the multiplication results between the weight matrix and the output of the previous layer and its summation with the bias vector is an afﬁne operation.

Original: In a CNN, the convolution and upsampling operations are obviously afﬁne operations.
Revised: In a CNN, the convolution and upsampling operations are obviously afﬁne operations.

Original: Even when an operation in a DNN is NOT represented as a single afﬁne operation, it is often the casethat the operation is represented by as a set of afﬁne operations, as in the following remark.
Revised: Even when an operation in a DNN is NOT represented as a single afﬁne operation, it is often the case that the operation is represented by as a set of afﬁne operations, as in the following remark.

Original: Remark 3.
Revised: Remark 3.

Original: (Max-pooling)
Revised: (Max-pooling)

Original: Although the max-pooling is not an afﬁne operation, it can be writtenas a set of linear inequalities.
Revised: Although the max-pooling is not an afﬁne operation, it can be written as a set of linear inequalities.

Original: For instance, v 1 = max { v 1 , v 2 , v 3 } can be written as a set { e > 1 v e > 2 v , e > 1 v  e > 3 v } , where v = ( v 1 , v 2 , v 3 ) > and e i is the standard basis vector with 1 at position i .
Revised: For instance, v 1 = max { v 1 , v 2 , v 3 } can be written as a set { e > 1 v  e > 2 v , e > 1 v  e > 3 v } , where v = ( v 1 , v 2 , v 3 ) > and e i is the standard basis vector with 1 at position i .

Original: Remark 4.
Revised: Remark 4.

Original: (Activation functions at the output layer)
Revised: (Activation functions at the output layer)

Original: In Remark 1, we mention that we need toperform piecewise linear approximation for non-piecewise linear activations.
Revised: In Remark 1, we mention that we need to perform piecewise linear approximation for non-piecewise linear activations.

Original: However, if thesefunctions are used at the output layer (e.g., sigmoid function or tanh function), we need not performthe approximation task, because we can deﬁne the set of linear inequalities based on the values priorto activation.
Revised: However, if these [Table] [Table] functions are used at the output layer (e.g., sigmoid function or tanh function), we need not perform the approximation task, because we can deﬁne the set of linear inequalities based on the values prior to activation.

Original: See the next Example 2 for the case of a sigmoid function.
Revised: See the next Example 2 for the case of a sigmoid function.

Original: As Lemma 2 and the above remarks suggest, the proposed conditional SI approach is applicable to aclass of piecewise linear networks (many DNNs are or can be well-approximated by piecewise linearnetworks). We would like to note that, since we are primarily interested in the reliability of a trainednetwork given new inputs (i.e., not training inputs), the validity of our proposed method does notdepend on DNN training.
Revised: The list of components that are commonly used in a DNN for segmentation task and can be represented as a set of linear inequalities is provided in Appendix E. We would like to note that, since we are primarily interested in the reliability of a trained network given new inputs (i.e., not training inputs), the validity of our proposed method does not depend on DNN training.

Original: Therefore, all of the operations that only operate in the training processsuch as Dropoutor Batch Normalization do not affect the applicability of the proposed conditional SIapproach and we do not need consider the selection event of those operations in our method.[Table] [Table]
Revised: Therefore, all of the operations that only operate in the training process such as Dropout do not affect the applicability of the proposed SI approach and we do not need consider the selection event of those operations in our method.

Original: Example 2.
Revised: Example 2.

Original: (The selection event of a basic CNN is a set of linear inequalities) Consider a basicnetwork structure for segmentation as shown in Fig.
Revised: (The selection event of a basic CNN is a set of linear inequalities) Consider a basic network structure for segmentation as shown in Fig.

Original: 3b and n is an even number.
Revised: 3b and n is an even number.

Original: Let us start with theﬁrst convolutional layer with 4 ﬁlters.
Revised: Let us start with the ﬁrst convolutional layer with 4 ﬁlters.

Original: Let R f 2 R p n ⇥p n be the matrix obtained by the f th ﬁlter.
Revised: Let R f 2 R p n ⇥p n be the matrix obtained by the f th ﬁlter.

Original: We apply ReLU activation on R f , 8 f 2 [4] .
Revised: We apply ReLU activation on R f , 8 f 2 [4] .

Original: The selection event of ReLU is written as
Revised: The selection event of ReLU is written as

Original: [Equation]

Original: [Equation]

Original: , if output of ReLU

Original: [Equation]
Revised: [Equation]

Original: < 0 , otherwise .

Original: Let P f 2 R p n ⇥p n is the matrix after applying ReLU on R f .
Revised: Let P f 2 R p n ⇥p n is the matrix after applying ReLU on R f .

Original: We now move to the pooling layer byapplying the max-pooling on P f , 8 f 2 [4] , with 2 ⇥ 2 window.
Revised: We now move to the pooling layer by applying the max-pooling on P f , 8 f 2 [4] , with 2 ⇥ 2 window.

Original: Selection event of max-pooling is
Revised: Selection event of max-pooling is

Original: [Equation]
Revised: [Equation]

Original: P [Equation] o , where q = max { P fij , P fi ( j +1) , P f ( i +1)
Revised: where q = max { P fij , P fi ( j +1) , P f ( i +1)

Original: j , P f
Revised: j , P f

Original: ( i +1)
Revised: ( i +1)

Original: ( j +1) }
Revised: ( j +1) }

Original: and I = { 2  + 1 :  2 { 0 , 1 , ..., ( n/ 2  1) }} .
Revised: and I = { 2  + 1 :  2 { 0 , 1 , ..., ( n/ 2  1) }} .

Original: We continue the forward process until the last convolutional layer.
Revised: We continue the forward process until the last convolutional layer.

Original: Let W 2 R p n ⇥p n be the matrixat this ﬁnal layer.
Revised: Let W 2 R p n ⇥p n be the matrix at this ﬁnal layer.

Original: We apply sigmoid function on W .
Revised: We apply sigmoid function on W .

Original: The selection event of sigmoid is written as
Revised: The selection event of sigmoid is written as

Original: [Equation]
Revised: [Equation]

Original: , if

Original: [Equation]

Original: , otherwise .

Original: 
Revised:

Original: Finally, the over-conditioning selection event isS = S ReLU
Revised: Finally, the over-conditioning selection event is

Original: S S MaxPool
Revised: [Equation]

Original: S S Sigmoid , which is a223 set of linear inequalities.
Revised: which is a set of linear inequalities.

Original: 3.4 Step 2: Homotopy Step
Revised: 3.4 Step 2: Homotopy Step

Original: We introduce homotopy method to compute A ( x ( z )) by combining multiple over-conditioning steps.
Revised: We introduce homotopy method to compute A ( x ( z )) by combining multiple over-conditioning steps.

Original: Lemma 3.
Revised: Lemma 3.

Original: Consider a real value z t .
Revised: Consider a real value z t .

Original: By applying a trained DNN to x ( z t ) , we obtain a set of linearinequalities ⇥  ( s ( x ( z t )))
Revised: By applying a trained DNN to x ( z t ) , we obtain a set of linear inequalities ⇥ [Equation] ( s [Equation]

Original: x ( z t )  ( s ( x ( z t ))) .
Revised: x [Equation] t ) [Equation]

Original: Then, the next breakpoint z t +1 > z t , at which the statusof one node is changed from active to inactive or vice versa—that is, the sign of one linear inequalitychanges.
Revised: Then, the next breakpoint z t +1 > z t , at which the status of one node is changed from active to inactive or vice versa—that is, the sign of one linear inequality changes.

Original: This breakpoint is calculated by
Revised: This breakpoint is calculated by

Original: [Equation]
Revised: [Equation]

Original:  The proof of Lemma 3 is provided in Appendix B. Algorithm 2 shows our solution to efﬁcientlyidentify A ( x ( z )) .
Revised: [Figure] [Figure] The proof of Lemma 3 is provided in Appendix B. Algorithm 2 shows our solution to efﬁciently identify A ( x ( z )) .

Original: In this algorithm, multiple breakpoints z 1 < z 2 < ... <
Revised: In this algorithm, multiple breakpoints z 1 < z 2 < ... <

Original: z |T | are computed one byone.
Revised: z |T | are computed one by one.

Original: Each breakpoint z t , t 2 [ |T | ] indicates the point at which the sign of one linear inequality is233 changed, i.e., the status of one node in the network will change from active to inactive or vice versa.
Revised: Each breakpoint z t , t 2 [ |T | ] indicates the point at which the sign of one linear inequality is changed, i.e., the status of one node in the network will change from active to inactive or vice versa.

Original: By identifying all these breakpoints { z t } t 2 [ |T | ] , the solution path is given by A ( x ( z ))
Revised: By identifying all these breakpoints { z t } t 2 [ |T | ] , the solution path is given by A ( x ( z ))

Original: =
Revised: =

Original: A ( x ( z t ))
Revised: A ( x ( z t ))

Original: [-ifz 
-] [ z t , z t +1 ] , t 2 [ |T | ] .
Revised: [-ifz 
-]if z 2 [ z t , z t +1 ] , t 2 [ |T | ] .

Original: Algorithm 1 shows the entire procedure to calculate the selective p -value in(11).
Revised: Algorithm 1 shows the entire procedure to calculate the selective p -value in (11).

Original: First, we apply a DNN on x obs to obtain the segmentation result A ( x obs ) .
Revised: First, we apply a DNN on x obs to obtain the segmentation result A ( x obs ) .

Original: Next, we calculate ⌘which is used to identify a parameterized line in R n .
Revised: Next, we calculate ⌘ which is used to identify a parameterized line in R n .

Original: Then, we compute A ( x ( z )) by using Algorithm2.
Revised: Then, we compute A ( x ( z )) by using Algorithm 2.

Original: After obtaining A ( x ( z )) , we identify Z in (10), which is the key factor for computing the selectivep -value.
Revised: After obtaining A ( x ( z )) , we identify Z in (10), which is the key factor for computing the selective p -value.

Original: Regarding the choice of [ z min , z max ] , under a normal distribution, neither very positive norvery negative z values affect the inference.
Revised: Regarding the choice of [ z min , z max ] , under a normal distribution, neither very positive nor very negative z values affect the inference.

Original: Therefore, it is reasonable to consider a range of values,e.g., [  20  , 20  ] [16], where  is the standard deviation of the distribution of the test statistic.[Figure] [Figure]
Revised: Therefore, it is reasonable to consider a range of values, e.g., [  20  , 20  ] [15], where  is the standard deviation of the distribution of the test statistic.

Original: Complexity.
Revised: Complexity.

Original: In the worst-case, the complexity ofthe proposed homotopy method still grows exponentially.
Revised: In the literature of homotopy method (a.k.a. parametric programming), it is known that the actual computational cost differs signiﬁcantly from the worst case.

Original: This is a common issue in other homotopy (a.k a. parametric programming) applicationssuch as Lasso regularization path.
Revised: A well-known application of the homotopy method in the ML community is the Lasso regularization path, which also has the worst-case computational cost on the exponential order of the number of features, but the actual cost is known to be nearly linear order.

Original: However, fortunately, it has been well-recognized that this worstcase rarely happens in practice, and this is also evident from our experimental results  (Fig. 6). 
Revised: Similarly, in the proposed homotopy method for SI, it is also evident from our experimental results that the number of breakpoints is almost linearly increasing in practice (Fig. 6). We additionally note that, at step 4 of Algorithm 2, we apply a trained DNN to the parametrized data x ( z t ) at any breakpoints z t . Therefore, the number of forward passes of the network is equal to the number of breakpoints. Class of networks.

Original: Experiment
Revised: The proposed method can be applied to a class of piecewise linear networks whose network operations are characterized by a set of linear inequalities or approximated by piecewise-linear functions (many state-of-the-art image segmentation networks are or can be wellapproximated by piecewise linear networks). This is due to the reason that all the theories and algorithms in §3 only depend on the property of each component and does not depend on the entire structure of the network. We believe that our method is fairly general because most of the basic operations in a trained neural network can be decomposed or approximated by afﬁne operations as we presented in Remarks 1-4, Example 2 as well as Appendix D.

Original: Experimental setup.
Revised: Experimental setup.

Original: We compared the proposed method (homotopy method) with proposed-methodoc (over-conditioning), naive method and permutation test.
Revised: We compared the proposed method (homotopy method) with proposed-methodoc (over-conditioning), naive method and permutation test.

Original: The details of the methods for comparisonare shown in Appendix E. We considered two covariance matrices:⌃ =
Revised: The details of the methods for comparison are shown in Appendix F. We considered two covariance matrices:

Original:  I n (independence) and⌃ =
Revised: • ⌃ = I n (independence) and • ⌃ =

Original: [0 . 5 | i  j | ] ij 2 R n ⇥ n (correlation).
Revised: [0 . 5 | i  j | ] ij 2 R n ⇥ n (correlation).

Original: Regarding the FPR experiments, we generated 120 nullimages X 
Revised: Regarding the FPR experiments, we generated 120 null images X =

Original: = ( X 1 , ..., X n ) ⇠ N ( 0 n , ⌃ ) for n 2 { 16 , 64 , 256 , 1024 } .
Revised: ( X 1 , ..., X n ) ⇠ N ( 0 n , ⌃ ) for n 2 { 16 , 64 , 256 , 1024 } .

Original: For each value of n , we run120 trials.
Revised: For each value of n , we run 120 trials.

Original: To test the power, we generated images X with n = 256 , in which the true averagedifference in the underlying model µ O x  µ B x =  µ 2 { 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 } .
Revised: To test the power, we generated images X with n = 256 , in which the true average difference in the underlying model µ O x  µ B x =  µ 2 { 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 } .

Original: For each  µ , we run254 120 trials.
Revised: For each  µ , we run 120 trials.

Original: We selected the signiﬁcance level ↵ = 0 .
Revised: We selected the signiﬁcance level ↵ = 0 .

Original: and we used the basic CNN shown in Fig.
Revised: and we used the basic CNN shown in Fig.

Original: 3b.
Revised: 3b.

Original:  In the literature, many network architectures have been proposed for segmentation problems.
Revised: [Figure] [Figure] In the literature, many network architectures have been proposed for segmentation problems.

Original: Although these architectures are very complex, if they are broken down into smaller elements, each canbe represented (or at least well approximated) as an afﬁne operation whose selection event can behandled by the proposed method.
Revised: Although these architectures are very complex, if they are broken down into smaller elements, each can be represented (or at least well approximated) as an afﬁne operation whose selection event can be handled by the proposed method.

Original: Because the main contribution of this study is to introduce conditional SI by which to evaluate the reliability of the segmentation results, empirical implementationsand evaluations of the complex architectures are beyond the scope of this study.
Revised: Because the main contribution of this study is to introduce conditional SI by which to evaluate the reliability of the segmentation results, empirical implementations and evaluations of the complex architectures are beyond the scope of this study.

Original: Numerical results.
Revised: Numerical results.

Original: The results of the FPR control are shown in Fig.
Revised: The results of the FPR control are shown in Fig.

Original: The proposed methodssuccessfully control the FPR in both cases of independence and correlation while the naive andpermutation methods cannot .
Revised: The proposed methods successfully control the FPR in both cases of independence and correlation while the naive and permutation methods cannot .

Original: Because the naive and permutation methods fail to control the FPR,we no longer consider the power.
Revised: Because the naive and permutation methods fail to control the FPR, we no longer consider the power.

Original: Fig.
Revised: Fig.

Original: 5 shows that the homotopy method has higher TPR than theover-conditioning option.
Revised: 5 shows that the homotopy method has higher TPR than the over-conditioning option.

Original: Fig.
Revised: Fig.

Original: 6 shows the reason why the proposed homotopy method is efﬁcient.
Revised: 6 shows the reason why the proposed homotopy method is efﬁcient.

Original: Intuitively, to overcome the over-conditioning issue, we need to consider all combinations of theactivenesses of the nodes in a trained DNN, which is exponentially increasing.
Revised: Intuitively, to overcome the over-conditioning issue, we need to consider all combinations of the activenesses of the nodes in a trained DNN, which is exponentially increasing.

Original: With the homotopymethod, we need only consider the number of encountered intervals on the line along the directionof the test statistic, which is almost linearly increasing in practice.Additionally, we conﬁrm the [Figure]
Revised: With the homotopy method, we need only consider the number of encountered intervals on the line along the direction of the test statistic, which is almost linearly increasing in practice.

Original: [Figure] [Table] robustness of the proposed method in terms of FPR control by applying our method to data following
Revised: Additionally, we conﬁrm the robustness of the proposed method in terms of FPR control by applying our method to data following

Original: Laplace distribution, skew normal distribution (skewness coefﬁcient 10), and t 20 distribution.
Revised: Laplace distribution, skew normal distribution (skewness coefﬁcient 10), and t 20 distribution.

Original: We alsoconsider the case where variance is also estimated from the data.
Revised: We also consider the case where variance is also estimated from the data.

Original: We test the FPR for both ↵ = 0 .
Revised: We test the FPR for both ↵ = 0 .and

Original: 05and ↵ = 0 .
Revised: ↵ = 0 .

Original: The FPR results are shown in Fig.
Revised: The FPR results are shown in Fig.

Original: Our method still maintains good performance.
Revised: Our method still maintains good performance.

Original: Real-data examples.
Revised: Regarding the last plot (estimated sigma) in Fig. 7, since we want to empirical check the performance [Table] on FPR control when breaking the assumption of ⌃ (i.e., ⌃ is known or estimated from independent data), we set ⌃ =

Revised: s 2 I where s 2 is the empirical (sample) variance

Revised: and I is the identity matrix.

Original:  We examine the brain image dataset extracted from the dataset used in [4],which includes 939 images with tumors and 941 images without tumors.
Revised: Real-data examples. We examine the brain image dataset extracted from the dataset used in [4], which includes 939 images with tumors and 941 images without tumors.

Original: We ﬁrst compare ourmethod to a permutation test in terms of FPR control.
Revised: We ﬁrst compare our method to a permutation test in terms of FPR control.

Original: The results are shown in Table 1.
Revised: The results are shown in Table 1.

Original: Becausethe permutation test cannot properly control the FPR, a comparison of power is no longer needed.
Revised: Because the permutation test cannot properly control the FPR, a comparison of power is no longer needed.

Original: Comparisons of the naive p -value and the selective p -value are shown in Fig.
Revised: Comparisons of the naive p -value and the selective p -value are shown in Fig.

Original: 8 and Fig.
Revised: 8 and Fig.

Original: The naivep -value remains small, even when the image has no tumor region; this indicated that naive p -valuescannot be used to quantify the reliability of DNN-based segmentation results.
Revised: The naive p -value remains small, even when the image has no tumor region; this indicated that naive p -values cannot be used to quantify the reliability of DNN-based segmentation results.

Original: The proposed methodsuccessfully identiﬁes false positive and true positive detections.
Revised: The proposed method successfully identiﬁes false positive and true positive detections.

Original:  Discussion
Revised:  Discussion

Original: We propose a novel conditional SI method to conduct inference on the DNN-based image segmentation result.
Revised: We propose a novel conditional SI method to conduct inference on the DNN-based image segmentation result.

Original: We believe that this study stands as a signiﬁcant step toward reliable artiﬁcial intelligenceand opens several directions for statistically evaluating the reliability of DNN-driven hypotheses.
Revised: We believe that this study stands as a signiﬁcant step toward reliable artiﬁcial intelligence and opens several directions for statistically evaluating the reliability of DNN-driven hypotheses.

Original: Some open questions remain, as follows:
Revised: Some open questions remain, as follows:

Original: • The proposed method currently does not support softmax operator.
Revised: • The proposed method currently does not support softmax and batch normalization operators.

Revised: Although these operators can in theory be approximated by

Original: Although the softmax operatorcan in theory be approximated by afﬁne operations, doing so will be challenging in reality, as weneed to work in a multidimensional space.
Revised: afﬁne operations, doing so will be challenging in reality, as we need to work in a multidimensional space.

Original: Although it is difﬁcult to apply the proposed method tonetwork architectures containing components that are difﬁcult to decompose into afﬁne operations,one possible solutionto this issue is to devise a new network architecture that can be expressed as aset of afﬁne functions but still has state-of-the-art segmentation performance.•
Revised: Although it is difﬁcult to apply the proposed method to network architectures containing components that are difﬁcult to decompose into afﬁne operations, one possible solution is to devise a new network architecture that can be expressed as a set of afﬁne functions but still has state-of-the-art segmentation performance. • Although there is no technical limitation in applying our proposed method to state-of-the-art network structures, it requires a considerable amount of implementation effort.

Revised: This is because, for a large and complex network, we have to implement the selection events for all the components in the network.

Revised: The future direction could be ﬁrst to implement the proposed method for a few commonly used network structures, and then to develop a generic software tool that can automatically compute the selection event when the network structure is given.

Original: Although this study mainly focuses on the image segmentation task, the proposed method canbe applied to a class of problems where (1) the test statistic is deﬁned as a linear contrast w.r.t thedata and (2) the operations of the trained neural network can be characterized by a set of linearinequalities (or approximated by piecewise-linear functions).
Revised: • Even though this study mainly focuses on the image segmentation task, the proposed method can be applied to a class of problems where (1) the test statistic is deﬁned as a linear contrast w.r.t the data and (2) the operations of the trained neural network can be characterized by a set of linear inequalities (or approximated by piecewise-linear functions).

Original: Therefore, widening the applicabilityof the proposed method to other computer vision tasks—as well as to other ﬁelds such as naturallanguage processing and signal processing—would also stand as a valuable future contribution. 
Revised: Therefore, widening the applicability of the proposed method to other computer vision tasks—as well as to other ﬁelds such as natural language processing and signal processing—would also stand as a valuable future contribution. Acknowledgments and Disclosure of Funding

Revised: We thank the anonymous reviewers and area chair for their comments.

Revised: This work was partially supported by MEXT KAKENHI (20H00601), JST CREST (JPMJCR21D3), JST Moonshot

Revised: R&D (JPMJMS2033-05), JST AIP Acceleration Research (JPMJCR21U2), NEDO (JPNP18002,

Revised: JPNP20006), RIKEN Center for Advanced Intelligence Project, and RIKEN Junior Research Associate Program.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes](b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] Please see Sec.
Revised: [Yes] Please see Sec.

Original: 5.(c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?[Yes]
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original:  (b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] Please see Appendix E. (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] Please see Appendix F. (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if 
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original:  applicable? 
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: [Table]

Original: [Table]