SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiarong Ding  
School of Mathematics and Statistics  
Xi'an Jiaotong University  
djr9901@stu.xjtu.edu.cn
Xuehu Zhu*  
School of Mathematics and Statistics  
Xi'an Jiaotong University  
zhuxuehu@xjtu.edu.cn
High-dimensional mediation analysis is often associated with a multiple testing problem for detecting significant mediators. Assessing the uncertainty of this detecting process via false discovery rate (FDR) has garnered great interest. To control the FDR in multiple testing, two essential steps are involved: ranking and selection. Existing approaches either construct p-values without calibration or disregard the joint information across tests, leading to conservation in FDR control or non-optimal ranking rules for multiple hypotheses. In this paper, we develop an adaptive mediation detection procedure (referred to as "AMDP") to identify relevant mediators while asymptotically controlling the FDR in high-dimensional mediation analysis. AMDP produces the optimal rule for ranking hypotheses and proposes a data-driven strategy to determine the threshold for mediator selection. This novel method captures information from the proportions of composite null hypotheses and the distribution of p-values, which turns the high dimensionality into an advantage instead of a limitation. The numerical studies on synthetic and real data sets illustrate the performances of AMDP compared with existing approaches.
Mediation analysis is regarded as a prevalent tool to dissect a mediation relationship between exposures and outcomes, and it has been widely applied in different fields, such as epidemiology [37], public health [21], policy evaluation [1], social sciences [25], neuroscience [10], and many others. Baron et al. [4] provided the basis for the advance of mediation analysis. They proposed a conventional regression-based approach, commonly referred to as the causal steps method, to examine the logical relationships among exposure, mediator, and outcome variables linking in a causal chain. While the causal steps method established necessary conditions for causal inference, it did not provide a joint test of the indirect effect of exposure on the outcome through a mediator, as recommended by MacKinnon et al. [32]. Thus, MacKinnon et al. [32] investigated the normality-based Sobel's test [38] as a means to detect mediation effects under the framework of the product-coefficient method, the product of the exposure-mediator and mediator-outcome effects. However, as highlighted by MacKinnon et al. [31], the distribution of product coefficients is not normally distributed, leading the Sobel's test being overly conservative. Hence, MacKinnon et al. [32] proposed the joint significance test (also name as the MaxP test) to alleviate this conservatism. Nonetheless, the joint significance test still suffers from low statistical power, as it overlooks the impact of the composite null structure in mediation analysis [33, 42].
To address the aforementioned issues, Taylor et al. [42] recommended the utilization of the distribution of the product method or bootstrapping as alternative procedures. These methods had been shown to exhibit higher power while still maintaining reasonable control over the Type I error rate. In the study of Kamukama et al. [27], a z-value based Sobel's test was introduced to investigate the mediation effect of competitive advantage on the relationship between intellectual capital and financial performance. Nuijten et al. [36] proposed an approach based on Bayesian models for testing the presence of an indirect effect. More recently, Zhang [46] developed two data-adaptive tests that outperform both Sobel's test and joint significance tests.
Despite these advances, a further challenge is that the methods mentioned above typically assume a low-dimensional mediator, whereas the development of high-throughput technologies promotes a growing need for research dealing with high-dimensional data. The multiple testing problem arising from high-dimension mediation analysis is aimed at identifying the relevant mediators that explain the effect of exposure on an outcome. Differ from a single test, multiple testing approaches are aimed to handle simultaneous testing, and often use the false discovery rate (FDR) to measure the uncertainty of detecting process [7], that is,
$$
\mathrm {F D P} = \frac {\# \left\{j : j \in \Delta_ {0} , j \in \widehat {\Delta} \right\}}{\# \{j \in \widehat {\Delta} \} \vee 1}, \quad \mathrm {F D R} = \mathbb {E} [ \mathrm {F D P} ], \tag {1}
$$
where  $j$  represents the index of mediator,  $\Delta_0$  is the index set of the null mediators,  $\widehat{\Delta}$  is the index set of selected mediators.
The control of FDR in multiple hypothesis testing consists of two primary steps: ranking and selection. In the initial step, a ranking statistic is calculated to evaluate the significance of each test, resulting in the ranking of hypotheses. The subsequent step is to maximize the selection set based on the established ranking order from the first step, while simultaneously maintaining the FDR at the target level. Although the p-value generated from the single hypothesis testing method can be considered as a ranking statistic and further combined with the BH procedure [7] for FDR control, there are still some limitations inherent to this approach. Firstly, the single p-value without calibration may be conservative and lead to excessive conservation in FDR control. Secondly, in the absence of joint information across multiple hypotheses, the ranking statistics based on the p-value may not be optimal.
To overcome these limitations, recent advancements in mediation analysis attempt to construct calibrated test statistics that account for the composite nature of the null hypothesis, and further achieve FDR control in multiple testing. For example, Dai et al. [12] developed a JS-mixture procedure, which utilizes the maxP statistics [32] as the ranking statistics and corrects the conservatism in the joint significance test by estimating the mixture distribution of p-values. Although JS-mixture sharply achieves FDR control, it is still underpowered for the reason that the maxP statistics does not account for the distribution information of two-dimension p-values under different hypotheses. Furthermore, Liu et al. [30] proposed a procedure called DACT focusing on constructing calibrated p-values for each single test by combining information across large-scale tests. Specially, DACT estimates the proportions of sub-null hypotheses and generates weighted p-values accordingly. However, DACT may suffer from underpowered performance in certain situations due to the following reasons. Intuitively, it ignores the alternative hypothesis information in its weighting scheme. On a deeper level, it does not fully consider the distributed information of two-dimensional p-values, limiting its ability to leverage valuable insights for improved power.
In this paper, we propose an adaptive mediation detection procedure (AMDP) for identifying relevant signals in high-dimensional mediation analysis. Our main contributions are summarized below:
- AMDP utilizes a two-dimensional p-value based local FDR as a test statistic, allowing for the comprehensive utilization of structural information of large-scale tests. Additionally, it determines the optimal rule for the order of selecting mediators.
- We establish theoretical results showing that AMDP enables asymptotic control of the FDR for selected mediators using the estimated local FDR.
- We reveal the critical importance of information retention in the ranking step for achieving optimal statistical power by discussing the limitations of the ranking statistics used in existing methods and conducting comparisons with AMDP.
- We empirically demonstrate the effectiveness of AMDP on synthetic and real data sets. Simulation results confirm the validity of our approach, and an application to a prostate
cancer dataset illustrates its satisfactory performance in identifying CpG methylation sites that mediate between risk SNPs and gene expression.
The remainder of this paper is organized as follows: Section 2 formally introduces an optimal ranking rule based on AMDP along with an estimator of local FDR. We theoretically prove the ability of AMDP in controlling the FDR while mimicking the optimal power. Section 3 presents the simulation studies to evaluate the performance of AMDP. In Section 4, we demonstrate the practical utility of AMDP by applying it to the prostate cancer dataset in TCGA 2015. We conclude the paper in Section 5. The technical proofs and additional discussion are postponed to Appendix.
2 AMDP: an optimal multiple testing procedure for FDR control
Let  $X$  be the exposure (independent variable),  $\{M_1, \dots, M_J\}$  be candidate mediators, and  $Y$  be the outcome (dependent variable). In the context of mediation analysis in the genome-wide association studies (GWASs),  $X$  often refers to the single nucleotide polymorphisms (SNPs),  $M$  corresponds to DNA methylation, and  $Y$  pertains to gene expression or a risk of disease. As stated by Baron et al. [4], the mediation relationship can be expressed by the following models:
$$
\mathbb {E} \left(M _ {j} \mid X\right) = \alpha_ {0 j} + \alpha_ {j} X,
$$
$$
\mathbb {E} \left(Y \mid M _ {j}, X\right) = \beta_ {0 j} + \beta_ {j} M _ {j} + \beta_ {1 j} X, \tag {2}
$$
where  $\alpha_{j}$  denotes the effect of  $X$  on  $M_{j}$ ,  $\beta_{j}$  represents the effect of  $M_{j}$  on  $Y$ , totally the product of  $\alpha_{j}$  and  $\beta_{j}$  represents indirect effect of  $X$  on  $Y$ .  $\beta_{1j}$  is the direct effect of  $X$  on  $Y$  with the  $M_{j}$  being fixed. We assume that there are no unmeasured confounding variables, also known as the sequential ignorability assumption [25, 43]. Any confounders can be adjusted by additional covariates [12], and such an adjustment is omitted in model (2) for simplification.
Testing whether  $\{M_1,\dots ,M_J\}$  plays an intermediary role in the causal path from  $X$  to  $Y$  in (2) can be transformed into a multiple testing problem:
$$
H _ {0 j}: \alpha_ {j} \beta_ {j} = 0 \text {v e r s u s} H _ {1 j}: \alpha_ {j} \beta_ {j} \neq 0. \tag {3}
$$
The above composite hypothesis can be decomposed into four disjoint cases as follows:
$$
\text {C a s e} 1, \mathrm {H} _ {0 0, j}: \quad \alpha_ {j} = 0 \text {a n d} \quad \beta_ {j} = 0,
$$
$$
\text {C a s e} 2, \mathrm {H} _ {0 1, j}: \quad \alpha_ {j} = 0 \text {a n d} \quad \beta_ {j} \neq 0,
$$
$$
\text {C a s e} 3, \mathrm {H} _ {1 0, j}: \quad \alpha_ {j} \neq 0 \text {a n d} \quad \beta_ {j} = 0,
$$
$$
\text {C a s e} 4, \mathrm {H} _ {1 1, j}: \quad \alpha_ {j} \neq 0 \text {a n d} \quad \beta_ {j} \neq 0,
$$
where Case 1-3 represents the composite null hypothesis, and Case 4 is the alternative hypothesis. A rejection of  $H_{0j}$  indicates the presence of a mediation effect by  $M_j$ . In this paper, the  $p$ -values for testing  $\alpha_j = 0$  and  $\beta_j = 0$  are respectively denoted as  $p_{1j} = 2\{1 - \Phi (|\hat{\alpha}_j| / \hat{\sigma}_{\alpha_j})\}$  and  $p_{2j} = 2\{1 - \Phi (|\hat{\beta}_j| / \hat{\sigma}_{\beta_j})\}$ , where  $\hat{\alpha}_j, \hat{\beta}_j, \hat{\sigma}_{\alpha_j}$  and  $\hat{\sigma}_{\beta_j}$  are the least squares estimators based on the models (2). Under the sequential ignorability assumption for mediation analysis,  $p_{1j}$  and  $p_{2j}$  are independent [30].
2.1 Optimal rejection region under the four-group model
To provide an optimal ranking guideline, we consider a pair of p-values  $p_j = (p_{1j}, p_{2j})$  under the empirical null inference framework [17]. Let  $H_{00,j} = 1$  if Case 1 holds,  $H_{01,j} = 1$  if Case 2 holds,  $H_{10,j} = 1$  if Case 3 holds, and  $H_{11,j} = 1$  otherwise. Assume the conditional density of  $p_j$  follows
$$
H _ {j} = \left\{ \begin{array}{l l} H _ {0 0, j} \sim \operatorname {B e r n o u l l i} \left\{\pi_ {0 0} \right\}, \\ H _ {0 1, j} \sim \operatorname {B e r n o u l l i} \left\{\pi_ {0 1} \right\}, \\ H _ {1 0, j} \sim \operatorname {B e r n o u l l i} \left\{\pi_ {1 0} \right\}, \\ H _ {1 1, j} \sim \operatorname {B e r n o u l l i} \left\{\pi_ {1 1} \right\}, \end{array} \quad p _ {j} \mid H _ {j} \sim \left\{ \begin{array}{l l} f _ {0 0} (p) & \text {i f} H _ {0 0, j} = 1, \\ f _ {0 1} (p) & \text {i f} H _ {0 1, j} = 1, \\ f _ {1 0} (p) & \text {i f} H _ {1 0, j} = 1, \\ f _ {1 1} (p) & \text {i f} H _ {1 1, j} = 1. \end{array} \right. \right. \tag {4}
$$
with  $p = \left(p^{(1)}, p^{(2)}\right) \in \mathbb{R}^2$ .  $\pi_{00}, \pi_{01}, \pi_{10}$ , and  $\pi_{11}$  represents the proportions of  $H_{00,j}$ ,  $H_{01,j}$ ,  $H_{10,j}$ , and  $H_{11,j}$ , respectively. It follows that  $\pi_{00} + \pi_{01} + \pi_{10} + \pi_{11} = 1$  due to the disjoint nature of the composite hypothesis.
Then, the density function of  $p_j$  follows the following four-group model, which can be considered as a variant version of the random mixture model [17].
$$
p _ {j} \sim f (p) = \pi_ {0 0} f _ {0 0} (p) + \pi_ {0 1} f _ {0 1} (p) + \pi_ {1 0} f _ {1 0} (p) + \pi_ {1 1} f _ {1 1} (p). \tag {5}
$$
Under the four-group model (5), the local FDR [18, 19] is defined as
$$
\operatorname {f d r} (p) = \mathbb {P} \left(H _ {0 0, j} \cup H _ {0 1, j} \cup H _ {1 0, j} = 1 \mid p _ {j} = p\right) = \frac {\pi_ {0 0} f _ {0 0} (p) + \pi_ {0 1} f _ {0 1} (p) + \pi_ {1 0} f _ {1 0} (p)}{f (p)}. \tag {6}
$$
It refers to the posterior probability that a hypothesis is null, given its corresponding p-value.
Before delving into the optimal ranking guidelines, we introduce several key definitions relevant to this objective. For any rejection region  $S \in [0,1]^2$ , we define the global FDR as
$$
\operatorname {g F D R} (S) = \mathbb {P} \left(H _ {0 0} \cup H _ {0 1} \cup H _ {1 0} = 1 \mid p _ {j} \in S\right), \tag {7}
$$
where  $H_{00}, H_{01}$ , and  $H_{10}$  are composite null hypotheses. The power is defined as
$$
\operatorname {P o w e r} (S) = \mathbb {P} \left\{p _ {j} \in S \mid H _ {1 1} = 1 \right\}. \tag {8}
$$
In the ranking step, the primary objective is to establish an optimal ranking rule that accurately reflects the significance order of the tests, while adhering to the optimality goal set in the selection step. Under the Neyman-Pearson framework [35], this optimality goal entails maximizing power while simultaneously controlling the global FDR at a targeted level of  $\alpha$ . This process can be formulated as a constrained optimization problem, i.e.
$$
\max  _ {S} \operatorname {P o w e r} (S) \quad \text {s u b j e c t t o} \mathrm {g F D R} (S) \leqslant \alpha . \tag {9}
$$
The optimal rule under the two-group model has been extensively studied in the literature. Researchers have proposed various methods for optimal decision-making based on different frameworks [5, 9]. Our optimality goal shares similarities with the work of Lei et al. [28]. They had demonstrated that, under Bayes rule, the optimal rejection thresholds are the level surfaces of local FDR. We extend this insight to p-values in two dimensions, and define the form of the rejection region as  $S(\zeta) = \{p : \mathrm{fdr}(p) \leq \zeta\}$ . A detailed and comprehensive explanation of this concept is provided in Theorem 1.
Theorem 1. Assume that
(i)  $f_{00}(p), f_{01}(p), f_{10}(p)$ , and  $f_{11}(p)$  are continuous;
(ii)  $\nu(p: \mathrm{fdr}(p) = t) = 0$  for any  $t \in (0,1]$ , where  $\nu$  is a Lebesgue-Stieltjes measure on the two-dimensional Borel space  $(\mathcal{R}^2, \mathcal{B}^2)$ .
Then, for any given global FDR level  $\alpha$ , there exists a unique value  $\zeta^{\star}$  such that  $S(\zeta^{\star})$  is the solution of the constrained optimization problem in (9). And the local FDR involved in  $S(\zeta^{\star})$  corresponds to the optimal ranking rule.
Remark 1. Genovese et al. [20] have shown that under weak conditions,  $\mathrm{gFDR} = \mathrm{FDR} + O\left(\frac{1}{\sqrt{J}}\right)$ , where  $J$  represents the number of mediators. Hence, controlling  $\mathrm{gFDR}$  and FDR are asymptotically equivalent as the number of mediators  $J$  tends to infinity. A similar result supporting this equivalence was also obtained by Storey [40].
2.2 The estimator of local FDR
From Theorem 1, we have established that the optimal ranking rule under the Neyman-Pearson framework is the local FDR. However, it is worth noting that the discussions in Section 2.1 are based on the assumption that the distribution of p-values and proportions of the composite hypothesis are known. In the following, we emphasize that our results still hold if the local FDR can be consistently estimated. Assuming that  $f_{00} \equiv 1$  ( $p$ -values follow uniform distribution in Case 1). The estimation of  $\mathrm{fdr}(p)$  can be divided into three parts: (i) The proportions of the composite null hypothesis  $\pi_{00}, \pi_{01}, \pi_{10}$ ; (ii) The mixture density  $f(p)$ ; (iii) The densities of the composite null hypothesis  $f_{01}(p), f_{10}(p)$ .
Motivated from Storey et al. [41],  $\pi_{01},\pi_{10}$ , and  $\pi_{00}$  can be estimated as follows:
$$
\hat {\pi} _ {0}. (\lambda) = \frac {\sum I \left(p _ {1 j} > \lambda\right)}{J (1 - \lambda)}, \quad \hat {\pi}. _ {0} (\lambda) = \frac {\sum I \left(p _ {2 j} > \lambda\right)}{J (1 - \lambda)}, \quad \hat {\pi} _ {0 0} (\lambda) = \frac {\sum I \left(p _ {1 j} > \lambda , p _ {2 j} > \lambda\right)}{J (1 - \lambda) ^ {2}}, \tag {10}
$$
where  $\hat{\pi}_{0}(\lambda)$  denotes the estimator of the proportion of null  $p_{1j}$ ,  $\hat{\pi}_{.0}(\lambda)$  denotes the estimator of the proportion of null  $p_{2j}$ .  $I(\cdot)$  is an indicator function, and  $\lambda \in [0,1)$  is a tuning parameter. In practice, there is a bias versus variance tradeoff for choosing a suitable  $\lambda$ . Further research on selecting an appropriate value of  $\lambda$  is detailed in the Appendix A. Following that
$$
\hat {\pi} _ {0 1} (\lambda) = \hat {\pi} _ {0}. (\lambda) - \hat {\pi} _ {0 0} (\lambda), \quad \hat {\pi} _ {1 0} (\lambda) = \hat {\pi}. _ {0} (\lambda) - \hat {\pi} _ {0 0} (\lambda). \tag {11}
$$
Next, we turn to the estimation of  $f(p)$ . For this purpose, we employed an adaptation of the beta kernel function proposed by Chen [11]. This choice is made considering the fact that  $p = (p^{(1)}, p^{(2)})$  falls within  $[0, 1]^2$ . The beta kernel function allows for a flexible and smooth estimation of  $f(p)$ , providing a suitable estimation approach for our analysis. Our beta kernel estimator is:
$$
\hat {f} (p) = \hat {f} \left(p ^ {(1)}, p ^ {(2)}\right) = J ^ {- 2} \left(\sum_ {j = 1} ^ {J} K _ {p ^ {(1)}, b} ^ {\star} \left(p _ {1 j}\right)\right) \left(\sum_ {j = 1} ^ {J} K _ {p ^ {(2)}, b} ^ {\star} \left(p _ {2 j}\right)\right), \tag {12}
$$
where  $K_{p,b}^{\star}$  is a boundary beta kernel defined as
$$
K _ {p, b} ^ {\star} (t) = \left\{ \begin{array}{l l} K _ {p / b, (1 - p) / b} (t) & \text {i f} p \in (2 b, 1 - 2 b), \\ K _ {\rho (p, b), (1 - p) / b} (t) & \text {i f} p \in [ 0, 2 b ], \\ K _ {p / b, \rho (1 - p)} (t) & \text {i f} p \in [ 1 - 2 b, 1 ], \end{array} \right.
$$
$K_{u,v}$  be the density function of a  $\operatorname{Beta}(u,v)$  random variable,  $b$  is a smoothing parameter, and  $\rho(p,b) = 2p^2 + 2.5 - \sqrt{4p^4 + 6p^2 + 2.25 - p^2 - p/b}$ .
In the context of mediation analysis, the density of p-values follows a mixture distribution, as indicated in (5). This mixture distribution involves three distinct types of null hypotheses:  $H_{01}$ ,  $H_{10}$ , and  $H_{00}$ . Distinguishing between  $H_{01}$  and  $H_{10}$ , as well as obtaining accurate estimators for  $f_{01}(p)$  (corresponding to  $H_{01}$ ) and  $f_{10}(p)$  (corresponding to  $H_{10}$ ) is indeed a challenging task. Motivated by the knockoff method [2], we consider leveraging the symmetry property of p-values under the composite null hypothesis to tackle this issue. Before diving into the details of utilization of the symmetry property for estimating  $f_{01}(p)$  and  $f_{10}(p)$ , we introduce some essential notations and assumptions.
Denote  $\Delta_{00},\Delta_{01}$ , and  $\Delta_{10}$  as the index set of the null mediators under the composite null hypothesis  $H_{00},H_{01}$ , and  $H_{10}$ , respectively. We define the region  $D$  as  $[0,0.5)^2$ , with its symmetric regions as follows:  $\tilde{D}_{01} = [0.5,1]\times [0,0.5)$ ,  $\tilde{D}_{10} = [0,0.5)\times [0.5,1]$ , and  $\tilde{D}_{00} = [0.5,1]^2$ . The assumptions are given as follows.
Assumption 1. For  $j \in \Delta_{00}$ , the sampling distribution of  $p_j$  is symmetric about  $p^{(1)} = 0.5$  and  $p^{(2)} = 0.5$ ; For  $j \in \Delta_{01}$ , the sampling distribution of  $p_j$  is symmetric about  $p^{(1)} = 0.5$ ; For  $j \in \Delta_{10}$ , the sampling distribution of  $p_j$  is symmetric about  $p^{(2)} = 0.5$ .
Assumption 2. The symmetric regions of  $D$  satisfy: (i) For  $p \in \tilde{D}_{00}$ ,  $\lim_{n \to \infty} f_{11}(p) = 0$ ,  $\lim_{n \to \infty} f_{01}(p) = 0$ ,  $\lim_{n \to \infty} f_{10}(p) = 0$ ; (ii) For  $p \in \tilde{D}_{01}$ ,  $\lim_{n \to \infty} f_{11}(p) = 0$  and  $\lim_{n \to \infty} f_{10}(p) = 0$ ; (iii) For  $p \in \tilde{D}_{10}$ ,  $\lim_{n \to \infty} f_{11}(p) = 0$  and  $\lim_{n \to \infty} f_{01}(p) = 0$ .
Assumption 1 is only required for the null mediators. It indicates that at least one of  $p_{1j}$  and  $p_{2j}$  follows a uniform distribution under the composite null hypothesis. Assumption 2 holds for any reasonable p-value. Since a non-null p-value should fall within [0, 0.5), we can infer that as the sample size  $n$  tends to infinity, the probability of p-values under alternatives falling within [0.5, 1] approaches zero. Additional explanations on Assumptions 1-2 are detailed in the Appendix D.2.
Remarkably, under Assumption 1, we can decompose  $f_{10}(p)$  and  $f_{01}(p)$  as follows:
$$
f _ {1 0} (p) = f _ {1 \cdot} \left(p ^ {(1)}\right) \cdot f _ {\cdot 0} \left(p ^ {(2)}\right) = f _ {1 \cdot} \left(p ^ {(1)}\right), \quad f _ {0 1} (p) = f _ {0 \cdot} \left(p ^ {(1)}\right) \cdot f _ {\cdot 1} \left(p ^ {(2)}\right) = f _ {\cdot 1} \left(p ^ {(2)}\right), \tag {13}
$$
where  $f_{0 \cdot}(p^{(1)}) = f_{\cdot 0}(p^{(2)}) = 1$ . This decomposition allows us to transform the problem into estimating  $f_{1} \cdot (p^{(1)})$  and  $f_{\cdot 1}(p^{(2)})$ , representing the marginal probability density of  $p_{1j}$  and  $p_{2j}$  under alternatives, respectively. Assumption 2 provides the inspiration to utilize the symmetric regions about  $D$  to address this estimation task, that is,
$$
k _ {1} \left(p ^ {(1)}\right) = \frac {\pi_ {0 0} + \pi_ {1 0} f _ {1} \cdot \left(p ^ {(1)}\right)}{\pi_ {0 0} + \pi_ {1 0}}, \quad k _ {2} \left(p ^ {(2)}\right) = \frac {\pi_ {0 0} + \pi_ {0 1} f _ {\cdot 1} \left(p ^ {(2)}\right)}{\pi_ {0 0} + \pi_ {0 1}}, \tag {14}
$$
where  $k_{1}(p^{(1)})$  denotes the density of  $p_{1j}$  under  $H_{10}$  and  $H_{00}$ , and  $k_{2}(p^{(2)})$  denotes the density of  $p_{2j}$  under  $H_{01}$  and  $H_{00}$ .
We apply the beta kernel function to  $p_{1j}$  in region  $\tilde{D}_{10}$  and  $\tilde{D}_{00}$ , as well as to  $p_{2j}$  in region  $\tilde{D}_{01}$  and  $\tilde{D}_{00}$ , to get the estimation of  $k_{1}(p^{(1)})$  and  $k_{2}(p^{(2)})$  as
$$
\hat {k} _ {1} \left(p ^ {(1)}\right) = J _ {1} ^ {- 1} \sum_ {j = 1} ^ {J _ {1}} K _ {p ^ {(1)}, b} ^ {\star} \left(p _ {1 j}\right), \quad \hat {k} _ {2} \left(p ^ {(2)}\right) = J _ {2} ^ {- 1} \sum_ {j = 1} ^ {J _ {2}} K _ {p ^ {(2)}, b} ^ {\star} \left(p _ {2 j}\right), \tag {15}
$$
where  $J_{1}$  is the number of p-values in region  $\tilde{D}_{10}$  and  $\tilde{D}_{00}$ ,  $J_{2}$  is the number of p-values in region  $\tilde{D}_{01}$  and  $\tilde{D}_{00}$ . Combining the estimators in (10)-(11), (13)-(15), we obtain the estimation of  $f_{1}(p^{(1)})$  and  $f_{\cdot 1}(p^{(2)})$  as:
$$
\hat {f} _ {1}. \left(p ^ {(1)}\right) = \frac {\left(\hat {\pi} _ {0 0} + \hat {\pi} _ {1 0}\right) \hat {k} _ {1} \left(p ^ {(1)}\right) - \hat {\pi} _ {0 0}}{\hat {\pi} _ {1 0}}, \quad \hat {f} _ {. 1} \left(p ^ {(2)}\right) = \frac {\left(\hat {\pi} _ {0 0} + \hat {\pi} _ {0 1}\right) \hat {k} _ {2} \left(p ^ {(2)}\right) - \hat {\pi} _ {0 0}}{\hat {\pi} _ {0 1}}. \tag {16}
$$