# Optimal Tests for Combining \(p\)-Values
## 5 Discussion and Conclusions  
In this paper, we studied a class of gamma distribution-based \(p\)-value combination methods, which include special cases that are equivalent to some existing popular methods. This class of tests provide unlimited choices for combining independent \(p\)-values. However, under a given situation, some of them may perform very poorly. Therefore, arbitrarily picking one of them may result in failing to detect true alternatives. On the other hand, if we try many different methods and report the smallest \(p\)-value, we need to adjust this \(p\)-value due to multiple comparison issue; otherwise, we will have more false findings than expected due to inflated type I error rate. Therefore, it is desirable to develop methods that can adaptively find the optimal approach from candidate tests. Our proposed CLRT-based test \(T_{CLRT}\) is one of such methods. We have shown that if the \(p\)-values to be combined are from a common density function \(f_{\alpha,c}(p)=(1-c)^{\alpha}e^{c-1}(1-p)\) for \(p\in(0,1)\), the gamma distribution-based test \(T_{G(\alpha)}\) is UMP when both parameters \(\alpha\) and \(c\) are known. When \(\alpha=\alpha_{0}\) is known but \(c\) unknown, both \(T_{G(\alpha)}\) and the CLRT-based test \(T_{CLRT,\alpha_{0}}\) are asymptotically UMP. Furthermore, when both \(\alpha\) and \(c\) are unknown, the proposed CLRT-based test \(T_{CLRT}\) is asymptotically UMP. In a meta-analysis, it is natural to assign different weights to individual studies [5, 7, 37, 38]. For instance, a larger weight can be assigned to a study with more subjects; hence, in the z test, a \(p\)-value from a larger study may receive a greater weight. Weights can also be assigned based on other quantities, such as the variances of the estimated effect sizes. However, there is no consensus on weight assignment. For our proposed tests, we can easily incorporate weights assigned to each individual \(p\)-value. For instance, the weighted gamma distribution-based tests can be constructed using \(T_{G(\alpha)}^{w}=\sum_{i=1}^{n}F_{G(\
u;\alpha)}^{-1}(1-P_{i})\), where \(w_{i}\) is the weight assigned to study \(i\) (\(i=1,\cdots,n\)). Based on the properties of gamma distributions, it is not difficult to show that \(\lim\limits_{\alpha\to 0}T_{G(\alpha)}^{w}\equiv\sum_{i=1}^{n}w_{i} \Phi^{-1}(1-P_{i})/\sqrt{\sum_{i=1}^{n}w_{i}^{2}}\), the weighted z test. Therefore, the class of weighted gamma distribution-based tests \(T_{G(\alpha)}^{w}\) are generalizations of the weighted z test. Likewise, the weighted log-likelihood function with given weights becomes \(l^{w}(\alpha,c)=\alpha ln(1-c)\sum_{i=1}^{n}w_{i}+c\sum_{i=1}^{n}F_{G(\
u; \alpha)}^{-1}(1-p_{i})\), from which the corresponding weighted CLRT-based optimal test \(T_{CLRT}^{W}\) can be constructed accordingly. Our proposed tests have much broader applications than in meta-analysis. In fact, they can be applied to almost all statistical testing problems when (asymptotically) independent \(p\)-values from individual components are available. For instance, in model selection, a typical step is to test whether a set of variables (or a single categorical variable with multiple levels) should be included in the final model. Often the time, the parameters, and the covariances of their estimates are estimated simultaneously through maximum likelihood estimation. Then the LRT via comparing the log-likelihood values from two models with and without the candidate variables, or the Wald chi-square test of the weighted sum of the squared estimated effect sizes, can be applied. For both LRT and the Wald test, a set of asymptotically independent \(p\)-values can be obtained through their asymptotically independent components (see, e.g., Chapter 16 of [39]). Hence, our proposed \(p\)-value combination methods, such as \(T_{CLRT}\), can be applied and may result in a better final model. Another example is the association test for two categorical variables in a two-way contingency table to which the Pearson chi-square test is usually applied.