constraint qualifications in convex vector semi infinite optimization abstract convex vector or multi objective semi infinite optimization deals with the simultaneous minimization of finitely many convex scalar functions subject to infinitely many convex constraints this paper provides characterizations of the weakly efficient efficient and properly efficient points in terms of cones involving the data and karush kuhn tucker conditions the latter characterizations rely on different local and global constraint qualifications the results in this paper generalize those obtained by the same authors on linear vector semi infinite optimization problems

influential features pca for high dimensional clustering we consider a clustering problem where we observe feature vectors x i in r p i 1 2 ldots n from k possible classes the class labels are unknown and the main interest is to estimate them we are primarily interested in the modern regime of p gg n where classical clustering methods face challenges we propose influential features pca if pca as a new clustering procedure in if pca we select a small fraction of features with the largest kolmogorov smirnov ks scores obtain the first k 1 left singular vectors of the post selection normalized data matrix and then estimate the labels by applying the classical k means procedure to these singular vectors in this procedure the only tuning parameter is the threshold in the feature selection step we set the threshold in a data driven fashion by adapting the recent notion of higher criticism as a result if pca is a tuning free clustering method we apply if pca to 10 gene microarray data sets the method has competitive performance in clustering especially in three of the data sets the error rates of if pca are only 29 or less of the error rates by other methods we have also rediscovered a phenomenon on empirical null by efron j amer statist assoc 99 2004 96 104 on microarray data with delicate analysis especially post selection eigen analysis we derive tight probability bounds on the kolmogorov smirnov statistics and show that if pca yields clustering consistency in a broad context the clustering problem is connected to the problems of sparse pca and low rank matrix recovery but it is different in important ways we reveal an interesting phase transition phenomenon associated with these problems and identify the range of interest for each

estimating the proportion of true null hypotheses with application to dna microarray data summary we consider the problem of estimating the proportion of true null hypotheses 0 i n a multiple hypothesis set up the tests are based on observed p values we first review published estimators based on the estimator that was suggested by schweder and spjotvoll then we derive new estimators based on nonparametric maximum likelihood estimation of the p value density restricting to decreasing and convex decreasing densities the estimators of 0 are all derived under the assumption of independent test statistics their performance under dependence is investigated in a simulation study we find that the estimators are relatively robust with respect to the assumption of independence and work well also for test statistics with moderate dependence

the false discovery rate for multiple testing in factorial experiments identifying the important factors and effects is an important goal in the analysis of two level fractional factorial experiments in moderate and large experiments the large number of potential effects presents problems of multiple statistical inference in this work we apply the false discovery rate fdr idea in multiple inference to effect identification in factorial experiments the fdr is the expected proportion of inert effects among those declared active which we believe is a more relevant quantity to control than the probability of a single erroneous declaration the criterion that has been adopted in previous studies we show how to combine the control of fdr with popular methods for estimating contrast standard error in unreplicated experiments we present simulations of unreplicated 16 and 32 run experiments that illustrate the improvements in power that can be obtained by controlling the fdr we also analyze data from an actual 128 run experiment with replication in which the fdr identifie

tweedie s formula and selection bias we suppose that the statistician observes some large number of estimates zi each with its own unobserved expectation parameter i the largest few of the zi s are likely to substantially overestimate their corresponding i s this being an example of selection bias or regression to the mean tweedie s formula first reported by robbins in 1956 offers a simple empirical bayes approach for correcting selection bias this article investigates its merits and limitations in addition to the methodology tweedie s formula raises more general questions concerning empirical bayes theory discussed here as relevance and empirical bayes information there is a close connection between applications of the formula and james stein estimation

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

to how many simultaneous hypothesis tests can normal student s t or bootstrap calibration be applied in the analysis of microarray data and in some other contemporary statistical problems it is not uncommon to apply hypothesis tests in a highly simultaneous way the number n say of tests used can be much larger than the sample sizes n to which the tests are applied yet we wish to calibrate the tests so that the overall level of the simultaneous test is accurate often the sampling distribution is quite different for each test so there may not be an opportunity to combine data across samples in this setting how large can n be as a function of n before level accuracy becomes poor here we answer this question in cases where the statistic under test is of student s t type we show that if either the normal or student t distribution is used for calibration then the level of the simultaneous test is accurate provided that log n increases at a strictly slower rate than n 1 3 as n diverges on the other hand if bootstrap methods are used for calibration then we may choose log n almost as large as

size power and false discovery rates modern scientific technology has provided a new class of large scale simultaneous inference problems with thousands of hypothesis tests to consider at the same time microarrays epitomize this type of technology but similar situations arise in proteomics spectroscopy imaging and social science surveys this paper uses false discovery rate methods to carry out both size and power calculations on large scale problems a simple empirical bayes approach allows the false discovery rate fdr analysis to proceed with a minimum of frequentist or bayesian modeling assumptions closed form accuracy formulas are derived for estimated false discovery rates and used to compare different methodologies local or tail area fdr s theoretical permutation or empirical null hypothesis estimates two microarray data sets as well as simulations are used to evaluate the methodology the power diagnostics showing why nonnull cases might easily fail to appear on a list of significant discoveries

convolution equations for ultradifferentiable functions and ultradistributions we characterize surjectivity of convolution operators on spaces of ultradifferentiable functions and ultradistributions of beurling type in the spirit of hormander s convexity conditions this completes results of bonet galbis and meise in contrast to the classical approach our proofs only use properties of ultradistributions and functional analytic tools

a sharper bonferroni procedure for multiple tests of significance summary a simple procedure for multiple tests of significance based on individual p values is derived this simple procedure is sharper than holm s 1979 sequentially rejective procedure both procedures contrast the ordered p values with the same set of critical values holm s procedure rejects an hypothesis only if its p value and each of the smaller p values are less than their corresponding critical values the new procedure rejects all hypotheses with smaller or equal p values to that of any one found less than its critical value

generalizations of slater s constraint qualification for infinite convex programs in this paper we study constraint qualifications and duality results for infinite convex programs p inf f x g x s x c whereg g1 g2 ands s1 s2 si are convex cones i 1 2 c is a convex subset of a vector spacex andf andgi are respectively convex andsi convex i 1 2 in particular we consider the special case whens2 is in afinite dimensional space g2 is affine ands2 is polyhedral we show that a recently introduced simple constraint qualification and the so called quasi relative interior constraint qualification both extend to p from the special case thatg g2 is affine ands s2 is polyhedral in a finite dimensional space the so called partially finite program this provides generalized slater type conditions for p which are much weaker than the standard slater condition we exhibit the relationship between these two constraint qualifications and show how to replace the affine assumption ong2 and the finite dimensionality assumption ons2 by a local compactness assumption we then introduce the notion of strong quasi relative interior to get parallel results for more general infinite dimensional programs without the local compactness assumption our basic tool reduces to guaranteeing the closure of the sum of two closed convex cones

a note on controlling the number of false positives lehmann and romano 2005 annals of statistics33 1138 1154 discuss a bonferroni type procedure that bounds the probability that the number of false positives is larger than a specified number we note that this procedure will have poor power as compared to a multivariate permutation test type procedure when the experimental design accommodates a permutation test an example is given involving gene expression microarray data of breast cancer tumors

parameter dependence of solutions of differential equations on spaces of distributions and the splitting of short exact sequences we show that a linear partial differential operator with constant coefficients p d is surjective on the space of e valued ultra distributions over an arbitrary convex set if e is a nuclear frechet space with property dn in particular this holds if e is isomorphic to the space of tempered distributions s or to the space of germs of holomorphic functions over a one point set h 0 this result has an interpretation in terms of solving the scalar equation p d u f such that the solution u depends on parameter whenever the right hand side f also depends on the parameter in the same way a suitable analogue for surjective convolution operators over rd is obtained as well to get the above results we develop a splitting theory for short exact sequences of the form 0 x y z 0 where z is a frechet schwartz space and x y are pls spaces like the spaces of distributions or real analytic functions or their subspaces in particular an extension of the dn splitting theorem of vogt and wagner is obtained

local shrinkage rules levy processes and regularized regression we use l evy processes to generate joint prior distributions and therefore penalty functions for a location parameter b b1 bp as p grows large this generalizes the class of local global shrinkage rules based on scale mixtures of normals illuminates new connections among disparate methods and leads to new results for computing posterior means and modes under a wide class of priors we extend this framework to large scale regularized regression problems where p n and provide comparisons with other methodologies

on a generalized false discovery rate the concept of k fwer has received much attention lately as an appropriate error rate for multiple testing when one seeks to control at least k false rejections for some flxed k 1 a less conservative notion the k fdr has been introduced very recently by sarkar 19 generalizing the false discovery rate of banjamini and hochberg 1 in this article we bring newer insight to the k fdr considering a mixture model involving independent p values before motivating the developments of some new procedures that control it we prove the k fdr control of the proposed methods under a slightly weaker condition than in the mixture model we provide numerical evidence of the proposed methods superior power performance over some kfwer and k fdr methods finally we apply our methods to a real data set

stepup procedures controlling generalized fwer and generalized fdr in many applications of multiple hypothesis testing where more than one false rejection can be tolerated procedures controlling error rates measuring at least k false rejections instead of at least one for some fixed k 1 can potentially increase the ability of a procedure to detect false null hypotheses the k fwer a generalized version of the usual familywise error rate fwer is such an error rate that has recently been introduced in the literature and procedures controlling it have been proposed a further generalization of a result on the k fwer is provided in this article in addition an alternative and less conservative notion of error rate the k fdr is introduced in the same spirit as the k fwer by generalizing the usual false discovery rate fdr a k fwer procedure is constructed given any set of increasing constants by utilizing the kth order joint null distributions of the p values without assuming any specific form of dependence among all the p values procedures controlling the k fdr are also developed by using the kth order joint null distributions of the p values first assuming that the sets of null and nonnull p values are mutually independent or they are jointly positively dependent in the sense of being multivariate totally positive of order two mtp2 and then discarding that assumption about the overall dependence among the p values

a split and merge bayesian variable selection approach for ultrahigh dimensional regression summary we propose a bayesian variable selection approach for ultrahigh dimensional linear regression based on the strategy of split and merge the approach proposed consists of two stages split the ultrahigh dimensional data set into a number of lower dimensional subsets and select relevant variables from each of the subsets and aggregate the variables selected from each subset and then select relevant variables from the aggregated data set since the approach proposed has an embarrassingly parallel structure it can be easily implemented in a parallel architecture and applied to big data problems with millions or more of explanatory variables under mild conditions we show that the approach proposed is consistent i e the true explanatory variables can be correctly identified by the approach as the sample size becomes large extensive comparisons of the approach proposed have been made with penalized likelihood approaches such as the lasso elastic net sure independence screening and iterative sure independence screening the numerical results show that the approach proposed generally outperforms penalized likelihood approaches the models selected by the approach tend to be more sparse and closer to the true model

on the conditional distributions of low dimensional projections from high dimensional data we study the conditional distribution of low dimensional projections from high dimensional data where the conditioning is on other low dimensional projections to fix ideas consider a random d vector z that has a lebesgue density and that is standardized so that ez 0 and ezz id moreover consider two projections defined by unit vectors and namely a response y z and an explanatory variable x z it has long been known that the conditional mean of y given x is approximately linear in x under some regularity conditions cf hall and li 1993 however a corresponding result for the conditional variance has not been available so far we here show that the conditional variance of y given x is approximately constant in x again under some regularity conditions these results hold uniformly in and for most s provided only that the dimension of z is large in that sense we see that most linear submodels of a high dimensional overall model are approximately correct our findings provide new insights in a variety of modeling scenarios we discuss several examples including sparse linear modeling generalized linear models under potential link violation sliced inverse regression sliced average variance estimation and kernel learning machines

an improved bonferroni procedure for multiple tests of significance summary a modification of the bonferroni procedure for testing multiple hypotheses is presented the method based on the ordered p values of the individual tests is less conservative than the classical bonferroni procedure but is still simple to apply a simulation study shows that the probability of a type i error of the procedure does not exceed the nominal significance level a for a variety of multivariate normal and multivariate gamma test statistics for independent tests the procedure has type i error probability equal to a the method appears particularly advantageous over the classical bonferroni procedure when several highly correlated test statistics are involved

a simple procedure for the selection of significant effects summary given a large number of test statistics a small proportion of which represent departures from the relevant null hypothesis a simple rule is given for choosing those statistics that are indicative of departure it is based on fitting by moments a mixture model to the set of test statistics and then deriving an estimated likelihood ratio simulation suggests that the procedure has good properties when the departure from an overall null hypothesis is not too small

strong control conservative point estimation and simultaneous conservative consistency of false discovery rates a unified approach summary the false discovery rate fdr is a multiple hypothesis testing quantity that describes the expected proportion of false positive results among all rejected null hypotheses benjamini and hochberg introduced this quantity and proved that a particular step up p value method controls the fdr storey introduced a point estimate of the fdr for fixed significance regions the former approach conservatively controls the fdr at a fixed predetermined level and the latter provides a conservatively biased estimate of the fdr for a fixed predetermined significance region in this work we show in both finite sample and asymptotic settings that the goals of the two approaches are essentially equivalent in particular the fdr point estimates can be used to define valid fdr controlling procedures in the asymptotic setting we also show that the point estimates can be used to estimate the fdr conservatively over all significance regions simultaneously which is equivalent to controlling the fdr at all levels simultaneously the main tool that we use is to translate existing fdr methods into procedures involving empirical processes this simplifies finite sample proofs provides a framework for asymptotic results and proves that these procedures are valid even under certain forms of dependence

the simes method for multiple hypothesis testing with positively dependent test statistics abstract the simes method for testing intersection of more than two hypotheses is known to control the probability of type i error only when the underlying test statistics are independent although this method is more powerful than the classical bonferroni method it is not known whether it is conservative when the test statistics are dependent this article proves that for multivariate distributions exhibiting a type of positive dependence that arise in many multiple hypothesis testing situations the simes method indeed controls the probability of type i error this extends some results established very recently in the special case of two hypotheses

controlling the familywise error rate with plug in estimator for the proportion of true null hypotheses summary estimation of the number or proportion of true null hypotheses in multiple testing problems has become an interesting area of research the first important work in this field was performed by schweder and spjotvoll among others they proposed to use plug in estimates for the proportion of true null hypotheses in multiple test procedures to improve the power we investigate the problem of controlling the familywise error rate fwer when such estimators are used as plug in estimators in single step or step down multiple test procedures first we investigate the case of independent p values under the null hypotheses and show that a suitable choice of plug in estimates leads to control of fwer in single step procedures we also investigate the power and study the asymptotic behaviour of the number of false rejections although step down procedures are more difficult to handle we briefly consider a possible solution to this problem anyhow plug in step down procedures are not recommended here for dependent p values we derive a condition for asymptotic control of fwer and provide some simulations with respect to fwer and power for various models and hypotheses

a direct approach to sparse discriminant analysis in ultra high dimensions summary sparse discriminant methods based on independence rules such as the nearest shrunken centroids classifier tibshirani et al 2002 and features annealed independence rules fan fan 2008 have been proposed as computationally attractive tools for feature selection and classification with high dimensional data a fundamental drawback of these rules is that they ignore correlations among features and thus could produce misleading feature selection and inferior classification we propose a new procedure for sparse discriminant analysis motivated by the least squares formulation of linear discriminant analysis to demonstrate our proposal we study the numerical and theoretical properties of discriminant analysis constructed via lasso penalized least squares our theory shows that the method proposed can consistently identify the subset of discriminative features contributing to the bayes rule and at the same time consistently estimate the bayes classification direction even when the dimension can grow faster than any polynomial order of the sample size the theory allows for general dependence among features simulated and real data examples show that lassoed discriminant analysis compares favourably with other popular sparse discriminant proposals

operating characteristics and extensions of the false discovery rate procedure summary we investigate the operating characteristics of the benjamini hochberg false discovery rate procedure for multiple testing this is a distribution free method that controls the expected fraction of falsely rejected null hypotheses among those rejected the paper provides a framework for understanding more about this procedure we first study the asymptotic properties of the deciding point d that determines the critical p value from this we obtain explicit asymptotic expressions for a particular risk function we introduce the dual notion of false non rejections and we consider a risk function that combines the false discovery rate and false non rejections we also consider the optimal procedure with respect to a measure of conditional risk

some probability inequalities for ordered rm mtp sb 2 random variables a proof of the simes conjecture some new probability inequalities involving the ordered components of an mtp 2 random vector are derived which provide an analytical proof of an important conjecture in the field of multiple hypothesis testing this conjecture has been mostly validated so far using simulation

constraint qualifications for convex optimization without convexity of constraints new connections and applications to best approximation abstract we study constraint qualifications and necessary and sufficient optimality conditions for a convex optimization problem with inequality constraints where the constraint functions are continuously differentiable but they are not assumed to be convex we present constraint qualifications under which the karush kuhn tucker conditions are necessary and sufficient for optimality without the convexity of the constraint functions and establish new links among various known constraint qualifications that guarantee necessary karush kuhn tucker conditions we also present a new constraint qualification which is the weakest constraint qualification for the karush kuhn tucker conditions to be necessary for optimality of the convex optimization problem consequently we present lagrange multiplier characterizations for the best approximation from a convex set in the face of nonconvex inequality constraints extending corresponding known results in the literature we finally give a table summarizing various links among the constraint qualifications

plots of p values to evaluate many tests simultaneously summary when a large number of tests are made possibly on the same data it is proposed to base a simultaneous evaluation of all the tests on a plot of cumulative p values using the observed significance probabilities the points corresponding to true null hypotheses should form a straight line while those for false null hypotheses should deviate from this line the line may be used to estimate the number of true hypotheses the properties of the method are studied in some detail for the problems of comparing all pairs of means in a one way layout testing all correlation coefficients in a large correlation matrix and the evaluation of all 2 x 2 subtables in a contingency table the plot is illustrated on real data

predicting false discovery proportion under dependence we present a flexible framework for predicting error measures in multiple testing situations under dependence our approach is based on modeling the distribution of the probit transform of the p values by mixtures of multivariate skew normal distributions the model can incorporate dependence among p values and it allows for shape restrictions on the p value density a nonparametric bayesian scheme for estimating the components of the mixture model is outlined and markov chain monte carlo algorithms are developed these lead to the prediction of false discovery proportion and related credible bands an expression for the positive false discovery rate for dependent observations is also derived the power of the mixture model in estimation of key quantities in multiple testing is illustrated by a simulation study a dataset on kidney transplant is also analyzed using the methods developed

empirical bayes analysis of a microarray experiment microarrays are a novel technology that facilitates the simultaneous measurement of thousands of gene expression levels a typical microarray experiment can produce millions of data points raising serious problems of data reduction and simultaneous inference we consider one such experiment in which oligonucleotide arrays were employed to assess the genetic effects of ionizing radiation on seven thousand human genes a simple nonparametric empirical bayes model is introduced which is used to guide the efficient reduction of the data to a single summary statistic per gene and also to make simultaneous inferences concerning which genes were affected by the radiation although our focus is on one specific experiment the proposed methods can be applied quite generally the empirical bayes inferences are closely related to the frequentist false discovery rate fdr criterion

an imputation regularized optimization algorithm for high dimensional missing data problems and beyond missing data are frequently encountered in high dimensional problems but they are usually difficult to deal with by using standard algorithms such as the expectation maximization algorithm and its variants to tackle this difficulty some problem specific algorithms have been developed in the literature but there still lacks a general algorithm this work is to fill the gap we propose a general algorithm for high dimensional missing data problems the algorithm works by iterating between an imputation step and a regularized optimization step at the imputation step the missing data are imputed conditionally on the observed data and the current estimates of parameters and at the regularized optimization step a consistent estimate is found via the regularization approach for the minimizer of a kullback leibler divergence defined on the pseudocomplete data for high dimensional problems the consistent estimate can be found under sparsity constraints the consistency of the averaged estimate for the true parameter can be established under quite general conditions the algorithm is illustrated by using high dimensional gaussian graphical models high dimensional variable selection and a random coefficient model

the strong conical hull intersection property for convex programming the strong conical hull intersection property chip is a geometric property of a collection of finitely many closed convex intersecting sets this basic property which was introduced by deutsch et al in 1997 is one of the central ingredients in the study of constrained interpolation and best approximation in this paper we establish that the strong chip of intersecting sets of constraints is the key characterizing property for optimality and strong duality of convex programming problems we first show that a sharpened strong chip is necessary and sufficient for a complete lagrange multiplier characterization of optimality for the convex programming model problem where c is a closed convex subset of a banach space x s is a closed convex cone which does not necessarily have non empty interior y is a banach space is a continuous convex function and g x y is a continuous s convex function we also show that the strong chip completely characterizes the strong duality for partially finite convex programs where y is finite dimensional and g x ax b and s is a polyhedral convex cone global sufficient conditions which are strictly weaker than the slater type conditions are given for the strong chip and for the sharpened strong chip

polynomial splines and their tensor products in extended linear modeling 1994 wald memorial lecture analysis of variance type models are considered for a regression function or for the logarithm of a probability function conditional probability function density function conditional density function hazard function conditional hazard function or spectral density function polynomial splines are used to model the main effects and their tensor products are used to model any interaction components that are included in the special context of survival analysis the baseline hazard function is modeled and nonproportionality is allowed in general the theory involves the l 2 rate of convergence for the fitted model and its components the methodology involves least squares and maximum likelihood estimation stepwise addition of basis functions using rao statistics stepwise deletion using wald statistics and model selection using the bayesian information criterion cross validation or an independent test set publicly available software written in c and interfaced to s s plus is used to apply this methodology to real data

optimal classification in sparse gaussian graphic model consider a two class classification problem where the number of features is much larger than the sample size the features are masked by gaussian noise with mean zero and covariance matrix sigma where the precision matrix omega sigma 1 is unknown but is presumably sparse the useful features also unknown are sparse and each contributes weakly i e rare and weak to the classification decision by obtaining a reasonably good estimate of omega we formulate the setting as a linear regression model we propose a two stage classification method where we first select features by the method of innovated thresholding it and then use the retained features and fisher s lda for classification in this approach a crucial problem is how to set the threshold of it we approach this problem by adapting the recent innovation of higher criticism thresholding hct we find that when useful features are rare and weak the limiting behavior of hct is essentially just as good as the limiting behavior of ideal threshold the threshold one would choose if the underlying distribution of the signals is known if only somewhat surprisingly when omega is sufficiently sparse its off diagonal coordinates usually do not have a major influence over the classification decision compared to recent work in the case where omega is the identity matrix proc natl acad sci usa 105 2008 14790 14795 philos trans r soc lond ser a math phys eng sci 367 2009 4449 4470 the current setting is much more general which needs a new approach and much more sophisticated analysis one key component of the analysis is the intimate relationship between hct and fisher s separation another key component is the tight large deviation bounds for empirical processes for data with unconventional correlation structures where graph theory on vertex coloring plays an important role

stochastic approximation with discontinuous dynamics and state dependent noise w p 1 and weak convergence abstract stochastic approximations of the form xn 1 xn anh xn n are treated where h might not be continuous and the noise sequence n might depend on xn an averaging and an ordinary differential equation method are combined to get w p 1 convergence for both the above algorithm and for the case where the interates are projected back onto a bounded set g if they ever leave it two examples are developed the first being an automata problem where the dynamics are not smooth and the noise is state dependent and the second a robbins monro process with observation averaging which causes the noise to be state dependent each example is typical of a larger class the results hold if an is a sequence of random variables an being dependent on xi i n if an 0 but an2

adaptive and dynamic adaptive procedures for false discovery rate control and estimation summary many methods for estimation or control of the false discovery rate fdr can be improved by incorporating information about 0 the proportion of all tested null hypotheses that are true estimates of 0 are often based on the number of p values that exceed a threshold we first give a finite sample proof for conservative point estimation of the fdr when the parameter is fixed then we establish a condition under which a dynamic adaptive procedure whose parameter is determined by data will lead to conservative 0 and fdr estimators we also present asymptotic results on simultaneous conservative fdr estimation and control for a class of dynamic adaptive procedures simulation results show that a novel dynamic adaptive procedure achieves more power through smaller estimation errors for 0 under independence and mild dependence conditions we conclude by discussing the connection between estimation and control of the fdr and show that several recently developed fdr control procedures can be cast in a unifying framework where the strength of the procedures can be easily evaluated

selective inference on multiple families of hypotheses summary in many complex multiple testing problems the hypotheses are divided into families given the data families with evidence for true discoveries are selected and hypotheses within them are tested neither controlling the error rate in each family separately nor controlling the error rate over all hypotheses together can assure some level of confidence about the filtration of errors within the selected families we formulate this concern about selective inference in its generality for a very wide class of error rates and for any selection criterion and present an adjustment of the testing level inside the selected families that retains control of the expected average error over the selected families

surjectivity of partial differential operators with good fundamental solutions abstract we characterize the surjectivity of a linear partial differential operator with constant coefficients on e as well as on d in terms of the existence of good shifted fundamental solutions this characterization complements results of meise taylor and vogt as well as frerick and the present author

an equivalent measure of partial correlation coefficients for high dimensional gaussian graphical models gaussian graphical models ggms are frequently used to explore networks such as gene regulatory networks among a set of variables under the classical theory of ggms the construction of gaussian graphical networks amounts to finding the pairs of variables with nonzero partial correlation coefficients however this is infeasible for high dimensional problems for which the number of variables is larger than the sample size in this article we propose a new measure of partial correlation coefficient which is evaluated with a reduced conditional set and thus feasible for high dimensional problems under the markov property and adjacency faithfulness conditions the new measure of partial correlation coefficient is equivalent to the true partial correlation coefficient in construction of gaussian graphical networks based on the new measure of partial correlation coefficient we propose a multiple hypothesis test based method for the construction of gaussian graphical networks furthermore we establish

estimating the proportion of false null hypotheses among a large number of independently tested hypotheses we consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses focusing on the situation in which the proportion of false null hypotheses is very small we propose a family of methods for establishing lower 100 1 a confidence bounds for this proportion based on the empirical distribution of the p values of the tests methods in this family are then compared in terms of ability to consistently estimate the proportion by letting 0 as the number of hypothesis tests increases and the proportion decreases this work is motivated by a signal detection problem that occurs in astronomy

penalized normal likelihood and ridge regularization of correlation and covariance matrices high dimensionality causes problems in various areas of statistics a particular situation that rarely has been considered is the testing of hypotheses about multivariate regression models in which

false discovery rates and copy number variation copy number changes the gains and losses of chromosome segments are a common type of genetic variation among healthy individuals as well as an important feature in tumour genomes microarray technology enables us to simultaneously measure with moderate accuracy copy number variation at more than a million chromosome locations and for hundreds of subjects this leads to massive data sets and complicated inference problems concerning which locations are more likely to vary in this paper we consider a relatively simple false discovery rate approach to copy number analysis more careful parametric change point methods can then be focused on promising regions of the genome

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

false discovery rate adjusted multiple confidence intervals for selected parameters often in applied research confidence intervals cis are constructed or reported only for parameters selected after viewing the data we show that such selected intervals fail to provide the assumed coverage probability by generalizing the false discovery rate fdr approach from multiple testing to selected multiple cis we suggest the false coverage statement rate fcr as a measure of interval coverage following selection a general procedure is then introduced offering fcr control at level q under any selection rule the procedure constructs a marginal ci for each selected parameter but instead of the confidence level 1 q being used marginally q is divided by the number of parameters considered and multiplied by the number selected if we further use the fdr controlling testing procedure of benjamini and hochberg for selecting the parameters the newly suggested procedure offers cis that are dual to the testing procedure and are shown to be optimal in the independent case under the positive re

generalizing simes test and hochberg s stepup procedure in a multiple testing problem where one is willing to tolerate a few false rejections procedure controlling the familywise error rate fwer can potentially be improved in terms of its ability to detect false null hypotheses by generalizing it to control the k fwer the probability of falsely rejecting at least k null hypotheses for some fixed k 1 simes test for testing the intersection null hypothesis is generalized to control the k fwer weakly that is under the intersection null hypothesis and hochberg s stepup procedure for simultaneous testing of the individual null hypotheses is generalized to control the k fwer strongly that is under any configuration of the true and false null hypotheses the proposed generalizations are developed utilizing joint null distributions of the k dimensional subsets of the p values assumed to be identical the generalized simes test is proved to control the k fwer weakly under the multivariate totally positive of order two mtp 2 condition j multivariate analysis 10 1980 467 498 of the joint null distribution of the p values by generalizing the original simes inequality it is more powerful to detect k or more false null hypotheses than the original simes test when the p values are independent a stepdown procedure strongly controlling the k fwer a version of generalized holm s procedure that is different from and more powerful than ann statist 33 2005 1138 1154 with independent p values is derived before proposing the generalized hochberg s procedure the strong control of the k fwer for the generalized hochberg s procedure is established in situations where the generalized simes test is known to control its k fwer weakly

application of non parametric empirical bayes estimation to high dimensional classification we consider the problem of classification using high dimensional features space in a paper by bickel and levina 2004 it is recommended to use naive bayes classifiers that is to treat the features as if they are statistically independent consider now a sparse setup where only a few of the features are informative for classification fan and fan 2008 suggested a variable selection and classification method called fair the fair method improves the design of naive bayes classifiers in sparse setups the improvement is due to reducing the noise in estimating the features means this reduction is since that only the means of a few selected variables should be estimated we also consider the design of naive bayes classifiers we show that a good alternative to variable selection is estimation of the means through a certain non parametric empirical bayes procedure in sparse setups the empirical bayes implicitly performs an efficient variable selection it also adapts very well to non sparse setups and has the advantage of making use of the information from many weakly informative variables which variable selection type of classification procedures give up on using we compare our method with fair and other classification methods in simulation for sparse and non sparse setups and in real data examples involving classification of normal versus malignant tissues based on microarray data

adaptive linear step up procedures that control the false discovery rate we provide a new two stage procedure in which the linear step up procedure is used in stage one to estimate mo providing a new level q which is used in the linear step up procedure in the second stage we prove that a general form of the two stage procedure controls the false discovery rate at the desired level q this framework enables us to study analytically the properties of other procedures that exist in the literature a simulation study is presented that shows that two stage adaptive procedures improve in power over the original procedure mainly because they provide tighter control of the false discovery rate we further study the performance of the current suggestions some variations of the procedures and previous suggestions in the case where the test statistics are positively dependent a case for which the original procedure controls the false discovery rate in the setting studied here the newly proposed two stage procedure is the only one that controls the false discovery rate the procedures are illustrated with two examples of biological importance

robust variable and interaction selection for logistic regression and general index models under the logistic regression framework we propose a forward backward method soda for variable selection with both main and quadratic interaction terms in the forward stage soda adds in predic

some results on false discovery rate in stepwise multiple testing procedures the concept of false discovery rate fdr has been receiving increasing attention by researchers in multiple hypotheses testing this paper produces some theoretical results on the fdr in the context of stepwise multiple testing procedures with dependent test statistics it was recently shown by benjamini and yekutieli that the benjamini hochberg step up procedure controls the fdr when the test statistics are positively dependent in a certain sense this paper strengthens their work by showing that the critical values of that procedure can be used in a much more general stepwise procedure under similar positive dependency it is also shown that the fdr controlling benjamini liu step down procedure originally developed for independent test statistics works even when the test statistics are positively dependent in some sense an explicit expression for the fdr of a generalized stepwise procedure and an upper bound to the fdr of a step down procedure are obtained in terms of probability distributions of ordered components of dependent random variables before establishing the main results 1 introduction the main purpose of this paper is to present some formulas and inequalities providing theoretical insight into when and what stepwise procedures control the false discovery rate fdr at a prespecified level in multiple hypotheses testing the fdr first introduced in multiple testing by benjamini and hochberg 1995 is the expected proportion of erroneously rejected null hypotheses with v and r representing respectively the number of true null hypotheses rejected and the total number of null hypotheses rejected in a multiple testing procedure let q v r if r 0a nd 0i fr 0 then e q is the fdr of that procedure as they have argued it is an appropriate error rate to control in many practical situations particularly where a large number of null hypotheses are involved they also put forward an fdr controlling stepup procedure that is more powerful than comparable procedures controlling the traditional familywise error fwe rate when the underlying test statistics are independent benjamini and liu 1999 recently introduced a step down procedure with the same property the concept of fdr is now being used in a wide variety of applications abramovich and benjamini 1996 basford and tukey 1997 drigalenko

nonlinearly constrained best approximation in hilbert spaces the strong chip and the basic constraint qualification we study best approximation problems with nonlinear constraints in hilbert spaces the strong conical hull intersection property chip and the basic constraint qualification bcq condition are discussed best approximations with differentiable constraints and convex constraints are characterized the analysis generalizes some linearly constrained results of recent works f deutsch w li and j ward j approx theory 90 1997 pp 385 444 f deutsch w li and j d ward siam j optim 10 1999 pp 252 268

a stochastic process approach to false discovery control this paper extends the theory of false discovery rates fdr pioneered by benjamini and hochberg j roy statist soc ser b 57 1995 289 300 we develop a framework in which the false discovery proportion fdp the number of false rejections divided by the number of rejections is treated as a stochastic process after obtaining the limiting distribution of the process we demonstrate the validity of a class of procedures for controlling the false discovery rate the expected fdp we construct a confidence envelope for the whole fdp process from these envelopes we derive confidence thresholds for controlling the quantiles of the distribution of the fdp as well as controlling the number of false discoveries we also investigate methods for estimating the p value distribution

estimating the false discovery rate using the stochastic approximation algorithm testing of multiple hypotheses involves statistics that are strongly dependent in some applications but most work on this subject is based on the assumption of independence we propose a new method for estimating the false discovery rate of multiple hypothesis tests in which the density of test scores is estimated parametrically by minimizing the kullback leibler distance between the unknown density and its estimator using the stochastic approximation algorithm and the false discovery rate is estimated using the ensemble averaging method our method is applicable under general dependence between test statistics numerical comparisons between our method and several competitors conducted on simulated and real data examples show that our method achieves more accurate control of the false discovery rate in almost all scenarios

on best approximation by nonconvex sets and perturbation of nonconvex inequality systems in hilbert spaces by virtue of convexification techniques we study best approximations to a closed set c in a hilbert space as well as perturbation conditions relative to c and a nonlinear inequality system some results on equivalence of the best approximation and the basic constraint qualification are established

the control of the false discovery rate in multiple testing under dependency benjamini and hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems a simple procedure was given there as an fdr controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate we prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses this condition for positive dependency is general enough to cover many problems of practical interest including the comparisons of many treatments with a single control multivariate normal test statistics with positive correlation matrix and multivariate t furthermore the test statistics may be discrete and the tested hypotheses composite without posing special difficulties for all other forms of dependency a simple conservative modification of the procedure controls the false discovery rate thus the range of problems for which a procedure with proven fdr control can be offered is greatly increased 1 1 simultaneous hypotheses testing the control of the increased type i error when testing simultaneously a family of hypotheses is a central issue in the area of multiple comparisons rarely are we interested only in whether all hypotheses are jointly true or not which is the test of the intersection null hypothesis in most applications we infer about the individual hypotheses realizing that some of the tested hypotheses are usually true we hope not all and some are not we wish to decide which ones are not true indicating statistical discoveries an important such problem is that of multiple endpoints in a clinical trial a new treatment is compared with an existing one in terms of a large number of potential benefits endpoints

a direct approach to false discovery rates summary multiple hypothesis testing involves guarding against much more complicated errors than single hypothesis testing whereas we typically control the type i error rate for a single hypothesis test a compound error rate is controlled for multiple hypothesis tests for example controlling the false discovery rate fdr traditionally involves intricate sequential p value rejection methods based on the observed data whereas a sequential p value method fixes the error rate and estimates its corresponding rejection region we propose the opposite approach we fix the rejection region and then estimate its corresponding error rate this new approach offers increased applicability accuracy and power we apply the methodology to both the positive false discovery rate pfdr and fdr and provide evidence for its benefits it is shown that pfdr is probably the quantity of interest over fdr also discussed is the calculation of the q value the pfdr analogue of the p value which eliminates the need to set the error rate beforehand as is traditionally done some simple numerical examples are presented that show that this new approach can yield an increase of over eight times in power compared with the benjamini hochberg fdr method

empirical bayes estimates for large scale prediction problems classical prediction methods such as fisher s linear discriminant function were designed for small scale problems in which the number of predictors n is much smaller than the number of observations n modern scientific devices often reverse this situation a microarray analysis for example might include n 100 subjects measured on n 10 000 genes each of which is a potential predictor this article proposes an empirical bayes approach to large scale prediction where the optimum bayes prediction rule is estimated employing the data from all of the predictors microarray examples are used to illustrate the method the results demonstrate a close connection with the shrunken centroids algorithm of tibshirani et al 2002 a frequentist regularization approach to large scale prediction and also with false discovery rate theory

adaptive false discovery rate control under independence and dependence in the context of multiple hypothesis testing the proportion 0 of true null hypotheses in the pool of hypotheses to test often plays a crucial role although it is generally unknown a priori a testing procedure using an implicit or explicit estimate of this quantity in order to improve its efficency is called adaptive in this paper we focus on the issue of false discovery rate fdr control and we present new adaptive multiple testing procedures with control of the fdr in a first part assuming independence of the p values we present two new procedures and give a unified review of other existing adaptive procedures that have provably controlled fdr we report extensive simulation results comparing these procedures and testing their robustness when the independence assumption is violated the new proposed procedures appear competitive with existing ones the overall best though is reported to be storey s estimator albeit for a specific parameter setting that does not appear to have been considered before in a second part we propose adaptive versions of step up procedures that have provably controlled fdr under positive dependence and unspecified dependence of the p values respectively in the latter case while simulations only show an improvement over non adaptive procedures in limited situations these are to our knowledge among the first theoretically founded adaptive multiple testing procedures that control the fdr when the p values are not independent

constraint qualification the strong chip and best approximation with convex constraints in banach spaces several fundamental concepts such as the basic constraint qualification bcq the strong conical hull intersection property chip and the perturbations for convex systems of inequalities in banach spaces over mathbb r or mathbb c are extended and studied here the systems are not necessarily finite their relationships with each other in connection with the best approximations are investigated as applications we establish results on the unconstrained reformulation of best approximations with infinitely many constraints in hilbert spaces also we give several characterizations of best restricted range approximations in c q under quite general constraints

continuous optimizationconstraint qualifications in convex vector semi infinite optimization convex vector or multi objective semi infinite optimization deals with the simultaneous minimization of finitely many convex scalar functions subject to infinitely many convex constraints this paper provides characterizations of the weakly efficient efficient and properly efficient points in terms of cones involving the data and karush kuhn tucker conditions the latter characterizations rely on different local and global constraint qualifications the results in this paper generalize those obtained by the same authors on linear vector semi infinite optimization problems

false discovery and false nondiscovery rates in single step multiple testing procedures results on the false discovery rate fdr and the false nondiscovery rate fnr are developed for single step multiple testing procedures in addition to verifying desirable properties of fdr and fnr as measures of error rates these results extend previously known results providing further insights particularly under dependence into the notions of fdr and fnr and related measures first considering fixed configurations of true and false null hypotheses inequalities are obtained to explain how an fdr or fnr controlling single step procedure such as a bonferroni or sidak procedure can potentially be improved two families of procedures are then constructed one that modifies the fdr controlling and the other that modifies the fnr controlling sidak procedure these are proved to control fdr or fnr under independence less conservatively than the corresponding families that modify the fdr or fnr controlling bonferroni procedure results of numerical investigations of the performance of the modified sidak fdr procedure over its competitors are presented second considering a mixture model where different configurations of true and false null hypotheses are assumed to have certain probabilities results are also derived that extend some of storey s work to the dependence case

a resampling based stochastic approximation method for analysis of large geostatistical data the gaussian geostatistical model has been widely used in modeling of spatial data however it is challenging to computationally implement this method because it requires the inversion of a large covariance matrix particularly when there is a large number of observations this article proposes a resampling based stochastic approximation method to address this challenge at each iteration of the proposed method a small subsample is drawn from the full dataset and then the current estimate of the parameters is updated accordingly under the framework of stochastic approximation since the proposed method makes use of only a small proportion of the data at each iteration it avoids inverting large covariance matrices and thus is scalable to large datasets the proposed method also leads to a general parameter estimation approach maximum mean log likelihood estimation which includes the popular maximum log likelihood estimation mle approach as a special case and is expected to play an important role

type ii generalized family wise error rate formulas with application to sample size determination multiple endpoints are increasingly used in clinical trials the significance of some of these clinical trials is established if at least r null hypotheses are rejected among m that are simultaneously tested the usual approach in multiple hypothesis testing is to control the family wise error rate which is defined as the probability that at least one type i error is made more recently the q generalized family wise error rate has been introduced to control the probability of making at least q false rejections for procedures controlling this global type i error rate we define a type ii r generalized family wise error rate which is directly related to the r power defined as the probability of rejecting at least r false null hypotheses we obtain very general power formulas that can be used to compute the sample size for single step and step wise procedures these are implemented in our r package rpowersamplesize available on the cran making them directly available to end users complexities of the formulas are presented to gain insight into computation time issues comparison with monte carlo strategy is also presented we compute sample sizes for two clinical trials involving multiple endpoints one designed to investigate the effectiveness of a drug against acute heart failure and the other for the immunogenicity of a vaccine strategy against pneumococcus copyright 2016 john wiley sons ltd

the positive false discovery rate a bayesian interpretation and the q value multiple hypothesis testing is concerned with controlling the rate of false positives when testing several hypotheses simultaneously one multiple hypothesis testing error measure is the false discovery rate fdr which is loosely defined to be the expected proportion of false positives among all significant hypotheses the fdr is especially appropriate for exploratory analyses in which one is interested in finding several significant results among many tests in this work we introduce a modified version of the fdr called the positive false discovery rate pfdr we discuss the advantages and disadvantages of the pfdr and investigate its statistical properties when assuming the test statistics follow a mixture distribution we show that the pfdr can be written as a bayesian posterior probability and can be connected to classification theory these properties remain asymptotically true under fairly general conditions even under certain forms of dependence also a new quantity called the q value is introduced and investigated which is a natural bayesian posterior p value or rather the pfdr analogue of the p value 1 introduction when testing a single hypothesis one is usually concerned with controlling the false positive rate while maximizing the probability of detecting an effect when one really exists in statistical terms we maximize the power conditional on the type i error rate being at or below some level the field of multiple hypothesis testing tries to extend this basic paradigm to the situation where several hypotheses are tested simultaneously one must define an appropriate compound error measure according to the rate of false positives one is willing to encounter then a procedure is developed that allows one to control the error rate at a desired level while maintaining the power of each test as much as possible the most commonly controlled quantity when testing multiple hypotheses is the family wise error rate fwer which is the probability of yielding one or more false positives out of all hypotheses tested the most familiar example of this is the bonferroni method if there are m hypothesis tests each test is controlled so that the probability of a false positive is less than or equal to m for some chosen value of it then follows that the overall fwer is less than or equal to m any

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

estimation of a two component mixture model with applications to multiple testing we consider a two component mixture model with one known component we develop methods for estimating the mixing proportion and the other unknown distribution nonparametrically given i i d data from the mixture model we use ideas from shape restricted function estimation and develop tuning parameter free estimators that are easily implementable and have good finite sample performance we establish the consistency of our procedures distribution free finite sample lower confidence bounds are developed for the mixing proportion the identifiability of the model and the estimation of the density of the unknown mixing distribution are also addressed we discuss the connection with the problem of multiple testing and compare our procedure with some of the existing methods in that area through simulation studies we also analyse two data sets one arising from an application in astronomy and the other from a microarray experiment

a note on the adaptive control of false discovery rates summary the use of a fixed rejection region for multiple hypothesis testing has been shown to outperform standard fixed error rate approaches when applied to control of the false discovery rate in this work it is demonstrated that if the original step up procedure of benjamini and hochberg is modified to exercise adaptive control of the false discovery rate its performance is virtually identical to that of the fixed rejection region approach in addition the dependence of both methods on the proportion of true null hypotheses is explored with a focus on the difficulties that are involved in the estimation of this quantity

bayesian subset modeling for high dimensional generalized linear models this article presents a new prior setting for high dimensional generalized linear models which leads to a bayesian subset regression bsr with the maximum a posteriori model approximately equivalent to the minimum extended bayesian information criterion model the consistency of the resulting posterior is established under mild conditions further a variable screening procedure is proposed based on the marginal inclusion probability which shares the same properties of sure screening and consistency with the existing sure independence screening sis and iterative sure independence screening isis procedures however since the proposed procedure makes use of joint information from all predictors it generally outperforms sis and isis in real applications this article also makes extensive comparisons of bsr with the popular penalized likelihood methods including lasso elastic net sis and isis the numerical results indicate that bsr can generally outperform the penalized likelihood methods the m

the bayesian bridge summary we propose the bayesian bridge estimator for regularized regression and classification two key mixture representations for the bayesian bridge model are developed a scale mixture of normal distributions with respect to an stable random variable a mixture of bartlett fejer kernels or triangle densities with respect to a two component mixture of gamma random variables both lead to markov chain monte carlo methods for posterior simulation and these methods turn out to have complementary domains of maximum efficiency the first representation is a well known result due to west and is the better choice for collinear design matrices the second representation is new and is more efficient for orthogonal problems largely because it avoids the need to deal with exponentially tilted stable random variables it also provides insight into the multimodality of the joint posterior distribution which is a feature of the bridge model that is notably absent under ridge or lasso type priors we prove a theorem that extends this representation to a wider class of densities representable as scale mixtures of beta distributions and we provide an explicit inversion formula for the mixing distribution the connections with slice sampling and scale mixtures of normal distributions are explored on the practical side we find that the bayesian bridge model outperforms its classical cousin in estimation and prediction across a variety of data sets both simulated and real we also show that the markov chain monte carlo algorithm for fitting the bridge model exhibits excellent mixing properties particularly for the global scale parameter this makes for a favourable contrast with analogous markov chain monte carlo algorithms for other sparse bayesian models all methods described in this paper are implemented in the r package bayesbridge an extensive set of simulation results is provided in two on line supplemental files

almost sure convergence of stochastic approximation algorithms with non additive noise abstract stochastic approximation algorithms with non additive noise are discussed in studying strong convergence of such algorithms traditionally one assumes that the iterates return to a bounded or compact set infinitely often or that the function under consideration grows with certain rate the usual projection algorithms require that the bounded projection region is known beforehand it is desirable to weaken these boundedness conditions by introducing randomly varying truncations chen and zhu 1986 achieved this for stochastic approximation algorithms with additive noise here we extend their result to a more general setting