estimating the proportion of true null hypotheses with application to dna microarray data summary we consider the problem of estimating the proportion of true null hypotheses 0 i n a multiple hypothesis set up the tests are based on observed p values we first review published estimators based on the estimator that was suggested by schweder and spjotvoll then we derive new estimators based on nonparametric maximum likelihood estimation of the p value density restricting to decreasing and convex decreasing densities the estimators of 0 are all derived under the assumption of independent test statistics their performance under dependence is investigated in a simulation study we find that the estimators are relatively robust with respect to the assumption of independence and work well also for test statistics with moderate dependence

enhanced false discovery rate using gaussian mixture models for thresholding fmri statistical maps a typical fmri data analysis proceeds via the generalized linear model glm with gaussian noise using a model based on the experimental paradigm this analysis ultimately results in the production of z statistic images corresponding to the contrasts of interest thresholding such z statistic images at uncorrected thresholds suitable for testing activation at a single voxel results in the problem of multiple comparisons a number of methods which account for the problem of multiple comparisons have been proposed including gaussian random field theory mixture modeling and false discovery rate fdr the focus of this paper is on the development of a generalized version of fdr gfdr in an empirical bayesian framework specially adapted for fmri thresholding that is more robust to modeling violations as compared to traditional fdr we show theoretically as well as by simulation that for real fmri data various factors lead to a mixture of gaussians mog density for the null distribution artificial data was used to systematically study the bias of fdr and gfdr under varying intensity of modeling violations signal to noise ratios and activation fractions for a range of q values gfdr was able to handle modeling violations and produce good results when fdr failed real fmri data was also used to confirm gfdr capabilities our results indicate that it is very important to account for the form and fraction of the null hypothesis adaptively from the data in order to obtain valid inference

multiple testing of general contrasts powerful improvements are possible for multiple testing procedures when the hypotheses are logically related closed testing with exhaustive tests provides a unifying framework for developing such procedures but can be computationally difficult and can be nonmonotonic in p values royen introduced a truncated closed testing method for the case of all pairwise comparisons in the analysis of variance that is monotonic in p values shaffer developed a similar truncated procedure for more general comparisons but using bonferroni tests rather than exhaustive tests and westfall extended shaffer s method to allow exhaustive tests this article extends royen s method to general contrasts and proves that it is equivalent to the extended shaffer procedure for k contrasts the method generally requires evaluation of o 2k critical values that correspond to subset intersection hypotheses and is computationally infeasible for large k the set of intersections is represented using a tree structure and a

group sequential enrichment design incorporating subgroup selection an important component of clinical trials in drug development is the analysis of treatment efficacy in patient subgroups subpopulations because of concerns of multiplicity and of the small sample sizes often involved such analyses can present substantial statistical challenges and may lead to misleading conclusions as a confirmatory seamless phase ii iii design we propose an adaptive enrichment group sequential procedure whereby resources can be concentrated on subgroups most likely to respond to treatment stopping boundaries are defined through upper and lower spending functions the procedure is presented in terms of the efficient score enabling the analysis of normal binary or time to event data it addresses the dilution effect by eliminating populations at the first stage that appear likely to derive no therapeutic benefit it subsequently proceeds with the definitive assessment of treatment efficacy among the remaining pooled populations using a group sequential design the procedure provides strong protection of familywise type i error rate and we employ a bootstrap algorithm to obtain point and interval estimates that are adjusted for the selection bias we give examples to demonstrate how the design is used we make comparisons with adaptive two stage combination test procedures and with a group sequential test that does not account for the presence of subgroups numerical results show that the procedure has high power to detect subgroup specific effects and the use of multiple interim analysis points can lead to substantial sample size savings

rumor processes in random environment on mathbb n and on galton watson trees the aim of this paper is to study rumor processes in random environment in a rumor process a signal starts from the stations of a fixed vertex the root and travels on a graph from vertex to vertex we consider two rumor processes in the firework process each station when reached by the signal transmits it up to a random distance in the reverse firework process on the other hand stations do not send any signal but they listen for it up to a random distance the first random environment that we consider is the deterministic 1 dimensional tree mathbb n with a random number of stations on each vertex in this case the root is the origin of mathbb n we give conditions for the survival extinction on almost every realization of the sequence of stations later on we study the processes on galton watson trees with random number of stations on each vertex we show that if the probability of survival is positive then there is survival on almost every realization of the infinite tree such that there is at least one station at the root we characterize the survival of the process in some cases and we give sufficient conditions for survival extinction

an improved bonferroni inequality and applications summary we present an improved bonferroni inequality which gives an upper bound for the probability of the union of an arbitrary sequence of events the bound is constructed in terms of the joint probability of pairs of events which are represented by edges on a graph examples of applications to periodicity location shift detection kolmogorovsmirnov tests and outlier detection are given

exact calculations for false discovery proportion with application to least favorable configurations in a context of multiple hypothesis testing we provide several new exact calculations related to the false discovery proportion fdp of step up and step down procedures for step up procedures we show that the number of erroneous rejections conditionally on the rejection number is simply a binomial variable which leads to explicit computations of the c d f the s th moment and the mean of the fdp the latter corresponding to the false discovery rate fdr for step down procedures we derive what is to our knowledge the first explicit formula for the fdr valid for any alternative c d f of the p values we also derive explicit computations of the power for both step up and step down procedures these formulas are explicit in the sense that they only involve the parameters of the model and the c d f of the order statistics of i i d uniform variables the p values are assumed either independent or coming from an equicorrelated multivariate normal model and an additional mixture model for the true false hypotheses is used this new approach is used to investigate new results which are of interest in their own right related to least most favorable configurations for the fdr and the variance of the fdp

method of transmitting or storing masked sub band coded audio signals in the transmission of audio signals the audio signal is digitally represented by use of quadrature mirror filtering in the form a plurality of spectral sub band signals the quantizing of the sample values in the sub bands e g 24 sub bands is controlled to the extent that the quantizing noise levels of the individual sub band signals are at approximately the same level difference from the masking threshold of the human auditory system resulting from the individual sub band signals the differences of the quantizing noise levels of the sub band signals with respect to the resulting masking threshold are set by the difference between the total information flow required for coding and the total information flow available for coding the available total information flow is set and may then fluctuate as a function of the signal

stepup procedures controlling generalized fwer and generalized fdr in many applications of multiple hypothesis testing where more than one false rejection can be tolerated procedures controlling error rates measuring at least k false rejections instead of at least one for some fixed k 1 can potentially increase the ability of a procedure to detect false null hypotheses the k fwer a generalized version of the usual familywise error rate fwer is such an error rate that has recently been introduced in the literature and procedures controlling it have been proposed a further generalization of a result on the k fwer is provided in this article in addition an alternative and less conservative notion of error rate the k fdr is introduced in the same spirit as the k fwer by generalizing the usual false discovery rate fdr a k fwer procedure is constructed given any set of increasing constants by utilizing the kth order joint null distributions of the p values without assuming any specific form of dependence among all the p values procedures controlling the k fdr are also developed by using the kth order joint null distributions of the p values first assuming that the sets of null and nonnull p values are mutually independent or they are jointly positively dependent in the sense of being multivariate totally positive of order two mtp2 and then discarding that assumption about the overall dependence among the p values

comparing connectomes across subjects and populations at different scales brain connectivity can be represented by a network that enables the comparison of the different patterns of structural and functional connectivity among individuals in the literature two levels of statistical analysis have been considered in comparing brain connectivity across groups and subjects 1 the global comparison where a single measure that summarizes the information of each brain is used in a statistical test 2 the local analysis where a single test is performed either for each node connection which implies a multiplicity correction or for each group of nodes connections where each subset is summarized by one single test in order to reduce the number of tests to avoid a penalizing multiplicity correction we comment on the different levels of analysis and present some methods that have been proposed at each scale we highlight as well the possible factors that could influence the statistical results and the questions that have to be addressed in such an analysis

comparing treatment variances in repeated measures bioavailability trials in repeated measures bioavailability bioequivalence studies involving two formulations the underlying measurements such as the area under the plasma concentration time curve auc and the maximum plasma concentration cmax typically have a block compound symmetric covariance matrix to test the equality of the variances in such a covariance matrix we derive in this paper an exact test and four asymptotic tests we compare these tests using simulation to determine those that have good performance in terms of controlling the type i error rate as well as providing good power we provide an example to demonstrate the proposed method copyright 1999 john wiley sons ltd

sequential selection procedures and false discovery rate control summary we consider a multiple hypothesis testing setting where the hypotheses are ordered and one is only permitted to reject an initial contiguous block of hypotheses a rejection rule in this setting amounts to a procedure for choosing the stopping point k this setting is inspired by the sequential nature of many model selection problems where choosing a stopping point or a model is equivalent to rejecting all hypotheses up to that point and none thereafter we propose two new testing procedures and prove that they control the false discovery rate in the ordered testing setting we also show how the methods can be applied to model selection by using recent results on p values in sequential model selection settings

an improved bonferroni procedure for multiple tests of significance summary a modification of the bonferroni procedure for testing multiple hypotheses is presented the method based on the ordered p values of the individual tests is less conservative than the classical bonferroni procedure but is still simple to apply a simulation study shows that the probability of a type i error of the procedure does not exceed the nominal significance level a for a variety of multivariate normal and multivariate gamma test statistics for independent tests the procedure has type i error probability equal to a the method appears particularly advantageous over the classical bonferroni procedure when several highly correlated test statistics are involved

strengthening some common multiple test procedures for discrete data this paper proposes modification of some commonly used multiple test procedures for testing problems that arise when the underlying distributions are discrete these procedures have been shown as conservative because the exact nominal levels are unattainable the amended procedures can achieve actual type i errors much closer to the nominal levels thus giving rise to more powerful tests

permutation based simultaneous confidence bounds for the false discovery proportion when multiple hypotheses are tested interest is often in ensuring that the proportion of false discoveries fdp is small with high confidence in this paper confidence upper bounds for the fdp are constructed which are simultaneous over all rejection cut offs in particular this allows the user to select a set of hypotheses post hoc such that the fdp lies below some constant with high confidence our method uses permutations to account for the dependence structure in the data so far only meinshausen provided an exact permutation based and computationally feasible method for simultaneous fdp bounds we provide an exact method which uniformly improves this procedure further we provide a generalization of this method it lets the user select the shape of the simultaneous confidence bounds this gives the user more freedom in determining the power properties of the method interestingly several existing permutation methods such as significance analysis of microarrays sam and westfall and young s maxt method are obtained as special cases

a class of improved hybrid hochberg hommel type step up multiple test procedures summary in this paper we derive a new p value based multiple testing procedure that improves upon the hommel procedure by gaining power as well as having a simpler step up structure similar to the hochberg procedure the key to this improvement is that the hommel procedure can be improved by a consonant procedure exact critical constants of this new procedure can be numerically determined the zeroth order approximations to the exact critical constants albeit slightly conservative are simple to use and need no tabling and hence are recommended in practice the proposed procedure is shown to control the familywise error rate under independence among the p values simulations empirically demonstrate familywise error rate control under positive and negative dependence power superiority of the proposed procedure over competing ones is also empirically demonstrated illustrative examples are given

strong control conservative point estimation and simultaneous conservative consistency of false discovery rates a unified approach summary the false discovery rate fdr is a multiple hypothesis testing quantity that describes the expected proportion of false positive results among all rejected null hypotheses benjamini and hochberg introduced this quantity and proved that a particular step up p value method controls the fdr storey introduced a point estimate of the fdr for fixed significance regions the former approach conservatively controls the fdr at a fixed predetermined level and the latter provides a conservatively biased estimate of the fdr for a fixed predetermined significance region in this work we show in both finite sample and asymptotic settings that the goals of the two approaches are essentially equivalent in particular the fdr point estimates can be used to define valid fdr controlling procedures in the asymptotic setting we also show that the point estimates can be used to estimate the fdr conservatively over all significance regions simultaneously which is equivalent to controlling the fdr at all levels simultaneously the main tool that we use is to translate existing fdr methods into procedures involving empirical processes this simplifies finite sample proofs provides a framework for asymptotic results and proves that these procedures are valid even under certain forms of dependence

the simes method for multiple hypothesis testing with positively dependent test statistics abstract the simes method for testing intersection of more than two hypotheses is known to control the probability of type i error only when the underlying test statistics are independent although this method is more powerful than the classical bonferroni method it is not known whether it is conservative when the test statistics are dependent this article proves that for multivariate distributions exhibiting a type of positive dependence that arise in many multiple hypothesis testing situations the simes method indeed controls the probability of type i error this extends some results established very recently in the special case of two hypotheses

an adaptive confirmatory trial with interim treatment selection practical experiences and unbalanced randomization this article has been motivated by an ongoing international adaptive confirmatory trial at the interim analysis of this two stage trial none one or two active treatment regimens are selected for further study in the second stage a combination test approach is used in this practical setting with an extension of the theory to unbalanced randomization we show that a combination test with suitable weights can still preserve the overall type i error rate provided that the test statistic is chosen appropriately and the unpooled z test for proportions is not used the accuracy of stagewise p values is also discussed in a more general framework monte carlo simulations confirm the validity of the approach retained and evaluate the necessary sample size additional issues addressed during the design of the trial are also examined such as multiplicity due to testing hypotheses on key secondary endpoints a non inferiority comparison to an active treatment and covariate adjusted analyses for various types of outcome

a stagewise rejective multiple test procedure based on a modified bonferroni test summary simes 1986 has proposed a modified bonferroni procedure for the test of an overall hypothesis which is the combination of n individual hypotheses in contrast to the classical bonferroni procedure it is not obvious how statements about individual hypotheses are to be made for this procedure in the present paper a multiple test procedure allowing statements on individual hypotheses is proposed it is based on the principle of closed test procedures marcus peritz gabriel 1976 and controls the multiple level a

robust asymptotic sampling theory for correlations in pedigrees methods to unravel the genetic determinants of non mendelian diseases lie at the next frontier of statistical approaches for human genetics it is generally agreed that before proceeding with segregation or linkage analysis the trait under study ought to be shown to exhibit familial correlation by coding dichotomous traits as binary variables a single robust approach in the estimation of pedigree correlations rather than two distinct approaches can be used to assess the potential heritability of a trait and latterly to examine the mode of inheritance the asymptotic theory to conduct hypothesis tests and confidence intervals for correlations among different members of nuclear families is well established but is applicable only if the nuclear families are independent as a further contribution to the literature we derive the asymptotic sampling distribution of correlations between random variables among arbitrary pairs of members in extended families for the pearson product moment estimator with generalized weights this derivation is done without assuming normality of the traits the sampling distribution is shown to be asymptotically normal to first order and hence large sample hypothesis tests and confidence intervals with estimates of the variances and correlation coefficients are proposed discussion concludes with an example and a suggestion for future research

efficiency and power of tests for multiple binary outcomes abstract global tests provide a useful tool for comparing two or more groups with respect to multiple correlated outcomes we adapt and compare the performance of tests that have been suggested for use with multiple continuous outcomes to the case of multiple binary outcomes comparisons and guidelines are based on asymptotic relative efficiencies are s and simulations these results are illustrated using an application from teratology we extend the work of lefkopoulou and ryan to include general m group comparisons alternatives where group effects may differ for each outcome a concise form for this general class of score tests is derived to compute the are s for this class of tests we devise a useful characterization of the alternative space based on multivariate polar coordinates our findings indicate that the common outcome effect tests are efficient for a remarkably large range of circumstances a simple formula applies to compute the maximum number of unaffected outcomes that can be included i

rumor processes on mathbb n and discrete renewal processes we study two rumor processes on mathbb n the dynamics of which are related to an si epidemic model with long range transmission both models start with one spreader at site 0 and ignorants at all the other sites of mathbb n but differ by the transmission mechanism in one model the spreaders transmit the information within a random distance on their right and in the other the ignorants take the information from a spreader within a random distance on their left we obtain the probability of survival information on the distribution of the range of the rumor and limit theorems for the proportion of spreaders the key step of our proofs is to show that in each model the position of the spreaders on mathbb n can be related to a suitably chosen discrete renewal process

a permutational step up method of testing multiple outcomes this paper describes a permutational step up multiple comparison procedure to adjust the p values from k related hypotheses the method is applicable when two groups of subjects are being compared on each of k outcomes it is related to the step down method of westfall and young 1993 resampling based multiple testing examples and methods for p value adjustment volume 1 and troendle 1995 journal of the american statistical association 90 370 378 and is an alternative to the analytic step up method of dunnett and tamhane 1992 journal of the american statistical association 87 162 170 which requires a specific distribution and correlation structure by conditioning on the data observed this permutational method avoids any specific distribution or correlation assumption it is shown very generally that the method asymptotically controls the familywise probability of a type i error

discovering findings that replicate from a primary study of high dimension to a follow up study we consider the problem of identifying whether findings replicate from one study of high dimension to another when the primary study guides the selection of hypotheses to be examined in the follow up study as well as when there is no division of roles into the primary and the follow up study we show that existing meta analysis methods are not appropriate for this problem and suggest novel methods instead we prove that our multiple testing procedures control for appropriate error rates the suggested family wise error rate controlling procedure is valid for arbitrary dependence among the test statistics within each study a more powerful procedure is suggested for false discovery rate fdr control we prove that this procedure controls the fdr if the test statistics are independent within the primary study and independent or have positive dependence in the follow up study for arbitrary dependence within the primary study and either arbitrary dependence or positive dependence in the follow up study

closure procedures for monotone bi factorial dose response designs les essais en doses multiples ont deux buts i demontrer une amelioration de l efficacite pour une combinaison donnee de chacun des composants ii identifier un intervalle de doses efficaces et sures les auteurs s interessent aux deux aspects quoique avec un interet plus marque pour le second avec une procedure de cloture qui garantit un controle fort du risque global d erreur on etudie deux familles differentes d hypotheses nulles pour des dispositifs bi factoriels monotones par rapport a un ordre partiel l une est adaptee a la recherche d une ou plusieurs doses efficaces minimums l autre est suffisamment large pour permettre d identifier les doses efficaces les plus elevees on applique des tests du rapport de vraisemblance et des contrastes multiples appropries a un exemple d essai clinique tire de hung 2000 statistics in medicine 19 2079 2087 le code ecrit en langage r est disponible dans sa totalite sur internet

simultaneous confidence intervals that are compatible with closed testing in adaptive designs we describe a general method for finding a confidence region for a parameter vector that is compatible with the decisions of a two stage closed test procedure in an adaptive experiment the closed test procedure is characterized by the fact that rejection or nonrejection of a null hypothesis may depend on the decisions for other hypotheses and the compatible confidence region will in general have a complex nonrectangular shape we find the smallest cross product of simultaneous confidence intervals containing the region and provide computational shortcuts for calculating the lower bounds on parameters corresponding to the rejected null hypotheses we illustrate the method with an adaptive phase ii iii clinical trial

on the performance of fdr control constraints and a partial solution the false discovery rate fdr paradigm aims to attain certain control on type i errors with relatively high power for multiple hypothesis testing the benjamini hochberg bh procedure is a wellknown fdr controlling procedure under a random effects model we show that in general unlike the fdr the positive fdr pfdr of the bh procedure cannot be controlled at an arbitrarily low level due to the limited evidence provided by the observations to separate false and true nulls this results in a criticality phenomenon which is characterized by a transition of the procedure s power from being positive to asymptotically 0 without any reduction in the pfdr once the target fdr control level is below a positive critical value to address the constraints on the power and pfdr control imposed by the criticality phenomenon we propose a procedure which applies bh type procedures at multiple locations in the domain of p values both analysis and simulations show that the proposed procedure can attain substantially improved power and pfdr control

on weighted hochberg procedures summary we consider different ways of constructing weighted hochberg type step up multiple test procedures including closed procedures based on weighted simes tests and their conservative step up short cuts and step up counterparts of two weighted holm procedures it is shown that the step up counterparts have some serious pitfalls such as lack of familywise error rate control and lack of monotonicity in rejection decisions in terms of p values therefore an exact closed procedure appears to be the best alternative its only drawback being lack of simple stepwise structure a conservative step up short cut to the closed procedure may be used instead but with accompanying loss of power simulations are used to study the familywise error rate and power properties of the competing procedures for independent and correlated p values although many of the results of this paper are negative they are useful in highlighting the need for caution when procedures with similar pitfalls may be used

multivariate and multiple permutation tests abstract in this article we consider the use of permutation tests for comparing multivariate parameters from two populations first the underlying properties of permutation tests when comparing parameter vectors from two distributions p and q are developed although an exact level test can be constructed by a permutation test when the fundamental assumption of identical underlying distributions holds permutation tests have often been misused indeed permutation tests have frequently been applied in cases where the underlying distributions need not be identical under the null hypothesis in such cases permutation tests fail to control the type 1 error even asymptotically however we provide valid procedures in the sense that even when the assumption of identical distributions fails one can establish the asymptotic validity of permutation tests in general while retaining the exactness property when all the observations are i i d in the multivariate testing problem for testing the global null hypothesis of equality of parameter vectors a modified hotelling s t 2 statistic as well as tests based on the maximum of studentized absolute differences are considered in the latter case a bootstrap prepivoting test statistic is constructed which leads to a bootstrapping after permuting algorithm then these tests are applied as a basis for testing multiple hypotheses simultaneously by invoking the closure method to control the familywise error rate lastly monte carlo simulation studies and an empirical example are presented

accurate p values for adaptive designs with binary endpoints adaptive designs encompass all trials allowing various types of design modifications over the course of the trial a key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family wise error rate this can be achieved by combining the p values for each arm and stage to account for adaptations including but not limited to treatment selection sample size adaptation and multiple stages while the theory for this is novel and well established in practice these methods can perform poorly especially for unbalanced designs and for small to moderate sample sizes the problem is that standard stagewise tests have inflated type i error rate especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests seriously inflating the family wise error rate we propose to fix this problem by feeding the adaptive test with second order accurate p values in particular bootstrap p values secondly an adjusted version of the simes procedure for testing intersection hypotheses that reduces the built in conservatism is suggested numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate at or below the nominal level across the board irrespective of the baseline rate stagewise sample sizes or allocation ratio copyright 2017 john wiley sons ltd

optimizing the order of hypotheses in serial testing of multiple endpoints in clinical trials clinical trials usually collect information on a large number of variables or endpoints including one or more primary endpoints as well as a number of secondary endpoints representing different aspects of treatment effectiveness and safety in this article we focus on serial testing procedures that test multiple endpoints in a pre specified order and consider how to optimize the order of endpoints subject to any clinical constraints with respect to the expected number of successes i e endpoints that reach statistical significance or the expected gain if endpoints are associated with numerical utilities we consider some common approaches to this problem and propose two new approaches a greedy algorithm based on conditional power and a simulated annealing algorithm that attempts to improve a given sequence in a random and iterative fashion simulation results indicate that the proposed algorithms are useful for finding a high performing sequence and that optimized fixed sequence procedures can be competitive against traditional multiple testing procedures such as holm s the methods and findings are illustrated with two examples concerning migraine and asthma

multiple hypotheses testing and expected number of type i errors the performance of multiple test procedures with respect to error control is an old issue assuming that all hypotheses are true we investigate the behavior of the expected number of type i errors ene as a characteristic of certain multiple tests controlling the familywise error rate fwer or the false discovery rate fdr at a prespecified level we derive explicit formulas for the distribution of the number of false rejections as well as for the ene for single step step down and step up procedures based on independent p values moreover we determine the corresponding asymptotic distributions of the number of false rejections as well as explicit formulae for the ene if the number of hypotheses tends to infinity in case of fwer control we mostly obtain poisson distributions and in one case a geometric distribution as limiting distributions in case of fdr control we obtain limiting distributions which are apparently not named in the literature surprisingly the ene is bounded by a small number regardless of the number of hypotheses under consideration finally it turns out that in case of dependent test statistics the ene behaves completely differently compared to the case of independent test statistics

a general multistage procedure for k out of n gatekeeping we generalize a multistage procedure for parallel gatekeeping to what we refer to as k out of n gatekeeping in which at least k out of n hypotheses 1 k n in a gatekeeper family must be rejected in order to test the hypotheses in the following family this gatekeeping restriction arises in certain types of clinical trials for example in rheumatoid arthritis trials it is required that efficacy be shown on at least three of the four primary endpoints we provide a unified theory of multistage procedures for arbitrary k with k 1 corresponding to parallel gatekeeping and k n to serial gatekeeping the theory provides an insight into the construction of truncated separable multistage procedures using the closure method explicit formulae for calculating the adjusted p values are given the proposed procedure is simpler to apply for this particular problem using a stepwise algorithm than the mixture procedure and the graphical procedure with memory using entangled graphs

a conditional error function approach for subgroup selection in adaptive clinical trials growing interest in personalised medicine and targeted therapies is leading to an increase in the importance of subgroup analyses if it is planned to view treatment comparisons in both a predefined subgroup and the full population as co primary analyses it is important that the statistical analysis controls the familywise type i error rate spiessens and debois cont clin trials 2010 31 647 656 recently proposed an approach specific for this setting which incorporates an assumption about the correlation based on the known sizes of the different groups and showed that this is more powerful than generic multiple comparisons procedures such as the bonferroni correction if recruitment is slow relative to the length of time taken to observe the outcome it may be efficient to conduct an interim analysis in this paper we propose a new method for an adaptive clinical trial with co primary analyses in a predefined subgroup and the full population based on the conditional error function principle the methodology is generic in that we assume test statistics can be taken to be normally distributed rather than making any specific distributional assumptions about individual patient data in a simulation study we demonstrate that the new method is more powerful than previously suggested analysis strategies furthermore we show how the method can be extended to situations when the selection is not based on the final but on an early outcome we use a case study in a targeted therapy in oncology to illustrate the use of the proposed methodology with non normal outcomes copyright 2012 john wiley sons ltd

the sequential rejection principle of familywise error control closed testing and partitioning are recognized as fundamental principles of familywise error control in this paper we argue that sequential rejection can be considered equally fundamental as a general principle of multiple testing we present a general sequentially rejective multiple testing procedure and show that many well known familywise error controlling methods can be constructed as special cases of this procedure among which are the procedures of holm shaffer and hochberg parallel and serial gatekeeping procedures modern procedures for multiple testing in graphs resampling based multiple testing procedures and even the closed testing and partitioning procedures themselves we also give a general proof that sequentially rejective multiple testing procedures strongly control the familywise error if they fulfill simple criteria of monotonicity of the critical values and a limited form of weak familywise error control in each single step the sequential rejection principle gives a novel theoretical perspective on many well known multiple testing procedures emphasizing the sequential aspect its main practical usefulness is for the development of multiple testing procedures for null hypotheses possibly logically related that are structured in a graph we illustrate this by presenting a uniform improvement of a recently published procedure

α investing a procedure for sequential control of expected false discoveries summary investing is an adaptive sequential methodology that encompasses a large family of procedures for testing multiple hypotheses all control mfdr which is the ratio of the expected number of false rejections to the expected number of rejections mfdr is a weaker criterion than the false discovery rate which is the expected value of the ratio we compensate for this weakness by showing that investing controls mfdr at every rejected hypothesis investing resembles spending that is used in sequential trials but it has a key difference when a test rejects a null hypothesis investing earns additional probability towards subsequent tests investing hence allows us to incorporate domain knowledge into the testing procedure and to improve the power of the tests in this way investing enables the statistician to design a testing procedure for a specific problem while guaranteeing control of mfdr

a comparison of two modified bonferroni procedures summary it is shown that hommel s 1988 modified bonferroni procedure is at least as powerful as hochberg s 1988 procedure and in general more powerful

stepwise multiple test procedures and control of directional errors one of the most difficult problems occurring with stepwise multiple test procedures for a set of two sided hypotheses is the control of directional errors if rejection of a hypothesis is accomplished with a directional decision in this paper we generalize a result for so called step down procedures derived by shaffer to a large class of stepwise or closed multiple test procedures in a unifying way we obtain results for a large class of order statistics procedures including step down as well as step up procedures hochberg rom but also a procedure of hommel based on critical values derived by simes our method of proof is also applicable in situations where directional decisions are mainly based on conditionally independent t statistics a closed f test procedure applicable in regression models with orthogonal design the modified s method of scheffe applicable in the analysis of variance and fisher s lsd test for the comparison of three means will be considered in more detail

on rumour propagation among sceptics junior et al j appl probab 48 624 636 2011 studied a model to understand the spread of a rumour their model consists of individuals situated at the integer points of the line mathbb n an individual at the origin 0 starts a rumour and passes it to all individuals in the interval 0 r 0 where r 0 is a non negative random variable an individual located at i in this interval receives the rumour and transmits it further among individuals in i i r i where r 0 and r i are i i d random variables the rumour spreads in this manner an alternate model considers individuals seeking to find the rumour from individuals who have already heard it for this s he asks individuals to the left of her him and lying in an interval of a random size we study these two models when the individuals are more sceptical and they transmit or accept the rumour only if they receive it from at least two different sources in stochastic geometry the equivalent of this rumour process is the study of coverage of the space mathbb n d by random sets our study here extends the study of coverage of space and considers the case when each vertex of mathbb n d is covered by at least two distinct random sets

a conservative test for multiple comparison based on highly correlated test statistics in genetics we often encounter a large number of highly correlated test statistics the most famous conservative bound for multiple comparison is bonferroni s bound which is suitable when the test statistics are independent but not when the test statistics are highly correlated this article proposes a new conservative bound that is easily calculated without multiple integration and is a good approximation when the test statistics are highly correlated the performance of the proposed method is evaluated by simulation and real data analysis

a comparison of one sided methods to identify significant individual outcomes in a multiple outcome setting stepwise tests or global tests with closed testing we compare two approaches to the identification of individual significant outcomes when a comparison of two groups involves multiple outcome variables the approaches are all designed to control the familywise error rate fwe with any subset of the null hypotheses being true in the strong sense the first approach is initially to use a global test of the overall hypothesis that the groups are equivalent for all variables followed by an application of the closed testing algorithm of marcus peritz and gabriel the global tests considered here are ordinary least squares ols generalized least squares gls an approximation to a likelihood ratio test alr and a new test based on an approximation to the most powerful similar test for simple alternatives the second approach is that of stepwise testing which tests the univariate hypotheses in a specific order with appropriate adjustment to the univariate p values for multiplicity the stepwise tests considered include both step down and step up tests of a general type and likewise permutation tests that incorporate the dependence structure of the data we illustrate the tests with two examples of birth outcomes a comparison of cocaine exposed new borns to control new borns on neurobehavioural and physical growth variables and in a separate study a comparison of babies born to diabetic mothers and babies born to non diabetic mothers on minor malformations after describing the methods and analysing the birth outcome data we use simulations on gaussian data to provide guidelines for the use of these procedures in terms of power and computation

truncated simes tests it is known that the one sided simes test controls the error rate if the underlying distribution is multivariate totally positive of order 2 mtp2 but not in general the two sided test also controls the error rate when the coordinate absolute value has an mtp2 distribution which holds more generally we prove mathematically that when the coordinate absolute value controls the error rate at level 2 then certain kinds of truncated simes tests also control the one sided error rate at level we also compare the closure of the truncated tests with the holms hochberg and hommel procedures in many scenarios when the test statistics are multivariate normal

a linear non gaussian acyclic model for causal discovery in recent years several methods have been proposed for the discovery of causal structure from non experimental data such methods make various assumptions on the data generating process to facilitate its identification from purely observational data continuing this line of research we show how to discover the complete causal structure of continuous valued data under the assumptions that a the data generating process is linear b there are no unobserved confounders and c disturbance variables have non gaussian distributions of non zero variances the solution relies on the use of the statistical method known as independent component analysis and does not require any pre specified time ordering of the variables we provide a complete matlab package for performing this lingam analysis short for linear non gaussian acyclic model and demonstrate the effectiveness of the method using artificially generated data and real world data

a uniform improvement of bonferroni type tests by sequential tests we propose a new method to uniformly improve cutoff tests as e g the bonferroni test by replacing them with a sequential testing procedure the proposed procedure provides a uniform improvement that is it rejects the null hypothesis at least in all cases where the original procedure does has level and never requires a sample size larger than that of the original procedure information on the often unknown joint distribution of the test statistics across the hypotheses is not required we discuss the applications to one and two sided tests for the intersection of multiple hypotheses as well as to tests of elementary hypotheses for the wide class of asymptotically linear test statistics and under unknown nuisance parameters we propose an asymptotic approximation to our improved procedure and show that it exhausts the nominal type i error rate asymptotically we present an application in a clinical study with multiple endpoints where the null hypothesis of no effect is tested against the altern

validity of the hochberg procedure revisited for clinical trial applications there is much interest in using the hochberg procedure hp for statistical tests on primary endpoints of confirmatory clinical trials the procedure is simple to use and enjoys more power than the bonferroni and the holm procedures however the hp is not assumption free like the other two procedures it controls the familywise type i error rate when test statistics used for statistical tests are independent or if dependent satisfy a conditionally independent formulation otherwise its properties for dependent tests at present are not fully understood consequently its use for confirmatory trials especially for their primary endpoints remains worrisome confirmatory trials are typically designed with 1 2 primary endpoints therefore a question was raised at the food and drug administration as to whether the hp is a valid test for the simple case of performing treatment to control comparisons on two primary endpoints when their test statistics are not independent confirmatory trials for statistical tests normally use simple test statistics such as the normal z student s t and chi square the literature does include some work on the hp for dependent cases covering these test statistics but concerns remain regarding its use for confirmatory trials for which endpoint tests are mostly of the dependent kind the purpose of this paper is therefore to revisit this procedure and provide sufficient details for better understanding of its performance for dependent cases related to the aforementioned question published 2015 this article is a u s government work and is in the public domain in the usa

multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active treatment frequently multiple doses of an active treatment and multiple endpoints are simultaneously considered in the designs of clinical trials for these trials traditional multiplicity adjustment procedures such as bonferroni hochberg and hommel procedures can be applied when treating the comparisons of different doses to the control on all endpoints at the same level however these approaches will not take into account the possible dose response relationship on each endpoint and therefore are less specific and may have lower power to gain power in this paper we consider the problem as a two dimensional multiplicity problem one dimension concerns the multiple doses and the other dimension concerns the multiple endpoints we propose procedures which consider the dose order to form the closure of the procedures and control the family wise type i error rate in a strong sense for this two dimensional problem numerical examples show that procedures proposed in this paper in general have higher power than the commonly used procedures e g the regular hochberg procedure especially for comparing the higher dose to the control

step up multiple testing of parameters with unequally correlated estimates we consider the problem of simultaneously testing k or to 2 hypotheses on parameters theta 1 theta k using test statistics t 1 t k such that a specified familywise error rate alpha is achieved dunnett and tamhane 1992a proposed a step up multiple test procedure in which testing starts with the hypothesis corresponding to the least significant test statistic and proceeds towards the most significant stopping the first time a significant test result is obtained and rejecting the hypotheses corresponding to that and any remaining test statistics the parameter estimates used in the t statistics were assumed to be normally distributed with a common variance which was a known multiple of an unknown sigma 2 and known correlations which were equal in the present article we show how the procedure can be extended to include unequally correlated parameter estimates unequal correlations occur for example in experiments involving comparisons among treatment groups with unequal sample sizes we also compare the step up and step down multiple testing approaches and discuss applications to some biopharmaceutical testing problems

system for subband coding of a digital audio signal a system for subband coding of a digital audio signal x k includes in the coder 1 a filter bank 3 for splitting the audio signal band with sampling rate reduction into subbands p 1 p of approximately critical bandwidth and in the decoder 2 a filter bank 5 for merging these subbands with sampling rate increase for each subband p the coder 1 comprises a detector 7 p for determining a parameter g p m representative of the signal level in a block p m of m samples of the subband signal xp k as well as a quantizer 8 p for adaptively block quantizing this subband signal in response to parameter g p m and the decoder 2 comprises a dequantizer 9 p for adaptively block dequantizing the quantized subband signal sp k in response to parameter g p m the quantizing characteristics are related to the noise masking curve of the human auditory system owing to which a high quality of the replica x k of audio signal x k is attained with an average number of approximately 2 5 bits per sample for representing the output signals of the coder 1 the occasional audibility of quantizing noise in this replica x k is reduced effectively in that the coder 1 and decoder 2 contain identical bit allocation means 23 24 responsive to a set of parameters g p m for the higher group of subbands pim p p within an allocation window fig 5 for allocating a number of b p m bits per sample from a fixed predetermined number of b bits for this allocation window to the quantizer 8 p and the dequantizer 9 p for the block p m of subband signal xp k and sp k respectively

the control of the false discovery rate in multiple testing under dependency benjamini and hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems a simple procedure was given there as an fdr controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate we prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses this condition for positive dependency is general enough to cover many problems of practical interest including the comparisons of many treatments with a single control multivariate normal test statistics with positive correlation matrix and multivariate t furthermore the test statistics may be discrete and the tested hypotheses composite without posing special difficulties for all other forms of dependency a simple conservative modification of the procedure controls the false discovery rate thus the range of problems for which a procedure with proven fdr control can be offered is greatly increased 1 1 simultaneous hypotheses testing the control of the increased type i error when testing simultaneously a family of hypotheses is a central issue in the area of multiple comparisons rarely are we interested only in whether all hypotheses are jointly true or not which is the test of the intersection null hypothesis in most applications we infer about the individual hypotheses realizing that some of the tested hypotheses are usually true we hope not all and some are not we wish to decide which ones are not true indicating statistical discoveries an important such problem is that of multiple endpoints in a clinical trial a new treatment is compared with an existing one in terms of a large number of potential benefits endpoints

exact distribution of a maximally selected wilcoxon and a new hybrid test of symmetry recently a maximally selected normalized wilcoxon whose asymptotic distribution is a brownian bridge was proposed for testing symmetry of a distribution about zero the test sequentially discards observations whose absolute value is below increasing thresholds the wilcoxon is obtained at each threshold and the maximum is the test statistic we develop a recursive function for the exact distribution of a modification of the max wilcoxon test mw and provide critical values and a program for computing the p value for a sample a new hybrid test that combines the sign and mw tests is introduced the power of mw and the new hybrid test are compared with modarres and gastwirth s hybrid test mgh and the max mcnemar mm under the generalized lambda distributions gld family and two normal mixture models the mw and the new hybrid test outperform the mgh which is superior to the mm test in the gld family in one mixture model mm is the least powerful test and the remaining three are essentially equivalent in the second mixture model when the zero median assumption is nearly valid the mw test does well its performance degrades when this assumption is violated in the latter case the mm performs better than mw for the same degree of skewness because the mm simultaneously tests both symmetry and zero median data from a genetic study of monozygotic twins discordant for major depressive disorder is used to illustrate the new tests copyright 2014 john wiley sons ltd

applying the multivariate time rescaling theorem to neural population models statistical models of neural activity are integral to modern neuroscience recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing however any statistical model must be validated by an appropriate goodness of fit test kolmogorov smirnov tests based on the time rescaling theorem have proven to be useful for evaluating point process based statistical models of single neuron spike trains here we discuss the extension of the time rescaling theorem to the multivariate neural population case we show that even in the presence of strong correlations between spike trains models that neglect couplings between neurons can be erroneously passed by the univariate time rescaling test we present the multivariate version of the time rescaling theorem and provide a practical step by step procedure for applying it to testing the sufficiency of neural population models using several simple analytically tractable models and more complex simulated and real data sets we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test

hochberg s step up method cutting corners off holm s step down method holm s method and hochberg s method for multiple testing can be viewed as step down and step up versions of the bonferroni test we show that both are special cases of partition testing the difference is that while holm s method tests each partition hypothesis using the largest order statistic setting a critical value based on the bonferroni inequality hochberg s method tests each partition hypothesis using all the order statistics setting a series of critical values based on simes inequality geometrically hochberg s step up method cuts corners off the acceptance regions of holm s step down method by making assumptions on the joint distribution of the test statistics as can be expected partition testing making use of the joint distribution of the test statistics is more powerful than partition testing using probabilistic inequalities thus if the joint distribution of the test statistics is available through modelling for example we recommend partition step down testing setting exact critical values based on the joint distribution

hommel s bonferroni type inequality for unequally spaced levels summary hommel 1983 established a sharp upper bound to the overall significance level for a generalization of the bonferroni inequality based upon equally spaced critical levels for the successive ordered p values this note generalizes the sharp upper bound to unequally spaced critical levels

false discovery and false nondiscovery rates in single step multiple testing procedures results on the false discovery rate fdr and the false nondiscovery rate fnr are developed for single step multiple testing procedures in addition to verifying desirable properties of fdr and fnr as measures of error rates these results extend previously known results providing further insights particularly under dependence into the notions of fdr and fnr and related measures first considering fixed configurations of true and false null hypotheses inequalities are obtained to explain how an fdr or fnr controlling single step procedure such as a bonferroni or sidak procedure can potentially be improved two families of procedures are then constructed one that modifies the fdr controlling and the other that modifies the fnr controlling sidak procedure these are proved to control fdr or fnr under independence less conservatively than the corresponding families that modify the fdr or fnr controlling bonferroni procedure results of numerical investigations of the performance of the modified sidak fdr procedure over its competitors are presented second considering a mixture model where different configurations of true and false null hypotheses are assumed to have certain probabilities results are also derived that extend some of storey s work to the dependence case

system for subband coding of a digital audio signal and coder and decoder constituting the same a system for subband coding of a digital audio signal x k includes in the coder 1 a filter bank 3 for splitting the audio signal band with sampling rate reduction into subbands p 1 p of approximately critical bandwidth and in the decoder 2 a filter bank 5 for merging these subbands with sampling rate increase for each subband p the coder 1 comprises a detector 7 p for determining a parameter g p m representative of the signal level in a block p m of m samples of the subband signal xp k as well as a quantizer 8 p for adaptively block quantizing this subband signal in response to parameter g p m and the decoder 2 comprises a dequantizer 9 p for adaptively block dequantizing the quantized subband signal sp k in response to parameter g p m the quantizing characteristics are related to the noise masking curve of the human auditory system owing to which a high quality of the replica x k of audio signal x k is attained with an average number of approximately 2 5 bits per sample for representing the output signals of the coder 1 the occasional audibility of quantizing noise in this replica x k is reduced effectively in that the coder 1 and decoder 2 contain identical bit allocation means 23 24 responsive to a set of parameters g p m for the higher group of subbands pim p p within an allocation window fig 5 for allocating a number of b p m bits per sample from a fixed predetermined number of b bits for this allocation window to the quantizer 8 p and the dequantizer 9 p for the block p m of subband signal xp k and sp k respectively

a sequential algorithm for false discovery rate control on directed acyclic graphs summary we propose a linear time single pass top down algorithm for multiple testing on directed acyclic graphs where nodes represent hypotheses and edges specify a partial ordering in which the hypotheses must be tested the procedure is guaranteed to reject a sub directed acyclic graph with bounded false discovery rate while satisfying the logical constraint that a rejected node s parents must also be rejected it is designed for sequential testing settings where the directed acyclic graph structure is known a priori but the p values are obtained selectively such as in a sequence of experiments however the algorithm is also applicable in nonsequential settings where all p values can be calculated in advance such as in model selection our algorithm provably controls the false discovery rate under independence positive dependence or arbitrary dependence of the p values and specializes to known algorithms in the special cases of trees and line graphs it simplifies to the classical benjamini hochberg procedure when the directed acyclic graph has no edges we explore the empirical performance of our algorithm through simulations and analysis of a real dataset corresponding to a gene ontology and we demonstrate its favourable performance in terms of computational time and power

screening for partial conjunction hypotheses summary we consider the problem of testing for partial conjunction of hypothesis which argues that at least u out of n tested hypotheses are false it offers an in between approach to the testing of the conjunction of null hypotheses against the alternative that at least one is not and the testing of the disjunction of null hypotheses against the alternative that all hypotheses are not null we suggest powerful test statistics for testing such a partial conjunction hypothesis that are valid under dependence between the test statistics as well as under independence we then address the problem of testing many partial conjunction hypotheses simultaneously using the false discovery rate fdr approach we prove that if the fdr controlling procedure in benjamini and hochberg 1995 journal of the royal statistical society series b 57 289 300 is used for this purpose the fdr is controlled under various dependency structures moreover we can screen at all levels simultaneously in order to display the findings on a superimposed map and still control an appropriate fdr measure we apply the method to examples from microarray analysis and functional magnetic resonance imaging fmri two application areas where the need for partial conjunction analysis has been identified

correction of the p value after multiple coding of an explanatory variable in logistic regression we propose a method and a program to determine a significance level for a series of codings of an explanatory variable in logistic regression dichotomous and box cox transformations are considered three methods of correcting the significance level are studied the bonferroni method efron s method which uses the correlation between successive tests and the exact calculation by numerical integration using all correlations a simulation study has led to a strategy for the choice and number of the different codings of the variable this method is illustrated using the data of a study of the relation between cholesterol and dementia

a stepwise resampling method of multiple hypothesis testing abstract this article introduces a method of multiple hypothesis testing that combines the idea of sequential multiple testing procedures with the structure of resampling methods the method can be seen as an alternative to the analytic method of dunnett and tamhane which requires a specific distributional form resampling incorporates the covariance structure of the data without the need for distributional assumptions recent work by westfall and young has shown that a step down resampling method is asymptotically consistent when adjusted p values can be obtained exactly for continuous data this article shows that in the case of a comparison of two groups on multiple outcomes those results are generalizable to discrete data where exact adjusted p values are not available it is shown that the method asymptotically attains the desired level for controlling the experimentwise probability of a type i error

a sequentially rejective test procedure based on a modified bonferroni inequality summary a sharper bonferroni procedure for multiple tests of significance is derived this procedure is an improvement of hochberg s 1988 procedure which contrasts the individual p values with corresponding critical points it is shown that hochberg s original procedure is conservative and can be made more powerful by enlarging the rejection region so that the type one error is exactly at the nominal level it is also shown that the modified procedure retains all the desired properties of the original procedure

audio signal subband encoder there is provided an audio signal subband encoder which attains improved quality of sound by controlling allocation of the number of quantization bits to subband signals on the basis of an evaluation function combining the noise to mask ratio criterion with noise to signal ratio criterion in order to lessen the influence of the application error of the auditory masking rule such as the aliasing distortion of subband filters

approximate multinormal probabilities applied to correlated multiple endpoints in clinical trials clinical trials with multiple endpoints incur increased familywise type i errors the bonferroni correction is a common method used to modify the p values to account for multiple significance testing for independent endpoints the bonferroni method is slightly conservative whereas with high correlation the conservatism is extreme as demonstrated by pocock et al this paper presents a procedure which allows for the correlation present whilst adjusting the multiple p values the method is based on an approximation derived for multinormal probabilities

tests for precision and accuracy of multiple measuring devices we propose a multivariate linear model for analyzing measurement device data that follow grubbs s model the multivariate model generates the well known regression multiple correlation test for equality of device variances it also yields a new simultaneous test for equality of variances and biases the multivariate model readily permits testing of interesting subhypotheses involving the variances and biases we examine the power of these tests for onedimensional hypotheses compare them to tests proposed by grubbs and propose generalizations of grubbs s tests we also consider extensions of grubbs s model that allow similar regression and simultaneous tests to be performed

is the simes improved bonferroni procedure conservative summary simes 1986 proposed a modified bonferroni procedure for conducting multiple tests of significance he proved that when the n test statistics are independent his procedure has exact size a already hommel 1983 had considered a similar procedure and shown that it need not be conservative he gave an attainable upper bound for its anticonservativeness here we show that for one sided tests based on positively correlated bivariate normal test statistics the procedure is conservative and for negatively correlated such variables it is anticonservative two sided tests based on bivariate normal variables are always conservative some numerical values are given which show that for all practical purposes the nominal a value can be used

effect modification and design sensitivity in observational studies in an observational study of treatment effects subjects are not randomly assigned to treatment or control so differing outcomes in treated and control groups may reflect a bias from nonrandom assignment rather than a treatment effect after adjusting for measured pretreatment covariates perhaps by matching a sensitivity analysis determines the magnitude of bias from an unmeasured covariate that would need to be present to alter the conclusions of the naive analysis that presumes adjustments eliminated all bias other things being equal larger effects tend to be less sensitive to bias than smaller effects effect modification is an interaction between a treatment and a pretreatment covariate controlled by matching so that the treatment effect is larger at some values of the covariate than at others in the presence of effect modification it is possible that results are less sensitive to bias in subgroups experiencing larger effects two cases are considered i an a priori grouping into a few categ

one sided significance tests for generalized linear models under dichotomous response summary dichotomous response models are common in many experimental settings often concomitant explanatory variables are recorded and a generalized linear model such as a logit model is fit in some cases interest in specific model parameters is directed only at one sided departures from some null effect in these cases procedures can be developed for testing the null effect against a one sided alternative these include bonferroni type adjustments of univariate wald tests and likelihood ratio tests that employ inequality constrained multivariate theory this paper examines such tests of significance monte carlo evaluations are undertaken to examine the small sample properties of the various procedures the procedures are seen to perform fairly well generally achieving their nominal sizes at total sample sizes near 100 experimental units extensions to the problem of one sided tests against a control or standard are also considered

multiple tests of a non hierarchical finite family of hypotheses this paper considers the problem of simultaneously testing a non hierarchical finite family of hypotheses several definitions have been introduced they are used to shed interesting light on the interrelationship between the four types of test that have previously been proposed