a modified bonferroni method for discrete data the bonferroni adjustment for multiple comparisons is a simple and useful method of controlling the overall false positive error rate when several significance tests are performed in the evaluation of an experiment in situations with categorical data the test statistics have discrete distributions the discreteness of the null distributions can be exploited to reduce the number of significance tests taken into account in the bonferroni procedure this reduction is accomplished by using only the information contained in the marginal totals

exact significance levels for multiple binomial testing with application to carcinogenicity screens a simple experimental design consisting of one control group and one or more treatment groups is considered relevant research often focuses on the presence or absence of any of several characteristics in the treatment group s the statistical analysis frequently includes the comparison of the control group with each treatment group by the use of fisher irwin exact tests for each of many 2 x 2 tables the multiplicity of comparisons has given rise to concern that individual fisher irwin tests could seriously overstate the experimental evidence in some situations this paper provides a method for calculating the exact permutational probability of at least one significant fisher irwin test when only one treatment group and one control group is used for multiple treatment group designs upper and lower bounds on the probability are provided emphasis is given throughout to carcinogenesis screening experiments and an example of such an experiment is provided

a sharper bonferroni procedure for multiple tests of significance summary a simple procedure for multiple tests of significance based on individual p values is derived this simple procedure is sharper than holm s 1979 sequentially rejective procedure both procedures contrast the ordered p values with the same set of critical values holm s procedure rejects an hypothesis only if its p value and each of the smaller p values are less than their corresponding critical values the new procedure rejects all hypotheses with smaller or equal p values to that of any one found less than its critical value

statistical power of multiplicity adjustment strategies for correlated binary endpoints there are numerous alternatives to the so called bonferroni adjustment to control for familywise type i error among multiple tests yet for the most part these approaches disregard the correlation among endpoints this can prove to be a conservative hypothesis testing strategy if the null hypothesis is false the james procedure was proposed to account for the correlation structure among multiple continuous endpoints here a simulation study evaluates the statistical power of the hochberg and james adjustment strategies relative to that of the bonferroni approach when used for multiple correlated binary variables the simulations demonstrate that relative to the bonferroni approach neither alternative sacrifices power the hochberg approach has more statistical power for 0 50 whereas the james procedure provides more statistical power with higher the common correlation among the multiple outcomes a study of gender differences in new york city homicides is used to illustrate the approaches copyright 2007 john wiley sons ltd

sample size calculation for planning group sequential longitudinal trials procedures are developed in this paper for sample size calculations for planning a group sequential longitudinal trial with various correlation structures using a test statistic based on generalized estimating equations lee kim and tsiatis and group sequential boundaries based on type i error spending functions lan and demets

on fuzzy familywise error rate and false discovery rate procedures for discrete distributions fuzzy multiple comparisons procedures are introduced as a solution to the problem of multiple comparisons for discrete test statistics the critical function of the randomized p values is proposed as a measure of evidence against the null hypotheses the classical concept of randomized tests is extended to multiple comparisons this approach makes all theory of multiple comparisons developed for continuously distributed statistics automatically applicable to the discrete case examples of familywise error rate and false discovery rate procedures are discussed and an application to linkage disequilibrium testing is given software for implementing the procedures is available

strengthening some common multiple test procedures for discrete data this paper proposes modification of some commonly used multiple test procedures for testing problems that arise when the underlying distributions are discrete these procedures have been shown as conservative because the exact nominal levels are unattainable the amended procedures can achieve actual type i errors much closer to the nominal levels thus giving rise to more powerful tests

a note on approximation schemes for multidimensional knapsack problems polynomial and fully polynomial approximation algorithms for single dimensional knapsack problems have been extensively studied and a number of such algorithms constructed this note shows that the problem of finding a fully polynomial approximation algorithm for multidimensional knapsack problems is np hard

a stagewise rejective multiple test procedure based on a modified bonferroni test summary simes 1986 has proposed a modified bonferroni procedure for the test of an overall hypothesis which is the combination of n individual hypotheses in contrast to the classical bonferroni procedure it is not obvious how statements about individual hypotheses are to be made for this procedure in the present paper a multiple test procedure allowing statements on individual hypotheses is proposed it is based on the principle of closed test procedures marcus peritz gabriel 1976 and controls the multiple level a

point and interval estimation following a sequential clinical trial summary the usual methodology employed in analysis after a sequential clinical trial is based on orderings of the possible samples resulting from the design however this approach lacks flexibility for use in wider applications in this paper two estimation techniques not based on orderings are considered and modified to obtain improved accuracy a bias adjusted maximum likelihood estimate together with a new and general method for setting confidence limits are discussed the realistic scenario of group sequential monitoring is assumed and methods for exact estimation are given accuracy of the methodology after a triangular test and an o brien fleming test are demonstrated through simulation

nonparametric rank based methods for group sequential monitoring of paired censored survival data summary this research gives methods for nonparametric sequential monitoring of paired censored survival data in the two sample problem using paired weighted log rank statistics with adjustments for dependence in survival and censoring outcomes the joint asymptotic closed form distribution of these sequentially monitored statistics has a dependent increments structure simulations validating operating characteristics of the proposed methods highlight power and size consequences of ignoring even mildly correlated data a motivating example is presented via the early treatment diabetic retinopathy study

discrete sequential boundaries for clinical trials summary pocock 1977 o brien fleming 1979 and slud wei 1982 have proposed different methods to construct discrete sequential boundaries for clinical trials these methods require that the total number of decision times be specified in advance in the present paper we propose a more flexible way to construct discrete sequential boundaries the method is based on the choice of a function a t which characterizes the rate at which the error level ac is spent the boundary at a decision time is determined by a t and by past and current decision times but does not depend on the future decision times or the total number of decision times

confidence intervals for secondary parameters following a sequential test in sequential studies formal interim analyses are usually restricted to a consideration of a single null hypothesis concerning a single parameter of interest valid frequentist methods of hypothesis testing and of point and interval estimation for the primary parameter have already been devised for use at the end of such a study however the completed data set may warrant a more detailed analysis involving the estimation of parameters corresponding to effects that were not used to determine when to stop and yet correlated with those that were this paper describes methods for setting confidence intervals for secondary parameters in a way which provides the correct coverage probability in repeated frequentist realizations of the sequential design used the method assumes that information accumulates on the primary and secondary parameters at proportional rates this requirement will be valid in many potential applications but only in limited situations in survival analysis

a bayesian approach to the multiplicity problem for significance testing with binomial data statistical analyses of simple tumor rates from an animal experiment with one control and one treated group typically consist of hypothesis testing of many 2 x 2 tables one for each tumor type or site the multiplicity of significance tests may cause excessive overall false positive rates this paper presents a bayesian approach to the problem of multiple significance testing we develop a normal logistic model that accommodates the incidences of all tumor types or sites observed in the current experiment simultaneously as well as their historical control incidences exchangeable normal priors are assumed for certain linear terms in the model posterior means standard deviations and bayesian p values are computed for an average treatment effect as well as for the effects on individual tumor types or sites model assumptions are checked using probability plots and the sensitivity of the parameter estimates to alternative priors is studied the method is illustrated using tumor data from a chronic animal experiment

on adjusting p values for multiplicity response 1 the development of proc multtest while controversial the use of multiplicity adjustments has gained acceptance recently in varied fields of scientific endeavor and their associated publications multiplicity adjustments have been considered important in pharmaceutical safety determinations involving multiple endpoints such as in adverse events analysis of clinical trials and in animal carcinogenicity studies in situations where the compound tested is completely safe it is likely to observe false positive indications of one or more untoward outcomes when unadjusted testing methods are used multiplicity adjustment is also important in epidemiology and other complicated areas of data analysis as it offers protection against conclusions that are driven by excessive data mining multiplicity concerns in toxicology and clinical trials prompted the development of the sas procedure proc multtest which calculates adjusted p values for a user supplied family of tests in a wide variety of applications biometrics readers will be interested to know that this software is readily available to calculate many but not all of the multiplicity adjusted p values described by wright 1992 in addition the software incorporates many improvements and enhancements such as the ability to incorporate correlations and nonnormal distributions the development of proc multtest started in may 1987 one of us westfall presented a resampling approach to calculating adjusted p values for multiple tests in multivariate binomial models with special application to the animal carcinogenicity problem the talk was entitled multivariate binomial testing and was given as the keynote address for the midwestern biopharmaceutical statistics workshop mbsw held in muncie indiana similar approaches to the same problem were developed concurrently and independently all of which have been published farrar and crump 1988 heyse and rom 1988 westfall and young 1989 an interesting biometrics connection should be mentioned here dr young thought that westfall s 1985 publication concerning simultaneous inference with multivariate binary data might be applied to the multiplicity problem in animal carcinogenicity studies dr young therefore invited dr westfall to speak at the mbsw conference on this application the initial precursor to proc multtest called proc mbin was developed in 1988 we wrote the specifications for the software and the coding was performed by youling lin at texas tech university under the auspices of the pharmaceutical manufacturing association a consortium of drug companies offered financial and intellectual support for the project specific individuals and companies who contributed are listed below in the acknowledgements section the development proceeded through a series of meetings at national and regional statistics conferences attendees included representatives from the funding organizations and from the united states food and drug administration from the outset it was decided that the primary form of output from the software would be the adjusted p value for the reasons wright describes in his opening paragraph the procedure proc mbin was described by westfall lin and young 1989 and was donated to sas institute inc the software performed multiplicity adjustments for multiple tests z score or exact permutation tests in multivariate possibly stratified multiple group binary outcome situations the main feature of the output of this software was the use of adjusted p values to summarize many statistical tests single step resampling bootstrap or permutation bonferroni and sidak methods were used to calculate the adjustments the tabular form of the output was much like tables 2 and 3 of wright showing unadjusted p values which we called raw p values side by side with various types of adjusted p values

an improved approximation for calculation of confidence intervals after a sequential clinical trial a method for the calculation of confidence intervals following a sequential clinical trial is proposed which is more accurate than methods based solely on continuous monitoring assumptions but is not as computationally laborious as previous methods suggested for group sequential trials the calculation takes account of the excess of the sample path over the boundary at the final inspection of the trial which may be substantial after group sequential monitoring and is ignored when continuous monitoring is assumed accuracy of such confidence intervals after a triangular test is investigated by simulation of coverage probabilities individual confidence limit probabilities and p value curves

p value adjustment in sequential clinical trials many randomized clinical trials utilize a group sequential monitoring procedure for assessing treatment efficacy during the course of the trial for a group sequential trial in which the null hypothesis is ultimately rejected the nominal p value generally overstates the statistical significance of the result and some adjustment is required several orderings of the sample space have been proposed for performing this adjustment each with unique operating characteristics in this article we compare four proposed methods with respect to the degree to which each captures the strength of evidence against the null hypothesis implied by the data we conclude that the ordering of the sample space induced by the observed z score has the most desirable operating characteristics

sequential monitoring for comparison of changes in a response variable in clinical studies the spending function approach proposed by lan and demets 1983 biometrika 70 659 663 for sequential monitoring of clinical trials is applied to situations where comparison of changes in a continuous response variable between two groups is the primary concern death loss to follow up and missed visits could cause follow up measurements to be right censored or missing for some participants furthermore the probability of being censored may be dependent on the parameter value of the response variable informative censoring we propose to compare treatment effects by comparing areas under the expected response change curves between the two groups when the response curves are linear as a function of time in both groups this comparison is equivalent to comparing the rates of change in the response variable covariances of the sequential test statistics are derived conditions for having independent increments are presented for studies designed to evaluate long term treatment effects spending functions obtained by shifting the usual spending functions kim and demets 1987 biometrika 74 149 154 to the right and then rescaling to the remaining interval are also proposed such a shifted spending function is applied to the monitoring plan for the lung health study anthonisen 1989 american review of respiratory diseases 140 871 872

p value adjustments for multiple tests in multivariate binomial models abstract data from rodent carcinogenicity preclinical and clinical studies involving new drugs may be modeled as having come from multivariate binomial distributions in two year rodent carcinogenicity studies there are typically 20 50 tissues examined for occurrence of any of several possible lesions for a particular treatment group the number of occurrences of a particular lesion at a particular tissue may be modeled as binomial and the vector of such frequencies may be considered multivariate binomial with unspecified dependence structure the same model may also apply to clinical side effects data in this case the marginal frequencies may represent occurrences of events ranging from headaches to ingrown toenails frequently the goal of such studies is to isolate site specific significant differences between treatment and control groups for example in rodent carcinogenicity analyses it is generally not sufficient to claim that a new compound causes an increase in tumors at some unspecified si

two sample repeated significance tests based on the modified wilcoxon statistic abstract the asymptotic distribution theory of sequentially computed modified wilcoxon scores is developed for two sample survival data with random staggered entry and random loss to follow up the asymptotic covariance indicates generally dependent modified wilcoxon increments contradicting the authors reading of jones and whitehead 1979 a repeated significance testing procedure is presented for testing the equality of two survival distributions based on the asymptotic theory the early stopping properties of this procedure are illustrated by a prostate cancer example

simultaneous group sequential analysis of rank based and weighted kaplan meier tests for paired censored survival data this research sequentially monitors paired survival differences using a new class of nonparametric tests based on functionals of standardized paired weighted log rank pwlr and standardized paired weighted kaplan meier pwkm tests during a trial these tests may alternately assume the role of the more extreme statistic by monitoring pemax the maximum between the absolute values of the standardized pwlr and pwkm one combines advantages of rank based rb and non rb paired testing paradigms simulations show that monitoring treatment differences using pemax maintains type i error and is nearly as powerful as using the more advantageous of the two tests in proportional hazards ph as well as non ph situations hence pemax preserves power more robustly than individually monitored pwlr and pwkm while maintaining a reasonably simple approach to design and analysis of results an example from the early treatment diabetic retinopathy study etdrs is given

genome scanning tests for comparing amino acid sequences between groups consider a placebo controlled preventive hiv vaccine efficacy trial an hiv amino acid sequence is measured from each volunteer who acquires hiv and these sequences are aligned together with the reference hiv sequence represented in the vaccine we develop genome scanning methods to identify positions at which the amino acids in infected vaccine recipient sequences either a are more divergent from the reference amino acid than the amino acids in infected placebo recipient sequences or b have a different frequency distribution than the placebo sequences irrespective of a reference amino acid we consider t test type statistics for problem a and euclidean mahalanobis and kullback leibler type statistics for problem b the test statistics incorporate weights to reflect biological information contained in different amino acid positions and mismatches position specific p values are obtained by approximating the null distribution of the statistics either by a permutation procedure or by a nonparametric estimation a permutation method is used to estimate a cut off p value to control the per comparison error rate at a prespecified level the methods are examined in simulations and are applied to two hiv examples the methods for problem b address the general problem of comparing discrete frequency distributions between groups in a high dimensional data setting

sequential methods for parametric survival models summary the asymptotic sequential distribution is derived for score tests and maximum likelihood estimates in parametric survival models in the presence of nuisance parameters where individuals enter over time and the data are subject to right censoring it is shown that these sequential tests and estimates have an independent increments structure this allows the use of standard group sequential methods for early stopping

one sided significance tests for generalized linear models under dichotomous response summary dichotomous response models are common in many experimental settings often concomitant explanatory variables are recorded and a generalized linear model such as a logit model is fit in some cases interest in specific model parameters is directed only at one sided departures from some null effect in these cases procedures can be developed for testing the null effect against a one sided alternative these include bonferroni type adjustments of univariate wald tests and likelihood ratio tests that employ inequality constrained multivariate theory this paper examines such tests of significance monte carlo evaluations are undertaken to examine the small sample properties of the various procedures the procedures are seen to perform fairly well generally achieving their nominal sizes at total sample sizes near 100 experimental units extensions to the problem of one sided tests against a control or standard are also considered