generalized monte carlo significance tests summary simple monte carlo significance testing has many applications particularly in the preliminary analysis of spatial data the method requires the value of the test statistic to be ranked among a random sample of values generated according to the null hypothesis however there are situations in which a sample of values can only be conveniently generated using a markov chain initiated by the observed data so that independence is violated this paper describes two methods that overcome the problem of dependence and allow exact tests to be carried out the methods are applied to the rasch model to the finite lattice ising model and to the testing of association between spatial processes power is discussed in a simple case

global envelope tests for spatial processes summary envelope tests are a popular tool in spatial statistics where they are used in goodness of fit testing these tests graphically compare an empirical function t r with its simulated counterparts from the null model however the type i error probability is conventionally controlled for a fixed distance r only whereas the functions are inspected on an interval of distances i in this study we propose two approaches related to barnard s monte carlo test for building global envelope tests on i ordering the empirical and simulated functions on the basis of their r wise ranks among each other and the construction of envelopes for a deviation test these new tests allow the a priori choice of the global and they yield p values we illustrate these tests by using simulated and real point pattern data

an importance sampling algorithm for exact conditional tests in log linear models summary a simple but quite general simulation method for conducting exact conditional lackof fit tests in log linear models is proposed our monte carlo approximation utilises an importance sampling method motivated by the crude normal approximation to the poisson distribution examples considered include tests of quasi symmetry and related models for square tables and tests concerning higher order interactions in multi way tables the method is competitive with direct simulation from the exact conditional distribution when this is feasible and outperforms alternative monte carlo procedures when direct simulation is infeasible provided the number of degrees of freedom of the test is not too large extension of the method to tests against non saturated alternatives is straightforward and is briefly discussed and illustrated

finite sample properties and asymptotic efficiency of monte carlo tests on donne une approche unifiee qui couvre le cas d une distribution nulle arbitraire afin d etudier les proprietes statistiques des tests de monte carlo

levy white noise measures on infinite dimensional spaces existence and characterization of the measurable support it is shown that a levy white noise measure always exists as a borel measure on the dual k of the space k of c functions on r with compact support then a characterization theorem that ensures that the measurable support of is contained in s is proved in the course of the proofs a representation of the levy process as a function on k is obtained and stochastic levy integrals are studied

algebraic algorithms for sampling from conditional distributions we construct markov chain algorithms for sampling from discrete exponential families conditional on a sufficient statistic examples include contingency tables logistic regression and spectral analysis of permutation data the algorithms involve computations in polynomial rings using grobner bases

graph animals subgraph sampling and motif search in large networks we generalize a sampling algorithm for lattice animals connected clusters on a regular lattice to a monte carlo algorithm for graph animals i e connected subgraphs in arbitrary networks as with the algorithm in n kashtan et al bioinformatics 20 1746 2004 it provides a weighted sample but the computation of the weights is much faster linear in the size of subgraphs instead of superexponential this allows subgraphs with up to ten or more nodes to be sampled with very high statistics from arbitrarily large networks using this together with a heuristic algorithm for rapidly classifying isomorphic graphs we present results for two protein interaction networks obtained using the tandem affinity purification tap method one of escherichia coli with 230 nodes and 695 links and one for yeast saccharomyces cerevisiae with roughly ten times more nodes and links we find in both cases that most connected subgraphs are strong motifs z scores 10 or antimotifs z scores 10 when the null model is the ensemble of networks with fixed degree sequence strong differences appear between the two networks with dominant motifs in e coli being nearly bipartite graphs and having many pairs of nodes that connect to the same neighbors while dominant motifs in yeast tend towards completeness or contain large cliques we also explore a number of methods that do not rely on measurements of z scores or comparisons with null models for instance we discuss the influence of specific complexes like the 26s proteasome in yeast where a small number of complexes dominate the k cores with large k and have a decisive effect on the strongest motifs with 6 8 nodes we also present zipf plots of counts versus rank they show broad distributions that are not power laws in contrast to the case when disconnected subgraphs are included

exact tests for the rasch model via sequential importance sampling rasch proposed an exact conditional inference approach to testing his model but never implemented it because it involves the calculation of a complicated probability this paper furthers rasch s approach by 1 providing an efficient monte carlo methodology for accurately approximating the required probability and 2 illustrating the usefulness of rasch s approach for several important testing problems through simulation studies our monte carlo methodology is shown to compare favorably to other monte carlo methods proposed for this problem in two respects it is considerably faster and it provides more reliable estimates of the monte carlo standard error

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

assessing the significance of the correlation between two spatial processes modified tests of association based on the correlation coefficient or the covariance between two spatially autocorrelated processes are presented these tests can be used both for lattice and nonlattice data they are based on the evaluation of an effective sample size that takes into account the spatial structure for positively autocorrelated processes the effective sample size is reduced a method for evaluating this reduction via an approximation of the variance of the correlation coefficient is developed the performance of the tests is assessed by monte carlo simulations the method is illustrated by examples from geographical epidemiology

nonparametric goodness of fit tests for the rasch model a monte carlo algorithm realizing a family of nonparametric tests for the rasch model is introduced which are conditional on the item and subject marginals the algorithm is based on random changes of elements of data matrices without changing the marginals most powerful tests against all alternative hypotheses are given for which a monotone characteristic may be computed from the data matrix alternatives may also be composed computation times are long but exactp values are approximated with the quality of approximation only depending on calculation time but not on the number of persons the power and the flexibility of the procedure is demonstrated by means of an empirical example where among others indicators for increased item similarities the existence of subscales violations of sufficiency of the raw score as well as learning processes were found many of the features described are implemented in the program t rasch 1 0 by ponocny and ponocny seliger 1999

intrinsic autoregressions and related models on the two dimensional lattice summary stationary autoregressions on a two dimensional lattice are generalized to intrinsic models where only increments are assumed to be stationary prediction formulae and the asymptotic behaviour of the semivariogram are derived for parameter estimation we propose an approximate maximum likelihood estimator a generalization of whittle s estimator it is derived also for general intrinsic models

the effect of simulation order on level accuracy and power of monte carlo tests we discuss the effect of simulation order on level accuracy and power of monte carlo tests in a very general setting both parametric problems with or without nuisance parameters and nonparametric problems are treated by a single unifying argument it is shown that if the level of a monte carlo test is known only nominally not precisely then the level error of a monte carlo test is an order of magnitude less than that of the corresponding asymptotic test this result is available whenever the test statistic is asymptotically pivotal even if the number of simulations is held fixed as the sample size n increases it implies that monte carlo methods are a real alternative to asymptotic methods we also show that even if the number of simulations is held fixed a monte carlo test is able to distinguish between the null hypothesis and alternative hypotheses distant n 12 from the null

link and subgraph likelihoods in random undirected networks with fixed and partially fixed degree sequences the simplest null models for networks used to distinguish significant features of a particular network from a priori expected features are random ensembles with the degree sequence fixed by the specific network of interest these fixed degree sequence fds ensembles are however famously resistant to analytic attack in this paper we introduce ensembles with partially fixed degree sequences pfds and compare analytic results obtained for them with monte carlo results for the fds ensemble these results include link likelihoods subgraph likelihoods and degree correlations we find that local structural features in the fds ensemble can be reasonably well estimated by simultaneously fixing only the degrees of a few nodes in addition to the total number of nodes and links as test cases we use two protein interaction networks escherichia coli saccharomyces cerevisiae the internet on the autonomous system as level and the world wide web fixing just the degrees of two nodes gives the mean neighbor degree as a function of node degree k k in agreement with results explicitly obtained from rewiring for power law degree distributions we derive the disassortativity analytically in the pfds ensemble the partition function can be expanded diagrammatically we obtain an explicit expression for the link likelihood to lowest order which reduces in the limit of large sparse undirected networks with l links and with k max l to the simple formula p k k kk 2l kk in a similar limit the probability for three nodes to be linked into a triangle reduces to the factorized expression p delta k 1 k 2 k 3 p k 1 k 2 p k 1 k 3 p k 2 k 3

biometrika centenary nonparametrics summary contributions in biometrika to nonparametric statistics are surveyed with particular emphasis on more recent work it is argued that even in the journal s very early years it had interests in smoothing methods for density estimation and in techniques for nonparametric regression in addition to its more obvious focus on classical nonparametric statistics such as ranking methods biometrika s early involvement with permutation testing has links to its contemporary work on bootstrap methods emphases on different types of nonparametric statistics have evolved over time however a rapidly increasing proportion of the journal s pages is devoted to papers that include some use of nonparametric ideas or methods

confidence intervals from monte carlo tests abstract monte carlo tests have exact level when the distribution of the test statistic is free of nuisance parameters confidence sets obtained by inverting such tests are also exact but may have a complicated structure the sets reduce to intervals if the sampling can be organized in a special way a sufficient condition is that the test statistic is monotone in the parameter of interest when all random drawings are kept fixed examples given include models from the one parameter exponential class a simple theory quantifying the impact of monte carlo uncertainty is also developed

plausibility functions and exact frequentist inference in the frequentist program inferential methods with exact control on error rates are a primary focus the standard approach however is to rely on asymptotic approximations which may not be suitable this article presents a general framework for the construction of exact frequentist procedures based on plausibility functions it is shown that the plausibility function based tests and confidence regions have the desired frequentist properties in finite samples no large sample justification needed an extension of the proposed method is also given for problems involving nuisance parameters examples demonstrate that the plausibility function based method is both exact and efficient in a wide variety of problems

evolutionary similarity among genes an evolutionary history of a set of organisms is a family tree or topology with branches of various lengths between vertices that describe how closely the organisms are related to each other we

fuzzy p values in latent variable problems we consider the problem of testing a statistical hypothesis where the scientifically meaningful test statistic is a function of latent variables in particular we consider detection of genetic linkage where the latent variables are patterns of inheritance at specific genome locations introduced by geyer meeden 2005 fuzzy p values are random variables described by their probability distributions that are interpreted as p values for latent variable problems we introduce the notion of a fuzzy p value as having the conditional distribution of the latent p value given the observed data where the latent p value is the random variable that would be the p value if the latent variables were observed the fuzzy p value provides an exact test using two sets of simulations of the latent variables under the null hypothesis one unconditional and the other conditional on the observed data it provides not only an expression of the strength of the evidence against the null hypothesis but also an expression of the uncertainty in that expression owing to lack of knowledge of the latent variables we illustrate these features with an example of simulated data mimicking a real example of the detection of genetic linkage

an efficient mcmc algorithm to sample binary matrices with fixed marginals uniform sampling of binary matrices with fixed margins is known as a difficult problem two classes of algorithms to sample from a distribution not too different from the uniform are studied in the literature importance sampling and markov chain monte carlo mcmc existing mcmc algorithms converge slowly require a long burn in period and yield highly dependent samples chen et al developed an importance sampling algorithm that is highly efficient for relatively small tables for larger but still moderate sized tables 300 30 chen et al s algorithm is less efficient this article develops a new mcmc algorithm that converges much faster than the existing ones and that is more efficient than chen s algorithm for large problems its stationary distribution is uniform the algorithm is extended to the case of square matrices with fixed diagonal for applications in social network theory

comparison between the morphological skeleton and morphological shape decomposition the morphological skeleton and morphological shape decomposition msd are two popular approaches for morphological shape representation each method represents an object as an algebraic combination of a number of components where each component is given by a locus of points dilated by a specified structuring element homothetic this correspondence develops a theoretical comparison between the two methods combining the theoretical results with several representation cost measures we make a concrete comparison of the efficiency of the two methods the results indicate that for complex objects i e objects requiring a full range of homothetic sizes in the morphological skeleton representation the msd represents objects more efficiently than the morphological skeleton for three of four suggested cost measures

exact sample size needed to detect dependence in 2 2 2 tables summary in the georgia centenarian study poon et al exceptional longevity 2006 centenarian cases and young controls are classified according to three categories age ethnic origin and single nucleotide polymorphisms snps of candidate longevity genes where each factor has two possible levels here we provide methodologies to determine the minimum sample size needed to detect dependence in 2 2 2 tables based on fisher s exact test evaluated exactly or by markov chain monte carlo mcmc assuming only the case total l and the control total n are known while our mcmc method uses serial computing parallel computing techniques are employed to solve the exact sample size problem these tools will allow researchers to design efficient sampling strategies and to select informative snps we apply our tools to 2 2 2 tables obtained from a pilot study of the georgia centenarians study and the sample size results provided important information for the subsequent major study a comparison between the results of an exact method and those of a mcmc method showed that the mcmc method studied needed much less computation time on average 10 16 times faster on average for situations examined with s e 2 60 but its sample size results were only valid as a rule for larger sample sizes in the hundreds

clustering algorithms for chains we consider the problem of clustering a set of chains to k clusters a chain is a totally ordered subset of a finite set of items chains are an intuitive way to express preferences over a set of alternatives as well as a useful representation of ratings in situations where the item specific scores are either difficult to obtain too noisy due to measurement error or simply not as relevant as the order that they induce over the items first we adapt the classical k means for chains by proposing a suitable distance function and a centroid structure we also present two different approaches for mapping chains to a vector space the first one is related to the planted partition model while the second one has an intuitive geometrical interpretation finally we discuss a randomization test for assessing the significance of a clustering to this end we present an mcmc algorithm for sampling random sets of chains that share certain properties with the original data the methods are studied in a series of experiments using real and artificial data results indicate that the methods produce interesting clusterings and for certain types of inputs improve upon previous work on clustering algorithms for orders

markov chains for monte carlo tests of genetic equilibrium in multidimensional contingency tables hardy weinberg equilibrium and linkage equilibrium are fundamental concepts in population genetics in practice testing linkage equilibrium in haplotype data is equivalent to testing independence in a large sparse multidimensional contingency table testing hardy weinberg and linkage equilibrium simultaneously on multilocus genotype data introduces the additional complications of missing information and symmetry constraints on marginal probabilities to avoid unreliable large sample approximations for sparse contingency tables one can use exact tests like fisher s classical test that condition on observed marginal totals unfortunately computing p values for exact tests is often infeasible because of the large number of tables consistent with the marginal totals of an observed table we develop here markov chains for sampling from the appropriate conditional distributions for testing genetic equilibrium these chains compare favorably with a parallel independent sampling method that we present for n haplotype observations on j loci the markov chains converge to their stationary distributions in j 1 n ln n 2 0 n steps and can be an efficient tool for estimating p values our theoretical treatment of these results involves strong stationary stopping times order statistics large deviations and the embedding of poisson processes we include some general results on the application of strong stationary times to bounding the precision and bias of sample average estimators

monte carlo exact conditional tests for log linear and logistic models summary the form of the exact conditional distribution of a sufficient statistic for the interest parameters given a sufficient statistic for the nuisance parameters is derived for a generalized linear model with canonical link general results for log linear and logistic models are given a gibbs sampling approach for generating from the conditional distribution is proposed which enables monte carlo exact conditional tests to be performed examples include tests for goodness of fit of the all two way interaction model for a 28 table and of a simple logistic model tests against non saturated alternatives are also considered

constrained monte carlo maximum likelihood for dependent data maximum likelihood estimates mles in autologistic models and other exponential family models for dependent data can be calculated with markov chain monte carlo methods the metropolis algorithm or the gibbs sampler which simulate ergodic markov chains having equilibrium distributions in the model from one realization of such a markov chain a monte carlo approximant to the whole likelihood function can be constructed the parameter value if any maximizing this function approximates the mle

spatial statistics and bayesian computation on wednesday may 6th 1992 professor b w silverman in the chair summary markov chain monte carlo mcmc algorithms such as the gibbs sampler have provided a bayesian inference machine in image analysis and in other areas of spatial statistics for several years founded on the pioneering ideas of ulf grenander more recently the observation that hyperparameters can be included as part of the updating schedule and the fact that almost any multivariate distribution is equivalently a markov random field has opened the way to the use of mcmc in general bayesian computation in this paper we trace the early development of mcmc in bayesian inference review some recent computational progress in statistical physics based on the introduction of auxiliary variables and discuss its current and future relevance in bayesian applications we briefly describe a simple mcmc implementation for the bayesian analysis of agricultural field experiments with which we have some practical experience

biometrika centenary theory and general methodology note reprinted in biometrika one hundred years edited by d m titterington and d r cox oxford university press 11 50 reference stat article 2001 002doi 10 1093 biomet 88 1 13view record in web of science record created on 2006 04 21 modified on 2017 05 12

logit models and logistic regressions for social networks iii valued relations this paper generalizes thep model for dichotomous social network data wasserman pattison 1996 to the polytomous case the generalization is achieved by transforming valued social networks into three way binary arrays this data transformation requires a modification of the hammersley clifford theorem that underpins thep class of models we demonstrate that provided that certain non observed data patterns are excluded from consideration a suitable version of the theorem can be developed we also show that the approach amounts to a model for multiple logits derived from a pseudo likelihood function estimation within this model is analogous to the separate fitting of multinomial baseline logits except that the hammersley clifford theorem requires the equating of certain parameters across logits the paper describes how to convert a valued network into a data array suitable for fitting the model and provides some illustrative empirical examples