exponential family mixture models with least squares estimators pour une densite arbitraire d une famille exponentielle a un parametre on montre comment construire une distribution de melange sur le parametre de telle sorte que la distribution resultante soit une famille exponentielle a deux ou plusieurs parametres

survival models for heterogeneous populations derived from stable distributions summary a new three parameter family of distributions on the positive numbers is proposed it includes the stable distributions on the positive numbers the gamma the degenerate and the inverse gaussian distributions the family is characterized by the laplace transform from which moments convolutions infinite divisibility unimodality and other properties are derived the density is complicated but a simple saddlepoint approximation is provided weibull and gompertz distributions are naturally mixed over some of the distributions the family is natural exponential in one of the parameters the distributions are relevant for application as frailty distributions in life table methods for heterogeneous populations desirable properties of such distributions are discussed as an example survival after myocardial infarction is considered

learning the information divergence information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems examples are nonnegative matrix tensor factorization stochastic neighbor embedding topic models and bayesian network optimization the success of such a learning task depends heavily on a suitable divergence a large variety of divergences have been suggested and analyzed but very few results are available for an objective choice of the optimal divergence for a given task here we present a framework that facilitates automatic selection of the best divergence among a given family based on standard maximum likelihood estimation we first propose an approximated tweedie distribution for the beta divergence family selecting the best beta then becomes a machine learning problem solved by maximum likelihood next we reformulate alpha divergence in terms of beta divergence which enables automatic selection of alpha by maximum likelihood with reuse of the learning principle for beta divergence furthermore we show the connections between gamma and beta divergences as well as renyi and alpha divergences such that our automatic selection framework is extended to non separable divergences experiments on both synthetic and real world data demonstrate that our method can quite accurately select information divergence across different learning problems and various divergence families

the relative frailty variance and shared frailty models the relative frailty variance among survivors provides a readily interpretable measure of how the heterogeneity of a population as represented by a frailty model evolves over time we discuss the properties of the relative frailty variance show that it characterizes frailty distributions and that suitably rescaled it may be used to compare patterns of dependence across models and data sets in shared frailty models the relative frailty variance is closely related to the cross ratio function which is estimable from bivariate survival data we investigate the possible shapes of the relative frailty variance function for the purpose of model selection and review available frailty distribution families in this context we introduce several new families with contrasting properties including simple but flexible time varying frailty models the benefits of the approach we propose are illustrated with two applications to bivariate current status data obtained from serological surveys

a copula transformation in multivariate mixed discrete continuous models abstract copulas allow a flexible and simultaneous modeling of complicated dependence structures together with various marginal distributions especially if the density function can be represented as the product of the marginal density functions and the copula density function this leads to both an intuitive interpretation of the conditional distribution and convenient estimation procedures however this is no longer the case for copula models with mixed discrete and continuous marginal distributions because the corresponding density function cannot be decomposed so nicely in this paper we introduce a copula transformation method that allows to represent the density function of a distribution with mixed discrete and continuous marginals as the product of the marginal probability mass density functions and the copula density function with the proposed method conditional distributions can be described analytically and the computational complexity in the estimation procedure can be reduced depending on the type of copula used

a wavelet based approach to revealing the tweedie distribution type in sparse data abstract we propose two approaches to the analysis of sparse stochastic data which exhibit a power law dependence between their first and second moments taylor s law and determining the respective power index when it has a value between 1 and 2 they are based on the analysis of components of the haar wavelet expansion the first method uses a dependence between the first iterations of averaging and wavelet coefficients with the subsequent studying the statistics of zero paddings on the line which corresponds the linear dependence between two first moments of the analysed distributions the second method refers to taylor s plot formed by the full set of wavelet coefficients it is discussed that such representations provide also an opportunity to check time scale stability of analysed data and distinguish between particular cases of the tweedie probabilistic distribution both simulated series and real marine species abundance data for five spatial regions of the pacific are used as example illustrating an applicability of these approaches

critical phenomena in colloid polymer mixtures interfacial tension order parameter susceptibility and coexistence diameter the critical behavior of a model colloid polymer mixture the so called asakura oosawa ao model is studied using computer simulations and finite size scaling techniques investigated are the interfacial tension the order parameter the susceptibility and the coexistence diameter our results clearly show that the interfacial tension vanishes at the critical point with exponent 2 ensuremath nu ensuremath approx 1 26 this is in good agreement with the 3d ising exponent also calculated are critical amplitude ratios which are shown to be compatible with the corresponding 3d ising values we additionally identify a number of subtleties that are encountered when finite size scaling is applied to the ao model in particular we find that close to the critical point the finite size extrapolation of the interfacial tension is most consistent when logarithmic size dependences are ignored

a quasi likelihood approach to nonnegative matrix factorization a unified approach to nonnegative matrix factorization based on the theory of generalized linear models is proposed this approach embeds a variety of statistical models including the exponential family within a single theoretical framework and provides a unified view of such factorizations from the perspective of quasi likelihood using this framework a family of algorithms for handling signal dependent noise is developed and its convergence proved using the expectation maximization algorithm in addition a measure to evaluate the goodness of fit of the resulting factorization is described the proposed methods allow modeling of nonlinear effects using appropriate link functions and are illustrated using an application in biomedical signal processing

modelling heterogeneity in clustered count data with extra zeros using compound poisson random effect in medical and health studies heterogeneities in clustered count data have been traditionally modeled by positive random effects in poisson mixed models however excessive zeros often occur in clustered medical and health count data in this paper we consider a three level random effects zero inflated poisson model for health care utilization data where data are clustered by both subjects and families to accommodate zero and positive components in the count response compatibly we model the subject level random effects by a compound poisson distribution our model displays a variance components decomposition which clearly reflects the hierarchical structure of clustered data a quasi likelihood approach has been developed in the estimation of our model we illustrate the method with analysis of the health care utilization data the performance of our method is also evaluated through simulation studies

automatic smoothing of regression functions in generalized linear models abstract we consider the penalized likelihood method for estimating nonparametric regression functions in generalized linear models nelder and wedderburn 1972 and present a generalized cross validation procedure for empirically assessing an appropriate amount of smoothing in these estimates asymptotic arguments and numerical simulations are used to show that the generalized cross validatory procedure preforms well from the point of view of a weighted mean squared error criterion the methodology adds to the battery of graphical tools for model building and checking within the generalized linear model framework included are two examples motivated by medical and horticultural applications

natural exponential families with quadratic variance functions statistical theory the normal poisson gamma binomial negative binomial and nefghs distributions are the six univariate natural exponential families nef with quadratic variance functions qvf this sequel to morris 1982 treats certain statistical topics that can be handled within this unified nef qvf formulation including unbiased estimation bhattacharyya and cramer rao lower bounds conditional distributions and moments quadratic regression conjugate prior distributions moments of conjugate priors and posterior distributions empirical bayes and g 2 minimax marginal distributions and their moments parametric empirical bayes and characterizations

the bayesian lasso the lasso estimate for linear regression parameters can be interpreted as a bayesian posterior mode estimate when the regression parameters have independent laplace i e double exponential priors gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances a connection with the inverse gaussian distribution provides tractable full conditional distributions the bayesian lasso provides interval estimates bayesian credible intervals that can guide variable selection moreover the structure of the hierarchical model provides both bayesian and likelihood methods for selecting the lasso parameter slight modifications lead to bayesian versions of other lasso related estimation methods including bridge regression and a robust variant

sampling models which admit a given general exponential family as a conjugate family of priors let k k be a family of sampling distributions for the data x on a sample space x which is indexed by a parameter and let f be a family of priors on then f is said to be conjugate for k if it is closed under sampling that is if the posterior distributions of given the data x belong to f for almost all x in this paper we set up a framework for the study of what we term the dual problem for a given family of priors f a subfamily of a general exponential family find the class of sampling models k for which f is conjugate in particular we show that k must be a general exponential family dominated by some measure q on x b where b is the borel field on x it is the class of such measures q that we investigate in this paper we study its geometric features and general structure and apply the results to some familiar examples

testing the parametric null model hypothesis in semiparametric partial and generalized spline models on etend les resultats de cox et koh 1986 aux modeles generalises de wahba 1985 et a divers autres modeles splines partiels on montre que la statistique de test a une relation etroite avec le comportement de la fonction de validation croisee generalisee

electronic noise modeling in statistical iterative reconstruction we consider electronic noise modeling in tomographic image reconstruction when the measured signal is the sum of a gaussian distributed electronic noise component and another random variable whose log likelihood function satisfies a certain linearity condition examples of such likelihood functions include the poisson distribution and an exponential dispersion ed model that can approximate the signal statistics in integration mode x ray detectors we formulate the image reconstruction problem as a maximum likelihood estimation problem using an expectation maximization approach we demonstrate that a reconstruction algorithm can be obtained following a simple substitution rule from the one previously derived without electronic noise considerations to illustrate the applicability of the substitution rule we present examples of a fully iterative reconstruction algorithm and a sinogram smoothing algorithm both in transmission ct reconstruction when the measured signal contains additive electronic noise our simulation studies show the potential usefulness of accurate electronic noise modeling in low dose ct applications

automatic relevance determination in nonnegative matrix factorization with the beta divergence this paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization nmf with the divergence the divergence is a family of cost functions that includes the squared euclidean distance kullback leibler kl and itakura saito is divergences as special cases learning the model order is important as it is necessary to strike the right balance between data fidelity and overfitting we propose a bayesian model based on automatic relevance determination ard in which the columns of the dictionary matrix and the rows of the activation matrix are tied together through a common scale parameter in their prior a family of majorization minimization mm algorithms is proposed for maximum a posteriori map estimation a subset of scale parameters is driven to a small lower bound in the course of inference with the effect of pruning the corresponding spurious components we demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data the swimmer dataset a music decomposition example and a stock price prediction task

nonlinear hyperspectral unmixing with robust nonnegative matrix factorization we introduce a robust mixing model to describe hyperspectral data resulting from the mixture of several pure spectral signatures the new model extends the commonly used linear mixing model by introducing an additional term accounting for possible nonlinear effects that are treated as sparsely distributed additive outliers with the standard nonnegativity and sum to one constraints inherent to spectral unmixing our model leads to a new form of robust nonnegative matrix factorization with a group sparse outlier term the factorization is posed as an optimization problem which is addressed with a block coordinate descent algorithm involving majorization minimization updates simulation results obtained on synthetic and real data show that the proposed strategy competes with the state of the art linear and nonlinear unmixing methods

testing for overdispersion in poisson and binomial regression models abstract in this article a method for obtaining tests for overdispersion with respect to a natural exponential family is derived the tests are designed to be powerful against arbitrary alternative mixture models where only the first two moments of the mixed distribution are specified various tests for extra poisson and extra binomial variation are obtained as special cases the use of a particular test may be motivated by a consideration of the mechanism through which the overdispersion may arise the common occurrence of extra poisson and extra binomial variation has been noted by several authors however the poisson and binomial models remain valid in many instances and because of their simplicity and appeal it is of real interest to ascertain when they apply this paper develops a unifying theory for testing for overdispersion and generalizes tests previously derived including those by fisher 1950 collings and margolin 1985 and prentice 1986 it also shows the pearson statistic to be a sc

sample and population exponents of generalized taylor s law has been verified extensively in ecology where it applies to population abundance physics and other natural sciences its ubiquitous empirical verification suggests a context independent mechanism sample exponents b measured empirically via the scaling of sample mean and variance typically cluster around the value b 2 some theoretical models of population growth however predict a broad range of values for the population exponent b pertaining to the mean and variance of population density depending on details of the growth process is the widely reported sample exponent b 2 the result of ecological processes or could it be a statistical artifact here we apply large deviations theory and finite sample arguments to show exactly that in a broad class of growth models the sample exponent is b 2 regardless of the underlying population exponent we derive a generalized tl in terms of sample and population exponents bjk for the scaling of the kth vs the jth cumulants the sample exponent bjk depends predictably on the number of samples and for finite samples we obtain bjk k j asymptotically in time a prediction that we verify in two empirical examples thus the sample exponent b 2 may indeed be a statistical artifact and not dependent on population dynamics under conditions that we specify exactly given the broad class of models investigated our results apply to many fields where tl is used although inadequately understood

generalized linear models with varying dispersion summary generalized linear models are further generalized to include a linear predictor for the dispersion as well as for the mean it is shown how the convenient structure of generalized linear models can be carried over to this more general setting by considering the mean and dispersion structure separately mean and dispersion submodels are formulated for this the dependent variable for the dispersion submodel being the deviance components of the mean submodel the fact that both submodels are essentially generalized linear models themselves is used to derive simple expressions for the likelihood equations and for asymptotic tests estimation algorithms are proposed which have good convergence properties the results apply mainly to the normal inverse gaussian and gamma distributions but can be extended to discrete distributions by using quasi likelihoods the methods developed are applied to a well known data set

spline smoothing in a partly linear model on donne des estimations qui minimisent la somme des sommes residuelles des carres et une penalite de difficulte

double exponential families and their use in generalized linear regression abstract in one parameter exponential families such as the binomial and poisson the variance is a function of the mean double exponential families allow the introduction of a second parameter that controls variance independently of the mean double families are used as constituent distributions in generalized linear regressions in which both means and variances are allowed to depend on observed covariates the theory is applied to two examples a logistic regression and a large two way contingency table in such cases the binomial model of variance is often untrustworthy for example because genuine random sampling was infeasible the subjects may have been obtained in clumps so that the statistician should really be using smaller sample sizes clumped sampling is just one of many possible causes of overdispersion a habitual source of concern to users of binomial and poisson models this article concerns a class of regression families that allow the statistician to model overdispersion while carrying

natural real exponential families with cubic variance functions pursuing the classification initiated by morris 1982 we describe all the natural exponential families on the real line such that the variance is a polynomial function of the mean with degree less than or equal to 3 we get twelve different types the first six appear in the fundamental paper by morris most of the other six appear as distributions of first passage times in the literature an explanation of this occurrence of stopping times is provided by the introduction of the notion of reciprocity between two measures of between two natural exponential families and by classical fluctuation theory

a variance components approach to categorical data models with heterogeneous cell populations analysis of spatial gradients in lung cancer mortality rates in north carolina counties a mixed categorical continuous variable model is proposed for the analysis of mortality rates this model differs from other available models such as weighted least squares and loglinear models in that the within cell populations are assumed to be heterogeneous in their levels of mortality risk heterogeneity implies that in addition to the sampling variance considered in other available models there will be a second component of variance due solely to within cell heterogeneity maximum likelihood procedures are presented for the estimation of the model parameters these procedures are based on the assumption that the distribution function for each cell death count is the negative binomial probability function this assumption is shown to be equivalent to assuming a mixture of poisson processes with the differential risk levels among individuals within each cell being governed by a two parameter gamma distribution the model is applied to data on lung cancer mortality for 1970 1975 for the 100 counties of north carolina the analysis shows that though a gradient in lung cancer mortality rates exist in space the gradient is restricted to specific demographic categories identified by race age and sex

diagnostics for overdispersion abstract diagnostic tools are proposed for assessing the dependence of extrabinomial or extra poisson variation on explanatory variables and for comparing several common models for overdispersion these tools are based on tests for regression terms in the dispersion parameter of a generalized linear model using double exponential family and pseudolikelihood formulations score tests do not require the full fitting of models for variation and lead to easy graphical and numerical procedures based on squared residuals deviance or pearson robust modifications of these are motivated by levene s test in linear models the diagnostic tools are intended primarily to ensure prudent modeling of the variance to make correct inferences about parameters in the mean

approximate inference in generalized linear mixed models statistical approaches to overdispersion correlated errors shrinkage estimation and smoothing of regression relationships may be encompassed within the framework of the generalized linear mixed model glmm given an unobserved vector of random effects observations are assumed to be conditionally independent with means that depend on the linear predictor through a specified link function and conditional variances that are specified by a variance function known prior weights and a scale factor the random effects are assumed to be normally distributed with mean zero and dispersion matrix depending on unknown variance components for problems involving time series spatial aggregation and smoothing the dispersion may be specified in terms of a rank deficient inverse covariance matrix approximation of the marginal quasi likelihood using laplace s method leads eventually to estimating equations based on penalized quasilikelihood or pql for the mean parameters and pseudo likelihood for the variances im

exponential dispersion models on etudie les proprietes generales de la classe des modeles de dispersion exponentiels qui est la generalisation multivariable des modeles de distribution d erreur de nelder et wedderburn 1972

maximum likelihood estimation and large sample inference for generalized linear and nonlinear regression models summary the class of generalized linear models is extended to allow for correlated observations nonlinear models and error distributions not of the exponential family form the extended class of models include a number of important examples particularly of the composite transformational type large sample inference and maximum likelihood estimation for the extended class of generalized linear models are discussed and the analysis of deviance is generalized to the extended class of models calculation of the maximum likelihood estimate for a general likelihood by fisher s scoring method and a related method is considered and the relation with the gauss newton method is discussed

some covariance models for longitudinal count data with overdispersion a family of covariance models for longitudinal counts with predictive covariates is presented these models account for overdispersion heteroscedasticity and dependence among repeated observations the approach is a quasi likelihood regression similar to the formulation given by liang and zeger 1986 biometrika 73 13 22 generalized estimating equations for both the covariate parameters and the variance covariance parameters are presented large sample properties of the parameter estimates are derived the proposed methods are illustrated by an analysis of epileptic seizure count data arising from a study of progabide as an adjuvant therapy for partial seizures

an interval estimate for the interclass correlation in beta binomial sampling the beta binomial distribution has been frequently applied to model intraclass correlation among dichotomous responses in cluster sampling to avoid using sophisticated numerical procedures to calculate maximum likelihood estimates many authors have proposed different moment estimators most of them focused their discussion however on point estimation only in this paper we have developed a closed form interval estimate for the intraclass correlation and have shown it has asymptotically correct coverage we have further used a monte carlo simulation to evaluate the performance of the proposed interval estimation in a variety of situations we have found that when the number of clusters is moderate 50 the proposed interval estimation performs reasonably well except for the situation where the intraclass correlation is large and the underlying mean response probability is small even when the number of clusters is as small as 20 the proposed interval estimation may still be useful in the situation where the intraclass correlation is small we have also extended the proposed interval estimate to accommodate stratified sampling in which each stratum has a different mean response probability but has a common intraclass correlation

rational variance functions exponential dispersion models play an important role in the context of generalized linear models where error distributions other than the normal are considered any statistical model expressible in terms of a variance mean relation v omega leads to an exponential dispersion model provided that v omega is a variance function of a natural exponential family here omega is the domain of means and v is the variance function of the natural exponential family therefore it is of a particular interest to examine whether a pair v omega can serve as the variance function of a natural exponential family in this study we consider the case where omega is bounded and examine whether v can be the restriction to omega of a rational function vanishing at the boundary points of omega the class of such functions is large and contains the important subclass of polynomials it is shown that apart from the binomial family possessing a quadratic variance function and affine transformations thereof there exists no natural exponential family with variance function belonging to this class such a result implies in particular that the only variance functions of natural exponential families among polynomials of at least third degree are those restricted to unbounded domains omega

parameter orthogonality and approximate conditional inference on propose une statistique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations etant donne les estimateurs du maximum de vraisemblance pour les parametres de nuisance

consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models generalized linear models are used for regression analysis in a number of cases including categorical responses where the classical assumptions are violated the statistical analysis of such models is based on the asymptotic properties of the maximum likelihood estimator we present mild general conditions which respectively assure weak or strong consistency or asymptotic normality most of the previous work has been concerned with natural link functions in this case our normality condition though obtained by a different approach is closely related to a condition of haberman 1977a examples show how the general conditions reduce to weak requirements for special exponential families further for regressors with a compact range sufficient conditions are given which do not involve the unknown parameter and are therefore easy to check in practice responses with a bounded range e g categorical responses and stochastic regressors also are treated

dispersion models and longitudinal data analysis dispersion models provide a flexible class of non normal distributions with many potential applications in biostatistics accommodating a wide range of continuous discrete and mixed data starting with liang and zeger s generalized estimating equation method we review some recent applications of dispersion models in longitudinal data analysis including state space models based on the tweedie class of exponential dispersion models in medical applications the latent process of a state space model may often be interpreted as an unobserved potential morbidity process which is modelled as a function of time varying covariates by allowing a multivariate response vector of symptoms the model integrates several response variables of mixed types into a single model for growth curve models the latent process reflects the true growth

a review of some extensions to generalized linear models although generalized linear models are reasonably well known they are not as widely used in medical statistics as might be appropriate with the exception of logistic log linear and some survival models at the same time the generalized linear modelling methodology is decidedly outdated in that more powerful methods involving wider classes of distributions non linear regression censoring and dependence among responses are required limitations of the generalized linear modelling approach include the need for the iterated weighted least squares iwls procedure for estimation and deviances for inferences these restrict the class of models that can be used and do not allow direct comparisons among models from different distributions powerful non linear optimization routines are now available and comparisons can more fruitfully be made using the complete likelihood function the link function is an artefact necessary for iwls to function with linear models but that disappears once the class is extended to truly non linear models restricting comparisons of responses under different treatments to differences in means can be extremely misleading if the shape of the distribution is changing this may involve changes in dispersion or of other shape related parameters such as the skewness in a stable distribution with the treatments or covariates any exact likelihood function defined as the probability of the observed data takes into account the fact that all observable data are interval censored thus directly encompassing the various types of censoring possible with duration type data in most situations this can now be as easily used as the traditional approximate likelihood based on densities finally methods are required for incorporating dependencies among responses in models including conditioning on previous history and on random effects one important procedure for constructing such likelihoods is based on kalman filtering

gaussian markov distributions over finite graphs on s interesse aux modeles de selection de covariance introduites par dempster 1972 et etudies par wermuth 1976

modelling data containing exact zeroes using zero degrees of freedom on etablit l existence et l unicite des estimations du maximum de vraisemblance pour la distribution d echelle du khi deux non centre a zero degre de liberte

random sampling of skewed distributions implies taylor s power law of fluctuation scaling taylor s law tl a widely verified quantitative pattern in ecology and other sciences describes the variance in a species population density or other nonnegative quantity as a power law function of the mean density or other nonnegative quantity approximately variance a mean b a 0 multiple mechanisms have been proposed to explain and interpret tl here we show analytically that observations randomly sampled in blocks from any skewed frequency distribution with four finite moments give rise to tl we do not claim this is the only way tl arises we give approximate formulae for the tl parameters and their uncertainty in computer simulations and an empirical example using basal area densities of red oak trees from black rock forest our formulae agree with the estimates obtained by least squares regression our results show that the correlated sampling variation of the mean and variance of skewed distributions is statistically sufficient to explain tl under random sampling without the intervention of any biological or behavioral mechanisms this finding connects tl with the underlying distribution of population density or other nonnegative quantity and provides a baseline against which more complex mechanisms of tl can be compared

sup norm approximation bounds for networks through probabilistic methods we consider the problem of approximating a smooth target function and its derivatives by networks involving superpositions and translations of a fixed activation function the approximation is with respect to the sup norm and the rate is shown to be of order o n sup 1 2 that is the rate is independent of the dimension d the results apply to neural and wavelet networks and extend the work of barren see proc 7th yale workshop on adaptive and learning systems may 1992 and ibid vol 39 p 930 1993 the approach involves probabilistic methods based on central limit theorems for empirical processes indexed by classes of functions

analysis of contingency tables by ideal point discriminant analysis cross classified data are frequently encountered in behavioral and social science research the loglinear model and dual scaling correspondence analysis are two representative methods of analyzing such data an alternative method based on ideal point discriminant analysis da is proposed for analysis of contingency tables which in a certain sense encompasses the two existing methods a variety of interesting structures can be imposed on rows and columns of the tables through manipulations of predictor variables and or as direct constraints on model parameters this along with maximum likelihood estimation of the model parameters allows interesting model comparisons this is illustrated by the analysis of several data sets