some aspects of the spline smoothing approach to non parametric regression curve fitting non parametric regression using cubic splines is an attractive flexible and widely applicable approach to curve estimation although the basic idea was formulated many years ago the method is not as widely known or adopted as perhaps it should be the topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology particular subjects covered include the basic principles of the method the relation with moving average and other smoothing methods the automatic choice of the amount of smoothing and the use of residuals for diagnostic checking and model adaptation the question of providing inference regions for curves and for relevant properties of curves is approached via a finite dimensional bayesian formulation

autoregressive spatial smoothing and temporal spline smoothing for mapping rates this article proposes generalized additive mixed models for the analysis of geographic and temporal variability of mortality rates this class of models accommodates random spatial effects and fixed and random temporal components spatiotemporal models that use autoregressive local smoothing across the spatial dimension and b spline smoothing over the temporal dimension are developed the objective is the identification of temporal trends and the production of a series of smoothed maps from which spatial patterns of mortality risks can be monitored over time regions with consistently high rate estimates may be followed for further investigation the methodology is illustrated by analysis of british columbia infant mortality data

time varying coefficient models for the analysis of air pollution and health outcome data in this article a time varying coefficient model is developed to examine the relationship between adverse health and short term acute exposure to air pollution this model allows the relative risk to evolve over time which may be due to an interaction with temperature or from a change in the composition of pollutants such as particulate matter over time the model produces a smooth estimate of these time varying effects which are not constrained to follow a fixed parametric form set by the investigator instead the shape is estimated from the data using penalized natural cubic splines poisson regression models using both quasi likelihood and bayesian techniques are developed with estimation performed using an iteratively re weighted least squares procedure and markov chain monte carlo simulation respectively the efficacy of the methods to estimate different types of time varying effects are assessed via a simulation study and the models are then applied to data from four cities that were part of the national morbidity mortality and air pollution study

stable and efficient multiple smoothing parameter estimation for generalized additive models representation of generalized additive models gam s using penalized regression splines allows gam s to be employed in a straightforward manner using penalized regression methods not only is inference facilitated by this approach but it is also possible to integrate model selection in the form of smoothing parameter selection into model fitting in a computationally efficient manner using well founded criteria such as generalized cross validation the current fitting and smoothing parameter selection methods for such models are usually effective but do not provide the level of numerical stability to which users of linear regression packages for example are accustomed in particular the existing methods cannot deal adequately with numerical rank deficiency of the gam fitting problem and it is not straightforward to produce methods that can do so given that the degree of rank deficiency can be smoothing parameter dependent in addition models with the potential flexibility of gam s can also present

automatic smoothing of regression functions in generalized linear models abstract we consider the penalized likelihood method for estimating nonparametric regression functions in generalized linear models nelder and wedderburn 1972 and present a generalized cross validation procedure for empirically assessing an appropriate amount of smoothing in these estimates asymptotic arguments and numerical simulations are used to show that the generalized cross validatory procedure preforms well from the point of view of a weighted mean squared error criterion the methodology adds to the battery of graphical tools for model building and checking within the generalized linear model framework included are two examples motivated by medical and horticultural applications

trace pursuit a general framework for model free variable selection abstractwe propose trace pursuit for model free variable selection under the sufficient dimension reduction paradigm two distinct algorithms are proposed stepwise trace pursuit and forward trace pursuit stepwise trace pursuit achieves selection consistency with fixed p forward trace pursuit can serve as an initial screening step to speed up the computation in the case of ultrahigh dimensionality the screening consistency property of forward trace pursuit based on sliced inverse regression is established finite sample performances of trace pursuit and other model free variable selection methods are compared through numerical studies supplementary materials for this article are available online

bayesian inference in semiparametric mixed models for longitudinal data we consider bayesian inference in semiparametric mixed models spmms for longitudinal data spmms are a class of models that use a nonparametric function to model a time effect a parametric function to model other covariate effects and parametric or nonparametric random effects to account for the within subject correlation we model the nonparametric function using a bayesian formulation of a cubic smoothing spline and the random effect distribution using a normal distribution and alternatively a nonparametric dirichlet process dp prior when the random effect distribution is assumed to be normal we propose a uniform shrinkage prior usp for the variance components and the smoothing parameter when the random effect distribution is modeled nonparametrically we use a dp prior with a normal base measure and propose a usp for the hyperparameters of the dp base measure we argue that the commonly assumed dp prior implies a nonzero mean of the random effect distribution even when a base measure with mean zero is specified this implies weak identifiability for the fixed effects and can therefore lead to biased estimators and poor inference for the regression coefficients and the spline estimator of the nonparametric function we propose an adjustment using a postprocessing technique we show that under mild conditions the posterior is proper under the proposed usp a flat prior for the fixed effect parameters and an improper prior for the residual variance we illustrate the proposed approach using a longitudinal hormone dataset and carry out extensive simulation studies to compare its finite sample performance with existing methods

hierarchical generalized linear models we consider hierarchical generalized linear models which allow extra error components in the linear predictors of generalized linear models the distribution of these components is not restricted to be normal this allows a broader class of models which includes generalized linear mixed models we use a generalization of henderson s joint likelihood called a hierarchical or h likelihood for inferences from hierarchical generalized linear models this avoids the integration that is necessary when marginal likelihood is used under appropriate conditions maximizing the h likelihood gives fixed effect estimators that are asymptotically equivalent to those obtained from the use of marginal likelihood at the same time we obtain the random effect estimates that are asymptotically best unbiased predictors an adjusted profile h likelihood is shown to give the required generalization of restricted maximum likelihood for the estimation of dispersion components a scaled deviance test for the goodness of fit a model selection criterion for choosing between various dispersion models and a graphical method for checking the distributional assumption of random effects are proposed the ideas of quasi likelihood and extended quasi likelihood are generalized to the new class we give examples of the poisson gamma binomial beta and gamma inverse gamma hierarchical generalized linear models a resolution is proposed for the apparent difference between population averaged and subject specific models a unified framework is provided for viewing and extending many existing methods

semiparametric models for longitudinal data with application to cd4 cell numbers in hiv seroconverters summary the paper describes a semiparametric model for longitudinal data which is illustrated by its application to data on the time evolution of cd4 cell numbers in hiv seroconverters the essential ingredients of the model are a parametric linear model for covariate adjustment a nonparametric estimation of a smooth time trend serial correlation between measurements on an individual subject and random measurement error a back fitting algorithm is used in conjunction with a cross validation prescription to fit the model a notable feature in the application is that the onset of hiv infection is associated with a sudden drop in cd4 cells followed by a longer term slower decay the model is also used to estimate an individual s curve by combining his data with the population curve shrinkage toward the population mean trajectory is controlled in a natural way by the estimated covariance structure of the data

a general maximum likelihood analysis of variance components in generalized linear models this paper describes an em algorithm for nonparametric maximum likelihood ml estimation in generalized linear models with variance component structure the algorithm provides an alternative analysis to approximate mql and pql analyses mcgilchrist and aisbett 1991 biometrical journal 33 131 141 breslow and clayton 1993 journal of the american statistical association 88 9 25 mcgilchrist 1994 journal of the royal statistical society series b 56 61 69 goldstein 1995 multilevel statistical models and to gee analyses liang and zeger 1986 biometrika 73 13 22 the algorithm first given by hinde and wood 1987 in longitudinal data analysis 110 126 is a generalization of that for random effect models for overdispersion in generalized linear models described in aitkin 1996 statistics and computing 6 251 262 the algorithm is initially derived as a form of gaussian quadrature assuming a normal mixing distribution but with only slight variation it can be used for a completely unknown mixing distribution giving a straightforward method for the fully nonparametric ml estimation of this distribution this is of value because the ml estimates of the glm parameters can be sensitive to the specification of a parametric form for the mixing distribution the nonparametric analysis can be extended straightforwardly to general random parameter models with full npml estimation of the joint distribution of the random parameters this can produce substantial computational saving compared with full numerical integration over a specified parametric distribution for the random parameters a simple method is described for obtaining correct standard errors for parameter estimates when using the em algorithm several examples are discussed involving simple variance component and longitudinal models and small area estimation

mixed effects smoothing spline analysis of variance we propose a general family of nonparametric mixed effects models smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function the random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters these parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method we derive a connection between a nonparametric mixed effects model and a linear mixed effects model this connection suggests a way of fitting a nonparametric mixed effects model by using existing programs the classical two way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis of variance decompositions to build nonparametric mixed effects models similarly to the classical analysis of variance components of these nonparametric mixed effects models can be interpreted as main effects and interactions the penalized likelihood estimates of the fixed effects in a two way mixed model are extensions of james stein shrinkage estimates to correlated observations in an example three nested nonparametric mixed effects models are fitted to a longitudinal data set

bayesian confidence intervals for the cross validated smoothing spline summary we consider the model y t g ti ei i 17 2 n where g t t 0 1 is a smooth function and the ei are independent n 0 a2 errors with g2 unknown the cross validated smoothing spline can be used to estimate g non parametrically from

the performance of cross validation and maximum likelihood estimators of spline smoothing parameters abstract an important aspect of nonparametric regression by spline smoothing is the estimation of the smoothing parameter in this article we report on an extensive simulation study that investigates the finite sample performance of generalized cross validation cross validation and marginal likelihood estimators of the smoothing parameter in splines of orders 2 and 3 the performance criterion for both the estimate of the function and its first derivative is measured by the square root of integrated squared error marginal likelihood using splines of degree 5 emerges as an attractive alternative to the other estimators in that it usually outperforms them and is also faster to compute

bayesian semiparametric regression for longitudinal binary processes with missing data longitudinal studies with binary repeated measures are widespread in biomedical research marginal regression approaches for balanced binary data are well developed whereas for binary process data where measurement times are irregular and may differ by individuals likelihood based methods for marginal regression analysis are less well developed in this article we develop a bayesian regression model for analyzing longitudinal binary process data with emphasis on dealing with missingness we focus on the settings where data are missing at random mar which require a correctly specified joint distribution for the repeated measures in order to draw valid likelihood based inference about the marginal mean to provide maximum flexibility the proposed model specifies both the marginal mean and serial dependence structures using nonparametric smooth functions serial dependence is allowed to depend on the time lag between adjacent outcomes as well as other relevant covariates inference is fully bayesian using simulations we show that adequate modeling of the serial dependence structure is necessary for valid inference of the marginal mean when the binary process data are mar longitudinal viral load data from the hiv epidemiology research study are analyzed for illustration

breather solutions in the diffraction managed nls equation we show the existence of localized breather solutions in an averaged version of the discrete nonlinear schrodinger equation nls with diffraction management a system that models coupled waveguide arrays with periodic diffraction management geometries the breather solutions are constrained extrema of the hamiltonian of the averaged system and their existence is shown by a discrete version of the concentration compactness principle the main assumptions are that the averaged diffraction is sufficiently small compared to strength of the nonlinearity and that the sign of the nonlinearity corresponds to the focusing case an interesting feature of the problem is that the nonlinear interaction between neighboring lattice sites can be large and is of infinite range on the other hand the interaction decays rapidly at sufficiently large distances and this plays an important role in the proof the results also apply to higher dimensional lattices and to the discrete nls equation

approximate inference in generalized linear mixed models statistical approaches to overdispersion correlated errors shrinkage estimation and smoothing of regression relationships may be encompassed within the framework of the generalized linear mixed model glmm given an unobserved vector of random effects observations are assumed to be conditionally independent with means that depend on the linear predictor through a specified link function and conditional variances that are specified by a variance function known prior weights and a scale factor the random effects are assumed to be normally distributed with mean zero and dispersion matrix depending on unknown variance components for problems involving time series spatial aggregation and smoothing the dispersion may be specified in terms of a rank deficient inverse covariance matrix approximation of the marginal quasi likelihood using laplace s method leads eventually to estimating equations based on penalized quasilikelihood or pql for the mean parameters and pseudo likelihood for the variances im

semiparametric regression using variational approximations semiparametric regression offers a flexible framework for modeling nonlinear relationships between a response and covariates a prime example are generalized additive models gams where splines s

semiparametric stochastic mixed models for longitudinal data abstract we consider inference for a semiparametric stochastic mixed model for longitudinal data this model uses parametric fixed effects to represent the covariate effects and an arbitrary smooth function to model the time effect and accounts for the within subject correlation using random effects and a stationary or nonstationary stochastic process we derive maximum penalized likelihood estimators of the regression coefficients and the nonparametric function the resulting estimator of the nonparametric function is a smoothing spline we propose and compare frequentist inference and bayesian inference on these model components we use restricted maximum likelihood to estimate the smoothing parameter and the variance components simultaneously we show that estimation of all model components of interest can proceed by fitting a modified linear mixed model we illustrate the proposed method by analyzing a hormone dataset and evaluate its performance through simulations

additive extensions to generalized estimating equation methods the methods underlying vector generalized additive models are extended to provide additive extensions to the generalized estimating equations approaches to multivariate regression problems of liang and zeger and the subsequent literature the methods are illustrated by two examples using correlated binary data concerning the presence or absence of several plant species at the same geographical site and chest pain in male workers from a workforce study with minor modification the extensions are shown to apply to longitudinal data

a comparison of gcv and gml for choosing the smoothing parameter in the generalized spline smoothing problem the partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood gml estimate for the smoothing parameter then this estimate is compared with the generalized cross validation gcv estimate both analytically and by monte carlo methods the comparison is based on a predictive mean square error criteria it is shown that if the true unknown function being estimated is smooth in a sense to be defined then the gml estimate undersmooths relative to the gcv estimate and the predictive mean square error using the gml estimate goes to zero at a slower rate than the mean square error using the gcv estimate if the true function is rough then the gcv and gml estimates have asymptotically similar behavior a monte carlo experiment was designed to see if the asymptotic results in the smooth case were evident in small sample sizes mixed results were obtained for n 32 gcv was somewhat better than gml for n 64 and gcv was decidedly superior for n 128 in the n 32 case gcv was better for smaller sigma 2 and the comparison close for larger sigma 2 the theoretical results are shown to extend to the generalized spline smoothing model which includes the estimate of functions given noisy values of various integrals of them

inference in semiparametric dynamic models for binary longitudinal data this article deals with the analysis of a hierarchical semiparametric model for dynamic binary longitudinal responses the main complicating components of the model are an unknown covariate function and serial correlation in the errors existing estimation methods for models with these features are of o n3 where n is the total number of observations in the sample therefore nonparametric estimation is largely infeasible when the sample size is large as in typical in the longitudinal setting here we propose a new o n markov chain monte carlo based algorithm for estimation of the nonparametric function when the errors are correlated thus contributing to the growing literature on semiparametric and nonparametric mixed effects models for binary data in addition we address the problem of model choice to enable the formal comparison of our semiparametric model with competing parametric and semiparametric specifications the performance of the methods is illustrated with detailed studies involving simul

generalized linear models with random effects a gibbs sampling approach abstract generalized linear models have unified the approach to regression for a wide variety of discrete continuous and censored response variables that can be assumed to be independent across experimental units in applications such as longitudinal studies genetic studies of families and survey sampling observations may be obtained in clusters responses from the same cluster cannot be assumed to be independent with linear models correlation has been effectively modeled by assuming there are cluster specific random effects that derive from an underlying mixing distribution extensions of generalized linear models to include random effects has thus far been hampered by the need for numerical integration to evaluate likelihoods in this article we cast the generalized linear random effects model in a bayesian framework and use a monte carlo method the gibbs sampler to overcome the current computational limitations the resulting algorithm is flexible to easily accommodate changes in the number

maximum likelihood algorithms for generalized linear mixed models abstract maximum likelihood algorithms are described for generalized linear mixed models i show how to construct a monte carlo version of the em algorithm propose a monte carlo newton raphson algorithm and evaluate and improve the use of importance sampling ideas calculation of the maximum likelihood estimates is feasible for a wide variety of problems where they were not previously i also use the newton raphson algorithm as a framework to compare maximum likelihood to the joint maximization or penalized quasi likelihood methods and explain why the latter can perform poorly

generalized nonlinear modeling with multivariate free knot regression splines a bayesian method is presented for the nonparametric modeling of univariate and multivariate non gaussian response data data adaptive multivariate regression splines are used where the number and location of the knot points are treated as random the posterior model space is explored using a reversible jump markov chain monte carlo sampler computational difficulties are partly alleviated by introducing a random residual effect in the model that leaves many of the posterior conditional distributions of the model parameters in standard form the use of the latent residual effect provides a convenient vehicle for modeling correlation in multivariate response data and as such our method can be seen to generalize the seemingly unrelated regression model to non gaussian data we illustrate the method on a number of examples including two previously unpublished datasets relating to the spatial smoothing of multivariate accident data in texas and the modeling of credit card use across multiple retail sectors

accurate approximations for posterior moments and marginal densities abstract this article describes approximations to the posterior means and variances of positive functions of a real or vector valued parameter and to the marginal posterior densities of arbitrary i e not necessarily positive parameters these approximations can also be used to compute approximate predictive densities to apply the proposed method one only needs to be able to maximize slightly modified likelihood functions and to evaluate the observed information at the maxima nevertheless the resulting approximations are generally as accurate and in some cases more accurate than approximations based on third order expansions of the likelihood and requiring the evaluation of third derivatives the approximate marginal posterior densities behave very much like saddle point approximations for sampling distributions the principal regularity condition required is that the likelihood times prior be unimodal

smoothing spline models for the analysis of nested and crossed samples of curves abstract we introduce a class of models for an additive decomposition of groups of curves stratified by crossed and nested factors generalizing smoothing splines to such samples by associating them with a corresponding mixed effects model the models are also useful for imputation of missing data and exploratory analysis of variance we prove that the best linear unbiased predictors blups from the extended mixed effects model correspond to solutions of a generalized penalized regression where smoothing parameters are directly related to variance components and we show that these solutions are natural cubic splines the model parameters are estimated using a highly efficient implementation of the em algorithm for restricted maximum likelihood reml estimation based on a preliminary eigenvector decomposition variability of computed estimates can be assessed with asymptotic techniques or with a novel hierarchical bootstrap resampling scheme for nested mixed effects models our methods are applied to me

variable selection and model averaging in semiparametric overdispersed generalized linear models we express the mean and variance terms in a double exponential regression model as additive functions of the predictors and use bayesian variable selection to determine which predictors enter the model and whether they enter linearly or flexibly when the variance term is null we obtain a generalized additive model which becomes a generalized linear model if the predictors enter the mean linearly the model is estimated using markov chain monte carlo simulation and the methodology is illustrated using real and simulated data sets

inference in generalized additive mixed modelsby using smoothing splines generalized additive mixed models are proposed for overdispersed and correlated data which arise frequently in studies involving clustered hierarchical and spatial designs this class of models allows flexible functional dependence of an outcome variable on covariates by using nonparametric regression while accounting for correlation between observations by using random effects we estimate nonparametric functions by using smoothing splines and jointly estimate smoothing parameters and variance components by using marginal quasi likelihood because numerical integration is often required by maximizing the objective functions double penalized quasi likelihood is proposed to make approximate inference frequentist and bayesian inferences are compared a key feature of the method proposed is that it allows us to make systematic inference on all model components within a unified parametric mixed model framework and can be easily implemented by fitting a working generalized linear mixed model by using existing statistical software a bias correction procedure is also proposed to improve the performance of double penalized quasi likelihood for sparse data we illustrate the method with an application to infectious disease data and we evaluate its performance through simulation

semiparametric frailty models for clustered failure time data summary we consider frailty models with additive semiparametric covariate effects for clustered failure time data we propose a doubly penalized partial likelihood dppl procedure to estimate the nonparametric functions using smoothing splines we show that the dppl estimators could be obtained from fitting an augmented working frailty model with parametric covariate effects whereas the nonparametric functions being estimated as linear combinations of fixed and random effects and the smoothing parameters being estimated as extra variance components this approach allows us to conveniently estimate all model components within a unified frailty model framework we evaluate the finite sample performance of the proposed method via a simulation study and apply the method to analyze data from a study of sexually transmitted infections sti

bias correction in generalized linear mixed models with multiple components of dispersion abstract general formulas are derived for the asymptotic bias in regression coefficients and variance components estimated by penalized quasi likelihood pql in generalized linear mixed models with canonical link function and multiple sets of independent random effects easily computed correction matrices result in variance component estimates that have satisfactory asymptotic behavior for small values of the variance components and significantly reduce bias for larger values both first order and second order correction procedures are developed for regression coefficients estimated by pql the methods are illustrated through an analysis of an experiment on salamander matings involving crossed male and female random effects and their properties are evaluated in a simulation study

longitudinal data analysis using generalized linear models summary this paper proposes an extension of generalized linear models to the analysis of longitudinal data we introduce a class of estimating equations that give consistent estimates of the regression parameters and of their variance under mild assumptions about the time dependence the estimating equations are derived without specifying the joint distribution of a subject s observations yet they reduce to the score equations for multivariate gaussian outcomes asymptotic theory is presented for the general class of estimators specific cases in which we assume independence m dependence and exchangeable correlation structures from each subject are discussed efficiency of the proposed estimators in two simple situations is considered the approach is closely related to quasi likelih ood some key ironh estimating equation generalized linear model longitudinal data quasi likelihood repeated measures