some aspects of the spline smoothing approach to non parametric regression curve fitting non parametric regression using cubic splines is an attractive flexible and widely applicable approach to curve estimation although the basic idea was formulated many years ago the method is not as widely known or adopted as perhaps it should be the topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology particular subjects covered include the basic principles of the method the relation with moving average and other smoothing methods the automatic choice of the amount of smoothing and the use of residuals for diagnostic checking and model adaptation the question of providing inference regions for curves and for relevant properties of curves is approached via a finite dimensional bayesian formulation

splines in statistics abstract this is a survey article that attempts to synthesize a broad variety of work on splines in statistics splines are presented as a nonparametric function estimating technique after a general introduction to the theory of interpolating and smoothing splines splines are treated in the nonparametric regression setting the method of cross validation for choosing the smoothing parameter is discussed and the general multivariate regression surface estimation problem is addressed an extensive discussion of splines as nonparametric density estimators is followed by a discussion of their role in time series analysis a comparison of the spline and isotonic regression methodologies leads to a formulation of a hybrid estimator the closing section provides a brief overall summary and formulates a number of open unsolved problems relating to splines in statistics

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

a fast and efficient cross validation method for smoothing parameter choice in spline regression abstract the spline smoothing approach to nonparametric regression is considered with particular reference to the problem of choosing how much to smooth both computational and statistical aspects of the method of cross validation are discussed an approximate cross validation method is proposed that requires a small linear amount of computer time a simulation study indicates that the method has very good statistical properties some mathematical justification and motivation are also provided

smoothing splines regression derivatives and deconvolution abstract the statistical properties of a cubic smoothing spline and its derivative are analyzed it is shown that unless unnatural boundary conditions hold the integrated squared bias is dominated by local effects near the boundary similar effects are shown to occur in the regularized solution of a translation kernel integral equation these results are derived by developing a fourier representation for a smoothing spline author

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

discrete mechanics of arbitrary order abstract on the basis of greenspan s discrete mechanics a new technique for obtaining difference dynamical equations of arbitrary high order is developed the presented method conserves most of the constants of motion numerical examples illustrate this fact

generalized linear array models with applications to multidimensional smoothing summary data with an array structure are common in statistics and the design or regression matrix for analysis of such data can often be written as a kronecker product factorial designs contingency tables and smoothing of data on multidimensional grids are three such general classes of data and models in such a setting we develop an arithmetic of arrays which allows us to define the expectation of the data array as a sequence of nested matrix operations on a coefficient array we show how this arithmetic leads to low storage high speed computation in the scoring algorithm of the generalized linear model we refer to a generalized linear array model and apply the methodology to the smoothing of multidimensional arrays we illustrate our procedure with the analysis of three data sets mortality data indexed by age at death and year of death spatially varying microarray background data and disease incidence data indexed by age at death year of death and month of death

low rank scale invariant tensor product smooths for generalized additive mixed models summary a general method for constructing low rank tensor product smooths for use as components of generalized additive models or generalized additive mixed models is presented a penalized regression approach is adopted in which tensor product smooths of several variables are constructed from smooths of each variable separately these marginal smooths being represented using a low rank basis with an associated quadratic wiggliness penalty the smooths offer several advantages i they have one wiggliness penalty per covariate and are hence invariant to linear rescaling of covariates making them useful when there is no natural way to scale covariates relative to each other ii they have a useful tuneable range of smoothness unlike single penalty tensor product smooths that are scale invariant iii the relatively low rank of the smooths means that they are computationally efficient iv the penalties on the smooths are easily interpretable in terms of function shape v the smooths can be generated completely automatically from any marginal smoothing bases and associated quadratic penalties giving the modeler considerable flexibility to choose the basis penalty combination most appropriate to each modeling task and vi the smooths can easily be written as components of a standard linear or generalized linear mixed model allowing them to be used as components of the rich family of such models implemented in standard software and to take advantage of the efficient and stable computational methods that have been developed for such models a small simulation study shows that the methods can compare favorably with recently developed smoothing spline anova methods

stable and efficient multiple smoothing parameter estimation for generalized additive models representation of generalized additive models gam s using penalized regression splines allows gam s to be employed in a straightforward manner using penalized regression methods not only is inference facilitated by this approach but it is also possible to integrate model selection in the form of smoothing parameter selection into model fitting in a computationally efficient manner using well founded criteria such as generalized cross validation the current fitting and smoothing parameter selection methods for such models are usually effective but do not provide the level of numerical stability to which users of linear regression packages for example are accustomed in particular the existing methods cannot deal adequately with numerical rank deficiency of the gam fitting problem and it is not straightforward to produce methods that can do so given that the degree of rank deficiency can be smoothing parameter dependent in addition models with the potential flexibility of gam s can also present

generalized additive models for exceedances of high thresholds with an application to return level estimation for u s wind gusts generalized additive model gam forms offer a flexible approach to capturing marginal variation such forms are used here to represent distributional variation in extreme values and presented in t

smoothing spline gaussian regression more scalable computation via efficient approximation summary smoothing splines via the penalized least squares method provide versatile and effective nonparametric models for regression with gaussian responses the computation of smoothing splines is generally of the order o n3 n being the sample size which severely limits its practical applicability we study more scalable computation of smoothing spline regression via certain low dimensional approximations that are asymptotically as efficient a simple algorithm is presented and the bayes model that is associated with the approximations is derived with the latter guiding the porting of bayesian confidence intervals the practical choice of the dimension of the approximating space is determined through simulation studies and empirical comparisons of the approximations with the exact solution are presented also evaluated is a simple modification of the generalized cross validation method for smoothing parameter selection which to a large extent fixes the occasional undersmoothing problem that is suffered by generalized cross validation

spline smoothing the equivalent variable kernel method on considere une approche spline pour la regression non parametrique et l estimation de courbe on montre que dans un certain sens le lissage spline correspond approximativement au lissage par une methode du noyau avec une largeur de bande dependant de la densite locale des points du plan de regression

discretized laplacian smoothing by fourier methods abstract an approach to multidimensional smoothing is introduced that is based on a penalized likelihood with a modified discretized laplacian penalty term the choice of penalty simplifies computational difficulties associated with standard multidimensional laplacian smoothing methods yet without compromising mean squared error characteristics at least on the interior of the region of interest for linear smoothing in hyper rectangular domains which has wide application in image reconstruction and restoration problems computations are carried out using fast fourier transforms nonlinear smoothing is accomplished by iterative application of the linear smoothing technique the iterative procedure is shown to be convergent under general conditions adaptive choice of the amount of smoothing is based on approximate cross validation type scores an importance sampling technique is used to estimate the degrees of freedom of the smooth the methods are implemented in one and two dimensional settings some i

arbitrary order numerical methods conserving integrals for solving dynamic equations abstract a number of problems in celestial mechanics some hamiltonian systems and non hamiltonian ones may be solved numerically by a modification of polynomial extrapolation which conserves integrals it means that it is possible to get numerical solutions of dynamic equations which possess the same properties that follow from the theoretical study this paper is a survey of problems for which this modification may be applied

a new statistical approach to detecting significant activation in functional mri there are many ways to detect activation patterns in a time series of observations at a single voxel in a functional magnetic resonance imaging study the critical problem is to estimate the statistical significance which depends on the estimation of both the magnitude of the response to the stimulus and the serial dependence of the time series and especially on the assumptions made in that estimation we show that for experimental designs with periodic stimuli only a few aspects of the serial dependence are important and these can be estimated reliably via nonparametric estimation of the spectral density of the time series whereas existing techniques are biased by their assumptions the linear model with stationary serially dependent errors can be analyzed entirely in frequency domain and doing so provides many insights in particular we introduce a technique to detect periodic activations and show that it has a distribution theory that enables us to assign significance levels down to 1 in 100 000 levels which are needed when a whole brain image is under consideration nonparametric spectral density estimation is shown to be self calibrating and accurate when compared to several other time domain approaches the technique is especially resistant to high frequency artefacts that we have found in some datasets and we demonstrate that time domain approaches may be sufficiently susceptible to these effects to give misleading results the method is easily generalized to handle event related designs we found it necessary to consider the trends in the time series carefully and use nonlinear filters to remove the trends and robust techniques to remove spikes using this in connection with our techniques allows us to detect activations in clumps of a few even one voxel in periodic designs yet produce essentially no false positive detections at any voxels in null datasets

plurality and resemblance in fmri data analysis abstract we apply nine analytic methods employed currently in imaging neuroscience to simulated and actual bold fmri signals and compare their performances under each signal type starting with baseline time series generated by a resting subject during a null hypothesis study we compare method performance with embedded focal activity in these series of three different types whose magnitudes and time courses are simple convolved with spatially varying hemodynamic responses and highly spatially interactive we then apply these same nine methods to bold fmri time series from contralateral primary motor cortex and ipsilateral cerebellum collected during a sequential finger opposition study paired comparisons of results across methods include a voxel specific concordance correlation coefficient for reproducibility and a resemblance measure that accommodates spatial autocorrelation of differences in activity surfaces receiver operating characteristic curves show considerable model differences in ranges less than 10 significance level false positives and greater than 80 power true positives concordance and resemblance measures reveal significant differences between activity surfaces in both data sets these measures can assist researchers by identifying groups of models producing similar and dissimilar results and thereby help to validate consolidate and simplify reports of statistical findings a pluralistic strategy for fmri data analysis can uncover invariant and highly interactive relationships between local activity foci and serve as a basis for further discovery of organizational principles of the brain results also suggest that a pluralistic empirical strategy coupled formally with substantive prior knowledge can help to uncover new brain behavior relationships that may remain hidden if only a single method is employed

generalized additive models some applications abstract generalized additive models have the form x fj x j where might be the regression function in a multiple regression or the logistic transformation of the posterior probability pr y 1 x in a logistic regression in fact these models generalize the whole family of generalized linear models x x where x g x is some transformation of the regression function we use the local scoring algorithm to estimate the functions fj xj nonparametrically using a scatterplot smoother as a building block we demonstrate the models in two different analyses a nonparametric analysis of covariance and a logistic regression the procedure can be used as a diagnostic tool for identifying parametric transformations of the covariates in a standard linear analysis a variety of inferential tools have been developed to aid the analyst in assessing the relevance and significance of the estimated functions these include confidence curves degrees of freedom estimates and approximat

invariantization of the crank nicolson method for burgers equation abstract in this article a geometric technique to construct numerical schemes for partial differential equations pdes that inherit lie symmetries is proposed the moving frame method enables one to adjust the numerical schemes in a geometric manner and systematically construct proper invariant versions of them to illustrate the method we study invariantization of the crank nicolson scheme for burgers equation with careful choice of normalization equations the invariantized schemes are shown to surpass the standard scheme successfully removing numerical oscillation around sharp transition layers

nonparametric empirical bayes growth curve analysis abstract nonparametric and semiparametric regression have been suggested as alternatives to parametric models for growth curve analysis in this article we demonstrate that empirical bayes estimation can be used to improve linear smoothing estimates when multiple curves are available

conservative numerical methods for x f x abstract two distinctly different numerical methods are developed for solving the conservative initial value problem x f x x o dot x o both methods conserve exactly the same total energy as does the given differential equation computer examples are described and

automatic smoothing of regression functions in generalized linear models abstract we consider the penalized likelihood method for estimating nonparametric regression functions in generalized linear models nelder and wedderburn 1972 and present a generalized cross validation procedure for empirically assessing an appropriate amount of smoothing in these estimates asymptotic arguments and numerical simulations are used to show that the generalized cross validatory procedure preforms well from the point of view of a weighted mean squared error criterion the methodology adds to the battery of graphical tools for model building and checking within the generalized linear model framework included are two examples motivated by medical and horticultural applications

a flexible approach to parametric inference in nonlinear and time varying time series models many structural break and regime switching models have been used with macroeconomic and financial data in this paper we develop an extremely flexible modeling approach which can accommodate virtually any of these specifications we build on earlier work showing the relationship between flexible functional forms and random variation in parameters our contribution is based around the use of priors on the time variation that is developed from considering a hypothetical reordering of the data and distance between neighboring reordered observations the range of priors produced in this way can accommodate a wide variety of nonlinear time series models including those with regime switching and structural breaks by allowing the amount of random variation in parameters to depend on the distance between reordered observations the parameters can evolve in a wide variety of ways allowing for everything from models exhibiting abrupt change e g threshold autoregressive models or standard structural break models to those which allow for a gradual evolution of parameters e g smooth transition autoregressive models or time varying parameter models bayesian econometric methods for inference are developed for estimating the distance function and types of hypothetical reordering conditional on a hypothetical reordering and distance function a simple reordering of the actual data allows us to estimate our models with standard state space methods by a simple adjustment to the measurement equation we use artificial data to show the advantages of our approach before providing two empirical illustrations involving the modeling of real gdp growth

varying coefficient models we explore a class of regression and generalized regression models in which the coefficients are allowed to vary as smooth functions of other variables general algorithms are presented for estimating the models flexibly and some examples are given this class of models ties together generalized additive models and dynamic generalized linear models into one common framework when applied to the proportional hazards model for survival data this approach provides a new way of modelling departures from the proportional hazards assumption

critical point theory for nondifferentiable functionals and applications abstract we consider minimization results for locally lipschitzian functionals on a banach space by introducing a compactness condition of the palais smale type which is suggested by the variational principle of ekeland applications are given to some semilinear elliptic problems with discontinuous nonlinearities

regularization of wavelet approximations in this paper we introduce nonlinear regularized wavelet estimators for estimating nonparametric regression functions when sampling points are not uniformly spaced the approach can apply readily to many other statistical contexts various new penalty functions are proposed the hard thresholding and soft thresholding estimators of donoho and johnstone are specific members of nonlinear regularized wavelet estimators they correspond to the lower and upper envelopes of a class of the penalized least squares estimators necessary conditions for penalty functions are given for regularized estimators to possess thresholding properties oracle inequalities and universal thresholding parameters are obtained for a large class of penalty functions the sampling properties of nonlinear regularized wavelet estimators are established and are shown to be adaptively minimax to efficiently solve penalized least squares problems nonlinear regularized sobolev interpolators nrsi are proposed as initial estimators wh

the l 1 method for robust nonparametric regression abstract consider the problem of estimating the regression function underlying a set of data that is contaminated by a heavy tailed error distribution the two standard approaches to such a problem are each flawed robust parametric least squares is appropriate only if there is good reason to believe that the underlying function has some particular form whereas most nonparametric regression methods are asymptotically equivalent to kernel regression methods which are not resistant against outliers existing algorithms for robustifying nonparametric regression procedures use either nonlinear optimization of an influence function or iterative solution of local polynomial fitting using reweighted least squares neither of these two approaches combines computational ease with asymptotic theoretical results furthermore application of the robust procedure has been limited almost exclusively to the case of a single explanatory variable with the response variable in this article a new hybrid method is propose

proprietes de convergence presque complete du predicteur a noyau let z n be a mixing process which is valued in mathbb e subset mathbb r q g be a real measurable function defined on mathbb e s and k be two positive integers we suppose the existence of a function r satisfying r e g z n s z n k 1 z n n n k and estimate the function r from a sequence z i i 1 n by r n with r n u frac sum limits i k n s g z i s k u z i k 1 cdots z i h n 1 sum limits i k n s k u z i k 1 cdots z i h n 1 forall u in mathbb e k where k is a kernel of kq and h n h n 0 we give various conditions on both the sequences h n and the sequence n which is associated with z n for the two following properties uniform complete convergence to r for the functional estimator r n and complete convergence to 0 for the real random variable r n z n k 1 z n r zn k 1 z n this last result concerns the predictor r n z n k 1 z n of g z n s from z i i 1 n when the process z n is stationary and markovian of order k

on a free boundary problem abstract in this paper we will be concerned with the existence of positive solutions of the following discontinuous problem p u f u h u in u h on where is the unit ball of r n n 3 and h is the heaviside function the problem is written in an equivalent integral equation form the shape of the free boundary problem x u x is studied for when is a positive real parameter f and h are given functions

variational methods for non differentiable functionals and their applications to partial differential equations the interest in extending the theory of nonlinear partial differential equations pde to pde with discontinuous nonlinearities dnde has been increasing daily the reasons are twofold 1 many free boundary problems may be reduced to boundary value problems of dnde cf chang 16 71 chang and jiang 9 and fleishman and mahar 121 2 sometimes in dealing with a pde problem it is beneficial to put the original pde into a large category for instance dnde cf chang 7 simply speaking a dnde is as follows

mixed effects smoothing spline analysis of variance we propose a general family of nonparametric mixed effects models smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function the random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters these parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method we derive a connection between a nonparametric mixed effects model and a linear mixed effects model this connection suggests a way of fitting a nonparametric mixed effects model by using existing programs the classical two way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis of variance decompositions to build nonparametric mixed effects models similarly to the classical analysis of variance components of these nonparametric mixed effects models can be interpreted as main effects and interactions the penalized likelihood estimates of the fixed effects in a two way mixed model are extensions of james stein shrinkage estimates to correlated observations in an example three nested nonparametric mixed effects models are fitted to a longitudinal data set

tensor graphical lasso teralasso the paper introduces a multiway tensor generalization of the bigraphical lasso which uses a two way sparse kronecker sum multivariate normal model for the precision matrix to model parsimoniously conditional dependence relationships of matrix variate data based on the cartesian product of graphs we call this tensor graphical lasso generalization teralasso we demonstrate by using theory and examples that the teralasso model can be accurately and scalably estimated from very limited data samples of high dimensional variables with multiway co ordinates such as space time and replicates statistical consistency and statistical rates of convergence are established for both the bigraphical lasso and teralasso estimators of the precision matrix and estimators of its support non sparsity set respectively we propose a scalable composite gradient descent algorithm and analyse the computational convergence rate showing that the composite gradient descent algorithm is guaranteed to converge at a geometric rate to the global minimizer of the teralasso objective function finally we illustrate teralasso by using both simulation and experimental data from a meteorological data set showing that we can accurately estimate precision matrices and recover meaningful conditional dependence graphs from high dimensional complex data sets

an iterative approach to two dimensional laplacian smoothing with application to image restoration abstract smoothing methods are increasingly important in modern graphically oriented data analysis there are several fast and reliable univariate cross validated smoothing algorithms but comparable methods are not available in higher dimensions this article develops an iterative approach to two dimensional laplacian spline smoothing based on tensor product cubic b splines estimating equations are developed and solved iteratively using two line symmetric successive over relaxation with conjugate gradient acceleration an asymptotic approximation for the generalized cross validation score is described the resulting algorithm is naturally suited to two dimensional scatterplot smoothing but has potential value for image restoration as well this is illustrated on a multicolor image example along with an image restoration an uncertainty assessment based on the entropy of the marginal posterior pixel value distribution is produced this analysis particularly highlights the statistical uncertainty in resol

bayesian confidence intervals for the cross validated smoothing spline summary we consider the model y t g ti ei i 17 2 n where g t t 0 1 is a smooth function and the ei are independent n 0 a2 errors with g2 unknown the cross validated smoothing spline can be used to estimate g non parametrically from

iterative solution of systems originating from integral equations and surface interpolation a simple procedure for generating diagonal dominance in certain linear systems is presented and the iterative solution of the transformed systems is discussed the method is presented for systems encountered in the numerical solution of fredholm equations of the first kind with singular kernels and for systems arising in the interpolation of surfaces by certain sets of radial functions in the second case the method provides new sets of basis functions generalizing in some sense the univariate b splines

backfitting in smoothing spline anova a computational scheme for fitting smoothing spline anova models to large data sets with a near tensor product design is proposed such data sets are common in spatial temporal analyses the proposed scheme uses the backfitting algorithm to take advantage of the tensor product design to save both computational memory and time several ways to further speed up the backfitting algorithm such as collapsing component functions and successive over relaxation are discussed an iterative imputation procedure is used to handle the cases of near tensor product designs an application to a global historical surface air temperature data set which motivated this work is used to illustrate the scheme proposed

approximate inference in generalized linear mixed models statistical approaches to overdispersion correlated errors shrinkage estimation and smoothing of regression relationships may be encompassed within the framework of the generalized linear mixed model glmm given an unobserved vector of random effects observations are assumed to be conditionally independent with means that depend on the linear predictor through a specified link function and conditional variances that are specified by a variance function known prior weights and a scale factor the random effects are assumed to be normally distributed with mean zero and dispersion matrix depending on unknown variance components for problems involving time series spatial aggregation and smoothing the dispersion may be specified in terms of a rank deficient inverse covariance matrix approximation of the marginal quasi likelihood using laplace s method leads eventually to estimating equations based on penalized quasilikelihood or pql for the mean parameters and pseudo likelihood for the variances im

a comparison of the reinsch and speckman splines in the context of nonparametric regression spline smoothers can be used both for the estimation of the curve itself and for the estimation of the error variance we compare two approaches that have been suggested the reinsch spline i e roughness penalty and the speckman spline minimax the comparison is made on the basis of the summed mean squared error for curve estimation and on the basis of a closely related criterion for variance estimation we show in both asymptotic and small sample studies that the reinsch smoothing spline is close to optimal both as a curve and as a variance estimator

arbitrary order hamiltonian conserving numerical solutions of calogero and toda systems abstract for calogero and toda dynamical equations two numerical methods of arbitrary high order conserving the hamiltonian are developed the methods consist of modifications of conventional polynomial extrapolation with the gragg method used as a basic method the theoretical study is confirmed by a number of numerical examples

linear models for field trials smoothing and cross validation abstract spatial methods for the analysis of agricultural field experiments are represented here as smoothing methods applied simultaneously with the estimation of treatment effects selection of both the form of the smoother and the degree of smoothing required may be based on cross validation particular emphasis is placed in this paper on generalized least squares estimation ion linear models but the principle applies quite generally keywords agricultural field experiments generalized least squares incomplete block designs neighbour methods papadakis method recovery of inter block formation smoothing and spatial models

constructing a bootstrap confidence interval for the unknown concentration in radioimmunoassay the statistical problem associated with radioimmunoassay is known as calibration or inverse regression in the current study we propose a bootstrap procedure aimed at constructing an inverse confidence interval for the univariate calibration problem the calibration curve is estimated either parametrically or by non parametric regression the methods are illustrated by an example

a comparison of gcv and gml for choosing the smoothing parameter in the generalized spline smoothing problem the partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood gml estimate for the smoothing parameter then this estimate is compared with the generalized cross validation gcv estimate both analytically and by monte carlo methods the comparison is based on a predictive mean square error criteria it is shown that if the true unknown function being estimated is smooth in a sense to be defined then the gml estimate undersmooths relative to the gcv estimate and the predictive mean square error using the gml estimate goes to zero at a slower rate than the mean square error using the gcv estimate if the true function is rough then the gcv and gml estimates have asymptotically similar behavior a monte carlo experiment was designed to see if the asymptotic results in the smooth case were evident in small sample sizes mixed results were obtained for n 32 gcv was somewhat better than gml for n 64 and gcv was decidedly superior for n 128 in the n 32 case gcv was better for smaller sigma 2 and the comparison close for larger sigma 2 the theoretical results are shown to extend to the generalized spline smoothing model which includes the estimate of functions given noisy values of various integrals of them

optimal curve fitting and smoothing using normalized uniform b splines a tool for studying complex systems the basic problem considered in this paper is how to reduce measures of complex systems to easily understood measures we show that optimal smoothing splines are an ideal means to solve the problem in some cases of course no single method will work for all complex systems and all measures the problem of constructing optimal curves for given set of data at discrete data points is considered both equally spaced and non equally spaced data points are treated the curves are constructed by using b splines as basis functions namely as weighted sum of shifted b splines of degree k it is then shown that an optimal approximation can be solved without any boundary conditions wherein explicit solution formulas are presented a problem of optimal interpolation is also considered in parallel we apply this technique to the dow jones industrial index for both long and short time periods we see that long term trends can be easily identified

pointwise influence matrices for functional response regression we extend the notion of an influence or hat matrix to regression with functional responses and scalar predictors for responses depending linearly on a set of predictors our definition is shown to reduce to the conventional influence matrix for linear models the pointwise degrees of freedom the trace of the pointwise influence matrix are shown to have an adaptivity property that motivates a two step bivariate smoother for modeling nonlinear dependence on a single predictor this procedure adapts to varying complexity of the nonlinear model at different locations along the function and thereby achieves better performance than competing tensor product smoothers in an analysis of the development of white matter microstructure in the brain

learning curves for gaussian process regression approximations and bounds we consider the problem of calculating learning curves i e average generalization performance of gaussian processes used for regression on the basis of a simple expression for the generalization error in terms of the eigenvalue decomposition of the covariance function we derive a number of approximation schemes we identify where these become exact and compare with existing bounds on learning curves the new approximations which can be used for any input space dimension generally get substantially closer to the truth we also study possible improvements to our approximations finally we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function

conservative difference formulations of calogero and toda hamiltonian systems abstract calogero and toda hamiltonian systems are reformulated using only differences the formulations prove to have the same fundamental invariants as the continuous systems and in addition are readily implementable on modern digital computers

nonparametric trend estimation in functional time series with application to annual mortality rates this research was supported by the king abdullah university of science and technology kaust

smoothing parameter and model selection for general smooth models abstractthis article discusses a general framework for smoothing parameter estimation for models with regular likelihoods constructed in terms of unknown smooth functions of covariates gaussian random effects and parametric terms may also be present by construction the method is numerically stable and convergent and enables smoothing parameter uncertainty to be quantified the latter enables us to fix a well known problem with aic for such models thereby improving the range of model selection tools available the smooth functions are represented by reduced rank spline like smoothers with associated quadratic penalties measuring function smoothness model estimation is by penalized likelihood maximization where the smoothing parameters controlling the extent of penalization are estimated by laplace approximate marginal likelihood the methods cover for example generalized additive models for nonexponential family responses e g beta ordered categorical scaled t distribution negative binomial a

the dual variational principle and elliptic problems with discontinuous nonlinearities on etudie des problemes aux valeurs limites elliptiques du type u f u p x x u 0 x ou r n est un domaine borne et f peut avoir des discontinuites on cherche des solutions en utilisant le principe d action duale de clarke

two graphical displays for outlying and influential observations in regression summary the paper describes two procedures for detecting observations with outlying values either in the response variable or in the explanatory variables in multiple regression these procedures are presented as half normal plots with envelopes derived from simulation in order to avoid overinterpretation of the data analysis of a well known data set leads to the use of a data transformation a simple test for which is commended and to some comments on the relationship with robust regression the widespread availability of sophisticated computer software has made the fitting of multiple regression equations painless but the very ease of these procedures may cause insufficient care to be given to checking and scrutiny of the data transcription punching and data manipulation errors may lead to bad values both of the observations y and of the independent or explanatory variables x it is the purpose of the present paper to describe and exemplify two plots which provide checks against such bad values it is argued that these plots and a test for transformations should accompany any thorough regression analysis bad values of the response or outliers have long been detected by a variety of plots of residuals bad values of the explanatory variables are less easily detected even when they lead to an extreme point in the design space since the fitted equation will pass close to such an influential observation the residual may not be especially large even after allowance has been made for the small variance of the fitted value at such a point to detect such behaviour a quantity is needed which exhibits the dependence of the fitted model on each point or group of points discussions of such quantities are given by hoaglin welsch 1978 and more recently by belsley kuh welsch 1980 2 1 by cook weisberg 1980 and by pregibon 1981 the quantity used here is a modification of a statistic proposed by cook 1977 which is derived in 2 two half normal plots are presented in 3 as an aid to interpretation envelopes to the plots are generated by simulation examples of the plots are given in 4 both for simulated data and for the stack loss data given by brownlee 1965 p 454 transformations of this data set are discussed in 5 the paper concludes with some comments on the relationship with robust

modelling and smoothing parameter estimation with multiple quadratic penalties summary penalized likelihood methods provide a range of practical modelling tools including spline smoothing generalized additive models and variants of ridge regression selecting the correct weights for penalties is a critical part of using these methods and in the single penalty case the analyst has several well founded techniques to choose from however many modelling problems suggest a formulation employing multiple penalties and here general methodology is lacking a wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimise kw 1 2 xp a y k 2 o p m i 1 iip 0 s i p p is a parameter vector x a design matrix si a non negative definite coefficient matrix defining the i th penalty with associated smoothing parameter ii w a diagonal weight matrix y a vector of data or pseudodata and o an overall smoothing parameter included for reasons of computational efficiency this paper shows how smoothing parameter selection can be performed efficiently by applying generalized cross validation to this problem and how this allows non linear generalized linear and linear models to be fitted using multiple penalties substantially increasing the scope of penalized modelling methods examples of non linear modelling generalized additive modelling and anisotropic smoothing are given

bayesian confidence intervals for smoothing splines abstract the frequency properties of wahba s bayesian confidence intervals for smoothing splines are investigated by a large sample approximation and by a simulation study when the coverage probabilities for these pointwise confidence intervals are averaged across the observation points the average coverage probability acp should be close to the nominal level from a frequency point of view this agreement occurs because the average posterior variance for the spline is similar to a consistent estimate of the average squared error and because the average squared bias is a modest fraction of the total average squared error these properties are independent of the bayesian assumptions used to derive this confidence procedure and they explain why the acp is accurate for functions that are much smoother than the sample paths prescribed by the prior this analysis accounts for the choice of the smoothing parameter bandwidth using cross validation in the case of natural splines an adaptive method for avo

supercomputer simulation of colliding microdrops of water abstract microdrop collisions are important in fast kinetic microwave nucleation and rainstorm studies in this paper a molecular model of colliding microdrops of water is simulated on a cray x mp 24 a least squares fit of the force field allows relatively large time steps in the numerical procedure rotating raindrop modes dumbell modes and oscillatory oblateness modes are demonstrated as are nonadhesive soft and hard collisions

covariate adjusted region referenced generalized functional linear model for eeg data electroencephalography eeg studies produce region referenced functional data in the form of eeg signals recorded across electrodes on the scalp it is of clinical interest to relate the highly structured eeg data to scalar outcomes such as diagnostic status in our motivating study resting state eeg is collected on both typically developing td children and children with autism spectrum disorder asd aged 2 to 12 years old the peak alpha frequency paf defined as the location of a prominent peak in the alpha frequency band of the spectral density is an important biomarker linked to neurodevelopment and is known to shift from lower to higher frequencies as children age to retain the most amount of information from the data we consider the oscillations in the spectral density within the alpha band rather than just the peak location as a functional predictor of diagnostic status td vs asd adjusted for chronological age a covariate adjusted region referenced generalized functional linear model is proposed for modeling scalar outcomes from region referenced functional predictors which utilizes a tensor basis formed from one dimensional discrete and continuous bases to estimate functional effects across a discrete regional domain while simultaneously adjusting for additional nonfunctional covariates such as age the proposed methodology provides novel insights into differences in neural development of td and asd children the efficacy of the proposed methodology is investigated through extensive simulation studies

the signal extraction approach to nonlinear regression and spline smoothing abstract this article shows how to fit a smooth curve polynomial spline to pairs of data values yi xi prior specification of a parametric functional form for the curve is not required the resulting curve can be used to describe the pattern of the data and to predict unknown values of y given x both point and interval estimates are produced the method is easy to use and the computational requirements are modest even for large sample sizes our method is based on maximum likelihood estimation of a signal in noise model of the data we use the kalman filter to evaluate the likelihood function and achieve significant computational advantages over previous approaches to this problem

penalized maximum likelihood estimation in logistic regression and discrimination summary maximum likelihood estimation of the parameters of the binary logistic regression model for pr h i x is discussed with separate discussion of sampling from i the conditional distribution of h given x ii the joint distribution of h and x and iii the conditional distribution of x given h difficulties associated with continuous x in the latter sampling scheme are discussed to avoid these penalized maximum likelihood estimates are introduced which give estimates of the logistic parameters and a nonparametric spline estimate of the marginal distribution of x extensions to multinomial logistic regression are outlined

fsem functional structural equation models for twin functional data the aim of this article is to develop a novel class of functional structural equation models fsems for dissecting functional genetic and environmental effects on twin functional data while chara

thin plate regression splines discuss the production of low rank smoothers for d greater than or equal to 1 dimensional data which can be fitted by regression or penalized regression methods the smoothers are constructed by a simple transformation and truncation of the basis that arises from the solution of the thin plate spline smoothing problem and are optimal in the sense that the truncation is designed to result in the minimum possible perturbation of the thin plate spline smoothing problem given the dimension of the basis used to construct the smoother by making use of lanczos iteration the basis change and truncation are computationally efficient the smoothers allow the use of approximate thin plate spline models with large data sets avoid the problems that are associated with knot placement that usually complicate modelling with regression splines or penalized regression splines provide a sensible way of modelling interaction terms in generalized additive models provide low rank approximations to generalized smoothing spline models appropriate for use with large data sets provide a means for incorporating smooth functions of more than one variable into non linear models and improve the computational efficiency of penalized likelihood models incorporating thin plate splines given that the approach produces spline like models with a sparse basis it also provides a natural way of incorporating unpenalized spline like terms in linear and generalized linear models and these can be treated just like any other model terms from the point of view of model selection inference and diagnostics

soap film smoothing summary conventional smoothing methods sometimes perform badly when used to smooth data over complex domains by smoothing inappropriately across boundary features such as peninsulas solutions to this smoothing problem tend to be computationally complex and not to provide model smooth functions which are appropriate for incorporating as components of other models such as generalized additive models or mixed additive models we propose a class of smoothers that are appropriate for smoothing over difficult regions of 2 which can be represented in terms of a low rank basis and one or two quadratic penalties the key features of these smoothers are that they do not smooth across boundary features that their representation in terms of a basis and penalties allows straightforward incorporation as components of generalized additive models mixed models and other non standard models that smoothness selection for these model components is straightforward to accomplish in a computationally efficient manner via generalized cross validation akaike s information criterion or restricted maximum likelihood for example and that their low rank means that their use is computationally efficient

extending the scope of wavelet regression methods by coefficient dependent thresholding abstract various aspects of the wavelet approach to nonparametric regression are considered with the overall aim of extending the scope of wavelet techniques to irregularly spaced data to regularly spaced datasets of arbitrary size to heteroscedastic and correlated data and to data that contain outliers the core of the methodology is an algorithm for finding ail of the variances and within level covariances in the wavelet table of a sequence with given covariance structure if the original covariance matrix is band limited then the algorithm is linear in the length of the sequence the variance calculation algorithm allows data on any set of independent variable values to be treated by first interpolating to a fine regular grid of suitable length and then constructing a wavelet expansion of the gridded data various thresholding methods are discussed and investigated exact risk formulas for the mean square error of the methodology for given design are derived good performance is obtained by nois

inference in generalized additive mixed modelsby using smoothing splines generalized additive mixed models are proposed for overdispersed and correlated data which arise frequently in studies involving clustered hierarchical and spatial designs this class of models allows flexible functional dependence of an outcome variable on covariates by using nonparametric regression while accounting for correlation between observations by using random effects we estimate nonparametric functions by using smoothing splines and jointly estimate smoothing parameters and variance components by using marginal quasi likelihood because numerical integration is often required by maximizing the objective functions double penalized quasi likelihood is proposed to make approximate inference frequentist and bayesian inferences are compared a key feature of the method proposed is that it allows us to make systematic inference on all model components within a unified parametric mixed model framework and can be easily implemented by fitting a working generalized linear mixed model by using existing statistical software a bias correction procedure is also proposed to improve the performance of double penalized quasi likelihood for sparse data we illustrate the method with an application to infectious disease data and we evaluate its performance through simulation

detecting generalized synchronization between chaotic signals a kernel based approach a unified framework for analysing generalized synchronization in coupled chaotic systems from data is proposed the key of the proposed approach is the use of the kernel methods recently developed in the field of machine learning several successful applications are presented which show the capability of the kernel based approach for detecting generalized synchronization and dynamical change of the coupling strength between two chaotic systems can be captured by the proposed approach it is also discussed how the kernel parameter is suitably chosen from data

energy conserving arbitrary order numerical solutions of then body problem in this paper an energy conserving modification of the well known extrapolation methods for solving then body problem is presented the method is compared with the most commonly used extrapolation methods it appears that the presented modification yields a better approximation of the exact solution computer examples are given

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

elliptic problems with discontinuities in this paper we prove two existence theorems for elliptic problems with discontinuities the first one is a noncoercive dirichlet problem and the second one is a neumann problem we do not use the method of upper and lower solutions for neumann problems we assume that f is nondecreasing we use the critical point theory for locally lipschitz functionals

a fast monte carlo cross validation procedure for large least squares problems with noisy data we propose a fast monte carlo algorithm for calculating reliable estimates of the trace of the influence matrixa involved in regularization of linear equations or data smoothing problems where is the regularization or smoothing parameter this general algorithm is simply as follows i generaten pseudo random valuesw 1 w n from the standard normal distribution wheren is the number of data points and letw w 1 w n t ii compute the residual vectorw a w iii take the normalized inner product w t w a w w t w as an approximation to 1 n tr i a we show both by theoretical bounds and by numerical simulations on some typical problems that the expected relative precision of these estimates is very good whenn is large enough and that they can be used in practice for the minimization with respect to of the well known generalized cross validation gcv function this permits the use of the gcv method for choosing in any particular large scale application with only a similar amount of work as the standard residual method numerical applications of this procedure to optimal spline smoothing in one or two dimensions show its efficiency