some aspects of the spline smoothing approach to non parametric regression curve fitting non parametric regression using cubic splines is an attractive flexible and widely applicable approach to curve estimation although the basic idea was formulated many years ago the method is not as widely known or adopted as perhaps it should be the topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology particular subjects covered include the basic principles of the method the relation with moving average and other smoothing methods the automatic choice of the amount of smoothing and the use of residuals for diagnostic checking and model adaptation the question of providing inference regions for curves and for relevant properties of curves is approached via a finite dimensional bayesian formulation

splines in statistics abstract this is a survey article that attempts to synthesize a broad variety of work on splines in statistics splines are presented as a nonparametric function estimating technique after a general introduction to the theory of interpolating and smoothing splines splines are treated in the nonparametric regression setting the method of cross validation for choosing the smoothing parameter is discussed and the general multivariate regression surface estimation problem is addressed an extensive discussion of splines as nonparametric density estimators is followed by a discussion of their role in time series analysis a comparison of the spline and isotonic regression methodologies leads to a formulation of a hybrid estimator the closing section provides a brief overall summary and formulates a number of open unsolved problems relating to splines in statistics

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

stochastic inversion for electromagnetic geophysics practical challenges and improving convergence efficiency traditional deterministic geophysical inversion algorithms are not designed to provide a robust evaluation of uncertainty that reflects the limitations of the geophysical technique stochastic inversions which do provide a sampling based measure of uncertainty are computationally expensive and not straightforward to implement for nonexperts nonstatisticians our results include stochastic inversion for magnetotelluric and controlled source electromagnetic data two markov chain sampling algorithms metropolis hastings and slice sampler can significantly decrease the computational expense compared to using either sampler alone the statistics of the stochastic inversion allow for 1 variances that better reveal the measurement sensitivities of the two different electromagnetic techniques than traditional techniques and 2 models defined by the median and modes of parameter probability density functions which produce amplitude and phase data that are consistent with the observed data in general pa

a fast and efficient cross validation method for smoothing parameter choice in spline regression abstract the spline smoothing approach to nonparametric regression is considered with particular reference to the problem of choosing how much to smooth both computational and statistical aspects of the method of cross validation are discussed an approximate cross validation method is proposed that requires a small linear amount of computer time a simulation study indicates that the method has very good statistical properties some mathematical justification and motivation are also provided

smoothing splines regression derivatives and deconvolution abstract the statistical properties of a cubic smoothing spline and its derivative are analyzed it is shown that unless unnatural boundary conditions hold the integrated squared bias is dominated by local effects near the boundary similar effects are shown to occur in the regularized solution of a translation kernel integral equation these results are derived by developing a fourier representation for a smoothing spline author

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

generalized linear array models with applications to multidimensional smoothing summary data with an array structure are common in statistics and the design or regression matrix for analysis of such data can often be written as a kronecker product factorial designs contingency tables and smoothing of data on multidimensional grids are three such general classes of data and models in such a setting we develop an arithmetic of arrays which allows us to define the expectation of the data array as a sequence of nested matrix operations on a coefficient array we show how this arithmetic leads to low storage high speed computation in the scoring algorithm of the generalized linear model we refer to a generalized linear array model and apply the methodology to the smoothing of multidimensional arrays we illustrate our procedure with the analysis of three data sets mortality data indexed by age at death and year of death spatially varying microarray background data and disease incidence data indexed by age at death year of death and month of death

the c algebra of singular integral operators with semi almost periodic coefficients abstract we establish a fredholm criterion for the operators belonging to the c algebra generated by singular integral operators with semi almost periodic matrix coefficients the result is applied to toeplitz like operators that are perturbed by integral operators with fixed singularities at infinity in which case it leads to an effectively verifiable fredholm criterion together with an index formula our approach is based on an isomorphism theorem for c algebras associated with c dynamical systems and the notion of canonical generalized ap factorization for almost periodic matrix functions

analysis of repeated measurements using nonparametric smoothers and randomization tests the mixed model analysis of variance anova which is commonly applied to repeated measurements taken over time depends on specialized assumptions about the error distribution and fails to exploit information contained in the ordering of the data points over time this paper describes a procedure that overcomes these disadvantages while preserving familiar features of the mixed model anova group profiles are estimated by nonparametric smoothing of observed mean profiles group and time main effects and the group by time interaction effect are tested using randomization tests results of zerbe 1979 journal of the american statistical association 74 215 221 are used to construct f test approximations for the randomization tests of the group and group by time effects a new approximate f test for time effect is proposed a simulation study demonstrates that the approximations perform well and that smoothing increases the power of the tests for time main effect and group by time interaction the procedure is applied to data on hormone levels in cows

stable and efficient multiple smoothing parameter estimation for generalized additive models representation of generalized additive models gam s using penalized regression splines allows gam s to be employed in a straightforward manner using penalized regression methods not only is inference facilitated by this approach but it is also possible to integrate model selection in the form of smoothing parameter selection into model fitting in a computationally efficient manner using well founded criteria such as generalized cross validation the current fitting and smoothing parameter selection methods for such models are usually effective but do not provide the level of numerical stability to which users of linear regression packages for example are accustomed in particular the existing methods cannot deal adequately with numerical rank deficiency of the gam fitting problem and it is not straightforward to produce methods that can do so given that the degree of rank deficiency can be smoothing parameter dependent in addition models with the potential flexibility of gam s can also present

spline smoothing the equivalent variable kernel method on considere une approche spline pour la regression non parametrique et l estimation de courbe on montre que dans un certain sens le lissage spline correspond approximativement au lissage par une methode du noyau avec une largeur de bande dependant de la densite locale des points du plan de regression

expanded uncertainty quantification in inverse problems hierarchical bayes and empirical bayes a common way to account for uncertainty in inverse problems is to apply bayes rule and obtain a posterior distribution of the quantities of interest given a set of measurements a conventional bayesian treatment however requires assuming specific values for parameters of the prior distribution and of the distribution of the measurement errors e g the standard deviation of the errors in practice these parameters are often poorly known a priori and choosing a particular value is often problematic moreover the posterior uncertainty is computed assuming that these parameters are fixed if they are not well known a priori the posterior uncertainties have dubious value this paper describes extensions to the conventional bayesian treatment that assign uncertainty to the parameters defining the prior distribution and the distribution of the measurement errors these extensions are known in the statistical literature as empirical bayes and hierarchical bayes we demonstrate the practical applicati

discretized laplacian smoothing by fourier methods abstract an approach to multidimensional smoothing is introduced that is based on a penalized likelihood with a modified discretized laplacian penalty term the choice of penalty simplifies computational difficulties associated with standard multidimensional laplacian smoothing methods yet without compromising mean squared error characteristics at least on the interior of the region of interest for linear smoothing in hyper rectangular domains which has wide application in image reconstruction and restoration problems computations are carried out using fast fourier transforms nonlinear smoothing is accomplished by iterative application of the linear smoothing technique the iterative procedure is shown to be convergent under general conditions adaptive choice of the amount of smoothing is based on approximate cross validation type scores an importance sampling technique is used to estimate the degrees of freedom of the smooth the methods are implemented in one and two dimensional settings some i

characteristic fixed point sets of semifree actions on spheres a group action is semifree if it is free away from its fixed point set p a smith showed that when a finite group of orderqacts semifreely on a sphere the fixed set is a modqhomology sphere conversely given a modqhomology sphere as a subset of a sphere one may try to construct a group action on the sphere fixing the subset the converse question was first systematically studied by jones and then by many others in this note we give new numerical congruences satisfied by the homology of the fixed sets and give a definitive solution to the problem for characteristic fixed point sets c 1999 john wiley sons inc

a new statistical approach to detecting significant activation in functional mri there are many ways to detect activation patterns in a time series of observations at a single voxel in a functional magnetic resonance imaging study the critical problem is to estimate the statistical significance which depends on the estimation of both the magnitude of the response to the stimulus and the serial dependence of the time series and especially on the assumptions made in that estimation we show that for experimental designs with periodic stimuli only a few aspects of the serial dependence are important and these can be estimated reliably via nonparametric estimation of the spectral density of the time series whereas existing techniques are biased by their assumptions the linear model with stationary serially dependent errors can be analyzed entirely in frequency domain and doing so provides many insights in particular we introduce a technique to detect periodic activations and show that it has a distribution theory that enables us to assign significance levels down to 1 in 100 000 levels which are needed when a whole brain image is under consideration nonparametric spectral density estimation is shown to be self calibrating and accurate when compared to several other time domain approaches the technique is especially resistant to high frequency artefacts that we have found in some datasets and we demonstrate that time domain approaches may be sufficiently susceptible to these effects to give misleading results the method is easily generalized to handle event related designs we found it necessary to consider the trends in the time series carefully and use nonlinear filters to remove the trends and robust techniques to remove spikes using this in connection with our techniques allows us to detect activations in clumps of a few even one voxel in periodic designs yet produce essentially no false positive detections at any voxels in null datasets

plurality and resemblance in fmri data analysis abstract we apply nine analytic methods employed currently in imaging neuroscience to simulated and actual bold fmri signals and compare their performances under each signal type starting with baseline time series generated by a resting subject during a null hypothesis study we compare method performance with embedded focal activity in these series of three different types whose magnitudes and time courses are simple convolved with spatially varying hemodynamic responses and highly spatially interactive we then apply these same nine methods to bold fmri time series from contralateral primary motor cortex and ipsilateral cerebellum collected during a sequential finger opposition study paired comparisons of results across methods include a voxel specific concordance correlation coefficient for reproducibility and a resemblance measure that accommodates spatial autocorrelation of differences in activity surfaces receiver operating characteristic curves show considerable model differences in ranges less than 10 significance level false positives and greater than 80 power true positives concordance and resemblance measures reveal significant differences between activity surfaces in both data sets these measures can assist researchers by identifying groups of models producing similar and dissimilar results and thereby help to validate consolidate and simplify reports of statistical findings a pluralistic strategy for fmri data analysis can uncover invariant and highly interactive relationships between local activity foci and serve as a basis for further discovery of organizational principles of the brain results also suggest that a pluralistic empirical strategy coupled formally with substantive prior knowledge can help to uncover new brain behavior relationships that may remain hidden if only a single method is employed

a fast monte carlo cross validation procedure for large least squares problems with noisy data we propose a fast monte carlo algorithm for calculating reliable estimates of the trace of the influence matrixa involved in regularization of linear equations or data smoothing problems where is the regularization or smoothing parameter this general algorithm is simply as follows i generaten pseudo random valuesw 1 w n from the standard normal distribution wheren is the number of data points and letw w 1 w n t ii compute the residual vectorw a w iii take the normalized inner product w t w a w w t w as an approximation to 1 n tr i a we show both by theoretical bounds and by numerical simulations on some typical problems that the expected relative precision of these estimates is very good whenn is large enough and that they can be used in practice for the minimization with respect to of the well known generalized cross validation gcv function this permits the use of the gcv method for choosing in any particular large scale application with only a similar amount of work as the standard residual method numerical applications of this procedure to optimal spline smoothing in one or two dimensions show its efficiency

generalized additive models some applications abstract generalized additive models have the form x fj x j where might be the regression function in a multiple regression or the logistic transformation of the posterior probability pr y 1 x in a logistic regression in fact these models generalize the whole family of generalized linear models x x where x g x is some transformation of the regression function we use the local scoring algorithm to estimate the functions fj xj nonparametrically using a scatterplot smoother as a building block we demonstrate the models in two different analyses a nonparametric analysis of covariance and a logistic regression the procedure can be used as a diagnostic tool for identifying parametric transformations of the covariates in a standard linear analysis a variety of inferential tools have been developed to aid the analyst in assessing the relevance and significance of the estimated functions these include confidence curves degrees of freedom estimates and approximat

nonparametric empirical bayes growth curve analysis abstract nonparametric and semiparametric regression have been suggested as alternatives to parametric models for growth curve analysis in this article we demonstrate that empirical bayes estimation can be used to improve linear smoothing estimates when multiple curves are available

bayesian inference for semiparametric regression using a fourier representation summary this paper presents the bayesian analysis of a semiparametric regression model that consists of parametric and nonparametric components the nonparametric component is represented with a fourier series where the fourier coefficients are assumed a priori to have zero means and to decay to 0 in probability at either algebraic or geometric rates the rate of decay controls the smoothness of the response function the posterior analysis automatically selects the amount of smoothing that is coherent with the model and data posterior probabilities of the parametric and semiparametric models provide a method for testing the parametric model against a non specific alternative the bayes estimator s mean integrated squared error compares favourably with the theoretically optimal estimator for kernel regression

noise estimation in signal restoration using regularization several estimators of variance are compared in the context of problems where smoothing is incorporated in the estimation of regression functions many of the estimators are scaled quadratic forms in the data and some incorporate steps that try to accommodate substantial nonconstancy in the response function the estimators are described and an account is provided of an empirical study based on four one dimensional source functions of varying degrees of irregularity

automatic smoothing of regression functions in generalized linear models abstract we consider the penalized likelihood method for estimating nonparametric regression functions in generalized linear models nelder and wedderburn 1972 and present a generalized cross validation procedure for empirically assessing an appropriate amount of smoothing in these estimates asymptotic arguments and numerical simulations are used to show that the generalized cross validatory procedure preforms well from the point of view of a weighted mean squared error criterion the methodology adds to the battery of graphical tools for model building and checking within the generalized linear model framework included are two examples motivated by medical and horticultural applications

a flexible approach to parametric inference in nonlinear and time varying time series models many structural break and regime switching models have been used with macroeconomic and financial data in this paper we develop an extremely flexible modeling approach which can accommodate virtually any of these specifications we build on earlier work showing the relationship between flexible functional forms and random variation in parameters our contribution is based around the use of priors on the time variation that is developed from considering a hypothetical reordering of the data and distance between neighboring reordered observations the range of priors produced in this way can accommodate a wide variety of nonlinear time series models including those with regime switching and structural breaks by allowing the amount of random variation in parameters to depend on the distance between reordered observations the parameters can evolve in a wide variety of ways allowing for everything from models exhibiting abrupt change e g threshold autoregressive models or standard structural break models to those which allow for a gradual evolution of parameters e g smooth transition autoregressive models or time varying parameter models bayesian econometric methods for inference are developed for estimating the distance function and types of hypothetical reordering conditional on a hypothetical reordering and distance function a simple reordering of the actual data allows us to estimate our models with standard state space methods by a simple adjustment to the measurement equation we use artificial data to show the advantages of our approach before providing two empirical illustrations involving the modeling of real gdp growth

flexible regression models with cubic splines we describe the use of cubic splines in regression models to represent the relationship between the response variable and a vector of covariates this simple method can help prevent the problems that result from inappropriate linearity assumptions we compare restricted cubic spline regression to non parametric procedures for characterizing the relationship between age and survival in the stanford heart transplant data we also provide an illustrative example in cancer therapeutics

brief fast spline smoothing via spectral factorization concepts when tuning the smoothness parameter of nonparametric regression splines the evaluation of the so called degrees of freedom is one of the most computer intensive tasks in the paper a closed form expression of the degrees of freedom is obtained for the case of cubic splines and equally spaced data when the number of data tends to infinity state space methods kalman filtering and spectral factorization techniques are used to prove that the asymptotic degrees of freedom are equal to the variance of a suitably defined stationary process the closed form expression opens the way to fast spline smoothing algorithms whose computational complexity is about one half of standard methods or even one fourth under further approximations

regularization of wavelet approximations in this paper we introduce nonlinear regularized wavelet estimators for estimating nonparametric regression functions when sampling points are not uniformly spaced the approach can apply readily to many other statistical contexts various new penalty functions are proposed the hard thresholding and soft thresholding estimators of donoho and johnstone are specific members of nonlinear regularized wavelet estimators they correspond to the lower and upper envelopes of a class of the penalized least squares estimators necessary conditions for penalty functions are given for regularized estimators to possess thresholding properties oracle inequalities and universal thresholding parameters are obtained for a large class of penalty functions the sampling properties of nonlinear regularized wavelet estimators are established and are shown to be adaptively minimax to efficiently solve penalized least squares problems nonlinear regularized sobolev interpolators nrsi are proposed as initial estimators wh

the l 1 method for robust nonparametric regression abstract consider the problem of estimating the regression function underlying a set of data that is contaminated by a heavy tailed error distribution the two standard approaches to such a problem are each flawed robust parametric least squares is appropriate only if there is good reason to believe that the underlying function has some particular form whereas most nonparametric regression methods are asymptotically equivalent to kernel regression methods which are not resistant against outliers existing algorithms for robustifying nonparametric regression procedures use either nonlinear optimization of an influence function or iterative solution of local polynomial fitting using reweighted least squares neither of these two approaches combines computational ease with asymptotic theoretical results furthermore application of the robust procedure has been limited almost exclusively to the case of a single explanatory variable with the response variable in this article a new hybrid method is propose

automated smoothing of image and other regularly spaced data this paper is primarily motivated by the problem of automatically removing unwanted noise from high dimensional remote sensing imagery the initial step involves the transformation of the data to a space of intrinsically lower dimensionality and the smoothing of images in the new space different images require different amounts of smoothing the signal assumed to be mostly smooth with relatively few discontinuities is estimated from the data using the method of generalized cross validation it is shown how the generalized cross validated thin plate smoothing spline with observations on a regular grid in d dimensions is easily approximated and computed in the fourier domain space domain approximations are also investigated the technique is applied to some remote sensing data

c algebra of nonlocal convolution type operators abstract the c subalgebra b of all bounded linear operators on the space l 2 r which is generated by all multiplication operators by piecewise slowly oscillating functions by all convolution operators with piecewise slowly oscillating symbols and by the range of a unitary representation of the group of all affine mappings on r is studied a faithful representation of the quotient c algebra b b k in a hilbert space where k is the ideal of compact operators on the space l 2 r is constructed by applying an appropriate spectral measure decompositions a local trajectory method and the fredholm symbol calculus for the c algebra of convolution type operators without shifts this gives a fredholm symbol calculus for the c algebra b and a fredholm criterion for the operators b b

proprietes de convergence presque complete du predicteur a noyau let z n be a mixing process which is valued in mathbb e subset mathbb r q g be a real measurable function defined on mathbb e s and k be two positive integers we suppose the existence of a function r satisfying r e g z n s z n k 1 z n n n k and estimate the function r from a sequence z i i 1 n by r n with r n u frac sum limits i k n s g z i s k u z i k 1 cdots z i h n 1 sum limits i k n s k u z i k 1 cdots z i h n 1 forall u in mathbb e k where k is a kernel of kq and h n h n 0 we give various conditions on both the sequences h n and the sequence n which is associated with z n for the two following properties uniform complete convergence to r for the functional estimator r n and complete convergence to 0 for the real random variable r n z n k 1 z n r zn k 1 z n this last result concerns the predictor r n z n k 1 z n of g z n s from z i i 1 n when the process z n is stationary and markovian of order k

an iterative approach to two dimensional laplacian smoothing with application to image restoration abstract smoothing methods are increasingly important in modern graphically oriented data analysis there are several fast and reliable univariate cross validated smoothing algorithms but comparable methods are not available in higher dimensions this article develops an iterative approach to two dimensional laplacian spline smoothing based on tensor product cubic b splines estimating equations are developed and solved iteratively using two line symmetric successive over relaxation with conjugate gradient acceleration an asymptotic approximation for the generalized cross validation score is described the resulting algorithm is naturally suited to two dimensional scatterplot smoothing but has potential value for image restoration as well this is illustrated on a multicolor image example along with an image restoration an uncertainty assessment based on the entropy of the marginal posterior pixel value distribution is produced this analysis particularly highlights the statistical uncertainty in resol

bayesian confidence intervals for the cross validated smoothing spline summary we consider the model y t g ti ei i 17 2 n where g t t 0 1 is a smooth function and the ei are independent n 0 a2 errors with g2 unknown the cross validated smoothing spline can be used to estimate g non parametrically from

iterative solution of systems originating from integral equations and surface interpolation a simple procedure for generating diagonal dominance in certain linear systems is presented and the iterative solution of the transformed systems is discussed the method is presented for systems encountered in the numerical solution of fredholm equations of the first kind with singular kernels and for systems arising in the interpolation of surfaces by certain sets of radial functions in the second case the method provides new sets of basis functions generalizing in some sense the univariate b splines

backfitting in smoothing spline anova a computational scheme for fitting smoothing spline anova models to large data sets with a near tensor product design is proposed such data sets are common in spatial temporal analyses the proposed scheme uses the backfitting algorithm to take advantage of the tensor product design to save both computational memory and time several ways to further speed up the backfitting algorithm such as collapsing component functions and successive over relaxation are discussed an iterative imputation procedure is used to handle the cases of near tensor product designs an application to a global historical surface air temperature data set which motivated this work is used to illustrate the scheme proposed

a comparison of the reinsch and speckman splines in the context of nonparametric regression spline smoothers can be used both for the estimation of the curve itself and for the estimation of the error variance we compare two approaches that have been suggested the reinsch spline i e roughness penalty and the speckman spline minimax the comparison is made on the basis of the summed mean squared error for curve estimation and on the basis of a closely related criterion for variance estimation we show in both asymptotic and small sample studies that the reinsch smoothing spline is close to optimal both as a curve and as a variance estimator

linear models for field trials smoothing and cross validation abstract spatial methods for the analysis of agricultural field experiments are represented here as smoothing methods applied simultaneously with the estimation of treatment effects selection of both the form of the smoother and the degree of smoothing required may be based on cross validation particular emphasis is placed in this paper on generalized least squares estimation ion linear models but the principle applies quite generally keywords agricultural field experiments generalized least squares incomplete block designs neighbour methods papadakis method recovery of inter block formation smoothing and spatial models

constructing a bootstrap confidence interval for the unknown concentration in radioimmunoassay the statistical problem associated with radioimmunoassay is known as calibration or inverse regression in the current study we propose a bootstrap procedure aimed at constructing an inverse confidence interval for the univariate calibration problem the calibration curve is estimated either parametrically or by non parametric regression the methods are illustrated by an example

smoothing spline anova models for large data sets with bernoulli observations and the randomized gacv we propose the randomized generalized approximate cross validation rangacv method for choosing multiple smoothing parameters in penalized likelihood estimates for bernoulli data the method is intended for application with penalized likelihood smoothing spline anova models in addition we propose a class of approximate numerical methods for solving the penalized likelihood variational problem which in conjunction with the rangacv method allows the application of smoothing spline anova models with bernoulli data to much larger data sets than previously possible these methods are based on choosing an approximating subset of the natural representer basis functions for the variational problem simulation studies with synthetic data including synthetic data mimicking demographic risk factor data sets is used to examine the properties of the method and to compare the approach with the grkpack code of wang 1997c bayesian confidence intervals are obtained for the fits and are shown in the simulation studies to have the across the function property usually claimed for these confidence intervals finally the method is applied to an observational data set from the beaver dam eye study with scientifically interesting results

a statistical model for signature verification a bayesian model for off line signature verification involving the representation of a signature through its curvature is developed the prior model makes use of a spatial point process for specifying the knots in an approximation restricted to a buffer region close to a template curvature along with an independent time warping mechanism in this way prior shape information about the signature can be built into the analysis the observation model is based on additive white noise superimposed on the underlying curvature the approach is implemented using markov chain monte carlo and applied to a collection of documented instances of william shakespeare s signature

a comparison of gcv and gml for choosing the smoothing parameter in the generalized spline smoothing problem the partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood gml estimate for the smoothing parameter then this estimate is compared with the generalized cross validation gcv estimate both analytically and by monte carlo methods the comparison is based on a predictive mean square error criteria it is shown that if the true unknown function being estimated is smooth in a sense to be defined then the gml estimate undersmooths relative to the gcv estimate and the predictive mean square error using the gml estimate goes to zero at a slower rate than the mean square error using the gcv estimate if the true function is rough then the gcv and gml estimates have asymptotically similar behavior a monte carlo experiment was designed to see if the asymptotic results in the smooth case were evident in small sample sizes mixed results were obtained for n 32 gcv was somewhat better than gml for n 64 and gcv was decidedly superior for n 128 in the n 32 case gcv was better for smaller sigma 2 and the comparison close for larger sigma 2 the theoretical results are shown to extend to the generalized spline smoothing model which includes the estimate of functions given noisy values of various integrals of them

optimal curve fitting and smoothing using normalized uniform b splines a tool for studying complex systems the basic problem considered in this paper is how to reduce measures of complex systems to easily understood measures we show that optimal smoothing splines are an ideal means to solve the problem in some cases of course no single method will work for all complex systems and all measures the problem of constructing optimal curves for given set of data at discrete data points is considered both equally spaced and non equally spaced data points are treated the curves are constructed by using b splines as basis functions namely as weighted sum of shifted b splines of degree k it is then shown that an optimal approximation can be solved without any boundary conditions wherein explicit solution formulas are presented a problem of optimal interpolation is also considered in parallel we apply this technique to the dow jones industrial index for both long and short time periods we see that long term trends can be easily identified

algebraic phase unwrapping based on two dimensional spline smoothing over triangles two dimensional 2d phase unwrapping is an estimation problem of a continuous phase function over a 2d domain from its wrapped samples in this paper we propose a novel approach for high resolution 2d phase unwrapping in the first step spline smoothing sps we construct a pair of the smoothest spline functions which minimize the energies of their local changes while interpolating respectively the cosine and the sine of given wrapped samples if these functions have no common zero over the domain the proposed estimate of the continuous phase function can be obtained by algebraic phase unwrapping in the second step algebraic phase unwrapping apu to avoid the occurrence of common zeros in sps due to phase noise in the observed wrapped samples we also propose a denoising step denoising by selective smoothing dss as preprocessing which selectively smooths unreliable wrapped samples by using convex optimization the smoothness of the proposed unwrapped phase function is guaranteed globally over the domain without losing any desired consistency with all reliable wrapped samples numerical experiments for terrain height estimation demonstrate the effectiveness of the proposed 2d phase unwrapping scheme

the selection of prior distributions by formal rules abstract subjectivism has become the dominant philosophical foundation for bayesian inference yet in practice most bayesian analyses are performed with so called noninformative priors that is priors constructed by some formal rule we review the plethora of techniques for constructing such priors and discuss some of the practical and philosophical issues that arise when they are used we give special emphasis to jeffreys s rules and discuss the evolution of his viewpoint about the interpretation of priors away from unique representation of ignorance toward the notion that they should be chosen by convention we conclude that the problems raised by the research on priors chosen by formal rules are serious and may not be dismissed lightly when sample sizes are small relative to the number of parameters being estimated it is dangerous to put faith in any default solution but when asymptotics take over jeffreys s rules and their variants remain reasonable choices we also provide an annotated b

learning curves for gaussian process regression approximations and bounds we consider the problem of calculating learning curves i e average generalization performance of gaussian processes used for regression on the basis of a simple expression for the generalization error in terms of the eigenvalue decomposition of the covariance function we derive a number of approximation schemes we identify where these become exact and compare with existing bounds on learning curves the new approximations which can be used for any input space dimension generally get substantially closer to the truth we also study possible improvements to our approximations finally we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function

a c algebra of singular integral operators with shifts admitting distinct fixed points abstract representations on hilbert spaces for a nonlocal c algebra b of singular integral operators with piecewise slowly oscillating coefficients extended by a group of unitary shift operators are constructed the group of unitary shift operators u g in the c algebra b is associated with a discrete amenable group g of orientation preserving piecewise smooth homeomorphisms g t t that acts topologically freely on t and admits distinct fixed points for different shifts a c algebra isomorphism of the quotient c algebra b k where k is the ideal of compact operators onto a c algebra of fredholm symbols is constructed by applying the local trajectory method spectral measures and a lifting theorem as a result a fredholm symbol calculus for the c algebra b or equivalently a faithful representation of the quotient c algebra b k on a suitable hilbert space is constructed and a fredholm criterion for the operators b b is established

inference in semiparametric dynamic models for binary longitudinal data this article deals with the analysis of a hierarchical semiparametric model for dynamic binary longitudinal responses the main complicating components of the model are an unknown covariate function and serial correlation in the errors existing estimation methods for models with these features are of o n3 where n is the total number of observations in the sample therefore nonparametric estimation is largely infeasible when the sample size is large as in typical in the longitudinal setting here we propose a new o n markov chain monte carlo based algorithm for estimation of the nonparametric function when the errors are correlated thus contributing to the growing literature on semiparametric and nonparametric mixed effects models for binary data in addition we address the problem of model choice to enable the formal comparison of our semiparametric model with competing parametric and semiparametric specifications the performance of the methods is illustrated with detailed studies involving simul

group actions on homology spheres this can be stated in a more symmetric manner let r be any positive integer not equal to 3 then n acts freely and homologically trivially on z r i ff n acts freely and homologically trivially on sl in fact there is a one to one correspondence between such actions on u and such actions on s r the classification of such actions is discussed in w in addition the actions constructed have the property that every group element is isotopic to the identity theorem a holds in the topological or piecewise linear categories well known counterexamples exist in the smooth category even for homotopy spheres theorem a would be false without the assumption of homological triviality or without the assumption of simple connectivity see remark 8 8 in fact there exist groups n and simply connected z in homology spheres z r such that n acts freely on u but not on sl also there are groups n which act freely on s r but do not act freely on certain lens spaces lp with p prime to in if the group n is cyclic a result similar to our main theorem was first proved by l sffler l by different methods the second author independently established the cyclic case in wel by a method that extends to cyclic group actions on more general manifolds see c w1 and via this paper to general finite groups however the difficulties in making this extension have application elsewhere the main theorem is motivated by the philosophy that all of the geometry of a group action is present at the order of the group thus it is only natural to expect that any manifold homologically resembling a sphere admitting a

two graphical displays for outlying and influential observations in regression summary the paper describes two procedures for detecting observations with outlying values either in the response variable or in the explanatory variables in multiple regression these procedures are presented as half normal plots with envelopes derived from simulation in order to avoid overinterpretation of the data analysis of a well known data set leads to the use of a data transformation a simple test for which is commended and to some comments on the relationship with robust regression the widespread availability of sophisticated computer software has made the fitting of multiple regression equations painless but the very ease of these procedures may cause insufficient care to be given to checking and scrutiny of the data transcription punching and data manipulation errors may lead to bad values both of the observations y and of the independent or explanatory variables x it is the purpose of the present paper to describe and exemplify two plots which provide checks against such bad values it is argued that these plots and a test for transformations should accompany any thorough regression analysis bad values of the response or outliers have long been detected by a variety of plots of residuals bad values of the explanatory variables are less easily detected even when they lead to an extreme point in the design space since the fitted equation will pass close to such an influential observation the residual may not be especially large even after allowance has been made for the small variance of the fitted value at such a point to detect such behaviour a quantity is needed which exhibits the dependence of the fitted model on each point or group of points discussions of such quantities are given by hoaglin welsch 1978 and more recently by belsley kuh welsch 1980 2 1 by cook weisberg 1980 and by pregibon 1981 the quantity used here is a modification of a statistic proposed by cook 1977 which is derived in 2 two half normal plots are presented in 3 as an aid to interpretation envelopes to the plots are generated by simulation examples of the plots are given in 4 both for simulated data and for the stack loss data given by brownlee 1965 p 454 transformations of this data set are discussed in 5 the paper concludes with some comments on the relationship with robust

bayesian confidence intervals for smoothing splines abstract the frequency properties of wahba s bayesian confidence intervals for smoothing splines are investigated by a large sample approximation and by a simulation study when the coverage probabilities for these pointwise confidence intervals are averaged across the observation points the average coverage probability acp should be close to the nominal level from a frequency point of view this agreement occurs because the average posterior variance for the spline is similar to a consistent estimate of the average squared error and because the average squared bias is a modest fraction of the total average squared error these properties are independent of the bayesian assumptions used to derive this confidence procedure and they explain why the acp is accurate for functions that are much smoother than the sample paths prescribed by the prior this analysis accounts for the choice of the smoothing parameter bandwidth using cross validation in the case of natural splines an adaptive method for avo

an efficient method for calculating smoothing splines using orthogonal transformations a procedure for calculating the mean squared residual and the trace of the influence matrix associated with a polynomial smoothing spline of degree 2m 1 using an orthogonal factorization is presented the procedure substantially overcomes the problem of ill conditioning encountered by a recently developed method which employs a cholesky factorization but still requires only orderm 2 n operations and ordermn storage

spectral measures in c algebras of singular integral operators with shifts abstract the c subalgebra b of b l 2 t generated by all multiplication operators by slowly oscillating and piecewise continuous functions by the cauchy singular integral operator and by the range of a unitary representation of an amenable group of diffeomorphisms which acts topologically freely on t is studied a symbol calculus for the c algebra b and a fredholm criterion for the operators b b are established by using a generalization of the local trajectory method for studying c algebras associated with c dynamical systems this method is related to the allan douglas local principle and its generalization is based on the notion of spectral measure

two ways to quantify uncertainty in geophysical inverse problems we present two approaches to invert geophysical measurements and estimate subsurface properties and their uncertainties when little is known a priori about the size of the errors associated with the data we illustrate these approaches by inverting first arrival traveltimes of seismic waves measured in a vertical well to infer the variation of compressional slowness in depth first we describe a bayesian formulation based on probability distributions that define prior knowledge about the slowness and the data errors we use an empirical bayes approach where hyperparameters are not well known ahead of time e g the variance of the data errors and are estimated from their most probable value given the data the second approach is a non bayesian formulation that we call spectral in the sense that it uses the power spectral density of the traveltime data to constrain the inversion e g to estimate the variance of the data errors in the spectral approach we vary assumptions made about the characteristics of the slowness signal and evaluate the resulting slowness estimates and their uncertainties this approach is computationally simple and starts from a few assumptions these assumptions can be checked during the analysis on the other hand it requires evenly spaced traveltime measurements and it cannot be extended easily e g to data that have gaps in contrast the bayesian framework is based on a general theory that can be generalized immediately but it is more involved computationally despite the conceptual and practical differences we find that the two approaches give the same results when they start from the same assumptions the allegiance to a bayesian or non bayesian formulation matters less than what one is willing to assume when solving the inverse problem

testing for heterogeneity of evoked potential signals using an approximation to an exact permutation test in brain evoked potential research and various other fields a signal embedded in noise is observed in each of many repeated trials we propose a new test of the null hypothesis that the signal is the same in every trial the test has power against the alternative hypothesis that the signal is changing slowly from trial to trial the test statistic is the distance between the average of the trials and a parametric or nonparametric estimate of a slowly changing signal we derive the exact mean and variance of the test statistic under permutation of the trials and use these moments to approximate the exact permutation test we use simulated evoked potential data to evaluate the proposed test and to compare it to three previous tests

kernel smoothing of data with correlated errors abstract kernel smoothing is a common method of estimating the mean function in the nonparametric regression model y f x e where f x is a smooth deterministic mean function and e is an error process with mean zero in this article the mean squared error of kernel estimators is computed for processes with correlated errors and the estimators are shown to be consistent when the sequence of error processes converges to a mixing sequence the standard techniques for bandwidth selection such as cross validation and generalized cross validation are shown to perform very badly when the errors are correlated standard selection techniques are shown to favor undersmoothing when the correlations are predominantly positive and oversmoothing when negative the selection criteria can however be adjusted to correct for the effect of correlation in simulations the standard selection criteria are shown to behave as predicted the corrected criteria are shown to be very effective when the correlation functi

kernel b splines on general lattices the aim of the paper is to provide a computationally effective way to construct stable bases on general non degenerate lattices in particular we define new stable bases on hexagonal lattices and we give some numerical examples which show their usefulness in applications

penalized maximum likelihood estimation in logistic regression and discrimination summary maximum likelihood estimation of the parameters of the binary logistic regression model for pr h i x is discussed with separate discussion of sampling from i the conditional distribution of h given x ii the joint distribution of h and x and iii the conditional distribution of x given h difficulties associated with continuous x in the latter sampling scheme are discussed to avoid these penalized maximum likelihood estimates are introduced which give estimates of the logistic parameters and a nonparametric spline estimate of the marginal distribution of x extensions to multinomial logistic regression are outlined

the signal extraction approach to nonlinear regression and spline smoothing abstract this article shows how to fit a smooth curve polynomial spline to pairs of data values yi xi prior specification of a parametric functional form for the curve is not required the resulting curve can be used to describe the pattern of the data and to predict unknown values of y given x both point and interval estimates are produced the method is easy to use and the computational requirements are modest even for large sample sizes our method is based on maximum likelihood estimation of a signal in noise model of the data we use the kalman filter to evaluate the likelihood function and achieve significant computational advantages over previous approaches to this problem

smoothing spline models for the analysis of nested and crossed samples of curves abstract we introduce a class of models for an additive decomposition of groups of curves stratified by crossed and nested factors generalizing smoothing splines to such samples by associating them with a corresponding mixed effects model the models are also useful for imputation of missing data and exploratory analysis of variance we prove that the best linear unbiased predictors blups from the extended mixed effects model correspond to solutions of a generalized penalized regression where smoothing parameters are directly related to variance components and we show that these solutions are natural cubic splines the model parameters are estimated using a highly efficient implementation of the em algorithm for restricted maximum likelihood reml estimation based on a preliminary eigenvector decomposition variability of computed estimates can be assessed with asymptotic techniques or with a novel hierarchical bootstrap resampling scheme for nested mixed effects models our methods are applied to me

soap film smoothing summary conventional smoothing methods sometimes perform badly when used to smooth data over complex domains by smoothing inappropriately across boundary features such as peninsulas solutions to this smoothing problem tend to be computationally complex and not to provide model smooth functions which are appropriate for incorporating as components of other models such as generalized additive models or mixed additive models we propose a class of smoothers that are appropriate for smoothing over difficult regions of 2 which can be represented in terms of a low rank basis and one or two quadratic penalties the key features of these smoothers are that they do not smooth across boundary features that their representation in terms of a basis and penalties allows straightforward incorporation as components of generalized additive models mixed models and other non standard models that smoothness selection for these model components is straightforward to accomplish in a computationally efficient manner via generalized cross validation akaike s information criterion or restricted maximum likelihood for example and that their low rank means that their use is computationally efficient

extending the scope of wavelet regression methods by coefficient dependent thresholding abstract various aspects of the wavelet approach to nonparametric regression are considered with the overall aim of extending the scope of wavelet techniques to irregularly spaced data to regularly spaced datasets of arbitrary size to heteroscedastic and correlated data and to data that contain outliers the core of the methodology is an algorithm for finding ail of the variances and within level covariances in the wavelet table of a sequence with given covariance structure if the original covariance matrix is band limited then the algorithm is linear in the length of the sequence the variance calculation algorithm allows data on any set of independent variable values to be treated by first interpolating to a fine regular grid of suitable length and then constructing a wavelet expansion of the gridded data various thresholding methods are discussed and investigated exact risk formulas for the mean square error of the methodology for given design are derived good performance is obtained by nois

detecting generalized synchronization between chaotic signals a kernel based approach a unified framework for analysing generalized synchronization in coupled chaotic systems from data is proposed the key of the proposed approach is the use of the kernel methods recently developed in the field of machine learning several successful applications are presented which show the capability of the kernel based approach for detecting generalized synchronization and dynamical change of the coupling strength between two chaotic systems can be captured by the proposed approach it is also discussed how the kernel parameter is suitably chosen from data

occam s inversion a practical algorithm for generating smooth models from electromagnetic sounding data the inversion of electromagnetic sounding data does not yield a unique solution but inevitably a single model to interpret the observations is sought we recommend that this model be as simple or smooth as possible in order to reduce the temptation to overinterpret the data and to eliminate arbitrary discontinuities in simple layered models to obtain smooth models the nonlinear forward problem is linearized about a starting model in the usual way but it is then solved explicitly for the desired model rather than for a model correction by parameterizing the model in terms of its first or second derivative with depth the minimum norm solution yields the smoothest possible model rather than fitting the experimental data as well as possible which maximizes the roughness of the model the smoothest model which fits the data to within an expected tolerance is sought a practical scheme is developed which optimizes the step size at each iteration and retains the computational efficiency of layered models resulting in a stable and rapidly convergent algorithm the inversion of both magnetotelluric and schlumberger sounding field data and a joint magnetotelluric resistivity inversion demonstrate the method and show it to have practical application

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

resolution and uncertainty in 1d csem inversion a bayesian approach and open source implementation we show that resolution and uncertainty in csem inversion is most naturally approached using a bayesian framework resolution can be inferred by either hierarchical models with free parameters for effective correlation lengths bayesian smoothing or model choice frameworks applied to variable resolution spatial models bayesian splitting merging we find that typical 1d csem data can be modelled with quite parsimonious models typically o 10 parameters per common midpoint efficient opt imizations for the csem problem must address the challenges of poor scaling strong nonlinearity multimodality and the necessity of bound constraints the posterior parameter uncertainties are frequently controlled by the nonlinearity and linearised approaches to uncertainty are usually very poor in markov chain monte carlo mcmc approaches the nonlinearity and poor scaling make good mixing hard to achieve a novel approximate frequentist method we call the bayesianized parametric bootstrap sometimes called randomized maximum likelihood is much more efficient than mcmc in this problem is considerably better than linearized analysis but tends to modestly overstate uncertainties the software that implements these ideas for the 1d csem problem is made available under an open source license agreement

a signal extraction approach to the estimation of treatment and control curves abstract suppose that the response of a group of subjects which we call the control group is monitored over time at a given time after the start of the experiment the experimenter intervenes and applies a treatment to a subset of the group we propose a signal extraction approach to the simultaneous estimation of the treatment and control mean response curves and their rates of change imposing a smooth transition constraint between the two curves at the time of intervention by expressing the model in state space form we show how to compute the likelihood and the cross validation function efficiently in o n operations together with the response curve estimates and their standard errors we also show that the curve estimates are solutions to a penalized least squares problem a small simulation study to assess the performance of the generalized cross validation and marginal likelihood estimates of the smoothing parameter is carried out and the methodology in the article is applied to an experiment

minimum bias multiple taper spectral estimation two families of orthonormal tapers are proposed for multitaper spectral analysis minimum bias tapers and sinusoidal tapers spl upsisup k where spl upsisub nsup k spl radic 2 n 1 sin spl pi kn n 1 and n is the number of points the resulting sinusoidal multitaper spectral estimate is s spl circ f 1 2k n 1 spl sigmasub j 1sup k y f j 2n 2 y f j 2n 2 sup 2 where y f is the fourier transform of the stationary time series s f is the spectral density and k is the number of tapers for fixed j the sinusoidal tapers converge to the minimum bias tapers like 1 n since the sinusoidal tapers have analytic expressions no numerical eigenvalue decomposition is necessary both the minimum bias and sinusoidal tapers have no additional parameter for the spectral bandwidth the bandwidth of the jth taper is simply 1 n centered about the frequencies spl plusmn j 2n 2 thus the bandwidth of the multitaper spectral estimate can be adjusted locally by simply adding or deleting tapers the band limited spectral concentration spl intsub wsup w v f sup 2 df of both the minimum bias and sinusoidal tapers is very close to the optimal concentration achieved by the slepian 1978 tapers in contrast the slepian tapers can have the local bias spl intsub 1 2 sup 1 2 f sup 2 v f sup 2 df much larger than of the minimum bias tapers and the sinusoidal tapers

rational l s category and its applications let s be a 1 connected cw complex of finite type and put cat0 s lusternik schnirelmann category of the localization sq this invariant is char acterized in terms of the minimal model of s it is shown that if is the fibre inclusion of a fibration it is also shown that if dim h s q oo then either dim it s q oo or the groups nk s q grow exponentially with k introduction in this paper topological space means a pointed path connected normal space s1 such that the inclusion of the basepoint is a cofibration and the rational singular homology is finite dimensional in each degree all vector spaces and algebras have q as ground field homology and cohomology of spaces is singular with rational coefficients we shall be dealing with category of spaces cat s in the sense of lusternik and schnirelmann 18 a recent survey is given in 15 but our definition differs by one from that of 15 so that for us spheres have category one not two our purpose is to provide a computationally useful characterization of the rational category cat0 5 of space 5 and to apply this in a variety of situations the definition of cat0 5 is sketched below and given precisely in 4 it satisfies cat0 s cauls when 5 is 1 connected cat0 5 coincides with the category of the localization sq which has been studied by toomer in 26 and 27 and lemaire and sigrist in 17 the characterization problem was posed by berstein 26 most of the applications depend on the following fundamental mapping theorem which we derive as a consequence of our characterization