on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

semiparametric nonlinear mixed effects models and their applications nonlinear mixed effects models nlmms and self modeling nonlinear regression semor models are often used to fit repeated measures data they use a common function shared by all subjects to model variation within each subject and some fixed and or random parameters to model variation between subjects the parametric nlmm may be too restrictive and the semiparametric semor model ignores correlations within each subject in this article we propose a class of semiparametric nonlinear mixed effects models snmms that extend nlmms semor models and many other existing models in a natural way a snmm assumes that the mean function depends on some parameters and nonparametric functions the parameters provide an interpretable data summary the nonparametric functions provide flexibility to allow the data to decide some unknown or uncertain components such as the shape of the mean response over time a second stage model with fixed and random effects is used to model the parameters smoothing splines are us

recurrence plots revisited abstract we show that recurrence plots rps give detailed characterizations of time series generated by dynamical systems driven by slowly varying external forces for deterministic systems we show that rps of the time series can be used to reconstruct the rp of the driving force if it varies sufficiently slowly if the driving force is one dimensional its functional form can then be inferred up to an invertible coordinate transformation the same results hold for stochastic systems if the rp of the time series is suitably averaged and transformed these results are used to investigate the nonlinear prediction of time series generated by dynamical systems driven by slowly varying external forces we also consider the problem of detecting a small change in the driving force and propose a surrogate data technique for assessing statistical significance numerically simulated time series and a time series of respiration rates recorded from a subject with sleep apnea are used as illustrative examples

nonlinear structures in electroencephalogram signals abstract we apply a nonlinear prediction algorithm to investigate the presence of nonlinear structure in electroencephalogram eeg recordings the eeg signal could be modeled as a realization of a nonlinear model plus a residual noise uncorrelated gaussian noise using linear and nonlinear models we analyze the statistical nature of these residual noises in the case of epileptic patients and normal subjects we found that the residual noise presents gaussian distribution for epileptic patients if a nonlinear model is used whereas in the case of normal subjects the residual noise will exhibit a gaussian distribution only if a linear model autoregressive is used these results provide another evidence of the nonlinear character of the epileptic seizure recordings while the normal eeg seems to be better described as linearly correlated noise

smoothing spline semiparametric density models density estimation plays a fundamental role in many areas of statistics and machine learning parametric nonparametric and semiparametric density estimation methods have been proposed in the lite

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

spatiotemporal reconstruction of list mode pet data we describe a method for computing a continuous time estimate of tracer density using list mode positron emission tomography data the rate function in each voxel is modeled as an inhomogeneous poisson process whose rate function can be represented using a cubic b spline basis the rate functions are estimated by maximizing the likelihood of the arrival times of detected photon pairs over the control vertices of the spline modified by quadratic spatial and temporal smoothness penalties and a penalty term to enforce nonnegativity randoms rate functions are estimated by assuming independence between the spatial and temporal randoms distributions similarly scatter rate functions are estimated by assuming spatiotemporal independence and that the temporal distribution of the scatter is proportional to the temporal distribution of the trues a quantitative evaluation was performed using simulated data and the method is also demonstrated in a human study using sup 11 c raclopride

smoothing spline anova for multivariate bernoulli observations with application to ophthalmology data we combine a smoothing spline analysis of variance ss anova model and a log linear model to build a partly flexible model for multivariate bernoulli data the joint distribution conditioning on the predictor variables is estimated the log odds ratio is used to measure the association between outcome variables a numerical scheme based on the block one step successive over relaxation sor newton ralphson algorithm is proposed to obtain an approximate solution for the variational problem we extend the generalized approximate cross validation gacv and the randomized gacv for choosing smoothing parameters to the case of multivariate bernoulli responses the randomized version is fast and stable to compute and is used to adaptively select smoothing parameters in each block onestep sor iteration approximate bayesian confidence intervals are obtained for the flexible estimates of the conditional logit functions simulation studies are conducted to check the performance of the proposed method using the com

smoothing spline density estimation a dimensionless automatic algorithm abstract as a sequel to an earlier article by gu and qiu this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines the algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem which was shown by gu and qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution the smoothing parameter is updated jointly with the estimate in a performance oriented iteration via a cross validation performance estimate where the performance is measured by proxies of the symmetrized kullback leibler distance between the true density and the estimate simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general the method is also applied to some real data sets the algorithm is implem

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

an adaptive orthogonal search algorithm for model subset selection and non linear system identification a new adaptive orthogonal search aos algorithm is proposed for model subset selection and non linear system identification model structure detection is a key step in any system identification problem this consists of selecting significant model terms from a redundant dictionary of candidate model terms and determining the model complexity model length or model size the final objective is to produce a parsimonious model that can well capture the inherent dynamics of the underlying system in the new aos algorithm a modified generalized cross validation criterion called the adjustable prediction error sum of squares apress is introduced and incorporated into a forward orthogonal search procedure the main advantage of the new aos algorithm is that the mechanism is simple and the implementation is direct and easy and more importantly it can produce efficient model subsets for most non linear identification problems

model misspecification tests model building and predictability in complex systems abstract in this paper we show that significant advantages can be realized by using functions of simple variables such as time lags as directions in reconstruction spaces for complicated time series first we show that more sensitive model misspecification tests can be constructed when the candidate model is used as one of the directions in the reconstruction space this naturally results in misspecification tests based on the construction of conditional probabilities which can reveal misspecified models even when other powerful methods such as the bds test applied to a sequence of residuals fails second we show that model building and predictability of the time series can be substantially improved by using an informational criterion to determine the functions with which to associate the directions of the reconstruction space this criterion is of the form of a conditional probability and is related to a measure of the short term predictability of the time series we consider specifically an example in which the search space of functions is linear combinations of time lags and we compare the resulting model with a linear least squares fit to the data we also show that this optimized reconstruction space can improve longer term predictability we demonstrate this by comparing longer term predictions made using the optimized space with predictions made using a simple time lagged reconstruction space

survival trees by goodness of split abstract a tree based method for censored survival data is developed based on maximizing the difference in survival between groups of patients represented by nodes in a binary tree the method includes a pruning algorithm with optimal properties analogous to the classification and regression tree cart pruning algorithm uniform convergence of the estimates of the conditional cumulative hazard and survival functions is discussed and an example is given to show the utility of the algorithm for developing prognostic classifications for patients

periodic solutions for a kind of second order neutral functional differential equations in this paper criteria are established for the existence of periodic solutions to a second order sublinear neutral differential equation our method is based on careful a priori estimation and continuation theorem and our sublinear condition is an improvement of the boundedness condition in some recent results

time series and dependent variables we present a new method for analyzing time series which is designed to extract inherent deterministic dependencies in the series the method is particularly suited to series with broad band spectra such as chaotic series with or without noise we derive quantities j e based on conditional probabilities whose magnitude roughly speaking is an indicator of the extent to which the kth element in the series is a deterministic function of the k j th element to within a measurement uncertainty e we apply our method to a number of deterministic time series generated by chaotic processes such as the tent logistic and h non maps as well as to sequences of quasi random numbers in all cases the 6j correctly indicate the expected dependencies we also show that the j are robust to the addition of substantial noise in a deterministic process in addition we derive a predictability index which is a measure of the extent to which a time series is predictable given some tolerance e finally we discuss the behavior of the 6i as e approaches zero

periodic solutions for generalized lienard neutral equation with variable parameter abstract by means of mawhin s continuation theorem existence criteria is established for the periodic solutions to a neutral equation with variable parameter it is worth stating that the coefficient c is not a constant which is different from the corresponding ones of past work

dimension reduction in binary response regression abstract the idea of dimension reduction without loss of information can be quite helpful for guiding the construction of summary plots in regression without requiring a prespecified model focusing on the central subspace we investigate such sufficient dimension reduction in regressions with a binary response three existing methods sliced inverse regression principal hessian direction and sliced average variance estimation and one new method difference of covariances are studied for their ability to estimate the central subspace and produce sufficient summary plots combining these numerical methods with the graphical methods proposed earlier by cook leads to a novel paradigm for the analysis of binary response regressions

using surrogate data to detect nonlinearity in time series we address the issue of reliably discriminating between chaos and noise from a time series in particular we are interested in avoiding claims of chaos when simpler models such as linearly correlated noise can explain the data we take a statistical approach and use a form of bootstrapping to detect nonlinearity by showing that a given linear model is unlikely to have produced the data our method requires the careful statement of a null hypothesis which characterizes a candidate linear process the generation of an ensemble of surrogate data sets which are similar to the original time series but consistent with the null hypothesis and the computation of a discriminating statistic for the original and for each of the surrogate data sets the idea is to test the original time series against the null hypothesis by checking whether the discriminating statistic computed for the original time series differs significantly from the statistics computed for each of the surrogate sets we present algorithms for generating surrogate data under various null hypotheses and we show the results of numerical experiments on artificial data using correlation dimension lyapunov exponent and forecasting error as discriminating statistics finally we consider a number of experimental timemore series including sunspots electroencephalogram eeg signals and fluid convection and evaluate the statistical significance of the evidence for nonlinear structure in each case 52 refs 10 figs less

dependent variables in broad band continuous time series in this paper we continue our development of new methods for the analysis of broad band time series by deriving quantities which are able to indicate deterministic dependence of an element in one time series on elements in other time series these methods are very broadly applicable and are particularly well suited to the study of continuous time series in which the value of the function may depend on derivatives of the function itself or on other quantities we apply our methods to a number of mathematical examples including the lorentz equation the h6non heiles equations the forced brusselator and the mackey glass equation we show that our methods are very successful at indicating deterministic dependencies in these systems even if the time series are highly chaotic statistical aspects of our procedure are discussed as are a number of interesting and surprising epistomological implications

local and global asymptotic inference in smoothing spline models this article studies local and global inference for smoothing spline estimation in a unified asymptotic framework we first introduce a new technical tool called functional bahadur representation which significantly generalizes the traditional bahadur representation in parametric models that is bahadur ann inst statist math 37 1966 577 580 equipped with this tool we develop four interconnected procedures for inference i pointwise confidence interval ii local likelihood ratio testing iii simultaneous confidence band iv global likelihood ratio testing in particular our confidence intervals are proved to be asymptotically valid at any point in the support and they are shorter on average than the bayesian confidence intervals proposed by wahba j r stat soc ser b stat methodol 45 1983 133 150 and nychka j amer statist assoc 83 1988 1134 1143 we also discuss a version of the wilks phenomenon arising from local global likelihood ratio testing it is also worth noting that our simultaneous confidence bands are the first ones applicable to general quasi likelihood models furthermore issues relating to optimality and efficiency are carefully addressed as a by product we discover a surprising relationship between periodic and nonperiodic smoothing splines in terms of inference

regularization in the selection of radial basis function centers subset selection and regularization are two well known techniques that can improve the generalization performance of nonparametric linear regression estimators such as radial basis function networks this paper examines regularized forward selection rfs a combination of forward subset selection and zero order regularization an efficient implementation of rfs into which either delete 1 or generalized cross validation can be incorporated and a reestimation formula for the regularization parameter are also discussed simulation studies are presented that demonstrate improved generalization performance due to regularization in the forward selection of radial basis function centers

hybrid adaptive splines abstract an adaptive spline method for smoothing is proposed that combines features from both regression spline and smoothing spline approaches one of its advantages is the ability to vary the amount of smoothing in response to the inhomogeneous curvature of true functions at different locations this method can be applied to many multivariate function estimation problems which is illustrated by an application to smoothing temperature data on the globe the method s performance in a simulation study is found to be comparable to the wavelet shrinkage methods proposed by donoho and johnstone the problem of how to count the degrees of freedom for an adaptively chosen set of basis functions is addressed this issue arises also in the mars procedure proposed by friedman and other adaptive regression spline procedures

state space reconstruction in the presence of noise takens theorem demonstrates that in the absence of noise a multidimensional state space can be reconstructed from a scalar time series this theorem gives little guidance however about practical considerations for reconstructing a good state space we extend takens treatment applying statistical methods to incorporate the effects of observational noise and estimation error we define the distortion matrix which is proportional to the conditional covariance of a state given a series of noisy measurements and the noise amplification which is proportional to root mean square time series prediction errors with an ideal model we derive explicit formulae for these quantities and we prove that in the low noise limit minimizing the distortion is equivalent to minimizing the noise amplification we identify several different scaling regimes for distortion and noise amplification and derive asymptotic scaling laws when the dimension and lyapunov exponents are sufficiently large these scaling laws show that no matter how the state space is reconstructed there is an explosion in the noise amplification from a practical point of view determinism is lost and the time series is effectively a random process in the low noise large data limit we show that the technique of local singular value decomposition is an optimal coordinate transformation in the sense that it achieves the minimum distortion in a state space of the lowest possible dimension however in numerical experiments we find that estimation error complicates this issue for local approximation methods we analyze the effect of reconstruction on estimation error derive a scaling law and suggest an algorithm for reducing estimation errors

periodic solutions of neutral differential equation with multiple deviating arguments in this paper by means of an abstract continuous theorem of k set contractive operator and some analysis techniques the authors study the existence of periodic solution of the nonlinear neutral differential equation with multiple deviating argumentsddt x t cx email protected email protected i 1 email protected i t f x email protected i t email protected j 1 email protected j t g x email protected j t p t a new result is obtained

periodic solutions for some nonlinear differential equations of neutral type where f ir x ir ir is a caratheodory function f x is 2n periodic for every x ci e r and r e io 2n this problem has been studied by sadovskii in 151 by hale and mawhin in 6 in the case ial 1 to the previous one our interest is particularly motivated by the fact that represents a simple and meaningful model for a general phenomenon which often arises in the study of functional differential equations in fact as is well known in some cases a delayed differential equation can be seen as a perturbation of an ordinary differential equation in particular this occurs when the delayed term in the equation can be considered smaller than the ordinary one or dominated by it when such domination is available many good properties can be proved and several results such as those of 6 and 16 take advantage of this fact on the contrary when no domination is present the situation is much more complicated and in general very little is known in problem the delayed term can be considered dominated by the ordinary one whenever ial 1 as a matter of fact in this case one can prove the invertibility of the differential operator l x x a r ux the compactness of its inverse and even some a priori estimates for the solutions consequently one can reduce to a fixed point problem and solve it by means of some topological degree argument however when 11 1 so that the weight of the two terms x t and x t t is the same the above properties fail and moreover for some delays the kernel of the differential operator lx x a r is infinite dimensional therefore one must look for different methods problem is thus a simple but nevertheless significant example of the considerations illustrated by hale in 5 about the difficulty of undertaking a unified approach in the study of functional differential equations at the same time it provides an analogy between neutral equations and the wave equation in the line suggested by hale in 5

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

periodic solutions to a second order nonlinear neutral functional differential equation in the critical case abstract in this paper the authors study the existence of periodic solutions for a second order neutral functional differential equation x t cx t f x t x t g t x t t e t in the critical case c 1 by employing mawhin s continuation theorem and some analysis techniques sufficient conditions are given for the existence of periodic solutions

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

smoothing spline density estimation theory in this article a class of penalized likelihood probability density estimators is proposed and studied the true log density is assumed to be a member of a reproducing kernel hilbert space on a finite domain not necessarily univariate and the estimator is defined as the unique unconstrained minimizer of a penalized log likelihood functional in such a space under mild conditions the existence of the estimator and the rate of convergence of the estimator in terms of the symmetrized kullback leibler distance are established to make the procedure applicable a semiparametric approximation of the estimator is presented which sits in an adaptive finite dimensional function space and hence can be computed in principle the theory is developed in a generic setup and the proofs are largely elementary algorithms are yet to follow

on the existence of periodic solutions for neutral functional differential equation abstract by means of mawhin s continuation theorem existence criteria are established for the periodic solutions of neutral functional differential equation

periodic solutions to a kind of neutral functional differential equation in the critical case in this paper we consider a kind of neutral functional differential equation as follows x t cx t gt xt t e t in the critical case c 1 by employing the topological degree theory and some analysis techniques we obtain some new results on the existence of periodic solutions

periodic solutions of impulsive systems with a small delay for an impulsive system with delay it is proved that if the corresponding system without delay has an isolated omega periodic solution then in any neighbourhood of this orbit the system considered also has an omega periodic solution if the delay is small enough

testing for nonlinearity in time series the method of surrogate data we describe a statistical approach for identifying nonlinearity in time series the method first specifies some linear process as a null hypothesis then generates surrogate data sets which are consistent with this null hypothesis and finally computes a discriminating statistic for the original and for each of the surrogate data sets if the value computed for the original data is significantly different than the ensemble of values computed for the surrogate data then the null hypothesis is rejected and nonlinearity is detected we discuss various null hypotheses and discriminating statistics the method is demonstrated for numerical data generated by known chaotic systems and applied to a number of experimental time series which arise in the measurement of superfluids brain waves and sunspots we evaluate the statistical significance of the evidence for nonlinear structure in each case and illustrate aspects of the data which this approach identifies

tensor product space anova models to deal with the curse of dimensionality in high dimensional nonparametric problems we consider using tensor product space anova models which extend the popular additive models and are able to capture interactions of any order the multivariate function is given an anova decomposition that is it is expressed as a constant plus the sum of functions of one variable main effects plus the sum of functions of two variables two factor interactions and so on we assume the interactions to be in tensor product spaces we show in both regression and white noise settings the optimal rate of convergence for the tps anova model is within a log factor of the one dimensional optimal rate and that the penalized likelihood estimator in tps anova achieves this rate of convergence the quick optimal rate of the tps anova model makes it very preferable in high dimensional function estimation many properties of the tensor product space of sobolev hilbert spaces are also given

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

statistical tests for deterministic effects in broad band time series we derive a normalized version of the indicators of savit and green and prove that these normalized statistics have asymptotically a normal distribution with a mean of zero and standard deviation of one if the time series is random in the sense of being lid independent and identically distributed we verify this result numerically and study the magnitude of the finite size effects we also show that these statistics are very sensitive to the existence of deterministic effects in the series even if the underlying deterministic structure is complex such as those generated by a chaotic system we show that with moderate amounts of data the statistics can easily indicate the presence of an underlying attractor even in the presence of iid noise which is as large as or greater than the signal finally we discuss the generalization of our approach to include 1 other null hypotheses besides lid which express assumptions of specific dependencies and 2 the study of deterministic effects between more than one time series

asymptotic optimality for c_p c_l cross validation and generalized cross validation discrete index set c p c l cross validation and generalized cross validation are useful data driven techniques for selecting a good estimate from a proposed class of linear estimates the asymptotic behaviors of these procedures are studied some easily interpretable conditions are derived to demonstrate the asymptotic optimality it is argued that cross validation and generalized cross validation can be viewed as some special ways of applying c l applications in nearest neighbor nonparametric regression and in model selection are discussed in detail

adaptive principal surfaces abstract we develop a nonlinear generalization of principal components analysis a principal surface of the data is constructed adaptively using some ideas from the mars procedure of friedman we explore applications to curve and surface reconstruction and to data summarization

reconstructing attractors from scalar time series a comparison of singular system and redundancy criteria abstract a delay vector phase space reconstruction in which the delay time satisfiesa minimum redundancy criterion is compared with a reconstruction obtained using a singular system approach minimum redundancy produces the better reconstruction the reconstructions are compared using a distortion functional d which measures how well the location of a point in the original phase space can be determined on the basis of its image under the reconstruction process the superiority of the redundancy analysis over the singular system analysis is found to arise from the former s foundation on the notion of general independence as opposed to the latter s foundation on the notion of linear independence

inference in smoothing spline analysis of variance summary smoothing spline analysis of variance decomposes a multivariate function into additive components this decomposition not only provides an efficient way to model a multivariate function but also leads to meaningful inference by testing whether a certain component equals 0 no formal procedure is yet available to test such a hypothesis we propose an asymptotic method based on the likelihood ratio to test whether a functional component is 0 this test allows us to choose an optimal model and to compare groups of curves we first develop the general theory by exploiting the connection between mixed effects models and smoothing splines we then apply this to compare two groups of curves and to select an optimal model in a two dimensional problem a small simulation is used to assess the finite sample performance of the likelihood ratio test