proximal decomposition on the graph of a maximal monotone operator we present an algorithm to solve find x y in a times a bot such that y in tx where a is a subspace and t is a maximal monotone operator the algorithm is based on the proximal decomposition on the graph of a monotone operator and we show how to recover spingarn s decomposition method we give a proof of convergence that does not use the concept of partial inverse and show how to choose a scaling factor to accelerate the convergence in the strongly monotone case numerical results performed on quadratic problems confirm the robust behaviour of the algorithm

dc programming and dca thirty years of developments the year 2015 marks the 30th birthday of dc difference of convex functions programming and dca dc algorithms which constitute the backbone of nonconvex programming and global optimization in this article we offer a short survey on thirty years of developments of these theoretical and algorithmic tools the survey is comprised of three parts in the first part we present a brief history of the field while in the second we summarize the state of the art results and recent advances we focus on main theoretical results and dca solvers for important classes of difficult nonconvex optimization problems and then give an overview of real world applications whose solution methods are based on dca the third part is devoted to new trends and important open issues as well as suggestions for future developments

some aspects of the spline smoothing approach to non parametric regression curve fitting non parametric regression using cubic splines is an attractive flexible and widely applicable approach to curve estimation although the basic idea was formulated many years ago the method is not as widely known or adopted as perhaps it should be the topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology particular subjects covered include the basic principles of the method the relation with moving average and other smoothing methods the automatic choice of the amount of smoothing and the use of residuals for diagnostic checking and model adaptation the question of providing inference regions for curves and for relevant properties of curves is approached via a finite dimensional bayesian formulation

large scale molecular optimization from distance matrices by a d c optimization approach a so called dca method based on a d c difference of convex functions optimization approach algorithm for solving large scale distance geometry problems is developed different formulations of equivalent d c programs in the l 1 approach are stated via the lagrangian duality without gap relative to d c programming and new nonstandard nonsmooth reformulations in the l infty approach resp the l 1 l infty approach are introduced substantial subdifferential calculations permit us to compute sequences of iterations in the dca quite simply the computations actually require matrix vector products and only one cholesky factorization resp with an additional solution of a convex program in the l 1 approach resp the l 1 l infty approach and allow the exploitation of sparsity in the large scale setting two techniques respectively using shortest paths between all pairs of atoms to generate the complete dissimilarity matrix and the spanning trees procedure are investigated in order to compute a good starting point for the dca finally many numerical simulations of the molecular optimization problems with up to 12567 variables are reported which prove the practical usefulness of the nonstandard nonsmooth reformulations the globality of found solutions and the robustness and efficiency of our algorithms

analytic derivatives for estimation of linear dynamic models abstract this paper develops two algorithms algorithm 1 computes the exact gaussian log likelihood function its exact gradient vector and an asymptotic approximation of its hessian matrix for discrete time linear dynamic models in state space form algorithm 2 derived from algorithm 1 computes the exact sample information matrix of this likelihood function the computed quantities are analytic not numerical approximations and should therefore be useful for reliably quickly and accurately i checking local identifiability of parameters by checking the rank of the information matrix ii using the gradient vector and hessian matrix to compute maximum likelihood estimates of parameters with newton methods and iii computing asymptotic covariances cramer rao bounds of the parameter estimates with the hessian or the information matrix the principal contribution of the paper is algorithm 2 which extends to multivariate models the univariate results of porat and friedlander 1 by relying on the kalman filter instead of the levinson durbin filter used by porat and friedlander algorithms 1 and 2 can automatically handle any pattern of missing or linearly aggregated data although algorithm 1 is well known it is treated in detail in order to make the paper self contained

smoothing splines for longitudinal data in a longitudinal data model with fixed and random effects polynomials are used to model the fixed effects and smoothing polynomial splines are used to model the within subject random effect curves the splines are generated by modelling the data for each subject as observations of an integrated random walk with observational error the initial conditions for each subject s deviation from the fixed effect curve are assumed to have zero mean and arbitrary covariance matrix which is estimated by maximum likelihood producing an empirical bayes estimate this is in contrast to modelling a single curve using a diffuse prior an example is presented using unbalanced longitudinal data from a pilot study in breast cancer patients

estimation prediction and interpolation for nonstationary series with the kalman filter abstract we show how our definition of the likelihood of an autoregressive integrated moving average arima model with missing observations alternative to that of kohn and ansley and based on the usual assumptions made in estimation of and forecasting with arima models permits a direct and standard state space representation of the nonstationary original data so that the ordinary kalman filter and fixed point smoother can be efficiently used for estimation forecasting and interpolation in this way the problem of estimating missing values in nonstationary series is considerably simplified the results are extended to regression models with arima errors and a computer program is available from the authors

the likelihood for a state space model summary this paper derives an expression for the likelihood for a state space model the expression can be evaluated with the kalman filter initialized at a starting state estimate of zero and associated estimation error covariance matrix of zero adjustment for initial conditions can be made after filtering accordingly initial conditions can be modelled without filtering implications in particular initial conditions can be modelled as diffuse the connection between the diffuse and concentrated likelihood is also displayed

the estimation of residual variance in nonparametric regression summary a wide class of estimators of the residual variance in nonparametric regression is considered namely those that are quadratic in the data unbiased for linear regression and always nonnegative the minimax mean squared error estimator over a natural class of regression functions is derived this optimal estimator has an interesting structure and is closely related to a minimax estimator of the regression curve itself

the exact likelihood for a state space model with stochastic inputs abstract in this work we derive exact and approximate expressions for the conditional mean and variance of the initial state of a state space model allowing for unit roots and stochastic inputs these results provide adequate initial conditions to compute the exact likelihood using the kalman filter the exact conditional moments are the best choice when the stochastic structure of the inputs is known if this is not the case the approximate expressions are a good alternative as some simulation results illustrate

filtering and smoothing algorithms for state space models abstract the paper reviews and generalizes recent filtering and smoothing algorithms for observations generated by a state model in particular the paper discusses the modified kalman filter derived by ansley and kohn 1985 and kohn and ansley 1986 to deal with state space models having partially diffuse initial conditions and shows how to compute the limiting normalized likelihood of the observations for such models the paper also discusses and generalizes the new smoothing algorithm presented by kohn and ansley 1987c 1989 and extends it to state space models with partially diffuse initial conditions

a d c optimization algorithm for solving the trust region subproblem this paper is devoted to difference of convex functions d c optimization d c duality local and global optimality conditions in d c programming the d c algorithm dca and its application to solving the trust region problem the dca is an iterative method that is quite different from well known related algorithms thanks to the particular structure of the trust region problem the dca is very simple requiring only matrix vector products and in practice converges to the global solution the inexpensive implicitly restarted lanczos method of sorensen is used to check the optimality of solutions provided by the dca when a nonglobal solution is found a simple numerical procedure is introduced both to find a feasible point having a smaller objective value and to restart the dca at this point it is shown that in the nonconvex case the dca converges to the global solution of the trust region problem using only matrix vector products and requiring at most 2m 2 restarts where m is the number of distinct negative eigenvalues of the coefficient matrix that defines the problem numerical simulations establish the robustness and efficiency of the dca compared to standard related methods especially for large scale problems

the nonparametric estimation of branching curves abstract the spline smoothing approach to nonparametric regression is extended to cover the estimation of curves that branch at certain specified points in time this extension is motivated and illustrated by data from an experiment on sunflowers apart from providing insight into this particular data analytic problem the work described demonstrates the power of the roughness penalty approach in coping with curve estimation of a nonstandard kind particularly in the context of certain experiments in horticulture and arboriculture the experimental material may be treated identically to the control up to a certain known time part of the way through the experiment to display and investigate the effect of treatments applied during growth it is of interest to estimate a branching curve in the simplest case we want to study two growth curves that are common up to a certain known time and then diverge or branch after the application of a treatment to part of the experimental material the spline smoothing

a bayesian approach to additive semiparametric regression abstract we present a bayesian approach to estimating an additive semiparametric regression model which is robust to outliers the unknown curves are estimated by posterior means and are shown to be smoothing splines by using markov chain monte carlo an o mn algorithm is produced where n is the sample size and m is the total number of markov chain iterations previous exact approaches required o n3 operations making the estimation of large data sets impractical efficient methods for estimating the posterior means using mixture and backfitting estimates are developed the properties of the curve estimates are studied empirically using both simulated and real examples

convergence of a subgradient method for computing the bound norm of matrices abstract let and be any norms on r m and r n respectively we study a subgradient method for computing the associated bound norm s a sup ax x 1 a nonconvex optimization problem it is proved that homodual method converges when one of the norms and is polyhedral

a reproducing kernel hilbert space approach for q ball imaging diffusion magnetic resonance mr imaging has enabled us to reveal the white matter geometry in the living human brain the q ball technique is widely used nowadays to recover the orientational heterogeneity of the intra voxel fiber architecture this article proposes to employ the funk radon transform in a hilbert space with a reproducing kernel derived from the spherical laplace beltrami operator thus generalizing previous approaches that assume a bandlimited diffusion signal the function estimation problem is solved within a tikhonov regularization framework while a gaussian process model allows for the selection of the smoothing parameter and the specification of confidence bands shortcomings of q ball imaging are discussed

on gibbs sampling for state space models summary we show how to use the gibbs sampler to carry out bayesian inference on a linear state space model with errors that are a mixture of normals and coefficients that can switch over time our approach simultaneously generates the whole of the state vector given the mixture and coefficient indicator variables and simultaneously generates all the indicator variables conditional on the state vectors the states are generated efficiently using the kalman filter we illustrate our approach by several examples and empirically compare its performance to another gibbs sampler where the states are generated one at a time the empirical results suggest that our approach is both practical to implement and dominates the gibbs sampler that generates the states one at a time

global continuation for distance geometry problems distance geometry problems arise in the determination of protein structure we consider the case where only a subset of the distances between atoms is given and formulate this distance geometry problem as a global minimization problem with special structure we show that global smoothing techniques and a continuation approach for global optimization can be used to determine global solutions of this problem reliably and efficiently the global continuation approach determines a global solution with less computational effort than is required by a standard multistart algorithm moreover the continuation approach usually finds the global solution from any given starting point while the multistart algorithm tends to fail

multimodal logic programming using equational and order sorted logic in previous work 2 we proposed a method for automated modal theorem proving based on algebraic concepts and equational techniques basically it uses the translation of modal logic into a specially tailored multi sorted logic called path logic in this paper we extend the method for multi modal logic and apply it to multi modal logic programming the multi modal systems we consider are arbitrary mixing of first order modal systems of type kd kt kd4 or kt4 with interaction axioms of the form ia ja roughly with each modal subsystem is associated a sort in path logic and a specific set of equations and the interaction axioms are captured by the order relation between sorts hence again all the modal logical rules are coded in the unification algorithm if one considers horn clauses in the usual sense we get a logic programming system pathlog for which standard theoretical results apply in pathlog one can either write programs directly in the language of path logic or in various multi modal logics the modal formulas being automatically translated and put in clausal form for instance we obtain as a particular case a system for temporal logic programming which subsumes templog of 1 and whose completeness results immediately from our general theorems

functional mixed effects models in this article a new class of functional models in which smoothing splines are used to model fixed effects as well as random effects is introduced the linear mixed effects models are extended to nonparametric mixed effects models by introducing functional random effects which are modeled as realizations of zero mean stochastic processes the fixed functional effects and the random functional effects are modeled in the same functional space which guarantee the population average and subject specific curves have the same smoothness property these models inherit the flexibility of the linear mixed effects models in handling complex designs and correlation structures can include continuous covariates as well as dummy factors in both the fixed or random design matrices and include the nested curves models as special cases two estimation procedures are proposed the first estimation procedure exploits the connection between linear mixed effects models and smoothing splines and can be fitted using existing software the second procedure is a sequential estimation procedure using kalman filtering this algorithm avoids inversion of large dimensional matrices and therefore can be applied to large data sets a generalized maximum likelihood gml ratio test is proposed for inference and model selection an application to comparison of cortisol profiles is used as an illustration

penalized likelihood approach to estimate a smooth mean curve on longitudinal data this paper aims to propose a penalized likelihood approach to estimate a smooth mean curve for the evolution with time of a gaussian variable taking into account the correlation structure of longitudinal data the model is an extension of the mixed effects linear model including an unspecified function of time f t the estimator f t is defined as the solution of the maximization of the penalized likelihood and is approximated on a basis of cubic m spline with a reduced number of knots we present modifications of four criteria cross validation generalized cross validation t of rice akaike s criterion to estimate the smoothing parameter when data are correlated these four criteria gave very similar results in the simulation study the simulation study showed also the superiority of the bayesian confidence bands of the mean curve over the frequentist ones we develop empirical bayes estimates of subject specific deviations this approach was applied to study the progression of cd4 lymphocyte counts in a cohort of hiv patients treated with protease inhibitors copyright 2002 john wiley sons ltd

efficient generalized cross validation for state space models summary for a signal plus noise model with a state space representation an efficient procedure is given for obtaining the trace of the influence matrix where the influence matrix expresses the estimated signal vector as a linear combination of the observed data this allows an 0 n evaluation of the generalized cross validation criterion function our approach is very efficient requiring the addition of only one equation to the ordinary kalman filter and extends to models with linear regressors in the observation equation important applications are to spline smoothing in regression and to variance components models for seasonal time series

semiparametric bayesian inference in smooth coefficient models abstract we describe procedures for bayesian estimation and testing in cross sectional panel data and nonlinear smooth coefficient models the smooth coefficient model is a generalization of the partially linear or additive model wherein coefficients on linear explanatory variables are treated as unknown functions of an observable covariate in the approach we describe points on the regression lines are regarded as unknown parameters and priors are placed on differences between adjacent points to introduce the potential for smoothing the curves the algorithms we describe are quite simple to implement for example estimation testing and smoothing parameter selection can be carried out analytically in the cross sectional smooth coefficient model we apply our methods using data from the national longitudinal survey of youth nlsy using the nlsy data we first explore the relationship between ability and log wages and flexibly model how returns to schooling vary with measured cognitive ability we also examine a model of female labor supply and use this example to illustrate how the described techniques can been applied in nonlinear settings

combining dc algorithms dcas and decomposition techniques for the training of nonpositive semidefinite kernels today decomposition methods are one of the most popular methods for training support vector machines svms with the use of kernels that do not satisfy mercer s condition new techniques must be designed to handle nonpositive semidefinite kernels resulting to this choice in this work we incorporate difference of convex dc functions optimization techniques into decomposition methods to tackle this difficulty the new approach needs no problem modification and we show that the only use of a truncated dc algorithms dcas in the decomposition scheme produces a sufficient decrease of the objective function at each iteration thanks to this property an asymptotic convergence proof of the new algorithm is produced without any blockwise convexity assumption on the objective function we also investigate a working set selection rule using second order information for sequential minimal optimization smo type decomposition in the spirit of dc optimization numerical results show the robustness and the efficiency of the new methods compared with state of the art software

the molecule problem exploiting structure in global optimization the molecule problem is that of determining the relative locations of a set of objects in euclidean space relying only upon a sparse set of pairwise distance measurements this np hard problem has applications in the determination of molecular conformation the molecule problem can be naturally expressed as a continuous global optimization problem but it also has a rich combinatorial structure this paper investigates how that structure can be exploited to simplify the optimization problem in particular we present a novel divide and conquer algorithm in which a large global optimization problem is replaced by a sequence of smaller ones since the cost of the optimization can grow exponentially with problem size this approach holds the promise of a substantial improvement in performance our algorithmic development relies upon some recently published results in graph theory we describe an implementation of this algorithm and report some results of its performance on a sample molecule

nonparametric regression analysis of growth curves in recent years nonparametric curve estimates have been extensively explored in theoretical work there has however been a certain lack of convincing applications in particular involving comparisons with parametric techniques the present investigation deals with the analysis of human height growth where longitudinal measurements were collected for a sample of boys and a sample of girls evidence is presented that kernel estimates of acceleration and velocity of height and of height itself might offer advantages over a parametric fitting via functional models recently introduced for the specific problem considered both approaches are biased but the parametric one shows qualitative and quantitative distortion which both are not easily predictable data analytic problems involved with kernel estimation concern the choice of kernels the choice of the smoothing parameter and also whether the smoothing parameter should be chosen uniformly for all subjects or individually

diagnosing shocks in time series abstract efficient means of modeling aberrant behavior in times series are developed our methods are based on state space forms and allow test statistics for various interventions to be computed from a single run of the kalman filter smoother the approach encompasses existing detection methodologies departures commonly observed in practice such as outlying values level shifts and switches are readily dealt with new diagnostic statistics are proposed implications for structural models autoregressive integrated moving average models and models with explanatory variables are given

longitudinal data with serial correlation a state space approach a general linear mixed model first order autoregressive errors state space representations the laird ware model in state space form autoregressive moving average errors nonlinear models multivariate models appendix fortran subroutines

trend estimation and de trending via rational square wave filters this paper gives an account of some techniques of linear filtering which can be used for extracting trends from economic time series of limited duration and for generating de trended series a family of rational square wave filters is described which enable designated frequency ranges to be selected or rejected their use is advocated in preference to other filters which are commonly used in quantitative economic analysis

a fast algorithm for signal extraction influence and cross validation in state space models summary a fast algorithm is developed for computing the conditional mean and variance of the signal given the observations in a signal plus noise model the resulting recursions can be applied immediately to provide new and efficient formulae for smoothing part or all of the state vector the ideas of studentized residuals and leverage from regression analysis are generalized to state space models and the new algorithm is used to compute the various measures the results are also applied to obtain a new efficient algorithm for polynomial spline smoothing suppose observations are generated by a gaussian signal plus noise process with the signal described by a state space model such a model for the signal occurs often in practice for example when the signal is the output of a stochastic difference or differential equation this paper presents a new algorithm for signal extraction that is the computa tion of the conditional mean and variance of the signal given the observations the usual method for signal extraction is to use the kalman filter see for example anderson moore 1979 p 105 followed by a smoothing step using an algorithm such as the fixed interval smoothing algorithm anderson moore 1979 p 187 our approach also uses the kalman filter but for the smoothing step introduces a new set of recursions which are more efficient than those for the fixed interval smoothing algorithm as for the fixed interval smoothing algorithm recursions for estimating the signal and its variance are carried out separately so that considerable additional savings can be made if the signal estimate alone is required the development uses ideas introduced by ansley kohn 1987a a remarkable property of the new recursions is that they can be applied immediately

nonparametric spline regression with prior information summary by using prior information about the regression curve we propose new nonparametric regression estimates we incorporate two types of information first we suppose that the regression curve is similar in shape to a family of parametric curves characterized as the solution to a linear differential equation the regression curve is estimated by penalized least squares with the differential operator defining the smoothness penalty we discuss in particular growth and decay curves and take a time transformation to obtain a tractable solution the second type of prior information is linear equality constraints we estimate unknown parameters by generalized cross validation or maximum likelihood and obtain efficient 0 n algorithms to compute the estimate of the regression curve and the cross validation and maximum likelihood criterion functions

mixed effects smoothing spline analysis of variance we propose a general family of nonparametric mixed effects models smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function the random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters these parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method we derive a connection between a nonparametric mixed effects model and a linear mixed effects model this connection suggests a way of fitting a nonparametric mixed effects model by using existing programs the classical two way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis of variance decompositions to build nonparametric mixed effects models similarly to the classical analysis of variance components of these nonparametric mixed effects models can be interpreted as main effects and interactions the penalized likelihood estimates of the fixed effects in a two way mixed model are extensions of james stein shrinkage estimates to correlated observations in an example three nested nonparametric mixed effects models are fitted to a longitudinal data set

estimation prediction and interpolation for arima models with missing data abstract we show how to define and then compute efficiently the marginal likelihood of an arima model with missing observations the computation is carried out by using the univariate version of the modified kalman filter introduced by ansley and kohn 1985a which allows a partially diffuse initial state vector we also show how to predict and interpolate missing observations and obtain the mean squared error of the estimate

estimating integrated higher order continuous time autoregressions with an application to money income causality an algorithm is presented for computing the gaussian maximum likelihood estimator of the parameters of a multivariate continuous time autoregressive process with multiple roots that equal zero some of the variables might be observed at a point in time stocks and some might be observed as an integral over a unit interval flows this algorithm based on the kalman bucy filter is used to investigate the possibility that previous findings that money granger causes industrial production spuriously arose because time averaged variables were analyzed using discrete time methods

signal extraction for finite nonstationary time series summary for a signal plus noise model where both signal and noise are generated by nonstation ary arima autoregressive integrated moving average models we use the transformation approach of ansley kohn 1985 to construct an estimate of the signal and obtain its mean squared error given a finite number of observations a method for efficient computa tion of the signal estimate and its mean squared error is also presented consider observations z t generated by the signal plus noise nmodel z t s t n t 1s1 where both signal s t and noise n t follow nonstationary arima processes we give a solution to the finite sample signal extraction problem by defining an estimate of the signal s t given observations z 1 z n and finding its mean squared error the major difficulty in handling the model with nonstationary components is that the distribution of the observations is not defined unless assumptions are made about the initial values of the series moreover it is usually very difficult to formulate reasonable assumptions about the initial values because they affect the correlation structure of the whole series see for example the discussion of bell 1984 p 652 kohn ansley 1986 use a transformation approach to overcome the initial value problem in defining the likelihood and obtaining predittors and interpolators for an arima model which may have missing values this approach does not require any assumptions about the initial values and generalizes the usual differencing approach of box jenkins 1976 for the case where there are no missing observations in the present paper we apply the transformation approach of kohn ansley 1986 and ansley kohn 1985 to solve the signal extraction problem we define a linear estimator a t of s t so that s t sa t is invariant to the initial values and furthermore is unbiased in the sense that e s t s t 0 moreover it has minimum mean squared error amongst all such estimators our approach allows missing observations and extends to signal extraction for several additive signals when each is an arima process we also show how to compute the signal estimate and its mean squared error efficiently by writing the model in state space form placing a diffuse prior on the initial values of the process and then using the modified kalman filter and the modified smoothing algorithm of ansley kohn 1985 and kohn ansley 1986 gersch kitagawa 1983 kitagawa gersch 1984 and harvey todd 1983 use

bayesian confidence intervals for the cross validated smoothing spline summary we consider the model y t g ti ei i 17 2 n where g t t 0 1 is a smooth function and the ei are independent n 0 a2 errors with g2 unknown the cross validated smoothing spline can be used to estimate g non parametrically from

euclidean distance geometry and applications euclidean distance geometry is the study of euclidean geometry based on the concept of distance this is useful in several applications where the input data consist of an incomplete set of distances and the output is a set of points in euclidean space realizing those given distances we survey the theory of euclidean distance geometry and its most important applications with special emphasis on molecular conformation problems

the performance of cross validation and maximum likelihood estimators of spline smoothing parameters abstract an important aspect of nonparametric regression by spline smoothing is the estimation of the smoothing parameter in this article we report on an extensive simulation study that investigates the finite sample performance of generalized cross validation cross validation and marginal likelihood estimators of the smoothing parameter in splines of orders 2 and 3 the performance criterion for both the estimate of the function and its first derivative is measured by the square root of integrated squared error marginal likelihood using splines of degree 5 emerges as an attractive alternative to the other estimators in that it usually outperforms them and is also faster to compute

a continuous dc programming approach to the strategic supply chain design problem from qualified partner set we present a new continuous approach based on the dc difference of convex functions programming and dc algorithms dca to the problem of supply chain design at the strategic level when production of a new market opportunity has to be launched among a set of qualified partners a well known formulation of this problem is the mixed integer linear program in this paper we reformulate this problem as a dc program by using an exact penalty technique the proposed algorithm is a combination of dca and branch and bound scheme it works in a continuous domain but provides mixed integer solutions numerical simulations on many empirical data sets show the efficiency of our approach with respect to the standard branch and bound algorithm

unbalanced repeated measures models with structured covariance matrices the question of how to analyze unbalanced or incomplete repeated measures data is a common problem facing analysts we address this problem through maximum likelihood analysis using a general linear model for expected responses and arbitrary structural models for the within subject covariances models that can be fit include standard univariate and multivariate models with incomplete data random effects models and models with time series and factor analytic error structures we describe newton raphson and fisher scoring algorithms for computing maximum likelihood estimates and generalized em algorithms for computing restricted and unrestricted maximum likelihood estimates an example fitting several models to a set of growth data is included

regularization networks fast weight calculation via kalman filtering regularization networks are nonparametric estimators obtained from the application of tychonov regularization or bayes estimation to the hypersurface reconstruction problem their main drawback back is that the computation of the weights scales as o n sup 3 where n is the number of data in this paper we show that for a class of monodimensional problems the complexity can be reduced to o n by a suitable algorithm based on spectral factorization and kalman filtering moreover the procedure applies also to smoothing splines

maximum likelihood estimation of regression models with stochastic trend components abstract this article is concerned with the estimation of a regression model with a stochastic trend component it is shown that the probability of estimating the trend to be deterministic is very sensitive to the type of likelihood function used as the basis of inference

a comparison of gcv and gml for choosing the smoothing parameter in the generalized spline smoothing problem the partially improper prior behind the smoothing spline model is used to obtain a generalization of the maximum likelihood gml estimate for the smoothing parameter then this estimate is compared with the generalized cross validation gcv estimate both analytically and by monte carlo methods the comparison is based on a predictive mean square error criteria it is shown that if the true unknown function being estimated is smooth in a sense to be defined then the gml estimate undersmooths relative to the gcv estimate and the predictive mean square error using the gml estimate goes to zero at a slower rate than the mean square error using the gcv estimate if the true function is rough then the gcv and gml estimates have asymptotically similar behavior a monte carlo experiment was designed to see if the asymptotic results in the smooth case were evident in small sample sizes mixed results were obtained for n 32 gcv was somewhat better than gml for n 64 and gcv was decidedly superior for n 128 in the n 32 case gcv was better for smaller sigma 2 and the comparison close for larger sigma 2 the theoretical results are shown to extend to the generalized spline smoothing model which includes the estimate of functions given noisy values of various integrals of them

exact initial kalman filtering and smoothing for nonstationary time series models abstract this article presents a new exact solution for the initialization of the kalman filter for state space models with diffuse initial conditions for example the regression model with stochastic trend seasonal and other nonstationary autoregressive integrated moving average components requires a partially diffuse initial state vector the proposed analytical solution is easy to implement and computationally efficient the exact solution for smoothing is also given missing observations are handled in a straightforward manner all proofs rely on elementary results

benchmarking small area estimators this paper considers benchmarking issues in the context of small area estimation we find optimal estimators within the class of benchmarked linear estimators under linear constraints this extends existing results for external and internal benchmarking and also links the two necessary and sufficient conditions for self benchmarking are found for an augmented model most results of this paper are found using ideas of orthogonal projection copyright 2013 oxford university press

estimation filtering and smoothing in state space models with incompletely specified initial conditions on definit la vraisemblance pour un modele d espace d etats avec des conditions initiales incompletement specifiees par transformation des donnees pour eliminer la dependance sur des conditions non specifiees on etend cette approche pour obtenir des estimations sur les vecteurs d etat les predicteurs et les interpolateurs pour des observations manquantes

fully bayesian spline smoothing and intrinsic autoregressive priors summary there is a well known bayesian interpretation for function estimation by spline smoothing using a limit of proper normal priors the limiting prior and the conditional and intrinsic autoregressive priors popular for spatial modelling have a common form which we call partially informative normal we derive necessary and sufficient conditions for the propriety of the posterior for this class of partially informative normal priors with noninformative priors on the variance components a condition crucial for successful implementation of the gibbs sampler the results apply for fully bayesian smoothing

estimation and prediction for a class of dynamic nonlinear statistical models abstract a class of nonlinear state space models characterized by a single source of randomness is introduced a special case the model underpinning the multiplicative holt winters method of forecasting is identified maximum likelihood estimation based on exponential smoothing instead of a kalman filter and with the potential to be applied in contexts involving non gaussian disturbances is considered a method for computing prediction intervals is proposed and evaluated on both simulated and real data

nonparametric spline regression with autoregressive moving average errors summary we estimate by spline nonparametric regression an unknown function observed with autocorrelated errors when the errors are modelled by an autoregressive moving average model unknown parameters are estimated by either maximum likelihood cross validation or generalized cross validation by expressing the problem in state space form we obtain 0 n algorithms to estimate the function and its derivatives and evaluate the marginal likelihood and cross validation functions the finite sample properties of the function estimates are evaluated by an extensive simulation study and examples are given

autoregressive growth curves and kalman filtering the first part of this paper describes how a kalman filter can be used to construct maximum likelihood ml estimates of autoregressive ar and polynomial parameters in polynomial growth curves with ar 1 errors and irregularly spaced data the second part introduces a disturbed highest derivative polynomial dhdp as a model for growth curves this model does not depend on regression coefficients variances of the highest derivative disturbance and the observation error are estimated by ml using a kalman filter the estimated dhdp growth curve is obtained by optimally smoothing the output of the filter equally spaced data is not required the dhdp model and analysis are developed for an individual and extended to a population growth curve using data from many individuals with covariates

fast likelihood evaluation and prediction for nonstationary state space models summary a recursive procedure for initializing the kalman filter is displayed the recursion is for nonstationary state space models the procedure imposes small computational and programming burden over and above the kalman filter the procedure is superior to other suggested approaches in both computational speed and general applicability general properties of the method are investigated details of the initialization for the arima p d q and basic structural models are considered

rank two beamforming and power allocation in multicasting relay networks in this paper we propose a novel single group multicasting relay beamforming scheme we assume a source that transmits common messages via multiple amplify and forward relays to multiple destinations to increase the number of degrees of freedom in the beamforming design the relays process two received signals jointly and transmit the alamouti space time block code over two different beams furthermore in contrast to the existing relay multicasting scheme of the literature we take into account the direct links from the source to the destinations we aim to maximize the lowest received quality of service by choosing the proper relay weights and the ideal distribution of the power resources in the network to solve the corresponding optimization problem we propose an iterative algorithm which solves sequences of convex approximations of the original non convex optimization problem simulation results demonstrate significant performance improvements of the proposed methods as compared with the existing relay multicasting scheme of the literature and an algorithm based on the popular semidefinite relaxation technique

exact likelihood of vector autoregressive moving average process with missing or aggregated data summary this note points out that by using the kalman filter with nonconstant coefficients we can compute the exact likelihood of an autoregressive moving average process observed with noise when some of our observations are either missing or aggregated

multimodal logic programming we give a framework for developing the least model semantics fixpoint semantics and sld resolution calculi for logic programs in multimodal logics whose frame restrictions consist of the conditions of seriality i e x y ri x y and some classical first order horn clauses our approach is direct and no special restriction on occurrences of i and i is required we apply our framework for a large class of basic serial multimodal logics which are parametefized by an arbitrary combination of generalized versions of axioms t b 4 5 in the form e g 4 i j k and i i j another part of the work is devoted to programming in multimodal logics intended for reasoning about multidegree belief for use in distributed systems of belief or for reasoning about epistemic states of agents in multiagent systems for that we also use the framework and although these latter logics belong to the mentioned class of basic serial multimodal logics the special sld resolution calculi proposed for them are more efficient

fast evaluation of the distribution of the durbin watson and other invariant test statistics in time series regression abstract a method is given for evaluating p values in o n operations for a general class of invariant test statistics that can be expressed as the ratio of quadratic forms in time series regression residuals the best known of these is the durbin watson statistic although several others have been discussed in the literature the method is numerically exact in the sense that the user specifies the error tolerance at the outset as with existing exact methods the problem is reexpressed in terms of the distribution function of a single quadratic form in independent normals which is evaluated by numerically inverting its characteristic function existing methods however calculate the characteristic function by reducing the matrix defining the quadratic form to either eigenvalue or tridiagonal form each of which requires o n 3 operations for sample size n whereas the new method uses a modification of the kalman filter to do it in o n operations moreover the new method has minimal storage requiremen

kalman filtering methods for computing information matrices for time invariant periodic and generally time varying varma models and samples abstract under general conditions the inverse sample information matrix can be used to establish a cramer rao lower bound of the covariance matrix of parameter estimates of a model and the inverse asymptotic information matrix is the asymptotic covariance matrix of the parameter estimates the paper does two things first it derives a recursive kalman filtering method for computing exact sample and asymptotic information matrices for time invariant periodic or generally time varying gaussian vector autoregressive moving average varma models and samples second it specializes the recursive method to a nonrecursive method for computing exact asymptotic information matrices for time invariant or periodic varma models and samples

bayesian confidence intervals for smoothing splines abstract the frequency properties of wahba s bayesian confidence intervals for smoothing splines are investigated by a large sample approximation and by a simulation study when the coverage probabilities for these pointwise confidence intervals are averaged across the observation points the average coverage probability acp should be close to the nominal level from a frequency point of view this agreement occurs because the average posterior variance for the spline is similar to a consistent estimate of the average squared error and because the average squared bias is a modest fraction of the total average squared error these properties are independent of the bayesian assumptions used to derive this confidence procedure and they explain why the acp is accurate for functions that are much smoother than the sample paths prescribed by the prior this analysis accounts for the choice of the smoothing parameter bandwidth using cross validation in the case of natural splines an adaptive method for avo

markov chain monte carlo in conditionally gaussian state space models summary a bayesian analysis is given for a state space model with errors that are finite mixtures of normals and with coefficients that can assume a finite number of different values a sequence of indicator variables determines which components the errors belong to and the values of the coefficients the computation is carried out using markov chain monte carlo with the indicator variables generated without conditioning on the states previous approaches use the gibbs sampler to generate the indicator variables conditional on the states in many problems however there is a strong dependence between the indicator variables and the states causing the gibbs sampler to converge unacceptably slowly or even not to converge at all the new sampler is implemented in o n operations where n is the sample size permitting an exact bayesian analysis of problems that previously had no computationally tractable solution we show empirically that the new sampler can be much more efficient than previous approaches and illustrate its applicability to robust nonparametric regression with discontinuities and to a time series change point problem

a bayesian approach to state space multivariate time series modeling abstract time series modeling is mainly a model specification exercise recognizing this fact a new bayesian procedure is developed that takes full advantage of the subjective probability foundations of the bayesian approach allowing logical consistency in the face of an unknown model the computational demands of this new procedure have been greatly reduced relative to earlier techniques through the application of linear systems theory linear systems theory substitutes a formal model approximation based on the rank of an autocovariance matrix of the time series for the more arbitrary model selection processes of some other procedures this formalization ensures that the most important elements of the series dynamics are included in the model model selection procedures and an approach to the construction of optimal composite forecasts are derived using bayesian methodologies two empirical applications demonstrate the tractability and the accuracy of the new procedure

random effects models for longitudinal data models for the analysis of longitudinal data must recognize the relationship between serial observations on the same unit multivariate models with general covariance structure are often difficult to apply to highly unbalanced data whereas two stage random effects models can be used easily in two stage models the probability distributions for the response vectors of different individuals belong to a single family but some random effects parameters vary across individuals with a distribution specified at the second stage a general family of models is discussed which includes both growth models and repeated measures models as special cases a unified approach to fitting these models based on a combination of empirical bayes and maximum likelihood estimation of model parameters and using the em algorithm is discussed two examples are taken from a current epidemiological study of the health effects of air pollution

fixed interval estimation in state space models when some of the data are missing or aggregated summary ansley kohn s 1982 geometrical derivation of the kalman filter fixed interval smoothing algorithm is extended to the case where some of the observations are missing or aggregated and the state covariance matrix may be singular estimates of missing observations are also obtained the theory is then applied to the interpolation of missing data in multivariate autoregressive moving average models

optimization based dc programming and dca for hierarchical clustering one of the most promising approaches for clustering is based on methods of mathematical programming in this paper we propose new optimization methods based on dc difference of convex functions programming for hierarchical clustering a bilevel hierarchical clustering model is considered with different optimization formulations they are all nonconvex nonsmooth optimization problems for which we investigate attractive dc optimization algorithms called dca numerical results on some artificial and real world databases are reported the results demonstrate that the proposed algorithms are more efficient than related existing methods

locating objects in the plane using global optimization techniques we address the problem of locating objects in the plane such as segments arcs of circumferences arbitrary convex sets their complements or their boundaries given a set of points we seek the rotation and translation for such an object optimizing a very general performance measure which includes as a particular case the classical objectives in semi obnoxious facility location in general the above mentioned model yields a global optimization problem whose resolution is dealt with using difference of convex dc techniques such as outer approximation or branch and bound

the signal extraction approach to nonlinear regression and spline smoothing abstract this article shows how to fit a smooth curve polynomial spline to pairs of data values yi xi prior specification of a parametric functional form for the curve is not required the resulting curve can be used to describe the pattern of the data and to predict unknown values of y given x both point and interval estimates are produced the method is easy to use and the computational requirements are modest even for large sample sizes our method is based on maximum likelihood estimation of a signal in noise model of the data we use the kalman filter to evaluate the likelihood function and achieve significant computational advantages over previous approaches to this problem

cuts matrix completions and graph rigidity this paper brings together several topics arising in distinct areas polyhedral combinatorics in particular cut and metric polyhedra matrix theory and semidefinite programming in particular completion problems for positive semidefinite matrices and euclidean distance matrices distance geometry and structural topology in particular graph realization and rigidity problems cuts and metrics provide the unifying theme indeed cuts can be encoded as positive semidefinite matrices this fact underlies the approximative algorithm for max cut of goemans and williamson and both positive semidefinite and euclidean distance matrices yield points of the cut polytope or cone after applying the functions 1 arccos or when fixing the dimension in the euclidean distance matrix completion problem we find the graph realization problem and the related question of unicity of realization which leads to the question of graph rigidity our main objective here is to present in a unified setting a number of results and questions concerning matrix completion graph realization and rigidity problems these problems contain indeed very interesting questions relevant to mathematical programming and we believe that research in this area could yield to cross fertilization between the various fields involved

smoothing spline models for the analysis of nested and crossed samples of curves abstract we introduce a class of models for an additive decomposition of groups of curves stratified by crossed and nested factors generalizing smoothing splines to such samples by associating them with a corresponding mixed effects model the models are also useful for imputation of missing data and exploratory analysis of variance we prove that the best linear unbiased predictors blups from the extended mixed effects model correspond to solutions of a generalized penalized regression where smoothing parameters are directly related to variance components and we show that these solutions are natural cubic splines the model parameters are estimated using a highly efficient implementation of the em algorithm for restricted maximum likelihood reml estimation based on a preliminary eigenvector decomposition variability of computed estimates can be assessed with asymptotic techniques or with a novel hierarchical bootstrap resampling scheme for nested mixed effects models our methods are applied to me

a signal extraction approach to the estimation of treatment and control curves abstract suppose that the response of a group of subjects which we call the control group is monitored over time at a given time after the start of the experiment the experimenter intervenes and applies a treatment to a subset of the group we propose a signal extraction approach to the simultaneous estimation of the treatment and control mean response curves and their rates of change imposing a smooth transition constraint between the two curves at the time of intervention by expressing the model in state space form we show how to compute the likelihood and the cross validation function efficiently in o n operations together with the response curve estimates and their standard errors we also show that the curve estimates are solutions to a penalized least squares problem a small simulation study to assess the performance of the generalized cross validation and marginal likelihood estimates of the smoothing parameter is carried out and the methodology in the article is applied to an experiment

forecasting time series with complex seasonal patterns using exponential smoothing an innovations state space modeling framework is introduced for forecasting complex seasonal time series such as those with multiple seasonal periods high frequency seasonality non integer seasonality and dual calendar effects the new framework incorporates box cox transformations fourier representations with time varying coefficients and arma error correction likelihood evaluation and analytical expressions for point forecasts and interval predictions under the assumption of gaussian errors are derived leading to a simple comprehensive approach to forecasting complex seasonal time series a key feature of the framework is that it relies on a new method that greatly reduces the computational burden in the maximum likelihood estimation the modeling framework is useful for a broad range of applications its versatility being illustrated in three empirical studies in addition the proposed trigonometric formulation is presented as a means of decomposing complex seasonal time series and it is show

marginal likelihood and unit roots we develop new tests for the hypothesis of unit roots that are based on the marginal likelihood of the general linear model the marginal likelihood allows the incorporation of invariance arguments in the likelihood function it turns out that marginal likelihood tests for unit roots appear to be more powerful than other unit root tests for some basic models power functions almost coincide with the power envelopes even in small samples general correlation structures can be incorporated either by standard likelihood procedures or by adjustments of the test statistics on the basis of asymptotic distributions

longitudinal data analysis for linear gaussian models with random disturbed highest derivative polynomial subject effects for linear regression analysis of longitudinal data with gaussian response i propose a new model to generalize the traditional class of random effects models in which the random effects are deterministic polynomials with coefficients randomly distributed over subjects with mean zero the generalization is accomplished by adding zero mean gaussian disturbances to the highest derivative of each random coefficient subject polynomial independently at each observation time the resulting random effects which have mean zero at each observation time are called disturbed highest derivative polynomials dhdps the disturbances induce serial correlation and also allow the subject specific dhdp time trends to be non linear i do not estimate the subject specific dhdp time trends analysis is based on the marginal model that is the fixed effects or population model obtained by integrating the random polynomial coefficients and all disturbances out of the joint distribution of themselves and the response vector this allows a population averaged interpretation one can select the dhdp order by an information criterion when the population time trend is not correctly modelled the optimal dhdp order will be larger than when it is correctly modelled one can make the covariance matrix of the regression coefficients robust to errors in modelling the within subject dependence i describe the relationship of a dhdp to a smoothing polynomial spline and show how to replace the dhdp model with a smoothing polynomial spline model for the within subject dependence in the marginal model

a signal extraction approach to modeling hormone time series with pulses and a changing baseline abstract hormones serve as regulating signals for many biological processes in recent years it was determined that many hormones are secreted in a pulsatile manner and that the pulsatile secretion pattern in addition to the absolute concentration level is important in regulating biological processes consequently it is necessary to characterize the latent secretion patterns from measurements of concentration levels the characterization is complicated by the presence of a biological circadian rhythm when hormone concentrations are plotted over time the resultant time series usually exhibits occasional short rises superimposed on a slowly changing baseline this is a result of a mixture of pulsatile secretions and a circadian rhythm in this article we present a signal extraction approach to model simultaneously a slowly changing component and a pulsatile component of a time series a smoothing spline is used to model the baseline and a multiprocess dynamic linear model is used to model the pulsati