estimation of gradients and coordinate covariation in classification we introduce an algorithm that simultaneously estimates a classification function as well as its gradient in the supervised learning framework the motivation for the algorithm is to find salient variables and estimate how they covary an efficient implementation with respect to both memory and time is given the utility of the algorithm is illustrated on simulated data as well as a gene expression data set an error analysis is given for the convergence of the estimate of the classification function and its gradient to the true classification function and true gradient

learning multiple tasks with kernel methods we study the problem of learning many related tasks simultaneously using kernel methods and regularization the standard single task kernel methods such as support vector machines and regularization networks are extended to the case of multi task learning our analysis shows that the problem of estimating many task functions with regularization can be cast as a single task learning problem if a family of multi task kernel functions we define is used these kernels model relations among the tasks and are derived from a novel form of regularizers specific kernels that can be used for multi task learning are provided and experimentally tested on two real data sets in agreement with past empirical work on multi task learning the experiments show that learning multiple related tasks simultaneously using the proposed approach can significantly outperform standard single task learning particularly when there are many related tasks but few data per task

on consistent nonparametric order determination and chaos summary we give a brief introduction to deterministic chaos and a link between chaotic deterministic models and stochastic time series models we argue that it is often natural to determine the embedding dimension in a noisy environment first in any systematic study of chaos setting the stochastic models within the framework of non linear autoregression we introduce the notion of a generalized partial autocorrelation and an order we approach the estimation of the embedding dimension via order determination of an unknown non linear autoregression by cross validation and give justification by proving its consistency under global boundedness as a by product we provide a theoretical justification of the final prediction error approach of auestad and tj0stheim some illustrations based on the henon map and several real data sets are given the bias of the residual sum of squares as essentially a noise variance estimator is quantified

a bootstrap method for structure detection of narmax models many systems may be described by narmax models using only a few terms however depending on the order of the system the number of candidate terms can become very large selection of a subset of these candidate terms is necessary for an efficient system description this is an unresolved issue in system identification for over parameterized models therefore in this paper we develop a bootstrap structure detection bsd algorithm as a means of determining the structure of highly over parameterized models the performance of this bsd technique was evaluated by using it to estimate the structure of a 1 simple narmax model 2 moderately over parameterized narmax model and 3 highly over parameterized narmax model the results demonstrate that the bsd algorithm is a robust method for detecting the structure of narmax models this method provides accurate estimates of parameter statistics without relying on assumptions made by traditional procedures and yields a parsimonious description of the system

on the non negative garrotte estimator summary we study the non negative garrotte estimator from three different aspects consistency computation and flexibility we argue that the non negative garrotte is a general procedure that can be used in combination with estimators other than the original least squares estimator as in its original form in particular we consider using the lasso the elastic net and ridge regression along with ordinary least squares as the initial estimate in the non negative garrotte we prove that the non negative garrotte has the nice property that with probability tending to 1 the solution path contains an estimate that correctly identifies the set of important variables and is consistent for the coefficients of the important variables whereas such a property may not be valid for the initial estimators in general we show that the non negative garrotte can turn a consistent estimate into an estimate that is not only consistent in terms of estimation but also in terms of variable selection we also show that the non negative garrotte has a piecewise linear solution path using this fact we propose an efficient algorithm for computing the whole solution path for the non negative garrotte simulations and a real example demonstrate that the non negative garrotte is very effective in improving on the initial estimator in terms of variable selection and estimation accuracy

learning coordinate covariances via gradients we introduce an algorithm that learns gradients from samples in the supervised learning framework an error analysis is given for the convergence of the gradient estimated by the algorithm to the true gradient the utility of the algorithm for the problem of variable selection as well as determining variable covariance is illustrated on simulated data as well as two gene expression data sets for square loss we provide a very efficient implementation with respect to both memory and time

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

newton kantorovich type convergence theorem for a family of new deformed chebyshev method abstract in this paper a family of new deformed chebyshev method with third order is proposed in banach space which is used to solve the nonlinear operator equations the newton kantorovich convergence theorem for the family of new deformed chebyshev method is established by using majorizing function the error estimate is also obtained finally few examples are provided to show the application of our theorems

a note on the halley method in banach spaces abstract we provide a convergence theorem under the usual ostrowski kantorovich conditions for an equivalent form of the halley method in banach spaces the best possible upper error bound for the method is obtained some applications to the solution of quadratic operator equations are provided

sixth order derivative free family of iterative methods in this study we develop a four parameter family of sixth order convergent iterative methods for solving nonlinear scalar equations methods of the family require evaluation of four functions per iteration these methods are totally free of derivatives convergence analysis shows that the family is sixth order convergent which is also verified through the numerical work though the methods are independent of derivatives computational results demonstrate that family of methods are efficient and demonstrate equal or better performance as compared with other six order methods and the classical newton method

a family of second derivative free fourth order continuation method for solving nonlinear equations in this paper we present a parameter based iterative method free from the second derivative for solving nonlinear equations of type f x 0 ezquerro and hernandez 1999 discussed the convergence analysis of a uniparametric family of an iterative method in banach space based on this idea we propose a uniparametric family of second derivative free iterative method in r we discuss the convergence analysis and observe that it possesses a fourth order convergence for solving nonlinear equations in r several numerical examples are worked out with our proposed methods for parameter 1 2 and the existing fourth order iterative method proposed in king 1973 chun 2007 soleymani et al 2012 and maheshwari 2009 finally from the comparison we observe that our method is more efficient than existing methods finally we compare basins of attraction of our methods with the second derivative free fourth order iterative method proposed in king 1973 chun 2007 soleymani et al 2012 and maheshwari 2009 observe that the proposed scheme is more efficient

variable selection via rival removing irrelevant variables amidst lasso iterations and its application to nuclear material detection in many situations the number of data points is fixed and the asymptotic convergence results of popular model selection tools may not be useful a new algorithm for model selection rival removing irrelevant variables amidst lasso iterations is presented and shown to be particularly effective for a large but fixed number of data points the algorithm is motivated by an application of nuclear material detection where all unknown parameters are to be non negative thus positive lasso and its variants are analyzed then rival is proposed and is shown to have some desirable properties namely the number of data points needed to have convergence is smaller than existing methods

a unifying framework in vector valued reproducing kernel hilbert spaces for manifold regularization and co regularized multi view learning this paper presents a general vector valued reproducing kernel hilbert spaces rkhs framework for the problem of learning an unknown functional dependency between a structured input space and a structured output space our formulation encompasses both vector valued manifold regularization and co regularized multi view learning providing in particular a unifying framework linking these two important learning approaches in the case of the least square loss function we provide a closed form solution which is obtained by solving a system of linear equations in the case of support vector machine svm classification our formulation generalizes in particular both the binary laplacian svm to the multi class multi view settings and the multi class simplex cone svm to the semi supervised multi view settings the solution is obtained by solving a single quadratic optimization problem as in standard svm via the sequential minimal optimization smo approach empirical results obtained on the task of object recognition using several challenging data sets demonstrate the competitiveness of our algorithms compared with other state of the art methods

on modified newton methods with cubic convergence abstract in this paper we present two new modified newton methods for solving a non linear equation permitting f x 0 in some points these new methods are showed to be cubically convergent from the practical application we show that these new methods are vast superior and possible in global convergence

universal multi task kernels in this paper we are concerned with reproducing kernel hilbert spaces hk of functions from an input space into a hilbert space y an environment appropriate for multi task learning the reproducing kernel k associated to hk has its values as operators on y our primary goal here is to derive conditions which ensure that the kernel k is universal this means that on every compact subset of the input space every continuous function with values in y can be uniformly approximated by sections of the kernel we provide various characterizations of universal kernels and highlight them with several concrete examples of some practical importance our analysis uses basic principles of functional analysis and especially the useful notion of vector measures which we describe in sufficient detail to clarify our results

nonparametric identification of nonlinear time series selecting significant lags abstract in this article we suggest a nonparametric procedure for selecting significant lags in the model description of a general nonlinear stationary time series the procedure can be applied to both the conditional mean and the conditional variance and is valid for heteroscedastic series the procedure is illustrated by simulations and sunspot data lynx data and blowfly data are analyzed it is indicated that projectors can be used in conjunction with the procedure for selecting significant lags to check the adequacy of an additive time series model

identification of non linear stochastic systems by state dependent parameter estimation the paper outlines how improved estimates of time variable parameters in models of stochastic dynamic systems can be obtained using recursive filtering and fixed interval smoothing techniques with the associated hyper parameters optimized by maximum likelihood based on prediction error decomposition it then shows how by exploiting special data re ordering and back fitting procedures similar recursive parameter estimation techniques can be utilized to estimate much more rapid state dependent parameter sdp variations in this manner it is possible to identify and estimate a widely applicable class of nonlinear stochastic systems as illustrated by several examples that include simulated and real data from chaotic processes finally the paper points out that such sdp models can form the basis for new methods of signal processing automatic control and state estimation for nonlinear stochastic systems

application of a least absolute shrinkage and selection operator to aeroelastic flight test data identification of non linear systems involves estimating unknown parameters and model regressor selection selection of a subset of candidate terms that best describes the observed output model selection is an important step in black box modelling of any observed process this procedure is concerned with selecting a subset of parameters to give a parsimonious description of the system which may afford greater insight into the functionality of the system or a simpler controller design in this study a least absolute shrinkage and selection operator lasso technique is investigated for computing efficient model descriptions of non linear aeroelastic systems the lasso minimises the residual sum of squares by the addition of an l1 penalty term on the parameter vector of the traditional l2 minimisation problem its use for model selection is a natural extension of this constrained minimisation approach to pseudolinear regression problems which produces some model parameters that are exactly zero and the

better subset regression using the nonnegative garrote a new method called the nonnegative nn garrote is proposed for doing subset regression it both shrinks and zeroes coefficients in tests on real and simulated data it produces lower prediction error than ordinary subset selection it is also compared to ridge regression if the regression equations generated by a procedure do not change drastically with small changes in the data the procedure is called stable subset selection is unstable ridge is very stable and the nn garrote is intermediate simulation results illustrate the effects of instability on prediction error

sparse estimation from noisy observations of an overdetermined linear system this note studies a method for the estimation of a finite number of unknown parameters from linear equations which are perturbed by gaussian noise in the case the unknown parameters have only few nonzero entries the proposed estimator performs more efficiently than a traditional approach the method consists of three steps 1 a classical least squares estimate lse 2 the support is recovered through a linear programming lp optimization problem which can be computed using a soft thresholding step 3 a de biasing step using a lse on the estimated support set the main contribution of this note is a formal derivation of an associated oracle property of the final estimate that is with probability 1 the estimate equals the lse based on the support of the true parameters when the number of observations goes to infinity

a fourth order iterative method for solving nonlinear equations the present paper illustrates an iterative numerical method to solve nonlinear equations of the form f x 0 especially those containing the partial and non partial involvement of transcendental terms comparative analysis shows that the present method is faster than newton raphson method hybrid iteration method new hybrid iteration method and others cost is also found to be minimum than these methods the beauty in our method can be seen because of the optimization in important effecting factors i e lesser number of iteration steps lesser number of functional evaluations and lesser value of absolute error in final as well as in individual step as compared to the other methods this work also demonstrates the higher order convergence of the present method as compared to others without going to the computation of second derivative

sparse estimators and the oracle property or the return of hodges estimator abstract we point out some pitfalls related to the concept of an oracle property as used in fan and li 2001 variable selection via nonconcave penalized likelihood and its oracle properties journal of the american statistical association 96 1348 1360 2002 variable selection for cox s proportional hazards model and frailty model annals of statistics 30 74 99 2004 new estimation and model selection procedures for semiparametric modeling in longitudinal data analysis journal of the american statistical association 99 710 723 which are reminiscent of the well known pitfalls related to hodges estimator the oracle property is often a consequence of sparsity of an estimator we show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function in particular the maximal risk diverges to infinity whenever the loss function is unbounded for ease of presentation the result is set in the framework of a linear regression model but generalizes far beyond that setting in a monte carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation scad estimator introduced in fan and li 2001 variable selection via nonconcave penalized likelihood and its oracle properties journal of the american statistical association 96 1348 1360 we find that this estimator can perform rather poorly in finite samples and that its worst case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity

a two stage algorithm for identification of nonlinear dynamic systems this paper investigates the two stage stepwise identification for a class of nonlinear dynamic systems that can be described by linear in the parameters models and the model has to be built from a very large pool of basis functions or model terms the main objective is to improve the compactness of the model that is obtained by the forward stepwise methods while retaining the computational efficiency the proposed algorithm first generates an initial model using a forward stepwise procedure the significance of each selected term is then reviewed at the second stage and all insignificant ones are replaced resulting in an optimised compact model with significantly improved performance the main contribution of this paper is that these two stages are performed within a well defined regression context leading to significantly reduced computational complexity the efficiency of the algorithm is confirmed by the computational complexity analysis and its effectiveness is demonstrated by the simulation results

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

additivity tests for nonlinear autoregression summary additivity is commonly used in the statistical literature to simplify data analysis especially in analysis of variance and in multivariate smoothing in this paper we propose three procedures for testing additivity in nonlinear time series analysis the first procedure combines some smoothing techniques with analysis of variance the second is a lagrange multiplier test using nonparametric estimation and the third is a permutation test which uses smoothing techniques to obtain the test statistic and its reference distribution we investigate properties of the proposed tests and use simulation to check their performance in finite samples applications of the tests to nonlinear time series analysis are discussed and illustrated by real examples

some variant of newton s method with third order convergence we present a new modification of the newton s method which produces iterative methods with order of convergence three a general error analysis providing the higher order of convergence is given and the best efficiency in term of function evaluations of two of this new methods is provided

identification of nonlinear time series first order characterization and order determination summary we study the possibility of identifying nonlinear time series using nonparametric estimates of the conditional mean and conditional variance it is shown that most nonlinear models satisfy the assumptions needed to apply nonparametric asymptotic theory sampling variations of the conditional quantities are studied by simulation and explained by asymptotic arguments for a number of first order nonlinear autoregressive processes the conditional mean and variance can be used for identification purposes but one must be aware of bias and misspecification effects we also propose a criterion for determining the order of a general nonlinear model the criterion is justified in parts by heuristics but encouraging results are obtained from a limited set of simulation experiments several

third order methods from quadrature formulae for solving systems of nonlinear equations we extend to p dimensional problems a modification of the newton method based on quadrature formulas of order at least one which produces iterative methods with order of convergence three a general error analysis providing the higher order of convergence is given these new methods may be more efficient then other third order methods as they do not require the use of the second order frechet derivative

modified chebyshev halley methods free from second derivative abstract in this paper we present some variants of chebyshev halley methods free from second derivative it is shown that the order of convergence of the proposed methods is three at least there are two functions and one first derivative evaluations per iteration several numerical examples are given to demonstrate the performance of the presented methods by comparing with some other methods

reproducing kernel banach spaces for machine learning we introduce the notion of reproducing kernel banach spaces rkbs and study special semi inner product rkbs by making use of semi inner products and the duality mapping properties of an rkbs and its reproducing kernel are investigated as applications we develop in the framework of rkbs standard learning schemes including minimal norm interpolation regularization network support vector machines and kernel principal component analysis in particular existence uniqueness and representer theorems are established

orthogonal least squares methods and their application to non linear system identification abstract identification algorithms based on the well known linear least squares methods of gaussian elimination cholesky decomposition classical gram schmidt modified gram schmidt householder transformation givens method and singular value decomposition are reviewed the classical gram schmidt modified gram schmidt and householder transformation algorithms are then extended to combine structure determination or which terms to include in the model and parameter estimation in a very simple and efficient manner for a class of multivariate discrete time non linear stochastic systems which are linear in the parameters

input output parametric models for non linear systems part ii stochastic non linear systems recursive input output models for non linear multivariate discrete time systems are derived and sufficient conditions for their existence are defined the paper is divided into two parts the first part introduces and defines concepts such as nerode realization multistructural forms and results from differential geometry which are then used to derive a recursive input output model for multivariable deterministic non linear systems the second part introduces several examples compares the derived model with other representations and extends the results to create prediction error or innovation input output models for non linear stochastic systems these latter models are the generalization of the multivariable arm ax models for linear systems and are referred to as narmax or non linear autoregressive moving average models with exogenous inputs

a uniparametric chebyshev type method free from second derivatives in this paper we present a family of new chebyshev type methods free from second derivatives for solving non linear equations analysis of convergence shows that these new methods are cubically convergent as particular cases we introduce two efficient methods in this family of the new methods the practical utility of which is demonstrated by numerical examples

geometric constructions of iterative functions to solve nonlinear equations in this paper we present the geometrical interpretation of several iterative methods to solve a nonlinear scalar equation in addition we also review the extension to general banach spaces and some computational aspects of these methods

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

fourth order variants of cauchy s method for solving non linear equations abstract in this paper we present some new variants of cauchy s method in which the second derivative is replaced by a finite difference between the first derivatives analysis of convergence shows that the new methods have fourth order convergence per iteration the new methods require one evaluations of the function and two of its first derivative numerical results show that the new methods are efficient

refinement of reproducing kernels we continue our recent study on constructing a refinement kernel for a given kernel so that the reproducing kernel hilbert space associated with the refinement kernel contains that with the original kernel as a subspace to motivate this study we first develop a refinement kernel method for learning which gives an efficient algorithm for updating a learning predictor several characterizations of refinement kernels are then presented it is shown that a nontrivial refinement kernel for a given kernel always exists if the input space has an infinite cardinal number refinement kernels for translation invariant kernels and hilbert schmidt kernels are investigated various concrete examples are provided

discussion the dantzig selector statistical estimation when p is much larger than n 1 introduction this is a fascinating paper on an important topic the choice of predictor variables in large scale linear models a previous paper in these pages attacked the same problem using the lars algorithm efron hastie johnstone and tibshirani 3 actually three algorithms including the lasso as middle case there are tantalizing similarities between the dantzig selector ds and the lars methods but they are not the same and produce somewhat different models we explore this relationship with the lasso and lars here 2 dantzig selector and the lasso the definition of the dantzig selector ds in 1 7 can be re expressed as

operator valued kernels for learning from functional response data in this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions a data is represented by a set of functions and the label is also a function we focus on the use of reproducing kernel hilbert space theory to learn from such functional data basic concepts and properties of kernel based learning are extended to include the estimation of function valued functions in this setting the representer theorem is restated a set of rigorously defined infinite dimensional operator valued kernels that can be valuably applied when the data are functions is described and a learning algorithm for nonlinear functional data analysis is introduced the methodology is illustrated through speech and audio signal processing experiments

modified chebyshev s method free from second derivative for non linear equations in this paper we present some new modifications of chebyshev s method free from second derivative the order of convergence of the new methods is three per iteration the methods require two evaluations of the function and one evaluation of its first derivative the convergence of the new methods for the case of multiple roots is also discussed numerical examples show that the new methods have definite practical utility

third order iterative methods free from second derivative for nonlinear systems in this work we present a family of predictor corrector methods free from second derivative for solving nonlinear systems we prove that the methods of this family are of third order convergence we also perform numerical tests that allow us to compare these methods with newton s method in addition the numerical examples improve theoretical results showing super cubic convergence for some methods of this family

refinement of operator valued reproducing kernels this paper studies the construction of a refinement kernel for a given operator valued reproducing kernel such that the vector valued reproducing kernel hilbert space of the refinement kernel contains that of the given kernel as a subspace the study is motivated from the need of updating the current operator valued reproducing kernel in multi task learning when underfitting or overfitting occurs numerical simulations confirm that the established refinement kernel method is able to meet this need various characterizations are provided based on feature maps and vector valued integral representations of operator valued reproducing kernels concrete examples of refining translation invariant and finite hilbert schmidt operator valued reproducing kernels are provided other examples include refinement of hessian of scalar valued translation invariant kernels and transformation kernels existence and properties of operator valued reproducing kernels preserved during the refinement process are also investigated

a suboptimal bootstrap method for structure detection of non linear output error models with application to human ankle dynamics identification of non linear systems involves estimating unknown parameters and structure detection selection of a subset of candidate terms that best describe the observed output this is a necessary procedure to compute an efficient system description which may afford greater insight into the functionality of the system or a simpler controller design for non linear systems simple output additive noise can generate multiplicative terms between the input output and noise the terms associated with noise need to be modelled to obtain unbiased parameter estimates significantly increasing the number of candidate terms to be estimated and considered in special cases it may be possible to use an output error oe model structure and the instrumental variable iv estimator to obtain unbiased parameters without modelling the noise this significantly reduces the dimensionality of the structure computation problem therefore in this paper we propose a suboptimal bootstrap structure detection sobsd algo

finding the embedding dimension and variable dependencies in time series we present a general method the test which establishes functional dependencies given a sequence of measurements the approach is based on calculating conditional probabilities from vector component distances imposing the requirement of continuity of the underlying function the obtained values of the conditional probabilities carry information on the embedding dimension and variable dependencies the power of the method is illustrated on synthetic time series with different time lag dependencies and noise levels and on the sunspot data the virtue of the method for preprocessing data in the context of feedforward neural networks is demonstrated also its applicability for tracking residual errors in output units is stressed

nonparametric identification of nonlinear time series projections abstract we study the possibility of identifying general linear and nonlinear time series models using nonparametric methods the kernel estimators of the conditional mean and variance are used as a basis and the properties of these quantities as model indicators are briefly discussed some drawbacks are pointed out and motivated by these we introduce projections as tools of identification the projections are especially useful for additive modeling expressions for the asymptotic bias and variance are obtained the projection of the conditional variance is suggested as a tool for identifying heteroscedastic time series the results are illustrated by simulations for both the estimators of the projections and the estimators of the conditional mean and variance

identification of mimo non linear systems using a forward regression orthogonal estimator abstract an orthogonal least squares estimator which was originally derived for single input single output systems is extended to multi input multi output non linear systems the estimator can provide information about the structure or which terms to include in the model and final parameter estimates in a very simple and efficient manner multivariable non linear model validity tests are also discussed and the results of applying the orthogonal estimator to both simulated and real data are included

on learning vector valued functions in this letter we provide a study of learning in a hilbert space of vectorvalued functions we motivate the need for extending learning theory of scalar valued functions by practical considerations and establish some basic results for learning vector valued functions that should prove useful in applications specifically we allow an output space y to be a hilbert space and we consider a reproducing kernel hilbert space of functions whose values lie in y in this setting we derive the form of the minimal norm interpolant to a finite set of data and apply it to study some regularization functionals that are important in learning theory we consider specific examples of such functionals corresponding to multiple output regularization networks and support vector machines for both regression and classification finally we provide classes of operator valued kernels of the dot product and translation invariant type

structure identification of nonlinear dynamic systems a survey on input output approaches abstract in the past several methods have been elaborated for the identification of nonlinear dynamic systems most of the methods assume that the structure of the system is given a priori therefore they are in reality parameter estimation algorithms and structure identification is thus usually performed by repeated parameter estimation however in nonlinear system theory several methods are known to determine the structure of a system in this paper structure identification of block oriented especially cascade models of semi linear dynamic models with signal dependent parameters and of nonlinear dynamic models being linear in the parameters will be considered different structure selection methods are summarized based on step and impulse tests frequency response measurements correlation analysis repeated reproducible tests and normal operating data

orthogonal parameter estimation algorithm for non linear stochastic systems an orthogonal parameter estimation algorithm is derived which allows each parameter in a linear in the parameters non linear difference equation model to be estimated sequentially and quite independently of the other parameters in the model the algorithm can be applied for any persistently exciting input and provides both unbiased estimates in the presence of correlated noise and an indication of which terms to include in the model several simulated examples are included to demonstrate the effectiveness of the algorithm

on halley type iterations with free second derivative in this paper we relax the convergence conditions required in ezquerro and hernandez int j pure appl math 6 1 2003 103 for a multipoint third order iteration of halley type where the conditions provided are the known ones for methods of order three

divergence free kernel methods for approximating the stokes problem we develop a new discretization scheme for the stokes equations using analytically divergence free approximation spaces our scheme is based upon a meshfree method using matrix valued kernels the scheme works in arbitrary space dimension and can produce arbitrarily smooth and hence high order approximations after deriving the scheme we analyze the discretization error in sobolev spaces and give a numerical example

linear model selection by cross validation abstract we consider the problem of selecting a model having the best predictive ability among a class of linear models the popular leave one out cross validation method which is asymptotically equivalent to many other model selection methods such as the akaike information criterion aic the c p and the bootstrap is asymptotically inconsistent in the sense that the probability of selecting the model with the best predictive ability does not converge to 1 as the total number of observations n we show that the inconsistency of the leave one out cross validation can be rectified by using a leave n v out cross validation with n v the number of observations reserved for validation satisfying n v n 1 as n this is a somewhat shocking discovery because nv n 1 is totally opposite to the popular leave one out recipe in cross validation motivations justifications and discussions of some practical aspects of the use of the leave n v out cross validation method are provided and results

discrete reproducing kernel hilbert spaces sampling and distribution of dirac masses we study reproducing kernels and associated reproducing kernel hilbert spaces rkhss h over infinite discrete and countable sets v in this setting we analyze in detail the distributions of the corresponding dirac point masses of v illustrations include certain models from neural networks an extreme learning machine elm is a neural network configuration in which a hidden layer of weights are randomly sampled and where the object is then to compute resulting output for rkhss h of functions defined on a prescribed countable infinite discrete set v we characterize those which contain the dirac masses x for all points x in v further examples and applications where this question plays an important role are i discrete brownian motion hilbert spaces i e discrete versions of the cameron martin hilbert space ii energy hilbert spaces corresponding to graph laplacians where the set v of vertices is then equipped with a resistance metric and finally iii the study of gaussian free fields