iterative scaling and coordinate descent methods for maximum entropy models maximum entropy maxent is useful in natural language processing and many other areas iterative scaling is methods are one of the most popular approaches to solve maxent with many variants of is methods it is difficult to understand them and see the differences in this paper we create a general and unified framework for iterative scaling methods this framework also connects iterative scaling and coordinate descent methods we prove general convergence results for is methods and analyze their computational complexity based on the proposed framework we extend a coordinate descent method for linear svm to maxent results show that it is faster than existing iterative scaling methods

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

on the asymptotic stability ofθ methods for delay differential equations stability regions of methods for the linear delay differential test equations begin gathered y t py t qy t tau t 0 hfill y t varphi t t in tau 0 hfill end gathered where is a positive constant are presented in the case thatp andq are real constant coefficients necessary and sufficient conditions on the stepsize for the stability of a method are obtained furthermore whenp andq are complex coefficients sufficient conditions for the stability of the methods are also given

quasilinear multistep methods and variable step predictor corrector methods for neutral functional differential equations the theory of quasilinear multistep methods and predictor corrector methods for neutral functional differential equations is developed the theorem about the asymptotic behaviour of the global discretization error of quasilinear methods which forms a theoretical basis for step control strategy for predictor corrector methods for neutral equations is given some problems of implementation of these methods are also addressed and the results of numerical experiments are presented

a numerical study of the limited memory bfgs method and the truncated newton method for large scale optimization this paper examines the numerical performances of two methods for large scale optimization a limited memory quasi newton method l bfgs and a discrete truncated newton method tn various ways of classifying test problems are discussed in order to better understand the types of problems that each algorithm solves well the l bfgs and tn methods are also compared with the polak ribiere conjugate gradient method

numerical solution of delay differential equations by uniform corrections to an implicit runge kutta method in this paper we develop a class of numerical methods to approximate the solutions of delay differential equations they are essentially based on a modified version in a predictor corrector mode of the one step collocation method atn gaussian points these methods applied to ode s provide a continuous approximate solution which is accurate of order 2n at the nodes and of ordern 1 uniformly in the whole interval in order to extend the methods to delay differential equations the uniform accuracy is raised to the order 2n by some a posteriori corrections numerical tests and comparisons with other methods are made on real life problems

on the asymptotic stability properties of runge kutta methods for delay differential equations in this paper asymptotic stability properties of runge kutta r k methods for delay differential equations ddes are considered with respect to the following test equation begin eqnarray left begin array ccl y t a y t b y t tau 0 3cm y t g t end array right hskip 0 3cm begin array r t 0 0 3cm tau leq t leq 0 end array end eqnarray where a b in bbb r tau 0 and g t is a continuous real valued function in the last few years stability properties of r k methods applied to ddes have been studied by numerous authors who have considered regions of asymptotic stability for any positive delay and thus independent of the specific value of tau

a fully discrete spectral method for delay differential equations in this paper a new lanczos tau method for solving linear functional differential equations is introduced the scheme has infinite order of accuracy both in time and in the delayed argument the high accuracy in time is obtained without increasing the computational work and memory space which is needed for a one step explicit difference scheme the article demonstrates how to implement the algorithm in a robust and efficient manner and to treat problems with piecewise continuous initial function numerical results illustrating the behavior of the method when faced with difficult problems are presented and the numerical results are compared to those obtained by using two other methods

computer solutions of state dependent delay differential equations abstract an adaptation of the runge kutta verner 5 6 formula pair is used to construct a numerical method for the solution of state dependent delay differential equations with nonvanishing lag a fifth degree divided difference newton backward interpolation polynomial is used to find the location of the derivative jump discontinuities of the solution in order to maintain the sixth order accuracy of the rkv pair the value of the solution at the delay is approximated by a three point hermite polynomial this new method is tested on some real life problems a fortran program called sysdel is available from the authors

stepsize control and continuity consistency for state dependent delay differential equations abstract derivative discontinuities occur frequently in the solutions of delay differential equations even when the functions defining them are all c unless correctly treated such discontinuities can undermine the continuity assumptions made by ode software in this paper we discuss some specific difficulties arising from this aspect in the treatment of state dependent problems a necessary property continuity consistency is introduced along with a class of methods for which it is satisfied

multicategory support vector machines two category support vector machines svm have been very popular in the machine learning community for classification problems solving multicategory problems by a series of binary classifiers is quite common in the svm paradigm however this approach may fail under various circumstances we propose the multicategory support vector machine msvm which extends the binary svm to the multicategory case and has good theoretical properties the proposed method provides a unifying framework when there are either equal or unequal misclassification costs as a tuning criterion for the msvm an approximate leave one out cross validation function called generalized approximate cross validation is derived analogous to the binary case the effectiveness of the msvm is demonstrated through the applications to cancer classification using microarray data and cloud classification with satellite radiance profiles

numerical treatment of stochastic delay differential equations a global error bound abstract motivated by the error analysis of numerical solution for stochastic delay differential equations a new result indicating the dependency of convergence rate of the scheme on two separate quantities p and q where the former is the global order of underlying scheme and the latter is that of interpolation approximating the history of system is presented as an objective instance the stochastic runge kutta srk schemes of global order 0 5 1 and 1 5 in the strong convergence sense are utilized to formulate three one step schemes to approximate the solution of the problem moreover the mean square stability of these schemes will be asymptotically developed finally the theoretical results will be confirmed by some test problems

numerical solution by lmms of stiff delay differential systems modelling an immune response we consider the application of linear multistep methods lmms for the numerical solution of initial value problem for stiff delay differential equations ddes with several constant delays which are used in mathematical modelling of immune response for the approximation of delayed variables the nordsieck s interpolation technique providing an interpolation procedure consistent with the underlying linear multistep formula is used an analysis of the convergence for a variable stepsize and structure of the asymptotic expansion of global error for a fixed stepsize is presented some absolute stability characteristics of the method are examined implementation details of the code difsub dde being a modification of the gear s difsub are given finally an efficiency of the code developed for solution of stiff ddes over a wide range of tolerances is illustrated on biomedical application model

fitting longitudinal reduced rank regression models by alternating least squares an alternating least squares method for iteratively fitting the longitudinal reduced rank regression model is proposed the method uses ordinary least squares and majorization substeps to estimate the unknown parameters in the system and measurement equations of the model in an example with cross sectional data it is shown how the results conform closely to results from eigenanalysis optimal scaling of nominal and ordinal variables is added in a third substep and illustrated with two examples involving cross sectional and longitudinal data

derivation of continuous explicit two step runge kutta methods of order three we describe a construction of continuous extensions to a new representation of two step runge kutta methods for ordinary differential equations this representation makes possible the accurate and reliable estimation of local discretization error facilitates the efficient implementation of these methods in variable stepsize environment and adapts readily to the numerical solution of a class of delay differential equations a number of numerical tests carried out on the obtained methods of order 3 with quadratic interpolants show their efficiency and robust performance which allow them to compete with the state of the art dde23 code from matlab

an interior point method for large scale l 1 regularized logistic regression logistic regression with l1 regularization has been proposed as a promising method for feature selection in classification problems in this paper we describe an efficient interior point method for solving large scale l1 regularized logistic regression problems small problems with up to a thousand or so features and examples can be solved in seconds on a pc medium sized problems with tens of thousands of features and examples can be solved in tens of seconds assuming some sparsity in the data a variation on the basic method that uses a preconditioned conjugate gradient method to compute the search step can solve very large problems with a million features and examples e g the 20 newsgroups data set in a few minutes on a pc using warm start techniques a good approximation of the entire regularization path can be computed much more efficiently than by solving a family of problems independently

training a support vector machine in the primal most literature on support vector machines svms concentrates on the dual optimization problem in this letter we point out that the primal problem can also be solved efficiently for both linear and nonlinear svms and that there is no reason for ignoring this possibility on the contrary from the primal point of view new families of algorithms for large scale svm training can be investigated

qmc methods for the solution of delay differential equations in this paper the quasi monte carlo methods for runge kutta solution techniques of differential equations which were developed by stengle lecot coulibaly and koudiraty are extended to delay differential equations of the form y t f t y t y t t the retarded argument is approximated by interpolation after which the conventional quasi monte carlo runge kutta methods can be applied we give a proof of the convergence of this method and its order in a general form which does not depend on a specific quasi monte carlo runge kutta method finally a numerical investigation shows that similar to ordinary differential equations this quasi randomized method leads to an improvement for heavily oscillating delay differential equations compared even to high order runge kutta schemes

gensvm a generalized multiclass support vector machine traditional extensions of the binary support vector machine svm to multiclass problems are either heuristics or require solving a large dual optimization problem here a generalized multiclass svm is proposed called gensvm in this method classification boundaries for a k class problem are constructed in a k 1 dimensional space using a simplex encoding additionally several different weightings of the misclassification errors are incorporated in the loss function such that it generalizes three existing multiclass svms through a single optimization problem an iterative majorization algorithm is derived that solves the optimization problem without the need of a dual formulation this algorithm has the advantage that it can use warm starts during cross validation and during a grid search which significantly speeds up the training phase rigorous numerical experiments compare linear gensvm with seven existing multiclass svms on both small and large data sets these comparisons show that the proposed method is competitive with existing methods in both predictive accuracy and training time and that it significantly outperforms several existing methods on these criteria

natural continuous extensions of runge kutta methods the present paper develops a theory of natural continuous extensions nces for the discrete approximate solution of an ode given by a runge kutta process these nces are defined in such a way that the continuous solutions furnished by the one step collocation methods are included

continuous parallel block methods and their applications continuous numerical methods have many applications in the numerical solution of discontinuous ordinary differential equations delay differential equations delay differential algebraic equations neutral delay differential equations integro differential equations etc this paper deals with a continuous extension for the discrete approximate solution of ordinary differential equations generated by a class of parallel block methods existence and uniqueness for the continuous extension are discussed convergence and absolute stability of the continuous parallel block methods for ordinary differential equations are studied as applications we adopt the continuous parallel block methods to solve delay differential equations neutral delay differential equations and delay differential algebraic equations and obtain sufficient and necessary conditions for the continuous parallel block methods to be asymptotically stable several numerical experiments are given to illustrate the performance of the continuous parallel block methods

pegasos primal estimated sub gradient solver for svm we describe and analyze a simple and effective stochastic sub gradient descent algorithm for solving the optimization problem cast by support vector machines svm we prove that the number of iterations required to obtain a solution of accuracy epsilon is tilde o 1 epsilon where each iteration operates on a single training example in contrast previous analyses of stochastic gradient descent methods for svms require omega 1 epsilon 2 iterations as in previously devised svm solvers the number of iterations also scales linearly with 1 where is the regularization parameter of svm for a linear kernel the total run time of our method is tilde o d lambda epsilon where d is a bound on the number of non zero features in each example since the run time does not depend directly on the size of the training set the resulting algorithm is especially suited for learning from large datasets our approach also extends to non linear kernels while working solely on the primal objective function though in this case the runtime does depend linearly on the training set size our algorithm is particularly well suited for large text classification problems where we demonstrate an order of magnitude speedup over previous svm learning methods

newton stein method an optimization method for glms via stein s lemma we consider the problem of efficiently computing the maximum likelihood estimator in generalized linear models glms when the number of observations is much larger than the number of coefficients n p 1 in this regime optimization algorithms can immensely benefit from approximate second order information we propose an alternative way of constructing the curvature information by formulating it as an estimation problem and applying a stein type lemma which allows further improvements through subsampling and eigenvalue thresholding our algorithm enjoys fast convergence rates resembling that of second order methods with modest per iteration cost we provide its convergence analysis for the general case where the rows of the design matrix are samples from a sub gaussian distribution we show that the convergence has two phases a quadratic phase followed by a linear phase finally we empirically demonstrate that our algorithm achieves the highest performance compared to various optimization algorithms on several data sets

numerical solution of retarded initial value problems local and global error and stepsize control retarded initial value problems are routinely replaced by an initial value problem of ordinary differential equations along with an appropriate interpolation scheme hence one can control the global error of the modified problem but not directly the actual global error of the original problem in this paper we give an estimate for the actual global error in terms of controllable quantities further we show that the notion of local error as inherited from the theory of ordinary differential equations must be generalized for retarded problems along with the new definition we are led to developing a reliable basis for a step selection scheme

numerical treatment of delay differential equations by hermite interpolation a class of numerical methods for the treatment of delay differential equations is developed these methods are based on the wellknown runge kutta fehlberg methods the retarded argument is approximated by an appropriate multipoint hermite interpolation the inherent jump discontinuities in the various derivatives of the solution are considered automatically problems with piecewise continuous right hand side and initial function are treated too real life problems are used for the numerical test and a comparison with other methods published in literature

d convergence of general linear methods for stiff delay differential equations abstract this paper is concerned with the numerical solution of delay differential equations ddes we focus on the error behaviour of general linear methods for stiff ddes a restricted type of interpolation procedure is considered for general linear methods d convergence properties of general linear methods with this interpolation procedure are investigated some extant results are unified

linear multistep methods for functional differential equations a new way to define linear multistep methods for functional differential equations is presented and some of their properties are analyzed the asymptotic behavior of the global discretization error is investigated finally milne s device is generalized to functional differen tial equations the effect of the nonsmoothness of the exact solution is taken into account 1 introduction consider the functional differential equation fde 1 x t f t xt to t t xto o where x t e ri 0 e c t 0 rn fe c to t x c t 0 rn ri and xt 9 x t 9 for 0 e t 0 t e to t it is well known that the solution x of 1 is usually not smooth that is even if f and 4 are c functions x may have jump discontinuities in its derivatives the occurrence of these jump discontinuities may lead to order breakdown for numerical methods if no special provisions are made it seems that all available techniques require information about the location of the jumps if the delay is not state dependent then these locations may be known a priori if not then they may be calculated numerically see feldstein and neves 6 when the location of a jump discontinuity is known two different techniques are available i make the location of the jump a grid point and restart in practice this

restricted canonical correlations given two random vectors y 1 and y 2 the first canonical correlation between them is defined as sup correlation y 1 y 2 e rp e rq however in many practical situations e g educational testing problems neural networks some natural restrictions on the coefficients and may arise which should be incorporated in this maximization procedure the maximum correlation subject to such constraints is referred to as the restricted canonical correlation this problem is treated here under the nonnegativity restriction on and the analysis is extended to more general form of inequality constraints and also when the restrictions are present only on some of the coefficients restricted versions of some other related measures are also discussed this includes principal component analysis and different modifications of canonical correlations anomalies with higher order correlation are also described some properties of restricted canonical correlation including its bounds are studied

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

solution of time varying delay systems using an adaptive collocation method abstract an adaptive legendre gauss radau collocation method is presented for solving time varying delay systems by dividing the domain of the problem into a uniform mesh the delay system is replaced with a sequence of initial value problems the method developed in this paper adjusts both the mesh spacing and the degree of the polynomial on each mesh interval in order a specified error tolerance is satisfied we investigate some error estimates on the whole domain of interest some numerical examples are given and comparisons with other methods are made to demonstrate the efficiency and accuracy of the proposed method

canonical correlations associated with symmetric reflexive generalized inverses of the dispersion matrix we consider the relations between two sets of canonical correlations one based on a possibly singular dispersion matrix v and the other on a symmetric reflexive generalized inverse of v special attention is paid to the number of unit canonical correlations in each set we establish series of results characterizing the situations where all canonical correlations in one or both sets are less than one it turns out that in all such situations the canonical correlations in the one set are uniformly comparable to those in the other set

reducing multiclass to binary a unifying approach for margin classifiers we present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a margin based binary learning algorithm the proposed framework unifies some of the most popular approaches in which each class is compared against all others or in which all pairs of classes are compared to each other or in which output codes with error correcting properties are used we propose a general method for combining the classifiers generated on the binary problems and we prove a general empirical multiclass loss bound given the empirical loss of the individual binary learning algorithms the scheme and the corresponding bounds apply to many popular classification learning algorithms including support vector machines adaboost regression logistic regression and decision tree algorithms we also give a multiclass generalization error analysis for general output codes with adaboost as the binary learner experimental results with svm and adaboost show that our scheme provides a viable alternative to the most commonly used multiclass algorithms

in defense of one vs all classification we consider the problem of multiclass classification our main thesis is that a simple one vs all scheme is as accurate as any other approach assuming that the underlying binary classifiers are well tuned regularized classifiers such as support vector machines this thesis is interesting in that it disagrees with a large body of recent published work on multiclass classification we support our position by means of a critical review of the existing literature a substantial collection of carefully controlled experimental work and theoretical arguments

variable step variable order 3 stage hermite birkhoff obrechkoff dde solver of order 4 to 14 abstract this article presents a solver for delay differential equations ddes called hbo414dde based on a hybrid variable step variable order 3 stage hermite birkhoff obrechkoff ode solver of order 4 to 14 the current version of our method solves ddes with state dependent non vanishing small vanishing and asymptotically vanishing delays except neutral type and initial value ddes delayed values are computed using hermite interpolation small delays are dealt with by extrapolation and discontinuities are located by a bisection method hbo414dde was tested on several problems and results were compared with those of known solvers like sysdel and the recent matlab dde solver ddesd and statistics show that it gives most of the time a smaller relative error than the other solvers for the same number of function evaluations

approximating the linear quadratic optimal control law for hereditary systems with delays in the control a factorization approach is presented for deriving approximations to the optimal feedback gains for the linear regulator quadratic cost problem associated with time varying functional differential equations with control delays the approach is based on a discretization of the state penalty which leads to a simple structure for the feedback control law general properties of the volterra factors of hilbert schmidt operators are then used to obtain convergence results for the controls trajectories and feedback kernels two algorithms are derived from the basic approximation scheme including a fast algorithm in the time invariant case a numerical example is also considered

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

on the algorithmic implementation of multiclass kernel based vector machines in this paper we describe the algorithmic implementation of multiclass kernel based vector machines our starting point is a generalized notion of the margin to multiclass problems using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks our notion of margin yields a direct method for training multiclass predictors by using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size we describe an efficient fixed point algorithm for solving the reduced optimization problems and prove its convergence we then discuss technical details that yield significant running time improvements for large datasets finally we describe various experiments with our approach comparing it to previously studied kernel based methods our experiments indicate that for multiclass problems we attain state of the art accuracy

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

trust region newton method for logistic regression large scale logistic regression arises in many applications such as document classification and natural language processing in this paper we apply a trust region newton method to maximize the log likelihood of the logistic regression model the proposed method uses only approximate newton steps in the beginning but achieves fast convergence in the end experiments show that it is faster than the commonly used quasi newton approach for logistic regression we also extend the proposed method to large scale l2 loss linear support vector machines svm

stability analysis of linearly implicit one step interpolation methods for stiff retarded differential equations the aim of this paper is to analyze the stability properties of the modified problem and of linearly implicit runge kutta interpolation methods for stiff retarded differential equations with respect to the model equation y t py qy t tau quad t geqq 0 y t varphi t quad t leqq 0 where p and q are co mplex constants and tau 0 is a constant d necessary and sufficient conditions for the stability of the modified problem i e the retarded term y t tau of the model equation is replaced by an interpolation polynomial and of linearly implicit one step interpolation methods are presented these conditions allow us to compute the boundary locus of the stability region it is shown that any modified problem and any linearly implicit runge kutta lagrange method with appropriate stability function are p beta stable and that linearly implicit runge kutta lagrange methods possess better stability properties than linearly implicit runge kutta hermite methods furthermore

2h step spline method for the solution of delay differential equations abstract an approximate spline is constructed for the solution of initial value problems in delay differential equations the existence and uniqueness of the approximate spline solution are investigated samples of our computational experiments are given and compared with other methods

canonical correlation analysis with linear constraints abstract we develop canonical correlation analysis by imposing linear constraints upon parameters corresponding to two sets of variables the results of our method which we call canolc are shown in terms of projection operators both orthogonal and oblique further calc correspondence analysis with linear constraints turns out to be a special case of canolc

spline collocation methods for solving delay differential equations the convergence and stability analysis of seventh c 3 spline collocation methods if applied to delay differential equations ddes will be considered letting the interior collocation points x i 1 c j x i 1 c jh j 1 3 be dependent on three parameters c 1 c 2 c email protected 0 1 it will be shown that these methods for ddes possess convergence rate of order seven if 0 7279115

on the solutions of a system of linear retarded and advanced differential equations by the bessel collocation approximation in this study we present a numerical approximation for the solutions of the system of high order linear retarded and advanced differential equations with variable coefficients under the mixed conditions this method is based on taking the truncated bessel expansion of the functions in the retarded and advanced differential equation system by the aid of the matrix operations and the collocation points the problem is transformed into a matrix equation with the unknown bessel coefficients by solving this matrix equation the unknown coefficients of the approximate solutions are computed in addition examples that illustrate the pertinent features of the method are presented and the results of this investigation are discussed

numerical experience with limited memory quasi newton and truncated newton methods computational experience with several limited memory quasi newton and truncated newton methods for unconstrained nonlinear optimization is described comparative tests were conducted on a well known test library j j more b s gaxbow and k e hillstrom acm trans math software 7 1981 pp 17 41 on several synthetic problems allowing control of the clustering of eigenvalues in the hessian spectrum and on some large scale problems in oceanography and meteorology the results indicate that among the tested limited memory quasi newton methods the l bfgs method d c liu and j nocedal math programming 45 1989 pp 503 528 has the best overall performance for the problems examined the numerical performance of two truncated newton methods differing in the inner loop solution for the search vector is competitive with that of l bfgs

differential algebraic equations with after effect in this paper we are concerned with the solution of delay differential algebraic equations these are differential algebraic equations with after effect or constrained delay differential equations the general semi explicit form of the problem consists of a set of delay differential equations combined with a set of constraints that may involve retarded arguments even simply stated problems of this type can give rise to difficult analytical and numerical problems the more tractable examples can be shown to be equivalent to systems of delay or neutral delay differential equations our purpose is to highlight some of the complexities and obstacles that can arise when solving these problems and to indicate problems that require further research

local linearization method for numerical integration of delay differential equations in this paper a new approach for the numerical computation of delay differential equations ddes is introduced the essential idea consists of obtaining numerical integrators that use a code expressly developed for linear ddes in contrast with the conventional approach of using a code for ordinary differential equations specifically two numerical schemes of this new class of integrators are proposed and their numerical viability analyzed it includes the estimation of the convergence rate the evaluation of the computational cost of the schemes and a simulation study it is proved that these one step explicit integrators converge uniformly with order two to the solution of nonlinear ddes and are able to integrate stiff equations in a satisfactory way with low computational cost

sensitivity analysis for dynamic systems with time lags many problems in bioscience for which observations are reported in the literature can be modelled by suitable functional differential equations incorporating time lags other terminology delays or memory effects parameterized by scientifically meaningful constant parameters p or and variable parameters for example control functions u t it is often desirable to have information about the effect on the solution of the dynamic system of perturbing the initial data control functions time lags and other parameters appearing in the model the main purpose of this paper is to derive a general theory for sensitivity analysis of mathematical models that contain time lags in this paper we use adjoint equations and direct methods to estimate the sensitivity functions when the parameters appearing in the model are not only constants but also variables of time to illustrate the results the methodology is applied numerically to an example of a delay differential model

the tunneling method for global optimization in multidimensional scaling this paper focuses on the problem of local minima of the stress function it turns out that unidimensional scaling is particularly prone to local minima whereas full dimensional scaling with euclidean distances has a local minimum that is global for intermediate dimensionality with euclidean distances it depends on the dissimilarities how severe the local minimum problem is for city block distances in any dimensionality many different local minima are found a simulation experiment is presented that indicates under what conditions local minima can be expected we introduce the tunneling method for global minimization and adjust it for multidimensional scaling with general minkowski distances the tunneling method alternates a local search step in which a local minimum is sought with a tunneling step in which a different configuration is sought with the same stress as the previous local minimum in this manner successively better local minima are obtained and experimentation so far shows that the last one is often a global minimum

p stability properties of runge kutta methods for delay differential equations recently the author and others have studied a class of runge kutta methods for ddes these methods make use of certain constrained meshes which allow to get optimal order results in this paper we study their asymptotic stability properties using the test equation begin gathered y t ay t by t tau t 0 hfill y t g t for tau leqq t leqq 0 hfill end gathered wherea b 0 andg t is continuous and complex valued it is known thaty t 0 ast if b re a in particular we show that any astable one step collocation method for odes inherits the same property when it is applied to ddes with a constrained mesh i e it is p stable

asymptotic expansions for the midpoint rule applied to delay differential equations let u be the solution of the delay differential equation begin gathered u t f t u t u t tau quad text for t 0 hfill u t varphi t qquad text for tau leqq t leqq 0 hfill end gathered for t j jh let u h t j be the approximation for u t j obtained by the explicit midpoint rule initialized by one euler step it is shown that if f and varphi are sufficiently differentiable and tau h in mathbb n there are functions e k such that for j even u h left t j right u left t j right sum k 1 r h 2k e k left t j right o left h 2r 2 right this expansion of the global discretization error exists in spite of the fact that the solution u usually has jump discontinuities in its derivatives at the points t n tau n in mathbb n thus repeated extrapolation to the limit richardson extrapolation may be applied to improve the approximation furthermore it is show

decomposition methods for linear support vector machines in this letter we show that decomposition methods with alpha seeding are extremely useful for solving a sequence of linear support vector machines svms with more data than attributes this strategy is motivated by keerthi and lin 2003 who proved that for an svm with data not linearly separable after c is large enough the dual solutions have the same free and bounded components we explain why a direct use of decomposition methods for linear svms is sometimes very slow and then analyze why alpha seeding is much more effective for linear than nonlinear svms we also conduct comparisons with other methods that are efficient for linear svms and demonstrate the effectiveness of alpha seeding techniques in model selection

stepsize control for delay differential equations using a pair of formulae abstract the effect of the local approximation error on the stepsize control at one step methods which are used for the numerical solution of delay differential equations are considered it is shown how to get a reliable estimate for the local error and so a working stepsize control by using a pair of formulae

convergence analysis of the solution of retarded and neutral delay differential equations by continuous numerical methods we have recently developed a generic approach to solving retarded and neutral delay differential equations ddes the approach is based on the use of an explicit continuous runge kutta formula and employs defect control the approach can be applied to problems with state dependent delays and vanishing delays in this paper we determine convergence properties for the numerical solution associated with methods that implement this approach we first analyze such properties for retarded ddes and then the analysis is extended to the case of neutral ddes ndes such an extension is particularly important since ndes have received little attention in the literature the main result we establish is that the global error of the numerical solution can be efficiently and reliably controlled by directly monitoring the magnitude of the associated defect our analysis can also be applied directly to other numerical methods which use defect error control based on runge kutta formulas for ddes

on the stability of exponential fitting bdf algorithms we present bdf type formulas capable of the exact integration with only round off errors of differential equations whose solutions are linear combinations of an exponential with parameter l and ordinary polynomials for l 0 new formulas reduces to the classical bdf formulas plots of their 0 stability regions in terms of l are provided plots of their regions absolute stability that include all the negative real axis are provided numerical examples shows the efficiency of the proposed codes specially when we are integrating stiff oscillatory problems