the factorization method for the numerical solution of two point boundary value problems for linear ode s the theoretical analysis and computational implementation of factorization based methods for the numerical solution of linear boundary value problems for ordinary differential equations are presented the methods are optimal with respect to certain clearly defined criteria numerical examples show the effectiveness of a general code based on the factorization method

conditioning of collocation matrices and discrete green s functions the conditioning of spline collocation matrices is considered in the case of a boundary value problem for an mth order equation for bases with continuity built in which corresponds to most current implementations of splines for solving odes the collocation matrix a is related to the corresponding multiple shooting matrix for a first order system this gives the explicit form for a 1 as a discrete green function we briefly discuss the implications for mesh selection and how our results relate to previous ones for finite differences

nonlinear non negative component analysis algorithms in this paper general solutions for nonlinear non negative component analysis for data representation and recognition are proposed motivated by a combination of the non negative matrix factorization nmf algorithm and kernel theory which has lead to a recently proposed nmf algorithm in a polynomial feature space we propose a general framework where one can build a nonlinear non negative component analysis method using kernels the so called projected gradient kernel non negative matrix factorization pgknmf in the proposed approach arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem moreover we propose fixed point algorithms for the special case of gaussian radial basis function rbf kernels we demonstrate the power of the proposed methods in face and facial expression recognition applications

invariant imbedding the box scheme and an equivalence between them we show for general linear two point boundary value problems that the box scheme and invariant imbedding are equivalent in the sense that a specific algorithm for solving the difference equations is valid if and only if an appropriate imbedding is valid a key element in the equivalence is the role played by pivoting to solve the linear equations and switching to get a finite imbedding part of the equivalence proof shows that imbedding is the completion of the box scheme multiple imbeddings are introduced and a lower bound for the critical length problem is obtained

tangent hyperplane kernel principal component analysis for denoising kernel principal component analysis kpca is a method widely used for denoising multivariate data using geometric arguments we investigate why a projection operation inherent to all existing kpca denoising algorithms can sometimes cause very poor denoising based on this we propose a modification to the projection operation that remedies this problem and can be incorporated into any of the existing kpca algorithms using toy examples and real datasets we show that the proposed algorithm can substantially improve denoising performance and is more robust to misspecification of an important tuning parameter

the state space extension method in the theory of absolute stability this paper shows how the state space extension method can be used to derive frequency domain criteria for stability of systems with different kinds of uncertainties the main result is illustrated by a number of examples

a riccati transformation method for solving linear bvps ii computational aspects in part i of this paper siamj numer anal 25 1988 pp 1055 1073 a riccati transformation method for solving linear boundary value problems for ordinary differential equations is presented here the implementation aspects of this method are discussed in particular a double sweep integration process which includes a strategy for solving a matrix riccati differential equation and forming a continuous representation of its solution is described the efficacy of the method is demonstrated on a number of singular perturbation problems particular attention is paid to the choice of scaling for the variables and its effect on the solution method the computational experience while not based upon a refined implementation indicates that the riccati method may well prove to be competitive with the other general methods

a note on conditioning stability and collocation matrices the concept of a well conditioned bvp as considered in recent literature essentially corresponds to the concept of a uniformly stable ivp but this terminology suggests a shift from absolute to relative quantities which makes it more difficult to understand the conditioning of numerical discretizations especially for higher order differential operators in this note we wish to make it clear that the notion of well conditioning of bvps is not always a direct analogue of the u the condition number of a discretization of a well conditioned bvp generally grows unboundedly when higher accuracy is desired because it contains a discretization of a differential operator this condition number gets large particularly rapidly if a direct discretization of a higher order differential operator is attempted for collocation an implementation which uses a local runge kutta solution representation produces a much better conditioning because the repeated numerical differentiation is avoided instead the highest derivative is approximated locally by a discontinuous piece wise polynomial and local integrations yield lower solution derivatives upon imposing the collocation equations the highest derivative approximation can be locally eliminated resulting in a sophisticated discretization of the corresponding first order system

stability of a three point scheme for linear second order singularly perturbed bvps with turning points in this paper we study stability properties of a numerical method applied to linear second order perturbed boundary value problems with boundary or interior layers the method consists of a three point difference scheme with variable stepsize the stability of which may be investigated by studying that of an lu factorization for the coefficient matrix of a suitable tridiagonal system we also propose a mesh selection strategy for treating boundary and interior layers which furnishes a good mesh on which the three point scheme is stable

on spline basis selection for solving differential equations the suitability of b splines as a basis for piecewise polynomial solution representation for solving differential equations is challenged two alternative local solution representations are considered in the context of collocating ordinary differential equations hermite type and lmonomial both are much easier and shorter to implement and somewhat more efficient than b splines a new condition number estimate for the b splines and hermite type representations is presented one choice of the hermite type representation is experimentally determined to produce roundoff errors at most as large as those for b splines the monomial representation is shown to have a much smaller condition number than the other ones and correspondingly produces smaller roundoff errors especially for extremely nonuniform meshes the operation counts for the two local representations considered are about the same the hermite type representation being slightly cheaper it is concluded that both representations are preferable

mercer kernel based clustering in feature space the article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data this work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure it is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data

a well posed shooting method for transferable dae s consider a tpbvp for transferable nonlinear dae s in general the shooting equation has a singular jacobian a multiple shooting method which has a nonsingular jacobian and also produces consistent initial values for the integration is presented the estimation of the condition of the jacobian shows the well posedness of the method some illustrative examples are given

the order of accuracy for deferred corrections using uncentered end formulas a result of christiansen and russell siam j numer math 35 1980 pp 21 33 on the order of accuracy of an iterated deferred corrections algorithm of lentini and pereyra siam j numer math 14 1977 pp 91 111 is improved at the same time providing a realistic example for a theoretical framework developed by the author for proving accuracy results for deferred corrections

the conditioning of linear boundary value problems investigation is made into the sensitivity of solutions of linear boundary value problems to perturbations of the boundary condition we derive a useful quantity to decide for well or ill conditioning from this it is deduced which kind of requirements the boundary conditions should meet in order to have a well conditioned problem it is shown that this quantity can fruitfully be used to explain why e g the multiple shooting technique is stable in contrast to the single shooting one for certain well posed problems having solutions with different growth behavior the results are sustained by a number of examples

the close relationships between methods for solving two point boundary value problems the well conditioning of a boundary value problem is shown to be related to bounding two quantities one involving the boundary conditions and the other involving the green s function in the case of a solution dichotomy they are related to known stability results these results easily explain why difficulties arise using superposition and reduced superposition it is shown how multiple shooting overcomes these difficulties by relating its matrix conditioning to the underlying boundary value problem some factorization methods for the multiple shooting method are considered these show the close relationships between multiple shooting the stabilized march and invariant imbedding and one factorization leads to an efficient new way to implement each of these three methods many of the results apply for collocation and finite difference methods too finally it is shown both theroetically and computationally when matrix compactification can lead to difficulty

on a fourth order finite difference method for singularly perturbed boundary value problems we present a fourth order finite difference method for singularly perturbed one dimensional reaction diffusion problem the problem is discretized using a bakhvalov type mesh we give a uniform convergence with respect to the perturbation parameter numerical examples are presented which demonstrate computationally the fourth order of the method

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

nonlinear denoising and analysis of neuroimages with kernel principal component analysis and pre image estimation we investigate the use of kernel principal component analysis pca and the inverse problem known as pre image estimation in neuroimaging i we explore kernel pca and pre image estimation as a means for image denoising as part of the image preprocessing pipeline evaluation of the denoising procedure is performed within a data driven split half evaluation framework ii we introduce manifold navigation for exploration of a nonlinear data manifold and illustrate how pre image estimation can be used to generate brain maps in the continuum between experimentally defined brain states classes we base these illustrations on two fmri bold data sets one from a simple finger tapping experiment and the other from an experiment on object recognition in the ventral temporal lobe

numerical solution of eigenvalue problems for linear boundary value odes abstract interrelationships between several popular approaches for solving eigenvalue problems for linear boundary value odes are given for linear eigenvalue problems the popular methods can be interpreted in a common framework this leads us to propose and justify alternative strategies the choice of numerical methods used here is motivated by the desire to solve eigenvalue problems for stiff odes in particular we consider a one step global method spline collocation and two initial value methods riccati and continuous orthonormalization to solve the orr sommerfeld equation a comparison of results for these methods using various implementation strategies is given

linear system solvers for boundary value odes abstract we investigate the stability properties of several linear system solvers for solving boundary value odes we consider the compactification algorithm gaussian elimination with row partial pivoting and a qr algorithm applied to linear systems arising from solving bvps for which the matrix is block bidiagonal except for bordering along the last n rows and columns we will particularly compare auto s original linear solver an lu decomposition with partial pivoting and our implementation of the analogous qr algorithm to auto two other factors the underlying continuation strategy and mesh selection strategy may affect the stability of the linear system solver for ode continuation codes as well and are also discussed in our numerical investigations

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

the pre image problem in kernel methods in this paper we address the problem of finding the pre image of a feature vector in the feature space induced by a kernel this is of central importance in some kernel applications such as on using kernel principal component analysis pca for image denoising unlike the traditional method in which relies on nonlinear optimization our proposed method directly finds the location of the pre image based on distance constraints in the feature space it is noniterative involves only linear algebra and does not suffer from numerical instability or local minimum problems evaluations on performing kernel pca and kernel clustering on the usps data set show much improved performance

boundary value problems and dichotomic stability since the conditioning of a boundary value problem bvp is closely related to the existence of a dichotomic fundamental solution i e where one set of modes is increasing and a complementary set is decreasing it is important to have discretization methods that conserve this dichotomy property the conditions this imposes on such a method are investigated in this paper they are worked out in more detail for scalar second order equations the central difference scheme and for linear first order systems as well for the latter type both one step methods including collocation and multistep methods those that may be used in multiple shooting are examined

numerical solution of two point boundary value problems abstract this paper presents an original formulation of two point boundary value and eigenvalue problems expressed as a system of first order equations the fundamental difference between the new method and other methods based on a first order approach is the introduction of conditions of an integral character to supplement the simultaneous set of first order equations which are hence never regarded as an initial value problem the consideration of integral conditions leads to establish a class of linear multipoint schemes for the numerical solution of boundary value problems for ordinary differential equations furthermore the global character of the integral conditions nonlocality combined with the block structure of the system of algebraic equations allow dealing with stiff problems by means of the classical procedure of iterative refinement introduced by wilkinson the properties of the numerical schemes are illustrated by the solution of linear and nonlinear problems and by the accurate and efficient determination of some eigensolutions of a difficult problem of hydrodynamic stability the proposed method is conceptually simpler and numerically more convenient than existing initial value methods while still retaining all the advantages of a formulation based on a first order system

on dichotomy and well conditioning in two point boundary value problems we investigate the close relationships between the stability constants and the growth behavior of the fundamental matrix to the general boundary value problem we also show that conditioning number is the right criterion to indicate possible error amplification of the perturbed boundary conditions

collocation for singular perturbation problems i first order systems with constant coefficients the application of collocation methods for the numerical solution of singularly perturbed ordinary differential equations is investigated collocation at gauss radau and lobatto points is considered for both initial and boundary value problems for first order systems with constant coefficients particular attention is paid to symmetric schemes for boundary value problems these problems may have boundary layers at both interval ends br our analysis shows that certain collocation schemes in particular those based on gauss or lobatto points do perform very well on such problems provided that a fine mesh with steps proportional to the layers width is used in the layers only and a coarse mesh just fine enough to resolve the solution of the reduced problem is used in between ways to construct appropriate layer meshes are proposed of all methods considered the lobatto schemes appear to be the most promising class of methods as they essentially retain their usual superconvergence power for the smooth reduced solution whereas gauss legendre schemes do not br we also investigate the conditioning of the linear systems of equations arizing in the discretization of the boundary value problem for a row equilibrated version of the discretized system we obtain a pleasantly small bound on the maximum norm condition number which indicates that these systems can be solved safely by gaussian elimination with scaled partial pivoting

on dichotomy and well conditioning in bvp we investigate the relationships between the stability bounds of the problem on the one hand and the growth behaviour of the fundamental solution on the other hand it is shown that if these stability bounds are moderate i e if the problem is well conditioned then the homogeneous solution space is dichotomic which means that it can be split into a subspace of nondecreasing and a complementary subspace of nonincreasing modes this is done by carefully examining the green s functions if these exhibit an exponential behaviour then the solution space is also exponentially dichotomic on the other hand we also show that exponential dichotomy implies moderate stability constants i e well conditioning from this it follows that both concepts are more or less equivalent

on stability constants and condition numbers of discretisation methods discretizations lead to systems of algebraic equations of steadily growing size if the accuracy of the approximation is increasing inverse stability constants and condition numbers describe the sensitivity of the solution of these equations with respect to perturbations of the data starting from an abstract result in this paper the inverse stability constants of finite difference and collocation methods for bvodes as well as of ritz galerkin methods for elliptic variational problems are shown to converge to those of their continuous counterparts as the discretization becomes finer and finer for two choices of basis systems the exact asymptotic behavior of the collocation method condition number is given

uniform fourth order difference scheme for a singular perturbation problem the numerical solution of a nonlinear singularly perturbed two point boundary value problem is studied the developed method is based on hermitian approximation of the second derivative on special discretization mesh numerical examples which demonstrate the effectiveness of the method are presented

conditions and conditioning stability and stabilization an overview is given of several concepts that are useful when dealing with numerical bvp in particular a generalisation of the well conditioning concept for nonlinear problems is considered this is done to be able to investigate a class of second order scalar nonlinear problems a detailed study of the solution structure is made for this class the results are applied to two specific problems korteweg de vries and burgers and a way is indicated to stabilize these ill posed bvp

decoupling and stability of algorithms for boundary value problems the ordinary differential equations occurring in linear boundary value problems characteristically have both stable and unstable solution modes therefore a stable numerical algorithm should avoid both forward and backward integration of solutions on large intervals it is shown that most methods like multiple shooting collocation invariant imbedding and difference methods derive their stability from the fact that they all decouple the continuous or the discrete problem sooner or later for instance when solving a linear system this decoupling is related to the dichotomy of the ordinary differential equations in fact it turns out that the inherent initial value instability is an important prerequisite for a stable utilization of the decoupled representations from which the solutions are computed how this stability is related to the use of the boundary conditions is also investigated

a riccati transformation method for solving linear bvps i theoretical aspects a method of solving linear two point boundary value problems bvps for first order systems of ordinary differential equations odes is considered of particular interest is the class of singularly perturbed bvps the method attempts to split the integration of the rapidly increasing decreasing solution components typical for such problems into different initial value problems ivps the case of separated boundary conditions bcs is considered and their ability to detect solution components that eventually increase decrease is exploited by using the riccati transformation the original bvp is then reformulated as three ivps one of them being a matrix riccati equation the riccati equation is nonlinear and a multiple embedding setting must be used the entire method is then fit theoretically into a multiple shooting like framework and the original bvp as a chain of subproblems is studied in this way a generalized decoupling result is presented which enables the study of stability to be brough