feasible nonparametric estimation of multiargument monotone functions abstract this article presents a two stage estimation procedure that uses an ad hoc but very easily implemented isotonization of a kernel estimator this procedure yields an isotonic estimator with the convergence properties of the kernel estimator although the isotonization in the second stage does not satisfy the least squares condition this hybrid estimator may be considered to be a multidimensional generalization of similar procedures for the one dimensional case suggested by friedman and tibshirani and by mukarjee we derive some of the asymptotic properties of our estimator and demonstrate other statistical properties with monte carlo studies we conclude by providing a real data example

a note on a result of sun ji guang sensitivity of the cs and gsv decompositions in siam j numer anal 20 1983 pp 611 625 sun ji guang gives a perturbation analysis of a generalized singular value decomposition this note rephrases some of his ideas and derives an improved version of one of his results using straightforward matrix ideas some elaboration is given on the meaning of the result and the analysis is extended to cover the case of most usual interest to the numerical analyst

relatively robust representations of symmetric tridiagonals let ldlt be the triangular factorization of an unreduced symmetric tridiagonal matrix t i small relative changes in the nontrivial entries of l and d may be represented by diagonal scaling matrices 1 and 2 ldlt 2l 1d 1lt 2 the effect of 2 on the eigenvalues i is benign in this paper we study the inner perturbations induced by 1 suitable condition numbers govern the relative changes in the eigenvalues i we show that when j is an eigenvalue then the relative condition number of m j m j is the same for all n twisted factorizations one of which is ldlt that could be used to represent t i see section 2 we prove that as j the smallest eigenvalue has relative condition number relcond 1 o j each relcond is a rational function of we identify the poles and then use orthogonal polynomial theory to develop upper bounds on the sum of the relconds of all the eigenvalues these bounds require o n operations for an n n matrix we show that the sum of all the relconds is bounded by trace l d lt and conjecture that n ldlt the quantity trace l d lt ldlt is a natural measure of element growth in the context of this paper an algorithm for computing numerically orthogonal eigenvectors without recourse to the gram schmidt process is sketched it requires that there exist values of close to each cluster of close eigenvalues such that all the relconds belonging to the cluster are modest say 10 the sensitivity of the other eigenvalues is not important for this reason we develop o n bounds on the sum of the relconds associated with a cluster none of our bounds makes reference to the nature of the distribution of the eigenvalues within a cluster which can be very complicated

a new stable bidiagonal reduction algorithm a new bidiagonal reduction method is proposed for x r m n f orm n it decomposes x into the product x ubv t where u r m n has orthonormal columns v r n n is orthogonal and b r n n is upper bidiagonal the matrix v is computed as a product of householder transformations the matrices u and b are constructed using a recurrence if u is desired from the computation the new procedure requires fewer operations than the golub kahan procedure siam j num anal ser b 2 1965 205 and similar procedures in floating point arithmetic the columns of u may be far from orthonormal but that departure from orthonormality is structured the application of any backward stable singular value decomposition procedure to b recovers the left singular vectors associated with the leading largest singular values of x to near orthogonality

implementation of jacobi rotations for accurate singular value computation in floating point arithmetic in this paper we con if begin displaymath 1 m r value decomposition svd a u sigma v tau of a a 1 a 2 in r m times 2 accurately in floating point arithmetic it is shown how the rotation angacobi rotation v the right singular vector matrix and how to compute av u sigma even if the floating point representation of v is the identity matrix in the case ns a 1 gg ns a 2 underflow can produce the identity matrix as the floating point value of v even for a 1 a 2 that are far from being mutually orthogonal this can cause loss of accuracy and failure of convergence of the floating point implementation of the jacobi method for computing the svd the modified jacobi method recommended in this paper can be implemented as a reliable and highly accurate procedure for computing the svd of general real matrices whenever the exact singular values do not exceed the underflow or overflow limits

towards a generalized singular value decomposition we suggest a form for and give a constructive derivation of the generalized singular value decomposition of any two matrices having the same number of columns we outline its desirable characteristics and compare it to an earlier suggestion by van loan siam j numer anal 13 1976 pp 76 83 the present form largely follows from the work of van loan but is slightly more general and computationally more amenable than that in the paper cited we also prove a useful extension of a theorem of stewart siam rev 19 1977 pp 634 662 on unitary decompositions of submatrices of a unitary matrix

history and generality of the cs decomposition abstract it is almost a quarter of a century since chandler davis and william kahan brought together the key ideas of what stewart later completed and defined to be the cs decomposition csd of a partitioned unitary matrix this paper outlines some germane points in the history of the csd pointing out the contributions of jordan of davis and kahan and of stewart and the relationship of the csd to the direct rotation of davis and kato the paper provides an easy to memorize constructive proof of the csd reviews one of its important uses and suggests a motivation for the csd which emphasizes how generally useful it is it shows the relation between the csd and generalized singular value decompositions and points out some useful nullity properties one form of the csd trivially reveals finally it shows how via the qr factorization the csd can be used to obtain interesting results for partitioned nonsingular matrices we suggest the csd be taught in its most general form with no restrictions on the two by two partition and initially with no mention of angles between subspaces

a relative perturbation bound for positive definite matrices abstract we give a sharp estimate for the eigenvectors of a positive definite hermitian matrix under a floating point perturbation the proof is elementary

singular value and generalized singular value decompositions and the solution of linear matrix equations abstract the solution of the linear matrix equations i axb cyd e and ii axb fxg e h are considered new necessary and sufficient conditions for the consistency of the equations are derived some using the generalized singular value decomposition special cases iii ax yd e and iv axb e are treated using the singular value decomposition numerical algorithms for the solutions are also suggested

rounding error and perturbation bounds for the indefinite qr factorization abstract indefinite qr factorization is a generalization of the well known qr factorization where q is a unitary matrix with respect to the given indefinite inner product matrix j this factorization can be used for accurate computation of eigenvalues of the hermitian matrix a g jg where g and j are initially given or naturally formed from initial data the classical example of such a matrix is a b b c c with given b and c in this paper we present the rounding error and perturbation bounds for the so called triangular case of the indefinite qr factorization these bounds fit well into the relative perturbation theory for hermitian matrices given in factorized form

computing thecs decomposition of a partitioned orthonormal matrix this paper describes an algorithm for simultaneously diagonalizing by orthogonal transformations the blocks of a partitioned matrix having orthonormal columns

partially linear monotone methods with automatic variable selection and monotonicity direction discovery in many statistical regression and prediction problems it is reasonable to assume monotone relationships between certain predictor variables and the outcome genomic effects on phenotypes are for instance often assumed to be monotone however in some settings it may be reasonable to assume a partially linear model where some of the covariates can be assumed to have a linear effect one example is a prediction model using both high dimensional gene expression data and low dimensional clinical data or when combining continuous and categorical covariates we study methods for fitting the partially linear monotone model where some covariates are assumed to have a linear effect on the response and some are assumed to have a monotone potentially nonlinear effect most existing methods in the literature for fitting such models are subject to the limitation that they have to be provided the monotonicity directions a priori for the different monotone effects we here present methods for fitting partially linear monotone models which perform both automatic variable selection and monotonicity direction discovery the proposed methods perform comparably to or better than existing methods in terms of estimation prediction and variable selection performance in simulation experiments in both classical and high dimensional data settings

accurate computation of the product induced singular value decomposition with applications we present a new algorithm for floating point computation of the singular value decomposition svd of the product b tau c where b and c are full row rank matrices the algorithm replaces the pair b c with an equivalent pair b c and then it uses the jacobi svd algorithm to compute the svd of the explicitly computed matrix b tau c in this way each nonzero singular value sigma is approximated with some sigma delta sigma where the relative error delta sigma sigma is up to a factor of the dimensions of order boldmath varepsilon unboldmath min delta in cal d kappa 2 delta b min delta in cal d kappa 2 delta c where cal d denotes the set of diagonal nonsingular matrices kappa 2 cdot denotes the spectral condition number and boldmath varepsilon unboldmath is the roundoff unit of floating point arithmetic the new algorithm is applied to the eigenvalue problem hm x lambda x with symmetric positive definite h and m it is shown that each eigenvalue lambda is computed with high relative accuracy and that the relative error delta lambda lambda of the computed approximation lambda delta lambda is up to a factor of the dimension of order boldmath varepsilon unboldmath min delta in cal d kappa 2 delta h delta min delta in cal d kappa 2 delta m delta the new algorithm can also be used for accurate svd computation of a single matrix g that admits an accurate factorization g b tau c

communication optimal parallel and sequential qr and lu factorizations we present parallel and sequential dense qr factorization algorithms that are both optimal up to polylogarithmic factors in the amount of communication they perform and just as stable as householder qr we prove optimality by deriving new lower bounds for the number of multiplications done by non strassen like qr and using these in known communication lower bounds that are proportional to the number of multiplications we not only show that our qr algorithms attain these lower bounds up to polylogarithmic factors but that existing lapack and scalapack algorithms perform asymptotically more communication we derive analogous communication lower bounds for lu factorization and point out recent lu algorithms in the literature that attain at least some of these lower bounds the sequential and parallel qr algorithms for tall and skinny matrices lead to significant speedups in practice over some of the existing algorithms including lapack and scalapack for example up to 6 7 times over scalapack a performance model for the parallel algorithm for general rectangular matrices predicts significant speedups over scalapack

from bareiss algorithm to the stable computation of partial correlations abstract this paper presents a new algorithm for the stable computation of sample partial correlation coefficients we start by generalizing bareiss algorithm for the solution of linear systems of equations with non symmetric toeplitz coefficient matrix to matrices that are not toeplitz and show that it computes their lu and ul factorizations for symmetric positive definite matrices b the normalized version of bareiss algorithm is the hyperbolic cholesky algorithm which computes the upper and lower triangular cholesky factors u and l of b by means of 2 2 hyperbolic rotations if b is a convariance matrix certain partial correlation coefficients may be obtained as the geometric means of the multiplier pairs in bareiss algorithm or as the hyperbolic tangents of the hyperbolic rotations the partial correlation coefficients for a given data matrix a may suffer from serious loss of numerical accuracy when computed from the covariance matrix b a t a while the data flow graph of bareiss algorithm suggests a method due to cybenko for computing partial correlation coefficients directly from the columns of a the graph of hyperbolic cholesky suggests an alternative that requires fewer operations nd exhibits a higher degree of parallelism in that method one computes the qr decomposition a qu of the data matrix and applies orthogonal rotations to transform u to l the sines of the rotation angles constitute a set of partial correlation coefficients we extend this approach to determine arbitrary partial correlation coefficients at little extra computational cost

quadratic convergence estimate of scaled iterates by j symmetric jacobi method abstract this paper estimates the quadratic convergence reduction of scaled iterates by j symmetric jacobi method numer math 64 1993 241 although the method is well defined for a general definite pair h j h ht j diag im in m the paper considers the most important case when h is positive definite in that case the method is an accurate floating point eigensolver for the pair h j as such it is used in a compound algorithm for accurate floating point computation of eigenvalues and eigenvectors of a non singular indefinite symmetric matrix the new result is proved for scaled diagonally dominant matrices in the general case of multiple eigenvalues it uses frobenius norm of the off diagonal part of symmetrically scaled iteration matrix and a relative gap in the spectrum of h j it can be effectively used in connection with stopping criterion of the method especially with its one sided version

efficient algorithms for computing a strong rank revealing qr factorization given anm n matrixm withm n it is shown that there exists a permutation fi and an integer k such that the qr factorization myi q ak ckbk reveals the numerical rank of m the k k upper triangular matrix ak is well conditioned ilckll2 is small and bk is linearly dependent on ak with coefficients bounded by a low degree polynomial in n existing rank revealing qr rrqr algorithms are related to such factorizations and two algorithms are presented for computing them the new algorithms are nearly as efficient as qr with column pivoting for most problems and take o ran2 floating point operations in the worst case

computing the cs and the generalized singular value decompositions if the columns of a matrix are orthonormal and it is partitioned into a 2 by 1 block matrix then the singular value decompositions of the blocks are related this is the essence of the cs decomposition the computation of these related svd s requires some care stewart has given an algorithm that uses the linpack svd algorithm together with a jacobitype clean up operation on a cross product matrix our technique is equally stable and fast but avoids the cross product matrix the simplicity of our technique makes it more amenable to parallel computation on systolic type computer architectures these developments are of interest because a good way to compute the generalized singular value decomposition of a matrix pair a b is to compute the cs decomposition of a certain orthogonal column matrix related toa andb

approximate eigenvectors as preconditioner given approximate eigenvector matrix u of a hermitian nonsingular matrix h the spectral decomposition of h can be obtained by computing h u hu and then diagonalizing h this work addresses the issue of numerical stability of the transition from h to h in finite precision arithmetic our analysis shows that the eigenvalues will be computed with small relative error if i the approximate eigenvectors are sufficiently orthonormal and ii the matrix h h 2 is of the form dad with diagonal d and well conditioned a in that case h can be efficiently and accurately diagonalized by the jacobi method if u is computed by fast eigensolver based on tridiagonalization this procedure usually gives the eigensolution with high relative accuracy and it is more efficient than accurate jacobi type methods on their own

perturbation theory for the eigenvalues of factorised symmetric matrices abstract we obtain eigenvalue perturbation results for a factorised hermitian matrix h gjg where j 2 i and g has full row rank and is perturbed into g g where g is small with respect to g this complements the earlier results on the easier case of g with full column rank applied to square factors g our results help to identify the so called quasidefinite matrices as a natural class on which the relative perturbation theory for the eigensolution can be formulated in a way completely analogous to the one already known for positive definite matrices

comparison of fuzzy numbers using a fuzzy distance measure a new approach for ranking fuzzy numbers based on a distance measure is introduced a new class of distance measures for interval numbers that takes into account all the points in both intervals is developed first and then it is used to formulate the distance measure for fuzzy numbers the approach is illustrated by numerical examples showing that it overcomes several shortcomings such as the indiscriminative and counterintuitive behavior of several existing fuzzy ranking approaches

relative perturbation theory for hyperbolic eigenvalue problem we give relative perturbation bounds for eigenvalues and perturbation bounds for eigenspaces of a hyperbolic eigenvalue problem hx jx where h is a positive definite matrix and j is a diagonal matrix of signs we consider two types of perturbations when a graded matrix h d ad is perturbed in a graded sense to h h d a a d and the multiplicative perturbations of the form h h i e h i e our bounds are simple to compute compare well to the classical results and can be used when analyzing numerical algorithms

perturbation analysis for the generalized singular value problem this paper discusses perturbation bounds for generalized singular values and for subspaces associated with the generalized singular value decomposition of two matrices having the same number of columns by a limiting procedure from our results we can derive those for the ordinary singular value problem which were proved by l mirsky quart j math oxford 11 1960 pp 50 59 and p a wedin bit 12 1972 pp 99 111

highly accurate symmetric eigenvalue decomposition and hyperbolic svd let g be a m n real matrix with full column rank and let j be a n n diagonal matrix of signs jii 1 1 the hyperbolic singular value decomposition hsvd of the pair g j is defined as g u v 1 where u is orthogonal is positive definite diagonal and v is j orthogonal matrix vtjv j we analyze when it is possible to compute the hsvd with high relative accuracy this essentially means that each computed hyperbolic singular value is guaranteed to have some correct digits even if they have widely varying magnitudes we show that one sided j orthogonal jacobi method method computes the hsvd with high relative accuracy more precisely let b gd 1 where d is diagonal such that the columns of b have unit norms essentially we show that the computed hyperbolic singular values of the pair g j will have log10 e min b correct decimal digits where e is machine precision we give the necessary relative perturbation bounds and error analysis of the algorithm our numerical tests confirmed all theoretical results for the symmetric non singular eigenvalue problem hx x we analyze the two step algorithm which consists of factorization h gjgt followed by the computation of the hsvd of the pair g j here g is square and non singular let b dg where d is diagonal such that the rows of b have unit norms and let b be defined as above essentially we show that the computed eigenvalues of h will have log10 e 2min b e min b correct decimal digits this accuracy can be much higher then the one obtained by the classical qr and jacobi methods applied to h where the accuracy depends on the spectral condition number of h particularly if the matrices b and b are well conditioned and we are interested in the accurate computation of tiny eigenvalues again we give the perturbation and error bounds and our theoretical predictions are confirmed by a series of numerical experiments we also give the corresponding results for eigenvectors and hyperbolic singular vectors

relative residual bounds for indefinite hermitian matrices abstract we prove several residual bounds for relative perturbations of the eigenvalues of indefinite hermitian matrix the bounds fall into two categories the weyl type bounds and the hofmann wielandt type bounds the bounds are expressed in terms of sines of acute principal angles between certain subspaces associated with the indefinite decomposition of the given matrix the bounds are never worse than the classical residual bounds and can be much sharper in some cases the bounds generalize the existing relative residual bounds for positive definite matrices to indefinite case

an alternative givens ordering a new givens ordering is shown empirically and by an approximate theoretical analysis to take appreciably fewer stages than the standard scheme sharper error bounds than gentleman s ensue and the scheme is better suited to parallel computation other schemes less efficient but more easily analysed are discussed the effect of a possible limit in practice on the number of simultaneous computations is considered

fuzzy estimation for process capability indices process capability indices are summary statistics which measure the actual or the potential performance of process characteristics relative to the target and specification limits in most traditional methods precise estimation is used to assess the capability of manufacturing processes in this paper we introduce an algorithm based on buckley s estimation approach and use a family of confidence intervals to estimate process capability indices c p c p k and c p m the estimators of these indices thus obtained are triangular shaped fuzzy numbers we also present and illustrate method for the comparison of estimated process capability indices numerical examples are given to show the performance of the method

minimizing a linear objective function under a fuzzy max t norm relation equation constraint the work examines the feasibility of minimizing a linear objective function subject to a max t fuzzy relation equation constraint where t is a continuous archimedean t norm conventional methods for solving this problem are significantly improved by first separating the problem into two sub problems according to the availability of positive coefficients this decomposition is thus more easily handled than in previous literature next based on use of the maximum solution of the constraint equation the sub problem with non positive coefficients is solved and the size of the sub problem with positive coefficients reduced as well this step is unique among conventional methods owing to its ability to determine as many optimal variables as possible additionally several rules are developed for simplifying the remaining problem finally those undecided optimal variables are obtained using the covering problem rather than the branch and bound methods three illustrative examples demonstrate that the proposed approach outperforms conventional schemes its potential applications are discussed as well

floating point perturbations of hermitian matrices abstract we consider the perturbation properties of the eigensolution of hermitian matrices for the matrix entries and the eigenvalues we use the realistic floating point error measure a a recently demmel and veselic considered the same problem for a positive definite matrix h showing that the floating point perturbation theory holds with constants depending on the condition number of the matrix a dhd where aii 1 and d is a diagonal scaling we study the general hermitian case along the same lines thus obtaining new classes of well behaved matrices and matrix pairs our theory is applicable to the already known class of scaled diagonally dominant matrices as well as to matrices given by factors like those in symmetric indefinite decompositions we also obtain norm estimates for the perturbations of the eigenprojections and show that some of our techniques extend to non hermitian matrices however unlike in the positive definite case we are still unable to describe simply the set of all well behaved hermitian matrices

effective condition number bounds for convex regularization we derive bounds relating renegar s condition number to quantities that govern the statistical performance of convex regularization in settings that include the ell 1 analysis setting using results from conic integral geometry we show that the bounds can be made to depend only on a random projection or restriction of the analysis operator to a lower dimensional space and can still be effective if these operators are ill conditioned as an application we get new bounds for the undersampling phase transition of composite convex regularizers key tools in the analysis are slepian s inequality and the kinematic formula from integral geometry

the general linear model and the generalized singular value decomposition abstract a new derivation is given for the generalized singular value decomposition of two matrices x and f having the same number of rows it is shown how this decomposition reveals the structure of the general gauss markov linear model y x 2 ff and exhibits the structure and solution of the generalized linear least squares problem used to provide the best linear unbiased estimator for the model the decomposition is used to prove optimality of the estimator and to reveal the structure of the covariance matrix of the error of the estimator

relative perturbation theory for a class of diagonalizable hermitian matrix pairs abstract veselic and slapnicar gave a general perturbation result for the eigenvalues of the hermitian matrix pair h k where k is positive definite in this paper their result is generalized to a wider class of hermitian matrix pairs especially estimates for the relative perturbation of eigenvalues of definite pairs are also obtained

the matrix nearness problem for symmetric matrices associated with the matrix equation atxa btxb c d abstract a direct method based on the projection theorem in inner products spaces the generalized singular value decomposition and the canonical correlation decomposition is presented for finding the optimal approximate solution x in the set s e to a given matrix x where s e denotes the least squares symmetric solution set of the matrix equation atxa btxb c d the analytical expression of the optimal approximate solution x is obtained and an algorithm for finding this solution is also suggested

linear or nonlinear automatic structure discovery for partially linear models partially linear models provide a useful class of tools for modeling complex data by naturally incorporating a combination of linear and nonlinear effects within one framework one key question in partially linear models is the choice of model structure that is how to decide which covariates are linear and which are nonlinear this is a fundamental yet largely unsolved problem for partially linear models in practice one often assumes that the model structure is given or known and then makes estimation and inference based on that structure alternatively there are two methods in common use for tackling the problem hypotheses testing and visual screening based on the marginal fits both methods are quite useful in practice but have their drawbacks first it is difficult to construct a powerful procedure for testing multiple hypotheses of linear against nonlinear fits second the screening procedure based on the scatterplots of individual covariate fits may provide an educated guess on the regressio

on estimation for monotone dose response curves abstract estimating monotone dose response curves with independent normal errors has been studied in detail in the literature a simple procedure using a generalization of the studentized maximum modulus technique is proposed for constructing confidence bands for monotone regression functions this procedure is shown to compare well with its predecessors the center of this smooth and efficient confidence band is then used to estimate a smooth monotone regression function its pointwise mean squared errors compare favorably with the maximum likelihood estimate a weighted average of the isotonic regression and the least squares line and the rank transformed regression a numerical example of a binding inhibition assay is included to illustrate the proposed interval and point estimation procedures

the hyberbolic singular value decomposition and applications a new generalization of the singular value decomposition svd the hyperbolic svd is advanced and its existence is established under mild restrictions the hyperbolic svd accurately and efficiently finds the eigenstructure of any matrix that is expressed as the difference of two matrix outer products signal processing applications where this task arises include the covariance differencing algorithm for bearing estimation in sensor arrays sliding rectangular windowing and array calibration two algorithms for effecting this decomposition are detailed one is sequential and follows a similar pattern to the sequential bidiagonal based svd algorithm the other is for parallel implementation and mimics hestenes svd algorithm 1958 numerical examples demonstrate that like its conventional counterpart the hyperbolic svd exhibits superior numerical behavior relative to explicit formation and solution of the normal equations furthermore the hyperbolic svd applies in problems where the conventional svd cannot be employed

the m n symmetric procrustes problem abstract an p q matrix a is said to be m n symmetric if man man t for given m r n p n r q n in this paper the following m n symmetric procrustes problem is studied find the m n symmetric matrix a which minimizes the frobenius norm of ax b where x and b are given rectangular matrices we use project theorem the singular value decomposition and the generalized singular value decomposition of matrices to analysis the problem and to derive a stable method for its solution the related optimal approximation problem to a given matrix on the solution set is solved furthermore the algorithm to compute the optimal approximate solution and the numerical experiment are given

isotonic regression in general dimensions we study the least squares regression function estimator over the class of real valued functions on 0 1 d that are increasing in each coordinate for uniformly bounded signals and with a fixed cubic lattice design we establish that the estimator achieves the minimax rate of order n min 2 d 2 1 d in the empirical l 2 loss up to polylogarithmic factors further we prove a sharp oracle inequality which reveals in particular that when the true regression function is piecewise constant on k hyperrectangles the least squares estimator enjoys a faster adaptive rate of convergence of k n min 1 2 d again up to polylogarithmic factors previous results are confined to the case d leq2 finally we establish corresponding bounds which are new even in the case d 2 in the more challenging random design setting there are two surprising features of these results first they demonstrate that it is possible for a global empirical risk minimisation procedure to be rate optimal up to polylogarithmic factors even when the corresponding entropy integral for the function class diverges rapidly second they indicate that the adaptation rate for shape constrained estimators can be strictly worse than the parametric rate

on relative residual bounds for the eigenvalues of a hermitian matrix abstract let h be a hermitian matrix x an orthonormal matrix and m x hx then the eigenvalues of m approximate some eigenvalues of h with an absolute error bounded by hx xm 2 the main interest in this work is the relative distance between the eigenvalues of m and some part of the spectrum of h it is shown that distance depends on the angle between the ranges of x and hx

a jacobi eigenreduction algorithm for definite matrix pairs we propose a jacobi eigenreduction algorithm for symmetric definite matrix pairsa j of small to medium size real symmetric matrices withj 2 i j diagonal neitherj nora itself need be definite our jacobi reduction works only on one matrix and usesj orthogonal elementary congruences which include both trigonometric and hyperbolic rotations and preserve the symmetry throughout the process for the rotation parameters only the pivotal elements of the current matrix are needed which facilitates parallelization we prove the global convergence of the method the quadratic convergence was observed in all experiments we apply our method in two situations i eigenreducing a single real symmetric matrix and ii eigenreducing an overdamped quadratic matrix pencil in both cases our method is preceded by a symmetric indefinite decomposition and performed in its one sided variant on the thus obtained factors our method outdoes the standard methods like standard jacobi orqr ql in accuracy in spite of the use of hyperbolic transformations which are not orthogonal a theoretical justification of this behaviour is made elsewhere the accuracy advantage of our method can be particularly drastic if the eigenvalues are of different order in addition in working with quadratic pencils our method is shown to either converge or to detect non overdampedness

additive isotonic models abstract additive isotonic models generalize linear models by replacing lines with isotonic nondecreasing transformations fitted transformations of several explanatory variables are added together and then transformed by a known function to yield fitted values of the response variable the isotonic transformations are chosen to minimize an explicit criterion such as the negative log likelihood by an algorithm that optimizes one transformation at a time while adjusting for the current fitted values of the others cycling until the criterion converges this approach can be used in various situations notably for generalizing ordinary linear regression and linear logistic regression at each step of the algorithm the needed optimal isotonic transformation is found using a simple generalization of the standard pool adjacent violators algorithm ayer brunk ewing reid and silverman 1955 the fitted transformations are always made up of flat steps so the technique is useful for finding optimal strati

component selection and smoothing in multivariate nonparametric regression we propose a new method for model selection and model fitting in multivariate nonparametric regression models in the framework of smoothing spline anova the cosso is a method of regularization with the penalty functional being the sum of component norms instead of the squared norm employed in the traditional smoothing spline method the cosso provides a unified framework for several recent proposals for model selection in linear models and smoothing spline anova models theoretical properties such as the existence and the rate of convergence of the cosso estimator are studied in the special case of a tensor product design with periodic functions a detailed analysis reveals that the cosso does model selection by applying a novel soft thresholding type operation to the function components we give an equivalent formulation of the cosso estimator which leads naturally to an iterative algorithm we compare the cosso with mars a popular method that builds functional anova models in simulations and real examples the cosso method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets the cosso gives very competitive performance in these studies

generalized additive and index models with shape constraints summary we study generalized additive models with shape restrictions e g monotonicity convexity and concavity imposed on each component of the additive prediction function we show that this framework facilitates a non parametric estimator of each additive component obtained by maximizing the likelihood the procedure is free of tuning parameters and under mild conditions is proved to be uniformly consistent on compact intervals more generally our methodology can be applied to generalized additive index models here again the procedure can be justified on theoretical grounds and like the original algorithm has highly competitive finite sample performance practical utility is illustrated through the use of these methods in the analysis of two real data sets our algorithms are publicly available in the r package scar short for shape constrained additive regression

existence uniqueness and continuity of trapezoidal approximations of fuzzy numbers under a general condition abstract the main aim of this paper is to characterize the set of real parameters associated to a fuzzy number represented in a general form which include the most important characteristics with the following property for any given fuzzy number there exists at least a trapezoidal fuzzy number which preserves a fixed parameter the uniqueness of the nearest trapezoidal fuzzy number with this property is proved the average euclidean distance being considered as an important property each resulting trapezoidal approximation operator is continuous the main results are illustrated by examples

parallel qr decomposition of a rectangular matrix we show that the greedy algorithm introduced in 1 and 5 to perform the parallel qr decomposition of a dense rectangular matrix of sizem n is optimal then we assume thatm n2 tends to zero asm andn go to infinity and prove that the complexity of such a decomposition is asymptotically2n when an unlimited number of processors is available

on the symmetric solutions of linear matrix equations abstract the symmetric solutions of linear matrix equations ax c and axb c are considered the necessary and sufficient conditions for the consistency of the equations with a symmetric condition on solutions are derived using the singular value decomposition and the generalized singular value decomposition numerical algorithms for solving the equations are suggested

relative perturbation theory for hyperbolic singular value problem we give relative perturbation bounds for singular values and perturbation bounds for singular subspaces of a hyperbolic singular value problem for the pair g j where g is a full rank matrix and j is a diagonal matrix of signs we consider two types of relative perturbations g g b b d and g g d b b depending whether g has full column or full row rank respectively in both cases we also consider relative element wise perturbations of g which typically occur in numerical computations

on integrated l 1 convergence rate of an isotonic regression estimator for multivariate observations we consider a general monotone regression estimation where we allow for independent and dependent regressors we propose a modification of the classical isotonic least squares estimator and establish its rate of convergence for the integrated l 1 loss function the methodology captures the shape of the data without assuming additivity or a parametric form for the regression function furthermore the degree of smoothing is chosen automatically and no auxiliary tuning is required for the theoretical analysis some simulations and two real data illustrations complement the study of the proposed estimator

weyl type relative perturbation bounds for eigensystems of hermitian matrices we present a weyl type relative bound for eigenvalues of hermitian perturbations a e of not necessarily definite hermitian matrices a this bound given in function of the quantity a 1 2ea 1 2 2 that was already known in the definite case is shown to be valid as well in the indefinite case we also extend to the indefinite case relative eigenvector bounds which depend on the same quantity as a consequence new relative perturbation bounds for singular values and vectors are also obtained using matrix differential calculus techniques we obtain for eigenvalues a sharper first order bound involving the logarithm matrix function which is smaller than not only for small e as expected but for any perturbation

spectral sensitivity of products of bidiagonals abstract simple formulae are derived for certain relative condition numbers tridiagonal matrices may be represented as products of bidiagonals in various ways depending on properties such as symmetry and positive definiteness the condition numbers give the amplification factor for relative changes in a nonzero eigenvalue caused by relative changes in an entry of a bidiagonal factor the formulae show that in many but not all cases these condition numbers are of modest size several examples illustrate the results and raise new questions

the symmetric solution of the matrix equations ax ya c axat bybt c and atxa btxb c d abstract the necessary and sufficient conditions for the existence of and the expressions for the symmetric solutions of matrix equations i ax ya c ii axa t byb t c and iii a t xa b t xb c d are derived in addition the minimum 2 norm least squares symmetric solution of equation i the minimum 2 norm symmetric solution of equation ii and the least squares solution of equation iii are obtained

a method for stochastic multiple criteria decision making based on dominance degrees this paper proposes a method for solving the stochastic multiple criteria decision making smcdm problem where consequences of alternatives with respect to criteria are represented by random variables with probability distributions firstly definitions and related analysis of dominance degree of one probability distribution over another are given then by calculating the dominance degrees the dominance degree matrix of alternative pairwise comparisons with respect to each criterion is built further using promethee ii method an overall dominance degree matrix of alternative pairwise comparisons is constructed and a net flow of each alternative is calculated based on the obtained net flows a ranking of alternatives is determined finally numerical examples for the three cases are given to illustrate the use of the proposed method

a tangent algorithm for computing the generalized singular value decomposition we present two new algorithms for floating point computation of the generalized singular values of a real pair a b of full column rank matrices and for floating point solution of the generalized eigenvalue problem hx lambda mx with symmetric positive definite matrices h and m the pair a b is replaced with an equivalent pair a b and the generalized singular values are computed as the singular values of the explicitly computed matrix f a b 1 the singular values of f are computed using the jacobi method the relative accuracy of the computed singular value approximations does not depend on column scalings of a and b that is the accuracy is nearly the same for all pairs ad 1 bd 2 with d 1 d 2 arbitrary diagonal nonsingular matrices similarly the pencil h lambda m is replaced with an equivalent pencil h lambda m and the eigenvalues of h lambda m are computed as the squares of the singular values of g l h l m 1 where l h l m are the cholesky factors of h m respectively and the matrix g is explicitly computed as the solution of a linear system of equations for the computed approximation lambda delta lambda of any exact eigenvalue lambda the relative error delta lambda lambda is of order p n mbox boldmath varepsilon max min delta in cal d kappa 2 delta h delta min delta in cal d kappa 2 delta m delta where p n is a modestly growing polynomial of the dimension of the problem mbox boldmath varepsilon is the round off unit of floating point arithmetic cal d denotes the set of diagonal nonsingular matrices and kappa 2 cdot is the spectral condition number furthermore floating point computation corresponds to an exact computation with h delta h m delta m where for all i j delta h ij sqrt h ii h jj and delta m ij sqrt m ii m jj are of order of mbox boldmath varepsilon times a modest function of n

sharp asymptotics for isotonic regression the asymptotic behavior of the isotonic estimator of a monotone regression function that is the least squares estimator under monotonicity restriction is investigated in particular it is proved that the 1 distance between the isotonic estimator and the true function is of magnitude n 1 3 moreover it is proved that a centered version of this 1 distance converges at the n1 2 rate to a gaussian variable with fixed variance

a note on one sided jacobi algorithm we propose a one sided or implicit variant of the jacobi diagonalization algorithm for positive definite matrices the variant is based on a previous cholesky decomposition and currently uses essentially one square array which on output contains the matrix of eigenvectors thus reaching the storage economy of the symmetric ql algorithm the current array is accessed only columnwise which makes the algorithm attractive for various parallelized and or vectorized implementations even on a serial computer our algorithm shows improved efficiency in particular if the cholesky step is made with diagonal pivoting on matrices of ordern 25 200 our algorithm is about 2 3 5 times slower than ql thus being almost on the halfway between the standard jacobi and ql algorithms the previous cholesky decomposition can be performed with higher precision without extra time or storage costs thus offering considerable gains in accuracy with highly conditioned input matrices

optimal algorithms for parallel givens factorization on a coarse grained pram we study the complexity of the parallel givens factorization of a square matrix of size n on a shared memory architecture composed with p identical processors coarse grained erew pram we show how to construct an asymptotically optimal algorithm we deduce that tje time complexity is equal to formule and that the minimum number of processors in order to compute the givens factorization in asymptotically optimal time 2n o n is equal to p opt n 2 2 o n these results complete previous analysis presented in the case where the number of processors is unlimited

absolute and relative perturbation bounds for invariant subspaces of matrices absolute and relative perturbation bounds are derived for angles between invariant subspaces of complex square matrices in the two norm and in the frobenius norm the absolute bounds can be considered extensions of davis and kahan s sin theorem to general matrices and invariant subspaces of any dimension the relative bounds are the most general relative bounds for invariant subspaces because they place no restrictions on the matrix or the perturbation when the perturbed subspace has dimension one the relative bound is implied by the absolute bound

fitting time series models to nonstationary processes a general minimum distance estimation procedure is presented for nonstationary time series models that have an evolutionary spectral representation the asymptotic properties of the estimate are derived under the assumption of possible model misspecification for autoregressive processes with time varying coefficients the estimate is compared to the least squares estimate furthermore the behavior of estimates is explained when a stationary model is fitted to a nonstationary process

a hierarchically blocked jacobi svd algorithm for single and multiple graphics processing units we present a hierarchically blocked one sided jacobi algorithm for the singular value decomposition svd targeting both single and multiple graphics processing units gpus the blocking structure reflects the levels of the gpus memory hierarchy the algorithm may outperform magma s textttdgesvd while retaining high relative accuracy to this end we developed a family of parallel pivot strategies on the gpu s shared address space but applicable also to inter gpu communication unlike common hybrid approaches our algorithm in a single gpu setting needs a cpu for the controlling purposes only while utilizing the gpu s resources to the fullest extent permitted by the hardware when required by the problem size the algorithm in principle scales to an arbitrary number of gpu nodes the scalability is demonstrated by more than twofold speedup for sufficiently large matrices on a tesla s2050 system with four gpus versus a single fermi card

generalized reflexive solutions of the matrix equation axb d and an associated optimal approximation problem let r c m x m and s c n x n be nontrivial unitary involutions i e r h r r 1 i m and s h s s 1 i n we say that g c m x n is a generalized reflexive matrix if rgs g the set of all mxn generalized reflexive matrices is denoted by grc m x n in this paper a sufficient and necessary condition for the matrix equation axb d where a c p x m b c n x q and d c p x q to have a solution x grc m x n is established and if it exists a representation of the solution set s x is given an optimal approximation between a given matrix x c m x n and the affine subspace s x is discussed an explicit formula for the unique optimal approximation solution is presented and a numerical example is provided

perturbations of the eigenprojections of a factorized hermitian matrix abstract we give the perturbation bounds for the eigenprojections of a hermitian matrix h gjg where g has full column rank and j is nonsingular under the perturbations of the factor g our bounds hold for example when g is given with elementwise relative error our bounds contain relative gaps between the eigenvalues and may thus be much less pessimistic than the standard norm estimates

least squares solution with the minimum norm for the matrix equation axb gxh c d based on the projection theorem in hilbert space by making use of the generalized singular value decomposition and the canonical correlation decomposition an analytical expression of the least squares solution for the matrix equation axb gxh c d with the minimum norm is derived an algorithm for finding the minimum norm solution is described and some numerical results have been given

cpk index estimation using fuzzy numbers process capacity indices pcis were developed and have been successfully used by companies to compete in and dominate the high profit markets by improving the quality and the productivity since the past two decades there is an essential assumption in the conventional application wherein the output process measurements are precise and distributed as normal random variables since the assumption of normal distribution is untenable errors can occur if the cpk index is computed using non normal data in the present study we address the situation that the output of data from measurement of the quality of a product is insufficiently precise or scarce this is possible when the quality measurement refers to the decision maker s subjective determination in such a situation the linguistic variable that is easier to capture the decision maker s subjective perception is applied to construct the pci cpk the present approach can mitigate the effect when the normal assumption is inappropriate and extends the application of cpk index

accuracy of two svd algorithms for 2 2 triangular matrices a new algorithm for the accurate computation of the singular value decomposition of 2x2 triangular matrices is proposed the algorithm is based on voevodin formulas sharp accuracy bounds are derived by using a subtle error analysis which tracks the signs of the errors of intermediate quantities and does not neglect the non linear parts of the errors the analysis is fine tuned for the case of almost diagonal matrices the same analysis is also used to analyze the errors for the xlasv2 computational routine of lapack the error estimates of the new algorithm compare favorably to those of the lapack routine

improving point and interval estimators of monotone functions by rearrangement suppose that a target function is monotonic and an available original estimate of this target function is not monotonic rearrangements univariate and multivariate transform the original estimate to a monotonic estimate that always lies closer in common metrics to the target function furthermore suppose an original confidence interval which covers the target function with probability at least 1 is defined by an upper and lower endpoint functions that are not monotonic then the rearranged confidence interval defined by the rearranged upper and lower endpoint functions is monotonic shorter in length in common norms than the original interval and covers the target function with probability at least 1 we illustrate the results with a growth chart example

development of fuzzy process accuracy index for decision making problems several statistical decision making tools and methods are available to organize evidence evaluate risks and aid in decision making process capability indices are the summary statistics to point out the process performance in this paper these indices are analyzed to obtain a new decision making tool process accuracy index c a measures the degree of process centering and gives alerts when the process mean departures from the target value it focuses on the location of process mean and the distance between mean and target value we modify the traditional process accuracy index to obtain a new tool under fuzziness with the proposed tool specification limits and process mean can be defined as triangular or trapezoidal fuzzy numbers the proposed tool is illustrated to solve a supplier selection problem

accuracy of the kogbetliantz method for scaled diagonally dominant triangular matrices the paper proves that kogbetliantz method computes all singular values of a scaled diagonally dominant triangular matrix which can be well scaled from both sides symmetrically to high relative accuracy special attention is paid to deriving sharp accuracy bounds for one step one batch and one sweep of the method by a simple numerical test it is shown that the methods based on bidiagonalization are generally not accurate on that class of well behaved matrices

optimal perturbation bounds for the hermitian eigenvalue problem there is now a large literature on structured perturbation bounds for eigenvalue problems of the formvbox hx mx where h and m are hermitian these results give relative error bounds on the ith eigenvalue i of the form i i i and bound the error in the ith eigenvector in terms of the relative gap minj i i j i j 1 2 in general this theory usually restricts h to be nonsingular and m to be positive definite we relax this restriction by allowing h to be singular for our results on eigenvalues weallow m to be positive semi definite and for a few results we allow it to be more general for these problems for eigenvalues that are not zero or infinity under perturbation it is possible to obtain local relative error bounds thus a wider class of problems may be characterized by this theory although it is impossible to give meaningful relative error bounds on eigenvalues that are not bounded away from zero we show that the error in the subspace associated with those eigenvalues can be characterized meaningfully

a general asymptotic scheme for inference under order restrictions limit distributions for the greatest convex minorant and its derivative are considered for a general class of stochastic processes including partial sum processes and empirical processes for independent weakly dependent and long range dependent data the results are applied to isotonic regression isotonic regression after kernel smoothing estimation of convex regression functions and estimation of monotone and convex density functions various pointwise limit distributions are obtained and the rate of convergence depends on the self similarity properties and on the rate of convergence of the processes considered

computational and statistical tradeoffs via convex relaxation modern massive datasets create a fundamental problem at the intersection of the computational and statistical sciences how to provide guarantees on the quality of statistical inference given bounds on computational resources such as time or space our approach to this problem is to define a notion of algorithmic weakening in which a hierarchy of algorithms is ordered by both computational efficiency and statistical efficiency allowing the growing strength of the data at scale to be traded off against the need for sophisticated processing we illustrate this approach in the setting of denoising problems using convex relaxation as the core inferential tool hierarchies of convex relaxations have been widely used in theoretical computer science to yield tractable approximation algorithms to many computationally intractable tasks in the current paper we show how to endow such hierarchies with a statistical characterization and thereby obtain concrete tradeoffs relating algorithmic runtime to amount of data

a bound for the condition of a hyperbolic eigenvector matrix the hyperbolic eigenvector matrix is a matrix x which simultaneously diagonalizes the pair h j where h is hermitian positive definite and j diag 1 such that x hx and x jx j we prove that the spectral condition of x x is bounded byk x mink d hd where the minimum is taken over all non singular matrices d which commute with j this bound is attainable and it can be simply computed similar results hold for other signature matrices j like in the discretized klein gordon equation

an overview of relative sin t theorems for invariant subspaces of complex matrices abstract relative perturbation bounds for invariant subspaces of complex matrices are reviewed with emphasis on bounding the sines of the largest principal angle between two subspaces i e sin theorems the goal is to provide intuition as well as an idea for why the bounds hold and why they look the way they do relative bounds have the advantage of being better at exploiting structure in a perturbation than absolute bounds therefore the reaction of subspaces to relative perturbations can be different than to absolute perturbations in particular there are certain classes of relative perturbations to which subspaces of indefinite hermitian matrices can be more sensitive than subspaces of definite matrices

on a variational formulation of the qsvd and the rsvd abstract recently m t chu r f funderlic and g h golub siam j matrix anal appl 18 1997 1082 1092 presented a variational formulation for the quotient singular value decomposition qsvd of two matrices a r n m c r p m which is a generalization to two matrices of the ordinary singular value decomposition svd and characterizes the role of two orthogonal matrices in qsvd in this paper we give an alternative derivation of this variational formulation and extend it to establish an analogous variational formulation for the restricted singular value decomposition rsvd of a matrix triplet a r n m b r n l c r p m which provides a new understanding of the orthogonal matrices appearing in this decomposition

implicit standard jacobi gives high relative accuracy we prove that the jacobi algorithm applied implicitly on a decomposition a xdx t of the symmetric matrix a where d is diagonal and x is well conditioned computes all eigenvalues of a to high relative accuracy the relative error in every eigenvalue is bounded by o epsilon kappa x where epsilon is the machine precision and kappa x equiv x 2 cdot x 1 2 is the spectral condition number of x the eigenvectors are also computed accurately in the appropriate sense we believe that this is the first algorithm to compute accurate eigenvalues of symmetric indefinite matrices that respects and preserves the symmetry of the problem and uses only orthogonal transformations

componentwise analysis of direct factorization of real symmetric and hermitian matrices abstract we derive componentwise error bound for the factorization h gjg t where h is a real symmetric matrix g has full column rank and j is diagonal with 1 s on the diagonal we also derive a componentwise forward error bound that is we bound the difference between the exact and the computed factor g in the cases where such a bound is possible we extend these results to the hermitian case and to the well known bunch parlett factorization finally we prove bounds for the scaled condition of the matrix g and show that the factorization can have the rank revealing property

risk bounds in isotonic regression nonasymptotic risk bounds are provided for maximum likelihood type isotonic estimators of an unknown nondecreasing regression function with general average loss at design points these bounds are optimal up to scale constants and they imply uniform n 1 3 consistency of the ell p risk for unknown regression functions of uniformly bounded variation under mild assumptions on the joint probability distribution of the data with possibly dependent observations

on the condition numbers of a multiple eigenvalue of a generalized eigenvalue problem for standard eigenvalue problems closed form expressions for the condition numbers of a multiple eigenvalue are known in particular they are uniformly 1 in the hermitian case and generally take different values in the non hermitian case we consider the generalized eigenvalue problem and identify the condition numbers our main result is that a multiple eigenvalue generally has multiple condition numbers even in the hermitian definite case the condition numbers are characterized in terms of the singular values of the outer product of the corresponding left and right eigenvectors

parallel complexity of numerically accurate linear system solvers we prove a number of negative results about practical i e work efficient and numerically accurate algorithms for computing the main matrix factorizations in particular we prove that the popular householder and givens methods for computing the qr decomposition are p complete and hence presumably inherently sequential under both real and floating point number models we also prove that gaussian elimination ge with a weak form of pivoting which aims only at making the resulting algorithm nondegenerate is likely to be inherently sequential as well finally we prove that ge with partial pivoting is p complete over gf 2 or when restricted to symmetric positive definite matrices for which it is known that even standard ge no pivoting does not fail altogether the results of this paper give further formal support to the widespread belief that there is a tradeoff between parallelism and accuracy in numerical algorithms