mismatch in high rate entropy constrained vector quantization bucklew s 1984 high rate vector quantizer mismatch result is extended from fixed rate coding to variable rate coding using a lagrangian formulation it is shown that if an asymptotically high rate optimal sequence of variable rate codes is designed for a k dimensional probability density function pdf g and then applied to another pdf f for which f g is bounded then the resulting mismatch or loss of performance from the optimal possible is given by the relative entropy or kullback leibler 1968 divergence i f spl par g it is also shown that under the same assumptions an asymptotically optimal code sequence for g can be converted to an asymptotically optimal code sequence for a mismatched source f by modifying only the lossless component of the code applications to quantizer design using uniform and gaussian densities are described including a high rate analog to the shannon rate distortion result of sakrison 1975 and lapidoth 1997 showing that the gaussian is the worst case for lossy compression of a source with known covariance by coupling the mismatch result with composite quantizers the worst case properties of uniform and gaussian densities are extended to conditionally uniform and gaussian densities which provides a lloyd clustering algorithm for fitting mixtures to general densities

an isogeometric reissner mindlin shell element based on bezier dual basis functions overcoming locking and improved coarse mesh accuracy abstract we develop a mixed geometrically nonlinear isogeometric reissner mindlin shell element for the analysis of thin walled structures that leverages bezier dual basis functions to address both shear and membrane locking and to improve the quality of computed stresses the accuracy of computed solutions over coarse meshes that have highly non interpolatory control meshes is achieved through the application of a continuous rotational approach the starting point of the formulation is the modified hellinger reissner variational principle with independent displacement membrane and shear strains as the unknown fields to overcome locking the strain variables are interpolated with lower order spline bases while the variations of the strain variables are interpolated with the corresponding bezier dual bases leveraging the orthogonality property of the bezier dual basis the strain variables are condensed out of the system with only a slight increase in the bandwidth of the resulting linear system the condensed approach preserves the accuracy of the non condensed mixed approach but with fewer degrees of freedom from a practical point of view since the bezier dual basis is completely specified through bezier extraction any spline space that admits bezier extraction can utilize the proposed approach directly

lqg control with an h sup infinity performance bound a riccati equation approach an lqg linear quadratic gaussian control design problem involving a constraint on h sup infinity disturbance attenuation is considered the h sup infinity performance constraint is embedded within the optimization process by replacing the covariance lyapunov equation by a riccati equation whose solution leads to an upper bound on l sub 2 performance in contrast to the pair of separated riccati equations of standard lqg theory the h sup infinity constrained gains are given by a coupled system of three modified riccati equations the coupling illustrates the breakdown of the separation principle for the h sup infinity constrained problem both full and reduced order design problems are considered with an h sup infinity attenuation constraint involving both state and control variables an algorithm is developed for the full order design problem and illustrative numerical results are given

an algebraic riccati equation approach to h optimization abstract this paper considers the general so called four block h optimal control problem with the assumption that system states are available for feedback it is shown that infimization of the h norm of the closed loop transfer function over all linear constant i e nondynamic stabilizing state feedback laws can be completely characterized via an algebraic riccati equation it is further shown that the optimal norm is not improved by allowing feedback to be dynamic thus the general state feedback h optimal control problem can be solved by iteratively solving one are and the controller can be chosen to be static gain

compressive sensing on manifolds using a nonparametric mixture of factor analyzers algorithm and performance bounds nonparametric bayesian methods are employed to constitute a mixture of low rank gaussians for data x rn that are of high dimension n but are constrained to reside in a low dimensional subregion of rn the number of mixture components and their rank are inferred automatically from the data the resulting algorithm can be used for learning manifolds and for reconstructing signals from manifolds based on compressive sensing cs projection measurements the statistical cs inversion is performed analytically we derive the required number of cs random measurements needed for successful reconstruction based on easily computed quantities drawing on block sparsity properties the proposed methodology is validated on several synthetic and real datasets

mixed and componentwise condition numbers of nonsymmetric algebraic riccati equation abstract in this paper the mixed and componentwise condition numbers of nonsymmetric algebraic riccati equation are considered the explicit expressions of two kinds of condition numbers are derived in the uniform manner and the upper bounds are proposed some simple numerical experiments show that the mixed and componentwise condition numbers can be sharp

an inverse free parallel spectral divide and conquer algorithm for nonsymmetric eigenproblems we discuss an inverse free highly parallel spectral divide and conquer algorithm it can compute either an invariant subspace of a nonsymmetric matrix a or a pair of left and right deflating subspaces of a regular matrix pencil a lambda b this algorithm is based on earlier ones of bulgakov godunov and malyshev but improves on them in several ways this algorithm only uses easily parallelizable linear algebra building blocks matrix multiplication and qr decomposition but not matrix inversion similar parallel algorithms for the nonsymmetric eigenproblem use the matrix sign function which requires matrix inversion and is faster but can be less stable than the new algorithm

mismatched estimation and relative entropy a random variable with distribution p is observed in gaussian noise and is estimated by a mismatched minimum mean square estimator that assumes that the distribution is q instead of p this paper shows that the integral over all signal to noise ratios snrs of the excess mean square estimation error incurred by the mismatched estimator is twice the relative entropy d p q in nats this representation of relative entropy can be generalized to nonreal valued random variables and can be particularized to give new general representations of mutual information in terms of conditional means inspired by the new representation we also propose a definition of free relative entropy which fills a gap in and is consistent with the literature on free probability

signal detection under mismatch corresp a binary detection problem of the neymann pearson type in which the probability density functions used are inaccurate versions of the true ones are considered the performance of the above suboptimal detection scheme as the number of observations increases is investigated a necessary and sufficient condition is given for the exponential convergence to zero of the two error probabilities as the number of observations increases the condition is in terms of an inequality between differences of asymptotic per sample informational divergence expressions

sensitivity to basis mismatch in compressed sensing the theory of compressed sensing suggests that successful inversion of an image of the physical world broadly defined to include speech signals radar sonar returns vibration records sensor array snapshot vectors 2 d images and so on for its source modes and amplitudes can be achieved at measurement dimensions far lower than what might be expected from the classical theories of spectrum or modal analysis provided that the image is sparse in an apriori known basis for imaging problems in spectrum analysis and passive and active radar sonar this basis is usually taken to be a dft basis however in reality no physical field is sparse in the dft basis or in any apriori known basis no matter how finely we grid the parameter space the sources may not lie in the center of the grid cells and consequently there is mismatch between the assumed and the actual bases for sparsity in this paper we study the sensitivity of compressed sensing to mismatch between the assumed and the actual sparsity bases we start by analyzing the effect of basis mismatch on the best k term approximation error which is central to providing exact sparse recovery guarantees we establish achievable bounds for the l1 error of the best k term approximation and show that these bounds grow linearly with the image or grid dimension and the mismatch level between the assumed and actual bases for sparsity we then derive bounds with similar growth behavior for the basis pursuit l1 recovery error indicating that the sparse recovery may suffer large errors in the presence of basis mismatch although we present our results in the context of basis pursuit our analysis applies to any sparse recovery principle that relies on the accuracy of best k term approximations for its performance guarantees we particularly highlight the problematic nature of basis mismatch in fourier imaging where spillage from off grid dft components turns a sparse representation into an incompressible one we substantiate our mathematical analysis by numerical examples that demonstrate a considerable performance degradation for image inversion from compressed sensing measurements in the presence of basis mismatch for problem sizes common to radar and sonar

mismatch in the classification of linear subspaces sufficient conditions for reliable classification this paper considers the classification of linear subspaces with mismatched classifiers in particular we assume a model where one observes signals in the presence of isotropic gaussian noise and the distribution of the signals conditioned on a given class is gaussian with a zero mean and a low rank covariance matrix we also assume that the classifier knows only a mismatched version of the parameters of input distribution in lieu of the true parameters by constructing an asymptotic low noise expansion of an upper bound to the error probability of such a mismatched classifier we provide sufficient conditions for reliable classification in the low noise regime that are able to sharply predict the absence of a classification error floor such conditions are a function of the geometry of the true signal distribution the geometry of the mismatched signal distributions as well as the interplay between such geometries namely the principal angles and the overlap between the true and the mismatched signal subspaces numerical results demonstrate that our conditions for reliable classification can sharply predict the behavior of a mismatched classifier both with synthetic data and in a motion segmentation and a hand written digit classification applications

efficient computation of the extreme solutions of x a x 1 a q and x a x 1 a q we propose a new quadratically convergent algorithm having a low computational cost per step and good numerical stability properties which allows the simultaneous approximation of the extreme solutions of the matrix equations x a x 1 a q and x a x 1 a q the algorithm is based on the cyclic reduction method

convergence rates of some iterative methods for nonsymmetric algebraic riccati equations arising in transport theory we determine and compare the convergence rates of various fixed point iterations for finding the minimal positive solution of a class of nonsymmetric algebraic riccati equations arising in transport theory

lambertian reflectance and linear subspaces we prove that the set of all lambertian reflectance functions the mapping from surface normals to intensities obtained with arbitrary distant light sources lies close to a 9d linear subspace this implies that in general the set of images of a convex lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low dimensional linear subspace explaining prior empirical results we also provide a simple analytic characterization of this linear space we obtain these results by representing lighting using spherical harmonics and describing the effects of lambertian materials as the analog of a convolution these results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce nonnegative lighting functions we also show a simple way to enforce nonnegative lighting when the images of an object lie near a 4d linear space we apply these algorithms to perform face recognition by finding the 3d model that best matches a 2d query image

low memory and low complexity iterative schemes for a nonsymmetric algebraic riccati equation arising from transport theory abstract we reconsider newton s method and two fixed point methods for finding the minimal positive solution of a nonsymmetric algebraic riccati equation arising from transport theory we rewrite the subproblem of the newton and fixed point iterative schemes into an equivalent form with some special structure by the use of the particular structure of the subproblem we then present low memory and low complexity versions of these iterative methods with a factored alternating direction implicit iteration some properties of eigenvalues for iterative coefficient matrices in solving the subproblem are derived and the convergence of the proposed methods is established numerical experiments show that the new iterative schemes are highly efficient to obtain the minimal positive solution the proposed low memory and low complexity newton s method is particularly efficient for solving large scale riccati equation arising from transport theory

modified alternately linearized implicit iteration method for m matrix algebraic riccati equations abstract research on the theories and efficient numerical methods of m matrix algebraic riccati equation mare has become a hot topic in recent years in this paper we consider numerical solution of m matrix algebraic riccati equation and propose a modified alternately linearized implicit iteration method mali for computing the minimal nonnegative solution of mare convergence of the mali method is proved by choosing proper parameters for the nonsingular m matrix or irreducible singular m matrix theoretical analysis and numerical experiments show that the mali method is effective and efficient in some cases

a structure preserving doubling algorithm for continuous time algebraic riccati equations abstract continuous time algebraic riccati equations cares can be transformed a la cayley to discrete time algebraic riccati equations dares the efficient structure preserving doubling algorithm sda for dares from e k w chu h y fan w w lin a structure preserving doubling algorithm for periodic discrete time algebraic riccati equations preprint 2002 28 ncts national tsing hua university hsinchu 300 taiwan 2003 e k w chu h y fan w w lin c s wang a structure preserving doubling algorithm for periodic discrete time algebraic riccati equations preprint 2002 18 ncts national tsing hua university hsinchu 300 taiwan 2003 can then be applied in this paper we develop the structure preserving doubling algorithm from a new point of view and show its quadratic convergence under assumptions which are weaker than stabilizability and detectability as well as practical issues involved in the application of the sda to cares a modified version of the sda developed for dares with a doubly symmetric structure is also presented extensive numerical results show that our approach is efficient and competitive

total linear least squares and the algebraic riccati equation abstract it is shown that the solution to a total linear least squares problem satisfies a quadratic matrix equation which turns into an algebraic riccati equation when the matrix of unknowns is square if there is an additional symmetry constraint on the solution the optimal solution is given by the anti stabilizing solution of this riccati equation

mismatched decoding error exponents second order rates and saddlepoint approximations this paper considers the problem of channel coding with a given possibly suboptimal maximum metric decoding rule a cost constrained random coding ensemble with multiple auxiliary costs is introduced and is shown to achieve error exponents and second order coding rates matching those of constant composition random coding while being directly applicable to channels with infinite or continuous alphabets the number of auxiliary costs required to match the error exponents and second order rates of constant composition coding is studied and is shown to be at most two for independent identically distributed random coding asymptotic estimates of two well known non asymptotic bounds are given using saddlepoint approximations each expression is shown to characterize the asymptotic behavior of the corresponding random coding bound at both fixed and varying rates thus unifying the regimes characterized by error exponents second order rates and moderate deviations for fixed rates novel exact asymptotics expressions are obtained to within a multiplicative 1 o 1 term using numerical examples it is shown that the saddlepoint approximations are highly accurate even at short block lengths

existence of algebraic matrix riccati equations arising in transport theory we consider the existence of positive solutions of a certain class of algebraic matrix riccati equations with two parameters c 0 c 1 and 0 1 here c denotes the fraction of scattering per collision and is an angular shift equations of this class are induced via invariant imbedding and the shifted gauss legendre quadrature formula from a simple transport model by establishing the existence of positive solutions of such equations the problem of the convergence of some iterative schemes for solving them can be completely solved

mismatched decoding revisited general alphabets channels with memory and the wide band limit the mismatch capacity of a channel is the highest rate at which reliable communication is possible over the channel with a given possibly suboptimal decoding rule this quantity has been studied extensively for single letter decoding rules over discrete memoryless channels dmcs here we extend the study to memoryless channels with general alphabets and to channels with memory with possibly non single letter decoding rules we also study the wide band limit and in particular the mismatch capacity per unit cost and the achievable rates on an additive noise spread spectrum system with single letter decoding and binary signaling

accurate solutions of m matrix sylvester equations this paper is concerned with a relative perturbation theory and its entrywise relatively accurate numerical solutions of an m matrix sylvester equation ax xb c by which we mean both a and b have positive diagonal entries and nonpositive off diagonal entries and p i m otimes a b rm t otimes i n is a nonsingular m matrix and c is entrywise nonnegative it is proved that small relative perturbations to the entries of a b and c introduce small relative errors to the entries of the solution x thus the smaller entries of x do not suffer bigger relative errors than its larger entries unlikely the existing perturbation theory for general sylvester equations we then discuss some minor but crucial implementation changes to three existing numerical methods so that they can be used to compute x as accurately as the input data deserve

a structure preserving doubling algorithm for nonsymmetric algebraic riccati equation in this paper we propose a structure preserving doubling algorithm sda for the computation of the minimal nonnegative solution to the nonsymmetric algebraic riccati equation nare based on the techniques developed for the symmetric cases this method allows the simultaneous approximation to the minimal nonnegative solutions of the nare and its dual equation requiring only the solutions to two linear systems and several matrix multiplications per iteration similar to newton s method and the fixed point iteration methods for solving nares we also establish global convergence for sda under suitable conditions using only elementary matrix theory we show that sequences of matrices generated by sda are monotonically increasing and quadratically convergent to the minimal nonnegative solutions of the nare and its dual equation numerical experiments show that the sda algorithm is feasible and effective and outperforms newton s iteration and the fixed point iteration methods

on achievable rates and error exponents for channels with mismatched decoding the problem of characterizing achievable rates and error exponents for discrete memoryless channels with mismatched decoding is addressed a mismatched cognitive multiple access channel is introduced and an inner bound on its capacity region is derived using two alternative encoding methods 1 superposition coding and 2 random binning the inner bounds are derived by analyzing the average error probability of the code ensemble for both methods and by a tight characterization of the resulting error exponents random coding converse theorems are also derived a comparison of the achievable regions shows that in the matched case random binning performs as well as superposition coding i e the region achievable by random binning is equal to the capacity region the achievability results are further specialized to obtain a lower bound on the mismatch capacity of the single user channel by investigating a cognitive multiple access channel whose achievable sum rate serves as a lower bound on the single user channel s capacity while the achievable rate presented here may not improve the rate achieved by lapidoth s scheme in optimizing over the parameters of the random coding scheme it can improve the achieved rate for given parameters and thereby may reduce the computational complexity required to find a good code

shape and motion from image streams under orthography a factorization method inferring scene geometry and camera motion from a stream of images is possible in principle but is an ill conditioned problem when the objects are distant with respect to their size we have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step an image stream can be represented by the 2fxp measurement matrix of the image coordinates of p points tracked through f frames we show that under orthographic projection this matrix is of rank 3 based on this observation the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively two of the three translation components are computed in a preprocessing stage the method can also handle and obtain a full solution from a partially filled in measurement matrix that may result from occlusions or tracking failures the method gives accurate results and does not introduce smoothing in either shape or motion we demonstrate this with a series of experiments on laboratory and outdoor image streams with and without occlusions

acquiring linear subspaces for face recognition under variable lighting previous work has demonstrated that the image variation of many objects human faces in particular under variable lighting can be effectively modeled by low dimensional linear spaces even when there are multiple light sources and shadowing basis images spanning this space are usually obtained in one of three ways a large set of images of the object under different lighting conditions is acquired and principal component analysis pca is used to estimate a subspace alternatively synthetic images are rendered from a 3d model perhaps reconstructed from images under point sources and again pca is used to estimate a subspace finally images rendered from a 3d model under diffuse lighting based on spherical harmonics are directly used as basis images in this paper we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis vectors of a low dimensional linear space and that this subspace is close to those acquired by the other methods more specifically there exist configurations of k point light source directions with k typically ranging from 5 to 9 such that by taking k images of an object under these single sources the resulting subspace is an effective representation for recognition under a wide range of lighting conditions since the subspace is generated directly from real images potentially complex and or brittle intermediate steps such as 3d reconstruction can be completely avoided nor is it necessary to acquire large numbers of training images or to physically construct complex diffuse harmonic light fields we validate the use of subspaces constructed in this fashion within the context of face recognition

sparse subspace clustering algorithm theory and applications many real world problems deal with collections of high dimensional data such as images videos text and web documents dna microarray data and more often such high dimensional data lie close to low dimensional structures corresponding to several classes or categories to which the data belong in this paper we propose and study an algorithm called sparse subspace clustering to cluster data points that lie in a union of low dimensional subspaces the key idea is that among the infinitely many possible representations of a data point in terms of other points a sparse representation corresponds to selecting a few points from the same subspace this motivates solving a sparse optimization program whose solution is used in a spectral clustering framework to infer the clustering of the data into subspaces since solving the sparse optimization program is in general np hard we consider a convex relaxation and show that under appropriate conditions on the arrangement of the subspaces and the distribution of the data the proposed minimization program succeeds in recovering the desired sparse representations the proposed algorithm is efficient and can handle data points near the intersections of subspaces another key advantage of the proposed algorithm with respect to the state of the art is that it can deal directly with data nuisances such as noise sparse outlying entries and missing entries by incorporating the model of the data into the sparse optimization program we demonstrate the effectiveness of the proposed algorithm through experiments on synthetic data as well as the two real world problems of motion segmentation and face clustering

an algorithm for the computation of the structured complex stability radius abstract perturbed state space models of the form x a bdc x are considered where a b c are given matrices a is stable and d is an unknown complex perturbation matrix a reliable and efficient algorithm for the computation of the norm of the minimal norm destabilizing matrix d is presented the performance of the algorithm and possible applications are illustrated

a generalized structure preserving doubling algorithm for generalized discrete time algebraic riccati equations in chu et al 2004 an efficient structure preserving doubling algorithm sda was proposed for the solution of discrete time algebraic riccati equations dares in this paper we generalize the sda to the g sda for the generalized dare e t xe a t xa a t xb c ts r b t xb 1 b t xa s tc c t qc using cayley transformation twice we transform the generalized dare to a dare in a standard symplectic form without any explicit inversions of possibly ill conditioned r and e the sda can then be applied selected numerical examples illustrate that the g sda is efficient out performing other algorithms

solving large scale nonlinear matrix equations by doubling we consider the solution of the large scale nonlinear matrix equation x bx 1a q 0 with a b q x cn n and in some applications b a or h the matrix q is assumed to be nonsingular and sparse with its structure allowing the solution of the corresponding linear system qv r in o n computational complexity furthermore b and a are respectively of ranks ra rb n the type 2 structure preserving doubling algorithm by lin and xu 2006 24 is adapted with the appropriate applications of the sherman morrison woodbury formula and the low rank updates of various iterates two resulting large scale doubling algorithms have an o ra rb 3 computational complexity per iteration after some pre processing of data in o n computational complexity and memory requirement and converge quadratically these are illustrated by the numerical examples

disturbance attenuation and h optimization a design method based on the algebraic riccati equation this note presents a method for designing a state feedback control law to reduce the effect of disturbances on the output of a given linear system the problem under consideration involves attenuating the disturbances to a prespecified level the construction of the state feedback control law requires the solution of a certain algebraic riccati equation by applying the procedure with successively smaller values of the prespecifled disturbance level the result also provides an approach to the h infty optimization problem for the state feedback case

a class of iterative methods for solving nonsymmetric algebraic riccati equations arising in transport theory in this paper we consider the nonsymmetric algebraic riccati equation arising in transport theory an important feature of this equation is that its minimal positive solution can be obtained via computing the minimal positive solution of a vector equation we propose a class of iterative methods to solve the vector equation the convergence analysis shows that the sequence of vectors generated by iterative methods with two kinds of specific iterative matrices is monotonically increasing and converges to the minimal positive solution of the vector equation numerical experiments show that the new methods outperform the modified simple iterative method and newton s method

structure preserving algorithms for periodic discrete time algebraic riccati equations in this paper we investigate structure preserving algorithms for computing the symmetric positive semi definite solutions to the periodic discrete time algebraic riccati equations p dares using a structure preserving swap and collapse procedure a single symplectic matrix pair in standard symplectic form is obtained the p dares can then be solved via a single dare using a structure preserving doubling algorithm we develop the structure preserving doubling algorithm from a new point of view and show its quadratic convergence under assumptions which are weaker than stabilizability and detectability with several numerical results the algorithm is shown to be efficient out performing other algorithms on a large set of benchmark problems