iterative reweighted algorithms for matrix rank minimization the problem of minimizing the rank of a matrix subject to affine constraints has applications in several areas including machine learning and is known to be np hard a tractable relaxation for this problem is nuclear norm or trace norm minimization which is guaranteed to find the minimum rank matrix under suitable assumptions in this paper we propose a family of iterative reweighted least squares algorithms irls p with 0 p 1 as a computationally efficient way to improve over the performance of nuclear norm minimization the algorithms can be viewed as locally minimizing certain smooth approximations to the rank function when p 1 we give theoretical guarantees similar to those for nuclear norm minimization that is recovery of low rank matrices under certain assumptions on the operator defining the constraints for p 1 irls p shows better empirical performance in terms of recovering low rank matrices than nuclear norm minimization we provide an efficient implementation for irls p and also present a related family of algorithms sirls p these algorithms exhibit competitive run times and improved recovery when compared to existing algorithms for random instances of the matrix completion problem as well as on the movielens movie recommendation data set

temporal second order difference schemes for the multi dimensional variable order time fractional sub diffusion equations abstract a special point on each time interval is found for the approximation of the variable order time caputo derivative which makes at least second order approximation accuracy be obtained on this basis two difference schemes are proposed for the multi dimensional variable order time fractional sub diffusion equations which have second order accuracy in time second order and fourth order accuracy in space respectively the obtained difference schemes are proved to be uniquely solvable the convergence and stability of the schemes in the discrete h 1 norm are analyzed by utilizing the energy method some numerical examples are presented to verify the theoretical results

a cubature h filter and its square root version in this paper we present a nonlinear state estimation algorithm based on the fusion of an extended h eh f and a cubature kalman filter ckf the resulting estimator is called a cubature h filter the recently developed ckf is a gaussian approximation of a bayesian filter and its performance over non gaussian noises may degrade in contrast the h filter is capable of estimating the states of linear systems with non gaussian noises and the extended h filter eh f can estimate the states of non linear and non gaussian systems similar to the h filter an eh f also does not make any assumptions about the statistics of the process or measurement noise but it does require jacobians during the state estimation of nonlinear systems which degrade the overall performance when the nonlinearities are severe the cubature h filter is developed to have the desirable features of both ckf and eh f for numerical accuracy a square root version of the cubature h filter is developed using j unitary transform

use of the kalman filter for inference in state space models with unknown noise distributions the kalman filter is frequently used for state estimation in state space models when the standard gaussian noise assumption does not apply a problem arises however in that inference based on the incorrect gaussian assumption can lead to misleading or erroneous conclusions about the relationship of the kalman filter estimate to the true unknown state this note shows how inequalities from probability theory associated with the probabilities of convex sets have potential for characterizing the estimation error of a kalman filter in such a non gaussian distribution free setting

a numerical integration perspective on gaussian filters this paper proposes a numerical integration perspective on the gaussian filters a gaussian filter is approximation of the bayesian inference with the gaussian posterior probability density assumption being valid there exists a variation of gaussian filters in the literature that derived themselves from very different backgrounds from the numerical integration viewpoint various versions of gaussian filters are only distinctive from each other in their specific treatments of approximating the multiple statistical integrations a common base is provided for the first time to analyze and compare gaussian filters with respect to accuracy efficiency and stability factor this study is expected to facilitate the selection of appropriate gaussian filters in practice and to help design more efficient filters by employing better numerical integration methods

unscented kalman filter with advanced adaptation of scaling parameter the paper deals with state estimation of the nonlinear stochastic systems by means of the unscented kalman filter with a focus on specification of the points their position is influenced by two design parameters the scaling parameter determining the spread of the points and a covariance matrix decomposition determining rotation of the points in this paper a choice of the scaling parameter is analyzed it is shown that considering other values than the standard choice may lead to increased quality of the estimate especially if the scaling parameter is adapted several different criteria for the adaptation are proposed and techniques to reduce computational costs of the adaptation are developed the proposed algorithm of the unscented kalman filter with advanced adaptation of the scaling parameter is illustrated in a numerical example

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

nonconvex nonsmooth low rank minimization via iteratively reweighted nuclear norm the nuclear norm is widely used as a convex surrogate of the rank function in compressive sensing for low rank matrix recovery with its applications in image recovery and signal processing however solving the nuclear norm based relaxed convex problem usually leads to a suboptimal solution of the original rank minimization problem in this paper we propose to use a family of nonconvex surrogates of l 0 norm on the singular values of a matrix to approximate the rank function this leads to a nonconvex nonsmooth minimization problem then we propose to solve the problem by an iteratively reweighted nuclear norm irnn algorithm irnn iteratively solves a weighted singular value thresholding problem which has a closed form solution due to the special properties of the nonconvex surrogate functions we also extend irnn to solve the nonconvex problem with two or more blocks of variables in theory we prove that the irnn decreases the objective function value monotonically and any limit point is a stationary point extensive experiments on both synthesized data and real images demonstrate that irnn enhances the low rank matrix recovery compared with the state of the art convex algorithms

gaussian filters for nonlinear filtering problems we develop and analyze real time and accurate filters for nonlinear filtering problems based on the gaussian distributions we present the systematic formulation of gaussian filters and develop efficient and accurate numerical integration of the optimal filter we also discuss the mixed gaussian filters in which the conditional probability density is approximated by the sum of gaussian distributions a new update rule of weights for gaussian sum filters is proposed our numerical tests demonstrate that new filters significantly improve the extended kalman filter with no additional cost and the new gaussian sum filter has a nearly optimal performance

nonlinear filtering revisited a spectral approach the objective of this paper is to develop an approach to nonlinear filtering based on the cameron martin version of wiener chaos expansion this approach gives rise to a new numerical scheme for nonlinear filtering the main feature of this algorithm is that it allows one to separate the computations involving the observations from those dealing only with the system parameters and to shift the latter off line

reduced rank regression via adaptive nuclear norm penalization we propose an adaptive nuclear norm penalization approach for low rank matrix approximation and use it to develop a new reduced rank estimation method for high dimensional multivariate regression the adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix and it is generally non convex under the natural restriction that the weight decreases with the singular value however we show that the proposed non convex penalized regression method has a global optimal solution obtained from an adaptively soft thresholded singular value decomposition the method is computationally efficient and the resulting solution path is continuous the rank consistency of and prediction estimation performance bounds for the estimator are established for a high dimensional asymptotic regime simulation studies and an application in genetics demonstrate its efficacy

gaussian map filtering using kalman optimization this paper deals with the update step of gaussian map filtering in this framework we seek a gaussian approximation to the posterior probability density function pdf whose mean is given by the maximum a posteriori map estimator we propose two novel optimization algorithms which are quite suitable for finding the map estimate although they can also be used to solve general optimization problems these are based on the design of a sequence of pdfs that become increasingly concentrated around the map estimate the resulting algorithms are referred to as kalman optimization ko methods we also provide the important relations between these ko methods and their conventional optimization algorithms coas counterparts i e newton s and levenberg marquardt algorithms our simulations indicate that ko methods are more robust than their coa equivalents

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

fourier hermite kalman filter in this note we shall present a new class of gaussian filters called fourier hermite kalman filters fourier hermite kalman filters are based on expansion of nonlinear functions with the fourier hermite series in same way as the traditional extended kalman filter is based on the taylor series the first order truncation of the fourier hermite series gives the previously known statistically linearized filter

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

robust recovery of subspace structures by low rank representation in this paper we address the subspace clustering problem given a set of data samples vectors approximately drawn from a union of multiple subspaces our goal is to cluster the samples into their respective subspaces and remove possible outliers as well to this end we propose a novel objective function named low rank representation lrr which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary it is shown that the convex program associated with lrr solves the subspace clustering problem in the following sense when the data is clean we prove that lrr exactly recovers the true subspace structures when the data are contaminated by outliers we prove that under certain conditions lrr can exactly recover the row space of the original data and detect the outlier as well for data corrupted by arbitrary sparse errors lrr can also approximately recover the row space with theoretical guarantees since the subspace membership is provably determined by the row space these further imply that lrr can perform robust subspace clustering and error correction in an efficient and effective way

moment estimation using a marginalized transform we present a method for estimating mean and covariance of a transformed gaussian random variable the method is based on evaluations of the transforming function and resembles the unscented transform and gauss hermite integration in that respect the information provided by the evaluations is used in a bayesian framework to form a posterior description of the parameters in a model of the transforming function estimates are then derived by marginalizing these parameters from the analytical expression of the mean and covariance an estimation algorithm based on the assumption that the transforming function can be described using hermite polynomials is presented and applied to the non linear filtering problem the resulting marginalized transform mt estimator is compared to the cubature rule the unscented transform and the divided difference estimator the evaluations show that the presented method performs better than these methods more specifically in estimating the covariance matrix contrary to the unscented transform the resulting approximation of the covariance matrix is guaranteed to be positive semidefinite

comment on a new method for the nonlinear transformation of means and covariances in filters and estimators with authors reply the above paper julier et al ieee trans automat contr vol 45 pp 477 82 2000 generalizes the kalman filter to nonlinear systems by transforming approximations of the probability distributions through the nonlinear process and measurement functions this comment derives exactly the same estimator by linearizing the process and measurement functions by a statistical linear regression through some regression points in contrast with the extended kalman filter which uses an analytic linearization in one point this insight allows one 1 to understand predict the performance of the estimator for specific applications and 2 to make adaptations to the estimator i e the choice of the regression points and their weights in those cases where the original formulation does not assure good results in reply the authors state that the commenters conclusion is unnecessarily narrow interpretation of results

fast and accurate matrix completion via truncated nuclear norm regularization recovering a large matrix from a small subset of its entries is a challenging problem arising in many real applications such as image inpainting and recommender systems many existing approaches formulate this problem as a general low rank matrix approximation problem since the rank operator is nonconvex and discontinuous most of the recent theoretical studies use the nuclear norm as a convex relaxation one major limitation of the existing approaches based on nuclear norm minimization is that all the singular values are simultaneously minimized and thus the rank may not be well approximated in practice in this paper we propose to achieve a better approximation to the rank of matrix by truncated nuclear norm which is given by the nuclear norm subtracted by the sum of the largest few singular values in addition we develop a novel matrix completion algorithm by minimizing the truncated nuclear norm we further develop three efficient iterative procedures tnnr admm tnnr apgl and tnnr admmap to solve the optimization problem tnnr admm utilizes the alternating direction method of multipliers admm while tnnr agpl applies the accelerated proximal gradient line search method apgl for the final optimization for tnnr admmap we make use of an adaptive penalty according to a novel update rule for admm to achieve a faster convergence rate our empirical study shows encouraging results of the proposed algorithms in comparison to the state of the art matrix completion algorithms on both synthetic and real visual datasets

a new method for the nonlinear transformation of means and covariances in filters and estimators this paper describes a new approach for generalizing the kalman filter to nonlinear systems a set of samples are used to parametrize the mean and covariance of a not necessarily gaussian probability distribution the method yields a filter that is more accurate than an extended kalman filter ekf and easier to implement than an ekf or a gauss second order filter its effectiveness is demonstrated using an example

nonlinear filtering via generalized edgeworth series and gauss hermite quadrature in this correspondence an approximate nonlinear filter is presented for systems with continuous time dynamics and discrete time measurements the filter is based on a combination of generalized edgeworth series ges expansion of probability density functions and gauss hermite quadrature ghq application to a passive tracking problem is also presented

cubature kalman filters in this paper we present a new nonlinear filter for high dimensional state estimation which we have named the cubature kalman filter ckf the heart of the ckf is a spherical radial cubature rule which makes it possible to numerically compute multivariate moment integrals encountered in the nonlinear bayesian filter specifically we derive a third degree spherical radial cubature rule that provides a set of cubature points scaling linearly with the state vector dimension the ckf may therefore provide a systematic solution for high dimensional nonlinear filtering problems the paper also includes the derivation of a square root version of the ckf for improved numerical stability the ckf is tested experimentally in two nonlinear state estimation problems in the first problem the proposed cubature rule is used to compute the second order statistics of a nonlinearly transformed gaussian random variable the second problem addresses the use of the ckf for tracking a maneuvering aircraft the results of both experiments demonstrate the improved performance of the ckf over conventional nonlinear filters

image super resolution via sparse representation this paper presents a new approach to single image superresolution based upon sparse signal representation research on image statistics suggests that image patches can be well represented as a sparse linear combination of elements from an appropriately chosen over complete dictionary inspired by this observation we seek a sparse representation for each patch of the low resolution input and then use the coefficients of this representation to generate the high resolution output theoretical results from compressed sensing suggest that under mild conditions the sparse representation can be correctly recovered from the downsampled signals by jointly training two dictionaries for the low and high resolution image patches we can enforce the similarity of sparse representations between the low resolution and high resolution image patch pair with respect to their own dictionaries therefore the sparse representation of a low resolution image patch can be applied with the high resolution image patch dictionary to generate a high resolution image patch the learned dictionary pair is a more compact representation of the patch pairs compared to previous approaches which simply sample a large amount of image patch pairs reducing the computational cost substantially the effectiveness of such a sparsity prior is demonstrated for both general image super resolution sr and the special case of face hallucination in both cases our algorithm generates high resolution images that are competitive or even superior in quality to images produced by other similar sr methods in addition the local sparse modeling of our approach is naturally robust to noise and therefore the proposed algorithm can handle sr with noisy inputs in a more unified framework