application of the karhunen loeve procedure for the characterization of human faces the use of natural symmetries mirror images in a well defined family of patterns human faces is discussed within the framework of the karhunen loeve expansion this results in an extension of the data and imposes even and odd symmetry on the eigenfunctions of the covariance matrix without increasing the complexity of the calculation the resulting approximation of faces projected from outside of the data set onto this optimal basis is improved on average

stochastic linear quadratic optimal control with constraint for discrete time systems in this paper we consider linear quadratic optimal control with constraint for discrete time stochastic systems with state and disturbance dependent noise with the aid of the lagrange multiplier theorem we present a necessary condition under which the problem is well posed and a state feedback solution can be derived moreover a sufficient condition is introduced for the case in which the quadratic term matrices are non negative in a way the previous results on stochastic linear quadratic optimal control without constraint can be regarded as corollaries of the theorems of this paper

generalized principal component analysis projection of saturated model parameters principal component analysis pca is very useful for a wide variety of data analysis tasks but its implicit connection to the gaussian distribution can be undesirable for discrete data such as bi

image outlier detection and feature extraction via l1 norm based 2d probabilistic pca this paper introduces an l1 norm based probabilistic principal component analysis model on 2d data l1 2dppca based on the assumption of the laplacian noise model the laplacian or l1 density function can be expressed as a superposition of an infinite number of gaussian distributions under this expression a bayesian inference can be established based on the variational expectation maximization approach all the key parameters in the probabilistic model can be learned by the proposed variational algorithm it has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers experiments on some publicly available databases show that the performance of l1 2dppca has largely been improved after identifying and removing sample outliers resulting in more accurate image reconstruction than the existing pca based methods the performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy

regularized coplanar discriminant analysis for dimensionality reduction abstract the dimensionality reduction methods based on linear embedding such as neighborhood preserving embedding npe sparsity preserving projections spp and collaborative representation based projections crp try to preserve a certain kind of linear representation for each sample after projection however in the transformed low dimensional space the linear relationship between the samples may be changed which cannot make the linear representation based classifiers such as sparse representation based classifier src to achieve higher recognition accuracy in this paper we propose a new linear dimensionality reduction algorithm called regularized coplanar discriminant analysis rcda to address this problem it simultaneously seeks a linear projection matrix and some linear representation coefficients that make the samples from the same class coplanar and the samples from different classes not coplanar the proposed regularization term balances the bias from the optimal linear representation and that from the class mean to avoid overfitting the training data and overcomes the matrix singularity in solving the linear representation coefficients an alternative optimization approach is proposed to solve the rcda model experiments are done on several benchmark face databases and hyperspectral image databases and results show that rcda can obtain better performance than other dimensionality reduction methods

piecewise linear skeletonization using principal curves proposes an algorithm to find piecewise linear skeletons of handwritten characters by using principal curves the development of the method was inspired by the apparent similarity between the definition of principal curves smooth curves which pass through the middle of a cloud of points and medial axes smooth curves that run equidistantly from the contours of a character image the central fitting and smoothing step of the algorithm is an extension of the polygonal line algorithm which approximates principal curves of data sets by piecewise linear curves the polygonal line algorithm is extended to find principal graphs and complemented with two steps specific to the task of skeletonization an initialization method to capture the approximate topology of the character and a collection of restructuring operations to improve the structural quality of the skeleton produced by the initialization method an advantage of our approach over existing methods is that we optimize the skeleton graph by minimizing an intuitive and explicit objective function that captures the two competing criteria of smoothing the skeleton and fitting it closely to the pixels of the character image we tested the algorithm on isolated handwritten digits and images of continuous handwriting the results indicated that the proposed algorithm can find a smooth medial axis in the great majority of a wide variety of character templates and that it substantially improves the pixel wise skeleton obtained by traditional thinning methods

a direct formulation for sparse pca using semidefinite programming given a covariance matrix we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination this problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis pca and has wide applications ranging from biology to finance we use a modification of the classical variational representation of the largest eigenvalue of a symmetric matrix where cardinality is constrained and derive a semidefinite programming based relaxation for our problem we also discuss nesterov s smooth minimization technique applied to the semidefinite program arising in the semidefinite relaxation of the sparse pca problem the method has complexity o n 4 sqrt log n epsilon where n is the size of the underlying covariance matrix and epsilon is the desired absolute accuracy on the optimal value of the problem

maximal likelihood correspondence estimation for face recognition across pose due to the misalignment of image features the performance of many conventional face recognition methods degrades considerably in across pose scenario to address this problem many image matching based methods are proposed to estimate semantic correspondence between faces in different poses in this paper we aim to solve two critical problems in previous image matching based correspondence learning methods 1 fail to fully exploit face specific structure information in correspondence estimation and 2 fail to learn personalized correspondence for each probe image to this end we first build a model termed as morphable displacement field mdf to encode face specific structure information of semantic correspondence from a set of real samples of correspondences calculated from 3d face models then we propose a maximal likelihood correspondence estimation mlce method to learn personalized correspondence based on maximal likelihood frontal face assumption after obtaining the semantic correspondence encoded in the learned displacement we can synthesize virtual frontal images of the profile faces for subsequent recognition using linear discriminant analysis method with pixel intensity features state of the art performance is achieved on three multipose benchmarks i e cmu pie feret and multipie databases owe to the rational mdf regularization and the usage of novel maximal likelihood objective the proposed mlce method can reliably learn correspondence between faces in different poses even in complex wild environment i e labeled face in the wild database

technical communique discrete time optimal control with control dependent noise and generalized riccati difference equations the optimal control law is derived for discrete time linear stochastic systems with quadratic performance criterion and control dependent noise the analysis includes the study of a generalized riccati difference equation and of the asymptotic behavior of its solutions

non stationary random vibration analysis of structures under multiple correlated normal random excitations abstract an algorithm that integrates karhunen loeve expansion kle and the finite element method fem is proposed to perform non stationary random vibration analysis of structures under excitations represented by multiple random processes that are correlated in both time and spatial domains in kle the auto covariance functions of random excitations are discretized using orthogonal basis functions the kle for multiple correlated random excitations relies on expansions in terms of correlated sets of random variables reflecting the cross covariance of the random processes during the response calculations the eigenfunctions of kle used to represent excitations are applied as forcing functions to the structure the proposed algorithm is applied to a 2dof system a 2d cantilever beam and a 3d aircraft wing under both stationary and non stationary correlated random excitations two methods are adopted to obtain the structural responses a the modal method and b the direct method both the methods provide the statistics of the dynamic response with sufficient accuracy the structural responses under the same type of correlated random excitations are bounded by the response obtained by perfectly correlated and uncorrelated random excitations the structural response increases with a decrease in the correlation length and with an increase in the correlation magnitude the proposed methodology can be applied for the analysis of any complex structure under any type of random excitation

on the oscillation of hill s equations under periodic forcing abstract we use the floquet theory of the hill s equation to prove the conjecture that all solutions of the second order forced linear differential equation y c sin t y cos t are oscillatory on 0 for all c 0

a flexible terminal approach to stochastic stability and stabilization of continuous time semi markovian jump systems with time varying delay abstract this paper addresses the stochastic stability and stabilization problems for a class of semi markovian jump systems smjss with time varying delay where the time varying delay t is assumed to satisfy 1 t 2 based on the flexible terminal approach the time varying delay t is first transformed such that 1 t t 2 t by utilizing a novel semi markovian lyapunov krasoviskii functional smlkf and an improved reciprocally convex inequality rci sufficient conditions are established to guarantee a feasible solution two illustrated examples are shown the effectiveness of the main results

nonoscillation of mathieu equations with two frequencies abstract as is well known mathieu s equation is a representative of mathematical models describing parametric excitation phenomena this paper deals with the oscillation problem for mathieu s equation with two frequencies the ratio of these two frequencies is not necessarily a rational number when the ratio is an irrational number the coefficient of mathieu s equation is quasi periodic but not periodic for this reason the basic knowledge for linear periodic systems such as floquet theory is not useful whether all solutions of mathieu s equation oscillate or not is determined by parameters and frequencies our results provide parametric conditions to guarantee that all solutions are nonoscillatory the advantage of the obtained parametric conditions is that it can be easily checked parametric nonoscillation region is drawn to understand these results easily finally several simulations are carried out to clarify the remaining problems

linear matrix inequalities riccati equations and indefinite stochastic linear quadratic controls this paper deals with an optimal stochastic linear quadratic lq control problem in infinite time horizon where the diffusion term in dynamics depends on both the state and the control variables in contrast to the deterministic case we allow the control and state weighting matrices in the cost functional to be indefinite this leads to an indefinite lq problem which may still be well posed due to the deep nature of uncertainty involved the problem gives rise to a stochastic algebraic riccati equation sare which is however fundamentally different from the classical algebraic riccati equation as a result of the indefinite nature of the lq problem to analyze the sare we introduce linear matrix inequalities lmis whose feasibility is shown to be equivalent to the solvability of the sare moreover we develop a computational approach to the sare via a semi definite programming associated with the lmis finally numerical experiments are reported to illustrate the proposed approach

a constrained em algorithm for independent component analysis we introduce a novel way of performing independent component analysis using a constrained version of the expectation maximization em algorithm the source distributions are modeled as d one dimensional mixtures of gaussians the observed data are modeled as linear mixtures of the sources with additive isotropic noise this generative model is fit to the data using constrained em the simpler soft switching approach is introduced which uses only one parameter to decide on the sub or supergaussian nature of the sources we explain how our approach relates to independent factor analysis

integral inequalities and second order linear oscillation on the interval 0 a u 0 below by a solution we mean a non trivial solution a solution is oscillatory if it has an infinite number of zeros numerous oscillation criteria are known and diversified approaches have proved fruitful among these is the use of the associated riccati equation although the theory of differential and integral inequalities has been used by some investigators see e g hartman 4 71 the authors believe that it has not been fully exploited the non oscillation of 1 1 is equivalent to the existence of a solution of the associated riccati equation on a half line thus one technique of proving oscillation is to show that any solution of the riccati equation diverges to i co at a finite point the theory of differential and integral inequalities comes in handy in the estimation of the growth of such a solution below we describe some known results that are relevant to results established in this paper although these are stated here for the case i co in the rest of the paper we consider both cases a 0 and a 00 it is well known that if q is large in the mean then 1 1 is oscillatory a typical example of such a criterion is due to fite leighton and wintner if

reconstructing bifurcation diagrams from noisy time series using nonlinear autoregressive models we introduce a formalism for the reconstruction of bifurcation diagrams from noisy time series the method consists in finding a parametrized predictor function whose bifurcation structure is similar to that of the given system the reconstruction algorithm is composed of two stages model selection and bifurcation parameter identification in the first stage an appropriate model that best represents all the given time series is selected a nonlinear autoregressive model with polynomial terms is employed in this study the identification of the bifurcation parameters from among the many model parameters is done in the second stage the algorithm works well even for a limited number of time series

generalization performance of regularization networks and support vector machines via entropy numbers of compact operators we derive new bounds for the generalization error of kernel machines such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers the proofs make use of a viewpoint that is apparently novel in the field of statistical learning theory the hypothesis class is described in terms of a linear operator mapping from a possibly infinite dimensional unit ball in feature space into a finite dimensional space the covering numbers of the class are then determined via the entropy numbers of the operator these numbers which characterize the degree of compactness of the operator can be bounded in terms of the eigenvalues of an integral operator induced by the kernel function used by the machine as a consequence we are able to theoretically explain the effect of the choice of kernel function on the generalization performance of support vector machines

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

sparse exponential family principal component analysis we propose a sparse exponential family principal component analysis sepca method suitable for any type of data following exponential family distributions to achieve simultaneous dimension reduction and variable selection for better interpretation of the results because of the generality of exponential family distributions the method can be applied to a wide range of applications in particular when analyzing high dimensional next generation sequencing data and genetic mutation data in genomics the use of sparsity inducing penalty helps produce sparse principal component loading vectors such that the principal components can focus on informative variables by using an equivalent dual form of the formulated optimization problem for sepca we derive optimal solutions with efficient iterative closed form updating rules the results from both simulation experiments and real world applications have demonstrated the superiority of our sepca in reconstruction accuracy and computational efficiency over traditional exponential family pca epca the existing sparse pca spca and sparse logistic pca slpca algorithms highlightsa sparse exponential family principal component analysis is proposed it has wide applications to any data following exponential family distributions efficient solutions can be achieved with closed form update rules the performance is enhanced with appropriate assumption of the data distribution this model is flexible and highly extensible

products of gaussians and probabilistic minor component analysis recently hinton introduced the products of experts architecture for density estimation where individual expert probabilities are multiplied and renormalized we consider products of gaussian pancakes equally elongated in all directions except one and prove that the maximum likelihood solution for the model gives rise to a minor component analysis solution we also discuss the covariance structure of sums and products of gaussian pancakes or one factor probabilistic principal component analysis models

danco an intrinsic dimensionality estimator exploiting angle and norm concentration abstract in the past decade the development of automatic intrinsic dimensionality estimators has gained considerable attention due to its relevance in several application fields however most of the proposed solutions prove to be not robust on noisy datasets and provide unreliable results when the intrinsic dimensionality of the input dataset is high and the manifold where the points are assumed to lie is nonlinearly embedded in a higher dimensional space in this paper we propose a novel intrinsic dimensionality estimator danco and its faster variant fastdanco which exploit the information conveyed both by the normalized nearest neighbor distances and by the angles computed on couples of neighboring points the effectiveness and robustness of the proposed algorithms are assessed by experiments on synthetic and real datasets by the comparative evaluation with state of the art methodologies and by significance tests

bayes optimality motivated linear and multilayered perceptron based dimensionality reduction dimensionality reduction is the process of mapping high dimension patterns to a lower dimension subspace when done prior to classification estimates obtained in the lower dimension subspace are more reliable for some classifiers there is also an improvement in performance due to the removal of the diluting effect of redundant information a majority of the present approaches to dimensionality reduction are based on scatter matrices or other statistics of the data which do not directly correlate to classification accuracy the optimality criteria of choice for the purposes of classification is the bayes error usually however bayes error is difficult to express analytically we propose an optimality criteria based on an approximation of the bayes error and use it to formulate a linear and a nonlinear method of dimensionality reduction the nonlinear method we propose relies on using a multilayered perceptron which produces as output the lower dimensional representation it thus differs from autoassociative like multilayered perceptrons which have been proposed and used for dimensionality reduction our results show that the nonlinear method is as anticipated superior to the linear method in that it can perform unfolding of a nonlinear manifold in addition the nonlinear method we propose provides substantially better lower dimension representation for classification purposes than fisher s linear discriminant fld and two other nonlinear methods of dimensionality reduction that are often used

anomaly preserving ell _ scriptscriptstyle 2 infty optimal dimensionality reduction over a grassmann manifold in this paper we address the problem of redundancy reduction of high dimensional noisy signals that may contain anomaly rare vectors which we wish to preserve since anomaly data vectors contribute weakly to the l2 norm of the signal as compared to the noise l2 based criteria are unsatisfactory for obtaining a good representation of these vectors as a remedy a new approach named min max svd mx svd was recently proposed for signal subspace estimation by attempting to minimize the maximum of data residual l2 norms denoted as l2 l and designed to represent well both abundant and anomaly measurements however the mx svd algorithm is greedy and only approximately minimizes the proposed l2 l norm of the residuals in this paper we develop an optimal algorithm for the minization of the l2 l norm of data misrepresentation residuals which we call maximum orthogonal complements optimal subspace estimation moose the optimization is performed via a natural conjugate gradient learning approach carried out on the set of n dimensional subspaces in irm m n which is a grassmann manifold the results of applying moose mx svd and l2 based approaches are demonstrated both on simulated and real hyperspectral data

note on wong s paper using the method of interval integral average we establish some new oscillation criteria which improve the oscillation criteria due to james s w wong

a least squares framework for component analysis over the last century component analysis ca methods such as principal component analysis pca linear discriminant analysis lda canonical correlation analysis cca locality preserving projections lpp and spectral clustering sc have been extensively used as a feature extraction step for modeling classification visualization and clustering ca techniques are appealing because many can be formulated as eigen problems offering great potential for learning linear and nonlinear representations of data in closed form however the eigen formulation often conceals important analytic and computational drawbacks of ca techniques such as solving generalized eigen problems with rank deficient matrices e g small sample size problem lacking intuitive interpretation of normalization factors and understanding commonalities and differences between ca methods this paper proposes a unified least squares framework to formulate many ca methods we show how pca lda cca lpp sc and its kernel and regularized extensions correspond to a particular instance of least squares weighted kernel reduced rank regression ls wkrrr the ls wkrrr formulation of ca methods has several benefits 1 provides a clean connection between many ca techniques and an intuitive framework to understand normalization factors 2 yields efficient numerical schemes to solve ca techniques 3 overcomes the small sample size problem 4 provides a framework to easily extend ca methods we derive weighted generalizations of pca lda sc and cca and several new ca techniques

oscillation and nonoscillation of hill s equation with periodic damping we prove new results on the oscillation and nonoscillation of the hill s equation with periodic damping y p t y q t y 0 t 0 where p t and q t are continuous and periodic the results show that the equation y sint y cost y 0 is nonoscillatory whilst the equation y cost y sint y 0 is oscillatory

exponential stability and extended dissipativity criteria for generalized discrete time neural networks with additive time varying delays abstract this paper is concerned with exponential stability and extended dissipativity criteria for generalized discrete time neural networks gdnns with additive time varying delays the generalized dissipativity analysis combines a few previous results into a framework such as l 2 l performance h performance passivity performance strictly q s r dissipative and strictly q s r dissipative the definition of exponential stability for gdnns is given with a new and more appropriate expression a novel augmented lyapunov krasovskii functional lkf which involves more information about the additive time varying delays is constructed by introducing more zero equalities and using a new double summation inequality together with finsler s lemma an improved delay dependent exponential stability and extended dissipativity criterion are derived in terms of convex combination technique cct finally numerical examples are given to illustrate the usefulness and advantages of the proposed methods

oscillation criterion for half linear differential equations with periodic coefficients abstract in this paper we present an oscillation criterion for second order half linear differential equations with periodic coefficients the method is based on the nonexistence of a proper solution of the related modified riccati equation our result can be regarded as an oscillatory counterpart to the nonoscillation criterion by sugie and matsumura 2008 these two theorems provide a complete half linear extension of the oscillation criterion of kwong and wong 2003 dealing with the hill s equation

em algorithms for ml factor analysis the details of em algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models the algorithm is essentially the same for both cases and involves only simple least squares regression operations the largest matrix inversion required is for aq q symmetric matrix whereq is the matrix of factors the example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation

nonlinear kernel based statistical pattern analysis the eigenstructure of the second order statistics of a multivariate random population can be inferred from the matrix of pairwise combinations of inner products of the samples therefore it can be also efficiently obtained in the implicit high dimensional feature spaces defined by kernel functions we elaborate on this property to obtain general expressions for immediate derivation of nonlinear counterparts of a number of standard pattern analysis algorithms including principal component analysis data compression and denoising and fisher s discriminant the connection between kernel methods and nonparametric density estimation is also illustrated using these results we introduce the kernel version of mahalanobis distance which originates nonparametric models with unexpected and interesting properties and also propose a kernel version of the minimum squared error mse linear discriminant function this learning machine is particularly simple and includes a number of generalized linear models such as the potential functions method or the radial basis function rbf network our results shed some light on the relative merit of feature spaces and inductive bias in the remarkable generalization properties of the support vector machine svm although in most situations the svm obtains the lowest error rates exhaustive experiments with synthetic and natural data show that simple kernel machines based on pseudoinversion are competitive in problems with appreciable class overlapping

stability analysis for a class of neutral type singular systems with time varying delay abstract this paper is concerned with the stability problem for a class of neutral type singular systems with time varying delay the considered systems contain delays both in their state and in their derivatives of state based on the singular system approach and the lyapunov krasovskii functional approach some sufficient conditions which guarantee the considered systems to be regular impulse free and stable are derived finally some numerical examples are provided to show the effectiveness of the presented methods

indefinite stochastic linear quadratic control and generalized differential riccati equation a stochastic linear quadratic lq control problem is indefinite when the cost weighting matrices for the state and the control are allowed to be indefinite indefinite stochastic lq theory has been extensively developed and has found interesting applications in finance however there remains an outstanding open problem which is to identify an appropriate riccati type equation whose solvability is it equivalent to the solvability of the indefinite stochastic lq problem this paper solves this open problem for lq control in a finite time horizon a new type of differential riccati equation called the generalized differential riccati equation is introduced which involves algebraic equality inequality constraints and a matrix pseudoinverse it is then shown that the solvability of the generalized riccati equation is not only sufficient but also necessary for the well posedness of the indefinite lq problem and the existence of optimal feedback open loop controls moreover all of the optimal controls can be identified via the solution to the riccati equation an example is presented to illustrate the theory developed

a nonoscillation theorem for half linear differential equations with periodic coefficients abstract the half linear differential equation p x a t p x b t p x 0 is considered under the assumption that the coefficient a t and an indefinite integral b t of b t are periodic functions with period t 0 it is proved that p 1 p b t a t b t 0 0 t t is sufficient for all nontrivial solutions to be nonoscillatory here p 1 and q y y q 2 y for q p or q p p p 1 the proof is given by means of riccati technique the condition is shown to be sharp sufficient conditions are also presented for all nontrivial solutions are oscillatory in the linear case p 2 some examples and simulations are included to illustrate our results

simple conditions for parametrically excited oscillations of generalized mathieu equations abstract the following equation is considered in this paper x cos t x 0 where and are real parameters and 0 this equation is referred to as mathieu s equation when 2 the parameters determine whether all solutions of this equation are oscillatory or nonoscillatory our results provide parametric conditions for oscillation and nonoscillation there is a feature in which it is very easy to check whether these conditions are satisfied or not parametric oscillation and nonoscillation regions are drawn to help understand the obtained results

infinite horizon quadratic optimal control of a class of nonlinear stochastic systems the author considers a class of discrete time nonlinear stochastic control systems whose nonlinearities are described by statistical means by introducing the proper mean square stabilizability and detectability properties of such systems and giving a characterization of these properties in terms of system parameters he examines the steady state properties of quadratically optimal controllers a robustness property of these controllers is pointed out

interval oscillation of a general class of second order nonlinear differential equations with nonlinear damping abstract the paper is concerned with the oscillation of a class of general type second order differential equations with nonlinear damping terms several new interval oscillation criteria are established for such a class of differential equations under quite general assumptions examples are also given to illustrate the results in particular it is shown that under some very mild conditions on k 1 k 2 and f the equation k 1 x x cos tk 2 x x x sin tf x 0 is oscillatory which extends the result obtained by kwong and wong j math anal appl 288 2003 15

an introduction to kernel based learning algorithms this paper provides an introduction to support vector machines kernel fisher discriminant analysis and kernel principal component analysis as examples for successful kernel based learning methods we first give a short background about vapnik chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations we illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and dna analysis

optimal harvesting strategy for a stochastic mutualism system in a polluted environment with regime switching abstract in this paper we stochastically perturb the harvesting mutualism system into stochastic system in a polluted environment by white noise telegraph noise and levy jump we first prove the existence and uniqueness of the global positive solution of the stochastic system and establish the sufficient condition for persistence in mean of the two cooperative species then using aggregation and ergodic methods the explicit forms of optimal harvesting strategy and the maximum of sustainable yield are obtained we also give some examples and numerical simulations to illustrate the theoretical results the impacts of toxicant in organism telegraph noise and levy jump on the persistence in mean and optimal harvesting strategy are revealed

the indian buffet process an introduction and review the indian buffet process is a stochastic process defining a probability distribution over equivalence classes of sparse binary matrices with a finite number of rows and an unbounded number of columns this distribution is suitable for use as a prior in probabilistic models that represent objects using a potentially infinite array of features or that involve bipartite graphs in which the size of at least one class of nodes is unknown we give a detailed derivation of this distribution and illustrate its use as a prior in an infinite latent feature model we then review recent applications of the indian buffet process in machine learning discuss its extensions and summarize its connections to other stochastic processes

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

independent factor analysis we introduce the independent factor analysis ifa method for recovering independent hidden sources from their observed mixtures ifa generalizes and unifies ordinary factor analysis fa principal component analysis pca and independent component analysis ica and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy ifa is a two step procedure in the first step the source densities mixing matrix and noise covariance are estimated from the observed data by maximum likelihood for this purpose we present an expectation maximization em algorithm which performs unsupervised learning of an associated probabilistic model of the mixing situation each source in our model is described by a mixture of gaussians thus all the probabilistic calculations can be performed analytically in the second step the sources are reconstructed from the observed data by an optimal nonlinear estimator a variational approximation of this algorithm is derived for cases with a large number of sources where the exact algorithm becomes intractable our ifa algorithm reduces to the one for ordinary fa when the sources become gaussian and to an em algorithm for pca in the zero noise limit we derive an additional em algorithm specifically for noiseless ifa this algorithm is shown to be superior to ica since it can learn arbitrary source densities from the data beyond blind separation ifa can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding

stochastic linear quadratic regulators with indefinite control weight costs ii in part i of this paper s chen x li and x zhou siam j control optim 36 1998 pp 1685 1702 an optimization model of stochastic linear quadratic regulators lqrs with indefinite control cost weighting matrices is proposed and studied in this sequel the problem of solving lqr models with system diffusions dependent on both state and control variables which is left open in part i is tackled first the solvability of the associated stochastic riccati equations sres is studied in the normal case namely all the state and control weighting matrices and the terminal matrix in the cost functional are nonnegative definite with at least one positive definite which in turn leads to an optimal state feedback control of the lqr problem in the general indefinite case the problem is decomposed into two optimal lqr problems one with a forward dynamics and the other with a backward dynamics the well posedness and solvability of the original lqr problem are then obtained by solving these two subproblems and an optimal control is explicitly constructed examples are presented to illustrate the results

probabilistic principal component analysis principal component analysis pca is a ubiquitous technique for data analysis and processing but one which is not based upon a probability model in this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model closely related to factor analysis we consider the properties of the associated likelihood function giving an em algorithm for estimating the principal subspace iteratively and discuss the advantages conveyed by the definition of a probability density function for pca

stochastic linear quadratic regulators with indefinite control weight costs this paper considers optimal minimizing control of stochastic linear quadratic regulators lqrs the assumption that the control weight costs must be positive definite inherited from the determ

optimally adaptive transform coding the optimal linear block transform for coding images is well known to be the karhunen loeve transformation klt however the assumption of stationarity in the optimality condition is far from valid for images images are composed of regions whose local statistics may vary widely across an image while the use of adaptation can result in improved performance there has been little investigation into the optimality of the criterion upon which the adaptation is based in this paper we propose a new transform coding method in which the adaptation is optimal the system is modular consisting of a number of modules corresponding to different classes of the input data each module consists of a linear transformation whose bases are calculated during an initial training period the appropriate class for a given input vector is determined by the subspace classifier the performance of the resulting adaptive system is shown to be superior to that of the optimal nonadaptive linear transformation this method can also be used as a segmentor the segmentation it performs is independent of variations in illumination in addition the resulting class representations are analogous to the arrangement of the directionally sensitive columns in the visual cortex

oscillation problems for hill s equation with periodic damping abstract this paper deals with the second order linear differential equation x a t x b t x 0 where a and b are periodic coefficients the main purpose is to present new criteria which guarantee that all nontrivial solutions are nonoscillatory and that those are oscillatory our nonoscillation theorem and oscillation theorem are proved by using the riccati technique in our theorem the composite function of an indefinite integral of b and a suitable multiple valued continuously differentiable function are focused and the composite function of them plays an important role the results obtained here include a result by kwong and wong 15 and a result by sugie and matsumura 26 an application to a equation of whittaker hill type is given to show the usefulness of our results finally simulations are also attached to illustrate that our oscillation criterion is sharp

solvability and asymptotic behavior of generalized riccati equations arising in indefinite stochastic lq controls the optimal control problem in a finite time horizon with an indefinite quadratic cost function for a linear system subject to multiplicative noise on both the state and control can be solved via a constrained matrix differential riccati equation in this paper we provide general necessary and sufficient conditions for the solvability of this generalized differential riccati equation furthermore its asymptotic behavior is investigated along with its connection to the generalized algebraic riccati equation associated with the linear quadratic control problem in finite time horizon examples are presented to illustrate the results established

input space versus feature space in kernel based methods this paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector sv kernel functions we first discuss the geometry of feature space in particular we review what is known about the shape of the image of input space under the feature space map and how this influences the capacity of sv methods following this we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel using the example of the class of inhomogeneous polynomial kernels which are often used in sv pattern recognition we then discuss the connection between feature space and input space by dealing with the question of how one can given some vector in feature space find a preimage exact or approximate in input space we describe algorithms to tackle this issue and show their utility in two applications of kernel methods first we use it to reduce the computational complexity of sv decision functions second we combine it with the kernel pca algorithm thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real world data

on kamenev type oscillation theorems for second order differential equations with damping abstract oscillation criteria for the nonlinear second order ordinary differential equation with damping x p t x f x 0 are given the results are extensions of integral averaging technique of kamenev and include earlier results of yeh yan and chen

learning harmonium models with infinite latent features undirected latent variable models represent an important class of graphical models that have been successfully developed to deal with various tasks one common challenge in learning such models is to determine the number of hidden units that are unknown a priori although bayesian nonparametrics have provided promising results in bypassing the model selection problem in learning directed bayesian networks very little effort has been made toward applying bayesian nonparametrics to learn undirected latent variable models in this paper we present the infinite exponential family harmonium iefh a bipartite undirected latent variable model that automatically determines the number of latent units from an unbounded pool we also present two important extensions of iefh to 1 multiview iefh for dealing with heterogeneous data and 2 infinite maximum margin harmonium immh for incorporating supervising side information to learn predictive latent features we develop variational inference algorithms to learn model parameters our methods are computationally competitive because of the avoidance of selecting the number of latent units our extensive experiments on real image datasets and text datasets appear to demonstrate the benefits of iefh and immh inherited from bayesian nonparametrics and max margin learning such results were not available until now and contribute to expanding the scope of bayesian nonparametrics to learn the structures of undirected latent variable models