nonlinear kolmogorov equations in infinite dimensional spaces the backward stochastic differential equations approach and applications to optimal control solutions of semilinear parabolic differential equations in infinite dimensional spaces are obtained by means of forward and backward infinite dimensional stochastic evolution equations parabolic equations are intended in a mild sense that reveals to be suitable also towards applications to optimal control

blind constant modulus equalization via convex optimization in this paper we formulate the problem of blind equalization of constant modulus cm signals as a convex optimization problem the convex formulation is obtained by performing an algebraic transformation on the direct formulation of the cm equalization problem using this transformation the original nonconvex cm equalization formulation is turned into a convex semidefinite program sdp that can be efficiently solved using interior point methods our sdp formulation is applicable to baud spaced equalization as well as fractionally spaced equalization performance analysis shows that the expected distance between the equalizer obtained by the sdp approach and the optimal equalizer in the noise free case converges to zero exponentially as the signal to noise ratio snr increases in addition simulations suggest that our method performs better than standard methods while requiring significantly fewer data samples

h 2 h control for stochastic systems with delay this paper is concerned with the h2 h control problem for stochastic linear systems with delay in state control and external disturbance dependent noise a necessary and sufficient condition for the existence of a unique solution to the control problem is derived the resulting solution is characterised by a kind of complex generalised forward backward stochastic differential equations with stochastic delay equations as forward equations and anticipated backward stochastic differential equations as backward equations especially we present the equivalent feedback solution via a new type of riccati equations to explain the theoretical results we apply them to a population control problem

optimal control of mean field jump diffusion systems with delay this paper is concerned with an optimal control problem under mean field jump diffusion systems with delay firstly some existence and uniqueness results are proved for a jump diffusion mean field stochastic delay differential equation and a jump diffusion mean field advanced backward stochastic differential equation then necessary and sufficient maximum principles for control systems of mean field type and with delay are established under certain conditions a mean field delayed linear quadratic control problem is finally discussed using the obtained maximum principles

a partial information non zero sum differential game of backward stochastic differential equations with applications this paper is concerned with a new kind of non zero sum differential game of backward stochastic differential equations bsdes it is required that the control is adapted to a sub filtration of the filtration generated by the underlying brownian motion we establish a necessary condition in the form of maximum principle with pontryagin s type for open loop nash equilibrium point of this type of partial information game and then give a verification theorem which is a sufficient condition for nash equilibrium point the theoretical results are applied to study a partial information linear quadratic lq game and a partial information financial problem

performance analysis of source image estimators in blind source separation blind methods often separate or identify signals or signal subspaces up to an unknown scaling factor sometimes it is necessary to cope with the scaling ambiguity which can be done through reconstructing signals as they are received by sensors because scales of the sensor responses images have known physical interpretations in this paper we analyze two approaches that are widely used for computing the sensor responses especially in frequency domain independent component analysis one approach is the least squares projection whereas the other one assumes a regular mixing matrix and computes its inverse both estimators are invariant to the unknown scaling although frequently used their differences were not studied yet a goal of this paper is to fill this gap the estimators are compared through a theoretical study perturbation analysis and simulations we point to the fact that the estimators are equivalent when the separated signal subspaces are orthogonal and vice versa two applications are shown one of which demonstrates a case where the estimators yield substantially different results

emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces olshausen and field 1996 applied the principle of independence maximization by sparse coding to extract features from natural images this leads to the emergence of oriented linear filters that have simultaneous localization in space and in frequency thus resembling gabor functions and simple cell receptive fields in this article we show that the same principle of independence maximization can explain the emergence of phase and shift invariant features similar to those found in complex cells this new kind of emergence is obtained by maximizing the independence between norms of projections on linear subspaces instead of the independence of simple linear filter outputs the norms of the projections on such independent feature subspaces then indicate the values of invariant features

joint independent subspace analysis using second order statistics this paper deals with a novel generalization of classical blind source separation bss in two directions first relaxing the constraint that the latent sources must be statistically independent this generalization is well known and sometimes termed independent subspace analysis isa second jointly analyzing several isa problems where the link is due to statistical dependence among corresponding sources in different mixtures when the data are one dimensional i e multiple classical bss problems this model known as independent vector analysis iva has already been studied in this paper we combine iva with isa and term this new model joint independent subspace analysis jisa we provide full performance analysis of jisa including closed form expressions for minimal mean square error mse fisher information and cramer rao lower bound in the separation of gaussian data the derived mse applies also for non gaussian data when only second order statistics are used we generalize previously known results on iva including its ability to uniquely resolve instantaneous mixtures of real gaussian stationary data and having the same arbitrary permutation at all mixtures numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm

brief paper maximum principle for the stochastic optimal control problem with delay and application in this paper we consider an optimal control problem for the stochastic system described by stochastic differential equations with delay we obtain the maximum principle for the optimal control of this problem by virtue of the duality method and the anticipated backward stochastic differential equations our results can be applied to a production and consumption choice problem the explicit optimal consumption rate is obtained

stochastic maximum principle for optimal control problems of forward backward systems involving impulse controls we consider a stochastic optimal control problem of a forward backward system in which the control variable consists of two components the continuous control and the impulse control the domain of the control is assumed to be convex necessary optimality conditions of the pontryagin maximum principle type are obtained for this stochastic optimal control problem we also give additional conditions under which the necessary optimality conditions turn out to be sufficient

semidefinite programming relaxations for semialgebraic problems a hierarchy of convex relaxations for semialgebraic problems is introduced for questions reducible to a finite number of polynomial equalities and inequalities it is shown how to construct a complete family of polynomially sized semidefinite programming conditions that prove infeasibility the main tools employed are a semidefinite programming formulation of the sum of squares decomposition for multivariate polynomials and some results from real algebraic geometry the techniques provide a constructive approach for finding bounded degree solutions to the positivstellensatz and are illustrated with examples from diverse application fields

maximum principle for mean field jump diffusion stochastic delay differential equations and its application to finance abstract this paper investigates a stochastic optimal control problem with delay and of mean field type where the controlled state process is governed by a mean field jump diffusion stochastic delay differential equation two sufficient maximum principles and one necessary maximum principle are established for the underlying system as an application a bicriteria mean variance portfolio selection problem with delay is studied to demonstrate the effectiveness and potential of the proposed techniques under certain conditions explicit expressions are provided for the efficient portfolio and the efficient frontier which are as elegant as those in the classical mean variance problem without delays

maximum principle for nonzero sum stochastic differential game with delays in this technical note we discuss a nonzero sum stochastic differential game with delays not only the state variable but also control variables of players involve delays this kind of games are motivated by some interesting problems arising from economics and finance using anticipated backward stochastic differential equations we establish a necessary condition and a sufficient condition of maximum principle for the delayed game problem to explain theoretical results we apply them to an economic problem

quadratic maximization and semidefinite relaxation in this paper we study a class of quadratic maximization problems and their semidefinite programming sdp relaxation for a special subclass of the problems we show that the sdp relaxation provides an exact optimal solution another subclass which is hard guarantees that the sdp relaxation yields an approximate solution with a worst case performance ratio of 0 87856 this is a generalization of the well known result of goemans and williamson for the maximum cut problem finally we discuss extensions of these results in the presence of a certain type of sign restrictions

optimal consumption and portfolio with both fixed and proportional transaction costs we consider a market model with one risk free and one risky asset in which the dynamics of the risky asset are governed by a geometric brownian motion in this market we consider an investor who consumes from the bank account and who has the opportunity at any time to transfer funds between the two assets we suppose that these transfers involve a fixed transaction cost k 0 independent of the size of the transaction plus a cost proportional to the size of the transaction the objective is to maximize the cumulative expected utility of consumption over a planning horizon we formulate this problem as a combined stochastic control impulse control problem which in turn leads to a nonlinear quasi variational hamilton jacobi bellman inequality qvhjbi we prove that the value function is the unique viscosity solution of this qvhjbi finally numerical results are presented

separation theorem for independent subspace analysis and its consequences independent component analysis ica the theory of mixed independent non gaussian sources has a central role in signal processing computer vision and pattern recognition one of the most fundamental conjectures of this research field is that independent subspace analysis isa the extension of the ica problem where groups of sources are independent can be solved by traditional ica followed by grouping the ica components the conjecture called isa separation principle i has been rigorously proven for some distribution types recently ii forms the basis of the state of the art isa solvers iii enables one to estimate the unknown number and the dimensions of the sources efficiently and iv can be extended to generalizations of the isa task such as different linear controlled post nonlinear complex valued partially observed problems as well as to problems dealing with nonparametric source dynamics here we shall review the advances on this field

technical communique a closed form optimal control for linear systems with equal state and input delays this paper presents the optimal regulator for a linear system with equal delays in state and input and a quadratic criterion the optimal regulator equations are obtained using the maximum principle performance of the obtained optimal regulator is verified in the illustrative example against the best linear regulators available for the linear system without delays and for a rational approximation of the original time delay system simulation graphs demonstrating better performance of the obtained optimal regulator are included

a tensor product matrix approximation problem in quantum physics we consider a matrix approximation problem arising in the study of entanglement in quantum physics this notion represents a certain type of correlations between subsystems in a composite quantum system the states of a system are described by a density matrix which is a positive semidefinite matrix with trace one the goal is to approximate such a given density matrix by a so called separable density matrix and the distance between these matrices gives information about the degree of entanglement in the system separability here is expressed in terms of tensor products we discuss this approximation problem for a composite system with two subsystems and show that it can be written as a convex optimization problem with special structure we investigate related convex sets and suggest an algorithm for this approximation problem which exploits the tensor product structure in certain subproblems finally some computational results and experiences are presented

undercomplete blind subspace deconvolution we introduce the blind subspace deconvolution bssd problem which is the extension of both the blind source deconvolution bsd and the independent subspace analysis isa tasks we examine the case of the undercomplete bssd ubssd applying temporal concatenation we reduce this problem to isa the associated high dimensional isa problem can be handled by a recent technique called joint f decorrelation jfd similar decorrelation methods have been used previously for kernel independent component analysis kernel ica more precisely the kernel canonical correlation kcca technique is a member of this family and as is shown in this paper the kernel generalized variance kgv method can also be seen as a decorrelation method in the feature space these kernel based algorithms will be adapted to the isa task in the numerical examples we i examine how efficiently the emerging higher dimensional isa tasks can be tackled and ii explore the working and advantages of the derived kernel isa methods

approximation bounds for quadratic optimization with homogeneous quadratic constraints we consider the np hard problem of finding a minimum norm vector in n dimensional real or complex euclidean space subject to m concave homogeneous quadratic constraints we show that a semidefinite programming sdp relaxation for this nonconvex quadratically constrained quadratic program qp provides an o m 2 approximation in the real case and an o m approximation in the complex case moreover we show that these bounds are tight up to a constant factor when the hessian of each constraint function is of rank 1 namely outer products of some given so called steering vectors and the phase spread of the entries of these steering vectors are bounded away from pi 2 we establish a certain constant factor approximation depending on the phase spread but independent of m and n for both the sdp relaxation and a convex qp restriction of the original np hard problem finally we consider a related problem of finding a maximum norm vector subject to m convex homogeneous quadratic constraints we show that an sdp relaxation for this nonconvex qp provides an o 1 ln m approximation which is analogous to a result of nemirovski e math program 86 1999 pp 463 473 for the real case

error bounds for some semidefinite programming approaches to polynomial minimization on the hypercube we consider the problem of minimizing a polynomial on the hypercube 0 1 n and derive new error bounds for the hierarchy of semidefinite programming approximations to this problem corresponding to the positivstellensatz of schmudgen math ann 289 1991 pp 203 206 the main tool we employ is bernstein approximations of polynomials which also gives constructive proofs and degree bounds for positivity certificates on the hypercube

blind separation of multi dimensional components via subspace decomposition performance analysis a prevalent approach to blind separation of multi dimensional data is a two step procedure in the first step the data are assigned a one dimensional model a separating algorithm is applied according to this model this step corresponds to classical blind source separation bss in the second step the output is assigned into groups representing the multi dimensional components in this paper we consider an even more general case in which the subpartition of the components in the first step may be into elements of any dimension not necessarily one we consider a piecewise stationary model and assume that the number and dimensions of the underlying multi dimensional components are known we obtain a closed form analytical expression for the mean square error mse in the estimation of the multi dimensional components using this two step procedure as expected this approach is suboptimal in the presence of finite data size errors therefore we can predict the expected gain from using the correct model of the components over any finer decomposition thereof in addition we demonstrate the dependence of this gain on the model parameters

second order multidimensional ica performance analysis independent component analysis ica and blind source separation bss deal with extracting a number of mutually independent elements from a set of observed linear mixtures motivated by various applications this paper considers a more general and more flexible model the sources can be partitioned into groups exhibiting dependence within a given group but independence between two different groups we argue that this is tantamount to considering multidimensional components as opposed to the standard ica case which is restricted to one dimensional components the core of the paper is devoted to the statistical analysis of the blind separation of multidimensional components based on second order statistics in a piecewise stationary model we develop the likelihood and the associated estimating equations for the gaussian case we obtain closed form expressions for the fisher information matrix and the cramer rao bound of the de mixing parameters as well as the mean square error mse of the component estimates the derived mse is valid also for non gaussian data our analysis is verified through numerical experiments and its performance is compared to classical ica in various dependence scenarios quantifying the gain in the accuracy of component recovery in presence of multidimensional components

linear equations modulo 2 and the l_1 diameter of convex bodies we design a randomized polynomial time algorithm which given a 3 tensor of real numbers a a ijk i j k 1 n such that for all i j k in 1 dots n we have a ijk a ikj a kji a jik a kij a jki and a iik a ijj a iji 0 computes a number operatorname alg a which satisfies with probability at least frac12 omega sqrt frac log n n t cdot max x in 1 1 n sum i j k 1 n a ijk x ix jx k le operatorname alg a le max x in 1 1 n sum i j k 1 n a ijk x ix jx k on the other hand we show via a simple reduction from a result of hastad and venkatesh random structures algorithms 25 2004 pp 117 149 that under the assumption np not subseteq dtime n log n o 1 for every epsilon 0 there is no algorithm that approximates max x in 1 1 n sum i j k 1 n a ijk x ix jx k within a factor of 2 log n 1 epsilon in time 2 log n o 1 our algorithm is based on a reduction to the problem of computing the diameter of a convex body in mathbb r n with respect to the l 1 norm we show that it is possible to do so up to a multiplicative error of o sqrt frac n log n while no randomized polynomial time algorithm can achieve accuracy o sqrt frac n log n this resolves a question posed by brieden et al in mathematika 48 2001 pp 63 105 we apply our new algorithm to improve the algorithm of hastad and venkatesh for the max e3 lin 2 problem given an overdetermined system mathcal e of n linear equations modulo 2 in n le n boolean variables such that in each equation only three distinct variables appear the goal is to approximate in polynomial time the maximum number of satisfiable equations in mathcal e minus frac n 2 i e we subtract the expected number of satisfied equations in a random assignment hastad and venkatesh obtained an algorithm which approximates this value up to a factor of o sqrt n we obtain an o sqrt frac n log n approximation algorithm by relating this problem to the refutation problem for random 3 cnf formulas we give evidence that obtaining a significant improvement over this approximation factor is likely to be difficult

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

blind separation of instantaneous mixtures of nonstationary sources most source separation algorithms are based on a model of stationary sources however it is a simple matter to take advantage of possible nonstationarities of the sources to achieve separation this paper develops novel approaches in this direction based on the principles of maximum likelihood and minimum mutual information these principles are exploited by efficient algorithms in both the off line case via a new joint diagonalization procedure and in the on line case via a newton like procedure some experiments showing the good performance of our algorithms and evidencing an interesting feature of our methods are presented their ability to achieve a kind of super efficiency the paper concludes with a discussion contrasting separating methods for non gaussian and nonstationary models and emphasizing that as a matter of fact what makes the algorithms work is strictly speaking not the nonstationarity itself but rather the property that each realization of the source signals has a time varying envelope

inhomogeneous polynomial optimization over a convex set an approximation approach in this paper we consider computational methods for optimizing a multivariate inhomogeneous polynomial function over a general convex set the focus is on the design and analysis of polynomial time approximation algorithms the methods are able to deal with optimization models with polynomial objective functions in any fixed degrees in particular we first study the problem of maximizing an inhomogeneous polynomial over the euclidean ball a polynomial time approximation algorithm is proposed for this problem with an assured relative worst case performance ratio which is dependent only on the dimensions of the model the method and approximation ratio are then generalized to optimize an inhomogeneous polynomial over the intersection of a finite number of co centered ellipsoids furthermore the constraint set is extended to a general convex compact set specifically we propose a polynomial time approximation algorithm with a relative worst case performance ratio for polynomial optimization over some convex compact sets e g a polytope finally numerical results are reported revealing good practical performance of the proposed algorithms for solving some randomly generated instances

an eigenvalue method for testing positive definiteness of a multivariate form in this paper we present an eigenvalue method for testing positive definiteness of a multivariate form this problem plays an important role in the stability study of nonlinear autonomous systems via lyapunov s direct method in automatic control at first we apply the d andrea dickenstein version of the classical macaulay formulas of the resultant to compute the symmetric hyperdeterminant of an even order supersymmetric tensor by using the supersymmetry property we give detailed computation procedures for the bezoutians and specified ordering of monomials in this approach we then use these formulas to calculate the characteristic polynomial of a fourth order three dimensional supersymmetric tensor and give an eigenvalue method for testing positive definiteness of a quartic form of three variables some numerical results of this method are reported

adapted solution of a backward stochastic differential equation abstract let wt t 0 1 be a standard k dimensional weiner process defined on a probability space f p and let ft denote its natural filtration given a f1 measurable d dimensional random vector x we look for an adapted pair of processes x t y t t 0 1 with values in rd and rd k respectively which solves an equation of the form x t t 1 f s x s y s ds t 1 g s x s y s dw s x a linearized version of that equation appears in stochastic control theory as the equation satisfied by the adjoint process we also generalize our results to the following equation x t t 1 f s x s y s ds t 1 g s x s y s dw s x under rather restrictive assumptions on g

joint independent subspace analysis uniqueness and identifiability this paper deals with the identifiability of joint independent subspace analysis jisa jisa is a recently proposed framework that subsumes independent vector analysis iva and independent subspace analysis isa each underlying mixture can be regarded as a dataset therefore jisa can be used for data fusion in this paper we assume that each dataset is an overdetermined mixture of several multivariate gaussian processes each of which has independent and identically distributed samples this setup is not identifiable when each mixture is considered individually given these assumptions jisa can be restated as coupled block diagonalization cbd of its correlation matrices hence jisa identifiability is tantamount to cbd uniqueness in this work we provide necessary and sufficient conditions for uniqueness and identifiability of jisa and cbd our analysis is based on characterizing all the cases in which the fisher information matrix is singular we prove that non identifiability may occur only due to pairs of underlying random processes with the same dimension our results provide further evidence that irreducibility has a central role in the uniqueness analysis of block based decompositions our contribution extends previous results on the uniqueness and identifiability of isa iva coupled matrix and tensor decompositions we provide examples to illustrate our results

on cones of nonnegative quadratic functions we derive linear matrix inequality lmi characterizations and dual decomposition algorithms for certain matrix cones which are generated by a given set using generalized co positivity these matrix cones are in fact cones of nonconvex quadratic functions that are nonnegative on a certain domain as a domain we consider for instance the intersection of a upper level set of a quadratic function and a half plane consequently we arrive at a generalization of yakubovich s s procedure result although the primary concern of this paper is to characterize the matrix cones by lmis we show as an application of our results that optimizing a general quadratic function over the intersection of an ellipsoid and a half plane can be formulated as semidefinite programming sdp thus proving the polynomiality of this class of optimization problems which arise e g from the application of the trust region method for nonlinear programming other applications are in control theory and robust optimization

polynomials nonnegative on a grid and discrete optimization we characterize the real valued polynomials on r n that are nonnegative not necessarily strictly positive on a grid k of points of r n in terms of a weighted sum of squares whose degree is bounded and known in advance we also show that the mimimization of an arbitrary polynomial on k a discrete optimization problem reduces to a convex continuous optimization problem of fixed size the case of concave polynomials is also investigated the proof is based on a recent result of curto and fialkow on the k moment problem

ito volterra optimal state estimation with continuous multirate randomly sampled and delayed measurements the optimal filter for continuous linear stochastic time varying systems described by the itocirc volterra equations with discontinuous measure is derived with an appropriately selected measure the result is applicable to a wide range of observation processes including the hybrid case of observations formed by an arbitrary combination of continuous and discrete measurements which may be sampled with a priori unknown changing and possibly random rates and delays the simultaneous presence of continuous and sampled measurements causes impulsive discontinuity in the inputs of the optimal filter equations which leads to a discontinuous change in state estimates every time a sampled measurement becomes available using the theory of vibrosolutions the explicit and unique expressions for the jumps in state estimates and estimation error covariance are derived several examples illustrate the procedure of modeling hybrid measurement systems by selecting an appropriate discontinuous measure we further show that the itocirc volterra model and the main result of the paper can be specialized to several important cases including state space systems for which we recover several known state estimation results and derive a novel optimal filter for continuous ltv systems with an arbitrary combination of continuous and delayed sampled measurements this optimal filter updates the state estimates for incoming measurements as soon as they become available and does not require prior knowledge of sampling instants and delays which makes it applicable when deterministic and random changes in sampling and delays are present several computational examples demonstrate the implementation of the developed filter and compare its performance to the traditional alternatives using monte carlo simulations

maximum block improvement and polynomial optimization in this paper we propose an efficient method for solving the spherically constrained homogeneous polynomial optimization problem the new approach has the following three main ingredients first we establish a block coordinate descent type search method for nonlinear optimization with the novelty being that we accept only a block update that achieves the maximum improvement hence the name of our new search method maximum block improvement mbi convergence of the sequence produced by the mbi method to a stationary point is proved second we establish that maximizing a homogeneous polynomial over a sphere is equivalent to its tensor relaxation problem thus we can maximize a homogeneous polynomial function over a sphere by its tensor relaxation via the mbi approach third we propose a scheme to reach a kkt point of the polynomial optimization provided that a stationary solution for the relaxed tensor problem is available numerical experiments have shown that our new method works very efficiently for a majority of the test instances that we have experimented with the method finds the global optimal solution at a low computational cost

approximating the cut norm via grothendieck s inequality the cut norm a c of a real matrix a a ij i in r j in s is the maximum over all i subset r j subset s of the quantity sum i in i j in j a ij this concept plays a major role in the design of efficient approximation algorithms for dense graph and matrix problems here we show that the problem of approximating the cut norm of a given real matrix is max snp hard and we provide an efficient approximation algorithm this algorithm finds for a given matrix a a ij i in r j in s two subsets i subset r and j subset s such that sum i in i j in j a ij geq rho a c where rho 0 is an absolute constant satisfying rho 0 56 the algorithm combines semidefinite programming with a rounding technique based on grothendieck s inequality we present three known proofs of grothendieck s inequality with the necessary modifications which emphasize their algorithmic aspects these proofs contain rounding techniques which go beyond the random hyperplane rounding of goemans and williamson j acm 42 1995 pp 1115 1145 allowing us to transfer various algorithms for dense graph and matrix problems to the sparse case

a pontryagin s maximum principle for non zero sum differential games of bsdes with applications this technical note is concerned with a maximum principle for a new class of non zero sum stochastic differential games the most distinguishing feature compared with the existing literature is that the game systems are described by backward stochastic differential equations bsdes this kind of games are motivated by some interesting phenomena arising from financial markets and can be used to characterize the players with different levels of utilities we establish a necessary condition and a sufficient condition in the form of maximum principle for open loop equilibrium point of the foregoing games respectively to explain the theoretical results we use them to study a financial problem

a tensor framework for nonunitary joint block diagonalization this paper introduces a tensor framework to solve the problem of nonunitary joint block diagonalization jbd of a set of real or complex valued matrices we show that jbd can be seen as a particular case of the block component decomposition bcd of a third order tensor the resulting tensor model fitting problem does not require the block diagonalizer to be a square matrix the over and underdetermined cases can be handled to compute the tensor decomposition we build an efficient nonlinear conjugate gradient ncg algorithm in the over and exactly determined cases we show that exact jbd can be computed by a closed form solution based on eigenvalue analysis in approximate jbd problems this solution can be used to efficiently initialize any iterative jbd algorithm such as ncg finally we illustrate the performance of our technique in the context of independent subspace analysis isa based on second order statistics sos

non orthogonal joint diagonalization in the least squares sense with application in blind source separation approximate joint diagonalization of a set of matrices is an essential tool in many blind source separation bss algorithms a common measure of the attained diagonalization of the set is the weighted least squares wls criterion however most well known algorithms are restricted to finding an orthogonal diagonalizing matrix relying on a whitening phase for the nonorthogonal factor often such an approach implies unbalanced weighting which can result in degraded performance we propose an iterative alternating directions algorithm for minimizing the wls criterion with respect to a general not necessarily orthogonal diagonalizing matrix under some mild assumptions we prove weak convergence in the sense that the norm of parameters update is guaranteed to fall below any arbitrarily small threshold within a finite number of iterations we distinguish between hermitian and symmetrical problems using bss simulations results we demonstrate the improvement in estimating the mixing matrix resulting from the relaxation of the orthogonality restriction

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

fast approximate joint diagonalization incorporating weight matrices we propose a new low complexity approximate joint diagonalization ajd algorithm which incorporates nontrivial block diagonal weight matrices into a weighted least squares wls ajd criterion often in blind source separation bss when the sources are nearly separated the optimal weight matrix for wls based ajd takes a nearly block diagonal form based on this observation we show how the new algorithm can be utilized in an iteratively reweighted separation scheme thereby giving rise to fast implementation of asymptotically optimal bss algorithms in various scenarios in particular we consider three specific yet common scenarios involving stationary or block stationary gaussian sources for which the optimal weight matrices can be readily estimated from the sample covariance matrices which are also the target matrices for the ajd comparative simulation results demonstrate the advantages in both speed and accuracy as well as compliance with the theoretically predicted asymptotic optimality of the resulting bss algorithms based on the weighted ajd both on large scale problems with matrices of the size 100times100

global minimization of normal quartic polynomials based on global descent directions a normal quartic polynomial is a quartic polynomial whose fourth degree term coefficient tensor is positive definite its minimization problem is one of the simplest cases of nonconvex global optimization and has engineering applications we call a direction a global descent direction of a function at a point if there is another point with a lower function value along this direction for a normal quartic polynomial we present a criterion to find a global descent direction at a noncritical point a saddle point or a local maximizer we give sufficient conditions to judge whether a local minimizer is global and give a method for finding a global descent direction at a local but not global minimizer we also give a formula at a critical point and a method at a noncritical point to find a one dimensional global minimizer along a global descent direction based upon these we propose a global descent algorithm for finding a global minimizer of a normal quartic polynomial when n 2 for the case n ge 3 we propose an algorithm for finding an epsilon global minimizer at each iteration of a second algorithm a system of constrained nonlinear equations is solved numerical tests show that these two algorithms are promising

a semidefinite relaxation scheme for multivariate quartic polynomial optimization with quadratic constraints we present a general semidefinite relaxation scheme for general n variate quartic polynomial optimization under homogeneous quadratic constraints unlike the existing sum of squares approach which relaxes the quartic optimization problems to a sequence of typically large linear semidefinite programs sdp our relaxation scheme leads to a possibly nonconvex quadratic optimization problem with linear constraints over the semidefinite matrix cone in mathbb r n times n it is shown that each alpha factor approximate solution of the relaxed quadratic sdp can be used to generate in randomized polynomial time an o alpha factor approximate solution for the original quartic optimization problem where the constant in o cdot depends only on problem dimension in the case where only one positive definite quadratic constraint is present in the quartic optimization problem we present a randomized polynomial time approximation algorithm which can provide a guaranteed relative approximation ratio of 1 o n 2

complex quadratic optimization and semidefinite programming in this paper we study the approximation algorithms for a class of discrete quadratic optimization problems in the hermitian complex form a special case of the problem that we study corresponds to the max 3 cut model used in a recent paper of goemans and williamson j comput system sci 68 2004 pp 442 470 we first develop a closed form formula to compute the probability of a complex valued normally distributed bivariate random vector to be in a given angular region this formula allows us to compute the expected value of a randomized with a specific rounding rule solution based on the optimal solution of the complex semidefinite programming relaxation problem in particular we present an m 2 1 cos frac 2 pi m 8 pi approximation algorithm and then study the limit of that model in which the problem remains np hard we show that if the objective is to maximize a positive semidefinite hermitian form then the randomization rounding procedure guarantees a worst case performance ratio of pi 4 approx 0 7854 which is better than the ratio of 2 pi approx 0 6366 for its counterpart in the real case due to nesterov furthermore if the objective matrix is real valued positive semidefinite with nonpositive off diagonal elements then the performance ratio improves to 0 9349

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

global optimization with polynomials and the problem of moments we consider the problem of finding the unconstrained global minimum of a real valued polynomial p x mathbb r n to mathbb r as well as the global minimum of p x in a compact set k defined by polynomial inequalities it is shown that this problem reduces to solving an often finite sequence of convex linear matrix inequality lmi problems a notion of karush kuhn tucker polynomials is introduced in a global optimality condition some illustrative examples are provided

eigenvalues and invariants of tensors abstract a tensor is represented by a supermatrix under a co ordinate system in this paper we define e eigenvalues and e eigenvectors for tensors and supermatrices by the resultant theory we define the e characteristic polynomial of a tensor an e eigenvalue of a tensor is a root of the e characteristic polynomial in the regular case a complex number is an e eigenvalue if and only if it is a root of the e characteristic polynomial we convert the e characteristic polynomial of a tensor to a monic polynomial and show that the coefficients of that monic polynomial are invariants of that tensor i e they are invariant under co ordinate system changes we call them principal invariants of that tensor the maximum number of principal invariants of mth order n dimensional tensors is a function of m and n we denote it by d m n and show that d 1 n 1 d 2 n n d m 2 m for m 3 and d m n m n 1 m for m n 3 we also define the rank of a tensor all real eigenvectors associated with nonzero e eigenvalues are in a subspace with dimension equal to its rank

biquadratic optimization over unit spheres and semidefinite programming relaxations this paper studies the so called biquadratic optimization over unit spheres min x in mathbb r n y in mathbb r m sum 1 leq i k leq n 1 leq j l leq m b ijkl x i y j x k y l subject to x 1 y 1 we show that this problem is np hard and there is no polynomial time algorithm returning a positive relative approximation bound then we present various approximation methods based on semidefinite programming sdp relaxations our theoretical results are as follows for general biquadratic forms we develop a frac 1 2 max m n 2 approximation algorithm under a slightly weaker approximation notion for biquadratic forms that are square free we give a relative approximation bound frac 1 nm when min n m is a constant we present two polynomial time approximation schemes ptass which are based on sum of squares sos relaxation hierarchy and grid sampling of the standard simplex for practical computational purposes we propose the first order sos relaxation a convex quadratic sdp relaxation and a simple minimum eigenvalue method and show their error bounds some illustrative numerical examples are also included

the stochastic maximum principle for linear convex optimal control with random coefficients this paper considers a stochastic control problem with linear dynamics convex cost criterion and convex state constraint in which the control enters both the drift and diffusion coefficients these coefficients are allowed to be random and no l p bounds are imposed on the control an explicit solution for the adjoint equation and a global stochastic maximum principle are obtained for this model this is evidently the first version of the stochastic maximum principle that covers the consumption investment problem the mathematical tools are those of stochastic calculus and convex analysis when it is assumed as in other versions of the stochastic maximum principle that the admissible controls are square integrable not only a necessary but also a sufficient condition for optimality is obtained it is then shown that this particular case of the general model may be applied to solve a variety of problems in stochastic control including the linear regulator predicted miss and benes problems

the stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls this paper is concerned with a pontryagin s maximum principle for stochastic optimal control problems of delay systems with random coefficients involving both continuous and impulse controls this kind of control problems is motivated by some interesting phenomena arising from economics and finance we establish a necessary maximum principle and a sufficient verification theorem by virtue of the duality and the convex analysis to explain the theoretical results we apply them to a production and consumption choice problem

anticipated backward stochastic differential equations in this paper we discuss new types of differential equations which we call anticipated backward stochastic differential equations anticipated bsdes in these equations the generator includes not only the values of solutions of the present but also the future we show that these anticipated bsdes have unique solutions a comparison theorem for their solutions and a duality between them and stochastic differential delay equations

on maximization of quadratic form over intersection of ellipsoids with common center we demonstrate that if a1 a m are symmetric positive semidefinite n n matrices with positive definite sum and a is an arbitrary symmetric n n matrix then the relative accuracy in terms of the optimal value of the semidefinite relaxation max x tr ax mid tr a ix le1 i 1 m x succeq0 eqno hbox sdp of the optimization program x tax to max mid x ta ix le 1 i 1 m eqno hbox p is not worse than 1 frac 1 2 ln 2m 2 it is shown that this bound is sharp in order as far as the dependence on m is concerned and that a feasible solution x to p with x tax ge frac opt hbox rm sdp 2 ln 2m 2 eqno can be found efficiently this somehow improves one of the results of nesterov 4 where bound similar to is established for the case when all ai are of rank 1

nonzero sum stochastic differential games with discontinuous feedback the existence of a nash equilibrium feedback is established for a two player nonzero sum stochastic differential game with discontinuous feedback this is obtained by studying a parabolic system strongly coupled by discontinuous terms

semidefinite relaxation bounds for indefinite homogeneous quadratic optimization this paper studies the relationship between the optimal value of a homogeneous quadratic optimization problem and its semidefinite programming sdp relaxation we consider two quadratic optimization models 1 min x c x mid x a k x ge 1 k 0 1 ldots m x in mathbb f n and 2 max x c x mid x a k x le 1 k 0 1 ldots m x in mathbb f n where mathbb f is either the real field mathbb r or the complex field mathbb c and a k c are symmetric matrices for the minimization model 1 we prove that if the matrix c and all but one of the a k s are positive semidefinite then the ratio between the optimal value of 1 and its sdp relaxation is upper bounded by o m 2 when mathbb f mathbb r and by o m when mathbb f mathbb c moreover when two or more of the a k s are indefinite this ratio can be arbitrarily large for the maximization model 2 we show that if c and at most one of the a k s are indefinite while other a k s are positive semidefinite then the ratio between the optimal value of 2 and its sdp relaxation is bounded from below by o 1 log m for both the real and the complex case this result improves the bound based on the so called approximate s lemma of ben tal nemirovski and roos siam j optim 13 2002 pp 535 560 when two or more of the a k s in 2 are indefinite we derive a general bound in terms of the problem data and the sdp solution for both optimization models we present examples to show that the derived approximation bounds are essentially tight

deterministic approximation algorithms for sphere constrained homogeneous polynomial optimization problems due to their fundamental nature and numerous applications sphere constrained polynomial optimization problems have received a lot of attention lately in this paper we consider three such problems i maximizing a homogeneous polynomial over the sphere ii maximizing a multilinear form over a cartesian product of spheres and iii maximizing a multiquadratic form over a cartesian product of spheres since these problems are generally intractable our focus is on designing polynomial time approximation algorithms for them by reducing the above problems to that of determining the l 2 diameters of certain convex bodies we show that they can all be approximated to within a factor of log n n d 2 1 deterministically where n is the number of variables and d is the degree of the polynomial this improves upon the currently best known approximation bound of 1 n d 2 1 in the literature we believe that our approach will find further applications in the design of approximation algorithms for polynomial optimization problems with provable guarantees

approximating quadratic programming with bound and quadratic constraints we consider the problem of approximating the global maximum of a quadratic program qp subject to bound and simple quadratic constraints based on several early results we show that a 4 7 approximate solution can be obtained in polynomial time

maximum principle for singular stochastic control problems in this paper an optimal singular stochastic control problem is considered for this model a general stochastic maximum principle is obtained by using a time transformation this is the first version of the stochastic maximum principle that covers nonlinear cases

portfolio selection with transaction costs in this paper optimal consumption and investment decisions are studied for an investor who has available a bank account paying a fixed rate of interest and a stock whose price is a log normal diffusion this problem was solved by merton and others when transactions between bank and stock are costless here we suppose that there are charges on all transactions equal to a fixed percentage of the amount transacted it is shown that the optimal buying and selling policies are the local times of the two dimensional process of bank and stock holdings at the boundaries of a wedge shaped region which is determined by the solution of a nonlinear free boundary problem an algorithm for solving the free boundary problem is given

stochastic equations with delay optimal control via bsdes and regular solutions of hamilton jacobi bellman equations we consider an ito stochastic differential equation with delay driven by brownian motion whose solution by an appropriate reformulation defines a markov process x with values in a space of continuous functions mathbf c with generator mathcal l we then consider a backward stochastic differential equation depending on x with unknown processes y z and we study properties of the resulting system in particular we identify the process z as a deterministic functional of x we next prove that the forward backward system provides a suitable solution to a class of parabolic partial differential equations on the space mathbf c driven by mathcal l and we apply this result to prove a characterization of the fair price and the hedging strategy for a financial market with memory effects we also include applications to optimal stochastic control of differential equation with delay in particular we characterize optimal controls as feedback laws in terms of the process x

optimal filtering for linear state delay systems in this note the optimal filtering problem for linear systems with state delay over linear observations is treated proceeding from the general expression for the stochastic ito differential of the optimal estimate and the error variance as a result the optimal estimate equation similar to the traditional kalman bucy one is derived however it is impossible to obtain a system of the filtering equations that is closed with respect to the only two variables the optimal estimate and the error variance as in the kalman bucy filter the resulting system of equations for determining the error variance consists of a set of equations whose number is specified by the ratio between the current filtering horizon and the delay value in the state equation and increases as the filtering horizon tends to infinity in the example performance of the designed optimal filter for linear systems with state delay is verified against the best kalman bucy filter available for linear systems without delays and two versions of the extended kalman bucy filter for time delay systems

approximation algorithms for homogeneous polynomial optimization with quadratic constraints in this paper we consider approximation algorithms for optimizing a generic multi variate homogeneous polynomial function subject to homogeneous quadratic constraints such optimization models have wide applications e g in signal processing magnetic resonance imaging mri data training approximation theory and portfolio selection since polynomial functions are non convex the problems under consideration are all np hard in general in this paper we shall focus on polynomial time approximation algorithms in particular we first study optimization of a multi linear tensor function over the cartesian product of spheres we shall propose approximation algorithms for such problem and derive worst case performance ratios which are shown to be dependent only on the dimensions of the model the methods are then extended to optimize a generic multi variate homogeneous polynomial function with spherical constraint likewise approximation algorithms are proposed with provable approximation performance ratios furthermore the constraint set is relaxed to be an intersection of co centered ellipsoids namely we consider maximization of a homogeneous polynomial over the intersection of ellipsoids centered at the origin and propose polynomial time approximation algorithms with provable worst case performance ratios numerical results are reported illustrating the effectiveness of the approximation algorithms studied