a new neural network for solving linear and quadratic programming problems a new neural network for solving linear and quadratic programming problems is presented and is shown to be globally convergent the new neural network improves existing neural networks for solving these problems it avoids the parameter turning problem it is capable of achieving the exact solutions and it uses only simple hardware in which no analog multipliers for variables are required furthermore the network solves both the primal problems and their dual problems simultaneously

prefiltering based esprit for estimating sinusoidal parameters in non gaussian arma noise addresses the esprit algorithm for estimating sinusoidal parameters in non gaussian arma noise the authors show that after prefiltering the output data via the estimated ar polynomial of the noise model a new esprit based on some extensions of esprit can be used to estimate sinusoidal parameters

parameter estimation problems with singular information matrices the case of a singular fisher information matrix fim represents a significant complication for the theory of the cramer rao lower bound crb that is usually handled by resorting to the pseudoinverse of the fisher matrix we take a different approach in which the crb is derived as the solution to an unconstrained quadratic maximization problem which enables us to handle the singular case in a simple yet rigorous manner when the fisher matrix is singular except under unusual circumstances any estimator having the specified bias derivatives that figure in the crb must have infinite variance

optimized estimation of moments for nonstationary signals the behavior of nonstationary moment estimators that are based on linear time invariant filtering is analyzed the performance of such estimators is evaluated in terms of their time averaged variance and time averaged squared bias optimal estimators that minimize a convex combination of bias and variance are derived the superiority of such optimally weighted filtering over the conventional exponentially windowed moment estimation technique is demonstrated by means of a simple example the same example also serves to illustrate the difficulties encountered when the construction of optimal estimators relies on uncertain prior information as well as to demonstrate the feasibility of overcoming such difficulties by using appropriately designed robust optimal estimators we also include a brief discussion of the relevance of the proposed moment estimation technique to identification of linear time variant systems and to estimation of time variant spectra

an algorithm for the minimization of mixed l sub 1 and l sub 2 norms with application to bayesian estimation the regularizing functional approach is widely used in many estimation problems in practice the solution is defined as one minimum point of a suitable functional the main part of which accounts for the underlying physical model whereas the regularizing part represents some prior information about the unknowns in the bayesian interpretation one has a maximum a posteriori map estimator in which the main and regularizing parts are represented respectively by likelihood and prior distributions when either the prior or likelihood is a laplace distribution and the other is a gaussian distribution one is led to consider functionals that include both absolute and square norms the authors present a characterization of the minimum points of such functionals together with a descent type algorithm for numerical computations the results of monte carlo simulations are also reported

metric geometry in homogeneous spaces of the unitary group of a c algebra part i minimal curves abstract we study the metric geometry of homogeneous spaces p of the unitary group of a c algebra a modulo the unitary group of a c subalgebra b where the invariant finsler metric is induced by the quotient norm of a b the main results include the following in the von neumann algebra context for each tangent vector x at a point p there is a geodesic t 0 x which is obtained by the action on of a 1 parameter group in the unitary group of a this geodesic is minimizing up to length 2

geometry of midpoint sets for thompson s metric abstract it is well known that the convex cone p m of m m positive definite matrices is a cartan hadamard riemannian manifold with respect to the riemannian trace metric where the geometric mean of two positive definite matrices coincides with the unique metric midpoint between them in this paper we consider the thompson metric on p m inherited from the spectral norm and study some geometric structures of the thompson midpoints we prove that there is a unique midpoint minimal geodesic between a and b if and only if the spectrum of a 1 b is contained in a a 1 for some a 0 and the set of thompson midpoints between a and b is compact and is convex in both riemannian and euclidean sense it is further shown that the set of all weighted midpoints between a and b is compact and convex in riemannian sense

a generalized least absolute deviation method for parameter estimation of autoregressive signals this paper proposes a generalized least absolute deviation glad method for parameter estimation of autoregressive ar signals under non gaussian noise environments the proposed glad method can improve the accuracy of the estimation of the conventional least absolute deviation lad method by minimizing a new cost function with parameter variables and noise error variables compared with second and high order statistical methods the proposed glad method can obtain robustly an optimal ar parameter estimation without requiring the measurement noise to be gaussian moreover the proposed glad method can be implemented by a cooperative neural network nn which is shown to converge globally to the optimal ar parameter estimation within a finite time simulation results show that the proposed glad method can obtain more accurate estimates than several well known estimation methods in the presence of different noise distributions

on the value of information in system identification bounded noise case assuming instantaneous bounds on the noise system parameter identification is formulated as membership set estimation problem sequential algorithms are constructed to estimate the membership sets of the parameters which are consistent with the measurements and the noise constraints the important new feature of the proposed algorithms is their ability to ignore redundant data the efficient data extraction property of the new algorithms is achieved with small computational effort and with improved performance when compared to the least square algorithm the convergence properties and the notion of identifiability in the set theoretic context are also studied

cumulant based order determination of non gaussian arma models the objective is the order determination of non gaussian and nonminimum phase autoregressive moving average arma models using higher order cumulant statistics the two methods developed assume knowledge of upper bounds on the arma orders the first method performs a linear dependence search among the columns of a higher order statistics matrix by means of the gram schmidt orthogonalization procedure in the second method the order of the ar part is found as the rank of the matrix formed by the higher order statistics sequence for numerically robust rank determination the singular value decomposition approach is adopted the argument principle and samples of the polyspectral phase are used to obtain the relative degree of the arma model from which the order of the ma part can be determined statistical analysis is included for determining the correct ma order with high probability when estimates of third order cumulants are only available simulations are used to verify the performance of the methods and compare autocorrelation with cumulant based order determination approaches

determination of the ma order of an arma process using sample correlations it is shown that autoregressive information and moving average information are implicitly contained in two different correlation matrices an algorithm for ma order determination which adopts only autocorrelations and the ar order without requiring ar coefficients is proposed numerical simulations are presented to show the practical value of the proposed singular value decomposition svd based algorithm

strongly consistent identification algorithms and noise insensitive mse criteria windowed cumulant projections of nongaussian linear processes yield autocorrelation estimators which are immune to additive gaussian noise of unknown covariance by establishing strong consistency of these estimators strongly consistent and noise insensitive recursive algorithms are developed for parameter estimation these computationally attractive schemes are shown to be optimal with respect to a modified mean square error mse criterion which implicitly exploits the high signal to noise ratio domain of cumulant statistics the novel mse objective function is expressed in terms of the noisy process but it is shown to be a scalar multiple of the standard mse criterion as if the latter was computed in the absence of noise simulations illustrate the performance of the proposed algorithms and compare them with the conventional algorithms

on the exponential metric increasing property a short and simple proof is given for the inequality that shows that positive definite matrices constitute a riemannian manifold of negative curvature the idea of the proof leads to generalisations to non riemannian metrics and to connections with some well known inequalities of mathematical physics

l sub 1 and l sub infinity minimization via a variant of karmarkar s algorithm simple iterative algorithms are presented for l sub 1 and l sub infinity minimization regression based on a variant of karmarkar s linear programming algorithm although these algorithms are based on entirely different theoretical principles to the popular irls iteratively reweighted least squares algorithm they have almost identical matrix operations also presented are the results of a monte carlo study comparing the numerical convergence properties of the karmarkar algorithm for l sub 1 minimization to those of an irls and a simplex algorithm the test problem involves l sub 1 estimation of ar autoregressive model parameters the karmarkar algorithm outperformed irls by achieving higher numerical accuracy in fewer iterations techniques for reducing the computational cost per iteration of the karmarkar l sub 1 algorithm are discussed

adaptive algorithms for constrained arma signals in the presence of noise a family of algorithms is developed for adaptive parameter estimation of constrained autoregressive moving average arma signals in the presence of noise these algorithms utilize a priori information about the signal s properties such as its spectral type for example low pass bandpass etc or a spatial domain characteristic special applications include modeling of autoregressions ar and signals of known spectral type in the presence of noise signal deconvolution image deblurring and multipath parameter estimation selected results of simulations are included to demonstrate the performance of the algorithms

comparison of some instrumental variable methods consistency and accuracy aspects a general analysis of various instrumental variable methods is given the investigation includes methods for open loop operation as well as methods designed for closed loop operation the instrumental variables are formed as different combinations of inputs delayed inputs delayed outputs filtered inputs and external setpoint variations necessary conditions as well as sufficient conditions for consistency are explicitly derived and discussed the paper also includes some simple counter examples to general consistency for some common instrumental variable methods also asymptotic accuracy properties are reviewed it is e g studied how the use of filters will influence the covariance matrix of the parameter estimates

set membership identification of systems with parametric and nonparametric uncertainty a method for parameter set estimation in which the system model is assumed to contain both parametric and nonparametric uncertainty is presented in the disturbance free case the parameter set estimate is guaranteed to contain the parameter set of the true plant in the presence of stochastic disturbances the parameter set estimate obtained from finite data records is shown to have the property that it contains the true plant parameter set with probability one as the data length tends to infinity

the linear l 1 estimator and the huber m estimator relationships between a linear l1 estimation problem and the huber m estimator problem can be easily established by their dual formulations the least norm solution of a linear programming problem studied by mangasarian and meyer siam j control optim 17 1979 pp 745 752 provides a key link between the dual problems based on the dual formulations we establish a local linearity property of the huber m estimators with respect to the tuning parameter gamma and prove that the solution set of the huber m estimator problem is lipschitz continuous with respect to perturbations of the tuning parameter gamma as a consequence the set of the linear l1 estimators is the limit of the set of the huber m estimators as gamma to 0 thus the huber m estimator problem has many solutions for small tuning parameter gamma if the linear l1 estimation problem has multiple solutions a recursive version of madsen and nielsen s algorithm siam j optim 3 1993 pp 223 235 based on computation of the huber m estimator is proposed for finding a linear l1 estimator

neural network for solving extended linear programming problems a neural network for solving extended linear programming problems is presented and is shown to be globally convergent to exact solutions the proposed neural network only uses simple hardware in which no analog multiplier for variables is required and has no parameter tuning problem finally an application of the neural network to the l sub 1 norm minimization problem is given

finsler geometry and actions of the p schatten unitary groups let p be an even positive integer and u p h the banach lie group of unitary operators u which verify that u 1 belongs to the p schatten ideal b p h let o be a smooth manifold on which u p h acts transitively and smoothly then one can endow o with a natural finsler metric in terms of the p schatten norm and the action of u p h our main result establishes that for any pair of given initial conditions x o and x to x there exists a curve 6 t e t z x in o with z a skew hermitian element in the p schatten class such that 0 x and 0 x which remains minimal as long as t z p 4 moreover is unique with these properties we also show that the metric space o d where d is the rectifiable distance is complete in the process we establish minimality results in the groups u p h and a convexity property for the rectifiable distance as an example of these spaces we treat the case of the unitary orbit o uau u u p h of a self adjoint operator a b h

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

optimal paths for symmetric actions in the unitary group given a positive and unitarily invariant lagrangian mathcal l defined in the algebra of matrices and a fixed time interval 0 t 0 subset mathbb r we study the action defined in the lie group of n times n unitary matrices mathcal u n by mathcal s alpha int 0 t 0 mathcal l dot alpha t dt where alpha 0 t 0 to mathcal u n is a rectifiable curve we prove that the one parameter subgroups of mathcal u n are the optimal paths provided the spectrum of the exponent is bounded by moreover if mathcal l is strictly convex we prove that one parameter subgroups are the unique optimal curves joining given endpoints finally we also study the connection of these results with unitarily invariant metrics in mathcal u n as well as angular metrics in the grassmann manifold

minimal curves in u n and gl n with respect to the spectral and the trace norms abstract consider the lie group of n n complex unitary matrices u n endowed with the bi invariant finsler metric given by the spectral norm x u u x x for any x tangent to a unitary operator u given two points in u n in general there exist infinitely many curves of minimal length in this paper we provide a complete description of such curves and as a consequence we give an equivalent condition for uniqueness similar studies are done for the grassmann manifolds on the other hand consider the cone of n n positive invertible matrices g l n endowed with the bi invariant finsler metric given by the trace norm x 1 a a 1 2 x a 1 2 1 for any x tangent to a g l n in this context also exist infinitely many curves of minimal length in this paper we provide a complete description of such curves proving first a characterization of the minimal curves joining two hermitian matrices x y h n the last description is also used to construct minimal paths in the group of unitary matrices u n endowed with the bi invariant finsler metric x 1 u u x 1 x 1 for any x tangent to u u n we also study the set of intermediate points in all the previous contexts

adaptive ar modeling in white gaussian noise autoregressive ar modeling is widely used in signal processing the coefficients of an ar model can be easily obtained with a least mean square lms prediction error filter however it is known that this filter gives a biased solution when the input signal is corrupted by white gaussian noise treichler 1979 suggested the spl gamma lms algorithm to remedy this problem and proved that the mean weight vector can converge to the wiener solution in this paper we develop a new algorithm that extends works of vijayan et al 1990 for adaptive ar modeling in the presence of white gaussian noise by theoretical analysis we show that the performance of the new algorithm is superior to the spl gamma lms filter simulations are also provided to support our theoretical results

fast adaptive algorithms for ar parameters estimation using higher order statistics time varying statistics in linear filtering and linear estimation problems necessitate the use of adaptive or time varying filters in the solution with the rapid availability of vast and inexpensive computation power models which are non gaussian even nonstationary are being investigated at increasing intensity statistical tools used in such investigations usually involve higher order statistics hos the classical instrumental variable iv principle has been widely used to develop adaptive algorithms for the estimation of arma processes despite the great number of iv methods developed in the literature the cumulant based procedures for pure autoregressive ar processes are almost nonexistent except lattice versions of iv algorithms this paper deals with the derivation and the properties of fast transversal algorithms hence by establishing a relationship between classical iv methods and cumulant based ar estimation problems new fast adaptive algorithms fast transversal recursive instrumental variable ftriv and generalized least mean squares glms are proposed for the estimation of ar processes the algorithms are seen to have better performance in terms of convergence speed and misadjustment even in low snr the extra computational complexity is negligible the performance of the algorithms as well as some illustrative tracking comparisons with the existing adaptive ones in the literature are verified via simulations the conditions of convergence are investigated for the glms

functional reproducibility in nonlinear systems using dynamic compensation a new sufficient condition for the existence of a control to generate a given real analytic function as the output of multivariable nonlinear systems with dynamic compensation is derived furthermore a prefilter right inverse system is constructed to generate the required control

a new order estimation technique for time series modeling a new approach to estimating the order of the autoregressive moving average model is proposed which is based on the approximate stochastic realization introduced in davis and fotopoulos 1991 the present approach is attractive because overparameterization a very common problem in order determination is avoided successfully simulation results are included to illustrate the effectiveness of the proposed order estimation approach