robust data fusion for multisensor detection systems minimax robust data fusion schemes for multisensor detection systems with discrete time observations characterized by statistical uncertainty are developed and analyzed block sequential and serial fusion rules are considered the performance measures used and made robust with respect to the uncertainties include the error probabilities of the hypothesis testing problem in the block fusion case and the error probabilities and expected numbers of samples or sensors in the sequential and serial fusion cases for different sensor observation statistics the minimax robust fusion rules are derived for two asymptotic cases of interest when the number of sensors is large and when the number of times the fusion center collects the local sensor decisions is large moreover for the case of identical sensor observation statistics and a large number of sensors it is shown that there is no loss in optimality if local tests using likelihood ratios and equal thresholds are used in the sequential fusion rule in all situations the robust decision rules at the sensors and the fusion center are shown to make use of likelihood ratios and thresholds that depend on the least favorable probability distributions of the uncertainty class describing the statistics of sensor observations

robust mse equalizer design for mimo communication systems in the presence of model uncertainties this paper considers a robust mean square error mse equalizer design problem for multiple input multiple output mimo communication systems with imperfect channel and noise information at the receiver when the channel state information csi and the noise covariance are known exactly at the receiver a minimum mean square error mmse equalizer can be employed to estimate the transmitted signal however in actual systems it is necessary to take into account channel and noise estimation errors we consider here a worst case equalizer design problem where the goal is to find the equalizer minimizing the equalization mse for the least favorable channel model within a neighborhood of the estimated model the neighborhood is formed by placing a bound on the kullback leibler kl divergence between the actual and estimated channel models lagrangian optimization is used to convert this min max problem into a convex min min problem over a convex domain which is solved by interchanging the minimization order the robust mse equalizer and associated least favorable channel model can then be obtained by solving numerically a scalar convex minimization problem simulation results are presented to demonstrate the mse and bit error rate ber performance of robust equalizers when applied to the least favorable channel model

the entropy theorem via coding bounds the entropy theorem also known as the shannon mcmillan breiman theory or the asymptotic equipartition theorem asserts that for a stationary ergodic finite alphabet process the sequence 1 n log p x sub 1 sup n converges almost surely to the entropy rate h of the process the entropy theorem has been used to establish asymptotic bounds on the performance of noiseless codes here the coding theorems are established without using the entropy theorem and the coding theorems are then used to prove the entropy theorem the principle feature is the direct use of coding ideas to obtain the entropy theorem

bayesian regression with network prior optimal bayesian filtering perspective the recently introduced intrinsically bayesian robust filter ibrf provides fully optimal filtering relative to a prior distribution over an uncertainty class of joint random process models whereas formerly the theory was limited to model constrained bayesian robust filters for which optimization was limited to the filters that are optimal for models in the uncertainty class this paper extends the ibrf theory to the situation where there are both a prior on the uncertainty class and sample data the result is optimal bayesian filtering obf where optimality is relative to the posterior distribution derived from the prior and the data the ibrf theories for effective characteristics and canonical expansions extend to the obf setting a salient focus of the present study is to demonstrate the advantages of bayesian regression within the obf setting over the classical bayesian approach in the context of linear gaussian models

robust mean squared error estimation of multiple signals in linear systems affected by model and noise uncertainties this paper is a continuation of the work in 11 and 2 on the problem of estimating by a linear estimator n unobservable input vectors undergoing the same linear transformation from noise corrupted observable output vectors whereas in the aforementioned papers only the matrix representing the linear transformation was assumed uncertain here we are concerned with the case in which the second order statistics of the noise vectors i e their covariance matrices are also subjected to uncertainty we seek a robust mean squared error estimator immuned against both sources of uncertainty we show that the optimal robust mean squared error estimator has a special form represented by an elementary block circulant matrix and moreover when the uncertainty sets are ellipsoidal like the problem of finding the optimal estimator matrix can be reduced to solving an explicit semidefinite programming problem whose size is independent of n

exact sample conditioned mse performance of the bayesian mmse estimator for classification error part i representation in recent years biomedicine has been faced with difficult high throughput small sample classification problems in such settings classifier error estimation becomes a critical issue because training and testing must be done on the same data a recently proposed error estimator places the problem in a signal estimation framework in the presence of uncertainty permitting a rigorous solution optimal in a minimum mean square error sense the uncertainty in this model is relative to the parameters of the feature label distributions resulting in a bayesian approach to error estimation closed form solutions are available for two important problems discrete classification with dirichlet priors and linear classification of gaussian distributions with normal inverse wishart priors in this work part i of a two part study we introduce the theoretical mean square error mse conditioned on the observed sample of any estimate of the classifier error including the bayesian error estimator for both bayesian models thus bayesian error estimation has a unique advantage in that its mathematical framework naturally gives rise to a practical expected measure of performance given an observed sample in part ii of the study we examine consistency of the error estimator demonstrate various mse properties and apply the conditional mse to censored sampling

minimax mse ratio estimation with signal covariance uncertainties in continuation to an earlier work we further consider the problem of robust estimation of a random vector or signal with an uncertain covariance matrix that is observed through a known linear transformation and corrupted by additive noise with a known covariance matrix while in the earlier work we developed and proposed a competitive minimax approach of minimizing the worst case mean squared error mse difference regret criterion here we study in the same spirit the minimum worst case mse ratio regret criterion namely the worst case ratio rather than difference between the mse attainable using a linear estimator ignorant of the exact signal covariance and the minimum mse mmse attainable by optimum linear estimation with a known signal covariance we present the optimal linear estimator under this criterion in two ways the first is as a solution to a certain semidefinite programming sdp problem and the second is as an expression that is of closed form up to a single parameter whose value can be found by a simple line search procedure we then show that the linear minimax ratio regret estimator can also be interpreted as the mmse estimator that minimizes the mse for a certain choice of signal covariance that depends on the uncertainty region we demonstrate that in applications the proposed minimax mse ratio regret approach may outperform the well known minimax mse approach the minimax mse difference regret approach and the plug in approach where in the latter one uses the mmse estimator with an estimated covariance matrix replacing the true unknown covariance

a group of non uniform exponential growth locally isomorphic to img z 2 i we prove that a sequence of marked three generated groups isomorphic to the iterated monodromy group of z 2 i converges to a group of non uniform exponential growth which is an extension of the infinite direct sum of cyclic groups of order 4 by a grigorchuk group

minimax mse estimation of deterministic parameters with noise covariance uncertainties in this paper a minimax mean squared error mse estimator is developed for estimating an unknown deterministic parameter vector in a linear model subject to noise covariance uncertainties the estimator is designed to minimize the worst case mse across all norm bounded parameter vectors and all noise covariance matrices in a given region of uncertainty the minimax estimator is shown to have the same form as the estimator that minimizes the worst case mse over all norm bounded vectors for a least favorable choice of the noise covariance matrix an example demonstrating the performance advantage of the minimax mse approach over the least squares and weighted least squares methods is presented

on minimax robustness a general approach and applications the minimax approach to the design of systems that are robust with respect to modeling uncertainties is studied using a game theoretic formulation in which the performance functional and the sets of modeling uncertainties and admissible design policies are arbitrary the existence and characterization of minimax robust solutions that form saddle points are discussed through various methods that take into account several common features of the games encountered in applications in particular it is shown that if the performance functional and the uncertainty set are convex then a certain type of regularity condition on the functional is sufficient to ensure that the optimal strategy for a least favorable element of the uncertainty set is minimax robust the efficacy of the methods proposed for a general game is tested in the problems of matched filtering wiener filtering quadratic detection and output energy filtering in which uncertainties in their respective signal and noise models are assumed to exist these problems are analyzed in a common hilbert space framework and they serve to point out the advantages and limitations of the proposed techniques

optimal robust classifiers qualitatively a filter is said to be robust if its performance degradation is acceptable for distributions close to the one for which it is optimal that is the one for which it has been designed this paper adapts the signal processing theory of optimal robust filters to classifiers the distribution class conditional distributions to which the classifier is to be applied is parameterized by a state vector and the principle issue is to choose a design state that is optimal in comparison to all other states relative to some measure of robustness a minimax robust classifier is one whose worst performance over all states is better than the worst performances of the other classifiers defined at the other states a bayesian robust classifier is one whose expected performance is better than the expected performances of the other classifiers the state corresponding to the bayesian robust classifier is called the maximally robust state minimax robust classifiers tend to give too much weight to states for which classification is very difficult and therefore our effort is focused on bayesian robust classifiers whereas the signal processing theory of robust filtering concentrates on design with full distributional knowledge and a fixed number of observation variables features design via training from sample data and feature selection are so important for classification that robustness optimality must be considered from these perspectives in particular for small samples in this context for a given sample size we will be concerned with the maximally robust state feature pair all definitions are independent of the classification rule however applications are only considered for linear and quadratic discriminant analysis for which there are parametric forms for the optimal discriminants

optimal and robust memoryless discrimination from dependent observations discrimination is considered between two possible sources based on dependent observations of their output the discrimination problem is modeled by means of a general binary hypothesis test the main emphasis being on situations that cannot be modeled as signals in additive noise the observations are modeled as stationary m dependent or rho mixing processes the structure of the discriminator is such that the observations are passed through a memoryless nonlinearity summed up to form a test statistic which is then compared to a threshold only fixed sample size tests are considered four different performance measures which resemble the signal to noise ratios encountered in the signal in additive noise problems are derived under different problem formulations the optimal nonlinearities for each of the performance measures are derived as solutions to various integral equations for three of the four performance measures the authors have successfully obtained robust nonlinearities for uncertainty in the marginal and the joint probability density functions of the observations computer simulation results that demonstrate the advantage of using these nonlinearities over the i i d nonlinearity under the probability of error criterion are presented

linear minimax regret estimation of deterministic parameters with bounded data uncertainties we develop a new linear estimator for estimating an unknown parameter vector x in a linear model in the presence of bounded data uncertainties the estimator is designed to minimize the worst case regret over all bounded data vectors namely the worst case difference between the mean squared error mse attainable using a linear estimator that does not know the true parameters x and the optimal mse attained using a linear estimator that knows x we demonstrate through several examples that the minimax regret estimator can significantly increase the performance over the conventional least squares estimator as well as several other least squares alternatives

maximum entropy and robust prediction on a simplex the related problems of finite length robust prediction and maximizing spectral entropy over a simplex of covariance matrices are considered general properties of iterative solutions of these problems are developed and monotone convergence proofs are presented for two algorithms that provide such solutions the analogous problems for simplexes of spectral densities are also considered

a probabilistic approach to multivariable robust filtering and open loop control a new approach to robust filtering prediction and smoothing of discrete time signal vectors is presented linear time invariant filters are designed to be insensitive to spectral uncertainty in signal models the goal is to obtain a simple design method leading to filters which are not overly conservative modeling errors are described by sets of models parameterized by random variables with known covariances these covariances could either be estimated from data or be used as robustness tuning knobs a robust design is obtained by minimizing the spl hscrsub 2 norm or equivalently the mean square estimation error averaged with respect to the assumed model errors a polynomial solution based on an averaged spectral factorization and a unilateral diophantine equation is derived the robust estimator is referred to as a cautious wiener filter it turns out to be only slightly more complicated to design than an ordinary wiener filter the methodology can be applied to any open loop filtering or control problem in particular we illustrate this for the design of robust multivariable feedforward regulators decoupling and model matching filters

minimax estimation of deterministic parameters in linear models with a random model matrix we consider the problem of estimating an unknown deterministic parameter vector in a linear model with a random model matrix with known second order statistics we first seek the linear estimator that minimizes the worst case mean squared error mse across all parameter vectors whose possibly weighted norm is bounded above we show that the minimax mse estimator can be found by solving a semidefinite programming problem and develop necessary and sufficient optimality conditions on the minimax mse estimator using these conditions we derive closed form expressions for the minimax mse estimator in some special cases we then demonstrate through examples that the minimax mse estimator can improve the performance over both a baysian approach and a least squares method we then consider the case in which the norm of the parameter vector is also bounded below since the minimax mse approach cannot account for a nonzero lower bound we consider in this case a minimax regret method in which we seek the estimator that minimizes the worst case difference between the mse attainable using a linear estimator that does not know the parameter vector and the optimal mse attained using a linear estimator that knows the parameter vector for analytical tractability we restrict our attention to the scalar case and develop a closed form expression for the minimax regret estimator

not residually finite groups of intermediate growth commensurability and non geometricity abstract we construct a series of not residually finite groups of intermediate growth in particular we construct groups of intermediate growth that are not commensurable up to finite kernels with any residually finite group also we show that certain properties of groups among those solvability of the word problem and the property of having torsion of unbounded order are not weakly geometric the argument is based in particular on studying commensurability up to finite kernels

a competitive minimax approach to robust estimation of random parameters we consider the problem of estimating in the presence of model uncertainties a random vector x that is observed through a linear transformation h and corrupted by additive noise we first assume that both the covariance matrix of x and the transformation h are not completely specified and develop the linear estimator that minimizes the worst case mean squared error mse across all possible covariance matrices and transformations h in the region of uncertainty although the minimax approach has enjoyed widespread use in the design of robust methods we show that its performance is often unsatisfactory to improve the performance over the minimax mse estimator we develop a competitive minimax approach for the case where h is known but the covariance of x is subject to uncertainties and seek the linear estimator that minimizes the worst case regret namely the worst case difference between the mse attainable using a linear estimator ignorant of the signal covariance and the optimal mse attained using a linear estimator that knows the signal covariance the linear minimax regret estimator is shown to be equal to a minimum mse mmse estimator corresponding to a certain choice of signal covariance that depends explicitly on the uncertainty region we demonstrate through examples that the minimax regret approach can improve the performance over both the minimax mse approach and a plug in approach in which the estimator is chosen to be equal to the mmse estimator with an estimated covariance matrix replacing the true unknown covariance we then show that although the optimal minimax regret estimator in the case in which the signal and noise are jointly gaussian is nonlinear we often do not lose much by restricting attention to linear estimators

robust competitive estimation with signal and noise covariance uncertainties robust estimation of a random vector in a linear model in the presence of model uncertainties has been studied in several recent works while previous methods considered the case in which the uncertainty is in the signal covariance and possibly the model matrix but the noise covariance is assumed to be completely specified here we extend the results to the case where the noise statistics may also be subjected to uncertainties we propose several different approaches to robust estimation which differ in their assumptions on the given statistics in the first method we assume that the model matrix and both the signal and the noise covariance matrices are uncertain and develop a minimax mean squared error mse estimator that minimizes the worst case mse in the region of uncertainty the second strategy assumes that the model matrix is given and tries to uniformly approach the performance of the linear minimum mse estimator that knows the signal and noise covariances by minimizing a worst case regret measure the regret is defined as the difference or ratio between the mse attainable using a linear estimator ignorant of the signal and noise covariances and the minimum mse possible when the statistics are known as we show earlier solutions follow directly from our more general results however the approach taken here in developing the robust estimators is considerably simpler than previous methods

the rate distortion function on classes of sources determined by spectral capacities the quantity sup a in cal r r a d is considered where cal r is a class of homogeneous n parameter sources and r a d denotes the single letter mean square error mse rate distortion function for the individual source a in particular the case in which the class cal r is specified in terms of spectral information is treated for general classes of spectral measures whose upper measures are capacities in the sense of choquet alternating of order two this type of class includes many common models for spectral uncertainty such as mixture models spectral band models and neighborhoods generated by variation and prohorov metrics it is shown that each such class contains a worst case source whose rate distortion function achieves the supremum over the class for each value of distortion this source is characterized as having a spectral density that is a derivative in the sense of huber and strassen of the upper spectral measure with respect to lebesgue measure on pi pi n moreover it is shown that the spectral measure of the worst case source is closest in a sense defined by directed divergence to lebesgue measure which corresponds to a memoryless source numerical results are presented for the particular case in which the source spectral measure is a mixture of a gauss markov spectrum and an unknown contaminating component

robust least squares estimation with a relative entropy constraint given a nominal statistical model we consider the minimax estimation problem consisting of finding the best least squares estimator for the least favorable statistical model within a neighborhood of the nominal model the neighborhood is formed by placing a bound on the kullback leibler kl divergence between the actual and nominal models for a gaussian nominal model and a finite observations interval or for a stationary gaussian process over an infinite interval the usual noncausal wiener filter remains optimal however the worst case performance of the filter is affected by the size of the neighborhood representing the model uncertainty on the other hand standard causal least squares estimators are not optimal and a characterization is provided for the causal estimator and the corresponding least favorable model the causal estimator takes the form of a risk sensitive estimator with an appropriately selected risk sensitivity coefficient

robust wiener filtering with non parametric spectral uncertainty in this article robust least squares filtering problems are considered for non parametric multivariate spectral uncertainty defined by the so called spectral band and generalised moment constraints its major aim is to provide a basis for computing approximate solutions to worst case wiener filtering optimisation problems involving causal filters and multivariate signals it hinges upon associating upper and lower bounds on the minimum worst case performance achievable with causal filters with linear cost linear matrix inequality lc lmi constraint optimisation problems on the basis of a lagrangean duality formulation for the worst case least squares performance of a given filter upper bounds on it are obtained as the optimal values of lc lmi problems then for linearly parameterised classes of filter transfer functions a causal filter which optimises such an upper bound on worst case performance can also be obtained from an lc lmi optimisation problem to estimate the amount of conservatism incur

minimax robust coding for channels with uncertainty statistics the problem of minimax robust coding for classes of channels with uncertainty in their statistical description is addressed specific consideration is given to 1 discrete memoryless channels with uncertainty in the probability transition matrices 2 discrete time stationary gaussian channels with spectral uncertainty and to uncertainty with classes determined by 2 alternating choquet capacities both block codes and convolutional codes are considered a robust maximum likelihood decoding rule is derived the rule guarantees that for all channels in the uncertainty class and all rates smaller than a critical rate the average probability of decoding error for the ensemble of random block codes and the ensemble of random time varying convolutional codes converges to zero exponentially with increasing block length or constraint length respectively the channel capacity and cut off rate of the class are then evaluated

robust wiener kolmogorov theory a minimax formulation is considered for the problem of designing robust linear causal estimators of linear functions of discrete time wide sense stationary signals when knowledge of the signal and or noise spectra is inexact the solution is given under mild regularity conditions in terms of a least favorable pair of spectra thus reducing the minimax problem to a direct maximization problem which in many cases can be solved easily it is noted that this design method leads in particular to robust n step predictors robust causal filters and robust n lag smoothers the method of design is illustrated by a thorough development of the special case of one step noiseless prediction further solutions are given explicitly for the problem of robust causal filtering of an uncertain signal in white noise and numerical examples are given for this case which illustrate the effectiveness of this design

nonlinear estimation for a class of systems this paper considers nonlinear estimation problems for classes of models and employs relative entropy to describe the uncertainty classes two optimization problems are formulated on general banach spaces and their solutions are sought 1 when the transition probability between the signal to be estimated x and the measurement y or stochastic kernel is unknown and 2 when the joint probability induced by the random variables rvs x y is unknown for both problems the uncertainty is described by a relative entropy constraint between the unknown distribution and a fixed nominal distribution the results include existence of the optimal measures using weak convergence techniques and properties associated with the estimate of the true distribution classical examples are chosen to illustrate the applicability of the results

robust deconvolution of deterministic and random signals we consider the problem of designing an estimation filter to recover a signal x n convolved with a linear time invariant lti filter h n and corrupted by additive noise our development treats the case in which the signal x n is deterministic and the case in which it is a stationary random process both formulations take advantage of some a priori knowledge on the class of underlying signals in the deterministic setting the signal is assumed to have bounded weighted energy in the stochastic setting the power spectra of the signal and noise are bounded at each frequency the difficulty encountered in these estimation problems is that the mean squared error mse at the output of the estimation filter depends on the problem unknowns and therefore cannot be minimized beginning with the deterministic setting we develop a minimax mse estimation filter that minimizes the worst case point wise mse between the true signal x n and the estimated signal over the class of bounded norm inputs we then establish that the mse at the output of the minimax mse filter is smaller than the mse at the output of the conventional inverse filter for all admissible signals next we treat the stochastic scenario for which we propose a minimax regret estimation filter to deal with the power spectrum uncertainties this filter is designed to minimize the worst case difference between the mse in the presence of power spectrum uncertainties and the mse of the wiener filter that knows the correct power spectra the minimax regret filter takes the entire uncertainty interval into account and as demonstrated through an example can often lead to improved performance over traditional minimax mse approaches for this problem

optimal control of uncertain stochastic systems subject to total variation distance uncertainty this paper is concerned with optimization of uncertain stochastic systems in which uncertainty is described by a total variation distance constraint between the measures induced by the uncertain systems and the measure induced by the nominal system while the payoff is a linear functional of the uncertain measure robustness at the abstract setting is formulated as a minimax game in which the control seeks to minimize the payoff over the admissible controls while the uncertainty aims at maximizing it over the total variation distance constraint it is shown that the maximizing measure in the total variation distance constraint exists while the resulting payoff is a linear combination of l 1 and l infty norms further the maximizing measure is characterized by a linear combination of a tilted measure and the nominal measure giving rise to a payoff which is a nonlinear functional on the space of measures to be minimized over the admissible controls the abstract formulation and results are subsequ

robust filtering and feedforward control based on probabilistic descriptions of model errors abstract a new approach to robust estimation of signals prediction of time series and robust feedforward control is considered modelling errors are parametrized by random variables with known covariances a robust design is obtained by minimizing the squared estimation error averaged both with respect to model errors and the noise a polynomial equations approach based on averaged spectral factorizations and averaged diophantine equations is derived mild solvability conditions guarantee the existence of stable optimal filters and feedforward regulators the robust design turns out to be no more complicated than the design of an ordinary wiener filter or lqg regulator the proposed approach avoids two drawbacks of minimax design first probabilistic descriptions of model uncertainties may have soft bounds these are more readily obtainable in a noisy environment than the hard bounds required for minimax design furthermore not only the range of uncertainties but also their likelihood is taken into account common model deviations will have a greater impact on an estimator design than do very rare worst cases the conservativeness is thus reduced

robust matched filters for noise uncertainty within two alternating capacity classes the problem of designing matched filters that are robust against uncertainty in the statistics of a noise process modeled by two alternating choquet capacities is addressed the robust design is based on the maximum signal to noise ratio the problem is formulated and solved for both discrete time and continuous time matched filters with uncertainty in either the autocorrelation function or the spectral measure of the noise explicit solutions that are characterized by the hyber strassen derivative of the capacity generating the class with respect to a lebesgue like measure on a suitable interval are obtained

robust prediction and interpolation for vector stationary processes robust multivariate prediction and interpolation problems for statistically contaminated vector valued second order stationary processes are considered the statistical contamination is modeled by requiring that the spectral density matrices of the processes lie within certain nonparametric classes both prediction and interpolation are then formalized as games whose saddle point solutions are sought finally such solutions are found and analyzed for two specific multivariate spectral classes

nonideal sampling and interpolation from noisy observations in shift invariant spaces digital analysis and processing of signals inherently relies on the existence of methods for reconstructing a continuous time signal from a sequence of corrupted discrete time samples in this paper a general formulation of this problem is developed that treats the interpolation problem from ideal noisy samples and the deconvolution problem in which the signal is filtered prior to sampling in a unified way the signal reconstruction is performed in a shift invariant subspace spanned by the integer shifts of a generating function where the expansion coefficients are obtained by processing the noisy samples with a digital correction filter several alternative approaches to designing the correction filter are suggested which differ in their assumptions on the signal and noise the classical deconvolution solutions least squares tikhonov and wiener are adapted to our particular situation and new methods that are optimal in a minimax sense are also proposed the solutions often have a similar structure and can be computed simply and efficiently by digital filtering some concrete examples of reconstruction filters are presented as well as simple guidelines for selecting the free parameters e g regularization of the various algorithms

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

on the algorithmic solvability of spectral factorization and applications spectral factorization is an operation which appears in many different engineering applications this paper studies whether spectral factorization can be algorithmically computed on an abstract machine a turing machine it is shown that there exist computable spectral densities with very good analytic properties i e smooth with finite energy such that the corresponding spectral factor cannot be determined on a turing machine further it will be proved that it is impossible to decide algorithmically whether or not a given computable density possesses a computable spectral factor this negative result has consequences for applications of spectral factorization in computer aided design because there it is necessary that this problem be decidable conversely this paper will show that if the logarithm of a computable spectral density belongs to certain sobolev space of sufficiently smooth functions then the spectral factor is always computable as an application the paper discusses the possibility of calculating the optimal causal wiener filter on an abstract machine

intrinsically bayesian robust kalman filter an innovation process approach in many contemporary engineering problems model uncertainty is inherent because accurate system identification is virtually impossible owing to system complexity or lack of data on account of availability time or cost the situation can be treated by assuming that the true model belongs to an uncertainty class of models in this context an intrinsically bayesian robust ibr filter is one that is optimal relative to the cost function in the classical sense and the prior distribution over the uncertainty class in the bayesian sense ibr filters have previously been found for both wiener and granulometric morphological filtering in this paper we derive the ibr kalman filter that performs optimally relative to an uncertainty class of state space models introducing the notion of bayesian innovation process and the bayesian orthogonality principle we show how the problem of designing an ibr kalman filter can be reduced to a recursive system similar to the classical kalman recursive equations except with effective counterparts such as the effective kalman gain matrix after deriving the recursive ibr kalman equations for discrete time we use the limiting method to obtain the ibr kalman bucy equations for continuous time finally we demonstrate the utility of the proposed framework for two real world problems sensor networks and gene regulatory network inference

thurston equivalence of topological polynomials we answer hubbard s question on determining the thurston equivalence class of twisted rabbits i e composita of the rabbit polynomial with nth powers of the dehn twists about its ears the answer is expressed in terms of the 4 adic expansion of n we also answer the equivalent question for the other two families of degree 2 topological polynomials with three post critical points in the process we rephrase the questions in group theoretical language in terms of wreath recursions

on robust wiener signal estimation a minimax formulation is considered for the problem of designing robust linear causal estimators of continuous time signals in noise it is shown that the minimax estimation problem can generally be reduced to finding a least favorable pair of spectra in particular robust predictors filters and finite lag smoothers are shown to be just the optimal predictors filters and smoothers respectively for a least favorable pair this result can often greatly simplify the design of robust estimators

the ergodic and entropy theorems revisited a new proof of birkhoff s ergodic theorem is given using a sample path covering idea an idea created by ornstein and weiss in their extension of the information convergence theorem to random fields a sketch of the ornstein weiss proof for processes is included in the appendix

extremum problems with total variation distance and their applications the aim of this paper is to investigate extremum problems with pay off being the total variation distance metric defined on the space of probability measures subject to linear functional constraints on the space of probability measures and vice versa that is with the roles of total variation metric and linear functional interchanged utilizing concepts from signed measures the extremum probability measures of such problems are obtained in closed form by identifying the partition of the support set and the mass of these extremum measures on the partition the results are derived for abstract spaces specifically complete separable metric spaces known as polish spaces while the high level ideas are also discussed for denumerable spaces endowed with the discrete topology these extremum problems often arise in many areas such as approximating a family of probability distributions by a given probability distribution maximizing or minimizing entropy subject to total variation distance metric constraints quantifying uncertainty of probability distributions by total variation distance metric stochastic minimax control and in many problems of information decision theory and minimax theory

optimal classifiers with minimum expected error within a bayesian framework part ii properties and performance analysis in part i of this two part study we introduced a new optimal bayesian classification methodology that utilizes the same modeling framework proposed in bayesian minimum mean square error mmse error estimation optimal bayesian classification thus completes a bayesian theory of classification where both the classifier error and our estimate of the error may be simultaneously optimized and studied probabilistically within the assumed model having developed optimal bayesian classifiers in discrete and gaussian models in part i here we explore properties of optimal bayesian classifiers in particular invariance to invertible transformations convergence to the bayes classifier and a connection to bayesian robust classifiers we also explicitly derive optimal bayesian classifiers with non informative priors and explore relationships to linear and quadratic discriminant analysis lda and qda which may be viewed as plug in rules under gaussian modeling assumptions finally we present several simulations addressing the robustness of optimal bayesian classifiers to false modeling assumptions companion website http gsp tamu edu publications supplementary dalton12a

robust matched filters for optical receivers the problem of designing optical receivers that are robust against uncertainty in the statistics of the observation process in photodetection is investigated in particular a modification in the design of the postdetection matched filter is proposed to account for possible uncertainty in the rate function of the incident light the rate of the dark current and in the statistics of the additive noise present at the input to the optical receiver this design is based on a game theoretic approach in which a filter is sought that has the maximum worst case output signal to noise ratio possible over the class of allowable statistics that is the design criterion is maximin signal to noise ratio a general characterization of maximin robust matched filters for observed poisson processes is presented in this context and specific solutions for several useful uncertainty models are obtained numerical results are presented for a specific example to illustrate the performance of the proposed technique

robust hypothesis testing for bounded classes of probability densities corresp least favorable pairs of probability density functions and robust tests are obtained for classes of density functions specified as bands with upper and lower bounds the results are shown to extend huber s solution for the epsilon contaminated classes which is contained as a special case a minimum distance property of the least favorable density pair is also made explicit

on the p point uncertainty class corresp the p point uncertainty class also known as sakrison s class b is considered although the p point class is not a 2 alternating choquet capacity class it is shown how many of the robust signal processing and communication theory results which have been developed for capacity classes can be applied to the p point class by embedding it in a slightly larger extended p point class

minimax robust prediction of discrete time series we discuss a robust approach for predicting a weakly stationary discrete time series whose spectral density f is not exactly known we assume that we know that f mathfrak d where mathfrak d is a convex set of spectral densities fulfilling some not too stringent conditions we proof the existence of a most indeterministic density f0 in mathfrak d and we show that the classical optimal linear predictor for a time series with spectral density f0 is mini max robust in the sense that it minimizes the maximal possible prediction error

brief paper an analysis of the effects of spectral uncertainty on wiener filtering a representative selection of results from an extensive study of the performance of wiener filtering under spectral uncertainty is presented for a variety of spectral uncertainty models the wiener filter is shown to have undesirable sensitivity to even small deviations from those signal and noise spectral densities which were used to design the filter performance of a robust filter designed to have the best possible performance when the uncertainty is worst is also examined in most cases the robust filter s insensitivity to spectral uncertainty makes it preferable to the traditional wiener filter

minimax linear smoothing for capacities the case of correlated signals and noise results of an earlier paper giving minimax linear smoothers for the problem of estimating a homogeneous signal field in an additive orthogonal noise field when both have uncertain spectral properties are extended to the case in which the signal and noise fields are arbitrarily correlated as before spectral uncertainty is modeled by assuming that the spectral measures of the signal and noise fields lie in classes of measures generated by two alternating choquet capacities it is demonstrated that this problem admits a general solution in terms of the huber strassen derivative between the capacities that generate the uncertainty sets and that the least favorable spectra for smoothing in orthogonal noise are also the least favorable marginal spectra for smoothing in correlated noise the resulting filter is seen to be a zonal filter that also arises as the solution to an analogous problem in nonparametric minimax hypothesis testing these new results extend the applicability of minimax robust smoothing techniques to application involving signal dependent noise phenomena such as multipath and clutter which are usually difficult to model precisely

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

filtering and smoothing in an h sup infinity setting the problems of filtering and smoothing are considered for linear systems in an h sup infinity setting i e the plant and measurement noises have bounded energies are in l sub 2 but are otherwise arbitrary two distinct situations for the initial condition of the system are considered the initial condition is assumed known in one case while in the other the initial condition is not known but the initial condition the plant and measurement noise are in some weighted ball of r sup n xl sub 2 finite horizon and infinite horizon cases are considered necessary and sufficient conditions are presented for the existence of estimators both filters and smoothers that achieve a prescribed performance bound and algorithms that result in performance within the bounds are developed in case of smoothers the optimal smoother is also presented the approach uses basic quadratic optimization theory in a time domain setting as a consequence of which both linear time varying and time invariant systems can be considered with equal ease in the smoothing problem for linear time varying systems one considers only the finite horizon case

constrained solutions in importance via robust statistics the problem of estimating estimating expectations of functions of random vectors via simulation is investigated monte carlo simulations also known as simple averaging have been used as a direct means of estimation a technique known as importance sampling can be used to modify the simulation via weighted averaging in the hope that the estimate will converge more rapidly to the expected value than standard monte carlo simulations a constrained optimal solution to the problem of minimizing the variance of the importance sampling estimator is derived this is accomplished by finding the distribution which is closest to the unconstrained optimal solution in the ali silvey sense s ali et al 1966 the solution from the constraint class is shown to be the least favorable density function in terms of bayes risk against the optimal density function examples of constraint classes which include epsilon mixture show that the constrained optimal solution can be made arbitrarily close to the optimal solution applications to estimating probability of error in communication systems are presented

intrinsically optimal bayesian robust filtering when designing optimal filters it is often unrealistic to assume that the statistical model is known perfectly the issue is then to design a robust filter that is optimal relative to an uncertainty class of processes robust filter design has been treated from minimax best worst case performance and bayesian best average performance perspectives heretofore the bayesian approach has involved finding a model specific optimal filter one that is optimal for some model in the uncertainty class lifting this constraint we optimize over the full class from which the original optimal filters were obtained for instance over all linear filters by extending the original characteristics determining the filter such as the power spectral density to effective characteristics that apply across the uncertainty class we demonstrate for both linear and morphological filtering that an intrinsically optimal bayesian robust filter can be represented in the same form as the standard solution to the optimal filter except via the effective characteristics solutions for intrinsic bayesian robust filters are more transparent and intuitive than solutions for model specific filters and also less tedious because effective characteristics push through the spectral theory into the bayesian setting whereas solutions in the model specific case depend on grinding out the optimization

minimax robust deconvolution filters under stochastic parametric and noise uncertainties the author consider the design of robust deconvolution filters for linear discrete time systems with stochastic parameter and noise uncertainties it is assumed that some large but bounded uncertainties exist in the driving and measurement noise covariances as well as the second order statistics of stochastic parameters and initial conditions three kinds of minimax sensitivity criteria are used to develop the techniques to the synthesis of minimax deconvolution filters under uncertain linear stochastic systems their approach is based on saddle point theory and the sensitivity analysis of kalman filters the design algorithms give the recursive realization of the minimax deconvolution filters for the time varying uncertain systems under fairly general conditions for the time invariant uncertain case the existence and solutions of steady state deconvolution filters are further developed finally the utility of the minimax design approaches and the properties of the resulting minimax deconvolution filters are illustrated by a numerical example

minimax linear smoothing for capacities abstract minimax linear smoothers are considered for the problem of estimating a homogeneous signal field in an additive orthogonal noise field a minimax game with the quadratic mean estimation error as an objective function is used to formulate this problem uncertainty in signal and noise field spectra is modeled using general nonparametric classes of measures proposed by huber and strassen for the problem of minimax hypothesis testing these classes which are described in terms of choquet alternating capacities of order 2 include the conventional models for spectral uncertainty and admit a general solution to the minimax linear smoothing problem author

the interactions between ergodic theory and information theory information theorists frequently use the ergodic theorem likewise entropy concepts are often used in information theory recently the two subjects have become partially intertwined as deeper results from each discipline find use in the other a brief history of this interaction is presented in this paper together with a more detailed look at three areas of connection namely recurrence theory blowing up bounds and direct sample path methods

robust linear filtering for multivariable stationary time series we consider the problem of asymptotic noncausal linear filtering for multivariable second order stationary time series under spectral uncertainty in both the signal and the noise processes the spectral uncertainty is modeled by e contaminated and p point classes for the case where both the signal and noise spectra are e contaminated we assume that the eigenvectors of the corresponding nominal spectral density matrices are identical the problem is formulated as a game whose saddle point solutions are found and analyzed

robust mean squared error estimation in the presence of model uncertainties we consider the problem of estimating an unknown parameter vector x in a linear model that may be subject to uncertainties where the vector x is known to satisfy a weighted norm constraint we first assume that the model is known exactly and seek the linear estimator that minimizes the worst case mean squared error mse across all possible values of x we show that for an arbitrary choice of weighting the optimal minimax mse estimator can be formulated as a solution to a semidefinite programming problem sdp which can be solved very efficiently we then develop a closed form expression for the minimax mse estimator for a broad class of weighting matrices and show that it coincides with the shrunken estimator of mayer and willke with a specific choice of shrinkage factor that explicitly takes the prior information into account next we consider the case in which the model matrix is subject to uncertainties and seek the robust linear estimator that minimizes the worst case mse across all possible values of x and all possible values of the model matrix as we show the robust minimax mse estimator can also be formulated as a solution to an sdp finally we demonstrate through several examples that the minimax mse estimator can significantly increase the performance over the conventional least squares estimator and when the model matrix is subject to uncertainties the robust minimax mse estimator can lead to a considerable improvement in performance over the minimax mse estimator

boundary behavior for groups of subexponential growth in this paper we introduce a method for partial description of the poisson boundary for a certain class of groups acting on a segment as an application we find among the groups of subexponential growth those that admit nonconstant bounded harmonic functions with respect to some symmetric infinitely supported measure of finite entropy h this implies that the entropy h of the corresponding random walk is finite and positive as another application we exhibit certain discontinuity for the recurrence property of random walks finally as a corollary of our results we get new estimates from below for the growth function of a certain class of grigorchuk groups in particular we exhibit the first example of a group generated by a finite state automaton such that the growth function is subexponential but grows faster than exp n for any 1 we show that in some of our examples the growth function satisfies exp n ln

performance bounds for discrimination problems with uncertain statistics corresp decision designs that are insensitive to modeling uncertainty are developed for the chernoff bounds on the performance of binary hypothesis testing problems these designs are based on observations with statistical uncertainty modeled by using general classes generated by 2 alternating capacities the results are illustrated for the two cases of independent identically distributed observations with uncertainty in the probability distribution and discrete time stationary gaussian observations with spectral uncertainty and they are applicable to several other cases as well for the chernoff upper bounds on the error probabilities a robust decision design based on the i kel hood ratio test between a least favorable pair of probability distributions or spectral measures respectively is derived it is then shown that for all elements in the uncertainty class this choice of likelihood ratio guarantees the exponential convergence of the aforementioned chernoff bounds to zero as the number of observations or the length of the observation interval increases

quantifying the objective cost of uncertainty in complex dynamical systems real world problems often involve complex systems that cannot be perfectly modeled or identified and many engineering applications aim to design operators that can perform reliably in the presence of such uncertainty in this paper we propose a novel bayesian framework for objective based uncertainty quantification uq which quantifies the uncertainty in a given system based on the expected increase of the operational cost that it induces this measure of uncertainty called mocu mean objective cost of uncertainty provides a practical way of quantifying the effect of various types of system uncertainties on the operation of interest furthermore the proposed uq framework provides a general mathematical basis for designing robust operators and it can be applied to diverse applications including robust filtering classification and control we demonstrate the utility and effectiveness of the proposed framework by applying it to the problem of robust structural intervention of gene regulatory networks an important application in translational genomics

an rkhs approach to robust l sup 2 estimation and signal detection the application of rkhs reproducing kernel hilbert space theory to the problems of robust signal detection and estimation is investigated it is shown that this approach provides a general and unified framework in which to analyze the problems of l sup 2 estimation matched filtering and quadratic detection in the presence of uncertainties regarding the second order structure of the random processes involved

minimax discrimination for observed poisson processes with uncertain rate functions the problem of robust design is considered in the context of testing hypotheses concerning the rate function of an observed point process designs that are insensitive to uncertainty in the rate functions are developed by applying a minimax formulation to two different measures of signal to noise ratio uncertainty in the rate is modeled by using general classes of rate measures generated by choquet 2 alternating capacities and solutions are characterized for this case by a radon nikodym type derivative between such classes it is shown that for uncertainty within capacity classes the robust decision design developed for the signal to noise ratio is also robust in a weaker sense for the chernoff upper bounds on the error probabilities furthermore the use of such a test guarantees the exponential convergence of these bounds to zero with increasing length of the observation interval for all rates in the uncertainty class