In recent years there has been much interest in robustness issues In recent years there has been a resurgence of interest in what are known as robust methods for statistical signal processing. Such methods are applicable whenever schemes are used to carry out functions such as signal detection, estimation, filtering, and coding, common examples being in radar and sonar signal processing, communication systems, pattern recognition, and speech and image processing. In the early days of development of the body of ideas we now possess for statistical signal processing, the emphasis was on the derivation of optimum schemes for use in specified signal and noise environments. A classic example of this is the matched filter which is optimum for a particular signal and noise model. Because the signals and noise in signal processing applications are usually modeled as random processes and performance measures therefore usually involve probabilistic quantities (such as mean squared error or probability of error), the theory of statistics has played a fundamental role in the development of optimum signal processing techniques. Suppose a signal processing scheme, say a detector for a signal with known waveform in additive noise, is designed to give optimum performance for noise possessing a specific statistical description. For example, one widespread model for noise is that it is a Gaussian process. An important question that arises is, how sensitive is the performance of such an optimum scheme to deviations in the signal and noise characteristics from those for which the scheme is designed? This is an important question because in practice one rarely has perfect knowledge of, say, the noise characteristics; the Gaussian or any other specific model is usually a nominal assumption which may at best be approximately valid most of the time. Unfortunately, it turns out that in many cases nominally optimum signal processing schemes can suffer a drastic degradation in performance even for apparently small deviations from nominal assumptions. It is this basic observation that motivates the search for robust signal processing techniques; that is, techniques with good performance under any nominal conditions and acceptable performance for signal and noise conditions other than the nominal which can range over the whole of allowable classes of possible characteristics. Thus in seeking robust methods it is recognized at the outset that a single, precise characterization of signal and noise conditions is unrealistic, and so classes of possible signal and noise characterizations are constructed and considered in the design of such methods. To illustrate the above observation with a concrete example consider further the detection problem mentioned above in discrete time. Thus suppose we have scalar observations X , , X , ; . . , X , , forming a vector X , which are 0018-9219/85/0300-0433901.00 4 9 8 5 IEEE A test based on the comparison of A(X) with a threshold is equivalent to one based on a comparison of the logarithm of A(X) with the logarithm of the original threshold. Taking the logarithm on both sides of (1.2), we have where If I L ( x ; s,B)I is unbounded as a function of x , the value of logA(X) can be influenced heavily by a single observation component X i for which I L ( X , ; s,,e)l is large. Such a component can therefore completely override the weight of a possibly large number of other components in the choice between H, and HI. While this effect is certainly acceptable if the model for the noise density function is accurate, it may also be observed because of an occasional completely erroneous measurement which the pdf model f does not take into account. In general, the assumed pdf f describes only an approximate or nominal model. Thus while the actual value of lL (X,;s,,O)l at some observed value X , = x , may not be large relative to that obtained at other observation components, for the assumed model this may happen. To illustrate this, suppose that f is assumed to be Gaussian, in which case [ ( x ; s,e) is linear in x and thus is unbounded. If the true density f has exponential, rather than Gaussian, tails, then the true L ( x ; s,@) is a constant for x in the tails, and so is bounded. For a model specifying exponential tails, an increasing absolute value for an observation component indicates increasing likelihood of one hypothesis over another only up to a "saturation" value; beyond it, larger absolute values do not indicate larger relative likelihood. If the noise density were truly exponential (or some other long-tailed pdf), then the performance of the test that is optimum for Gaussian noise could be very poor because of the unexpected number of large noise values. It Often a class of allowable characteristics, say for a noise pdf, is constructed by starting with a nominal characteristic and then including in the class all other characteristics that are "close," in some well-defined sense, to this nominal one. Then a signal processing scheme that is robust may have performance at the nominal which is not quite as good as the scheme that is optimum for the nominal case, but its overall performance with respect to the defined class of characteristics will be good or acceptable. This loose definition of robustness is perfectly reasonable, but it does not provide a systematic approach to obtaining robust schemes. In order to achieve this we must first specify a measure of "overall" performance of a scheme with respect to a class of allowable conditions at the input. One such measure that has been widely used and which leads to interesting and useful results in many situations is the worst case performance of a scheme over a class of input conditions. Clearly, if its worst case performance is good we may say that a given scheme is robust. On the other hand, to find such a robust scheme we can look for the scheme that optimizes worst case Performance. This approach leads to what are known as minimax' robust schemes. Implicit in our association of minimax schemes with robust schemes is the expectation that the worst case performance of a minimax scheme will be acceptably good, being the best that can be achieved. Another expectation one has in defining robust schemes in this way is that at any nominal operating point the performance of the minimax scheme will not be very far below that of the nominally optimum scheme, which on the other hand will have much poorer performance away from the nominal point. Fortunately, it does turn out that minimax schemes for the signal processing applications of interest usually have the above 'A scheme that minimizes the maximum possible value of a loss function is minimax; if performance is measured by a gain function then a maximin scheme would be sought. We shall use the term minimax as a general description for such schemes in all cases. PROCEEDINGS OF THE IEEE, V O L 73. NO. 3, MARCH 1985 characteristics. They may therefore be said to have a more "stable" performance than schemes lacking these characteristics (in the literature the terms robust and stable are sometimes used to mean the same thing). We should emphasize that the classes of allowable characteristics one deals with in robust signal processing are generally nonparametric function classes, such as the class of all power spectral density functions with specified total power (area under the function) and which lie between specified upper and lower bounding functions. For uncertainties expressed by parametric classes of allowable values for finite-dimensional parameters (such as the mean and variance of a Gaussian pdf) one can of course use minimax designs as well, although alternative parametric approaches of statistical theory can also be applied in such situations. In this paper we will concentrate on minimax robust schemes. There are useful formulations of robustness other than the minimax one, most notably the stability or qualitative robustness ideas introduced by Root [2] in the context of signal detection and by Hampel [3] in the context of parameter estimation. These formulations utilize the idea of robustness as a continuity property of some performance measure as a function of the underlying model, and some brief discussion of these ideas is included here. However, from the viewpoint of design, the minimax approach has had the most impact on robust signal processing schemes. ~l s o we will not survey adaptive procedures, which may be used as robust schemes when input conditions are not precisely known and may be time-varying. Adaptive procedures, which attempt to learn about input conditions and adjust their specific signal processing structure accordingly to maintain good performance, are generally more complex than fixed minimax schemes. Adaptive schemes are more desirable when the a priori uncertainty is so large that the guaranteed level of performance of a minimax scheme would be too poor to be acceptable and when adequate time and data for adapting are available. Conversely, minimax procedures would be more desirable under more constrained uncertainty classes, and especially as robust procedures to guard against excessive performance degradation of nominally optimum schemes for deviations from nominal assumptions. Minimax schemes may be used in conjunction with an adaptive approach, because the learning mechanism in adaptive schemes can never be expected to perform perfectly given any finite time for adaptation to take place. The application of minimax concepts to obtain robust versions of optimum adaptive procedures has also been considered [4]. While our primary concern here is with minimax robust schemes we will mention other specific techniques whenever it is appropriate. Most The pre-I964 investigations of minimax signal processing schemes tend to be characterized by two attributes. One is that they were generally not concerned directly with pdf variations but rather with power spectral density function or related variations. Secondly, minimax schemes were advocated simply as reasonable approaches when designing systems for operation under conditions at the inputs which could not be determined a priori. Thus the possible nonrobustness of optimum schemes for nominal assumptions on the input was not explicitly recognized as an issue. The term "robust" was first used in describing desirable statistical procedures by Box in 1953 [17]. As we have remarked, minimax robustness of estimation and hypothesis testing schemes was considered by Huber in [6] and [7], and since then a large number of results on minimax and alternative formulations of robustness have been generated in the statistics literature. In a recent paper [I81 Huber has given a most interesting account of some early concerns about robustness of statistical procedures and specific schemes, some of which date back to the last century. This survey will focus specifically on minimax robust signal processing schemes, so that only a small part of the large body of the statistics literature will be mentioned explicitly. Most of the recent developments in robust signal processing have of course been influenced directly by the developments in robust statistics. However, signal processing problems impose their own distinct requirements which are not always standard in problems of statistics. Thus it turns out that some recent developments in robust signal processing have provided new results in robust statistics. Most of the results we survey here are of the post-1965 period. Two of the earliest papers in the signal processing area from this period are those of Wolff and Gastwirth [29] and Martin and Schwartz [30], and they have been responsible for driving much of the subsequent work in robust signal processing. Thus a considerable literature has arisen on robust signal processing just in the last ten to fifteen years. The statistical descriptions of input conditions in signal processing are usually stated in terms of power spectral density or correlation functions and pdfs. We shall discuss results which have been obtained on minimax robust  filtering for signal estimation (e.g., Wiener filtering) in Section I 1 and for signal detection (eg, matched filtering) in Section Ill. Here the uncertainty classes are for spectral density or correlation functions. In Section IV results on minimax robust nonlinear signal detection schemes for distributional uncertainties are surveyed. Nonlinear parame fer estimation schemes are surveyed more briefly in Section V, since on this topic there is much already available in review form in the statistics literature. Also included in Section V is a brief survey of nonlinear modifications of the Kalman filter for robustness against non-Gaussian pdfs for the observation and process noise components. Section Vi treats the problem of robust quantization of data with unknown statistics, and we close with some concluding remarks in Section VII. Although our survey begins with robust linear filtering, studies on this topic are of more recent vintage than those on nonlinear signal detection and estimation. We feel, however, that the very widespread use of schemes such as Wiener and matched filters in signal processing justifies our beginning with robust versions of such linear processing schemes. Before we begin let us note some other review, tutorial, and survey articles which are available in the literature. A tutorial on this subject by the authors has been published recently [31]. VandeLinde has given a brief survey in [32]. Ershov [25] and Krasnenker [33] have surveyed nonlinear estimation and detection schemes, respectively. Poor [34] has recently given a more mathematically detailed survey of robust detection schemes. Kleiner, Martin, and Thomson [35] and Martin and Thomson [36] treat the robust estimation of power spectral density functions. Robust methods for time series analysis have been considered by Martin in [37], and robust methods for system identification have been described by Poljak and Tsypkin in [38]. As a final introductory comment we should note that the literature in the area of robust statistical methods is vast and broad. Thus although this survey touches on what we feel to be the major contributions in robust signal processing, it is by no means exhaustive. However, the many results and methods that are not discussed here are accessible to the reader through the references provided. One of the most common signal processing tasks arising in applications is that of estimating (e.g., filtering, predicting, or smoothing) a signal waveform from a noisy measurement. This task arises, for example, in radar and sonar tracking systems, in observers for automatic control systems, in demodulators for analog communication systems, and in medical imaging systems. Conventional design procedures for optimum signal estimation algorithms often require an exact knowledge of the statistical behavior both of the signal of interest and of the noise corrupting the measurement. For example, in the design of optimum linear estimation algorithms we must know the spectral or autocorrelation properties of the signal and noise in order to specify the optimum procedures, and (as we shall see below) procedures designed to be optimum for a given model can be undesirably sensitive to inaccuracies in the model. As noted in the Introduction, robust procedures can overcome problems arising due to modeling inaccuracy by incorporating modeling uncertainty into the design from the outset. In this section, we will discuss the design of robust estimation procedures primarily within the context of the stationary linear (i.e., Wiener-Kolmogorov) estimation problem. Several other signal estimation problems have been treated in the context of robust design, including recursive nonlinear filtering and identification. Results on these problems will be discussed briefly in Section V. Consider the observation model Y( t) = S( t) + N( t), -CO < t < 00 (2.1) where { S( t); -00 < t < oo} and { N( t); -oo < t < 00) are real, zero-mean, orthogonal, wide-sense-stationary (wss) random processes representing signal and noise, respectively. We assume that {S( t); -03 t < GO} and { N( t); -w < t < oo} have power spectral densities as and a,, respectively. (Most of these assumptions can be relaxed, as is discussed below.) Given the observation process { Y( t); -03 < t < oo} we wish t o form an estimate of S( t) of the form where h is the impulse response of a time-invariant linear filter. A common performance criterion for signal estimates is the mean squared error (MSE), which for estimates of the form of (2.2) is given straightforwardly by where H is the transfer function associated with h (i.e., H is the Fourier transform of h). If as and @ , are known, then the M S E of (2.3) can be minimized over H to find the optimum filter transfer function for linear minimum-MSE estimation. It is straightforward to show (see, e.g., Thomas [39]) that the minimizing solution is given by and that the corresponding minimum value of the MSE is given by ew(@s,@,.,). (2.5) Suppose that we design a filter H, via (2.4) to be optimum for some nominal signal and noise spectral pair
The signal processing problem considered in this paper is most relevant to robust statistical inference REF .
Robust techniques for signal processing: A survey
Proceedings of the IEEE
Proceedings of the IEEE
Computer Science