a general weighted median filter structure admitting negative weights weighted median smoothers which were introduced by edgemore in the context of least absolute regression over 100 years ago have received considerable attention in signal processing during the past two decades although weighted median smoothers offer advantages over traditional linear finite impulse response fir filters it is shown in this paper that they lack the flexibility to adequately address a number of signal processing problems in fact weighted median smoothers are analogous to normalized fir linear filters constrained to have only positive weights it is also shown that much like the mean is generalized to the rich class of linear fir filters the median can be generalized to a richer class of filters admitting positive and negative weights the generalization follows naturally and is surprisingly simple in order to analyze and design this class of filters a new threshold decomposition theory admitting real valued input signals is developed the new threshold decomposition framework is then used to develop fast adaptive algorithms to optimally design the real valued filter coefficients the new weighted median filter formulation leads to significantly more powerful estimators capable of effectively addressing a number of fundamental problems in signal processing that could not adequately be addressed by prior weighted median smoother structures

quadratic weighted median filters for edge enhancement of noisy images quadratic volterra filters are effective in image sharpening applications the linear combination of polynomial terms however yields poor performance in noisy environments weighted median wm filters in contrast are well known for their outlier suppression and detail preservation properties the wm sample selection methodology is naturally extended to the quadratic sample case yielding a filter structure referred to as quadratic weighted median qwm that exploits the higher order statistics of the observed samples while simultaneously being robust to outliers arising in the higher order statistics of environment noise through statistical analysis of higher order samples it is shown that although the parent gaussian distribution is light tailed the higher order terms exhibit heavy tailed distributions the optimal combination of terms contributing to a quadratic system i e cross and square is approached from a maximum likelihood perspective which yields the wm processing of these terms the proposed qwm filter structure is analyzed through determination of the output variance and breakdown probability the studies show that the qwm exhibits lower variance and breakdown probability indicating the robustness of the proposed structure the performance of the qwm filter is tested on constant regions edges and real images and compared to its weighted sum dual the quadratic volterra filter the simulation results show that the proposed method simultaneously suppresses the noise and enhances image details compared with the quadratic volterra sharpener the qwm filter exhibits superior qualitative and quantitative performance in noisy image sharpening

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

a training framework for stack and boolean filtering fast optimal design procedures and robustness case study a training framework is developed in this paper to design optimal nonlinear filters for various signal and image processing tasks the targeted families of nonlinear filters are the boolean filters and stack filters the main merit of this framework at the implementation level is perhaps the absence of constraining models making it nearly universal in terms of application areas we develop fast procedures to design optimal or close to optimal filters based on some representative training set furthermore the training framework shows explicitly the essential part of the initial specification and how it affects the resulting optimal solution symmetry constraints are imposed on the data and consequently on the resulting optimal solutions for improved performance and ease of implementation the case study is dedicated to natural images the properties of optimal boolean and stack filters when the desired signal in the training set is the image of a natural scene are analyzed specifically the effect of changing the desired signal using various natural images and the characteristics of the noise the probability distribution function the mean and the variance is analyzed elaborate experimental conditions were selected to investigate the robustness of the optimal solutions using a sensitivity measure computed on data sets a remarkably low sensitivity and consequently a good generalization power of boolean and stack filters are revealed boolean based filters are thus shown to be not only suitable for image restoration but also robust making it possible to build libraries of optimal filters which are suitable for a set of applications

block median pyramidal transform analysis and denoising applications a nonlinear multiscale pyramidal transform based on nonoverlapping block decompositions using the median operation and a polynomial approximation is considered it is shown that this structure can be useful for denoising of oneand two dimensional 1 d and 2 d signals various denoising techniques are analyzed including methods based on spatially adaptive thresholding and partial cycle spinning algorithms an analytical method for deriving the distribution function of the transform coefficients is also presented this in turn can be used for the selection of thresholds for denoising applications

output distributions of stack filters based on mirrored threshold decomposition the output distribution formula for stack filters based on mirrored threshold decomposition is derived this formula allows one to compute the cumulative distribution function of the output of a stack filter for a given independent and identically distributed i i d input noise distribution mirrored threshold decomposition permits us to analyze the input output characteristics of the stack filter in the binary domain the sliding window operation of the stack filter is modeled by a deterministic finite automaton the output distribution of the filter is obtained by interpreting the automaton as a markov chain whose transition probabilities depend on the probabilistic description of the binary input signal an example illustrating the use of the output distribution formula is provided

stack filter design using selection probabilities selection probabilities viz rank and sample selection probabilities quantify the impulse suppression robustness lowpass filtering ability and detail preserving ability of stack filters closed form expressions exist for computing selection probabilities from positive boolean function describing the stack filter we first present an analysis framework that interprets a boolean vector to denote membership in a partition set using this framework a new algorithm to compute selection probabilities from positive boolean functions is derived next a method to synthesize a stack filter from specified selection probabilities is presented an underdetermined system of linear diophantine equations is derived in the unknown elements of the c matrix and the desired selection probabilities this system of equations is solved to obtain the candidate solutions for the c matrix for each candidate c matrix an algorithm based on partition sets is used to derive one or more positive boolean functions of the stack filter or show that none exists

fuzzy stack filters their definitions fundamental properties and application in image processing a new fuzzy filter called fuzzy stack filter fsf is proposed to extend the filtering capability of conventional stack filter sf which is based on the positive boolean function pbf as its window operator we fuzzify the onset and off set of a given pbf to obtain two types of fuzzy pbfs then we adopt the architecture of threshold decomposition to develop this new fuzzy filter with a fuzzy pbf as its window operator each fuzzy pbf is associated with a set of control parameters therefore the original pbf can be estimated from above and below by two fuzzy pbfs with appropriate control parameters furthermore we can apply the fuzzy modifiers to modify the fuzzy pbfs such that the pbfs can be completely estimated by the fuzzy pbfs hence the stack filter is a special case of fuzzy stack filter since some control parameters are added in this new filter the neural learning algorithms can be easily developed under the flexibility of the given control parameters we first propose the fuzzy m n rank order filter to test our proposed learning algorithm in this simple learning algorithm we can remove the noise corrupted images very well in contrast to the filtering behavior of rank order filters we believe that the results presented will lead to more fruitful research on more advanced and powerful learning algorithms dedicated to the appropriate applications

estimation of error variance via ridge regression summarywe propose a novel estimator of error variance and establish its asymptotic properties based on ridge regression and random matrix theory the proposed estimator is valid under both low and high dimensional models and performs well not only in nonsparse cases but also in sparse ones the finite sample performance of the proposed method is assessed through an intensive numerical study which indicates that the method is promising compared with its competitors in many interesting scenarios

hybrid polynomial filters for gaussian and non gaussian noise environments traditional polynomial filtering theory based on linear combinations of polynomial terms is able to approximate important classes of nonlinear systems the linear combination of polynomial terms however yields poor performance in environments characterized by gaussian and heavy tailed distributions weighted median and weighted myriad filters in contrast are well known for their outlier suppression and detail preservation properties it is shown here that the weighted median and weighted myriad methodologies are naturally extended to the polynomial sample case yielding hybrid filter structures that exploits the higher order statistics of the observed samples while simultaneously being robust to outliers for both gaussian and heavy tailed distributions environments moreover the introduced hybrid polynomial filter classes are well motivated by analysis of cross and square term statistics of gaussian and heavy tailed distributions a presented asymptotic tail mass analysis shows that polynomial terms both under gaussian and heavy tailed noise statistics have heavier tails than the observed samples indicating that robust combination methods should be utilized to avoid undue influence of outliers further analysis shows weighted median processing of polynomial terms for the gaussian noise case and weighted median and weighted myriad processing of cross and square terms respectively for the heavy tailed noise case are justified from a maximum likelihood perspective filters parameter optimization procedures are also presented finally the effectiveness of hybrid filters is demonstrated through simulations that include temporal spectrum and bispectrum analysis

generalized mean median filtering for robust frequency selective applications huber proposed the pepsi family of epsi contaminated normal distributions to model environments characterized by heavy tailed distributions based on this two component mixture distribution mean median mem filters were proposed the mem filter output is a combination of the sample mean and the sample median where observation samples are weighted uniformly this property of mem filters constrains them to the class of smoothers lacking frequency selective filtering capabilities this paper extends mem filtering to the weighted sum median wsm filtering structure admitting real valued weights thereby enabling more general filtering characteristics i e bandpass and high pass filtering the proposed filter structure is also well motivated from a presented maximum likelihood ml estimate analysis under epsi contaminated statistics the ml analysis demonstrates the need for a combination of weighted sum ws and weighted median wm type filters for processing of signals corrupted by epsi contaminated noise the wsm filter is statistically analyzed through the determination of filter output variance and breakdown probability the combination parameter alpha is optimized to minimize the filter output variance which is a measure of noise attenuation capability moreover filter design procedures that yield a desired spectral response are detailed finally the proposed wsm filter structure is tested utilizing signal processing applications including low pass bandpass and high pass filtering and image processing applications including image sharpening and denoising evaluating and comparing the wsm filter performance to that of the ws wm and mem filters

stack filters the median and other rank order operators possess two properties called the threshold decomposition and the stacking properties the first is a limited superposition property which leads to a new architecture for these filters the second is an ordering property which allows an efficient vlsi implementation of the threshold decomposition architecture motivated by the success of rank order filters in a wide variety of applications and by the ease with which they can now be implemented we consider in this paper a new class of filters called stack filters they share the threshold decomposition and stacking properties of rank order filters but are otherwise unconstrained they are shown to form a very large class of easily implemented nonlinear filters which includes the rank order operators as well as all compositions of morphological operators the convergence properties of these filters are investigated using techniques similar to those used to determine root signal behavior of median filters the results obtained include necessary conditions for a stack filter to preserve monotone regions or edges in signals the output distribution for these filters is also found all the stack filters of window width 3 are determined along with their convergence properties among these filters are found two which we have named asymmetric median filters they share all the properties of median filters except that they remove impulses of one sign only that is one removes only positive going edges the other removes only negative going edges while the median filter removes impulses of both signs this investigation of the properties of stack filters thus produces several new useful and easily implemented filters

boolean derivatives weighted chow parameters and selection probabilities of stack filters the theory of boolean derivatives the activities of the arguments of a boolean function bf and chow 1961 parameters are studied from the point of view of their application in the statistical analysis of a class of nonlinear filters stack filters the connection between the partial derivatives of a positive bf pbf and the selection probabilities of stack filters is established the notions of the weighted activities of the variables of the pbf and weighted chow parameters are introduced for the analysis the computation of the joint selection probability matrix and the sample selection probability vector of a continuous stack filter spectral approaches to the selection probabilities of stack filters are derived in particular spectral algorithms with computational complexity o 2 sup n where n is the number of input samples within an input window are given for the computation of sample selection probability vectors the difference of the spectral algorithms presented from the nonspectral ones is that spectral algorithms are universal i e their complexities are independent of the pbf which is used as the base for stack filtering they are also straightforward to implement and fast spectral transforms exist

explicit granger causality in kernel hilbert spaces granger causality gc is undoubtedly the most widely used method to infer cause effect relations from observational time series several nonlinear alternatives to gc have been proposed based on kernel methods we generalize kernel granger causality by considering the variables cross relations explicitly in hilbert spaces the framework is shown to generalize the linear and kernel gc methods and comes with tighter bounds of performance based on rademacher complexity we successfully evaluate its performance in standard dynamical systems as well as to identify the arrow of time in coupled r ossler systems and it is exploited to disclose the el ni no southern oscillation phenomenon footprints on soil moisture globally

stack filters and selection probabilities based on the fact that the output of a given stack filter can be determined if the ranks of the samples in the input window are known and that this output always equals one of the samples in the input window rank and sample selection probabilities are defined the output distribution of the stack filter of size n with independent identically distributed i i d inputs can be expressed as a weighted sum of the ith i 1 2 n order statistics where the rank selection probabilities are the weights the sample selection probabilities equal the impulse response coefficients of a finite impulse response fir filter whose output spectrum is closest of all linear filters to that of the stack filter for i i d gaussian inputs results are also derived for correlated inputs robustness and detail preserving properties of stack filters are related to the selection probabilities other statistical properties are also derived finally methods to compute the selection probabilities of the stack filter from its positive boolean function and the selection probabilities of the weighted median filter from its weights are given in detail

tests for high dimensional regression coefficients with factorial designs we propose simultaneous tests for coefficients in high dimensional linear regression models with factorial designs the proposed tests are designed for the large p small n situations where the conventional f test is no longer applicable we derive the asymptotic distribution of the proposed test statistic under the high dimensional null hypothesis and various scenarios of the alternatives which allow power evaluations we also evaluate the power of the f test for models of moderate dimension the proposed tests are employed to analyze a microarray data on yorkshire gilts to find significant gene ontology terms which are significantly associated with the thyroid hormone after accounting for the designs of the experiment

output distributional influence function when a filter is being selected for an application it is often essential to know that the behavior of the filter does not change significantly if there are small deviations from the initial assumptions this robustness of a filter is traditionally explored by means of the influence function if and change of variance function cvf however as these are asymptotic measures there is uncertainty of the applicability of the obtained results to the finite length filters used in the real world filtering applications we present a new method called the output distributional influence function odif that examines the robustness of the finite length filters the method gives most extensive information about the robustness for filters with a known output distribution function as examples the odifs for the distribution function density function expectation and variance are given for the well known mean and median filters and are interpreted in detail

on the robustness of the class of stack filters a widely held view in the nonlinear signal processing community is that the class of stack filters is robust although this is supported by extensive experimental evidence no systematic theoretical justification exists despite the availability of analytical tools for studying robustness of individual stack filters we focus on rank selection probabilities rsps as measures of robustness as it is well known that other statistical characterizations of stack filters such as output distributions breakdown probabilities and output distributional influence functions can be represented in terms of rsps we show in a very general sense that the class of stack filters is highly robust it is also shown that almost all stack filters have very similar output distributions for independent and identically distributed i i d input signals and thus very similar statistical behavior

meridian filtering for robust signal processing a broad range of statistical processes is characterized by the generalized gaussian statistics for instance the gaussian and laplacian probability density functions are special cases of generalized gaussian statistics moreover the linear and median filtering structures are statistically related to the maximum likelihood estimates of location under gaussian and laplacian statistics respectively in this paper we investigate the well established statistical relationship between gaussian and cauchy distributions showing that the random variable formed as the ratio of two independent gaussian distributed random variables is cauchy distributed we also note that the cauchy distribution is a member of the generalized cauchy distribution family recently proposed myriad filtering is based on the maximum likelihood estimate of location under cauchy statistics an analogous relationship is formed here for the laplacian statistics as the ratio of laplacian statistics yields the distribution referred here to as the meridian interestingly the meridian distribution is also a member of the generalized cauchy family the maximum likelihood estimate under the obtained statistics is analyzed motivated by the maximum likelihood estimate under meridian statistics meridian filtering is proposed the analysis presented here indicates that the proposed filtering structure exhibits characteristics more robust than that of median and myriad filtering structures the statistical and deterministic properties essential to signal processing applications of the meridian filter are given the meridian filtering structure is extended to admit real valued weights utilizing the sign coupling approach finally simulations are performed to evaluate and compare the proposed meridian filtering structure performance to those of linear median and myriad filtering

a fast algorithm for designing stack filters stack filters are a class of nonlinear filters with excellent properties for signal restoration unfortunately present algorithms for designing stack filters can only be used for small window sizes because of either their computational overhead or their serial nature this paper presents a new adaptive algorithm for determining a stack filter that minimizes the mean absolute error criterion the new algorithm retains the iterative nature of many current adaptive stack filtering algorithms but significantly reduces the number of iterations required to converge to an optimal filter this algorithm is faster than all currently available stack filter design algorithms is simple to implement and is shown in this paper to always converge to an optimal stack filter extensive comparisons between this new algorithm and all existing algorithms are provided the comparisons are based both on the performance of the resulting filters and upon the time and space complexity of the algorithms they demonstrate that the new algorithm has three advantages it is faster than all other available algorithms it can be used on standard workstations sparc 5 with 48 mb to design filters with windows containing 20 or more points and its highly parallel structure allows very fast implementations on parallel machines this new algorithm allows cascades of stack filters to be designed stack filters with windows containing 72 points have been designed in a matter of minutes under this new approach

stack filters stack smoothers and mirrored threshold decomposition stack smoothers have received considerable attention in signal processing in the past decade stack smoothers define a large class of nonlinear smoothers based on positive boolean functions pbf applied in the binary domain of threshold decomposition although stack smoothers can offer some advantages over traditional linear fir filters they are in essence smoothers lacking the flexibility to adequately address a number of signal processing problems that require bandpass or highpass filtering characteristics in this paper mirrored threshold decomposition is introduced which together with the associated binary pbf define the significantly richer class of stack filters using threshold logic representation a number of properties of stack filters are derived notably stack filters defined in the binary domain of mirrored threshold decomposition require the use of double weighting of each sample in the integer domain the class of recursive stack filters and the corresponding recursive weighted median rwm filters in the integer domain admitting negative weights are introduced the new stack filter formulation leads to a more powerful class of estimators capable of effectively addressing a number of fundamental problems in signal processing that could not adequately be addressed by prior stack smoother structures

adaptive stack filtering under the mean absolute error criterion an adaptive filter algorithm is developed for the class of stack filters which is a class of nonlinear filters obeying a weak superposition property the adaptation algorithm can be interpreted as a learning algorithm for a group of decision making units the decisions of which are subject to a set of constraints called the stacking constraints under a rather weak statistical assumption on the training inputs the decision strategy adopted by the group which evolves according to the proposed learning algorithm is shown to converge to an optimal strategy in the sense that it corresponds to an optimal stack filter under the mean absolute error criterion this adaptive algorithm requires only increment decrement and comparison operations and only local interconnections between the learning units implementation of the algorithm in hardware is therefore very feasible an example is provided to show how the adaptive stack filtering algorithm can be used in an application in image processing

polynomial weighted median filtering this paper extends weighted median wm filters to the class of polynomial weighted median pwm filters traditional polynomial filtering theory based on linear combinations of polynomial terms is able to approximate important classes of nonlinear systems the linear combination of polynomial terms however yields poor performance in environments characterized by heavy tailed distributions weighted median filters in contrast are well known for their outlier suppression and detail preservation properties the weighted median sample selection methodology is naturally extended to the polynomial sample case yielding a filter structure that exploits the higher order statistics of the observed samples while simultaneously being robust to outliers moreover the pwm filter class is well motivated by an analysis of cross and square term statistics a presented probability density function analysis shows that these terms have heavier tails than the observed samples indicating that robust combination methods should be utilized to avoid undue influence of outliers further analysis shows weighted median processing of polynomial terms is justified from a maximum likelihood perspective the established pwm filter class is statistically analyzed through the determination of the filter output distribution and breakdown probability filter parameter optimization procedures are also presented finally the effectiveness of pwm filters is demonstrated through simulations that include temporal spectrum and bispectrum analysis

stack filters and the mean absolute error criterion a method to determine the stack filter which minimizes the mean absolute error between its output and a desired signal given noisy observations of this desired signal is presented specifically an optimal window width b stack filter can be determined with a linear program with o b2 sup b variables this algorithm is efficient since the number of different inputs to a window width b filter is m sup b if the filter has m valued input and the number of stack filters grows faster than 2 raised to the 2 sup b 2 power it is shown that optimal stack filtering under the mean absolute error criterion is analogous to optimal linear filtering under the mean squared error criterion both linear filters and stack filters are defined by superposition properties both classes are implementable and both have tractable procedures for finding the optimal filter under an appropriate error criterion

analysis of the properties of median and weighted median filters using threshold logic and stack filter representation the deterministic properties of weighted median wm filters are analyzed threshold decomposition and the stacking property together establish a unique relationship between integer and binary domain filtering the authors present a method to find the weighted median filter which is equivalent to a stack filter defined by a positive boolean function because the cascade of wm filters can always be expressed as a single stack filter this allows expression of the cascade of wm filters as a single wm filter a direct application is the computation of the output distribution of a cascade of wm filters the same method is used to find a nonrecursive expansion of a recursive wm filter as applications of theoretical results several interesting deterministic and statistical properties of wm filters are derived

finding of optimal stack filter by graphic searching methods an efficient process to filter noise via an optimal stack filter is proposed the graphic search based techniques are employed to speed up the finding of the optimal stack filter experimental results and performance evaluation are demonstrated to show the efficiency of our proposed method

a two sample test for high dimensional data with applications to gene set testing we proposed a two sample test for means of high dimensional data when the data dimension is much larger than the sample size the classical hotelling s t 2 test does not work for this large p small n situation the proposed test does not require explicit conditions on the relationship between the data dimension and sample size this offers much flexibility in analyzing high dimensional data an application of the proposed test is in testing significance for sets of genes which we demonstrate in an empirical study on a leukemia data set

fuzzy logic and mathematical morphology implementation by stack filters mathematical morphology has proven invaluable for signal image processing fuzzy theory is enjoying great success in control and signal processing we show that stack filters which are already related to morphology require slight modification to implement basic fuzzy operators the article therefore provides a fast errorless method for implementation of basic fuzzy operations

median filtering by threshold decomposition median filters are a special class of ranked order filters used for smoothing signals repeated application of the filter on a quantized signal of finite length ultimately results in a sequence termed a root signal which is invariant to further passes of the median filter in this paper it is shown that median filtering an arbitrary level signal to its root is equivalent to decomposing the signal into binary signals filtering each binary signal to a root with a binary median filter and then reversing the decomposition this equivalence allows problems in the analysis and the implementation of median filters for arbitrary level signals to be reduced to the equivalent problems for binary signals since the effects of median filters on binary signals are well understood this technique is a powerful new tool

on the lp which finds a mmae stack filter two methods are proposed to modify the linear program lp developed by e j coyle and j h lin 1988 to find a stack filter which minimizes the mean absolute error mae in the first approach the number of constraints is substantially reduced at the expense of requiring a zero one lp to solve for an optimal filter this scheme reduces the number of constraints from o n2 sup n to o 28 sup n which is exactly the cardinality of the set of possible binary vectors which can appear in the window of the filter in the second approach the lp is transformed into a max flow problem this guarantees that the problem can be solved in time which is a polynomial function of the number of variables in the lp as opposed to the worst case exponential time that may occur with the simplex method it also allows the many fast algorithms for the max flow problem to be used to find an optimal stack filter recursive algorithms for construction of the window width n constraint matrix for both the original lp and the max flow modification are also provided