continuous discrete time observers for a class of mimo nonlinear systems this technical note addresses the observer design problem for a class of continuous time dynamical systems with non uniformly sampled measurements more specifically we propose an observer that runs in continuous time with an output error correction term that is updated in a mixed continuous discrete fashion the proposed observer is actually an impulsive system since it is described by a set of differential equations with instantaneous state impulses corresponding to the measured samples and their estimates nevertheless we shall show that such an impulsive system can be put under the form of a hybrid system composed of a continuous time high gain observer coupled with an inter sample output predictor two design features of the proposed observer are worth emphasizing firstly the observer calibration is achieved through the tuning of a scalar design parameter secondly the exponential convergence to zero of the observation error is established under a well defined condition on the maximum value of the sampling partition diameter simulations results involving a flexible joint robot arm are given in order to highlight the performance of the proposed observer

robust observer design under measurement noise with gain adaptation and saturated estimates abstract we use incremental homogeneity gain adaptation and incremental observability for proving new results on robust observer design for systems with noisy measurement and bounded trajectories a state observer is designed by dominating the incrementally homogeneous nonlinearities of the observation error system with its linear approximation while gain adaptation and incremental observability guarantee an asymptotic upper bound for the estimation error depending on the limsup of the norm of the measurement noise a characteristic and innovative feature of this observer is the mixed low high gain structure in combination with saturated state estimates and dynamically tuned gains and saturation levels the gain adaptation is implemented as the output of a stable filter using the squared norm of the measured output estimation error and the mismatch between each estimate and its saturated value

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

estimates of error probability for complex gaussian channels with generalized likelihood ratio detection we derive approximate expressions for the probability of error in a two class hypothesis testing problem in which the two hypotheses are characterized by zero mean complex gaussian distributions these error expressions are given in terms of the moments of the test statistic employed and we derive these moments for both the likelihood ratio test appropriate when class densities are known and the generalized likelihood ratio test appropriate when class densities must be estimated from training data these moments are functions of class distribution parameters which are generally unknown so we develop unbiased moment estimators in terms of the training data with these accurate estimates of probability of error can be calculated quickly for both the optimal and plug in rules from available training data we present a detailed example of the behavior of these estimators and demonstrate their application to common pattern recognition problems which include quantifying the incremental value of larger training data collections evaluating relative geometry in data fusion from multiple sensors and selecting a good subset of available features

a note on the gamma test analysis of noisy input output data and noisy time series abstract in a smooth input output process y f x if the input data x r d is noise free and only the output data y is corrupted by noise then a near optimal smooth model g will be a close approximation to f however as previously observed for example in h kantz t schreiber nonlinear time series analysis 2nd ed cambridge univ press 2004 if the input data is also corrupted by noise then this is no longer the case with noise on the inputs the best predictive smooth model based on noisy data need not be an approximation to the actual underlying process rather the best predictive model depends on both the underlying process and the noise a corollary of this observation is that one cannot readily infer the nature of a process from noisy data since almost all data has associated noise this conclusion has some unsettling implications in this note we show how these effects can be quantified using the gamma test in particular we examine the gamma test analysis of noisy time series data we show that the noise level on the best predictive smooth model based on the noisy data can be much higher than the noise level on individual time series measurements and we give an upper bound for the first in terms of the second

multidimensional probability density function approximations for detection classification and model order selection this paper addresses the problem of calculating the multidimensional probability density functions pdfs of statistics derived from known many to one transformations of independent random variables rvs with known distributions the statistics covered in the paper include reflection coefficients autocorrelation estimates cepstral coefficients and general linear functions of independent rvs through pdf transformation these results can be used for general pdf approximation detection classification and model order selection a model order selection example that shows significantly better performance than the akaike and mdl method is included

probability density function estimation using the eef with application to subset feature selection we describe a method for estimating a probability density function when some of the sufficient statistics are known the general form of the pdf is within the exponential family and is known as the exponentially embedded family using the proposed estimator new approaches to choosing features from a set of possible features become available with the error metric being the kullback liebler distance applications to subset selection in the context of multipath estimation as well as linear regression for machine learning are used to illustrate the practical utility of the method

technical report an mgf based unified framework to determine the joint statistics of partial sums of ordered i n d random variables the joint statistics of partial sums of ordered random variables rvs are often needed for the accurate performance characterization of a wide variety of wireless communication systems a unified analytical framework to determine the joint statistics of partial sums of ordered independent and identically distributed i i d random variables was recently presented however the identical distribution assumption may not be valid in several real world applications with this motivation in mind we consider in this paper the more general case in which the random variables are independent but not necessarily identically distributed i n d more specifically we extend the previous analysis and introduce a new more general unified analytical framework to determine the joint statistics of partial sums of ordered i n d rvs our mathematical formalism is illustrated with an application on the exact performance analysis of the capture probability of generalized selection combining gsc based rake receivers operating over frequency selective fading channels with a non uniform power delay profile we also discussed a couple of other sample applications of the generic results presented in this work

an mgf based unified framework to determine the joint statistics of partial sums of ordered random variables order statistics find applications in various areas of communications and signal processing in this paper we introduce an unified analytical framework to determine the joint statistics of partial sums of ordered random variables rvs with the proposed approach we can systematically derive the joint statistics of any partial sums of ordered statistics in terms of the moment generating function mgf and the probability density function pdf our mgf based approach applies not only when all the k ordered rvs are involved but also when only the ks ks k best rvs are considered in addition we present the closed form expressions for the exponential rv special case these results apply to the performance analysis of various wireless communication systems over fading channels

uniform manifold sampling ums sampling the maximum entropy pdf maximum entropy pdf projection mepp is a way to construct generative models from feature transformations corresponding to each dimension reducing feature mapping such as a feed forward neural network or an algorithm to calculate linear prediction coefficients from time series and given a prior distribution for the features there exists a unique generative model for the input data which subject to mild requirements is maximum entropy maxent among all probability density functions pdfs that are consistent with the given feature prior in this paper we consider the problem of sampling from these maxent projected pdfs the sampling process consists of drawing a sample from the given feature prior distribution then drawing samples uniformly distributed on the inversion set set of input samples consistent with the drawn feature value usually a manifold the process is called uniform manifold sampling ums we describe ums for simple nonlinear and iterative feature transformations then focus on linear transformations with input data constraints x i 0 or 0 leq x i leq 1 which require mcmc based sampling we discover that the manifold centroid sample mean for a fixed feature value is useful as a deterministic maxent feature inversion solution we show how to predict the centroid efficiently without sampling and demonstrate its usefulness in speeding up mcmc by an order of magnitude and in spectral estimation and image reconstruction finally we provide an example of ums in a classification experiment in which we use monte carlo integration to create true generative models from arbitrary classifiers

reducts and constructs in classic and dominance based rough sets approach abstract the idea of the reduct as defined in the classic rough sets approach crsa has proven to be inspiring enough to get into closely related theories including the dominance based rough sets approach drsa the procedure of reduction is generally similar to that of feature selection but narrower as it is the descriptive rather than the predictive aspect of data exploration that constitutes its principal goal crsa reducts are thus defined as minimal subsets of attributes that retain sufficiently high quality of object description developed within crsa the crsa reducts have given rise to the generalized notion of crsa constructs which have turned out to be superior to reducts in numerous practical experiments with real life data sets the generalization process is continued in this paper in which a definition of constructs in the context of drsa is introduced the definition fully analogous to that of crsa constructs differs only in that it is context based in drsa while context free in crsa consequently the presented drsa constructs are expected to have analogous properties to that of crsa constructs including superiority to drsa reducts in experiments with real life data sets

fault estimation for a class of nonlinear systems based on intermediate estimator this technical note investigates the problem of fault estimation for a class of nonlinear systems with lipschitzian nonlinearities in most of the existing fault estimation approaches the observer matching condition should be satisfied however this condition is hard to meet for many systems to overcome this restriction a new fault estimation method is proposed where an intermediate variable is introduced and a specially designed intermediate estimator is proposed to estimate the state and the faults simultaneously by exploiting the properties of fault distribution matrix it is proved that the states of the error system are uniformly ultimately bounded simulation examples verify the effectiveness of the proposed method

robust design of nonlinear internal models without adaptation we propose a solution to the problem of semiglobal output regulation for nonlinear minimum phase systems driven by uncertain exosystems that does not rely upon conventional adaptation schemes to estimate the frequency of the exogenous signals rather the proposed approach relies upon regression like arguments used to derive a nonlinear internal model able to offset the presence of an unknown number of harmonic exogenous inputs of uncertain amplitude phase and frequency the design methodology guarantees asymptotic regulation if the dimension of the regulator exceeds a lower bound determined by the actual number of harmonic components of the exogenous input if this is not the case a bounded steady state regulation error is ensured whose amplitude though can be arbitrarily decreased by acting on a design parameter of the regulator

a general wrapper approach to selection of class dependent features in this paper we argue that for a c class classification problem c 2 class classifiers each of which discriminating one class from the other classes and having a characteristic input feature subset should in general outperform or at least match the performance of a c class classifier with one single input feature subset for each class we select a desirable feature subset which leads to the lowest classification error rate for this class using a classifier for a given feature subset search algorithm to fairly compare all models we propose a weight method for the class dependent classifier i e assigning a weight to each model s output before the comparison is carried out the method s performance is evaluated on two artificial data sets and several real world benchmark data sets with the support vector machine svm as the classifier and with the relief class separability and minimal redundancy maximal relevancy mrmr as attribute importance measures our results indicate that the class dependent feature subsets found by our approach can effectively remove irrelevant or redundant features while maintaining or improving sometimes substantially the classification accuracy in comparison with other feature selection methods

ekf like observer with stability for a class of nonlinear systems an extended kalman filter ekf like observer is derived from a former observer result for a class of nonlinear systems which can be written as a linear part in the unmeasured states on the one hand and some additive nonlinearity with a triangular jacobian on the other hand it is shown how the previously presented excitation condition for exponential stability of the observer extends to this ekf version the observer is illustrated in simulation with two challenging examples the first one in leak detection and the second one in chaos synchronization

brief paper an adaptive high gain observer for nonlinear systems in this paper the authors provide a solution to the noise sensitivity of high gain observers the resulting nonlinear observer possesses simultaneously 1 extended kalman filter s good noise filtering properties and 2 the reactivity of the high gain extended kalman filter with respect to large perturbations the authors introduce innovation as the quantity that drives the gain adaptation they prove a general convergence result propose guidelines to practical implementation and show simulation results for an example

class specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition the polynomial classifier pc that takes the binomial terms of reduced subspace features as inputs has shown superior performance to multilayer neural networks in pattern classification in this paper we propose a class specific feature polynomial classifier cfpc that extracts class specific features from class specific subspaces unlike the ordinary pc that uses a class independent subspace the cfpc can be viewed as a hybrid of ordinary pc and projection distance method the class specific features better separate one class from the others and the incorporation of class specific projection distance further improves the separability the connecting weights of cfpc are efficiently learned class by class to minimize the mean square error on training samples to justify the promise of cfpc we have conducted experiments of handwritten digit recognition and numeral string recognition on the nist special database 19 sd19 the digit recognition task was also benchmarked on two standard databases usps and mnist the results show that the performance of cfpc is superior to that of ordinary pc and is competitive with support vector classifiers svcs

class specific feature sets in classification in this correspondence we present a new approach to the design of probabilistic classifiers that circumvents the dimensionality problem rather than working with a common high dimensional feature set the classifier is written in terms of likelihood ratios with respect to a common class using sufficient statistics chosen specifically for each class

the observer follower filter a new approach to nonlinear suboptimal filtering this paper investigates the state estimation problem for a class of stochastic nonlinear differential systems a novel algorithm is proposed denoted as observer follower filter off based on a two steps mixed approach the first step makes use of a high gain observer based estimator for nonlinear systems applied to the system equations in order to provide the trajectory around which a n degree carleman approximation of the stochastic differential system is achieved second step in principle any other high gain estimator can be used but in this note we prove that the one here proposed provides a bounded mean square error numerical simulations show the effectiveness of the proposed methodology and the improvements of the off with respect to the standard extended kalman bucy filter ekbf obtained by increasing the order of the carleman approximation

time varying increasing gain observers for nonlinear systems a full order state observer for a class of nonlinear continuous time systems is presented as generalization of the high gain observer for having a time varying gain that is let to be small in the first time instants increases over time up to its maximum and then is kept constant the global stability of the resulting estimation error is proved by means of a lyapunov functional via a change of coordinate the design of such an observer is obtained by solving a nonlinear programming problem and using series expansions to set the time varying gain

the relationship between attribute reducts in rough sets and minimal vertex covers of graphs the problems to find attribute reduction in rough sets and to obtain the minimal vertex cover for graphs are both np hard problems this paper studies the relationship between the two problems the vertex cover problem for graphs from the perspective of rough sets is first investigated the attribute reduction of an information system is then studied in the framework of graph theory the results in this paper show that finding the minimal vertex cover of a graph is equivalent to finding the attribute reduction of an information system induced from the graph conversely the attribute reduction computation can be translated into the calculation of the minimal vertex cover of a derivative graph finally a new algorithm for the vertex cover problem based on rough sets is presented furthermore experiments are conducted to verify the effectiveness of the proposed method

increasing gain observers for nonlinear systems for the purpose to estimate the state of nonlinear continuous time systems we focus on the increasing gain observer regarded as generalization of the well known high gain observer as compared with the previous results on increasing gain observers the assumptions under which the global asymptotic stability of the estimation error is proved are relaxed the stability analysis is drawn by using a more general type of lyapunov functional where design parameters can be tuned to set the time varying gain a new design method based on such a lyapunov functional is proposed to construct the observer simulation results affirm the effectiveness of the increasing gain approach as compared with the high gain observer

a multi resolution hidden markov model using class specific features we apply the pdf projection theorem to generalize the hidden markov model hmm to accommodate multiple simultaneous segmentations of the raw data and multiple feature extraction transformations different segment sizes and feature transformations are assigned to each state the algorithm averages over all allowable segmentations by mapping the segmentations to a proxy hmm and using the forward procedure a by product of the algorithm is the set of a posteriori state probability estimates that serve as a description of the input data these probabilities have simultaneously the temporal resolution of the smallest processing windows and the processing gain and frequency resolution of the largest processing windows the method is demonstrated on the problem of precisely modeling the consonant t in order to detect the presence of a distinct burst component we compare the algorithm against standard speech analysis methods using data from the timit corpus

quick attribute reduction in inconsistent decision tables this paper focuses on three types of attribute reducts in inconsistent decision tables assignment reduct distribution reduct and maximum distribution reduct it is quite inconvenient to judge these three types of reduct directly according to their definitions this paper proposes judgment theorems for the assignment reduct the distribution reduct and the maximum distribution reduct which are expected to greatly simplify the judging of these three types of reducts on this basis we derive three new types of attribute significance measures and construct the q ara quick assignment reduction algorithm the q dra quick distribution reduction algorithm and the q mdra quick maximum distribution reduction algorithm these three algorithms correspond to the three types of reducts we conduct a series of comparative experiments with twelve uci machine learning data repository university of california at irvine data sets including consistent and inconsistent decision tables to evaluate the performance of the three reduction algorithms proposed with the relevant algorithm quickreduct 9 34 the experimental results show that quickreduct possesses weak robustness because it cannot find the reduct even for consistent data sets whereas our proposed three algorithms show strong robustness because they can find the reduct for each data set in addition we compare the q dra quick distribution reduction algorithm with the cebarknc conditional entropy based algorithm for reduction of knowledge without a computing core 43 because both find the distribution reduct by using a heuristic search the experimental results demonstrate that q dra runs faster than cebarknc does because the distribution function of q dra has a lower calculation cost instructive conclusions for these reduction algorithms are drawn from the perspective of classification performance for the c4 5 and rbf svm classifiers last we make a comparison between discernibility matrix based methods and our algorithms the experimental results indicate that our algorithms are efficient and feasible

state and unknown input observers for nonlinear systems with bounded exogenous inputs a systematic design methodology for state observers for a large class of nonlinear systems with bounded exogenous inputs disturbance inputs and sensor noise is proposed the nonlinearities under consideration are characterized by an incremental quadratic constraint parameterized by a set of multiplier matrices linear matrix inequalities are developed to construct observer gains which ensure that a performance output based on the state estimation error satisfies a prescribed degree of accuracy furthermore conditions guaranteeing estimation of the unknown inputs to arbitrary degrees of accuracy are provided the proposed scheme is illustrated with a numerical example which does not satisfy the so called matching conditions

relative reducts in consistent and inconsistent decision tables of the pawlak rough set model a relative reduct can be considered as a minimum set of attributes that preserves a certain classification property this paper investigates three different classification properties and suggests three distinct definitions accordingly in the pawlak rough set model while the three definitions yield the same set of relative reducts in consistent decision tables they may result in different sets in inconsistent tables relative reduct construction can be carried out based on a discernibility matrix the study explicitly stresses a fact that the definition of a discernibility matrix should be tied to a certain property regarding the three classification properties we can define three distinct definitions accordingly based on the common structure of the specific definitions of relative reducts and discernibility matrices general definitions of relative reducts and discernibility matrices are suggested

support vector machines and the multiple hypothesis test problem two enhancements are proposed to the application and theory of support vector machines the first is a method of multicategory classification based on the binary classification version of the support vector machine svm the method which is called the m ary svm represents each category in binary format and to each bit of that representation is assigned a conventional svm this approach requires only log sub 2 k svms where k is the number of classes we give an example of classification on an octaphase shift keying 8 psk pattern space to illustrate the main concepts the second enhancement is that of adding equality constraints to the conventional binary classification svm this allows pinning the classification boundary to points that are known a priori to lie on the boundary applications of this method often arise in problems having some type of symmetry we present one such example where the m ary svm is used to classify symbols of a cdma two user multiuser detection pattern space

approximations for densities of sufficient estimators abstract a simple method of obtaining asymptotic expansions for the densities of sufficient estimators is described it is an extension of the one developed by o barndorff nielsen and d r cox 1979 for exponential families a series expansion in powers of n 1 is derived of which the first term has an error of order n 1 which can effectively be reduced to n 3 2 by renormalization the results obtained are similar to those given by h e daniels s 1954 saddlepoint method but the derivations are simpler a brief treatment of approximations to conditional densities is given theorems are proved which extend the validity of the multivariate edgeworth expansion to parametric families of densities of statistics which need not be standardized sums of independent and identically distributed vectors these extensions permit the treatment of problems arising in time series analysis the technique is used by j durbin 1980 to obtain approximations to the densities of partial serial correlation coefficients

an observer based controller for nonlinear systems a gain scheduling approach abstract this paper presents an observer based controller for discrete time nonlinear systems whose control inputs are subject to a set of equality and or inequality constraints firstly a regularized least square estimator is developed to estimate the states of the nonlinear system then the estimated states are used to generate a feedback control law using the jacobean linearization in conjunction with gain scheduling the stability of the closed loop control system is rigorously investigated applications of the proposed observer based control scheme to two numerical examples are presented then the developed controller is used to control a synchronous machine which is subject to constraints on the control the simulation results show the effectiveness of the proposed observer based control scheme

attribute reduction in decision theoretic rough set models rough set theory can be applied to rule induction there are two different types of classification rules positive and boundary rules leading to different decisions and consequences they can be distinguished not only from the syntax measures such as confidence coverage and generality but also the semantic measures such as decision monotocity cost and risk the classification rules can be evaluated locally for each individual rule or globally for a set of rules both the two types of classification rules can be generated from and interpreted by a decision theoretic model which is a probabilistic extension of the pawlak rough set model as an important concept of rough set theory an attribute reduct is a subset of attributes that are jointly sufficient and individually necessary for preserving a particular property of the given information table this paper addresses attribute reduction in decision theoretic rough set models regarding different classification properties such as decision monotocity confidence coverage generality and cost it is important to note that many of these properties can be truthfully reflected by a single measure c in the pawlak rough set model on the other hand they need to be considered separately in probabilistic models a straightforward extension of the c measure is unable to evaluate these properties this study provides a new insight into the problem of attribute reduction

finding rough and fuzzy rough set reducts with sat feature selection refers to the problem of selecting those input features that are most predictive of a given outcome a problem encountered in many areas such as machine learning pattern recognition and signal processing in particular solution to this has found successful application in tasks that involve datasets containing huge numbers of features in the order of tens of thousands which would otherwise be impossible to process further recent examples include text processing and web content classification rough set theory has been used as such a dataset pre processor with much success but current methods are inadequate at finding globally minimal reductions the smallest sets of features possible this paper proposes a technique that considers this problem from a propositional satisfiability perspective in this framework globally minimal subsets can be located and verified

attribute reduction based on evidence theory in incomplete decision systems attribute reduction is a basic issue in knowledge representation and data mining this paper deals with attribute reduction in incomplete information systems and incomplete decision systems based on dempster shafer theory of evidence the concepts of plausibility reduct and belief reduct in incomplete information systems as well as relative plausibility reduct and relative belief reduct in incomplete decision systems are introduced it is shown that in an incomplete information system an attribute set is a belief reduct if and only if it is a classical reduct and a plausibility consistent set must be a classical consistent set in a consistent incomplete decision system the concepts of relative reduct relative plausibility reduct and relative belief reduct are all equivalent in an inconsistent incomplete decision system an attribute set is a relative plausibility reduct if and only if it is a relative reduct a plausibility consistent set must be a belief consistent set and a belief consistent set is not a plausibility consistent set in general

twenty one ml estimators for model selection classical approaches to determine a suitable model structure from observed input output data are based on hypothesis tests and information based criteria recently the model structure has been considered as a stochastic variable and standard estimation techniques have been proposed the resulting estimators are closely related to the aforementioned methods however it turns out that there are a number of prior choices in the problem formulation which are crucial for the estimators behavior the contribution of this paper is to clarify the role of the prior choices to examine a number of possibilities and to show which estimators are consistent this is done in a linear regression framework for autoregressive models we also investigate a novel prior assumption on stability and give the estimator for the model order and the parameters themselves

conditional model order estimation a new approach to model order selection is proposed based on the theory of sufficient statistics the method does not require any prior knowledge of the model parameters it is able to discriminate between models by basing the decision on the part of the data that is independent of the model parameters this is accomplished conceptually by transforming the data into a sufficient statistic and an ancillary statistic with respect to the model parameters it is the probability density function of the ancillary statistic when adjusted for its dimensionality that is used to estimate the order furthermore the rule is directly tied to the goal of minimizing the probability of error and does not employ any asymptotic approximations the estimator can be shown to be consistent and via computer simulation is found to outperform the minimum description length estimator

high gain observers in the presence of measurement noise a switched gain approach this paper considers output feedback control using high gain observers in the presence of measurement noise for a class of nonlinear systems we study stability in the presence of measurement noise and illustrate the tradeoff when selecting the observer gain between state reconstruction speed and robustness to model uncertainty on the one hand versus amplification of noise on the other based on this tradeoff we propose a high gain observer that switches between two gain values this scheme is able to quickly recover the system states during large estimation error and reduce the effect of measurement noise in a neighborhood of the origin of the estimation error we argue boundedness and ultimate boundedness of the closed loop system under switched gain output feedback

error covariance matrix estimation of noisy and dynamically coupled time series we estimate the covariance matrix of the errors in several dynamically coupled time series corrupted by measurement errors we say that several scalar time series are dynamically coupled if they record the values of measurements of the state variables of the same smooth dynamical system the estimation of the covariance matrix of the errors is made using a noise reduction algorithm that efficiently exploits the information contained jointly in the dynamically coupled noisy time series the method is particularly powerful for short length time series with high uncertainties

on the proper forms of bic for model order selection the bayesian information criterion bic is often presented in a form that is only valid in large samples and under a certain condition on the rate at which the fisher information matrix fim increases with the sample length this form has been improperly used previously in situations in which the conditions mentioned above do not hold in this correspondence we describe the proper forms of bic in several practically relevant cases that do not satisfy the above assumptions in particular we present a new form of bic for high signal to noise ratio snr cases the conclusion of this study is that bic remains one of the most successful existing rules for model order selection if properly used

a nonlinear high gain observer for systems with measurement noise in a feedback control framework we address the problem of state estimation for a class of nonlinear systems with measurement noise in the context of feedback control it is well known that high gain observers are robust against model uncertainty and disturbances but sensitive to measurement noise when implemented in a feedback loop this work presents the benefits of a nonlinear gain structure in the innovation process of the high gain observer in order to overcome the tradeoff between fast state reconstruction and measurement noise attenuation the goal is to generate a larger observer gain during the transient response than in the steady state response thus by reducing the observer gain after achieving satisfactory state estimates the effect of noise on the steady state performance is reduced moreover the nonlinear gain observer presented in this paper is shown to surpass the system performance achieved when using comparable linear gain observers the proof argues boundedness and ultimate boundedness of the closed loop system under the proposed output feedback

a high gain observer with updated gain for a class of mimo non triangular systems a high gain like observer with updated gain is proposed for a class of cascade nonlinear and non triangular systems that are observable for any input the objective of the gain adaptation is to perform an admissible tradeoff between state reconstruction dynamics on the one hand versus noise amplification on the other hand to this end the gain of the proposed observer is tuned through a single scalar time varying parameter governed by an adequate differential riccati equation the involved adaptation process is mainly driven by the power of the output observation error norm computed on a moving horizon window simulation results are given to show the effectiveness of the proposed observer namely its exponential convergence and its insensitivity with respect to noise measurements

state space reconstruction in the presence of noise takens theorem demonstrates that in the absence of noise a multidimensional state space can be reconstructed from a scalar time series this theorem gives little guidance however about practical considerations for reconstructing a good state space we extend takens treatment applying statistical methods to incorporate the effects of observational noise and estimation error we define the distortion matrix which is proportional to the conditional covariance of a state given a series of noisy measurements and the noise amplification which is proportional to root mean square time series prediction errors with an ideal model we derive explicit formulae for these quantities and we prove that in the low noise limit minimizing the distortion is equivalent to minimizing the noise amplification we identify several different scaling regimes for distortion and noise amplification and derive asymptotic scaling laws when the dimension and lyapunov exponents are sufficiently large these scaling laws show that no matter how the state space is reconstructed there is an explosion in the noise amplification from a practical point of view determinism is lost and the time series is effectively a random process in the low noise large data limit we show that the technique of local singular value decomposition is an optimal coordinate transformation in the sense that it achieves the minimum distortion in a state space of the lowest possible dimension however in numerical experiments we find that estimation error complicates this issue for local approximation methods we analyze the effect of reconstruction on estimation error derive a scaling law and suggest an algorithm for reducing estimation errors

asymptotic map criteria for model selection the two most popular model selection rules in signal processing literature have been akaike s 1974 criterion aic and rissanen s 1978 principle of minimum description length mdl these rules are similar in form in that they both consist of data and penalty terms their data terms are identical but the penalties are different mdl being more stringent toward overparameterization aic penalizes for each additional model parameter with an equal incremental amount of penalty regardless of the parameter s role in the model in most of the literature on model selection mdl appears in a form that also suggests equal penalty for every unknown parameter this mdl criterion we refer to as naive mdl in this paper we show that identical penalization for every parameter is not appropriate and that the penalty has to depend on the model structure and type of model parameters the approach to showing this is bayesian and it relies on large sample theory we derive maximum a posteriori map rules for several different families of competing models and obtain forms that are similar to aic and naive mdl for some families however we find that the derived penalties are different in those cases our extensive simulations show that the map rule outperforms aic and naive mdl

convergent fault estimation for linear systems with faults and disturbances the problem of convergent fault estimation for a class of linear systems with faults and disturbances is addressed for the first time in this technical note a differential equation based convergence theorem of mean sequence of estimates is proposed which provides an iterative mean estimation technique that has not been fully reported yet in the literature based on the technique a convergent fault estimation strategy for the linear system in consideration is designed and the convergence of mean sequence of fault estimates is proved theoretically an iterative algorithm is given subsequently to compute the fault estimates numerical simulation results demonstrate the effectiveness and merits of the proposed method

class specific attribute reducts in rough set theory abstract the concept of attribute reducts plays a fundamental role in rough set analysis there are at least two possibilities to define an attribute reduct a classification based or global attribute reduct is a minimal subset of condition attributes that preserves the positive region of the decision classification namely the positive regions of all decision classes in a decision table a class specific class dependent or local attribute reduct is a minimal subset of condition attributes that preserves the positive region of a particular decision class while a classification based reduct may not work equally well for every decision class a class specific attribute reduct is optimally tailored to a particular decision class however studies in rough set theory are dominated by classification based reducts there is very limited investigation on class specific reducts an objective of this paper is to draw attention to class specific reducts we systematically compare the two types of reducts and investigate their relationships with respect to both individual reducts and families of all reducts it is possible to derive a class specific reduct from a classification based reduct and to derive a classification based reduct from a family of class specific reducts the families of all class specific reducts provide a pair of lower and upper bounds of the family of all classification based reducts based on a three way classification of attributes into the pair wise disjoint sets of core marginal and nonuseful attributes we examine relationships between the corresponding classes of classification based and class specific attributes the union of the sets of class specific core attributes is the set of classification based core attributes it is only possible to obtain an upper bound for the set of classification based marginal attributes and a lower bound for the set of classification based nonuseful attributes from the family of the class specific corresponding sets of attributes

knowledge reduction based on the equivalence relations defined on attribute set and its power set one of the key problems of knowledge discovery is knowledge reduction this paper proposes a new method for knowledge reduction in information systems first two families of closed sets c r and c r are defined where r and r are equivalence relations defined on the attribute set and its power set respectively the properties of c r and c r are also discussed the necessary and sufficient condition for c r c r is then given and employed to construct an approach to attribute reduction in information systems it is also proved that under the condition c r c r the proposed approach to knowledge reduction is equivalent to the well accepted one in reference w x zhang y leung w z wu information systems and knowledge discovery science publishing company beijing 2003

maximum entropy pdf design using feature density constraints applications in signal processing this paper revisits an existing method of constructing high dimensional probability density functions pdfs based on the pdf at the output of a dimension reducing feature transformation we show how to modify the method so that it can provide the pdf with the highest entropy among all pdfs that generate the given low dimensional pdf the method is completely general and applies to arbitrary feature transformations the chain rule is described for multi stage feature calculations typically used in signal processing examples are given including mfcc and auto regressive features experimental verification of the results using simulated data is provided including a comparison with competing generative methods

observer design based on immersion technics and canonical form abstract a crucial problem in immersion based observer design is the case where the dimension of the immersed system is greater than that of the original system and the analytical expression of the inverse of the immersion is unknown in this paper we have proposed a method for constructing observers for such autonomous nonlinear systems

distributed moving horizon state estimation of two time scale nonlinear systems abstract in this paper we focus on distributed moving horizon estimation dmhe for a class of two time scale nonlinear systems described in the framework of singularly perturbed systems by taking advantage of the time scale separation property a two time scale system is first decomposed into a reduced order fast system and a reduced order slow system the slow system is further decomposed into several interconnected slow subsystems in the proposed distributed state estimation scheme a local estimator is designed for each slow subsystem and for the reduced order fast system the slow subsystem estimators communicate with each other to exchange information and they are only required to send information to the fast system one directionally the fast system estimator does not send out any information the local estimators are designed as observer enhanced moving horizon estimators sufficient conditions on the convergence of the estimation error of the dmhe are derived the application of the proposed dmhe to a chemical process example demonstrates its applicability and effectiveness

saddlepoint approximations of the distribution of the person parameter in the two parameter logistic model large sample theory states the asymptotic normality of the maximum likelihood estimator of the person parameter in the two parameter logistic 2pl model in short tests however the assumption of normality can be grossly wrong as a consequence intended coverage rates may be exceeded and confidence intervals are revealed to be overly conservative methods belonging to the higher order theory more specifically saddlepoint approximations are a convenient way to deal with small sample problems confidence bounds obtained by these means hold the approximate confidence level for a broad range of the person parameter moreover an approximation to the exact distribution permits to compute median unbiased estimates mue that are as likely to overestimate as to underestimate the true person parameter additionally in small samples these mue are less mean biased than the often used maximum likelihood estimator

a proof of the gamma test from a dataset of input output vectors the gamma test estimates the variance of the noise on an output modulo any smooth model with bounded partial derivatives we present a proof of the gamma test under fairly weak hypotheses

dynamic observers as asymptotic limits of recursive filters special cases a method for constructing observers for dynamical systems as asymptotic limits of filters is described the program is carried out in detail for linear systems and in addition an observer is obtai

model estimation and classification via model structure determination in model estimation we often face problems with unknown parameters in the candidate models this paper proposes the model structure determination msd for model estimation with unknown parameters we start with the problem of model order selection and decompose the probability density function pdf into the information provided by the data about the model parameters and that of the model structure the factor that depends on the model parameters is approximated using a minimax procedure and the msd depends on the model structure only it is shown that the msd is equivalent to the exponentially embedded family eef for model order selection under some conditions finally we apply the msd to a classification problem where we have partial knowledge about the parameters and simulation results show that it outperforms the pseudo maximum likelihood pseudo ml rule

model order selection rules for covariance structure classification in radar the adaptive classification of the interference covariance matrix structure for radar signal processing applications is addressed in this paper this represents a key issue because many detection architectures are synthesized assuming a specific covariance structure which may not necessarily coincide with the actual one due to the joint action of the system and environment uncertainties the considered classification problem is cast in terms of a multiple hypotheses test with some nested alternatives and the theory of model order selection mos is exploited to devise suitable decision rules several mos techniques such as the akaike takeuchi and bayesian information criteria are adopted and the corresponding merits and drawbacks are discussed at the analysis stage illustrating examples for the probability of correct model selection are presented showing the effectiveness of the proposed rules

convergence acceleration for observers by gain commutation increasing convergence rates of observers and differentiators for a class of nonli near systems in the output canonical form under presence of bounded matched disturbances lipschitz uncertainties and measurement noises is investigated a supervisory algorithm is designed that switches among different values of observer gains to accelerate the estimation in the noise free case the presented switched gain observer ensures global uniform time of convergence of the estimation error to the origin in the presence of noise the goals of overshoot reducing for the initial phase acceleration of convergence and improvement of asymptotic precision of estimation are achieved efficiency of the proposed switching gain observer is demonstrated by numerical comparison with a sliding mode and linear high gain observers

asymptotically optimal approximation of multidimensional pdf s by lower dimensional pdf s probability density functions pdf s of high dimensionality are impractical to estimate from real data for accurate estimation the dimensionality of the pdf can be at most 5 10 in order to reduce the dimensionality a sufficient statistic is usually employed when none is available there is no universal agreement on how to proceed we show how to construct a high dimension pdf based on the pdf of a low dimensional statistic that is closest to the true one in the sense of divergence the latter criterion asymptotically minimizes the probability of error in a decision rule an application to feature selection for classification is described

the pdf projection theorem and the class specific method we present the theoretical foundation for optimal classification using class specific features and provide examples of its use a new probability density function pdf projection theorem makes it possible to project probability density functions from a low dimensional feature space back to the raw data space an m ary classifier is constructed by estimating the pdfs of class specific features then transforming each pdf back to the raw data space where they can be fairly compared although statistical sufficiency is not a requirement the classifier thus constructed becomes equivalent to the optimal bayes classifier if the features meet sufficiency requirements individually for each class this classifier is completely modular and avoids the dimensionality curse associated with large complex problems by recursive application of the projection theorem it is possible to analyze complex signal processing chains we apply the method to feature sets including linear functions of independent random variables cepstrum and mel cepstrum in addition we demonstrate how it is possible to automate the feature and model selection process by direct comparison of log likelihood values on the common raw data domain

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

sufficiency classification and the class specific feature theorem a new proof of the class specific feature theorem is given the proof makes use of the observed data as opposed to the set of sufficient statistics as in the original formulation we prove the theorem for the classical case in which the parameter vector is deterministic and known as well as for the bayesian case in which the parameter vector is modeled as a random vector with known prior probability density function the essence of the theorem is that with a suitable normalization the probability density function of the sufficient statistic for each probability density function family can be used for optimal classification one need not have knowledge of the probability density functions of the data under each hypothesis

efficient attribute reduction from the viewpoint of discernibility attribute reduction is an important preprocessing step in pattern recognition machine learning and data mining as an effective method for attribute reduction rough set theory offers a useful and formal methodology it retains the discernibility power of the original datasets thus attribute reduction has been extensively studied in rough set theory however the inefficiency of the existing attribute reduction algorithms limits the application of rough sets in this paper we first analyse the limitations of existing attribute reduction algorithms then a novel measure of attribute quality called the relative discernibility degree is proposed based on the discernibility theoretical analysis shows that this measure can find relative dispensable attributes and remain unchanged after removing the relative dispensable attributes and redundant objects in the process of selecting attributes this property can be used to reduce the search space and accelerate the heuristic process of attribute reduction consequently a new attribute reduction algorithm is proposed from the viewpoint of discernibility furthermore the relationships among the reduction definitions of the algebra view information view and discernibility view are derived some non equivalent relationships among these views of rough set theory in inconsistent decision tables are discovered a set of numerical experiments was conducted on uci datasets experimental results show that the proposed algorithm is effective and efficient and is applicable to the case of large scale datasets

reconstruction expansion as a geometry based framework for choosing proper delay times abstract the quality of attractor reconstruction using the method of delays is known to be sensitive to the delay parameter here we develop a new computationally efficient approach to choosing that quantifies reconstruction expansion from the identity line of the embedding space we show that reconstruction expansion is related to the concept of reconstruction signal strength and that increased expansion corresponds to diminished effects of measurement error thus reconstruction expansion represents a simple geometrical framework for choosing furthermore we describe the role of dynamical error in attractor expansion and argue that algorithms for determining should be considered as attempts at estimating an upper bound to the optimal delay

on the stability of a differential riccati equation for continuous discrete observers abstractthis article deals with the stability properties of a continuous discrete riccati differential equation the main motivation comes from the theory of kalman filters for continuous time nonlinear systems with sampled measurements where this type of equation often arises stability is to be understood in the following sense under appropriate hypotheses the solution matrix is always symmetric positive definite and bounded from above and below no uniformity is expected from the sampling procedure only an upper bound on the elapsed time between consecutive samples is needed the exposition consists of two parts first the stability properties are proven and second by means of three examples it is shown how the article s main result applies

high snr consistent linear model order selection and subset selection high snr consistency of model order selection criteria in linear regression models has attracted a lot of attention in the signal processing community recently it is now known that exponentially embedded family versions of minimum description length etc are high snr consistent however a general framework for high snr consistency in linear regression is still missing this paper fills this gap by deriving necessary and sufficient conditions nscs for model order selection criteria in both known and unknown noise variance situations to be high snr consistent many popular model order selection criteria are proved to be high snr consistent using these nscs we also provide a convergence rate analysis to discriminate between various high snr consistent model order selection criteria a direct application of model order selection techniques to the very important subset selection problem is computationally infeasible this paper establish the high snr consistency of an existing t statistics based index ordering scheme that allows the conversion of a subset selection problem into a model order selection problem combining this index ordering with high snr consistent model order selection criteria leads to a novel subset selection procedure ssp that is numerically shown to outperform existing high snr consistent ssps

nullity based matroid of rough sets and its application to attribute reduction rough sets were proposed to deal with vagueness and incompleteness of knowledge in information systems in this field there are many optimization issues such as attribute reduction matroids generalized from matrices have been widely used in many fields particularly greedy algorithm design which plays an important role in attribute reduction therefore it is meaningful to combine matroids with rough sets to solve the optimization problems in this paper we construct a type of matroid of rough sets based on the concept of nullity and apply it to attribute reduction first we propose a nullity operator for rough sets to induce a matroid and then we present a specific type of matroid called a nullity based matroid second given the relationship between nullities and matrices we present two types of matrices to characterize this type of matroid and its nullity operator third the dual of this type of matroid is induced by the second type of matrix finally we apply the obtained matroids to attribute reduction issues in information systems in summary this paper provides a new approach to studying rough sets

a strong tracking extended kalman observer for nonlinear discrete time systems the authors show how the extended kalman filter used as an observer for nonlinear discrete time systems or extended kalman observer eko becomes a useful state estimator when the arbitrary matrices namely r sub k and q sub k are adequately chosen as a first step we use the linearization technique given by boutayed et al 1997 which consists of introducing unknown diagonal matrices to take the approximation errors into account it is shown that the decreasing lyapunov function condition leads to a linear matrix inequality lmi problem which points out the connection between a good convergence behavior of the eko and the instrumental matrices r sub k and q sub k in order to satisfy the obtained lmi a particular design of q sub k is given high performances of the proposed technique are shown through numerical examples under the worst conditions

observer design for a class of uncertain nonlinear systems with sampled outputs application to the estimation of kinetic rates in bioreactors a continuous discrete time observer is proposed for a class of uncertain nonlinear systems where the output is available only at non uniformly spaced sampling instants the underlying correction term depends on the output observation error and is updated in a mixed continuous discrete fashion the proposed observer is first introduced under a set of differential equations with instantaneous state impulses corresponding to the measured samples and their estimates two features of the proposed observer are worth to be pointed out the first one consists in the simplicity of its calibration while the second one lies in its comprehensive convergence analysis more specifically it is shown that in the case of noise free sampled outputs the observation error lies in a ball centered at the origin and its radius is proportional to the bounds of the uncertainties and the sampling partition diameter moreover in the free uncertainties case the exponential convergence to zero of the observation error is established under a well defined condition on the maximum value of the sampling partition diameter the ability of the proposed observer to perform a suitable estimation of the reactions rates in biochemical reactors is highlighted through a simulation study dealing with an ethanolic fermentation

observer design for uncertain nonlinear systems with unmodeled dynamics this paper is concerned with the observer design problem for a class of multi input multi output nonlinear systems with the unmodeled dynamics unknown parameters and external disturbance a dynamic signal which can dominate the unmodeled dynamics is firstly constructed then two types of observer schemes that is adaptive observer and robust observer are respectively proposed the observation error in the two schemes can be made arbitrarily small by choosing the appropriate design parameters an illustrative example is provided to demonstrate the validity of the proposed design methods

peaking free output feedback adaptive neural control under a nonseparation principle high gain observers have been extensively applied to construct output feedback adaptive neural control anc for a class of feedback linearizable uncertain nonlinear systems under a nonlinear separation principle yet due to static gain and linear properties high gain observers are usually subject to peaking responses and noise sensitivity existing adaptive neural network nn observers cannot effectively relax the limitations of high gain observers this paper presents an output feedback indirect anc strategy under a nonseparation principle where a hybrid estimation scheme that integrates an adaptive nn observer with state variable filters is proposed to estimate plant states by applying a single lyapunov function candidate to the entire system it is proved that the closed loop system achieves practical asymptotic stability under a relatively low observer gain dominated by controller parameters our approach can completely avoid peaking responses without control saturation while keeping favourable noise rejection ability simulation results have shown effectiveness and superiority of this approach

efficiency of the extended kalman filter for nonlinear systems with small noise the problem of nonlinear filtering is studied asymptotically as the noise tends to zero detectability conditions ensuring that the filtering error tends to zero are derived and under these conditions it is proved that the extended kalman filter provides a good suboptimal filter the smoothing problem is also studied the proofs use some estimates for the linearization of stochastic systems some changes of probability and stochastic differentiation techniques