a new kernel based approach for linear system identification this paper describes a new kernel based approach for linear system identification of stable systems we model the impulse response as the realization of a gaussian process whose statistics differently from previously adopted priors include information not only on smoothness but also on bibo stability the associated autocovariance defines what we call a stable spline kernel the corresponding minimum variance estimate belongs to a reproducing kernel hilbert space which is spectrally characterized compared to parametric identification techniques the impulse response of the system is searched for within an infinite dimensional space dense in the space of continuous functions overparametrization is avoided by tuning few hyperparameters via marginal likelihood maximization the proposed approach may prove particularly useful in the context of robust identification in order to obtain reduced order models by exploiting a two step procedure that projects the nonparametric estimate onto the space of nominal models the continuous time derivation immediately extends to the discrete time case on several continuous and discrete time benchmarks taken from the literature the proposed approach compares very favorably with the existing parametric and nonparametric techniques

intrinsic difficulties in using the doubly infinite time axis for input output control theory points out that the natural definitions of stability and causality in input output control theory lead to certain inconsistencies when inputs and outputs are allowed to have support on the doubly infinite time axis in particular linear time invariant systems with right half plane poles cannot be considered to be both causal and stabilizable in contrast there is no such conflict when the semi infinite time axis is used

maximum likelihood estimators and worst case optimal algorithms for system identification abstract for system identification problems with stochastic noise maximum likelihood estimators are frequently used if noise is deterministic worst case optimal algorithms should be considered in this paper we study the following problem under what circumstances are maximum likelihood estimators optimal in the worst case

strong consistence of recursive identification for wiener systems the paper concerns identification of the wiener system consisting of a linear subsystem followed by a static nonlinearity f with no invertibility and structure assumption recursive estimates are given for coefficients of the linear subsystem and for the value f v at any fixed v the main contribution of the paper consists in establishing convergence with probability one of the proposed algorithms to the true values this probably is the first strong consistency result for this kind of wiener systems a numerical example is given which justifies the theoretical analysis

optimal sampling schedule for parameter estimation of linear models with unknown but bounded measurement errors the problem of optimal sampling design for parameter estimation when data are generated by linear models is addressed the measurements are assumed to be corrupted by an unknown but bounded additive noise the sampling design assumes that the number of samples is unconstrained and no replication is allowed two main results are shown 1 for particular classes of linear models the optimal number of measurements is equal to the number of parameters as in the statistical context 2 the uncertainty intervals of the parameter estimates are bounded from above by quantities that can be computer a priori knowing only the model and the error structure

compositional strategy synthesis for stochastic games with multiple objectives abstract design of autonomous systems is facilitated by automatic synthesis of controllers from formal models and specifications we focus on stochastic games which can model interaction with an adverse environment as well as probabilistic behaviour arising from uncertainties our contribution is twofold first we study long run specifications expressed as quantitative multi dimensional mean payoff and ratio objectives we then develop an algorithm to synthesise e optimal strategies for conjunctions of almost sure satisfaction for mean payoffs and ratio rewards in general games and boolean combinations of expected mean payoffs in controllable multi chain games second we propose a compositional framework together with assume guarantee rules which enables winning strategies synthesised for individual components to be composed to a winning strategy for the composed game the framework applies to a broad class of properties which also include expected total rewards and has been implemented in the software tool prism games

adaptive filtering using quantized output measurements a normalized stochastic gradient adaptive filtering algorithm based on a finite impulse response fir model is discussed the algorithm identifies the system exactly given only coarsely quantized output measurements a description of the quantizer is included in the overall input output model and the scheme exploits an approximation of the derivative of the quantizer using an associated differential equation global convergence is established to a zero output error except for possible colored measurement disturbances parameter setting or to the boundary of the model set

asymptotic properties of sign algorithms for adaptive filtering this paper develops asymptotic properties of a class of sign error algorithms with expanding truncation bounds for adaptive filtering under merely stationary ergodicity and finite second moments of the reference and output signals and using trajectory subsequence ts method it is proved that the algorithm convergers almost surely then a mean squares estimate is derived for the estimation error and a suitably scaled sequence of the estimation error is shown to converge to a diffusion process the scaling factor together with the stationary covariance gives the rate of convergence result moreover an algorithm under mean squares criterion with expanding truncation bounds is also examined compared with the existing results in the literature sufficient conditions for almost sure convergence are much relaxed a simple example is provided for demonstration purpose

optimal estimation theory for dynamic systems with set membership uncertainty an overview in many problems such as linear and nonlinear regressions parameter and state estimation of dynamic systems state space and time series prediction interpolation smoothing and functions approximation one has to evaluate some unknown variable using available data the data are always associated with some uncertainty and it is necessary to evaluate how this uncertainty affects the estimated variables typically the problem is approached assuming a probabilistic description of uncertainty and applying statistical estimation theory an interesting alternative approach referred to as set membership or unknown but bounded ubb error description has been investigated since the late 1960s in this approach uncertainty is described by an additive noise which is known only to have given integral typically l2 of l1 or componentwise l bounds in this paper we review the main results of this theory with special attention to the most recent advances obtained in the case of componentwise bounds

identification of discrete time output error model using filtered input excitation for integrating processes with time delay a discrete time output error oe model identification method is proposed for industrial integrating processes with time delay to facilitate the discrete time controller design and implementation in sampled systems by subtly introducing a differential filter to the input excitation an open loop identification test can be performed for such a process while complying with the output limit correspondingly the parameter estimation is equivalently transformed to identify a stable type oe model with time delay a recursive identification algorithm is developed to simultaneously estimate all the model parameters including the integer delay and a linear interpolation of the input excitation is given for the recursive computation to guarantee consistent estimation the convergence of parameter estimation is analyzed with a proof an illustrative example is used to demonstrate the effectiveness of the proposed method

sequential kernel density approximation and its application to real time visual tracking visual features are commonly modeled with probability density functions in computer vision problems but current methods such as a mixture of gaussians and kernel density estimation suffer from either the lack of flexibility by fixing or limiting the number of gaussian components in the mixture or large memory requirement by maintaining a nonparametric representation of the density these problems are aggravated in real time computer vision applications since density functions are required to be updated as new data becomes available we present a novel kernel density approximation technique based on the mean shift mode finding algorithm and describe an efficient method to sequentially propagate the density modes over time although the proposed density representation is memory efficient which is typical for mixture densities it inherits the flexibility of nonparametric methods by allowing the number of components to be variable the accuracy and compactness of the sequential kernel density approximation technique is illustrated by both simulations and experiments sequential kernel density approximation is applied to online target appearance modeling for visual tracking and its performance is demonstrated on a variety of videos

spatiotemporal gmm for background subtraction with superpixel hierarchy we propose a background subtraction algorithm using hierarchical superpixel segmentation spanning trees and optical flow first we generate superpixel segmentation trees using a number of gaussian mixture models gmms by treating each gmm as one vertex to construct spanning trees next we use the m smoother to enhance the spatial consistency on the spanning trees and estimate optical flow to extend the m smoother to the temporal domain experimental results on synthetic and real world benchmark datasets show that the proposed algorithm performs favorably for background subtraction in videos against the state of the art methods in spite of frequent and sudden changes of pixel values

asymptotically efficient parameter estimation using quantized output observations this paper studies identification of systems in which only quantized output observations are available an identification algorithm for system gains is introduced that employs empirical measures from multiple sensor thresholds and optimizes their convex combinations strong convergence of the algorithm is first derived the algorithm is then extended to a scenario of system identification with communication constraints in which the sensor output is transmitted through a noisy communication channel and observed after transmission the main results of this paper demonstrate that these algorithms achieve the cramer rao lower bounds asymptotically and hence are asymptotically efficient algorithms furthermore under some mild regularity conditions these optimal algorithms achieve error bounds that approach optimal error bounds of linear sensors when the number of thresholds becomes large these results are further extended to finite impulse response and rational transfer function models when the inputs are designed to be periodic and full rank

parameter identification of wiener systems with discontinuous nonlinearities abstract the paper deals with modeling and parameter identification of wiener type nonlinear dynamic systems having discontinuous asymmetric piecewise linear characteristics with preloads and dead zones multiple application of a decomposition technique provides a special form of static nonlinearity description its incorporation into the wiener model leads to an input output equation where all the model parameters are separated and can be estimated iteratively using input output data and internal variable estimates more illustrative examples are included to demonstrate the feasibility of the proposed approach

experiment design in a bounded error context comparison with d optimality abstract the problem of optimal experiment design for linear regression models is addressed the measurement errors are supposed to be bounded with no other hypothesis on their distribution contrary to the classical statistical approach an experiment is considered here as optimal if it minimizes the volume of the set of all parameters that are consistent with the data the model structure and the noise bounds this new design policy is compared to the classical d optimal design used in a statistical context when the number of measurements is equal to the number of parameters to be estimated the new optimality criterion although relying on quite different hypotheses on the noise distribution is shown to lead to the same optimal policy as a d optimality criterion this is no longer true when more measurements are to be made depending on the shape of the admissible experimental domain it is sometimes possible to design optimal experiments without any calculations an algorithmic procedure is suggested to be used when this is not so

δ entropy definition properties and applications in system identification with quantized data recently the minimum error entropy criterion an information theoretic alternative to the traditional mean square error criterion has been successfully used in the contexts of machine learning and signal processing for system identification however the mee criterion will be no longer suitable if the training data are discrete valued since minimizing error s discrete entropy cannot constrain error s dispersion in this paper to make the mee criterion suitable for the discrete valued data cases we give a new entropy definition for the discrete random variables i e the d entropy based on riemann sums for finite size partitions a probability weighted formula is established to calculate the average partition this new entropy retains some important properties of the differential entropy and reduces to discrete entropy under certain conditions unlike discrete entropy the d entropy is sensitive to the dynamic range of the data and can be used as a superior optimality criterion in system identification problems also we present a plug in estimate of d entropy analyze its asymptotic behavior and explore the links to the kernel based and m spacing based estimates for differential entropy finally the d entropy criterion is applied in system identification with coarsely quantized input output data to search for the optimum parameter set monte carlo simulations demonstrate the performance improvement that may be achieved with the d entropy criterion

an optimal two stage identification algorithm for hammerstein wiener nonlinear systems abstract an optimal two stage identification algorithm is presented for hammerstein wiener systems where two static nonlinear elements surround a linear block the proposed algorithm consists of two steps the first one is the recursive least squares and the second one is the singular value decomposition of two matrices whose dimensions are fixed and do not increase as the number of the data point increases moreover the algorithm is shown to be convergent in the absence of noise and convergent with probability one in the presence of white noise

on the value of information in system identification bounded noise case assuming instantaneous bounds on the noise system parameter identification is formulated as membership set estimation problem sequential algorithms are constructed to estimate the membership sets of the parameters which are consistent with the measurements and the noise constraints the important new feature of the proposed algorithms is their ability to ignore redundant data the efficient data extraction property of the new algorithms is achieved with small computational effort and with improved performance when compared to the least square algorithm the convergence properties and the notion of identifiability in the set theoretic context are also studied

identification and application of bounded parameter models abstract the place of parameter bounding algorithms in identification methodology is discussed such algorithms provide a radical alternative to the computation of parameter point estimates and covariances instead of p d f or mean and covariance for the noise and prior parameter estimates they require bounds in an essentially deterministic model formulation the paper is a preliminary review of some standard tasks namely experiment design testing for outliers toleranced prediction and worst case control design in the context of parameter bounding identification computational requirements of these tasks are examined and the limitations of existing parameter bounding algorithms are noted

consistent parameter bounding identification for linearly parametrized model sets abstract in standard parameter bounding identification a time domain bound on the noise signal is used to construct parameter bounds one of the properties of this standard approach is that there is no consistency if a conservative noise bound has been chosen it is shown that alternative bounds on the noise signal give rise to consistent parameter bounding identification methods i e asymptotically in the number of data samples the feasible parameter set converges to the true parameter vector the first alternative noise bound that is introduced is a bound on the cross covariance between the noise and some instrumental input signal the noise bound is represented by a small number of linear inequalities which can be used in parameter bounding by linear programming it is shown that there is consistency under fairly general conditions even when a conservative bound has been chosen additionally a procedure is presented to estimate the cross covariance bound from data similar consistency results are shown for two other types of noise bounds a bound on the discrete fourier transform of the noise in combination with sinusoidal excitation and a time domain bound on the noise after measurement averaging with periodic excitation

optimal algorithms theory for robust estimation and prediction this paper deals with the theory of optimal algorithms for problems which cannot be solved exactly the theory developed allows for the derivation of new and interesting results in parameter estimation and in time series prediction in situations where no reliable statistical hypothesis can be made on the functions and modeling errors involved but only a bound on them is known in particular the derivation of computationally simple optimal algorithms for these two problems is investigated the practical effectiveness of the algorithms obtained is illustrated by several numerical examples

analytical methods for parameter space delimination and application to shallow lake phytoplankton dynamics modeling the first step in parameter estimation is to reduce the dimensionality of the problem by deriving estimates from independent experimentation and from the literature in addition insensitive parameters are either removed or fixed in the remaining lower dimensional problem parameter space delimitation is possible by analytical means three conjunctive methods are derived period average analysis extremum analysis and quasisteady state analysis the basic idea is to find conditions for the parameters that must be fulfilled in order to comply with average and extreme values in the observations the approach is applied to the modeling of the phytoplankton dynamics of lake balaton the analytical techniques prove to supply valuable insight into parameter interrelationships and model adequacy and can serve as satisfactory substitutes for formal parameter estimation techniques in the early stages of model development

convergence analysis of an adaptive filter equipped with the sign sign algorithm convergence with probability 1 of the sign sign algorithm with a decreasing step size is proven when it is used in adaptive plant identification the plant input and plant noise are assumed stationary and m dependent the assumptions do not pose any upper bound on the memory m of the plant input and plant noise and hence they allow arbitrarily strong correlation in these signals a main interest of the paper is that it provides a condition of persistent excitation of the algorithm examples satisfying this condition are provided

computation of nonconservative stability perturbation bounds for systems with nonlinearly correlated uncertainties consideration is given to the problem of robust stability analysis of linear dynamic systems with uncertain physical parameters entering as polynomials in the state equation matrices a method is proposed giving necessary and sufficient conditions for computing the uncertain system stability margin in parameter space which provides a measure of maximal parameter perturbations preserving stability of the perturbed system around a known stable nominal system a globally convergent optimization algorithm that enables solutions to the problem to be obtained is presented using the polynomial structure of the problem the algorithm generates a convergent sequence of interval estimates of the global extremum these intervals provide a measure of the accuracy of the approximating solution achieved at each step of the iterative procedure some numerical examples are reported showing attractive features of the algorithm from the point of view of computational burden and convergence behavior

recursive identification for wiener model with discontinuous piece wise linear function this paper deals with identification of wiener systems with nonlinearity being a discontinuous piece wise linear function recursive estimation algorithms are proposed to estimate six unknown parameters contained in the nonlinearity and all unknown coefficients of the linear subsystem by using the iid gaussian inputs the estimates are proved to converge to the corresponding true values with probability one a numerical example is given to justify the obtained theoretical results

quiver diagrams and signed adaptive filters a graphical technique for exploring the behavior of deterministic discrete time adaptive algorithms excited by periodic inputs is examined the constraint to periodic inputs is convenient the underlying concepts used also apply to stationary inputs drawn from finite alphabets the resulting graph is based on plotting single period parameter trajectories in the parameter error space and is essentially a discrete version of flow diagrams associated with continuous time systems for the least mean square lms adaptive fir filter with the algorithm step size sufficiently small the change of adapted parameters from a particular initial value is given by the total update over the input period with the parameterization frozen at its initial value this results in the familiar elliptical hyperparaboloid surface steepest descent interpretation this method can be extended to the lms adaptive infinite input response iir filter with suitable approximations based on a small step size

on autoregressive models the parsimony principle and their use in control oriented system identification a perspective is given on the use and limitations of autoregressive type models and the parsimony principle in the context of control oriented system identification it is demonstrated that the non systematic application of the parsimony principle tends to favour low order autoregressive type models that can be detrimental in subsequent control design especially for unstable or poorly damped systems furthermore an existence condition for smooth coprime factorizations and system stabilizability is given

persistent identification of time varying systems identification of time varying systems especially slowly time varying systems is of importance in the development of a comprehensive theory of adaptation the persistent identification measures employed in this paper capture a main characterization in such identification problems namely one input signal must be used for identification of all possible observation windows this paper establishes several essential features in persistent identification problems which highlight their potential utility in adaptation 1 they have computable upper and lower bounds for typical classes of prior uncertainty sets 2 any full rank n periodic signals are optimal and the simple least squares estimates are optimal identification algorithms 3 optimal probing inputs can be approximately generated in a closed loop configuration when the plant and the controller are slowly time varying and 4 n periodic signals are asymptotically optimal for slowly time varying systems the main results of this paper have been successfully combined with a certain slow h sup spl infin design to derive an adaptive stabilization scheme

robust estimation and filtering in the presence of bounded noise optimal algorithms for robust estimation and filtering are constructed the noise is considered a deterministic variable belonging to a set described by a norm previous results obtained for complete one to one and approximate information by m milanese and r tempo ibid vol 30 p 730 8 1985 are extended to partial and approximate information this information is considered useful in dealing with dynamic systems not completely identifiable and or with two different sources of noise such as process and measurement noise for different norms characterizing the noise optimal algorithms in a min max sense are shown in particular for hilbert norms a linear optimal algorithm is the well known minimum variance estimator for l sub infinity norm an optimal algorithm computable by linear programming is presented state estimation is formalized in the context of the general theory for stable systems an approximate state estimation is obtained by neglecting higher order powers an upper bound determining the approximation introduced is derived a numerical example illustrates the application of the theory

optimal inner bounds of feasible parameter set in linear estimation with bounded noise some problems arising in parameter estimation theory with unknown but bounded measurement errors are discussed in this theory a key role is played by the feasible parameter set i e the set of all parameter values consistent with the system model and the error bounds if a linear relationship between parameters and measurements is assumed this set is a polytope an exact representation of this polytope may be too complex for practical use and approximate descriptions in terms of simple shaped sets contained in the feasible parameter set inner bounds have shown to be useful in several applications balls in l sub infinity norms boxes l sub 2 norms ellipsoids and l sub 1 norms diamonds are used as bounding sets results are given on the computation of maximal balls when their shape is either known or partially free a numerical example is given in which the results of a computation of inner boxes of fixed shape and fixed orientation are given and compared

recursive identification of hammerstein systems with discontinuous nonlinearities containing dead zones this note deals with the recursive parameter identification of hammerstein systems with discontinuous nonlinearities i e two segment piecewise linear with dead zones and preloads a special form of the hammerstein model with this type of nonlinearity is incorporated into the recursive least squares identification scheme supplemented with the estimation of model internal variables the proposed method is illustrated by examples

nonlinear least absolute values and minimax model fitting abstract up to now the least absolute values l 1 and minimax l criteria have seldom been used in model fitting even if they are preferable for theoretical reasons their non differentiability causes numerical and analytical difficulties fortunately a recent reformulation of l 1 and l as equivalent constrained differentiable problems enables one to numerically solve them using standard nonlinear programming software this paper exploits this reformulation for analytical purposes in particular limits are derived concerning the ability of l 1 and l to resolve highly similar nonlinearly parametric terms from error corrupted observations examples are exponentials of nearly equal decay in compartmental models and almost coinciding peaks in spectroscopy

adaptive filtering with binary reinforcement recently there has been increased interest in high speed adaptive filtering where the usual stochastic gradient or least mean square lms algorithm is replaced with the simpler algorithm where adaptation is guided only by the polarity of the error signal in this paper the convergence of this binary reinforcement br algorithm is proved under the usual independence assumption and the surprising observation is made that unlike the lms algorithm convergence occurs for any positive value of the step size parameter while the stochastic gradient algorithm attempts to minimize a mean square error cost function the binary reinforcement algorithm in fact attempts to minimize a mean absolute error cost function it is proved for the binary reinforcement algorithm that the tap weight vector converges in distribution to a random vector that is suitably concentrated about the optimal value based on a least mean absolute error cost function for a sufficiently small step size the expected cost of the asymptotic weight vector can be made as close as desired to thc minimum cost attain ed by the optimum weight vector

background foreground modeling based on spatiotemporal sparse subspace clustering background estimation and foreground segmentation are important steps in many high level vision tasks many existing methods estimate background as a low rank component and foreground as a sparse matrix without incorporating the structural information therefore these algorithms exhibit degraded performance in the presence of dynamic backgrounds photometric variations jitter shadows and large occlusions we observe that these backgrounds often span multiple manifolds therefore constraints that ensure continuity on those manifolds will result in better background estimation hence we propose to incorporate the spatial and temporal sparse subspace clustering into the robust principal component analysis rpca framework to that end we compute a spatial and temporal graph for a given sequence using motion aware correlation coefficient the information captured by both graphs is utilized by estimating the proximity matrices using both the normalized euclidean and geodesic distances the low rank component must be able to efficiently partition the spatiotemporal graphs using these laplacian matrices embedded with the rpca objective function these laplacian matrices constrain the background model to be spatially and temporally consistent both on linear and nonlinear manifolds the solution of the proposed objective function is computed by using the linearized alternating direction method with adaptive penalty optimization scheme experiments are performed on challenging sequences from five publicly available datasets and are compared with the 23 existing state of the art methods the results demonstrate excellent performance of the proposed algorithm for both the background estimation and foreground segmentation

the complexity of multi mean payoff and multi energy games in mean payoff games the objective of the protagonist is to ensure that the limit average of an infinite sequence of numeric weights is nonnegative in energy games the objective is to ensure that the running sum of weights is always nonnegative multi mean payoff and multi energy games replace individual weights by tuples and the limit average resp running sum of each coordinate must be resp remain nonnegative we prove finite memory determinacy of multi energy games and show inter reducibility of multi mean payoff and multi energy games for finite memory strategies we improve the computational complexity for solving both classes with finite memory strategies we prove conp completeness improving the previous known expspace bound for memoryless strategies we show that deciding the existence of a winning strategy for the protagonist is np complete we present the first solution of multi mean payoff games with infinite memory strategies we show that mean payoff sup objectives can be decided in np conp whereas mean payoff inf objectives are conp complete

guaranteed performance regions in markovian systems with competing decision makers a decision maker is facing a dynamic system which is being controlled by himself herself as well as by other decision makers he she considers a vector of performance measures acceptable performance is defined through a set in the space of performance vectors whether this decision maker can guarantee a time averaged performance vector which approaches this desired set is considered for the worst case scenario in which other decision makers may for selfish reasons try to exclude his her vector from the desired set for a controlled markov model of the system a sufficient condition for approachability is given and appropriate control strategies are constructed under certain recurrence conditions a complete characterization of approachability is then provided for convex sets the mathematical formulation leads to a theory of approachability for stochastic games with vector payoffs a simple queuing example illustrates this approach

tracking and identification of regime switching systems using binary sensors this work is concerned with tracking and system identification for time varying parameters the parameters are markov chains and the observations are binary valued with noise corruption to overcome the difficulties due to the limited measurement information wonham type filters are developed first then based on the filters two popular estimators namely mean squares estimator msq and maximum posterior map estimator are constructed for the mean squares estimator we derive asymptotic normality in the sense of weak convergence and in the sense of strong approximation the asymptotic normality is then used to derive error bounds when the markov chain is infrequently switching we derive error bounds for map estimators when the markovian parameters are fast varying we show that the averaged behavior of the parameter process can be derived from the stationary measure of the markov chain and that can be estimated using empirical measures upper and lower error bounds on estimation errors are also established

edge preserving smoothers for image processing abstract classical smoothers have limited usefulness in image processing because sharp edges tend to be blurred there is a literature on edge preserving smoothers but these all require moderately large smooth stretches here we discuss an approach to this problem called sigma filtering and propose an improvement based on running m estimation both computational and theoretical aspects are developed for image processing the methods have a niche between standard filtering approaches and bayes markov random field analysis

estimation theory for nonlinear models and set membership uncertainty this chapter studies the problem of estimating a given function of a vector of unknowns called the problem element by using measurements depending non linearly on the problem element and affected by unknown but bounded noise assuming that both the solution sought and the measurements depend polynomially on the unknown problem element a method is given to compute the axis aligned box of minimal volume containing the feasible solution set i e the set of all unknowns consistent with the actual measurements and the given bound on the noise the center of this box is a point estimate of the solution which enjoys useful optimality properties the sides of the box represent the intervals of possible variation of the estimates important problems like parameter estimation of exponential models time series prediction with arma models and parameter estimates of discrete time state space models can be formalized and solved by using the developed theory

optimality of central and projection algorithms for bounded uncertainty this paper investigates optimality properties of central and projection algorithms for linear problems in the field of system identification in a context in which uncertainty is described in a deterministic rather than statistical way particular attention is devoted to least squares algorithms when the measurement noise is assumed to be unknown but bounded in a hilbert norm a major contribution of this paper consists in proving that least squares algorithms enjoy strong optimality properties on the contrary it is pointed out that these properties do not hold for other frequently used projection algorithms such as least absolute values or minimax algorithms corresponding to a description of the measurement error in h or l norm respectively

focusing on unbounded input output models on the basic one operator input output plant model y gu has fundamental limitations in feedback and closed loop modelling problems on the full time axis when g is an unbounded linear operator it has been a deeply rooted misunderstanding that there is no essential difference between singly infinite and doubly infinite time axis formulations of feedback problems using this basic plant model we present here several new types of results on the limitations of the one operator plant model y gu in discrete time feedback and modelling problems on the full time axis a detailed analysis of existence of solutions in two concrete feedback problems is also provided we discuss also the errors in the classical transform domain analysis of the unbounded plant mapping case on the full time axis an important conclusion of the present and related earlier work is that the two operator plant model ay bu should be seen as the fundamental input output plant model this model allows the modeling of open loop unstable

tree filtering efficient structure preserving smoothing with a minimum spanning tree we present a new efficient edge preserving filter tree filter to achieve strong image smoothing the proposed filter can smooth out high contrast details while preserving major edges which is not achievable for bilateral filter like techniques tree filter is a weighted average filter whose kernel is derived by viewing pixel affinity in a probabilistic framework simultaneously considering pixel spatial distance color intensity difference as well as connectedness pixel connectedness is acquired by treating pixels as nodes in a minimum spanning tree mst extracted from the image the fact that an mst makes all image pixels connected through the tree endues the filter with the power to smooth out high contrast fine scale details while preserving major image structures since pixels in small isolated region will be closely connected to surrounding majority pixels through the tree while pixels inside large homogeneous region will be automatically dragged away from pixels outside the region the tree filter can be separated into two other filters both of which turn out to have fast algorithms we also propose an efficient linear time mst extraction algorithm to further improve the whole filtering speed the algorithms give tree filter a great advantage in low computational complexity linear to number of image pixels and fast speed it can process a 1 megapixel 8 bit image at 0 25 s on an intel 3 4 ghz core i7 cpu including the construction of mst the proposed tree filter is demonstrated on a variety of applications

convergence of adaptive control schemes using least squares parameter estimates the stability convergence asymptotic optimality and self tuning properties of stochastic adaptive control schemes based on least squares estimates of the unknown parameters are examined it is assumed that the additive noise is i i d and gaussian and that the true system is of minimum phase the bayesian embedding technique is used to show that the recursive least squares parameter estimates converge in general the normal equations of least squares are used to establish that all stable control law designs used in a certainty equivalent i e indirect procedure generally yield a stable adaptive control system four results are given to characterize the limiting behavior precisely a certainty equivalent self tuning regulator is shown to yield strongly consistent parameter estimates when the delay is strictly greater than one even without any excitation in the reference trajectory

recursive computation of inner bounds for the parameters of linear models minimum volume ellipsoidal outer bounds on the parameters of linear regression type models with bounded model output error can be computed by an established algorithm a complementary algorithm to compute maximum volume ellipsoidal inner bounds is derived it is little more complicated than the existing outer bounding algorithm in conjunction with that algorithm it allows parameter values to be classified as acceptable dubious or unacceptable it is also found to be effective in determining output error bounds by trial and error during development of parameter bounding models

joint identification of plant rational models and noise distribution functions using binary valued observations system identification of plants with binary valued output observations is of importance in understanding modeling capability and limitations for systems with limited sensor information establishing relationships between communication resource limitations and identification complexity and studying sensor networks this paper resolves two issues arising in such system identification problems first regression structures for identifying a rational model contain non smooth nonlinearities leading to a difficult nonlinear filtering problem by introducing a two step identification procedure that employs periodic signals empirical measures and identifiability features rational models can be identified without resorting to complicated nonlinear searching algorithms second by formulating a joint identification problem we are able to accommodate scenarios in which noise distribution functions are unknown convergence of parameter estimates is established recursive algorithms for joint identification and their key properties are further developed

estimation theory and uncertainty intervals evaluation in presence of unknown but bounded errors linear families of models and estimators the problem of parameter estimation and of the evaluation of related uncertainty intervals is considered in the case that a probabilistic description of noise and errors is not available of suitable but only a bound on them is known in the present paper only linear parametrizations and estimators are considered very simple and computationally feasible algorithms are derived for evaluating two different types of uncertainty intervals estimates uncertainty intervals parameter uncertainty intervals the relationships between the eui s and the pui s are established and the solution to the problem of the minimum uncertainty intervals estimator is given the latter can be obtained by means of a simple linear programming algorithm

density based multifeature background subtraction with support vector machine background modeling and subtraction is a natural technique for object detection in videos captured by a static camera and also a critical preprocessing step in various high level computer vision applications however there have not been many studies concerning useful features and binary segmentation algorithms for this problem we propose a pixelwise background modeling and subtraction technique using multiple features where generative and discriminative techniques are combined for classification in our algorithm color gradient and haar like features are integrated to handle spatio temporal variations for each pixel a pixelwise generative background model is obtained for each feature efficiently and effectively by kernel density approximation kda background subtraction is performed in a discriminative manner using a support vector machine svm over background likelihood vectors for a set of features the proposed algorithm is robust to shadow illumination changes spatial variations of background we compare the performance of the algorithm with other density based methods using several different feature combinations and modeling techniques both quantitatively and qualitatively

structure identification of parameter bounding models by use of noise structure bounds the selection of a model for a given set of records is discussed for the case when the output error is required to be always within some specified bounds a new selection criterion is proposed the model should be the simplest giving sufficiently unstructured output errors the lack of structure is specified by requiring the candidate model to be capable of giving sufficiently small sample autocorrelations of the model output errors over a range of lags computational algorithms to apply the criterion are presented and illustrated by tests on ar processes

recursive identification of systems with binary valued outputs and with arma noises abstract this paper considers the identification problem of the arma system followed by a binary sensor in which the internal variables are corrupted by additive arma noises recursive estimates for the parameters of the linear system and for the threshold of the binary sensor are given by the stochastic approximation algorithms with expanding truncations saawet under reasonable conditions all the estimates are proved to be convergent to the true values with probability one the almost surely convergence rates are also investigated a simulation example is included to justify the theoretical results

asymptotically convergent modified recursive least squares with data dependent updating and forgetting factor for systems with bounded noise continual updating of estimates required by most recursive estimation schemes often involves redundant usage of information and may result in system instabilities in the presence of bounded output disturbances an algorithm which eliminates these difficulties is investigated based on a set theoretic assumption the algorithm yields modified least squares estimates with a forgetting factor it updates the estimates selectively depending on whether the observed data contain sufficient information the information evaluation required at each step involves very simple computations in addition the parameter estimates are shown to converge asymptotically at an exponential rate to a region around the true parameter

identification input design for consistent parameter estimation of linear systems with binary valued output observations input design is of essential importance in system identification for providing sufficient probing capabilities to guarantee convergence of parameter estimates to their true values this paper presents conditions on input signals that characterize their probing richness for strongly consistent parameter estimation of linear systems with binary valued output observations necessary and sufficient conditions on periodic signals are derived for sufficient richness these conditions are further studied under different system configurations including open loop and feedback systems and different scenarios of noises including actuator noise input measurement noise and output measurement noise in addition to system parameter estimation essential properties of identifiability and input conditions are also derived when sensor thresholds or noise distribution functions are unknown the findings of this paper provide a foundation to study identification of systems that either use binary valued or quantized sensors or involve communication channels which mandate quantization of signals

identification of wiener systems with binary valued output observations this work is concerned with identification of wiener systems whose outputs are measured by binary valued sensors the system consists of a linear fir finite impulse response subsystem of known order followed by a nonlinear function with a known parametrization structure the parameters of both linear and nonlinear parts are unknown input design identification algorithms and their essential properties are presented under the assumptions that the distribution function of the noise is known and the nonlinearity is continuous and invertible it is shown that under scaled periodic inputs identification of wiener systems can be decomposed into a finite number of core identification problems the concept of joint identifiability of the core problem is introduced to capture the essential conditions under which the wiener system can be identified with binary valued observations under scaled full rank conditions and joint identifiability a strongly convergent algorithm is constructed the algorithm is shown to be asymptotically efficient for the core identification problem hence achieving asymptotic optimality in its convergence rate for computational simplicity recursive algorithms are also developed

closed loop persistent identification of linear systems with unmodeled dynamics and stochastic disturbances the essential issues of time complexity and probing signal selection are studied for persistent identification of linear time invariant systems in a closed loop setting by establishing both upper and lower bounds on identification accuracy as functions of the length of observation size of unmodeled dynamics and stochastic disturbances we demonstrate the inherent impact of unmodeled dynamics on identification accuracy reduction of time complexity by stochastic averaging on disturbances and probing capability of full rank periodic signals for closed loop persistent identification these findings indicate that the mixed formulation in which deterministic uncertainty of system dynamics is blended with random disturbances is beneficial to reduction of identification complexity