fuzzy rule selection by multi objective genetic local search algorithms and rule evaluation measures in data mining this paper shows how a small number of simple fuzzy if then rules can be selected for pattern classification problems with many continuous attributes our approach consists of two phases candidate rule generation by rule evaluation measures in data mining and rule selection by multi objective evolutionary algorithms in our approach first candidate fuzzy if then rules are generated from numerical data and prescreened using two rule evaluation measures i e confidence and support in data mining then a small number of fuzzy if then rules are selected from the prescreened candidate rules using multi objective evolutionary algorithms in rule selection we use three objectives maximization of the classification accuracy minimization of the number of selected rules and minimization of the total rule length thus the task of multi objective evolutionary algorithms is to find a number of non dominated rule sets with respect to these three objectives the main contribution of this paper is to propose an idea of utilizing the two rule evaluation measures as prescreening criteria of candidate rules for fuzzy rule selection an arbitrarily specified number of candidate rules can be generated from numerical data for high dimensional pattern classification problems through computer simulations we demonstrate that such a prescreening procedure improves the efficiency of our approach to fuzzy rule selection we also extend a multi objective genetic algorithm moga in our former studies to a multi objective genetic local search mogls algorithm where a local search procedure adjusts the selection i e inclusion or exclusion of each candidate rule furthermore a learning algorithm of rule weights i e certainty factors is combined with our mogls algorithm such extensions to our moga for fuzzy rule selection are another contribution of this paper

multiobjective evolutionary optimization of type 2 fuzzy rule based systems for financial data classification classification techniques are becoming essential in the financial world for reducing risks and possible disasters managers are interested in not only high accuracy but in interpretability and transparency as well it is widely accepted now that the comprehension of how inputs and outputs are related to each other is crucial for taking operative and strategic decisions furthermore inputs are often affected by contextual factors and characterized by a high level of uncertainty in addition financial data are usually highly skewed toward the majority class with the aim of achieving high accuracies preserving the interpretability and managing uncertain and unbalanced data this paper presents a novel method to deal with financial data classification by adopting type 2 fuzzy rule based classifiers frbcs generated from data by a multiobjective evolutionary algorithm moea the classifiers employ an approach denoted as scaled dominance for defining rule weights in such a way to help minority classes to be correctly classified in particular we have extended paes rcs an moea based approach to learn concurrently the rule and data bases of frbcs for managing both interval type 2 fuzzy sets and unbalanced datasets to the best of our knowledge this is the first work that generates type 2 frbcs by concurrently maximizing accuracy and minimizing the number of rules and the rule length with the objective of producing interpretable models of real world skewed and incomplete financial datasets the rule bases are generated by exploiting a rule and condition selection rcs approach which selects a reduced number of rules from a heuristically generated rule base and a reduced number of conditions for each selected rule during the evolutionary process the weight associated with each rule is scaled by the scaled dominance approach on the fuzzy frequency of the output class in order to give a higher weight to the minority class as regards the data base learning the membership function parameters of the interval type 2 fuzzy sets used in the rules are learned concurrently to the application of rcs unbalanced datasets are managed by using in addition to complexity selectivity and specificity as objectives of the moea rather than only the classification rate we tested our approach named it2 paes rcs on 11 financial datasets and compared our results with the ones obtained by the original paes rcs with three objectives and with and without scaled dominance the frbcs fuzzy association rule based classification model for high dimensional dataset farc hd and fuzzy unordered rules induction algorithm furia the classical c4 5 decision tree algorithm and its cost sensitive version using nonparametric statistical tests we will show that it2 paes rcs generates frbcs with on average accuracy statistically comparable with and complexity lower than the ones generated by the two versions of the original paes rcs further the frbcs generated by farc hd and furia and the decision trees computed by c4 5 and its cost sensitive version despite the highest complexity result to be less accurate than the frbcs generated by it2 paes rcs finally we will highlight how these frbcs are easily interpretable by showing and discussing one of them

the optimum scalar data nonlinearity in lms adaptation for arbitrary iid inputs the authors show that the optimum nonlinear scale operation upon the elements of the observation vector in the lms algorithm is exactly x 1 mu x sup 2 for any independent stochastic data input and any noise density moreover use of such a nonlinearity can yield a significant performance improvement in fast adaptation situations

performance advantage of complex lms for controlling narrow band adaptive arrays in narrow band adaptive array applications the mean square convergence of the discrete time real least mean square lms algorithm is slowed by image frequency noises generated in the lms loops the complex lms algorithm proposed by widrow et al is shown to eliminate these noises yielding convergence of the mean squared error mse at slightly over twice the rate this paper includes a comprehensive analysis of the mse of adaptation for lms the analysis is based upon the method developed in the 1968 dissertation by k d senne and it represents the most complete treatment of the subject published to date

fuzzy theoretic approach to signals and systems static systems abstract fuzzy theoretic approach to signals and systems assumes all system variables and parameters as uncertain i e being characterized by membership functions develops a mathematical theory for analytically determining the membership functions on system variables and parameters derives algorithms for estimating the parameters of membership functions and establishes robustness and convergence properties of the estimation algorithms the membership functions are analytically determined via solving a variational optimization problem that maximizes the over uncertainties averaged log membership of the observed data around an initial guess this paper develops the analytical fuzzy theory for the particular case of a multi input single output static system affected by noises the theory facilitates designing an adaptive filtering algorithm the robustness of the adaptive filtering algorithm is proved theoretically via a mathematical analysis numerical experiments further demonstrate the robustness of the filtering algorithm a comparison of the algorithm with the state of art methods is made using the practical biomedical applications related to the modeling and analysis of heart rate signals for assessing the physiological state of an individual

genetic training instance selection in multiobjective evolutionary fuzzy systems a coevolutionary approach when dealing with datasets that are characterized by a large number of instances multiobjective evolutionary learning moel of fuzzy rule based systems frbss suffers from high computational costs mainly because of the fitness evaluation the use of a reduced set of representative instances in place of the overall training set ts would considerably lessen the computational effort even though a large number of papers have proposed instance selection approaches mainly in classification problems how this selection should be performed especially in the context of regression is still an open issue in this paper we tackle the instance selection problem in the framework of moel of frbss through a coevolutionary approach in the execution of the moel periodically a single objective genetic algorithm soga evolves a population of reduced tss the soga aims to maximize a purposely defined index which measures how much the pareto fronts computed by using respectively the reduced ts and the overall ts are close to each other the closer the fronts the more the reduced ts is representative of the overall ts during the execution of the moel the rule base and the membership function parameters of the fuzzy sets are concurrently learned by maximizing the accuracy and minimizing the complexity we tested our approach on 12 large datasets we adopted reduced tss composed of 5 10 and 20 of the overall ts using nonparametric statistical tests we verified that with 10 and 20 of the overall ts the pareto front approximations that are generated by our coevolutionary approach are comparable with the ones generated by applying the moel with the overall ts although the coevolution allows us to save up to 86 36 of the execution time in addition the analysis of the behavior of three representative solutions on the test set highlights that the use of the reduced tss does not affect the generalization capabilities of the generated frbss

interpretability of linguistic fuzzy rule based systems an overview of interpretability measures linguistic fuzzy modelling developed by linguistic fuzzy rule based systems allows us to deal with the modelling of systems by building a linguistic model which could become interpretable by human beings linguistic fuzzy modelling comes with two contradictory requirements interpretability and accuracy in recent years the interest of researchers in obtaining more interpretable linguistic fuzzy models has grown whereas the measures of accuracy are straightforward and well known interpretability measures are difficult to define since interpretability depends on several factors mainly the model structure the number of rules the number of features the number of linguistic terms the shape of the fuzzy sets etc moreover due to the subjectivity of the concept the choice of appropriate interpretability measures is still an open problem in this paper we present an overview of the proposed interpretability measures and techniques for obtaining more interpretable linguistic fuzzy rule based systems to this end we will propose a taxonomy based on a double axis complexity versus semantic interpretability considering the two main kinds of measures and rule base versus fuzzy partitions considering the different components of the knowledge base to which both kinds of measures can be applied the main aim is to provide a well established framework in order to facilitate a better understanding of the topic and well founded future works

adaptive filter performance with nonlinearities in the correlation multiplier it is well known that the correlation multiplier used to form the update signal in adaptive transversal filters need not be a true multiplier by using an approximate multiplier such as a sign bit only multiplier implementation can be made less costly in this paper we derive expressions for the convergence rate and misalignment noise of adaptive transversal filters with arbitrary correlation multiplier non linearities the analysis shows any such nonlinearities to always harm performance and quantifies the penalty suffered sign bit only multiplication suffers a significant penalty and power of two multiplication is suggested as a better alternative its performance is almost equal to that of a true multiplier and in a digital implementation at least it is only slightly more difficult to realize than a sign bit only multiplier the theory is developed in a way emphasizing physical understanding and has been verified to be in good agreement with simulation

junction times in singular optimal control finding junction times on partially singular optimal control trajectories is in general difficult analytically and especially so when the optimal control is explicitly dependent on both the state and adjoint variables this paper presents a new approach for linear problems whereby variations in the final state measured from a reference optimal solution can be used to find corresponding junction times the approach is shown to be effective when applied to a problem which has an optimal solution involving a single bang singular junction for certain choices of end state

a dynamically constrained multiobjective genetic fuzzy system for regression problems in this paper a multiobjective genetic fuzzy system gfs to learn the granularities of fuzzy partitions tuning the membership functions mfs and learning the fuzzy rules is presented it uses dynamic constraints which enable three parameter mf tuning to improve the accuracy while guaranteeing the transparency of fuzzy partitions the fuzzy models fms are initialized by a method that combines the benefits of wang mendel wm and decision tree algorithms thus the initial fms have less rules rule conditions and input variables than if wm initialization were to be used moreover the fuzzy partitions of initial fms are always transparent our approach is tested against recent multiobjective and monoobjective gfss on six benchmark problems it is concluded that the accuracy and interpretability of our fms are always comparable or better than those in the comparative studies furthermore on some benchmark problems our approach clearly outperforms some comparative approaches suitability of our approach for higher dimensional problems is shown by studying three benchmark problems that have up to 21 input variables

locally optimum adaptive signal processing algorithms we propose a new analytic method for comparing constant gain adaptive signal processing algorithms specifically estimates of the convergence speed of the algorithms allow for the definition of a local measure of performance called the efficacy that can be theoretically evaluated by definition the efficacy is consistent with the fair comparison techniques used in signal processing applications using the efficacy as a performance measure we prove that the lms newton algorithm is optimum and is thus the fastest algorithm within a very rich algorithmic class furthermore we prove that regular lms is better than any of its variants that apply the same nonlinear transformation on the elements of the regression vector such as signed regressor quantized regressor etc for an important class of input signals simulations support all our theoretical conclusions

the complex lms adaptive algorithm transient weight mean and covariance with applications to the ale the transient and steady state mean and covariance of the complex valued lms adaptive element weights are investigated when the inputs are samples from circularly normal processes it is shown that the data covariance diagonalizing matrix also diagonalizes the weight covariance matrix this result permits describing the complete transient behavior of the adaptive line enhancer ale weight covariance matrix in closed form for the case of multiple equal power narrow band statistically independent orthogonal rayleigh fading sinusoids in broad band noise based on these transient covariance results it is shown that for any stage of adaptation there exists an optimum ale gain constant that minimizes the excess mean squared prediction error this result is of particular significance when processing time limited data

on error saturation nonlinearities in lms adaptation the effect of a saturation type error nonlinearity in the weight update equation in least mean squares lms adaptation is investigated for a white gaussian data model nonlinear difference equations are derived for the eight first and second moments which include the effect of an error function erf saturation type nonlinearity on the error sequence driving the algorithm a nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures the steady state second moment weight behavior is evaluated exactly for the erf nonlinearity using the above results the tradeoff between the extent of error saturation steady state excess mean square error and rate of algorithm convergence is studied the tradeoff shows that 1 starting with a sign detector the convergence rate is increased by nearly a factor of two for each additional bit and 2 as the number of bits is increased further the additional bit by very little in convergence speed asymptotically approaching the behavior of the linear algorithm

nonlinear quantization effects in the frequency domain complex scalar lms adaptive algorithm nonlinear quantization effects in the frequency domain complex scalar lms adaptive algorithm are analyzed by using conditional expectations the probability density function of the quantizer input conditioned on the weight is derived the density is applied to finding the conditional characteristic function and the mth conditional moment at the quantizer output the first and second conditional moments of the quantizer output are used to derive difference equations that approximate the dynamical behavior of the first and second weight moments these difference equations are solved numerically and compare favorably to simulation results a model of the quantizer as an additive noise source is of no analytical value since the quantizatian noise has negligible effect on the mean square error when the model is valid finally a design approach is proposed for selecting the number of bits in the weight accumulator the moment equations are also used to determine the algorithm mean square error for different quantizer step sizes and the optimum algorithm step size when a fixed amount of input data is available for adaptation

nonlinear quantization effects in the lms and block lms adaptive algorithms a comparison digital implementations of the least mean square lms and block lms blms algorithms are compared with respect to finite word effects the algorithm stalling phenomenon is studied using gaussian data models and conditional expectation arguments it is shown that the blms algorithm requires 1 2 log sub 2 l k fewer bits for the same stalling behavior l block length and k lies between 0 2 and 1 0 depending on the precise definition of algorithm stalling on the other hand the lms algorithm requires log sub 2 l fewer bits than blms for the same level of saturation behavior transient response at algorithm initialization hence the lms algorithm requires 1 2 log sub 2 l k fewer bits than the blms algorithm for the same saturation and stalling effects

modeling with discrete time recurrent fuzzy systems via mixed integer optimization in this paper we present a new approach for modeling of dynamical systems with discrete time recurrent fuzzy systems dtrfs the modeling problem is divided into two subproblems the first subproblem considers the identification of the rule base of the dtrfs which can be interpreted as structural modeling this problem is recast into integer and mixed integer optimization problems respectively dependent on the chosen cost function and system structure the second subproblem considers the optimization of the parameters of the system i e quantitative modeling as a result we obtain a linguistically interpretable model of the dynamical system which additionally can be interpreted as a linguistic automaton the applicability of the approach is shown by modeling a thermofluidic process

on joining non singular and singular optimal control subarcs necessary conditions for junctions between non singular and singular optimal control subarcs are proved for the case where the singular problem order is greater than one the generalized legendre clebsch condition glc vanishes at the junction and the derivative of the glc expression is non zero at the junction the proof requires the derivation of two identities which are system properties of higher order singular problems

new insights on the transient and steady state behavior of the quantized lms algorithm this correspondence investigates the transient and steady state behavior of the quantized lms algorithm for gaussian inputs it is shown here that the so called stopping phenomenon is really a slow down phenomenon which because of an extremely slow convergence rate looks as if the algorithm has stopped the true steady state mse is shown to be nearly independent of the number of bits in the digital word length and very nearly the steady state mse of the infinite precision lms realization these results assume that the algorithm misadjustment effects due to coefficient quantization are negligible in comparison with those due to the stopping phenomena since the true steady state is rarely achievable with a finite number of iterations determination of the step size spl mu that minimizes the residual mse must be based on a stochastic model for the transient mode of algorithm operation it is shown that the finite word length and infinite precision design cases differ only in degree and not in kind as far as the selection of spl mu is concerned

tracking analysis of the sign algorithm in nonstationary environments a tracking analysis of the adaptive filters equipped with the sign algorithm and operating in nonstationary environments is presented under the assumption that the nonstationary can be modeled using a random disturbance it is shown that the long term time average of the mean absolute error is bounded and that there exists an optimal choice of the convergence constant mu which minimizes this quality applying the commonly used independence assumption and under the assumption that the nonstationarity is solely due to the time varying behavior of the optimal coefficients it is shown that the distributions of the successive coefficient misalignment vectors converge to a limiting distribution when the adaptive filter is used in the system identification mode finally under the additional assumption that the signals involved are zero mean and gaussian a set of nonlinear difference equations is derived that characterizes the mean and mean squared behavior of the filter coefficients and the mean squared estimation error during adaptation and tracking results of several experiments that show very good correlation with the theoretical analyses are presented

ten years of genetic fuzzy systems current framework and new trends fuzzy systems have demonstrated their ability to solve different kinds of problems in various application domains currently there is an increasing interest to augment fuzzy systems with learning and adaptation capabilities two of the most successful approaches to hybridise fuzzy systems with learning and adaptation methods have been made in the realm of soft computing neural fuzzy systems and genetic fuzzy systems hybridise the approximate reasoning method of fuzzy systems with the learning capabilities of neural networks and evolutionary algorithms the objective of this paper is to provide an account of genetic fuzzy systems with special attention to genetic fuzzy rule based systems after a brief introduction to models and applications of genetic fuzzy systems the field is overviewed new trends are identified a critical evaluation of genetic fuzzy systems for fuzzy knowledge extraction is elaborated and open questions that remain to be addressed in the future are raised the paper also includes some of the key references required to quickly access implementation details of genetic fuzzy systems

quantized kernel least mean square algorithm in this paper we propose a quantization approach as an alternative of sparsification to curb the growth of the radial basis function structure in kernel adaptive filtering the basic idea behind this method is to quantize and hence compress the input or feature space different from sparsification the new approach uses the redundant data to update the coefficient of the closest center in particular a quantized kernel least mean square qklms algorithm is developed which is based on a simple online vector quantization method the analytical study of the mean square convergence has been carried out the energy conservation relation for qklms is established and on this basis we arrive at a sufficient condition for mean square convergence and a lower and upper bound on the theoretical value of the steady state excess mean square error static function estimation and short term chaotic time series prediction examples are presented to demonstrate the excellent performance

transient and tracking performance analysis of the quantized lms algorithm for time varying system identification this paper investigates the statistical behavior of the finite precision lms adaptive filter in the identification of an unknown time varying stochastic system nonlinear recursions are derived for the mean and mean square behavior of the adaptive weights transient and tracking algorithm performance curves are generated from the recursions and shown to be in excellent agreement with monte carlo simulations our results demonstrate that linear models are inappropriate for analyzing the transient and the steady state algorithm behavior the performance curves indicate that the transient and tracking capabilities cannot be determined from perturbations about the infinite precision case it is shown that the transient phase of the algorithm increases as the digital wordlength or the speed of variation of the unknown system decrease design examples illustrate how the theory can be used to select the algorithm step size and the number of bits in the quantizer

complexity measures of supervised classification problems we studied a number of measures that characterize the difficulty of a classification problem focusing on the geometrical complexity of the class boundary we compared a set of real world problems to random labelings of points and found that real problems contain structures in this measurement space that are significantly different from the random sets distributions of problems in this space show that there exist at least two independent factors affecting a problem s difficulty we suggest using this space to describe a classifier s domain of competence this can guide static and dynamic selection of classifiers for specific problems as well as subproblems formed by confinement projection and transformations of the feature vectors

a proposal for the genetic lateral tuning of linguistic fuzzy systems and its interaction with rule selection linguistic fuzzy modeling allows us to deal with the modeling of systems by building a linguistic model which is clearly interpretable by human beings however since the accuracy and the interpretability of the obtained model are contradictory properties the necessity of improving the accuracy of the linguistic model arises when complex systems are modeled to solve this problem one of the research lines in recent years has led to the objective of giving more accuracy to linguistic fuzzy modeling without losing the interpretability to a high level in this paper a new postprocessing approach is proposed to perform an evolutionary lateral tuning of membership functions with the main aim of obtaining linguistic models with higher levels of accuracy while maintaining good interpretability to do so we consider a new rule representation scheme base on the linguistic 2 tuples representation model which allows the lateral variation of the involved labels furthermore the cooperation of the lateral tuning together with fuzzy rule reduction mechanisms is studied in this paper presenting results on different real applications the obtained results show the good performance of the proposed approach in high dimensional problems and its ability to cooperate with methods to remove unnecessary rules

adaptability interpretability and rule weights in fuzzy rule based systems this paper discusses interpretability in two main categories of fuzzy systems fuzzy rule based classifiers and interpolative fuzzy systems our goal is to show that the aspect of high level interpretability is more relevant to fuzzy classifiers whereas fuzzy systems employed in modeling and control benefit more from low level interpretability we also discuss the interpretability accuracy tradeoff and observe why various rule weighting schemes that have been brought into play to increase adaptability of fuzzy systems rather just increase computational overhead and seriously compromise interpretability of fuzzy systems

multi objective hierarchical genetic algorithm for interpretable fuzzy rule based knowledge extraction a new scheme based on multi objective hierarchical genetic algorithm mohga is proposed to extract interpretable rule based knowledge from data the approach is derived from the use of multiple objective genetic algorithm moga where the genes of the chromosome are arranged into control genes and parameter genes these genes are in a hierarchical form so that the control genes can manipulate the parameter genes in a more effective manner the effectiveness of this chromosome formulation enables the fuzzy sets and rules to be optimally reduced some important concepts about the interpretability are introduced and the fitness function in the moga will consider both the accuracy and interpretability of the fuzzy model in order to remove the redundancy of the rule base proactively we further apply an interpretability driven simplification method to newborn individuals in our approach we first apply the fuzzy clustering to generate an initial rule based model then the multi objective hierarchical genetic algorithm and the recursive least square method are used to obtain the optimized fuzzy models the accuracy and the interpretability of fuzzy models derived by this approach are studied and presented in this paper we compare our work with other methods reported in the literature on four examples a synthetic nonlinear dynamic system a nonlinear static system the lorenz system and the mackey glass system simulation results show that the proposed approach is effective and practical in knowledge extraction

a fast and efficient multi objective evolutionary learning scheme for fuzzy rule based classifiers during the last years multi objective evolutionary algorithms moeas have been extensively used to generate fuzzy rule based systems characterized by different trade offs between accuracy and complexity in this paper we propose an moea based approach to learn concurrently the rule and data bases of fuzzy rule based classifiers frbcs in particular the rule bases are generated by exploiting a rule and condition selection rcs strategy which selects a reduced number of rules from a heuristically generated set of candidate rules and a reduced number of conditions for each selected rule during the evolutionary process rcs can be considered as a rule learning in a constrained search space as regards the data base learning the membership function parameters of each linguistic term used in the rules are learned concurrently to the application of rcs we tested our approach on twenty four classification benchmarks and compared our results with the ones obtained by two similar state of the art moea based approaches and by two well known non evolutionary classification algorithms namely furia and c4 5 using non parametric statistical tests we show that our approach generates frbcs with accuracy and complexity statistically comparable to and sometimes better than the ones generated by the two moea based approaches exploiting however only the 5 of the number of fitness evaluations used by these approaches further the classifiers generated by our approach result to be more interpretable than the ones generated by the furia and c4 5 algorithms while achieving the same accuracy level

a counterexample to a conjecture of mcdanell abstract a partially singular optimal control trajectory will contain one or more junctions at which nonsingular and singular subarcs are joined mcdanell s conjecture concerns the necessary conditions for optimality at such a junction the conjecture is divided into two parts each part distinguished by the relative magnitudes of two parameters until now the conjecture has been proved correct for several different classes of problem however this paper presents a partially singular optimal control trajectory containing one junction for which one part of the conjecture is untrue attempts to construct a similar trajectory in order to disprove the second part of the conjecture have so far been unsuccessful

multiobjective programming for type 2 hierarchical fuzzy inference trees this paper proposes a design of hierarchical fuzzy inference tree hfit an hfit produces an optimum tree like structure i e a natural hierarchical structure that accommodates simplicity by combining several low dimensional fuzzy inference systems fiss such a natural hierarchical structure provides a high degree of approximation accuracy the construction of the hfit takes place in two phases first a nondominated sorting based multiobjective genetic programming mogp is applied to obtain a simple tree structure a low complexity model with a high accuracy second the differential evolution algorithm is applied to optimize the obtained tree s parameters in the derived tree each node acquires a different input s combination where the evolutionary process governs the input s combination hence hfit nodes are heterogeneous in nature which leads to a high diversity among the rules generated by the hfit additionally the hfit provides an automatic feature selection because it uses mogp for the tree s structural optimization that accepts inputs only relevant to the knowledge contained in data the hfit was studied in the context of both type 1 and type 2 fiss and its performance was evaluated through six application problems moreover the proposed multiobjective hfit was compared both theoretically and empirically with recently proposed fiss methods from the literature such as mcit2fis tscit2fnn sit2fnn rit2fns wb et2fis mrit2nfs it2fnn svr etc from the obtained results it was found that the hfit provided less complex and highly accurate models compared to the models produced by the most of other methods hence the proposed hfit is an efficient and competitive alternative to the other fiss for function approximation and feature selection

comparison and design of interpretable linguistic vs scatter frbss gm3m generalization and new rule meaning index for global assessment and local pseudo linguistic representation this work is devoted to defining more general interpretability indexes to be applied to any scatter or linguistic model implemented by any type of membership functions they are based on metrics that should take into account the semantic and inference issues the semantic issue in order to preserve the meaning of the linguistic labels and the inference issue since this can influence the behavior of the rules on the other hand these metrics have been designed to be intuitive in order to support the analysis or selection of a final model and to favor a low computational cost within an optimization process in order to check their usefulness a multi objective evolutionary algorithm simultaneously performing a rule selection and an adjustment of the fuzzy partitions is guided by the proposed indexes on several benchmark data sets to obtain models with different degrees of accuracy and interpretability in addition using these metrics a local analysis can be carried out between models of a different nature this local analysis through the model components gives support to the user to make the best choice from among the models

adaptive equalizer using finite bit power of two quantizer the stochastic gradient algorithm using a simplified arithmetic is analyzed in this paper a power of two quantizer is used for the input of the multiplier to reduce the multiplication to at most a simple shift in spite of its simple implementation the performance is shown to be comparable to the classical lms algorithm a linearized approximation to the quantizer is first derived followed by the analysis of an exact nonlinear model the derivation is based on the gaussian assumption and the effects of removing the gaussian assumption are later considered the roundoff error due to the finite bit computation is calculated computer simulation results are provided to support the analysis

genetic generation of fuzzy systems with rule extraction using formal concept analysis fuzzy classification systems have been widely researched with many approaches proposed in the literature several methods are available for the automatic definition of fuzzy classification systems which basically comprehend two tasks i the definition of the attributes in terms of fuzzy sets and ii the generation of a rule set containing the domain knowledge named fuzzy rule base genetic fuzzy systems are used to learn or tune in fuzzy classification systems some genetic approaches for learning the fuzzy rule base require the previous extraction of a set of rules to be used as the genetic search space in this paper we present the fca based method a proposal for the automatic generation of fuzzy rule bases which extracts a set of rules using the formal concept analysis theory directly from data after extracting the rules forming the genetic search space fca based uses a genetic algorithm to select the final rule base the last step of the fca based method is a rule pruning step in order to improve the interpretability of the fuzzy rule bases the extraction of rules proposed for the fca based algorithm presents polynomial complexity and does not require the predefinition of the number of rules to be extracted as it extracts rules directly from data the proposed method avoids the random extraction of rules it also presents the advantage of automatically extracting rules with variable number of conditions in their antecedents a feature subset selection method specifically designed for fuzzy classification systems is integrated into the fca based method in order to reduce the search space of solutions the fca based method is detailed and compared to eight different rule based fuzzy systems experimental results using 27 benchmark datasets and a 10 fold cross validation strategy show that fca based presents higher accuracy and statistically significant difference with seven of the eight compared methods

stochastic gradient adaptation under general error criteria examines a family of adaptive filter algorithms of the form w sub k 1 w sub k spl mu f d sub k w sub k sup t x sub k x sub k in which f spl middot is a memoryless odd symmetric nonlinearity acting upon the error such algorithms are a generalization of the least mean square lms adaptive filtering algorithm for even symmetric error criteria for this algorithm family the authors derive general expressions for the mean and mean square convergence of the filter coefficients for both arbitrary stochastic input data and gaussian input data they then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate using the calculus of variations it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as p x p x where p x is the probability density function of the uncorrelated plant noise for faster adaptation under the white gaussian input and noise assumptions the nonlinearity is shown to be x 1 spl mu spl lambda x sup 2 spl sigma sub k sup 2 where spl lambda is the input signal power and spl sigma sub k sup 2 is the conditional error power thus the optimum stochastic gradient error criterion for gaussian noise is not mean square it is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white gaussian input simulations verify the results for a host of noise interferences and indicate the improvement using non mean square error criteria

excursions of adaptive algorithms via the poisson clumping heuristic the authors detail the application of the poisson clumping heuristic pch to the least mean square lms adaptive algorithm and its signed variants under certain conditions on the input and disturbance statistics the parameter estimate errors form a markov process the pch asserts that large excursions of the parameter estimates occur in clumps and that these clumps are distributed in a poisson manner with parameter lambda sub b where b is the maximum parameter error expressions are derived for each of the four algorithms in the scalar case which allows calculation of lambda sub b in a relatively straightforward manner these values are compared to simulations of the algorithms given that the results are asymptotic in b the close agreement between simulated and theoretical values is striking even for very modest b the four algorithms are then compared in terms of lambda sub b some observations are made regarding the relative performance of the four variants no single lms variant always outperforms the others suggestions are made as to how this technique might be applied in the vector case and a crucial monotonicity property is verified

a nonlinear analytical model for the quantized lms algorithm the power of two step size case presents a study of the quantization effects in the finite precision lms algorithm with power of two step sizes deterministic nonlinear recursions are presented for the mean and second moment matrix of the weight vector about the wiener weight for white gaussian data models and small algorithm step size spl mu the numerical solutions of these recursions are shown to agree very closely with the monte carlo simulations during all phases of the adaptation process design examples demonstrate the selection of the number of quantizer bits and the adaptation step size spl mu to yield a desired transient behavior and cancellation performance the results obtained indicate that previous models are too conservative in predicting the converged mse for a given number of bits

multiobjective optimization and comparison of nonsingleton type 1 and singleton interval type 2 fuzzy logic systems singleton interval type 2 fuzzy logic systems flss have been widely applied in several real world applications where it was shown that the singleton interval type 2 flss outperform their singleton type 1 counterparts in applications with high uncertainty levels however one of the main criticisms of singleton interval type 2 flss is the fact that they outperform singleton type 1 flss solely based on their use of extra degrees of freedom extra parameters and that type 1 flss with a sufficiently large number of parameters may provide the same performance as interval type 2 flss in addition most works on type 2 flss only compare their results with singleton type 1 flss but fail to consider nonsingleton type 1 systems in this paper we aim to directly address and investigate this criticism in order to do so we will perform a comparative study between optimized singleton type 1 nonsingleton type 1 and singleton interval type 2 flss under the presence of noise we will also present a multiobjective evolutionary algorithm moea for the optimization of singleton type 1 nonsingleton type 1 and singleton interval type 2 fuzzy systems for function approximation problems the moea will aim to satisfy two objectives to maximize the accuracy of the fls and minimize the number of rules in the fls thus improving its interpretability furthermore we will present a methodology to obtain optimal consequents for the flss hence this paper has two main contributions first it provides a common methodology to learn the three types of flss i e singleton type 1 nonsingleton type 1 and singleton interval type 2 flss from data samples the second contribution is the creation of a common framework for the comparison of type 1 and type 2 flss that allows us to address the aforementioned criticism we provide details of a series of experiments and include statistical analysis showing that the type 2 fls is able to handle higher levels of noise than its nonsingleton and singleton type 1 counterparts

adaptive algorithms with nonlinear data and error functions the tools of nonlinear system theory are used to examine several common nonlinear variants of the lms algorithm and derive a persistence of excitation criterion for local exponential stability the condition is tight when the inputs are periodic and a generic counterexample is demonstrated which gives local instability for a large class of such nonlinear versions of lms specifically those which utilize a nonlinear data function the presence of a nonlinear error function is found to be relatively benign in that it does not affect the stability of the error system rather it defines the cost function the algorithm tends to minimize specific examples include the dead zone modification the cubed data nonlinearity the cubed error nonlinearity the signed regressor algorithm and a single layer version of the backpropagation algorithm

convergence of the signed output error adaptive identifier this paper considers an adaptive output error identifier with a signum function in its update kernel it is shown that the classical strict positive real spr condition required for the convergence of traditional adaptive identifiers does not suffice for the convergence of this signed identifier instead what is needed is a stronger operator condition called the strict dominant passive sdp condition we give an analog of the kalman yakubovic popov lemma for the sdp conditions and using it give a convergence proof under the assumptions of persistent excitation and sdp

a roundoff error analysis of the lms adaptive algorithm the steady state output error of the least mean square lms adaptive algorithm due to the finite precision arithmetic of a digital processor is analyzed it is found to consist of three terms 1 the error due to the input data quantization 2 the error due to the rounding of the arithmetic operations in calculating the filter s output and 3 the error due to the deviation of the filter s coefficients from the values they take when infinite precision arithmetic is used the last term is of paricular interest because its mean squared value is inversely proportional to the adaptation step size both fixed and floating point arithmetics are examined and the expressions for the final mean square error are found to be similar the relation between the quantization error and the error that occurs when adaptation possibly ceases due to quantization is also investigated

a review of the application of multiobjective evolutionary fuzzy systems current status and further directions over the past few decades fuzzy systems have been widely used in several application fields thanks to their ability to model complex systems the design of fuzzy systems has been successfully performed by applying evolutionary and in particular genetic algorithms and recently this approach has been extended by using multiobjective evolutionary algorithms which can consider multiple conflicting objectives instead of a single one the hybridization between multiobjective evolutionary algorithms and fuzzy systems is currently known as multiobjective evolutionary fuzzy systems this paper presents an overview of multiobjective evolutionary fuzzy systems describing the main contributions on this field and providing a two level taxonomy of the existing proposals in order to outline a well established framework that could help researchers who work on significant further developments finally some considerations of recent trends and potential research directions are presented

adaptive threshold nonlinear algorithm for adaptive filters with robustness against impulse noise a new adaptation algorithm for adaptive filters is proposed introducing adaptive threshold in the nonlinear correlation function the adaptive threshold controlled by the long term average of the error signal power makes the filtering system highly robust against impulse noise which is demonstrated by the results of simulation and theoretical calculation

junction points in singular optimal control optimal control trajectories which contain both non singular and singular subarcs have been known for almost 40 years points where these two types of subarc are joined are known as junction points necessary and sufficient conditions for the optimality of these so called partially singular trajectories are well known necessary conditions for optimality at junction points have also been found however for many years the status of mcdanell s conjecture has been unknown one part of the conjecture was proved false in 1987 but the remaining part has been hard to prove or disprove there has been much evidence that the whole of the conjecture is false in general although it is true in several special cases this paper presents a counterexample for the remaining part of mcdanell s conjecture a minimum of nine equations and two inequalities had to be satisfied at the junction in order for the counterexample to be constructed the resulting system is of third order and the problem is of lagrange type

convergence analysis of an algorithm for blind equalization the authors consider a key algorithm for blind equalization and derive expressions for the evolution of the equalizer output error trajectory the authors develop a model to examine the convergence behavior of this algorithm suitable approximations are incorporated into the model to facilitate analysis the validity of these approximations is demonstrated for a typical communication channel it is shown that even for a channel for which the assumption of gaussianity of the equalizer input data may not be very good the analysis presented predicts the convergence behavior reasonably well

fuzzy wavelet polynomial neural networks analysis and design in this study we propose a concept of fuzzy wavelet polynomial neural networks fwpnns based on concepts and constructs of polynomial neural networks and fuzzy wavelet neurons fwns these networks exhibit a rule based architecture while each rule in the fwn consists of the premise part and consequence part the premise part is realized by using c means clustering method while the consequence part is realized by means of wavelet functions whose parameters are estimated with the aid of the least square method in some sense the fwpnn can be regarded as a generalized fuzzy wavelet neural network fwnn unlike gaussian membership functions that are commonly utilized to implement the premise part of the rules in typical fwnns c means method is employed here to overcome a possible curse of dimensionality polynomial neural networks pnns are used to express the nonlinearity of a complex system furthermore the particle swarm optimization is used to optimize the design parameters of the proposed network based on the pnns and fwnns the proposed fwpnns take advantages of these two neural networks it exhibits the abilities to describe high order nonlinear relations between input and output variables and it is beneficial to describe models impacted by uncertainty the proposed fwpnns are applied for time series prediction and regression problems e g control of dynamic plants several well known modeling benchmarks including regression and time series are considered to evaluate the performance of the proposed fwpnns a comparative analysis shows that the proposed fwpnns result in better performance when comparing with some previous models reported in the literature

correlation function for the weight sequence of the frequency domain adaptive filter the statistical behavior of the single complex adaptive weight in the frequency domain adaptive filter is further investigated in a previous note 8 an expression was evaluated for the expected value of the magnitude square of the weight sequence at any time in this note the covariance of the weight at different times is evaluated and in steady state shown to be geometric with the same time constant as the frequency domain adaptive filter algorithm itself using these results the reduction in the fluctuations of the weight values are computed for an exponentially decaying post algorithm filter

on weight update saturation nonlinearities in lms adaptation the effect of a saturation type nonlinearity in the weight update equation in least mean square lms adaptation is investigated for a white gaussian data model nonlinear difference equations that include the effect of a 1 e sup ax saturation type nonlinearity on the update term driving the algorithm are derived for the weight first and second moments a nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures the steady state second moment weight behavior is evaluated approximately for the nonlinearity using these results the tradeoff between the extent of weight update saturation steady state excess mean square error and rate of algorithm convergence is studied for the same steady state misadjustment error the tradeoff results are discussed

the search for a counter example in a class of optimal control problems ii second order systems a certain theorem in the literature concerns necessary conditions for optimality at a junction joining a singular and non singular subare in a previous investigation it was shown that the theorem is correct fur all first order systems and any performance index of given typo under a chosen set of parameter values the present paper considers a special class of second order bilinear systems and a performance index of mayer type it is demonstrated that contrary to expectations the earlier analysis carries over to this more general system and the theorem is again found to be correct the authors conjecture that a similar result is also true for the most general form of second order system

comparison of the convergence of two algorithms for adaptive fir digital filters the convergence properties of two different algorithms for the updating of the coefficients of an adaptive fir digital filter are investigated and compared with one another these algorithms are the stochastic iteration algorithm and the sign algorithm in this latter algorithm a one bit gradient estimation is used which makes its implementation very simple the convergence is characterized by the residual echo variance after convergence and a parameter that indicates the speed of the convergence it is shown that the convergence of the sign algorithm can always be assured but is much slower than that of the stochastic iteration algorithm if the same variance of the residual echo is to be obtained

a nonlinear analytical model for the quantized lms algorithm the arbitrary step size case this paper extends conditional moment techniques previously developed for the study of nonlinear versions of the lms algorithm to the study of the effects of quantizers in the finite precision case deterministic nonlinear recursions are derived for the mean and second moment matrix of the weight vector about the wiener weight for white gaussian data models and small algorithm step sizes spl mu these recursions are solved numerically and shown to be in very close agreement with the monte carlo simulations during all phases of the adaptation process a design example is presented that demonstrates how the theory can be used to select the number of quantizer bits and the adaptation step size spl mu to yield a desired transient behavior and cancellation performance

characterizing persistent excitation for the sign sign equation error identifier abstract the sign sign ss algorithm is a computationally efficient adaptive identifier often used in signal processing tasks it is obtained by introducing signum functions on both the regressor and the prediction error multiplicands in the update kernel of the well known lms algorithm this paper gives a deterministic persistent excitation condition on the regressor sequence which guarantees ss convergence it also gives conditions under which ss may diverge and discusses how these persistent excitation conditions may be verified through a finite amount of computations

improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm convergence analysis of stochastic gradient adaptive filters using the sign algorithm is presented in this paper the methods of analysis currently available in literature assume that the input signals to the filter are white this restriction is removed for gaussian signals in our analysis expressions for the second moment of the coefficient vector and the steady state error power are also derived simulation results are presented and the theoretical and empirical curves show a very good match

analysis and implementation of variable step size adaptive algorithms stochastic gradient adaptive filtering algorithms using variable step sizes are investigated the variable step size algorithm improves the convergence rate while sacrificing little in steady state error expressions describing the convergence of the mean and mean squared values of the coefficients are developed and used to calculate the mean square error evolution the initial convergence rate and the steady state error are also investigated the performance of the algorithm is studied when a power of two quantizer algorithm is used and finite word length effects are considered the analytical results are verified with simulations encompassing variable applications two cmos implementations of the variable step size power of two quantizer algorithm are presented to demonstrate that the performance gains are attainable with only a modest increase in circuit complexity

a multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy rule based systems in this paper we propose the use of a multiobjective evolutionary approach to generate a set of linguistic fuzzy rule based systems with different tradeoffs between accuracy and interpretability in regression problems accuracy and interpretability are measured in terms of approximation error and rule base rb complexity respectively the proposed approach is based on concurrently learning rbs and parameters of the membership functions of the associated linguistic labels to manage the size of the search space we have integrated the linguistic two tuple representation model which allows the symbolic translation of a label by only considering one parameter with an efficient modification of the well known 2 2 pareto archived evolution strategy paes we tested our approach on nine real world datasets of different sizes and with different numbers of variables besides the 2 2 paes we have also used the well known nondominated sorting genetic algorithm nsga ii and an accuracy driven single objective evolutionary algorithm ea we employed these optimization techniques both to concurrently learn rules and parameters and to learn only rules we compared the different approaches by applying a nonparametric statistical test for pairwise comparisons thus taking into consideration three representative points from the obtained pareto fronts in the case of the multiobjective eas finally a data complexity measure which is typically used in pattern recognition to evaluate the data density in terms of average number of patterns per variable has been introduced to characterize regression problems results confirm the effectiveness of our approach particularly for possibly high dimensional datasets with high values of the complexity metric

a dynamic convergence analysis of blind equalization algorithms blind equalizers do not require a training sequence to start up or to restart after a communications breakdown making them particularly useful in applications such as broadcast and point to multipoint networks we study in parallel the dynamic convergence behavior of three blind equalization algorithms the multimodulus algorithm mma the constant modulus algorithm cma and the reduced constellation algorithm rca using a conditional gaussian approximation we first derive the theoretical mean squared error mse trajectory for mma this analysis leads to accurate but somewhat cumbersome expressions alternatively we apply a taylor series approximation to derive mse trajectories for mma cma and rca this approach yields simpler but somewhat less accurate expressions for the steady state operation however we derive even simpler formulas that accurately predict the asymptotic mse values we finally study the convergence rates of the three blind algorithms using their theoretical mse trajectories computer simulations and a laboratory experiment

transient analysis of adaptive filters with error nonlinearities the paper develops a unified approach to the transient analysis of adaptive filters with error nonlinearities in addition to deriving earlier results in a unified manner the approach also leads to new performance results without restricting the regression data to being gaussian or white the framework is based on energy conservation arguments and avoids the need for explicit recursions for the covariance matrix of the weight error vector

on the optimum data nonlinearity in lms adaptation the effect of an arbitrary nonlinear operation on the data input to the weight update equation in the lms adaptive algorithm is investigated for a gaussian data model exact difference equations are derived for the weight first and second moments that include the effects of an arbitrary nonlinear operation on the data sequence the difference equations are used to obtain expressions for the transient behavior of the mean square error the mean square error is minimized over the choice of nonlinearity for a fixed transient behavior the best choice of nonlinearity is shown exactly to be of the form x 1 bx2 when the input data are white for an arbitrary data covariance matrix the optimum nonlinearity is shown to be linear when the product of the algorithm step size and input power is much less than unity

identification of transparent compact accurate and reliable linguistic fuzzy models transparency accuracy compactness and reliability all appear to be vital even though somewhat contradictory requirements when it comes down to linguistic fuzzy modeling this paper presents a methodology for simultaneous optimization of these criteria by chaining previously published various algorithms a heuristic fully automated identification algorithm that is able to extract sufficiently accurate yet reliable and transparent models from data and two algorithms for subsequent simplification of the model that are able to reduce the number of output parameters as well as the number of fuzzy rules with only a marginal negative effect to the accuracy of the model

i first order systems some optimal control trajectories include both non singular and singular arcs at points where these two types of arc are joined called junctions certain necessary conditions for optimality are known the status of one theorem from the literature concerning a particular sot of conditions has been the subject of previous investigation this paper describes the initial work in a search for a counter example to the results of that theorem although the theory yields evidence to support the existence of a counter example it is shown that the theorem is in fact true for all first order systems and any performance index when specified values are given to certain parameters

adaptive frequency sampling filters we present two new structures for adaptive filters based on the idea of frequency sampling filters and gradient based estimation algorithms these filters have a finite impulse response fir and can be thought of as attempting to approximate a desired frequency response at given points on the unit circle the filters operate in real time with no batch processing of signals as is the case when using the discrete fourier transform they result in a marked reduction in dimension of the time domain problem of fitting an nth order fir transversal filter to a collection of length 2 transversal filters and further to a collection of n scalar filters the advantages of this are then discussed

an extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons in a recently published paper in jmlr dem sar 2006 recommends a set of non parametric statistical tests and procedures which can be safely used for comparing the performance of classifiers over multiple data sets after studying the paper we realize that the paper correctly introduces the basic procedures and some of the most advanced ones when comparing a control method however it does not deal with some advanced topics in depth regarding these topics we focus on more powerful proposals of statistical procedures for comparing n n classifiers moreover we illustrate an easy way of obtaining adjusted and comparable p values in multiple comparison procedures

performance analysis of adaptive filters equipped with the dual sign algorithm a convergence analysis for stochastic gradient adaptive filters equipped with the dual sign algorithm dsa is presented expressions for the mean and mean squared values of the coefficient misalignment vector are derived under the assumption that the input signal is gaussian the main differences between the current analysis and previous analyses are 1 the present analysis is valid for arbitrary autocorrelation matrices and 2 the analysis does not approximate the behavior for the dsa as switching between two sign algorithms a certain amount of time after the dsa is initialized a simulation example comparing the analytical results with empirical ones is presented and the two curves show excellent agreement with each other

a class of adaptive step size control algorithms for adaptive filters a class of new adaptive step size control algorithms which is applicable to most of the lms derived tap weight adaptation algorithms is proposed analysis yields a set of difference equations for theoretically calculating the transient behavior of the filter convergence and derives an explicit formula for the steady state excess mean square error emse experiments for some examples prove that the proposed algorithm is highly effective in improving the convergence rate in both transient and tracking phases the theoretically calculated convergence is shown to be in good agreement with that obtained through simulations alternative formulae of the step size adaptation for specific tap weight adaptation algorithms are also proposed

weak convergence and local stability properties of fixed step size recursive algorithms a recursive equation that subsumes several common adaptive filtering algorithms is analyzed for general stochastic inputs and disturbances by relating the motion of the parameter estimate errors to the behavior of an unforced deterministic ordinary differential equation ode the odes describing the motion of several common adaptive filters are examined in some simple settings including the least mean square lms algorithm and all three of its signed variants the signed regressor the signed error and the sign sign algorithms stability and instability results are presented in terms of the eigenvalues of a correlation like matrix this generalizes known results for lms signed regressor lms and signed error lms and gives new stability criteria for the sign sign algorithm the ability of the algorithms to track moving parameterizations can be analyzed in a similar manner by relating the time varying system to a forced ode the asymptotic distribution about the forced ode is an ornstein uhlenbeck process the properties of which can be described in a straightforward manner

three objective genetics based machine learning for linguisitc rule extraction abstract this paper shows how a small number of linguistically interpretable fuzzy rules can be extracted from numerical data for high dimensional pattern classification problems one difficulty in the handling of high dimensional problems by fuzzy rule based systems is the exponential increase in the number of fuzzy rules with the number of input variables another difficulty is the deterioration in the comprehensibility of fuzzy rules when they involve many antecedent conditions our task is to design comprehensible fuzzy rule based systems with high classification ability this task is formulated as a combinatorial optimization problem with three objectives to maximize the number of correctly classified training patterns to minimize the number of fuzzy rules and to minimize the total number of antecedent conditions we show two genetic algorithm based approaches one is rule selection where a small number of linguistically interpretable fuzzy rules are selected from a large number of prespecified candidate rules the other is fuzzy genetics based machine learning where rule sets are evolved by genetic operations these two approaches search for non dominated rule sets with respect to the three objectives

application of orthonormal transforms to implementation of quasi lms newton algorithm an efficient implementation of the lms newton lmsn algorithm is proposed the scheme uses a quasi inverse of the correlation matrix of the input instead of its exact inverse the proposed algorithm which is an alternative formulation of a class of transform domain adaptive filters tdafs has some advantages over the tdaf a feature of the scheme that greatly simplifies its implementations is the possibility of coarse quantization of the stochastic gradient terms in the adjustment recursion to back this up an analysis of the lmsn algorithm which includes the effect of coarse quantization of the stochastic gradient terms is given for correlated gaussian data computer simulation results that support the developed theories are also presented

normalized data nonlinearities for lms adaptation properly designed nonlinearly modified lms algorithms in which various quantities in the stochastic gradient estimate are operated upon by memoryless nonlinearities have been shown to perform better than the lms algorithm in system identification type problems the authors investigate one such algorithm given by w sub k l w sub k spl mu d sub k w sub k sup t x sub k x sub k f x sub k in which the function f x sub k is a scalar function of the sum of the squares of the n elements of the input data vector x sub k this form of algorithm generalizes the so called normalized lms nlms algorithm they evaluate the expected behavior of this nonlinear algorithm for both independent input vectors and correlated gaussian input vectors assuming the system identification model by comparing the nonlinear algorithm s behavior with that of the lms algorithm they then provide a method of optimizing the form of the nonlinearity for the given input statistics in the independent input case they show that the optimum nonlinearity is a single parameter version of the nlms algorithm with an additional constant in the denominator and show that this algorithm achieves a lower excess mean square error mse than the lms algorithm with an equivalent convergence rate additionally they examine the optimum step size sequence for the optimum nonlinear algorithm and show that the resulting algorithm performs better and is less complex to implement than the optimum step size algorithm derived for another form of the nlms algorithm simulations verify the theory and the predicted performance improvements of the optimum normalized data nonlinearity algorithm

analysis of adaptive filters using normalized signed regressor lms algorithm in this paper adaptive filters using the normalized signed regressor lms algorithm nsra with gaussian reference inputs are proposed and analyzed to yield difference equations for theoretically calculating expected convergence of the filters a simple difference equation for mean squared error mse is derived when the filter input is a white and gaussian process whereas approximate difference equations for colored gaussian inputs are proposed and tested stability conditions and residual mse after convergence are also obtained agreement of theoretical results with those of simulation in the experiment with some examples of filter convergence shows sufficient accuracy of the theory and assures the usefulness of the difference equations in estimating filter performances thus facilitating the design of adaptive filters using the nsra