on optimal and achievable fix free codes fix free codes are prefix condition codes which can also be decoded in the reverse direction they have attracted attention from several communities and are used in video standards two variations with additional constraints have also been considered for joint source channel coding 1 symmetric codes which require the codewords to be palindromes 2 codes with distance constraints on pairs of codewords approaches to determine the existence of a fix free code with a given set of codeword lengths for each of the three variations are proposed these appear to involve the first use of boolean satisfiability sat for the existence and design of source codes branch and bound algorithms to find the collection of optimal codes for asymmetric and symmetric fix free codes are described the first bound for the performance of optimal symmetric binary fix free codes is provided an earlier conjecture on optimal symmetric binary fix free codes is proven and related results for asymmetric fix free codes are presented a variation of the 3 4 conjecture for fix free codes is introduced a key idea in the first conjecture and the new one is a definition of how one sequence of nondecreasing natural numbers dominates another

a fast linear programming algorithm for blind equalization a fast implementation of a special non mse cost function for blind equalization is presented here this baud rate equalization algorithm is based on a convex cost function coupled with a simple linear constraint on the equalizer parameters for a generic class of channels with persistently exciting quadrature amplitude modulation input signals this new algorithm allows the convergence of equalizer parameters to a unique global minimum achieving intersymbol interference suppression and carrier phase recovery

convergence of the sign algorithm for adaptive filtering with correlated data convergence of a decreasing gain sign algorithm sa for adaptive filtering is analyzed the presence of the hard limiter in the algorithm makes a rigorous analysis difficult therefore there are few results available such results normally include restrictive assumptions such as the assumptions that successive observation vectors are independent and the new error signal of the adaptive filter has a time invariant probability density function the former assumption is not valid in the context of adaptive filtering since two successive observation vectors share most of their components while the latter assumption is a restriction on the adaptive weights whose evolution is a priori unknown in lieu of using these assumptions an almost sure convergence of the sa is proved under the assumption that the sequence of observation vectors is m dependent this assumption allows strong correlation between successive observations

asymptotic properties of sign algorithms for adaptive filtering this paper develops asymptotic properties of a class of sign error algorithms with expanding truncation bounds for adaptive filtering under merely stationary ergodicity and finite second moments of the reference and output signals and using trajectory subsequence ts method it is proved that the algorithm convergers almost surely then a mean squares estimate is derived for the estimation error and a suitably scaled sequence of the estimation error is shown to converge to a diffusion process the scaling factor together with the stationary covariance gives the rate of convergence result moreover an algorithm under mean squares criterion with expanding truncation bounds is also examined compared with the existing results in the literature sufficient conditions for almost sure convergence are much relaxed a simple example is provided for demonstration purpose

convergence analysis of finite alphabet beamformers for digital cochannel signals in this paper we examine the convergence behavior of finite alphabet fa beamformers the two most popular implementations of fa beamformers for digital communication signals are the iterative least squares with projection ilsp and the minimum mean square error mmse beamformer to facilitate the analysis for the binary communications case we derive closed form expressions for the mean weight vector signal to noise ratio and signal to interference ratio for both the ilsp and mmse beamformers in terms of the bit error rate ber performance at each iteration next we generalize the analysis for the m ary pulse amplitude modulation and m ary phase shift keying cases we show that both ilsp and mmse beamformers have previously unreported bias terms in the array response vectors which are functions of the ber for each iteration furthermore as the ber becomes arbitrarily small we show that our solutions converge to the well known asymptotic expressions widely published in the literature next we provide a geometric interpretation of the effects of the noise bias vector in terms of angles between subspaces based on our analysis we were able to develop necessary and sufficient conditions for convergence in the mean we conclude with monte carlo simulations to validate our analysis

on line blind multichannel equalization based on mutually referenced filters this paper presents a novel approach to the blind linear equalization of possibly nonminimum phase and time varying communication channels in the context of channel diversity we introduce the concept of mutually referenced equalizers mres in which several filters are considered the outputs of which act as training signals for each other a corresponding constrained multidimensional mean square error mse cost function is derived the minimization of which is shown to be a necessary and sufficient condition for equalization the links with a standard linear prediction problem are demonstrated the proposed technique exhibits properties of important practical concern 1 the proposed algorithm is globally convergent 2 simple closed form solutions exist for the mres but the mres also lend themselves readily to adaptive implementation in particular the recursive least squares rls algorithm can be used to offer optimal convergence rate 3 the mre method provides a solution for all equalization delays which results in robustness properties with respect to snr and ill defined channel lengths

analysis of boundedness and convergence of online gradient method for two layer feedforward neural networks this paper presents a theoretical boundedness and convergence analysis of online gradient method for the training of two layer feedforward neural networks the well known linear difference equation is extended to apply to the general case of linear or nonlinear activation functions based on this extended difference equation we investigate the boundedness and convergence of the parameter sequence of concern which is trained by finite training samples with a constant learning rate we show that the uniform upper bound of the parameter sequence which is very important in the training procedure is the solution of an inequality regarding the bound it is further verified that for the case of linear activation function a solution always exists and moreover the parameter sequence can be uniformly upper bounded while for the case of nonlinear activation function some simple adjustment methods on the training set or the activation function can be derived to improve the boundedness property then for the convergence analysis it is shown that the parameter sequence can converge into a zone around an optimal solution at which the error function attains its global minimum where the size of the zone is associated with the learning rate particularly for the case of perfect modeling a strong global convergence result where the parameter sequence can always converge to an optimal solution is proved

blind equalization based on joint minimum mse criterion a joint minimum mean square error mse criterion is presented for blind adaptive channel equalization of qam signals by jointly estimating the transmitted sequence and the equalizer parameters through the minimization of mse the equalizer parameters can be optimized and the channel input sequence can be restored at the equalizer output to improve computational efficiency a reduced constellation recursive least squares rls algorithm is introduced for blind equalisation simulation results show the success of both the least square and the reduced constellation rls algorithms

a soft switching blind equalization scheme via convex combination of adaptive filters blind equalizers avoid the transmission of pilot sequences allowing a more efficient use of the channel bandwidth normally after a first rough equalization is achieved it is necessary to switch these equalizers to a decision directed dd mode to reduce the steady state mean square error mse to acceptable levels the selection of an appropriate mse threshold for switching between the blind and the dd modes is critical to obtain a good overall performance however this is not an easy task since it depends on several factors such as the signal constellation the communication channel or the signal to noise ratio in this paper we propose an equalization scheme that adaptively combines a blind and a dd equalizers running in parallel the combination is itself adapted in a blind manner and as a result the overall scheme can automatically switch between the component filters avoiding the need to set the transition mse level a priori the performance of our proposal is illustrated both analytically and through an extensive set of simulations where we show its advantages with respect to existing hard and soft switching equalization schemes

convergence analysis of self adaptive equalizers a theoretical analysis of self adaptive equalization for data transmission is carried out starting from known convergence results for the corresponding trained adaptive filter the development relies on a suitable ergodicity model for the sequence of observations at the output of the transmission channel thanks to the boundedness of the decision function used for data recovery it can be proved that the algorithm is bounded strong convergence results can be reached when a perfect noiseless equalizer exists the algorithm will converge to it if the eye pattern is initially open otherwise convergence may take place towards certain other stationary points of the algorithm for which domains of attraction have been defined some of them will result in a poor error rate the case of a noisy channel exhibits limit points for the algorithm that differ from those of the classical trained algorithm the stronger the noise the greater the difference is one of the principal results of this study is the proof of the stability of the usual decision feedback algorithms once the learning period is over

on constructing the huffman code based reversible variable length codes in this letter we propose a generic and efficient algorithm that can construct both asymmetrical and symmetrical reversible variable length codes rvlcs starting from a given huffman code the construction is based on two developed codeword selection mechanisms for the symmetrical case and the asymmetrical case respectively it is shown that the two mechanisms possess simple features and can generate efficient rvlcs easily in addition two new asymmetrical rvlcs are constructed and shown to be very efficient for further reducing the coding overheads in mpeg 4 when operating in the reversible decoding mode

an a based algorithm for constructing reversible variable length codes with minimum average codeword length variable length codes vlcs are widely adopted in many compression standards due to their good coding efficiency on average codeword length however an inherent problem with a vlc is that an error of even one bit can cause serious error propagation and thus loss of synchronization at the receiver which would lead to a series of non correctly decoded symbols reversible variable length codes rvlcs were introduced to significantly mitigate this phenomenon in this work a method to find an optimal rvlc in terms of the minimum average codeword length is first formulated as a tree searching problem and then instead of performing an exhaustive search an a based construction algorithm is proposed to find an optimal rvlc the proposed algorithm has been applied to several benchmarks for sources and has found respective optimal symmetric and asymmetric rvlcs

robust blind learning algorithm for nonlinear equalization using input decision information in this paper we propose a new blind learning algorithm namely the benveniste goursat input output decision bg iod to enhance the convergence performance of neural network based equalizers for nonlinear channel equalization in contrast to conventional blind learning algorithms where only the output of the equalizer is employed for updating system parameters the bg iod exploits a new type of extra information the input decision information obtained from the input of the equalizer to mitigate the influence of the nonlinear equalizer structure on parameters learning thereby leading to improved convergence performance we prove that with the input decision information a desirable convergence capability that the output symbol error rate ser is always less than the input ser if the input ser is below a threshold can be achieved then the bg soft switching technique is employed to combine the merits of both input and output decision information where the former is used to guarantee ser convergence and the latter is to improve ser performance simulation results show that the proposed algorithm outperforms conventional blind learning algorithms such as stochastic quadratic distance and dual mode constant modulus algorithm in terms of both convergence performance and ser performance for nonlinear equalization

joint signal detection and parameter estimation in multiuser communications the problem of simultaneously detecting the information bits and estimating signal amplitudes and phases in a k user asynchronous direct sequence spread spectrum multiple access communication system is addressed the joint maximum likelihood ml estimator has a computational complexity that is exponential in the total number of bits transmitted and thus does not represent a practical solution to the problem an estimator that combines a suboptimum tree search algorithm with a recursive least squares estimator of complex signal amplitude is considered the complexity of this estimator is o k sup 2 computations per decoded bit and its performance is very close to that of the joint ml receiver this receiver has the advantage that the transmitted signal powers and phases are extracted from the received signal in an adaptive fashion without using a test sequence

on the convergence of blind adaptive equalizers for constant modulus signals this paper studies the behavior of the error sequence of stop anti go variants of two adaptive blind equalizers namely cma2 2 and sato s 1975 algorithm it is shown that for transmitted signals with constant modulus spl gamma the equalizer output can be made to lie within the circle of radius spl gamma spl radic c infinitely often for some value of c that is only slightly larger than one

length and cost dependent local minima of unconstrained blind channel equalizers baud rate linear blind equalizers may converge to undesirable stable equilibria due to different mechanisms one such mechanism is the use of linear fir filters as equalizers it is shown that this type of local minima exist for all unconstrained blind equalizers whose cost functions satisfy two general conditions the local minima generated by this mechanism are thus called length dependent local minima another mechanism is generated by the cost function adopted by the blind algorithm itself this type of local minima are called cost dependent local minima it is shown that several well designed algorithms do not have cost dependent local minimum whereas other algorithms such as the decision directed equalizer and the stop and go algorithm sga do unlike many existing convergence analysis the convergence of the godard 1980 algorithms gas and standard cumulant algorithms scas under gaussian noise is also presented computer simulations are used to verify the analytical results

new classes of finite dimensional nonlinear filters abstract we derive sufficient conditions for the estimation algebras of two special classes of nonlinear estimation systems to be finite dimensional by carrying out the brockett wei norman program new classes of finite dimensional filters are obtained these systems can be regarded respectively as extensions of the kalman bucy filters and the benes filters

the construction of huffman codes is a submodular convex optimization problem over a lattice of binary trees we show that the space of all binary huffman codes for a finite alphabet defines a lattice ordered by the imbalance of the code trees representing code trees as path length sequences we show that the imbalance ordering is closely related to a majorization ordering on real valued sequences that correspond to discrete probability density functions furthermore this tree imbalance is a partial ordering that is consistent with the total orderings given by either the external path length sum of tree path lengths or the entropy determined by the tree structure on the imbalance lattice we show the weighted path length of a tree the usual objective function for huffman coding is a submodular function as is the corresponding function on the majorization lattice submodular functions are discrete analogues of convex functions these results give perspective on huffman coding and suggest new approaches to coding as optimization over a lattice

analysis of mean square error and transient speed of the lms adaptive algorithm for the least mean square lms algorithm we analyze the correlation matrix of the filter coefficient estimation error and the signal estimation error in the transient phase as well as in steady state we establish the convergence of the second order statistics as the number of iterations increases and we derive the exact asymptotic expressions for the mean square errors in particular the result for the excess signal estimation error gives conditions under which the lms algorithm outperforms the wiener filter with the same number of taps we also analyze a new measure of transient speed we do not assume a linear regression model the desired signal and the data process are allowed to be nonlinearly related the data is assumed to be an instantaneous transformation of a stationary markov process satisfying certain ergodic conditions

on the penalty of optimal fix free codes in this paper the difference between the redundancy of the optimal asymmetric symmetric fix free code and that of the optimal prefix free code is considered as the penalty of benefiting from the desired properties of fix free codes this penalty is studied from different perspectives in particular it is shown that the average penalty of asymmetric fix free codes is less than 0 21 bit per symbol moreover it is proved that when the source alphabet size is sufficiently large for almost all sources the penalty is less than or equal to 0 182 bit per symbol regarding symmetric fix free codes it is shown that the average penalty tends to infinity as the source alphabet size increases

quiver diagrams and signed adaptive filters a graphical technique for exploring the behavior of deterministic discrete time adaptive algorithms excited by periodic inputs is examined the constraint to periodic inputs is convenient the underlying concepts used also apply to stationary inputs drawn from finite alphabets the resulting graph is based on plotting single period parameter trajectories in the parameter error space and is essentially a discrete version of flow diagrams associated with continuous time systems for the least mean square lms adaptive fir filter with the algorithm step size sufficiently small the change of adapted parameters from a particular initial value is given by the total update over the input period with the parameterization frozen at its initial value this results in the familiar elliptical hyperparaboloid surface steepest descent interpretation this method can be extended to the lms adaptive infinite input response iir filter with suitable approximations based on a small step size

blind equalization by direct examination of the input sequences this paper presents a novel approach to blind equalization deconvolution which is based on direct examination of possible input sequences in contrast to many other approaches it does not rely on a model of the approximative inverse of the channel dynamics to start with the blind equalization identifiability problem for a noise free finite impulse response channel model is investigated a necessary condition for the input which is algorithm independent for blind deconvolution is derived this condition is expressed in an information measure of the input sequence a sufficient condition for identifiability is also inferred which imposes a constraint on the true channel dynamics the analysis motivates a recursive algorithm where all permissible input sequences are examined the exact solution is guaranteed to be found as soon as it is possible an upper bound on the computational complexity of the algorithm is given this algorithm is then generalized to cope with time varying infinite impulse response channel models with additive noise the estimated sequence is an arbitrary good approximation of the maximum a posteriori estimate the proposed method is evaluated on a rayleigh fading communication channel the simulation results indicate fast convergence properties and good tracking abilities

theoremes de convergence presque sure pour une classe d algorithmes stochastiques a pas decroissant the paper studies the pathwise asymptotic behaviour of stochastic algorithms of the following general form theta n 1 theta n gamma n 1 f theta n y n 1 the hypotheses allowing discontinuities on the adaptation termf the process y n n 0 is a markov chain controlled by n for each fixed the markov chain y n n 0 is essentially of positive recurrent type

on convergence analysis of fractionally spaced adaptive blind equalizers in this paper we study the convergence analysis of fractionally spaced adaptive blind equalizers we show that based on the trivial and nontrivial nullspaces of a channel convolution matrix all equilibria can be classified as channel dependent equilibria cde or algorithm dependent equilibria ade because oversampling provides channel diversity the nullspace of the channel convolution matrix is affected we show that fractionally spaced equalizers fses do not possess any cde if a length zero condition is satisfied the convergence behavior of these fse are clearly determined by the specific choice of cost function alone we characterize the global convergence ability of several popular algorithms simply based on their ade we also present an fse implementation of the super exponential algorithm we show that the fse implementation does not introduce any nonideal approximation simulation results are also presented to illustrate the robustness and the improved performance of fse under the super exponential algorithm

the most likely optimal symmetric rvlc for each alphabet size up to 30 a symmetric reversible variable length code rvlc is determined which is most likely to be optimal in average codeword length it is shown that these codes are optimal for more than half of all the possible sources

second order convergence analysis of stochastic adaptive linear filtering the convergence of an adaptive filtering vector is studied when it is governed by the mean square error gradient algorithm with constant step size we consider the mean square deviation between the optimal filter and the actual one during the steady state this quantity is known to be essentially proportional to the step size of the algorithm however previous analyses were either heuristic or based upon the assumption that successive observations were independent which is far from being realistic actually in most applications two successive observation vectors share a large number of components and thus they are strongly correlated in this work we deal with the case of correlated observations and prove that the mean square deviation is actually of the same order or less than the step size of the algorithm this result is proved without any boundedness or barrier assumption for the algorithm as it has been done previously in the literature to ensure the nondivergence our assumptions are reduced to the finite strong memory assumption and the finite moments assumption for the observation they are satisfied in a very wide class of practical applications

stochastic approximation with correlated data new almost sure convergence results for a special form of the multidimensional robbins monro stochastic approximation procedure are developed the results are applicable to cases where the training data is heavily correlated no conditional expectation properties or boundedness assumptions are required to apply the new results for example when the data sequence is normal and i m dependent ii autoregressive moving average or iii band limited the results can be used to establish the almost sure convergence of each algorithm treated the special form of the robbins monro procedure considered is motivated by a consideration of several algorithms that have been proposed for discrete time adaptive signal processing applications most of these algorithms can also be viewed as stochastic gradient following algorithm the ease with which the new results can be applied is illustrated

convergence of an adaptive linear estimation algorithm in this work we prove the almost sure convergence of an adaptive linear estimator governed by a stochastic gradient algorithm with decreasing step size in the presence of correlated observations two complementary contributions are added to the famous 1977 ljung theorem first we drop the condition of nondivergence of the algorithm assumed by ljung while that condition can be ensured by adding a barrier the convergence of the suitably bounded algorithm itself is not established even on the basis of ljung theorem here the barrier problem is overcome by proving that it is not necessary for the convergence our second contribution is to generalize the model describing the correlated observations no state space model is used and no linear relationship between the observations and the signal to be estimated needs to be assumed instead we use a decreasing covariance model that agrees with a very wide class of practical applications

static and dynamic convergence behavior of adaptive blind equalizers this paper presents a theoretical analysis of the static and dynamic convergence behavior for a general class of adaptive blind equalizers we first study the properties of prediction error functions of blind equalization algorithms and then we use these properties to analyze the static and dynamic convergence behavior based on the independence assumption we prove in this paper that with a small step size the ensemble average of equalizer coefficients will converge to the minimum of the cost function near the channel inverse however the convergence is not consistent the correlation matrix or equalizer coefficients at equilibrium are determined by a lyapunov equation according to our analysis results for a given channel and stepsize there is an optimal length for an equalizer to minimize the intersymbol interference this result implies that a longer length blind equalizer does not necessarily outperform a shorter one which is contrary to what is conventionally conjectured the theoretical analysis results are confirmed by computer simulations

modified symmetrical reversible variable length code and its theoretical bounds reversible variable length codes rvlcs have been adopted in emerging video coding standards h 263 and mpeg 4 to enhance their error resilience capabilities which are important and essential in error prone environments this study proposes an efficient algorithm to construct a symmetrical rvlc from a given huffman code in addition theoretical bounds on the maximum codeword length for fixed length huffman codes and on the optimal average codeword lengths for sources with exponential distribution are provided

error energy bounds for adaptive gradient algorithms the paper establishes robustness optimality and convergence properties of the widely used class of instantaneous gradient adaptive algorithms the analysis is carried out in a purely deterministic framework and assumes no a priori statistical information it employs the cauchy schwarz inequality for vectors in an euclidean space and derives local and global error energy bounds that are shown to highlight as well as explain relevant aspects of the robust performance of adaptive gradient filters along the lines of h sup spl infin theory

new criteria for blind deconvolution of nonminimum phase systems channels a necessary and sufficient condition for blind deconvolution without observing the input of nonminimum phase linear time invariant systems channels is derived based on this condition several optimization criteria are proposed and their solution is shown to correspond to the desired response these criteria involve the computation only of second and fourth order moments implying a simple tap update procedure the proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the unobserved input sequence it is shown that in several important cases e g when the additive noise is gaussian the proposed criteria are essentially unaffected

continuous time tracking algorithms involving two time scale markov chains this work is concerned with least mean squares lms algorithms in continuous time for tracking a time varying parameter process a distinctive feature is that the true parameter process is changing at a fast pace driven by a finite state markov chain the states of the markov chain are divisible into a number of groups within each group the transitions take place rapidly among different groups the transitions are infrequent introducing a small parameter into the generator of the markov chain leads to a two time scale formulation the tracking objective is difficult to achieve nevertheless a limit result is derived yielding algorithms for limit systems moreover the rates of variation of the tracking error sequence are analyzed under simple conditions it is shown that a scaled sequence of the tracking errors converges weakly to a switching diffusion in addition a numerical example is provided and an adaptive step size algorithm developed

nonlinear blind equalization schemes using complex valued multilayer feedforward neural networks among the useful blind equalization algorithms stochastic gradient iterative equalization schemes are based on minimizing a nonconvex and nonlinear cost function however as they use a linear fir filter with a convex decision region their residual estimation error is high in the paper four nonlinear blind equalization schemes that employ a complex valued multilayer perceptron instead of the linear filter are proposed and their learning algorithms are derived after the important properties that a suitable complex valued activation function must possess are discussed a new complex valued activation function is developed for the proposed schemes to deal with qam signals of any constellation sizes it has been further proven that by the nonlinear transformation of the proposed function the correlation coefficient between the real and imaginary parts of input data decreases when they are jointly gaussian random variables last the effectiveness of the proposed schemes is verified in terms of initial convergence speed and mse in the steady state in particular even without carrier phase tracking procedure the proposed schemes correct an arbitrary phase rotation caused by channel distortion