complex elliptically symmetric random variables generation characterization and circularity tests complex elliptically symmetric ces distributions constitute a flexible and broad class of distributions for many engineering applications and include the widely used complex gaussian complex complex generalized gaussian and symmetric stable distributions for example their careful statistical characterization is needed moreover the mechanisms for generating random variables from these distributions are not well defined in literature in this paper we provide such treatment in order to provide a better insight on their statistical properties and simplifying proofs and derivations for example insightful expressions for complex kurtosis coefficients of ces distributions are derived providing an interpretation on what complex kurtosis really measures also derived are asymptotic distributions of circularity measures the sample circularity quotient and coefficient assuming i i d samples from an unspecified ces distribution also new wald s type detectors of circularity are proposed that are valid within ces distributions with finite fourth order moments these results are accompanied with examples that are of interest in developing signal processing algorithms for complex valued signals

performance advantage of complex lms for controlling narrow band adaptive arrays in narrow band adaptive array applications the mean square convergence of the discrete time real least mean square lms algorithm is slowed by image frequency noises generated in the lms loops the complex lms algorithm proposed by widrow et al is shown to eliminate these noises yielding convergence of the mean squared error mse at slightly over twice the rate this paper includes a comprehensive analysis of the mse of adaptation for lms the analysis is based upon the method developed in the 1968 dissertation by k d senne and it represents the most complete treatment of the subject published to date

adaptive nonlinear system identification in the short time fourier transform domain in this paper we introduce an adaptive algorithm for nonlinear system identification in the short time fourier transform stft domain the adaptive scheme consists of a parallel combination of a linear component represented by crossband filters between subbands and a quadratic component which is modeled by multiplicative cross terms we adaptively update the model parameters using the least mean square lms algorithm and derive explicit expressions for the transient and steady state mean square error mse in frequency bins for white gaussian inputs we show that estimation of the nonlinear component improves the mse performance only when the power ratio of nonlinear to linear components is relatively high furthermore as the number of crossband filters increases a lower steady state mse may be obtained at the expense of slower convergence experimental results support the theoretical derivations

randomness and recursive enumerability one recursively enumerable real alpha dominates another one beta if there are nondecreasing recursive sequences of rational numbers a n n in omega approximating alpha and b n n in omega approximating beta and a positive constant c such that for all n c alpha a n geq beta b n see r m solovay draft of a paper or series of papers on chaitin s work manuscript ibm thomas j watson research center yorktown heights ny 1974 p 215 and g j chaitin ibm j res develop 21 1977 pp 350 359 we show that every recursively enumerable random real dominates all other recursively enumerable reals we conclude that the recursively enumerable random reals are exactly the omega numbers g j chaitin ibm j res develop 21 1977 pp 350 359 second we show that the sets in a universal martin lof test for randomness have random measure and every recursively enumerable random number is the sum of the measures represented in a universal martin lof test

augmented performance bounds on strictly linear and widely linear estimators with complex data novel physical insights are provided into the performances of strictly linear sl and widely linear wl estimators of the generality of complex valued data both proper second order circular and improper second order noncircular this is achieved by first performing a novel complementary mean square error cmse analysis in order to quantify the degrees of improperness second order noncircularity of the sl and wl estimation errors the exact bounds on the cmse difference between the sl and wl estimators are investigated to show that only a joint consideration of the standard mse analysis and the proposed cmse analysis has enough degrees of freedom for a rigorous account of the performance of wl and sl estimators this also makes it possible to rigourously quantify the contributions to the wl performance advantage from the individual real and imaginary channels an important finding not possible to obtain by using the standard mse analysis only

blind frequency offset estimation for ofdm oqam systems this paper deals with the problem of blind carrier frequency offset cfo estimation in orthogonal frequency division multiplexing ofdm systems based on offset quadrature amplitude modulation ofdm oqam specifically by assuming that the number of subcarriers is sufficiently large the received signal is modeled as a complex gaussian random vector cgrv since the ofdm oqam signal results to be a noncircular nc process i e its relation function is different from zero by exploiting the generalized probability density function for nc cgrvs the unconditional maximum likelihood algorithm for cfo estimation in nondispersive channel is derived moreover due to its computational complexity a more feasible estimator is considered the performance of the proposed estimators assessed via computer simulation is compared with that of recently proposed cfo estimators exploiting second order cyclostationarity and with the gaussian crameacuter rao bound

widely linear estimation with complex data mean square estimation of complex and normal data is not linear as in the real case but widely linear the purpose of the paper is to calculate the optimum widely linear mean square estimate and to present its main properties the advantage with respect to the linear procedure is especially analyzed

the multivariate complex normal distribution a generalization the multivariate complex normal distribution usually employed in the literature is a special case since certain restrictions have been imposed on the covariances of the real and imaginary parts of its variables a more general distribution is proposed of which the usual distribution is shown to be a special case

the complex lms adaptive algorithm transient weight mean and covariance with applications to the ale the transient and steady state mean and covariance of the complex valued lms adaptive element weights are investigated when the inputs are samples from circularly normal processes it is shown that the data covariance diagonalizing matrix also diagonalizes the weight covariance matrix this result permits describing the complete transient behavior of the adaptive line enhancer ale weight covariance matrix in closed form for the case of multiple equal power narrow band statistically independent orthogonal rayleigh fading sinusoids in broad band noise based on these transient covariance results it is shown that for any stage of adaptation there exists an optimum ale gain constant that minimizes the excess mean squared prediction error this result is of particular significance when processing time limited data

second order analysis of improper complex random vectors and processes we present a comprehensive treatment of the second order theory of complex random vectors and wide sense stationary wss signals the main focus is on the improper case in which the complementary covariance does not vanish accounting for the information present in the complementary covariance requires the use of widely linear transformations based on these we present the eigenanalysis of complex vectors and apply it to the problem of rank reduction through principal components we also investigate joint properties of two complex vectors by introducing canonical correlations which paves the way for a discussion of the wiener filter and its rank reduced version we link the concepts of propriety and joint propriety to eigenanalysis and canonical correlation analysis respectively our treatment is extended to wss signals in particular we give a result on the asymptotic distribution of eigenvalues and examine the connection between wss proper and analytic signals

complex valued signal processing the proper way to deal with impropriety complex valued signals occur in many areas of science and engineering and are thus of fundamental interest in the past it has often been assumed usually implicitly that complex random signals are proper or circular a proper complex random variable is uncorrelated with its complex conjugate and a circular complex random variable has a probability distribution that is invariant under rotation in the complex plane while these assumptions are convenient because they simplify computations there are many cases where proper and circular random signals are very poor models of the underlying physics when taking impropriety and noncircularity into account the right type of processing can provide significant performance gains there are two key ingredients in the statistical signal processing of complex valued data 1 utilizing the complete statistical characterization of complex valued random signals and 2 the optimization of real valued cost functions with respect to complex parameters in this overview article we review the necessary tools among which are widely linear transformations augmented statistical descriptions and wirtinger calculus we also present some selected recent developments in the field of complex valued signal processing addressing the topics of model selection filtering and source separation

a time domain feedback analysis of filtered error adaptive gradient algorithms this paper provides a time domain feedback analysis of gradient based adaptive schemes a key emphasis is on the robustness performance of the adaptive filters in the presence of disturbances and modeling uncertainties along the lines of h sup spl infin theory and robust filtering the analysis is carried out in a purely deterministic framework and assumes no prior statistical information or independence conditions it is shown that an intrinsic feedback structure can be associated with the varied adaptive schemes the feedback structure is motivated via energy arguments and is shown to consist of two major blocks a time variant lossless i e energy preserving feedforward path and a time variant feedback path the configuration is further shown to lend itself to analysis via a so called small gain theorem thus leading to stability and robustness conditions that require the contractivity of certain operators choices for the step size parameter in order to guarantee faster rates of convergence are also derived and simulation results are included to demonstrate the theoretical findings in addition the time domain analysis provided in this paper is shown to extend the so called transfer function approach to a general time variant scenario without any approximations

proper complex random processes with applications to information theory the covariance of complex random variables and processes when defined consistently with the corresponding notion for real random variables is shown to be determined by the usual complex covariance together with a quantity called the pseudo covariance a characterization of uncorrelatedness and wide sense stationarity in terms of covariance and pseudo covariance is given complex random variables and processes with a vanishing pseudo covariance are called proper it is shown that properness is preserved under affine transformations and that the complex multivariate gaussian density assumes a natural form only for proper random variables the maximum entropy theorem is generalized to the complex multivariate case the differential entropy of a complex random vector with a fixed correlation matrix is shown to be maximum if and only if the random vector is proper gaussian and zero mean the notion of circular stationarity is introduced for the class of proper complex random processes a discrete fourier transform correspondence is derived relating circular stationarity in the time domain to uncorrelatedness in the frequency domain an application of the theory is presented

tracking of time varying mobile radio channels 1 the wiener lms algorithm adaptation algorithms with constant gains are designed for tracking smoothly time varying parameters of linear regression models in particular channel models occurring in mobile radio communications in a companion paper an application to channel tracking in the is 136 tdma system is discussed the proposed algorithms are based on two key concepts first the design is transformed into a wiener filtering problem second the parameters are modeled as correlated arima processes with known dynamics this leads to a new framework for systematic and optimal design of simple adaptation laws based on prior information the algorithms can be realized as wiener filters called learning filters or as lms newton updates complemented by filters that provide predictions or smoothing estimates the simplest algorithm named the wiener lms is presented all parameters are here assumed governed by the same dynamics and the covariance matrix of the regressors is assumed known the computational complexity is of the same order of magnitude as that of lms for regressors which are either white or have autoregressive statistics the tracking performance is however substantially improved

improper signaling for symbol error rate minimization in k user interference channel the rate maximization for the k user interference channels ics has been investigated extensively in the literature however the practical problem of minimizing the error probability with given signal modulations and or data rates of the users is less studied in this paper by utilizing additional degrees of freedom from the improper signaling versus the conventional proper signaling we seek to optimize the precoding matrices for the k user single input single output siso ics to minimize pair wise error probability pep and symbol error rate ser with two proposed algorithms respectively compared with conventional proper signaling and other state of the art improper signaling designs our proposed improper signaling schemes achieve notable error rate improvement in siso ics under both the additive white gaussian noise awgn and cellular system setups with or without channel coding our study provides another viewpoint for optimizing transmissions in ics and further justifies the practical benefit of improper signaling in interference limited communication systems

second order statistics of complex signals the second order statistical properties of complex signals are usually characterized by the covariance function however this is not sufficient for a complete second order description and it is necessary to introduce another moment called the relation function its properties and especially the conditions that it must satisfy are analyzed both for stationary and nonstationary signals this leads to a new perspective concerning the concept of complex white noise as well as the modeling of any signal as the output of a linear system driven by a white noise finally this is applied to complex autoregressive signals and it is shown that the classical prediction problem must be reformulated when the relation function is taken into consideration

receivers with widely linear processing for frequency selective channels we propose several equalization schemes based on widely linear processing wlp the received signal and its complex conjugate are separately filtered and the results are linearly combined it is shown that wlp yields a gain in performance if the noiseless received signal can be interpreted as the convolution of a real valued data sequence and an equivalent complex valued intersymbol interference channel impulse response such a model applies to e g amplitude shift keying offset quadrature amplitude modulation and binary minimum shift keying type modulation we consider receivers without and with decision feedback finite impulse response filters are derived for these structures which are optimum with respect to the zero forcing and minimum mean squared error mmse criteria respectively in the mmse case adaptive algorithms for filter adjustment are given infinite filter orders are investigated in order to obtain analytical performance results furthermore suboptimum trellis based detection with widely linear preprocessing is briefly discussed it is demonstrated analytically and by numerical examples that widely linear schemes may outperform conventional schemes significantly depending on the considered application

partial update lms algorithms partial updating of lms filter coefficients is an effective method for reducing computational load and power consumption in adaptive filter implementations this paper presents an analysis of convergence of the class of sequential partial update lms algorithms s lms under various assumptions and shows that divergence can be prevented by scheduling coefficient updates at random which we call the stochastic partial update lms algorithm spu lms specifically under the standard independence assumptions for wide sense stationary signals the s lms algorithm converges in the mean if the step size parameter spl mu is in the convergent range of ordinary lms relaxing the independence assumption it is shown that s lms and lms algorithms have the same sufficient conditions for exponential stability however there exist nonstationary signals for which the existing algorithms s lms included are unstable and do not converge for any value of spl mu on the other hand under broad conditions the spu lms algorithm remains stable for nonstationary signals expressions for convergence rate and steady state mean square error of spu lms are derived the theoretical results of this paper are validated and compared by simulation through numerical examples

exact expectation analysis of the lms adaptive filter in almost all analyses of the least mean square lms adaptive filter it is assumed that the filter coefficients are statistically independent of the input data currently in filter memory an assumption that is incorrect for shift input data we present a method for deriving a set of linear update equations that can be used to predict the exact statistical behavior of a finite impulse response fir lms adaptive filter operating upon finite time correlated input data using our method we can derive exact bounds upon the lms step size to guarantee mean and mean square convergence our equation deriving procedure is recursive and algorithmic and we describe a program written in the maple symbolic manipulation software package that automates the derivation for arbitrarily long adaptive filters operating on input data with stationary statistics using our analysis we present a search algorithm that determines the exact step size mean square stability bound for a given filter length and input correlation statistics extensive computer simulations indicate that the exact analysis is more accurate than previous analyses in predicting adaptation behavior our results also indicate that the exact step size bound is much more stringent than the bound predicted by the independence assumption analysis for correlated input data

transmit optimization with improper gaussian signaling for interference channels this paper studies the achievable rates of gaussian interference channels with additive white gaussian noise awgn when improper or circularly asymmetric complex gaussian signaling is applied for the gaussian multiple input multiple output interference channel mimo ic with the interference treated as gaussian noise we show that the user s achievable rate can be expressed as a summation of the rate achievable by the conventional proper or circularly symmetric complex gaussian signaling in terms of the users transmit covariance matrices and an additional term which is a function of both the users transmit covariance and pseudo covariance matrices the additional degrees of freedom in the pseudo covariance matrix which is conventionally set to be zero for the case of proper gaussian signaling provide an opportunity to further improve the achievable rates of gaussian mimo ics by employing improper gaussian signaling to this end this paper proposes widely linear precoding which efficiently maps proper information bearing signals to improper transmitted signals at each transmitter for any given pair of transmit covariance and pseudo covariance matrices in particular for the case of two user gaussian single input single output interference channel siso ic we propose a joint covariance and pseudo covariance optimization algorithm with improper gaussian signaling to achieve the pareto optimal rates by utilizing the separable structure of the achievable rate expression an alternative algorithm with separate covariance and pseudo covariance optimization is also proposed which guarantees the rate improvement over conventional proper gaussian signaling

on approximate diagonalization of correlation matrices in widely linear signal processing the so called augmented statistics of complex random variables has established that both the covariance and pseudocovariance are necessary to fully describe second order properties of noncircular complex signals to jointly decorrelate the covariance and pseudocovariance matrix the recently proposed strong uncorrelating transform sut requires two singular value decompositions svds in this correspondence we further illuminate the structure of these matrices and demonstrate that for univariate noncircular data it is sufficient to diagonalize the pseudocovariance matrix this ensures that the covariance matrix is also approximately diagonal the proposed approach is shown to result in lower computational complexity and enhanced numerical stability and to enable elegant new formulations of performance bounds in widely linear signal processing the analysis is supported by illustrative case studies and simulation examples

forbidden information godel incompleteness theorem leaves open a way around it vaguely perceived for a long time but not clearly identified thus godel believed informal arguments can answer any math question closing this loophole does not seem obvious and involves kolmogorov complexity this is unrelated to well studied before complexity quantifications of the usual godel effects i consider extensions u of the universal partial recursive predicate or say peano arithmetic i prove that any u either leaves an n bit input statement unresolved or contains nearly all information about the n bit prefix of any r e real which is n bits for some i argue that creating significant information about a specific math sequence is impossible regardless of the methods used similar problems and answers apply to other unsolvability results for tasks allowing multiple solutions for example nonrecursive tilings

performance analysis of the deficient length lms adaptive algorithm in almost all analyzes of the least mean square lms finite impulse response fir adaptive algorithm it is assumed that the length of the adaptive filter is equal to that of the unknown system impulse response however in many practical situations a deficient length adaptive filter whose length is less than that of the unknown system is employed and analysis results for the sufficient length lms algorithm are not necessarily applicable to the deficient length case therefore there is an essential need to accurately quantify the behavior of the lms algorithm for realistic situations where the length of the adaptive filter is deficient in this paper we present a performance analysis for the deficient length lms adaptive algorithm for correlated gaussian input data and using the common independence assumption exact expressions that completely characterize the transient and steady state mean square performances of the algorithm are developed which lead to new insights into the statistical behavior of the deficient length lms algorithm simulation experiments illustrate the accuracy of the theoretical results in predicting the convergence behavior of the algorithm

convergence analysis of lms filters with uncorrelated gaussian data statistical analysis of the least mean squares lms adaptive algorithm with uncorrelated gaussian data is presented exact analytical expressions for the steady state mean square error mse and the performance degradation due to weight vector misadjustment are derived necessary and sufficient conditions for the convergence of the algorithm to the optimal wiener solution within a finite variance are derived it is found that the adaptive coefficient which controls the rate of convergence of the algorithm must be restricted to an interval significantly smaller than the domain commonly stated in the literature the outcome of this paper therefore places fundamental limitations on the mse performance and rate of convergence of the lms adaptive scheme

on circularity circularity is an assumption that was originally introduced for the definition of the probability distribution function of complex normal vectors however this concept can be extended in various ways for nonnormal vectors the first purpose of the paper is to introduce and compare some possible definitions of circularity from these definitions it is also possible to introduce the concept of circular signals and to study whether or not the spectral representation of stationary signals introduces circular components therefore the relationships between circularity and stationarity are analyzed in detail finally the theory of linear mean square estimation for complex signals exhibits some connections with circularity and it is shown that without this assumption the estimation theory must be reformulated

performance analysis of deficient length quaternion least mean square adaptive filters quaternion adaptive filters have been widely used for processing 3d and 4d phenomena deficient length quaternion adaptive filters are explicitly or implicitly used in many practical applications where the length of system impulse response is large or unknown however their statistical behaviors are yet to be fully understood as theoretical results on the class of full length quaternion least mean square qlms algorithms do not necessarily apply to their deficient length versions this article fills this void and analyses the mean and mean square convergence of the deficient length qlms algorithms both for the strictly linear and widely linear cases transient and steady state performance is characterised by exploiting the augmented statistics of noncircular quaternion random vectors a novel decorrelation technique in the quaternion domain is shown to allow for the development of intuitive closed form solutions for correlated quaternion gaussian inputs thus unveiling the relationship between the algorithm behaviour and the noncircularity of quaternion input data the analysis also provides a general framework whereby the strictly linear and semi widely linear qlms algorithms can be seen as deficient length versions of the widely linear qlms numerical simulations validate the accuracy of the theoretical results and support the behaviour of the considered algorithms

complex valued filtering based on the minimization of complex error entropy in this paper we consider the training of complex valued filter based on the information theoretic method we first generalize the error entropy criterion to complex domain to present the complex error entropy criterion ceec due to the difficulty in estimating the entropy of complex valued error directly the entropy bound minimization ebm method is used to compute the upper bounds of the entropy of the complex valued error and the tightest bound selected by the ebm algorithm is used as the estimator of the complex error entropy then based on the minimization of complex error entropy mcee and the complex gradient descent approach complex valued learning algorithms for both the linear transverse filter and the nonlinear neural network are derived the algorithms are applied to complex valued linear filtering and complex valued nonlinear channel equalization to demonstrate their effectiveness and advantages

a full mean square analysis of clms for second order noncircular inputs a full mean square transient and steady state analysis of the complex least mean square clms algorithm is provided for strictly linear estimation of general second order noncircular improper gaussian inputs to this end we also consider the performance assessment in terms of the evolution of the complementary mean square error cmse and the complementary covariance pseudocovariance matrix of the weight error vector of clms this makes it possible to measure the degrees of noncircularity of the output error and the weight error vector which arise due to second order noncircularity improperness of the system input and system noise the recently introduced approximate uncorrelating transform which allows for joint direct diagonalization of both the input covariance and complementary covariance matrices with a single singular value decomposition is then employed in order to derive a unified bound on the step size which guarantees the convergence of both the standard mse and the proposed cmse a joint consideration of the standard mean square performance analysis and the proposed complementary performance analysis is shown to provide full second order closed form statistical descriptions of both the transient and steady state performances of clms for second order noncircular improper gaussian input data simulations in the system identification setting support the analysis

a unified approach to the steady state and tracking analyses of adaptive filters most adaptive filters are inherently nonlinear and time variant systems the nonlinearities in the update equations tend to lead to difficulties in the study of their steady state performance as a limiting case of their transient performance this paper develops a unified approach to the steady state and tracking analyses of adaptive algorithms that bypasses many of these difficulties the approach is based on the study of the energy flow through each iteration of an adaptive filter and it relies on a fundamental error variance relation

analysis of stability and performance of adaptation algorithms with time invariant gains adaptation laws that track parameters of linear regression models are investigated the considered class of algorithms apply linear time invariant filtering on the instantaneous gradient vector and includes least mean squares lms as its simplest member the asymptotic stability and steady state tracking performance for prediction and smoothing estimators is analyzed for parameter variations described by stochastic processes with time invariant statistics the analysis is based on a novel technique that decomposes the inherent feedback of adaptation algorithms into one time invariant loop and one time varying loop the impact of the time varying feedback on the tracking error covariance can be neglected under certain conditions and the performance analysis then becomes straightforward performance analysis in the presence of a non negligible time varying feedback is performed for algorithms that use scalar measurements convergence in mean square error mse and the mse tracking performance is investigated assuming independent consecutive regression vectors closed form expressions for the tracking mse are thereafter derived without this independence assumption for a subclass of algorithms applied to finite impulse response fir models with white inputs this class includes wiener lms adaptation

second order complex random vectors and normal distributions complex random vectors are usually described by their covariance matrix this is insufficient for a complete description of second order statistics and another matrix called the relation matrix is necessary some of its properties are analyzed and used to express the probability density function of normal complex vectors various consequences are presented

on the convergence behavior of the lms and the normalized lms algorithms it is shown that the normalized least mean square nlms algorithm is a potentially faster converging algorithm compared to the lms algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics a very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the lms and nlms algorithms is proposed using this model answers can be obtained to questions for which no answers are currently available using other perhaps more realistic models examples are given to illustrate that even quantitatively the answers obtained can be good approximations it is emphasized that the convergence of the nlms algorithm can be speeded up significantly by employing a time varying step size the optimal step size sequence can be specified a priori for the case of a white input signal with arbitrary distribution

complex random vectors and ica models identifiability uniqueness and separability in this paper the conditions for identifiability separability and uniqueness of linear complex valued independent component analysis ica models are established these results extend the well known conditions for solving real valued ica problems to complex valued models relevant properties of complex random vectors are described in order to extend the darmois skitovich theorem for complex valued models this theorem is used to construct a proof of a theorem for each of the above ica model concepts both circular and noncircular complex random vectors are covered examples clarifying the above concepts are presented

complex valued linear and widely linear filtering using mse and gaussian entropy in this paper we study the performance of mean square error mse and gaussian entropy criteria for linear and widely linear complex filtering the mse criterion has been extensively studied and with a widely linear filter form it can take into account the full second order statistics of the input signal however it cannot exploit the full second order statistics of the error and doubles the dimension of the parameter vector to be estimated in this paper we introduce the use of gaussian entropy criterion such that full second order statistics of the error can be taken into account and compare the performance of the gaussian entropy and mse criteria for a linear and widely linear filter implementation in batch and adaptive implementations detailed performance analysis with numerical examples is presented to investigate the relationship and performance differences of the two criteria in diverse scenarios