multiscale codes in the nervous system the problem of noise correlations and the ambiguity of periodic scales encoding information about continuous variables using noisy computational units is a challenge nonetheless asymptotic theory shows that combining multiple periodic scales for coding can be highly precise despite the corrupting influence of noise mathis herz and stemmler phys rev lett 109 018103 2012 indeed the cortex seems to use periodic multiscale grid codes to represent position accurately here we show how such codes can be read out without taking the long term limit even on short time scales the precision of such codes scales exponentially in the number n of neurons does this finding also hold for neurons that are not firing in a statistically independent fashion to assess the extent to which biological grid codes are subject to statistical dependences we first analyze the noise correlations between pairs of grid code neurons in behaving rodents we find that if the grids of two neurons align and have the same length scale the noise correlations between the neurons can reach values as high as 0 8 for increasing mismatches between the grids of the two neurons the noise correlations fall rapidly incorporating such correlations into a population coding model reveals that the correlations lessen the resolution but the exponential scaling of resolution with n is unaffected

representational accuracy of stochastic neural populations fisher information is used to analyze the accuracy with which a neural population encodes d stimulus features it turns out that the form of response variability has a major impact on the encoding capacity and therefore plays an important role in the selection of an appropriate neural model in particular in the presence of baseline firing the reconstruction error rapidly increases with d in the case of poissonian noise but not for additive noise the existence of limited range correlations of the type found in cortical tissue yields a saturation of the fisher information content as a function of the population size only for an additive noise model we also show that random variability in the correlation coefficient within a neural population as found empirically considerably improves the average encoding quality finally the representational accuracy of populations with inhomogeneous tuning properties either with variability in the tuning widths or fragmented into specialized subpopulations is superior to the case of identical and radially symmetric tuning curves usually considered in the literature

a feedback approach to the steady state performance of fractionally spaced blind adaptive equalizers this paper proposes a new approach to the analysis of the steady state performance of constant modulus algorithms cma which are among the most popular adaptive schemes for blind equalization a major feature of the proposed feedback approach is that it bypasses the need for working directly with the weight error covariance matrix in so doing approximate expressions for the steady state mean square error of several cm algorithms are derived including cma2 2 cma1 2 normalized cma and a new normalized cma variant with less bias a comparison among the various algorithms is also performed along with several simulation results the conclusions confirm the superior performance of cma2 2

steady state mse performance analysis of mixture approaches to adaptive filtering in this paper we consider mixture approaches that adaptively combine outputs of several parallel running adaptive algorithms these parallel units can be considered as diversity branches that can be exploited to improve the overall performance we study various mixture structures where the final output is constructed as the weighted linear combination of the outputs of several constituent filters although the mixture structure is linear the combination weights can be updated in a highly nonlinear manner to minimize the final estimation error such as in singer and feder 1999 arenas garcia figueiras vidal and sayed 2006 lopes satorius and sayed 2006 bershad bermudez and tourneret 2008 and silva and nascimento 2008 we distinguish mixture approaches that are convex combinations where the linear mixture weights are constrained to be nonnegative and sum up to one singer and feder 1999 arenas garcia figueiras vidal and sayed 2006 affine combinations where the linear mixture weights are constrained to sum up to one bershad bermudez and tourneret 2008 and finally unconstrained linear combinations of constituent filters kozat and singer 2000 we investigate mixture structures with respect to their final mean square error mse and tracking performance in the steady state for stationary and certain nonstationary data respectively we demonstrate that these mixture approaches can greatly improve over the performance of the constituent filters our analysis is also generic such that it can be applied to inhomogeneous mixtures of constituent adaptive branches with possibly different structures adaptation methods or having different filter lengths

reduced complexity self tuning adaptive algorithms in application to channel estimation in this letter reduced complexity self tuning algorithms are proposed using simplified parameter updating procedures convergence analysis based on the independence assumption and the ordinary differential equation ode method shows that the tuning parameter of the proposed algorithm attains the same limit as the conventional self tuning adaptive algorithm simulations are carried out for channel estimation to support the analysis and performance of the proposed algorithms

tracking analysis of the sign algorithm in nonstationary environments a tracking analysis of the adaptive filters equipped with the sign algorithm and operating in nonstationary environments is presented under the assumption that the nonstationary can be modeled using a random disturbance it is shown that the long term time average of the mean absolute error is bounded and that there exists an optimal choice of the convergence constant mu which minimizes this quality applying the commonly used independence assumption and under the assumption that the nonstationarity is solely due to the time varying behavior of the optimal coefficients it is shown that the distributions of the successive coefficient misalignment vectors converge to a limiting distribution when the adaptive filter is used in the system identification mode finally under the additional assumption that the signals involved are zero mean and gaussian a set of nonlinear difference equations is derived that characterizes the mean and mean squared behavior of the filter coefficients and the mean squared estimation error during adaptation and tracking results of several experiments that show very good correlation with the theoretical analyses are presented

convergence analysis of the sign algorithm without the independence and gaussian assumptions the paper is concerned with rigorous convergence analysis of the sign algorithm sa in the context of adaptive plant identification asymptotic time averaged convergence for the mean absolute weight misalignment is proved for all values of the algorithm step size and initial weight vector the paper has three main contributions with respect to available convergence results of the sa the first is the deletion of the gaussian assumption which is important when covering the case of discrete valued data no assumption about the distribution of the regressor sequence is used except for the usual assumption of positive definite covariance matrix the assumptions used about the noise allow nonexistence unboundedness and vanishing of the noise probability density function for arguments strictly different from zero the second contribution is the deletion of the assumption of independent successive regressors this deletion is important since in applications two successive regressors usually share all their components except two hence they are strongly dependent even for white plant input the case of colored noise is also analyzed finally the third contribution is the extension of the above results to the nonstationary case the used assumptions allow nonstationarity of the plant input plant noise and plant parameters

transient and steady state analysis of the affine combination of two adaptive filters in this paper we propose an approach to the transient and steady state analysis of the affine combination of one fast and one slow adaptive filters the theoretical models are based on expressions for the excess mean square error emse and cross emse of the component filters which allows their application to different combinations of algorithms such as least mean squares lms normalized lms nlms and constant modulus algorithm cma considering white or colored inputs and stationary or nonstationary environments since the desired universal behavior of the combination depends on the correct estimation of the mixing parameter at every instant its adaptation is also taken into account in the transient analysis furthermore we propose normalized algorithms for the adaptation of the mixing parameter that exhibit good performance good agreement between analysis and simulation results is always observed

improving the tracking capability of adaptive filters via convex combination as is well known hessian based adaptive filters such as the recursive least squares algorithm rls for supervised adaptive filtering or the shalvi weinstein algorithm swa for blind equalization converge much faster than gradient based algorithms such as the least mean squares algorithm lms or the constant modulus algorithm cma however when the problem is tracking a time variant filter the issue is not so clear cut there are environments for which each family presents better performance given this we propose the use of a convex combination of algorithms of different families to obtain an algorithm with superior tracking capability we show the potential of this combination and provide a unified theoretical model for the steady state excess mean square error for convex combinations of gradient and hessian based algorithms assuming a random walk model for the parameter variations the proposed model is valid for algorithms of the same or different families and for supervised lms and rls or blind cma and swa algorithms

accuracy of rate coding when shorter time window and higher spontaneous activity help it is widely accepted that neuronal firing rates contain a significant amount of information about the stimulus intensity nevertheless theoretical studies on the coding accuracy inferred from the exact spike counting distributions are rare we present an analysis based on the number of observed spikes assuming the stochastic perfect integrate and fire model with a change point representing the stimulus onset for which we calculate the corresponding fisher information to investigate the accuracy of rate coding we analyze the effect of changing the duration of the time window and the influence of several parameters of the model in particular the level of the presynaptic spontaneous activity and the level of random fluctuation of the membrane potential which can be interpreted as noise of the system the results show that the fisher information is nonmonotonic with respect to the length of the observation period this counterintuitive result is caused by the discrete nature of the count of spikes we observe also that the signal can be enhanced by noise since the fisher information is nonmonotonic with respect to the level of spontaneous activity and in some cases also with respect to the level of fluctuation of the membrane potential

tracking properties and steady state performance of rls adaptive filter algorithms adaptive signal processing algorithms derived from ls least squares cost functions are known to converge extremely fast and have excellent capabilities to track an unknown parameter vector this paper treats analytically and experimentally the steady state operation of rls recursive least squares adaptive filters with exponential windows for stationary and nonstationary inputs a new formula for the estimation noise has been derived involving second and fourth order statistics of the filter input as well as the exponential windowing factor and filter length furthermore it is shown that the adaptation process associated with lag effects depends solely on the exponential weighting parameter in addition the calculation of the excess mean square error due to the lag for an assumed markov channel provides the necessary information about tradeoffs between speed of adaptation and steady state error it is also the basis for comparison to the simple lms algorithm in a simple case of channel identification it is shown that the lms and rls adaptive filters have the same tracking behavior finally in the last part we present new rls restart procedures applied to transversal structures for mitigating the disastrous results of the third source of noise namely finite precision arithmetic

transient and tracking performance bounds of the sign sign algorithm the paper provides a rigorous tracking analysis of the sign sign algorithm when used in the identification of a time varying plant with a white gaussian input the plant parameters vary according to a random walk model the assumptions allow nonstationarity of the plant input plant noise and increments of the plant parameters upper bounds are derived for the long term averages of the mean of the weight misalignment norm mean absolute error mean square weight misalignment and mean square error these bounds hold for all values of the algorithm step size all initial filter weight settings and all degrees of nonstationarity of the plant input plant noise and plant parameter increments lower bounds of the mean square weight misalignment and mean square error are also derived the step sizes that minimize the above bounds are derived a transient analysis of the algorithm is done in the case of a time invariant plant a tight lower bound of the convergence time is derived the above analytical results are supported by computer simulations

optimal population codes for space grid cells outperform place cells rodents use two distinct neuronal coordinate systems to estimate their position place fields in the hippocampus and grid fields in the entorhinal cortex whereas place cells spike at only one particular spatial location grid cells fire at multiple sites that correspond to the points of an imaginary hexagonal lattice we study how to best construct place and grid codes taking the probabilistic nature of neural spiking into account which spatial encoding properties of individual neurons confer the highest resolution when decoding the animal s position from the neuronal population response a priori estimating a spatial position from a grid code could be ambiguous as regular periodic lattices possess translational symmetry the solution to this problem requires lattices for grid cells with different spacings the spatial resolution crucially depends on choosing the right ratios of these spacings across the population we compute the expected error in estimating the position in both the asymptotic limit using fisher information and for low spike counts using maximum likelihood estimation achieving high spatial resolution and covering a large range of space in a grid code leads to a trade off the best grid code for spatial resolution is built of nested modules with different spatial periods one inside the other whereas maximizing the spatial range requires distinct spatial periods that are pairwisely incommensurate optimizing the spatial resolution predicts two grid cell properties that have been experimentally observed first short lattice spacings should outnumber long lattice spacings second the grid code should be self similar across different lattice spacings so that the grid field always covers a fixed fraction of the lattice period if these conditions are satisfied and the spatial tuning curves for each neuron span the same range of firing rates then the resolution of the grid code easily exceeds that of the best possible place code with the same number of neurons

multidimensional encoding strategy of spiking neurons neural responses in sensory systems are typically triggered by a multitude of stimulus features using information theory we study the encoding accuracy of a population of stochastically spiking neurons characterized by different tuning widths for the different features the optimal encoding strategy for representing one feature most accurately consists of narrow tuning in the dimension to be encoded to increase the single neuron fisher information and broad tuning in all other dimensions to increase the number of active neurons extremely narrow tuning without sufficient receptive field overlap will severely worsen the coding this implies the existence of an optimal tuning width for the feature to be encoded empirically only a subset of all stimulus features will normally be accessible in this case relative encoding errors can be calculated that yield a criterion for the function of a neural population based on the measured tuning curves

computing with continuous attractors stability and online aspects two issues concerning the application of continuous attractors in neural systems are investigated the computational robustness of continuous attractors with respect to input noises and the implementation of bayesian online decoding in a perfect mathematical model for continuous attractors decoding results for stimuli are highly sensitive to input noises and this sensitivity is the inevitable consequence of the system s neutral stability to overcome this shortcoming we modify the conventional network model by including extra dynamical interactions between neurons these interactions vary according to the biologically plausible hebbian learning rule and have the computational role of memorizing and propagating stimulus information accumulated with time as a result the new network model responds to the history of external inputs over a period of time and hence becomes insensitive to short term fluctuations also since dynamical interactions provide a mechanism to convey the prior knowledge of stimulus that is the information of the stimulus presented previously the network effectively implements online bayesian inference this study also reveals some interesting behavior in neural population coding such as the trade off between decoding stability and the speed of tracking time varying stimuli and the relationship between neural tuning width and the tracking speed

comparison of rls lms and sign algorithms for tracking randomly time varying channels the performance of adaptive fir filters governed by the recursive least squares rls algorithm the least mean square lms algorithm and the sign algorithm sa are compared when the optimal filtering vector is randomly time varying the comparison is done in terms of the steady state excess mean square estimation error spl xi and the steady state mean square weight deviation spl eta it is shown that spl xi does not depend on the spread of eigenvalues of the input covariance matrix r in the cases of the lms algorithm and the sa while it does in the case of the rls algorithm in the three algorithms spl eta is found to be increasing with the eigenvalue spread the value of the adaptation parameter that minimizes spl xi is different from the one that minimizes spl eta it is shown that the minimum values of spl xi and spl eta attained by the rls algorithm are equal to the ones attained by the lms algorithm in any one of the three following cases 1 if r has equal eigenvalues 2 if the fluctuations of the individual elements of the optimal vector are mutually uncorrelated and have the same mean square value or 3 if r is diagonal and the fluctuations of the individual elements of the optimal vector have the same mean square value conditions that make the values of spl xi and spl eta of the lms algorithm smaller or greater than the ones of the rls algorithm are derived for gaussian input data the minimum values of spl xi and spl eta attained by the sa are found to exceed the ones attained by the lms algorithm by 1 db independently of r and the mutual correlation between the elements of the optimal vector

gradient based variable forgetting factor rls algorithm in time varying environments in this paper a new control mechanism for the variable forgetting factor vff of the recursive least square rls adaptive algorithm is presented the control algorithm is basically a gradient based method of which the gradient is derived from an improved mean square error analysis of rls the new mean square error analysis exploits the correlation of the inverse of the correlation matrix with itself that yields improved theoretical results especially in the transient and steady state mean square error it is shown that the theoretical analysis is close to simulation results for different forgetting factors and different model orders the analysis yields a dynamic equation of mean square error that can be used to derive a dynamic equation of the gradient of mean square error to control the forgetting factor the dynamic equation can produce a positive gradient when the error is large and a negative gradient when the error is in the steady state compared with other variable forgetting factor algorithms the new control algorithm gives fast tracking and small mean square model error for different signal to noise ratios snrs

efficient neural codes that minimize lp reconstruction error the efficient coding hypothesis assumes that biological sensory systems use neural codes that are optimized to best possibly represent the stimuli that occur in their environment most common models use information theoretic measures whereas alternative formulations propose incorporating downstream decoding performance here we provide a systematic evaluation of different optimality criteria using a parametric formulation of the efficient coding problem based on the reconstruction error of the maximum likelihood decoder this parametric family includes both the information maximization criterion and squared decoding error as special cases we analytically derived the optimal tuning curve of a single neuron encoding a one dimensional stimulus with an arbitrary input distribution we show how the result can be generalized to a class of neural populations by introducing the concept of a meta tuning curve the predictions of our framework are tested against previously measured characteristics of some early visual systems found in biology we find solutions that correspond to low values of suggesting that across different animal models neural representations in the early visual pathways optimize similar criteria about natural stimuli that are relatively close to the information maximization criterion

optimal signal estimation in neuronal models we study optimal estimation of a signal in parametric neuronal models on the basis of interspike interval data fisher information is the inverse asymptotic variance of the best estimator its dependence on the parameter value indicates accuracy of estimation our models assume that the input signal is estimated from neuronal output interspike interval data where the frequency transfer function is sigmoidal if the coefficient of variation of the interspike interval is constant with respect to the signal the fisher information is unimodal and its maximum for the most estimable signal can be found we obtain a general result and compare the signal producing maximal fisher information with the inflection point of the sigmoidal transfer function in several basic neuronal models

low complexity variable forgetting factor mechanism for blind adaptive constrained constant modulus algorithms in this paper we propose a novel low complexity variable forgetting factor vff mechanism for blind adaptive constrained constant modulus ccm recursive least squares rls algorithms applied to linear interference suppression in direct sequence code division multiple access ds cdma systems the proposed variable forgetting factor mechanism employs an updated component related to the time average of the constant modulus cm cost function to automatically adjust the forgetting factor in order to ensure good tracking of the interference and the channel convergence and tracking analyses are carried out analytical expressions for predicting the mean squared error of the proposed adaptation technique are obtained simulation results are presented for nonstationary environments and show that the proposed variable forgetting factor mechanism achieves superior performance to previously reported methods at a reduced complexity

neural population coding is optimized by discrete tuning curves the sigmoidal tuning curve that maximizes the mutual information for a poisson neuron or population of poisson neurons is obtained the optimal tuning curve is found to have a discrete structure that results in a quantization of the input signal the number of quantization levels undergoes a hierarchy of phase transitions as the length of the coding window is varied we postulate using the mammalian auditory system as an example that the presence of a subpopulation structure within a neural population is consistent with an optimal neural code

optimization of adaptive identification for time varying filters adaptive identification in a time varying context is studied when controlled by the lms algorithm with constant gain under the assumption of correlated successive input vectors it is well known by experience that the tracking mean square error mse epsilon mu results from the tradeoff between the gradient part which is increasing and the lag contribution which is decreasing in this note we clarify the relative roles of the gradient and lag errors by proving their decoupled character this property relies upon independence between the additive noise at the output of the plant to be identified and the information vector at the plant input convergence of the mse is established rather than assumed quantitative evaluations of upper and lower bounds allow an approximate optimization of the gain in two important cases the optimum is exact one of these cases is slow variations it is defined in a quantitative manner thanks to the ratio of the variation noise to the output additive noise

an improved approximation algorithm for capacitated multicast routings in networks let g v e be a connected graph such that each edge email protected e is weighted by nonnegative real w e let s be a vertex designated as a source k be a positive integer and email protected v be a set of terminals the capacitated multicast tree routing problem cmtr asks to find a partition z 1 z 2 z of s and a set t 1 t 2 t of trees of g such that z i consists of at most k terminals and each t i spans z email protected s the objective is to minimize i 1 w t i where w t i denotes the sum of weights of all edges in t i in this paper we propose a 3 2 4 3 r approximation algorithm to the cmtr where r is the best achievable approximation ratio for the steiner tree problem

mean square performance of a family of affine projection algorithms affine projection algorithms are useful adaptive filters whose main purpose is to speed the convergence of lms type filters most analytical results on affine projection algorithms assume special regression models or gaussian regression data the available analysis also treat different affine projection filters separately this paper provides a unified treatment of the mean square error tracking and transient performances of a family of affine projection algorithms the treatment relies on energy conservation arguments and does not restrict the regressors to specific models or to a gaussian distribution simulation results illustrate the analysis and the derived performance expressions

optimal short term population coding when fisher information fails efficient coding has been proposed as a first principle explaining neuronal response properties in the central nervous system the shape of optimal codes however strongly depends on the natural limitations of the particular physical system here we investigate how optimal neuronal encoding strategies are influenced by the finite number of neurons n place constraint the limited decoding time window length t time constraint the maximum neuronal firing rate fmax power constraint and the maximal average rate f max energy constraint while fisher information provides a general lower bound for the mean squared error of unbiased signal reconstruction its use to characterize the coding precision is limited analyzing simple examples we illustrate some typical pitfalls and thereby show that fisher information provides a valid measure for the precision of a code only if the dynamic range fmint fmaxt is sufficiently large in particular we demonstrate that the optimal width of gaussian tuning curves depends on the available decoding time t within the broader class of unimodal tuning functions it turns out that the shape of a fisher optimal coding scheme is not unique we solve this ambiguity by taking the minimum mean square error into account which leads to fiat tuning curves the tuning width however remains to be determined by energy constraints rather than by the principle of efficient coding

analytical performance of the lms algorithm on the estimation of wide sense stationary channels the performance of the least mean square lms algorithm on the estimation of time varying channels is analytically evaluated using the estimation error correlation matrix the mean square weight error mswe and the mean square estimation error mse as parameters expressions for those parameters are obtained from a set of hypotheses usually adopted in the communication systems context the channel is modeled as a wide sense stationary wss discrete time stochastic field with known autocorrelation the expressions for the steady state mswe and mse are particularized for the class of wss channel models and an original analysis of the optimum lms step size parameter for usual channel models is addressed for the sake of comparison with other works the analytical step size optimization for random walk models is also considered several estimates of mswe curves obtained by computer simulation are compared with analytical results for validation purposes a very good agreement between simulated and analytical results for both the mswe expressions and the optimum value of the lms step size parameter is shown

convergence analysis of adaptive filtering algorithms with singular data covariance matrix the paper provides a rigorous analysis of the behavior of adaptive filtering algorithms when the covariance matrix of the filter input is singular the analysis is done in the context of adaptive plant identification the considered algorithms are lms rls sign sa and signed regressor sra algorithms both the signal and weight behavior of the algorithms are considered the signal behavior is evaluated in terms of the moments of the excess output error of the filter the weight behavior is evaluated in terms of the moments of the filter weight misalignment vector it is found that the rls and sra diverge when the input covariance matrix is singular the steady state signal behavior of the lms and sa can be made arbitrarily fine by using sufficiently small step sizes of the algorithms indeed the long term average of the mean square excess error of the lms is proportional to the algorithm step size the long term average of the mean absolute excess error of the sa is proportional to the square root of the algorithm step size on the other hand the steady state weight behavior of both the lms and sa have biases that depend on the weight initialization the analytical results of the paper are supported by simulations

shrinkage linear and widely linear complex valued least mean squares algorithms for adaptive beamforming in this paper shrinkage linear complex valued least mean squares sl clms and shrinkage widely linear complex valued least mean squares swl clms algorithms are devised for adaptive beamforming by exploiting the relationship between the noise free a posteriori and a priori error signals the sl clms method is able to provide a variable step size to update the weight vector for the adaptive beamformer significantly enhancing the convergence speed and decreasing the steady state misadjustment on the other hand besides adopting a variable step size determined by minimizing the square of the augmented noise free a posteriori errors the swl clms approach exploits the noncircular properties of the signal of interest which considerably improves the steady state performance simulation results are presented to illustrate their superiority over the clms complex valued normalized lms variable step size recursive least squares rls algorithms and their corresponding widely linear based schemes additionally our proposed algorithms are more computationally efficient than the rls solutions though they may have a slightly slower convergence rate

parameter extraction from population codes a critical assessment in perceptual systems a stimulus parameter can be extracted by determining the center of gravity of the response profile of a population of neural sensors likewise at the motor end of a neural system center of gravity decoding also known as vector decoding generates a movement direction from the neural activation profile we evaluate these schemes from a statistical perspective by comparing their statistical variance with the minimum variance possible for an unbiased parameter extraction from the noisy neuronal ensemble activation profile center of gravity decoding can be statistically optimal this is the case for regular arrays of sensors with gaussian tuning profiles that have an output described by poisson statistics and for arrays of sensors with a sinusoidal tuning profile for the angular parameter estimated however there are also many cases in which center of gravity decoding is highly inefficient this includes the important case where sensor positions are very irregular finally we study the robustness of center of gravity decoding against response nonlinearities at different stages of an information processing hierarchy we conclude that in neural systems instead of representing a parameter explicitly it is safer to leave the parameter coded implicitly in a neuronal ensemble activation profile

fisher and shannon information in finite neural populations the precision of the neural code is commonly investigated using two families of statistical measures shannon mutual information and derived quantities when investigating very small populations of neurons and fisher information when studying large populations these statistical tools are no longer the preserve of theorists and are being applied by experimental research groups in the analysis of empirical data although the relationship between information theoretic and fisher based measures in the limit of infinite populations is relatively well understood how these measures compare in finite size populations has not yet been systematically explored we aim to close this gap we are particularly interested in understanding which stimuli are best encoded by a given neuron within a population and how this depends on the chosen measure we use a novel monte carlo approach to compute a stimulus specific decomposition of the mutual information the ssi for populations of up to 256 neurons and show that fisher information can be used to accurately estimate both mutual information and ssi for populations of the order of 100 neurons even in the presence of biologically realistic variability noise correlations and experimentally relevant integration times according to both measures the stimuli that are best encoded are those falling at the flanks of the neuron s tuning curve in populations of fewer than around 50 neurons however fisher information can be misleading

optimal stimulus and noise distributions for information transmission via suprathreshold stochastic resonance suprathreshold stochastic resonance ssr is a form of noise enhanced signal transmission that occurs in a parallel array of independently noisy identical threshold nonlinearities including model neurons unlike most forms of stochastic resonance the output response to suprathreshold random input signals of arbitrary magnitude is improved by the presence of even small amounts of noise in this paper the information transmission performance of ssr in the limit of a large array size is considered using a relationship between shannon s mutual information and fisher information a sufficient condition for optimality i e channel capacity is derived it is shown that capacity is achieved when the signal distribution is jeffrey s prior as formed from the noise distribution or when the noise distribution depends on the signal distribution via a cosine relationship these results provide theoretical verification and justification for previous work in both computational neuroscience and electronics

tracking analysis of the sign sign algorithm for nonstationary adaptive filtering with gaussian data this correspondence is concerned with the analysis of the sign sign algorithm ssa when used to track a plant with randomly time varying parameters the input of the plant and the plant noise are assumed stationary and gaussian the paper derives expressions of the steady state excess mean square error spl xi the steady state mean square weight misalignment spl eta and the step sizes that minimize each one of them the paper also presents a comparison among the tracking properties of the ssa the sign algorithm sa and the signed regressor algorithm sra it is found that the three algorithms share the features that 1 spl xi does not depend on the eigenvalue spread of the input covariance matrix 2 spl eta depends on the eigenvalue spread and 3 the step size that minimizes spl xi is different from the one that minimizes spl eta the minimum values of spl xi attained by the sa and the sra are equal to each other and they are 1 db less than the one attained by the ssa the ratio of the minimum value of spl eta of the ssa to the one of the sa is found to be dependent on the input eigenvalue spread for equal eigenvalues this ratio is equal to 1 db the minimum value of spl eta of the ssa is found to be 1 db higher than the one of the sra independently of the input eigenvalue spread it is found that an advantage of the ssa with respect to both the sa and the sra is that the two optimum step sizes of the ssa are independent of the mean square plant input and the mean square plant noise

a unified approach to the steady state and tracking analyses of adaptive filters most adaptive filters are inherently nonlinear and time variant systems the nonlinearities in the update equations tend to lead to difficulties in the study of their steady state performance as a limiting case of their transient performance this paper develops a unified approach to the steady state and tracking analyses of adaptive algorithms that bypasses many of these difficulties the approach is based on the study of the energy flow through each iteration of an adaptive filter and it relies on a fundamental error variance relation

transient analysis of adaptive filters with error nonlinearities the paper develops a unified approach to the transient analysis of adaptive filters with error nonlinearities in addition to deriving earlier results in a unified manner the approach also leads to new performance results without restricting the regression data to being gaussian or white the framework is based on energy conservation arguments and avoids the need for explicit recursions for the covariance matrix of the weight error vector

krylov proportionate adaptive filtering techniques not limited to sparse systems this paper proposes a novel adaptive filtering scheme named the krylov proportionate normalized least mean square kpnlms algorithm kpnlms exploits the benefits i e fast convergence for sparse unknown systems of the proportionate nlms algorithm but its applications are not limited to sparse unknown systems a set of orthonormal basis vectors is generated from a certain krylov sequence it is proven that the unknown system is sparse with respect to the basis vectors in case of fairly uncorrelated input data different adaptation gain is allocated to a coefficient of each basis vector and the gain is roughly proportional to the absolute value of the corresponding coefficient of the current estimate kpnlms enjoys i fast convergence ii linear complexity per iteration and iii no use of any a priori information numerical examples demonstrate significant advantages of the proposed scheme over the reduced rank method based on the multistage wiener filter mwf and the transform domain adaptive filter tdaf both in noisy and silent situations

neuronal tuning to sharpen or broaden sensory and motor variables are typically represented by a population of broadly tuned neurons a coarser representation with broader tuning can often improve coding accuracy but sometimes the accuracy may also improve with sharper tuning the theoretical analysis here shows that the relationship between tuning width and accuracy depends crucially on the dimension of the encoded variable a general rule is derived for how the fisher information scales with the tuning width regardless of the exact shape of the tuning function the probability distribution of spikes and allowing some correlated noise between neurons these results demonstrate a universal dimensionality effect in neural population coding

maximally informative stimuli and tuning curves for sigmoidal rate coding neurons and populations a general method for deriving maximally informative sigmoidal tuning curves for neural systems with small normalized variability is presented the optimal tuning curve is a nonlinear function of the cumulative distribution function of the stimulus and depends on the mean variance relationship of the neural system the derivation is based on a known relationship between shannon s mutual information and fisher information and the optimality of jeffrey s prior it relies on the existence of closed form solutions to the converse problem of optimizing the stimulus distribution for a given tuning curve it is shown that maximum mutual information corresponds to constant fisher information only if the stimulus is uniformly distributed as an example the case of sub poisson binomial firing statistics is analyzed in detail

optimum step size of sign algorithm for nonstationary adaptive filtering the adaptive filtering sign algorithm is analyzed in the case of nonstationary and correlated data the excess time average absolute estimation error eaae is used as a performance index of the algorithm it is proved that the eaae is the sum of two terms one proportional to the algorithm step size mu and the other proportional to 1 mu the optimum step size that minimizes the eaae is found equal to square root q p with q being the power of fluctuations of the optimal filter weights and p being the power of filter input this formula is derived under the assumption that increments of optimal filter weights are white computer simulations are used to examine the validity of the formula for other models of variation of the optimal filter simulation results show a good fit with the obtained optimum step size formula

efficient sensory encoding and bayesian inference with heterogeneous neural populations the efficient coding hypothesis posits that sensory systems maximize information transmitted to the brain about the environment we develop a precise and testable form of this hypothesis in the context of encoding a sensory variable with a population of noisy neurons each characterized by a tuning curve we parameterize the population with two continuous functions that control the density and amplitude of the tuning curves assuming that the tuning widths vary inversely with the cell density this parameterization allows us to solve in closed form for the information maximizing allocation of tuning curves as a function of the prior probability distribution of sensory variables for the optimal population the cell density is proportional to the prior such that more cells with narrower tuning are allocated to encode higher probability stimuli and that each cell transmits an equal portion of the stimulus probability mass we also compute the stimulus discrimination capabilities of a perceptual system that relies on this neural representation and find that the best achievable discrimination thresholds are inversely proportional to the sensory prior we examine how the prior information that is implicitly encoded in the tuning curves of the optimal population may be used for perceptual inference and derive a novel decoder the bayesian population vector that closely approximates a bayesian least squares estimator that has explicit access to the prior finally we generalize these results to sigmoidal tuning curves correlated neural variability and a broader class of objective functions these results provide a principled embedding of sensory prior information in neural populations and yield predictions that are readily testable with environmental physiological and perceptual data

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection