dual capacity upper bounds for noisy runlength constrained channels binary input memoryless channels with a run length constrained input are considered upper bounds to the capacity of such noisy run length constrained channels are derived using the dual capacity method with markov test distributions satisfying the karush kuhn tucker conditions for the capacity achieving output distribution simplified algebraic characterizations of the bounds are presented for the binary erasure channel and the binary symmetric channel these upper bounds are very close to achievable rates and improve upon previously known feedback based bounds for a large range of channel parameters for the binary input additive white gaussian noise channel the upper bound is simplified to a small scale numerical optimization problem these results provide some of the simplest upper bounds for an open capacity problem that has theoretical and practical relevance

arbitrarily varying channels with general alphabets and states previous results of csiszar and narayan 1988 1989 1991 on the capacity of discrete arbitrarily varying channels with input and state constraints are extended to the case of arbitrary alphabets and state sets for channels with scalar or vector inputs and additive interference consisting of a deterministic part and noise both arbitrarily varying subject to power constraints the capacity for deterministic codes is shown to be equal to the random coding capacity if the input power exceeds the power of the deterministic interference and zero otherwise explicit capacity formulas are also given

remarks on a recent theorem about conserved quantities a recently proved theorem is used to derive a conserved quantity associated with a velocity dependent symmetry for lagrangian systems in addition a generalization of the theorem is given

on the capacity of the wiener phase noise channel bounds and capacity achieving distributions in this paper the capacity of the additive white gaussian noise awgn channel affected by time varying wiener phase noise is investigated tight upper and lower bounds on the capacity of this channel are developed the upper bound is obtained by using the duality approach and by considering a specific distribution over the output of the channel in order to lower bound the capacity first a family of capacity achieving input distributions is found by solving a functional optimization of the channel mutual information then lower bounds on the capacity are obtained by drawing samples from the proposed distributions through monte carlo simulations the proposed capacity achieving input distributions are circularly symmetric non gaussian and the input amplitudes are correlated over time the evaluated capacity bounds are tight for a wide range of signal to noise ratio snr values and thus they can be used to quantify the capacity specifically the bounds follow the well known awgn capacity curve at low snr while at high snr they coincide with the high snr capacity result available in the literature for the phase noise channel

a strong version of the redundancy capacity theorem of universal coding the capacity of the channel induced by a given class of sources is well known to be an attainable lower bound on the redundancy of universal codes with respect to this class both in the minimax sense and in the bayesian maximin sense we show that this capacity is essentially a lower bound also in a stronger sense that is for most sources in the class this result extends rissanen s 1984 1986 lower bound for parametric families we demonstrate the applicability of this result in several examples e g parametric families with growing dimensionality piecewise fixed sources arbitrarily varying sources and noisy samples of learnable functions finally we discuss implications of our results to statistical inference

proper complex random processes with applications to information theory the covariance of complex random variables and processes when defined consistently with the corresponding notion for real random variables is shown to be determined by the usual complex covariance together with a quantity called the pseudo covariance a characterization of uncorrelatedness and wide sense stationarity in terms of covariance and pseudo covariance is given complex random variables and processes with a vanishing pseudo covariance are called proper it is shown that properness is preserved under affine transformations and that the complex multivariate gaussian density assumes a natural form only for proper random variables the maximum entropy theorem is generalized to the complex multivariate case the differential entropy of a complex random vector with a fixed correlation matrix is shown to be maximum if and only if the random vector is proper gaussian and zero mean the notion of circular stationarity is introduced for the class of proper complex random processes a discrete fourier transform correspondence is derived relating circular stationarity in the time domain to uncorrelatedness in the frequency domain an application of the theory is presented

on single antenna rayleigh block fading channels at finite blocklength this article concerns the maximum coding rate at which data can be transmitted over a noncoherent single antenna rayleigh block fading channel using an error correcting code of a given blocklength with a block error probability not exceeding a given value a high snr normal approximation of the maximum coding rate is presented that becomes accurate as the signal to noise ratio snr and the number of coherence intervals l over which we code tend to infinity numerical analyses suggest that the approximation is accurate at snr values above 15db and when the number of coherence intervals is 10 or more

a new entropy power inequality a strengthened version of shannon s entropy power inequality for the case where one of the random vectors involved is gaussian is proved in particular it is shown that if independent gaussian noise is added to an arbitrary multivariate random variable the entropy power of the resulting random variable is a concave function of the variance power of the added noise the strengthened inequality is shown to hold for the class of stable distributions

fading channels how perfect need perfect side information be the analysis of flat fading channels is often performed under the assumption that the additive noise is white and gaussian and that the receiver has precise knowledge of the realization of the fading process these assumptions imply the optimality of gaussian codebooks and of scaled nearest neighbor decoding here we study the robustness of this communication scheme with respect to errors in the estimation of the fading process we quantify the degradation in performance that results from such estimation errors and demonstrate the lack of robustness of this scheme for some situations we suggest the rule of thumb that in order to avoid degradation the estimation error should be negligible compared to the reciprocal of the signal to noise ratio snr

generalized multiscale finite element method gmsfem for elastic wave propagation in heterogeneous anisotropic media it is important to develop fast yet accurate numerical methods for seismic wave propagation to characterize complex geological structures and oil and gas reservoirs however the computational cost of conventional numerical modeling methods such as finite difference method and finite element method becomes prohibitively expensive when applied to very large models we propose a generalized multiscale finite element method gmsfem for elastic wave propagation in heterogeneous anisotropic media where we construct basis functions from multiple local problems for both the boundaries and interior of a coarse node support or coarse element the application of multiscale basis functions can capture the fine scale medium property variations and allows us to greatly reduce the degrees of freedom that are required to implement the modeling compared with conventional finite element method for wave equation while restricting the error to low values we formulate the continuous galerkin and discontinuous galerkin formulation of the multiscale method both of which have pros and cons applications of the multiscale method to three heterogeneous models show that our multiscale method can effectively model the elastic wave propagation in anisotropic media with a significant reduction in the degrees of freedom in the modeling system

a rigorous approach to high resolution entropy constrained vector quantization the nonnegativity of relative entropy implies that the differential entropy of a random vector x with probability density function pdf f is upper bounded by mathsf e left log g text x right for any arbitrary pdf g using this inequality with a cleverly chosen g we derive a lower bound on the asymptotic excess rate of entropy constrained vector quantization for d dimensional sources and r th power distortion where the asymptotic excess rate is defined as the difference between the smallest output entropy of a vector quantizer satisfying the distortion constraint and the rate distortion function in the limit as the distortion tends to zero specialized to the one dimensional case this lower bound coincides with the asymptotic excess rate achieved by a uniform quantizer thereby recovering the result by gish and pierce that uniform quantizers are asymptotically optimal as the allowed distortion tends to zero furthermore in the one dimensional case the derivation of the lower bound reveals a necessary condition for a sequence of quantizers to be asymptotically optimal this condition implies that any sequence of asymptotically optimal almost regular quantizers must converge to a uniform quantizer as the distortion tends to zero while the obtained lower bound itself is not novel to the best of our knowledge we present the first rigorous derivation that follows the direct approach by gish and pierce without resorting to heuristic high resolution approximations commonly found in the quantization literature furthermore our derivation holds for all d dimensional sources having finite differential entropy and whose integer part has finite entropy in contrast to gish and pierce we do not require additional constraints on the continuity or decay of the source pdf

models and information rates for wiener phase noise channels a waveform channel is considered where the transmitted signal is corrupted by wiener phase noise and additive white gaussian noise a discrete time channel model that considers the effect of filtering on the phase noise is developed the model is based on a multi sample receiver i e an integrate and dump filter whose output is sampled at a rate higher than the signaling rate it is shown that at high signal to noise ratio snr the multi sample receiver achieves a rate that grows logarithmically with the snr if the number of samples per symbol oversampling factor grows with the cubic root of the snr moreover the pre log factor is at least 1 2 and can be achieved by amplitude modulation for an approximate discrete time model of the multi sample receiver the capacity pre log at high snr is shown to be at least 3 4 if the number of samples per symbol grows with the square root of the snr the analysis shows that phase modulation achieves a pre log of at least 1 4 while amplitude modulation still achieves a pre log of 1 2 this is strictly greater than the capacity pre log of the approximate discrete time wiener phase noise channel with only one sample per symbol which is 1 2 numerical simulations are used to compute lower bounds on the information rates achieved by the multi sample receiver the simulations show that oversampling is beneficial for both strong and weak phase noise at high snr in fact the information rates are sometimes substantially larger than when using commonly used approximate discrete time models

capacity bounds via duality with applications to multiple antenna systems on flat fading channels a technique is proposed for the derivation of upper bounds on channel capacity it is based on a dual expression for channel capacity where the maximization of mutual information over distributions on the channel input alphabet is replaced with a minimization of average relative entropy over distributions on the channel output alphabet we also propose a technique for the analysis of the asymptotic capacity of cost constrained channels the technique is based on the observation that under fairly mild conditions capacity achieving input distributions escape to infinity the above techniques are applied to multiple antenna flat fading channels with memory where the realization of the fading process is unknown at the transmitter and unknown or only partially known at the receiver it is demonstrated that for high signal to noise ratio snr the capacity of such channels typically grows only double logarithmically in the snr to better understand this phenomenon and the rates at which it occurs we introduce the fading number as the second order term in the high snr asymptotic expansion of capacity and derive estimates on its value for various systems it is suggested that at rates that are significantly higher than the fading number communication becomes extremely power inefficient thus posing a practical limit on practically achievable rates upper and lower bounds on the fading number are also presented for single input single output siso systems the bounds coincide thus yielding a complete characterization of the fading number for general stationary and ergodic fading processes we also demonstrate that for memoryless multiple input single output miso channels the fading number is achievable using beam forming and we derive an expression for the optimal beam direction this direction depends on the fading law and is in general not the direction that maximizes the snr on the induced siso channel using a new closed form expression for the expectation of the logarithm of a noncentral chi square distributed random variable we provide some closed form expressions for the fading number of some systems with gaussian fading including siso systems with circularly symmetric stationary and ergodic gaussian fading the fading number of the latter is determined by the fading mean fading variance and the mean squared error in predicting the present fading from its past it is not directly related to the doppler spread for the rayleigh ricean and multiple antenna rayleigh fading channels we also present firm nonasymptotic upper and lower bounds on channel capacity these bounds are asymptotically tight in the sense that their difference from capacity approaches zero at high snr and their ratio to capacity approaches one at low snr

simple proof of the concavity of the entropy power with respect to added gaussian noise a very simple proof of m h costa s result see ibid vol it 31 p 751 60 1985 that the entropy power of x sub t x n o ti is concave in t is derived as an immediate consequence of an inequality concerning fisher information this relationship between fisher information and entropy is found to be useful for proving the central limit theorem thus one who seeks new entropy inequalities should try first to find new equalities about fisher information or at least to exploit the existing ones in new ways

capacity bounds for discrete time amplitude constrained additive white gaussian noise channels the capacity achieving input distribution of the discrete time additive white gaussian noise awgn channel with an amplitude constraint is discrete and seems difficult to characterize explicitly a dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector awgn channels the scalar bound improves on mckellips bound and is within 0 1 bit of capacity for all signal to noise ratios snrs the 2 d bound is within 0 15 bits of capacity provably up to 4 5 db numerical evidence suggests a similar gap for all snrs as the snr tends to infinity these bounds are accurate and match with a volume based lower bound for the 2 d complex case an analytic lower bound is derived by using a concentric constellation and is shown to be within 1 bit of capacity

on the limits of communication with low precision analog to digital conversion at the receiver as communication systems scale up in speed and bandwidth the cost and power consumption of high precision e g 8 12 bits analog to digital conversion adc becomes the limiting factor in modern transceiver architectures based on digital signal processing in this work we explore the impact of lowering the precision of the adc on the performance of the communication link specifically we evaluate the communication limits imposed by low precision adc e g 1 3 bits for transmission over the real discrete time additive white gaussian noise awgn channel under an average power constraint on the input for an adc with k quantization bins i e a precision of log2 k bits we show that the input distribution need not have any more than k 1 mass points to achieve the channel capacity for 2 bin 1 bit symmetric quantization this result is tightened to show that binary antipodal signaling is optimum for any signal to noise ratio snr for multi bit quantization a dual formulation of the channel capacity problem is used to obtain tight upper bounds on the capacity the cutting plane algorithm is employed to compute the capacity numerically and the results obtained are used to make the following encouraging observations a up to a moderately high snr of 20 db 2 3 bit quantization results in only 10 20 reduction of spectral efficiency compared to unquantized observations b standard equiprobable pulse amplitude modulated input with quantizer thresholds set to implement maximum likelihood hard decisions is asymptotically optimum at high snr and works well at low to moderate snrs as well

the capacity of discrete time memoryless rayleigh fading channels we consider transmission over a discrete time rayleigh fading channel in which successive symbols face independent fading and where neither the transmitter nor the receiver has channel state information subject to an average power constraint we study the capacity achieving distribution of this channel and prove it to be discrete with a finite number of mass points one of them located at the origin we numerically compute the capacity and the corresponding optimal distribution as a function of the signal to noise ratio snr the behavior of the channel at low snr is studied and finally a comparison is drawn with the ideal additive white gaussian noise channel

a general minimax result for relative entropy suppose nature picks a probability measure p sub spl theta on a complete separable metric space x at random from a measurable set p sub spl theta p spl theta spl theta spl isin spl theta then without knowing spl theta a statistician picks a measure q on s finally the statistician suffers a loss d p sub 0 spl par q the relative entropy between p sub spl theta and q we show that the minimax and maximin values of this game are always equal and there is always a minimax strategy in the closure of the set of all bayes strategies this generalizes previous results of gallager 1979 and davisson and leon garcia 1980

communication on the grassmann manifold a geometric approach to the noncoherent multiple antenna channel we study the capacity of multiple antenna fading channels we focus on the scenario where the fading coefficients vary quickly thus an accurate estimation of the coefficients is generally not available to either the transmitter or the receiver we use a noncoherent block fading model proposed by marzetta and hochwald see ibid vol 45 p 139 57 1999 the model does not assume any channel side information at the receiver or at the transmitter but assumes that the coefficients remain constant for a coherence interval of length t symbol periods we compute the asymptotic capacity of this channel at high signal to noise ratio snr in terms of the coherence time t the number of transmit antennas m and the number of receive antennas n while the capacity gain of the coherent multiple antenna channel is min m n bits per second per hertz for every 3 db increase in snr the corresponding gain for the noncoherent channel turns out to be m 1 m t bits per second per hertz where m min m n t 2 the capacity expression has a geometric interpretation as sphere packing in the grassmann manifold

transmit precoding in a noncoherent relay channel with channel mean feedback this paper investigates the optimal source transmit precoding to maximize the ergodic mutual information between the input and the output of a noncoherent amplify and forward relay channel with multiple antennas at the source and with a single antenna at both the relay and the destination it is assumed that only the source relay channel mean information is available at the source the challenge here is that relaying introduces a nonconvex structure in the objective function therefore previous methods dealing with channel mean feedback which generally require the concavity of the objective function cannot be applied to circumvent the difficulty at hand a different approach based on stochastic ordering is employed the stochastic optimization problem here is ultimately transformed into the comparison of two nonnegative random variables in the laplace transform order it is shown that the optimal source transmit strategy is to transmit along the known channel mean and its orthogonal eigen channels our result subsumes as an asymptotic case the optimal precoding for multiple input single output miso channels without relaying under mean feedback furthermore the analysis can be partially extended to the case with multiple antennas at the relay numerical examples are provided to complement and corroborate the analysis

information theoretic proofs of entropy power inequalities while most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information up to now shannon s entropy power inequality epi is an exception existing information theoretic proofs of the epi hinge on representations of differential entropy using either fisher information or minimum mean square error mmse which are derived from de bruijn s identity in this paper we first present an unified view of these proofs showing that they share two essential ingredients 1 a data processing argument applied to a covariance preserving linear transformation 2 an integration over a path of a continuous gaussian perturbation using these ingredients we develop a new and brief proof of the epi through a mutual information inequality which replaces stam and blachman s fisher information inequality fii and an inequality for mmse by guo shamai and verdu used in earlier proofs the result has the advantage of being very simple in that it relies only on the basic properties of mutual information these ideas are then generalized to various extended versions of the epi zamir and feder s generalized epi for linear transformations of the random variables takano and johnson s epi for dependent variables liu and viswanath s covariance constrained epi and costa s concavity inequality for the entropy power

on the capacity and mutual information of memoryless noncoherent rayleigh fading channels the present dissertation consists of a collection of five papers and an introduction the papers are dedicated firstly to the analysis and design of efficient wireless communication systems operating over fading channels and secondly to the information theoretic characterisation of fading channels under different sets of assumptions the wireless systems that are considered operate over frequency flat block fading channels under the assumption that perfect channel state information csi is available both at the transmitter and at the receiver one of the issues that is addressed is the energy efficient design of constant power link adaptation schemes in which circuit energy consumption is included in the total energy budget and in which the system s instantaneous bit error rate must never exceed a predefined threshold this is done assuming that a number of different transmission schemes are available for use and that the objective is to maximise the system s average spectral efficiency ase another issue is how to optimally choose the rates of a given finite number of capacity achieving codes in order to maximise the ase of dual branch multiple input multiple output mimo systems that is to say mimo systems with either two transmit or two receive antennae yet another is the derivation of expressions for the signal to noise ratio snr distributions of the independent subchannels which can be obtained by decoupling a dual branch mimo channel using linear precoding and decoding assuming that the available power is distributed among these subchannels in accordance with the water filling solution it is also shown that the knowledge of the subchannel snr distributions can be very helpful when optimising mimo communication systems the information theoretic characterisation of fading channels under different sets of assumptions is the other main concern of this thesis the first channel that is considered from this perspective is the frequency flat block fading channel discussed above the power adaptation strategy that maximises the average information rate that can be reliably transmitted over such a channel assuming the availability of perfect transmitter and receiver csi assuming that any transmitted codeword spans a single fading block and assuming the input signal is subject to both average and peak power constraints is characterised by means of a theorem and numerically computed in different scenarios the next channel that is considered is the memoryless noncoherent rayleigh fading channel in which the channel state is assumed to change on a symbol by symbol basis closed form expressions for the mutual information between the output and the input of this channel when the input magnitude distribution is discrete and restricted to having two mass points are derived and it is shown how these expressions can be used to obtain closed form expressions for the capacity of this channel for snr values of up to approximately 0 db the final channels to be examined are noncoherent rayleigh fading channels with memory in which the channel state is once again assumed to change on a symbol by symbol basis but where the fading process has memory which is modelled by an autoregressive process of arbitrary order for such channels it is shown that for any input magnitude distribution it is optimum from a capacity perspective to choose the phase of the input independent and identically distributed with a distribution which is uniform over the interval 0 2