estimation efficiency under privacy constraints we investigate the problem of estimating a random variable y under a privacy constraint dictated by another correlated random variable x when x and y are discrete we express the underlying privacy utility tradeoff in terms of the privacy constrained guessing probability mathcal h p xy varepsilon and the maximum probability mathsf p mathsf c y z of correctly guessing y given an auxiliary random variable z where the maximization is taken over all p z y ensuring that mathsf p mathsf c x z leq varepsilon for a given privacy threshold varepsilon geq 0 we prove that mathcal h p xy cdot is concave and piecewise linear which allows us to derive its expression in closed form for any varepsilon when x and y are binary in the non binary case we derive mathcal h p xy varepsilon in the high utility regime i e for sufficiently large but nontrivial values of varepsilon under the assumption that y and z have the same alphabets we also analyze the privacy constrained guessing probability for two scenarios in which x y and z are binary vectors when x and y are continuous random variables we formulate the corresponding privacy utility tradeoff in terms of mathsf sensr p xy varepsilon the smallest normalized minimum mean squared error mmse incurred in estimating y from a gaussian perturbation z here the minimization is taken over a family of gaussian perturbations z for which the mmse of f x given z is within a factor 1 varepsilon from the variance of f x for any non constant real valued function f we derive tight upper and lower bounds for mathsf sensr when y is gaussian for general absolutely continuous random variables we obtain a tight lower bound for mathsf sensr p xy varepsilon in the high privacy regime i e for small varepsilon

estimation with a helper who knows the interference we consider the problem of estimating a signal corrupted by independent interference with the assistance of a cost constrained helper who knows the interference causally or noncausally when the interference is known causally we characterize the minimum distortion incurred in estimating the desired signal in the noncausal case we present a general achievable scheme for discrete memoryless systems and novel lower bounds on the distortion for the binary and gaussian settings our gaussian setting coincides with that of assisted interference suppression introduced by grover and sahai our lower bound for this setting is based on the relation recently established by verdu between divergence and minimum mean squared error we illustrate with a few examples that this lower bound can improve on those previously developed our bounds also allow us to characterize the optimal distortion in several interesting regimes moreover we show that causal and noncausal estimation are not equivalent for this problem finally we consider the case where the desired signal is also available at the helper we develop new lower bounds for this setting that improve on those previously developed and characterize the optimal distortion up to a constant multiplicative factor for some regimes of interest

optimal phase transitions in compressed sensing compressed sensing deals with efficient recovery of analog signals from linear encodings this paper presents a statistical study of compressed sensing by modeling the input signal as an i i d process with known distribution three classes of encoders are considered namely optimal nonlinear optimal linear and random linear encoders focusing on optimal decoders we investigate the fundamental tradeoff between measurement rate and reconstruction fidelity gauged by error probability and noise sensitivity in the absence and presence of measurement noise respectively the optimal phase transition threshold is determined as a functional of the input distribution and compared to suboptimal thresholds achieved by popular reconstruction algorithms in particular we show that gaussian sensing matrices incur no penalty on the phase transition threshold with respect to optimal nonlinear encoding our results also provide a rigorous justification of previous results based on replica heuristics in the weak noise regime

optimal m estimation in high dimensional regression we consider in the modern setting of high dimensional statistics the classic problem of optimizing the objective function in regression using m estimates when the error distribution is assumed to be known we propose an algorithm to compute this optimal objective function that takes into account the dimensionality of the problem although optimality is achieved under assumptions on the design matrix that will not always be satisfied our analysis reveals generally interesting families of dimension dependent objective functions

mmse of bad codes we examine codes over the additive gaussian noise channel designed for reliable communication at some specific signal to noise ratio snr and constrained by the permitted minimum mean square error mmse at lower snrs the maximum possible rate is below point to point capacity and hence these are nonoptimal codes alternatively referred to as bad codes we show that the maximum possible rate is the one attained by superposition codebooks moreover the mmse and mutual information behavior as a function of snr for any code attaining the maximum rate under the mmse constraint is known for all snr we also provide a lower bound on the mmse for finite length codes as a function of the error probability of the code

mismatched estimation and relative entropy a random variable with distribution p is observed in gaussian noise and is estimated by a mismatched minimum mean square estimator that assumes that the distribution is q instead of p this paper shows that the integral over all signal to noise ratios snrs of the excess mean square estimation error incurred by the mismatched estimator is twice the relative entropy d p q in nats this representation of relative entropy can be generalized to nonreal valued random variables and can be particularized to give new general representations of mutual information in terms of conditional means inspired by the new representation we also propose a definition of free relative entropy which fills a gap in and is consistent with the literature on free probability

estimation in poisson noise properties of the conditional mean estimator this paper considers estimation of a random variable in poisson noise with signal scaling coefficient and dark current as explicit parameters of the noise model specifically the paper focuses on properties of the conditional mean estimator as a function of the scaling coefficient the dark current parameter the distribution of the input random variable and channel realizations with respect to the scaling coefficient and the dark current several identities in terms of derivatives are established for example it is shown that the gradient of the conditional mean estimator with respect to the scaling coefficient and dark current parameter is proportional to the conditional variance moreover a score function is proposed and a tweedie like formula for the conditional expectation is recovered with respect to the distribution several regularity conditions are shown for instance it is shown that the conditional mean estimator uniquely determines the input distribution moreover it is shown that if the conditional expectation is close to a linear function in terms of mean squared error then the input distribution is approximately gamma in the levy distance furthermore sufficient and necessary conditions for linearity are found interestingly it is shown that the conditional mean estimator cannot be linear when the dark current parameter of the poisson noise is non zero

second order asymptotics of mutual information a formula for the second order expansion of the input output mutual information of multidimensional channels as the signal to noise ratio snr goes to zero is obtained while the additive noise is assumed to be gaussian we deal with very general classes of input and channel distributions as special cases these channel models include fading channels channels with random parameters and channels with almost gaussian noise when the channel is unknown at the receiver the second term in the asymptotic expansion depends not only on the covariance matrix of the input signal but also on the fourth mixed moments of its components the study of the second order asymptotics of mutual information finds application in the analysis of the bandwidth power tradeoff achieved by various signaling strategies in the wideband regime

on mmse crossing properties and implications in parallel vector gaussian channels the scalar additive gaussian noise channel has the single crossing point property between the minimum mean square error mmse in the estimation of the input given the channel output assuming a gaussian input to the channel and the mmse assuming an arbitrary input this paper extends the result to the parallel vector additive gaussian channel in three phases the channel matrix is the identity matrix and we limit the gaussian input to a vector of gaussian i i d elements the single crossing point property is with respect to the signal to noise ratio as in the scalar case the channel matrix is arbitrary and the gaussian input is limited to an independent gaussian input a single crossing point property is derived for each diagonal element of the mmse matrix the gaussian input is allowed to be an arbitrary gaussian random vector a single crossing point property is derived for each eigenvalue of the difference matrix between the two mmse matrices these three extensions are then translated to new information theoretic properties on the mutual information using the i mmse relationship a fundamental relationship between estimation theory and information theory revealed by guo and coworkers the results of the last phase are also translated to a new property of fisher information finally the applicability of all three extensions on information theoretic problems is demonstrated through a proof of a special case of shannon s vector entropy power inequality a converse proof of the capacity region of the parallel degraded broadcast channel bc under an input per antenna power constraint and under an input covariance constraint and a converse proof of the capacity region of the compound parallel degraded bc under an input covariance constraint

estimation in gaussian noise properties of the minimum mean square error consider the minimum mean square error mmse of estimating an arbitrary random variable from its observation contaminated by gaussian noise the mmse can be regarded as a function of the signal to noise ratio snr as well as a functional of the input distribution of the random variable to be estimated it is shown that the mmse is concave in the input distribution at any given snr for a given input distribution the mmse is found to be infinitely differentiable at all positive snr and in fact a real analytic function in snr under mild conditions the key to these regularity results is that the posterior distribution conditioned on the observation through gaussian channels always decays at least as quickly as some gaussian density furthermore simple expressions for the first three derivatives of the mmse with respect to the snr are obtained it is also shown that as functions of the snr the curves for the mmse of a gaussian input and that of a non gaussian input cross at most once over all snrs these properties lead to simple proofs of the facts that gaussian inputs achieve both the secrecy capacity of scalar gaussian wiretap channels and the capacity of scalar gaussian broadcast channels as well as a simple proof of the entropy power inequality in the special case where one of the variables is gaussian

heat flow derivatives and minimum mean square error in gaussian noise we connect recent developments on gaussian noise estimation and the minimum mean square error to earlier results on entropy and fisher information heat flow expansion in particular the derivatives of the minimum mean square error with respect to the noise parameter are related to the heat flow derivatives of the fisher information and a special lie algebra structure on iterated gradients the results lead in particular to a partial answer to the minimum mean square error conjecture

generalized entropy power inequalities and monotonicity properties of information new families of fisher information and entropy power inequalities for sums of independent random variables are presented these inequalities relate the information in the sum of n independent random variables to the information contained in sums over subsets of the random variables for an arbitrary collection of subsets as a consequence a simple proof of the monotonicity of information in central limit theorems is obtained both in the setting of independent and identically distributed i i d summands as well as in the more general setting of independent summands with variance standardized sums

gaussian mixture noise channels with minimum and peak amplitude constraints motivated by the idea of transmitting energy and information simultaneously we investigate in this paper the impact of constraints on the amount of energy that individual symbols carry i e minimum amplitude constraints we consider a gaussian mixture noise channel with both minimum and peak amplitude constraints first we prove that the capacity achieving input has a discrete distribution with a finite number of probability mass points then we further investigate the number and positions of the probability mass points for the capacity achieving input specifically when the interference is constant and known at both the transmitter and the receiver it can be totally eliminated so that the channel operates like an awgn channel in this case we give a theorem to determine whether the optimal input is binary for more general cases such as non binary inputs and non constant interference we investigate optimal inputs and capacities via numerical computations

gradient of mutual information in linear vector gaussian channels this paper considers a general linear vector gaussian channel with arbitrary signaling and pursues two closely related goals i closed form expressions for the gradient of the mutual information with respect to arbitrary parameters of the system and ii fundamental connections between information theory and estimation theory generalizing the fundamental relationship recently unveiled by guo shamai and verdu spl acute we show that the gradient of the mutual information with respect to the channel matrix is equal to the product of the channel matrix and the error covariance matrix of the best estimate of the input given the output gradients and derivatives with respect to other parameters are then found via the differentiation chain rule

measurement matrix design for phase retrieval based on mutual information in phase retrieval problems a signal of interest soi is reconstructed based on the magnitude of a linear transformation of the soi observed with additive noise the linear transform is typically referred to as a measurement matrix many works on phase retrieval assume that the measurement matrix is a random gaussian matrix which in the noiseless scenario with sufficiently many measurements guarantees invertability of the transformation between the soi and the observations up to an inherent phase ambiguity however in many practical applications the measurement matrix corresponds to an underlying physical setup and is therefore deterministic possibly with structural constraints in this paper we study the design of deterministic measurement matrices based on maximizing the mutual information between the soi and the observations we characterize necessary conditions for the optimality of a measurement matrix and analytically obtain the optimal matrix in the low signal to noise ratio regime practical methods for designing general measurement matrices and masked fourier measurements are proposed simulation tests demonstrate the performance gain achieved by the suggested techniques compared to random gaussian measurements for various phase recovery algorithms

functional properties of minimum mean square error and mutual information in addition to exploring its various regularity properties we show that the minimum mean square error mmse is a concave functional of the input output joint distribution in the case of additive gaussian noise the mmse is shown to be weakly continuous in the input distribution and lipschitz continuous with respect to the quadratic wasserstein distance for peak limited inputs regularity properties of mutual information are also obtained several applications to information theory and the central limit theorem are discussed

mmse bounds for additive noise channels under kullback leibler divergence constraints on the input distribution upper and lower bounds on the minimum mean square error for additive noise channels are derived when the input distribution is constrained to be close to a gaussian reference distribution in terms of the kullback leibler divergence the upper bound is tight and is attained by a gaussian distribution whose mean is identical to that of the reference distribution and whose covariance matrix is defined implicitly via a system of non linear equations the estimator that attains the upper bound is identified as a minimax optimal estimator that is robust against deviations from the assumed prior the lower bound provides an alternative to well known inequalities in estimation and information theory such as the cramer rao lower bound stam s inequality or the entropy power inequality that is potentially tighter and defined for a larger class of input distributions several examples of applications in signal processing and information theory illustrate the usefulness of the proposed bounds in practice

randomly spread cdma asymptotics via statistical physics this paper studies randomly spread code division multiple access cdma and multiuser detection in the large system limit using the replica method developed in statistical physics arbitrary input distributions and flat fading are considered a generic multiuser detector in the form of the posterior mean estimator is applied before single user decoding the generic detector can be particularized to the matched filter decorrelator linear minimum mean square error mmse detector the jointly or the individually optimal detector and others it is found that the detection output for each user although in general asymptotically non gaussian conditioned on the transmitted symbol converges as the number of users go to infinity to a deterministic function of a hidden gaussian statistic independent of the interferers thus the multiuser channel can be decoupled each user experiences an equivalent single user gaussian channel whose signal to noise ratio snr suffers a degradation due to the multiple access interference mai the uncoded error performance e g symbol error rate and the mutual information can then be fully characterized using the degradation factor also known as the multiuser efficiency which can be obtained by solving a pair of coupled fixed point equations identified in this paper based on a general linear vector channel model the results are also applicable to multiple input multiple output mimo channels such as in multiantenna systems

hessian and concavity of mutual information differential entropy and entropy power in linear vector gaussian channels within the framework of linear vector gaussian channels with arbitrary signaling the jacobian of the minimum mean square error and fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper capitalizing on prior research where the minimum mean square error and fisher information matrices were linked to information theoretic quantities through differentiation the hessian of the mutual information and the entropy are derived these expressions are then used to assess the concavity properties of mutual information and entropy under different channel conditions and also to derive a multivariate version of an entropy power inequality due to costa

the degrees of freedom of isotropic mimo interference channels without state information at the transmitters this paper fully determines the degree of freedom dof region of two user interference channels with arbitrary number of transmit and receive antennas in the case of isotropic and independent or block wise independent fading where the channel state information is available to the receivers but not to the transmitters thus the dof gap between previous upper and lower bounds due to the authors vaze and varanasi and huang jafar shamai and vishwanath is closed the dof result characterizes the capacity region to the first order of the logarithm of the signal to noise ratio snr in the high snr regime the dof region is achievable by using random gaussian codebooks independent of the channel states which implies that it is impossible to increase the dof using beamforming and interference alignment in the absence of channel state information at the transmitters

large system analysis of multiuser detection with an unknown number of users a high snr approach we analyze multiuser detection under the assumption that the number of users accessing the channel is unknown by the receiver in this environment users activity must be estimated along with any other parameters such as data power and location our main goal is to determine the performance loss caused by the need for estimating the identities of active users which are not known a priori to prevent a loss of optimality we assume that identities and data are estimated jointly rather than in two separate steps we examine the performance of multiuser detectors when the number of potential users is large statistical physics methodologies are used to determine the macroscopic performance of the detector in terms of its multiuser efficiency special attention is paid to the fixed point equation whose solution yields the multiuser efficiency of the optimal maximum a posteriori detector in the large signal to noise ratio regime our analysis yields closed form approximate bounds to the minimum mean squared error in this regime these illustrate the set of solutions of the fixed point equation and their relationship with the maximum system load next we study the maximum load that the detector can support for a given quality of service specified by error probability

normal covariant quantization maps we consider questions related to quantizing complex valued functions defined on a locally compact topological group in the case of bounded functions we generalize r werner s approach to prove the characterization of the associated normal covariant quantization maps

mutual information and minimum mean square error in gaussian channels this paper deals with arbitrarily distributed finite power input signals observed through an additive gaussian noise channel it shows a new formula that connects the input output mutual information and the minimum mean square error mmse achievable by optimal estimation of the input given the output that is the derivative of the mutual information nats with respect to the signal to noise ratio snr is equal to half the mmse regardless of the input statistics this relationship holds for both scalar and vector signals as well as for discrete time and continuous time noncausal mmse estimation this fundamental information theoretic result has an unexpected consequence in continuous time nonlinear estimation for any input signal with finite power the causal filtering mmse achieved at snr is equal to the average value of the noncausal smoothing mmse achieved with a channel whose snr is chosen uniformly distributed between 0 and snr

on the precoder design of a wireless energy harvesting node in linear vector gaussian channels with arbitrary input distribution a wireless energy harvesting node wehn operating in linear vector gaussian channels with arbitrarily distributed input symbols is considered in this paper the precoding strategy that maximizes the mutual information along n independent channel accesses is studied under non causal knowledge of the channel state and harvested energy commonly known as offline approach it is shown that at each channel use the left singular vectors of the precoder are equal to the eigenvectors of the gram channel matrix additionally an expression that relates the optimal singular values of the precoder with the energy harvesting profile through the minimum mean square error mmse matrix is obtained then the specific situation in which the right singular vectors of the precoder are set to the identity matrix is considered in this scenario the optimal offline power allocation named mercury water flowing is derived and an intuitive graphical representation is presented two optimal offline algorithms to compute the mercury water flowing solution are proposed and an exhaustive study of their computational complexity is performed moreover an online algorithm is designed which only uses causal knowledge of the harvested energy and channel state finally the achieved mutual information is evaluated through simulation

solution of shannon s problem on the monotonicity of entropy it is shown that if x1 x2 are independent and identically distributed square integrable random variables then the entropy of the normalized sum ent x1 xn over n is an increasing function of n this resolves an old problem which goes back to 6 7 5 the result also has a version for non identically distributed random variables or random vectors

a simple proof of the entropy power inequality this correspondence gives a simple proof of shannon s entropy power inequality epi using the relationship between mutual information and minimum mean square error mmse in gaussian channels

monotonic decrease of the non gaussianness of the sum of independent random variables a simple proof artstein ball barthe and naor have recently shown that the non gaussianness divergence with respect to a gaussian random variable with identical first and second moments of the sum of independent and identically distributed i i d random variables is monotonically nonincreasing we give a simplified proof using the relationship between non gaussianness and minimum mean square error mmse in gaussian channels as artstein we also deal with the more general setting of nonidentically distributed random variables

mmse dimension if n is standard gaussian the minimum mean square error mmse of estimating a random variable x based on snr x n vanishes at least as fast as 1 snr as snr we define the mmse dimension of x as the limit as snr of the product of snr and the mmse mmse dimension is also shown to be the asymptotic ratio of nonlinear mmse to linear mmse for discrete absolutely continuous or mixed distribution we show that mmse dimension equals renyi s information dimension however for a class of self similar singular x e g cantor dis tribution we show that the product of snr and mmse oscillates around information dimension periodically in snr db we also show that these results extend considerably beyond gaussian noise under various technical conditions

mimo gaussian channels with arbitrary inputs optimal precoding and power allocation in this paper we investigate the linear precoding and power allocation policies that maximize the mutual information for general multiple input multiple output mimo gaussian channels with arbitrary input distributions by capitalizing on the relationship between mutual information and minimum mean square error mmse the optimal linear precoder satisfies a fixed point equation as a function of the channel and the input constellation for non gaussian inputs a nondiagonal precoding matrix in general increases the information transmission rate even for parallel noninteracting channels whenever precoding is precluded the optimal power allocation policy also satisfies a fixed point equation we put forth a generalization of the mercury waterfilling algorithm previously proposed for parallel noninterfering channels in which the mercury level accounts not only for the non gaussian input distributions but also for the interference among inputs

optimum power allocation for parallel gaussian channels with arbitrary input distributions the mutual information of independent parallel gaussian noise channels is maximized under an average power constraint by independent gaussian inputs whose power is allocated according to the waterfilling policy in practice discrete signaling constellations with limited peak to average ratios m psk m qam etc are used in lieu of the ideal gaussian signals this paper gives the power allocation policy that maximizes the mutual information over parallel channels with arbitrary input distributions such policy admits a graphical interpretation referred to as mercury waterfilling which generalizes the waterfilling solution and allows retaining some of its intuition the relationship between mutual information of gaussian channels and nonlinear minimum mean square error mmse proves key to solving the power allocation problem

local and networked mean square estimation with high dimensional log concave noise we consider the problem of mean square estimation of the state of a discrete time dynamical system having additive non gaussian noise we assume that the noise has the structure that at each time instant it is a projection of a fixed high dimensional noise vector with a log concave density we derive conditions which guarantee that as the dimension of this noise vector grows large the optimal estimator of the problem with gaussian noise becomes near optimal for the problem with non gaussian noise the results are derived by first showing an asymptotic gaussian lower bound on the minimum error which holds even for nonlinear systems the bound is asymptotically tight for linear systems for nonlinear systems this bound is tight provided and the noise has a strongly log concave density with some additional structure these results imply that the estimate obtained by employing the gaussian estimator on the non gaussian problem satisfies an approximate orthogonality principle and the difference between this estimate and the optimal estimate vanishes strongly in l 2 for linear systems with high dimensional log concave noise of this structure we get that the kalman filter serves as a near optimal estimator a key ingredient in the proofs is a recent central limit theorem of eldan and klartag the minimum mean square error is in general neither weakly upper semicontinuous nor weakly lower semicontinuous our results show that in the setting we consider it is weakly continuous we then consider mean square estimation over a noisy network where the problem is to jointly design strategies for sensing and estimation to minimize a composite quadratic cost these problems do not have a static information structure whereby the previously derived results do not apply we consider two different cases when the estimation part of the problem corresponds to estimating the original source sensed by the sensor and when this part corresponds to the estimation of the action of the sensor in the former case we show that results analogous to the above continue to hold the latter case corresponds to a variant of witsenhausen s problem wherein we show only partial results

the relationship between causal and noncausal mismatched estimation in continuous time awgn channels a continuous time finite power process with distribution p is observed through an awgn channel at a given signal to noise ratio snr and is estimated by an estimator that would have minimized the mean square error if the process had distribution q we show that the causal filtering mean square error mse achieved at snr level snr is equal to the average value of the noncausal smoothing mse achieved with a channel whose snr is chosen uniformly distributed between 0 and snr emerging as the bridge for equating these two quantities are mutual information and relative entropy our result generalizes that of guo shamai and verdu 2005 from the nonmismatched case where p q to general p and q among our intermediate results is an extension of duncan s theorem that relates mutual information and causal mmse to the case of mismatched estimation some further extensions and implications are discussed key to our findings is the recent result of verdu on mismatched estimation and relative entropy

on the minimum mean p th error in gaussian noise channels and its applications the problem of estimating an arbitrary random vector from its observation corrupted by additive white gaussian noise where the cost function is taken to be the minimum mean p th error mmpe is considered the classical minimum mean square error mmse is a special case of the mmpe several bounds properties and applications of the mmpe are derived and discussed the optimal mmpe estimator is found for gaussian and binary input distributions properties of the mmpe as a function of the input distribution signal to noise ratio snr and order p are derived the single crossing point property scpp which provides an upper bound on the mmse and which together with the mutual information mmse relationship is a powerful tool in deriving converse proofs in multi user information theory is extended to the mmpe moreover a complementary bound to the scpp is derived as a first application of the mmpe a bound on the conditional differential entropy in terms of the mmpe is provided which then yields a generalization of the ozarow wyner lower bound on the mutual information achieved by a discrete input on a gaussian noise channel as a second application the mmpe is shown to improve on previous characterizations of the phase transition phenomenon that manifests in the limit as the length of the capacity achieving code goes to infinity as a discontinuity of the mmse as a function of snr as a final application the mmpe is used to show new bounds on the second derivative of mutual information or the first derivative of the mmse

optimal graph algorithms on a fixed size linear array parallel algorithms for computing the minimum spanning tree of a weighted undirected graph and the bridges and articulation points of an undirected graphs on a fixed size linear array of processors are presented for a graph of n vertices the algorithms operate on a linear array of p processors and require o n2 p time for all p 1 p n in particular using n processors the algorithms require o n time which is optimal on this model the paper describes two approaches to limit the communication requirements for solving the problems the first is a divide and conquer strategy applied to sollin s algorithm for finding the minimum spanning tree of a graph the second uses a novel data reduction technique that constructs an auxiliary graph with no more than 2n 2 edges whose bridges and articulation points are the bridges and articulation points of the original graph

comparison of the achievable rates in ofdm and single carrier modulation with i i d inputs we compare the maximum achievable rates in single carrier sc and orthogonal frequency division multiplexing ofdm modulation schemes under the practical assumptions of independent identically distributed finite alphabet inputs and linear intersymbol interference with additive gaussian noise we show that the shamai laroia approximation serves as a bridge between the two rates while it is well known that this approximation is often a lower bound on the sc achievable rate it is revealed to also essentially upper bound the ofdm achievable rate we apply information estimation relations in order to rigorously establish this result for both general input distributions and to sharpen it for commonly used pulse amplitude modulation pam and quadratic amplitude modulation constellations to this end novel bounds on minimum mean square error estimation of pam inputs to a scalar gaussian channel are derived which may be of general interest our results show that under reasonable assumptions optimal sc schemes may offer spectral efficiency significantly superior to that of ofdm motivating further research of such systems

gaussian wiretap channel with amplitude and variance constraints we consider the gaussian wiretap channel with amplitude and variance constraints on the channel input we first show that the entire rate equivocation region of the gaussian wiretap channel with an amplitude constraint is obtained by discrete input distributions with finite support we prove this result by considering the existing single letter description of the rate equivocation region and showing that discrete distributions with finite support exhaust this region our result highlights an important difference between the peak power amplitude constrained and the average power variance constrained cases although in the average power constrained case both the secrecy capacity and the capacity can be achieved simultaneously our results show that in the peak power constrained case in general there is a tradeoff between the secrecy capacity and the capacity in the sense that both may not be achieved simultaneously we also show that under sufficiently small amplitude constraints the possible tradeoff between the secrecy capacity and the capacity does not exist and they are both achieved by the symmetric binary distribution finally we prove the optimality of discrete input distributions in the presence of an additional variance constraint

lower bounds and approximations for the information rate of the isi channel we consider the discrete time intersymbol interference isi channel model with additive gaussian noise and fixed independent identically distributed inputs in this setting we investigate the expression put forth by shamai and laroia as a conjectured lower bound for the input output mutual information after application of a minimum mean square error decision feedback equalizer receiver a low signal to noise ratio snr expansion is used to prove that the conjectured bound does not hold under general conditions and to characterize inputs for which it is particularly ill suited one such input is used to construct a counterexample indicating that the shamai laroia expression does not always bound even the achievable rate of the channel thus excluding a natural relaxation of the original conjectured bound however this relaxed bound is then shown to hold for any finite entropy input and isi channel when the snr is sufficiently high we derive two conditions under which the relaxed bound holds involving compound channel capacity and quasiconvexity arguments finally new simple bounds for the achievable rate are proven and compared with other known bounds information estimation relations and estimation theoretic bounds play a key role in establishing our results