On Convexity of Error Rates in Digital Communications

Convexity properties of error rates of a class of decoders, including the maximum-likelihood/min-distance one as a special case, are studied for arbitrary constellations, bit mapping, and coding. Earlier results obtained for the additive white Gaussian noise channel are extended to a wide class of noise densities, including unimodal and spherically invariant noise. Under these broad conditions, symbol and bit error rates are shown to be convex functions of the signal-to-noise ratio (SNR) in the high-SNR regime with an explicitly determined threshold, which depends only on the constellation dimensionality and minimum distance, thus enabling an application of the powerful tools of convex optimization to such digital communication systems in a rigorous way. It is the decreasing nature of the noise power density around the decision region boundaries that ensures the convexity of symbol error rates in the general case. The known high/low-SNR bounds of the convexity/concavity regions are tightened and no further improvement is shown to be possible in general. The high-SNR bound fits closely into the channel coding theorem: all codes, including capacity-achieving ones, whose decision regions include the hardened noise spheres (from the noise sphere hardening argument in the channel coding theorem), satisfy this high-SNR requirement and thus has convex error rates in both SNR and noise power. We conjecture that all capacity-achieving codes have convex error rates. Convexity properties in signal amplitude and noise power are also investigated. Some applications of the results are discussed. In particular, it is shown that fading is convexity-preserving and is never good in low dimensions under spherically invariant noise, which may also include any linear diversity combining.

Related papers:

title: Convex Optimization
abstract: Convex optimization problems arise frequently in many different fields. A comprehensive introduction to the subject, this book shows in detail how such problems can be solved numerically with great efficiency. The focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. The text contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance, and economics.

title: Principles of Communication Engineering
abstract: Textbook on communication engineering emphasizing random processes, information and detection theory, statistical communication theory, applications, etc

title: Convexity properties in binary detection problems
abstract: The author investigates the convexity properties of error probability in the detection of binary-valued scalar signals corrupted by additive noise. It is shown that the error probability of the maximum-likelihood receiver is a convex function of the signal power when the noise has a unimodal distribution. Based on this property, the results of the optimal time-sharing strategies of transmitters and jammers, and of the optimal use of multiple channels are obtained.

title: Symbol Error Rates of Maximum-Likelihood Detector: Convex/Concave Behavior and Applications
abstract: Convexity/concavity properties of symbol error rates (SER) of the maximum likelihood detector operating in the AWGN channel (non-fading and fading) are studied. Generic conditions are identified under which the SER is a convex/concave function of the SNR. Universal bounds for the SER 1st and 2nd derivatives are obtained, which hold for arbitrary constellations and are tight for some of them. Applications of the results are discussed, which include optimum power allocation in spatial multiplexing systems, optimum power/time sharing to decrease or increase (jamming problem) error rate, and implication for fading channels.

title: Log-Concavity Property of the Error Probability With Application to Local Bounds for Wireless Communications
abstract: A clear understanding of the behavior of error probability (EP) as a function of signal-to-noise ratio (SNR) and other system parameters is fundamental for assessing the design of digital wireless communication systems. We propose an analytical framework based on the log-concavity property of the EP which we prove for a wide family of multidimensional modulation formats in the presence of Gaussian disturbances and fading. Based on this property, we construct a class of local bounds for the EP that improve known generic bounds in a given region of the SNR and are invertible, as well as easily tractable for further analysis. This concept is motivated by the fact that communication systems often operate with performance in a certain region of interest (ROI) and, thus, it may be advantageous to have tighter bounds within this region instead of generic bounds valid for all SNRs. We present a possible application of these local bounds, but their relevance is beyond the example made in this paper.

title: Statistical-Physical Models of Electromagnetic Interference
abstract: Most man-made and natural electromagnetic interference, or "noise," are highly non-Gaussian random processes, whose degrading effects on system performance can be severe, particularly on most conventional systems, which are designed for optimal or near optimal performance against normal noise. In addition, the nature, origins, measurement, and prediction of the general EM interference environment are a major concern of any adequate spectral management program. Accordingly, this study is devoted to the development of analytically tractable, experimentally verifiable, statistical-physical models of such electromagnetic interference. Here, classification into three major types of noise is made: Class A (narrow band vis-á-vis the receiver), Class B (broad band vis-á-vis the receiver), and Class C (= Class A + Class B). First-order statistical models are constructed for the Class A and Class B cases. In particular, the APD (a posteriori probability distribution) or exceedance probability, PD, vis;P1 (¿ > ¿o)A,B, (and the associated probability densities, pdf's w1(¿)A,B,[1]) of the envelope are obtained; (the phase is shown to be uniformly distributed in (0, 2¿). These results are canonical, i.e., their analytic forms are invariant of the particular noise source and its quantifying parameter values, levels, etc. Class A interference is described by a 3-parameter model, Class B noise by a 6-parameter model.

title: Non-Gaussian Noise Models in Signal Processing for Telecommunications: New Methods and Results for Class A and Class B Noise Models
abstract: The subject here is generalized (i.e., non-Gaussian) noise models, and specifically their first-order probability density functions (PDFs). Attention is focused primarily on the author's canonical statistical-physical Class A and Class B models. In particular, Class A noise describes the type of electromagnetic interference (EMI) often encountered in telecommunication applications, where this ambient noise is largely due to other, "intelligent" telecommunication operations. On the other hand, ambient Class B noise usually represents man-made or natural "nonintelligent"-i.e., nonmessage-bearing noise-and is highly impulsive.  A noise is not an /spl alpha/-stable process, nor is it reducible to such, except in the limiting Gaussian cases of high-density noise (by the central limit theorem). Class B noise is also asymptotically normal (before model approximation). Under rather broad conditions, principally governed by the source propagation and distribution scenarios, the PDF of Class B noise alone (no Gaussian component) can usually be approximated by (1) a symmetric Gaussian /spl alpha/-stable (S/spl alpha/S) model in the case of narrowband reception, or when the PDF /spl omega//sub 1/(/spl alpha/) of the amplitude is symmetric; and (2) a nonsymmetric /spl alpha/-stable (NS/spl alpha/S) model (no Gaussian component) can be constructed in broadband regimes. New results here include: (i) counting functional methods for constructing the general qth-order characteristic functions (CFs) of Class A and Class B noise, from which (all) moments and (in principle), the PDFs follow; (ii) the first-order CFs, PDFs, and cumulative probabilities (APDs) of nonsymmetric broadband Class B noise, extended to include additive Gauss noise (AGN); (iii) proof of the existence of all moments in the basic Class A and Class B models; (iv) the key physical role of AGN and the fact that AGN removes /spl alpha/-stability; (v) the explicit roles of the propagation and distribution scenarios; and (vi) extension to noise fields. Although telecommunication applications are emphasized, Class A and Class B noise models apply selectively, but equally well, to other physical regimes, e.g., underwater acoustics and EM (radar, optics, etc.). Supportive empirical data are included.

title: Canonical detection in spherically invariant noise
abstract: The paper deals with the detection of signals with unknown parameters in impulsive noise, modeled as a spherically symmetric random process. The proposed model subsumes several interesting families of noise amplitude distributions: generalized Cauchy, generalized Laplace, generalized Gaussian, contaminated normal. It also allows handling of the case of correlated noise by a whitening approach. The generalized maximum likelihood decision strategy is adopted, resulting in a canonical detector, which is independent of the amplitude distribution of the noise. A general method for performance evaluation is outlined, and a comprehensive performance analysis is carried out for the case of M-ary equal-energy orthogonal signals under several distributional assumptions for the noise. The performance is contrasted with that of the maximum likelihood receiver for completely known signals, so as to assess the loss due to the a-priori uncertainty as to the signal parameters.

title: Some Characteristic Properties of Gaussian Stochastic Processes
abstract: In the paper spherically invariant processes are defined. The characteristic function of these processes $(\xi (t))$ in accordance with Shonberg’s theorem  is of the form \[ \chi (\eta ) \equiv {\bf M}e^{i\eta } = f({\bf D}\eta ) = \int_0^\infty {e^{ - \gamma {\bf D}\eta } } G(d\gamma ), \]$\eta = \int {\xi (t)\eta (t)dt} $, where G is some measure on $[ {0,\infty } )$. Only if the process is spherically invariant, then 1) every extrapolation problem has a linear solution, 2) every functional transformation leaving the correlation function of the process invariant retains its measure in the space of realizations.If a spherically invariant process is stationary and ergodic, then it is Gaussian.

title: Optimum detection of fading signals in impulsive noise
abstract: This paper deals with the synthesis and the analysis of optimum receivers to detect one out of M equally likely, equi-energy, fading signals in impulsive noise, modelled as a compound Gaussian, possibly correlated process. We show that the conventional coherent and incoherent detectors are still optimum, independent of the noise as well as the fading probability density functions. The performance analysis has been carried on with reference to the general case of arbitrarily distributed disturbance: in order to simplify the analysis, asymptotical expressions have been developed for high signal-to-noise ratios as well as high signal space dimensionality. Interestingly enough, this allows separating the effect of the noise spikyness from that of the fading law. Results indicate that, for deep fading, the noise marginal distribution does not dramatically affect the error probability, nor is it influential on the limit operating characteristics corresponding to infinite signal space dimensions. For non-fluctuating signals, instead, the noise distribution plays a primary role: spiky noise usually produces performance impairment; moreover, the limit performance in impulsive disturbance may exhibit marked deviations from the well-known stepwise shape which is typical of Gaussian channels.

title: Principles of Digital Transmission: With Wireless Applications
abstract: From the Publisher: 
Principles of Digital Transmission is designed for advanced undergraduate and graduate level students and professions in telecommunications. Teachers and learners can mix and match chapters to create four distinct courses: (1) a one-term basic course in digital communications; (2) a one-term course in advanced digital communications; (3) a one-term course in information theory and coding; (4) a two-term course sequence in digital communications and coding. 
 
The book provides rigorous mathematical tools for the analysis and design of digital transmission systems. The authors emphasize methodology in their aim to teach the reader how to do it rather than how it is done. They apply the fundamental tools of the discipline onto a number of systems, such as wireless data transmission systems.

title: Detection in the presence of spherically symmetric random vectors
abstract: A random n -vector X=(X_{1}, \cdots , X_{n}) is said to be spherically symmetric (SS) if its joint characteristic function (CF) can be expressed as a function of the quadratic form u \rho u \prime , where u = (u_{1}, \cdots , u_{n}) and \rho is an n \times n positive definite matrix. The investigation in this paper is concerned with the properties of such vectors and some detection problems involving them. We first prove a theorem characterizing the form of SS random vectors X and use it to find the form of the probability density functions (pdf's) of X and of X + N , where N \sim {\cal N (0, \sigma^{2}I) is an independent Gaussian vector and I is the identity matrix. Applying these results we look at the problem of detecting a known signal vector in the presence of X + N when \rho =I . For the k -ary detection problem we present two conditions under which the "minimum distance" receiver is optimum. Lastly, we discuss an application of our findings to the problem of coherent detection of binary phase-shift keyed (PSK) signals in the presence of multiple co-channel interferences and white Gaussian noise.

title: Gaussian class multivariate Weibull distributions: theory and applications in fading channels
abstract: Ascertaining on the suitability of the Weibull distribution to model fading channels, a theoretical framework for a class of multivariate Weibull distributions, originated from Gaussian random processes, is introduced and analyzed. Novel analytical expressions for the joint probability density function (pdf), moment-generating function (mgf), and cumulative distribution function (cdf) are derived for the bivariate distribution of this class with not necessarily identical fading parameters and average powers. Two specific distributions with arbitrary number of correlated variates are considered and studied: with exponential and with constant correlation where their pdfs are introduced. Both cases assume equal average fading powers, but not necessarily identical fading parameters. For the multivariate Weibull distribution with exponential correlation, useful corresponding formulas, as for the bivariate case, are derived. The presented theoretical results are applied to analyze the performance of several diversity receivers employed with selection, equal-gain, and maximal-ratio combining (MRC) techniques operating over correlated Weibull fading channels. For these diversity receivers, several useful performance criteria such as the moments of the output signal-to-noise ratio (SNR) (including average output SNR and amount of fading) and outage probability are analytically derived. Moreover, the average symbol error probability for several coherent and noncoherent modulation schemes is studied using the mgf approach. The proposed mathematical analysis is complemented by various evaluation results, showing the effects of the fading severity as well as the fading correlation on the diversity receivers performance.

title: The Weibull Distribution: A Handbook
abstract: Preface Genesis, Theory, and Description History and Meaning of the Weibull Distribution Definition and Properties of the Weibull Distribution Related Distributions Weibull Processes and Weibull Renewal Theory Order Statistics and Related Variables Characterizations Applications and Inference Weibull Applications and Aids in Doing Weibull Analysis Collecting Life Data Parameter Estimation-Graphical Approaches Parameter Estimation-Least Squares and Linear Approaches Parameter Estimation-Maximum Likelihood Approaches Parameter Estimation-Methods of Moments Parameter Estimation-More Classical Approaches and Comparisons Parameter Estimation-Bayesian Approaches Parameter Estimation-Further Approaches Parameter Estimation in Accelerated Life Testing Parameter Estimation for Mixed Weibull Models Inference of Weibull Processes Estimation of Percentiles and Reliability Including Tolerance Limits Prediction of Future Random Quantities Weibull Parameter Testing Weibull Goodness-of-Fit Testing and Related Problems Appendices Table of the Gamma, Digamma and Trigamma Functions Abbreviations Mathematical and Statistical Notations Bibliography Author Index Subject Index

title: Multivariate Exponential Power Distributions as Mixtures of Normal Distributions with Bayesian Applications
abstract: This paper shows that a multivariate exponential power distribution is a scale mixture of normal distributions, with respect to a probability distribution function, when its kurtosis parameter belongs to the interval (0, 1]. The corresponding mixing probability distribution function is presented. This result is used to design, through a Bayesian hierarchical model, an algorithm to generate samples of the posterior distribution; this is applied to a problem of quantitative genetics.

title: Optimum Transmission Ranges in a Direct-Sequence Spread-Spectrum Multihop Packet Radio Network
abstract: The authors obtain the optimum transmission ranges to maximize throughput for a direct-sequence spread-spectrum multihop packet radio network. In the analysis, they model the network self-interference as a random variable which is equal to the sum of the interference power of all other terminals plus background noise. The model is applicable to other spread-spectrum schemes where the interference of one user appears as a noise source with constant power spectral density to the other users. The network terminals are modeled as a random Poisson field of interference power emitters. The statistics of the interference power at a receiving terminal are obtained and shown to be the stable distributions of a parameter that is dependent on the propagation power loss law. The optimum transmission range in such a network is of the form CK/sup alpha / where C is a constant, K is a function of the processing gain, the background noise power spectral density, and the degree of error-correction coding used, and alpha is related to the power loss law. The results obtained can be used in heuristics to determine optimum routing strategies in multihop networks.

title: A representation theorem and its applications to spherically-invariant random processes
abstract: The n th-order characteristic functions (cf) of spherically-invariant random processes (sirp) with zero means are defined as cf, which are functions of n th-order quadratic forms of arbitrary positive definite matrices p . Every n th-order spherically-invariant characteristic function (sicf) is represented as a weighted Lebesgue-Stieltjes integral transform of an arbitrary univariate probability distribution function F(\cdot) on [0,\infty) . Furthermore, every n th-order sicf has a corresponding spherically-invariant probability density (sipd). Then we show that every n th-order sicf (or sipd) is a random mixture of a n th-order Gaussian cf [or probability density]. The randomization is performed on \nu^2 \rho , where \nu is a random variable (tv) specified by the F(\cdot) function. Examples of sirp are given. Relations to previously known results are discussed. Various expectation properties of Gaussian random processes are valid for sirp. Related conditional expectation, mean-square estimation, semMndependence, martingale, and closure properties are given. Finally, the form of the unit threshold likelihood ratio receiver in the detection of a known deterministic signal in additive sirp noise is shown to be a correlation receiver or a matched filter. The associated false-alarm and detection probabilities are expressed in closed forms.