Channel Coding at Low Capacity Lowcapacity scenarios have become increasingly important in the technology of Internet of Things (IoT) and the next generation of mobile networks. Such scenarios require efficient and reliable transmission of information over channels with an extremely small capacity. Within these constraints, the performance of stateoftheart coding techniques is far from optimal in terms of either rate or complexity. Moreover, the current nonasymptotic laws of optimal channel coding provide inaccurate predictions for coding in the lowcapacity regime. In this paper, we provide the first comprehensive study of channel coding in the lowcapacity regime. We will investigate the fundamental nonasymptotic limits for channel coding as well as challenges that must be overcome for efficient code design in lowcapacity scenarios.

On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote sourcecoding under logarithmic loss distortion measure, information combining, common reconstruction, the WynerAhlswedeKorner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal tradeoffs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (features) distributions under power limitations maximizing the relevance for the Gaussian information bottleneck, under complexity constraints.