clustered model reduction of positive directed networks this paper proposes a clustered model reduction method for semistable positive linear systems evolving over directed networks in this method we construct a set of clusters i e disjoint sets of state variables based on a notion of cluster reducibility defined as the uncontrollability of local states by aggregating the reducible clusters with aggregation coefficients associated with the frobenius eigenvector we obtain an approximate model that preserves not only a network structure among clusters but also several fundamental properties such as semistability positivity and steady state characteristics furthermore it is found that the cluster reducibility can be characterized for semistable systems based on a projected controllability gramian that leads to an a priori h 2 error bound of the state discrepancy caused by aggregation the efficiency of the proposed method is demonstrated through an illustrative example of enzyme catalyzed reaction systems described by a chemical master equation this captures the time evolution of chemical reaction systems in terms of a set of ordinary differential equations

uniform random number generation from markov chains non asymptotic and asymptotic analyses in this paper we derive non asymptotic achievability and converse bounds on the random number generation with without side information our bounds are efficiently computable in the sense that the computational complexity does not depend on the block length we also characterize the asymptotic behaviors of the large deviation regime and the moderate deviation regime by using our bounds which implies that our bounds are asymptotically tight in those regimes we also show the second order rates of those problems and derive single letter forms of the variances characterizing the second order rates furthermore we address the relative entropy rate and the modified mutual information rate for these problems

renyi resolvability and its applications to the wiretap channel the conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy in contrast to previous works in this paper we use the normalized or unnormalized renyi divergence with the renyi parameter in 0 2 cup infty to measure the level of approximation we also provide asymptotic expressions for normalized renyi divergence when the renyi parameter is larger than or equal to 1 as well as lower and upper bounds for the case when the same parameter is smaller than 1 we characterize the renyi resolvability which is defined as the minimum rate required to ensure that the renyi divergence vanishes asymptotically the renyi resolvabilities are the same for both the normalized and unnormalized divergence cases in addition when the renyi parameter smaller than 1 consistent with the traditional case where the renyi parameter is equal to 1 the renyi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution when the renyi parameter is larger than 1 the renyi resolvability is in general larger than the mutual information the optimal renyi divergence is proven to vanish at least exponentially fast for both of these two cases as long as the code rate is larger than the renyi resolvability the optimal exponential rate of decay for i i d random codes is also characterized exactly we apply these results to the wiretap channel and completely characterize the optimal tradeoff between the rates of the secret and non secret messages when the leakage measure is given by the unnormalized renyi divergence this tradeoff differs from the conventional setting when the leakage is measured by the traditional mutual information

secret key generation for a pairwise independent network model we consider secret key generation for a pairwise independent network model in which every pair of terminals observes correlated sources that are independent of sources observed by all other pairs of terminals the terminals are then allowed to communicate publicly with all such communication being observed by all the terminals the objective is to generate a secret key shared by a given subset of terminals at the largest rate possible with the cooperation of any remaining terminals secrecy is required from an eavesdropper that has access to the public interterminal communication a single letter formula for secret key capacity brings out a natural connection between the problem of secret key generation and a combinatorial problem of maximal packing of steiner trees in an associated multigraph an explicit algorithm is proposed for secret key generation based on a maximal packing of steiner trees in a multigraph the corresponding maximum rate of steiner tree packing is thus a lower bound for the secret key capacity when only two of the terminals or when all the terminals seek to share a secret key the mentioned algorithm achieves secret key capacity in which case the bound is tight

distributed source coding using abelian group codes a new achievable rate distortion region a distributed source coding problem with a joint distortion criterion that depends on the sources and the reconstruction is considered in this work while the prevalent trend in information theory has been to prove achievability results using shannon s random coding arguments using structured random codes offer rate gains over unstructured random codes for many problems motivated by this a new achievable rate distortion region an inner bound to the performance limit is presented for this problem for discrete memoryless sources based on good structured random nested codes built over abelian groups for certain sources and distortion functions the new rate region is shown to be strictly bigger than the berger tung rate region which has been the best known achievable rate region for this problem till now this is done using numerical plots achievable rates for single user source coding using abelian group codes are also obtained as a corollary of the main coding theorem it is shown that nested linear codes achieve the shannon rate distortion function in the arbitrary discrete memoryless case

constrained source coding with side information the source coding problem with side information at the decoder is studied subject to a constraint that the encoder to whom the side information is unavailable be able to compute the decoder s reconstruction sequence to within some distortion for discrete memoryless sources and finite single letter distortion measures an expression is given for the minimal description rate as a function of the joint law of the source and side information and of the allowed distortions at the encoder and at the decoder the minimal description rate is also computed for a memoryless gaussian source with squared error distortion measures a solution is also provided to a more general problem where there are more than two distortion constraints and each distortion measure may be a function of three arguments the source symbol the encoder s reconstruction symbol and the decoder s reconstruction symbol

on the structure of real time encoding and decoding functions in a multiterminal communication system a real time communication system with two encoders communicating with a single receiver over separate noisy channels is considered the two encoders make distinct partial observations of a markov source each encoder must encode its observations into a sequence of discrete symbols the symbols are transmitted over noisy channels to a finite memory receiver that attempts to reconstruct some function of the state of the markov source encoding and decoding must be done in real time that is the distortion measure does not tolerate delays under the assumption that the encoders observations are conditionally independent markov chains given an unobserved time invariant random variable results on the structure of optimal real time encoding and decoding functions are obtained it is shown that there exist finite dimensional sufficient statistics for the encoders the problem with noiseless channels and perfect memory at the receiver is then considered a new methodology to find the structure of optimal real time encoders is employed a sufficient statistic with a time invariant domain is found for this problem this methodology exploits the presence of common information between the encoders and the receiver when communication is over noiseless channels

information theoretic key agreement of multiple terminals part i we study the problem of information theoretically secure secret key agreement under the well known source model and channel model in both of these models multiple terminals wish to create a shared secret key that is secure from a passive eavesdropper the terminals have access to a noiseless public communication channel and an additional resource that depends on the model in the source model the resource is an external source that repeatedly beams correlated randomness to the terminals whereas in the channel model the resource is a secure but noisy discrete memoryless broadcast channel we derive new lower and upper bounds on the secret key capacity under both the source model and the channel model the technique used for deriving our bound for the source model is to find certain properties of functions of joint probability distributions which applied to the joint distribution of the source will imply that they dominate the secret key capacity and then prove the bound by a verification argument a similar technique is used for the channel model finally we also define a problem of communication for omniscience by a neutral observer and establish the equivalence between this new problem and the problem of secret key agreement this generalizes an earlier result of csiszar and narayan

change of multivariate mutual information from local to global we study the change of multivariate mutual information among a set of random variables when some common randomness is added to or removed from a subset of the random variables this is formulated more precisely as two new multiterminal secret key agreement problems that respectively ask how one can increase the secrecy capacity efficiently by adding common randomness to a small subset of users and how one can simplify the source model by removing redundant common randomness that does not contribute to the secrecy capacity characterizations and strongly polynomial time computations are derived for the rates of change maximum usable increment and redundancy these results can be applied to study the communication complexity for secret key agreement

numerical stationary distribution and its convergence for nonlinear stochastic differential equations to avoid finding the stationary distributions of stochastic differential equations by solving the nontrivial kolmogorov fokker planck equations the numerical stationary distributions are used as the approximations instead this paper is devoted to approximate the stationary distribution of the underlying equation by the backward euler maruyama method currently existing results mao et al 2005 yuan et al 2005 yuan et al 2004 are extended in this paper to cover larger range of nonlinear sdes when the linear growth condition on the drift coefficient is violated

determining optimal rates for communication for omniscience this paper considers the communication for omniscience problem a set of users observe a discrete memoryless multiple source and want to recover the entire multiple source via noise free broadcast communications we study the problem of how to determine an optimal rate vector that attains omniscience with the minimum sum rate the total number of communications the results cover both asymptotic and non asymptotic models where the transmission rates are real and integral respectively we propose a modified decomposition algorithm mda and a sum rate increment algorithm sia for the asymptotic and non asymptotic models respectively both of which determine the value of the minimum sum rate and a corresponding optimal rate vector in polynomial time for the coordinate saturation capacity algorithm a nesting algorithm in mda and sia we propose to implement it by a fusion method and show by experimental results that this fusion method contributes to a reduction in computation complexity finally we show that the separable convex minimization problem over the optimal rate vector set in the asymptotic model can be decomposed by the fundamental partition the optimal partition of the user set that determines the minimum sum rate so that the problem can be solved more efficiently

secrecy capacities for multiple terminals we derive single letter characterizations of strong secrecy capacities for models with an arbitrary number of terminals each of which observes a distinct component of a discrete memoryless multiple source with unrestricted and interactive public communication permitted between the terminals a subset of these terminals can serve as helpers for the remaining terminals in generating secrecy according to the extent of an eavesdropper s knowledge three kinds of secrecy capacity are considered secret key sk private key pk and wiretap secret key wsk capacity the characterizations of the sk and pk capacities highlight the innate connections between secrecy generation and multiterminal source coding without secrecy requirements a general upper bound for wsk capacity is derived which is tight in the case when the eavesdropper can wiretap noisy versions of the components of the underlying multiple source provided randomization is permitted at the terminals these secrecy capacities are seen to be achievable with noninteractive communication between the terminals the achievability results are also shown to be universal

two way source coding with a fidelity criterion let x i y i i 1 2 cdots be a sequence of independent identically distributed bivariate random variables and consider the following communication situation the x component of the process is observed at some location say a while the y component is observed at a different location say b the x y component of the process has to be reproduced at location b a it is desired to find the amount of information that should be exchanged between the two locations so that the distortions incurred will not exceed some predetermined tolerance a single letter characterization of a certain region cal k subset cal r 4 of rates and distortions is given this region contains all and only the achievable rates and distortions for the special case where block coding is used and the information is conveyed through a one way link that can switch direction only k times per block

achievable rate region for three user discrete broadcast channel based on coset codes we consider the problem of developing coding techniques and deriving achievable rate regions for discrete memoryless broadcast channels with three receivers 3 dbc we begin by identifying a novel vector additive 3 dbc for which we characterize an upper bound on the the largest achievable rate region based on unstructured codes henceforth referred to as pmb mathscr u mathcal m region we propose a coding technique based on coset codes that yield an achievable rate triple not contained within pmb mathscr u mathcal m region we generalize the proposed coding technique using a new ensemble of codes partitioned coset codes pcc containing both empirical and algebraic properties and evaluate its performance to derive an achievable rate region for the general 3 dbc the new elements in this derivation are binning and joint typicality encoding and decoding of statistically correlated pccs we validate the utility of this technique by identifying non additive instances of 3 dbc for which the proposed coding techniques based on pcc yield strictly larger rates

secret key generation for correlated gaussian sources secret key generation by multiple terminals is considered based on their observations of jointly distributed gaussian signals followed by public communication among themselves exploiting an inherent connection between secrecy generation and lossy data compression two main contributions are made the first is a characterization of strong secret key capacity and entails a converse proof technique that is valid for real valued and not necessarily gaussian as well as finite valued signals the capacity formula acquires a simple form when the terminals observe symmetrically correlated jointly gaussian signals for the latter setup with two terminals considering schemes that involve quantization at one terminal the best rate of an achievable secret key is characterized as a function of quantization rate secret key capacity is attained as the quantization rate tends to infinity structured codes are shown to attain the optimum tradeoff between secret key rate and quantization rate constituting our second main contribution

nested stochastic simulation algorithms for chemical kinetic systems with multiple time scales we present an efficient numerical algorithm for simulating chemical kinetic systems with multiple time scales this algorithm is an improvement of the traditional stochastic simulation algorithm ssa also known as gillespie s algorithm it is in the form of a nested ssa and uses an outer ssa to simulate the slow reactions with rates computed from realizations of inner ssas that simulate the fast reactions the algorithm itself is quite general and seamless and it amounts to a small modification of the original ssa our analysis of such multi scale chemical kinetic systems allows us to identify the slow variables in the system derive effective dynamics on the slow time scale and provide error estimates for the nested ssa efficiency of the nested ssa is discussed using these error estimates and illustrated through several numerical examples

coding for positive rate in the source model key agreement problem a two party key agreement problem with public discussion known as the source model problem is considered by relating key agreement to hypothesis testing a new coding scheme is developed that yields a sufficient condition to achieve a positive secret key sk rate in terms of renyi divergence the merits of this coding scheme are illustrated by applying it to an erasure model for eve s side information and by deriving an upper bound on eve s erasure probabilities for which the sk capacity is zero this bound strictly improves on the best known single letter lower bound on the sk capacity moreover the bound is tight when alice s or bob s source is binary which extends a previous result for a doubly symmetric binary source the results motivate a new measure for the correlation between two random variables which is of independent interest

equivocations exponents and second order coding rates under various renyi information measures we evaluate the asymptotics of equivocations their exponents as well as their second order coding rates under various renyi information measures specifically we consider the effect of applying a hash function on a source and we quantify the level of non uniformity and dependence of the compressed source from another correlated source when the number of copies of the sources is large unlike previous works that use shannon information measures to quantify randomness information or uniformity we define our security measures in terms of a more general class of information measures the renyi information measures and their gallager type counterparts a special case of these renyi information measure is the class of shannon information measures we prove tight asymptotic results for the security measures and their exponential rates of decay we also prove bounds on the second order asymptotics and show that these bounds match when the magnitudes of the second order coding rates are large we do so by establishing new classes non asymptotic bounds on the equivocation and evaluating these bounds using various probabilistic limit theorems asymptotically

statistical inference under multiterminal data compression this paper presents a survey of the literature on the information theoretic problems of statistical inference under multiterminal data compression with rate constraints significant emphasis is put on problems 1 multiterminal hypothesis testing 2 multiterminal parameter estimation and 3 multiterminal pattern classification in either case of positive rates or zero rates in addition the paper includes three new results i e the converse theorems for all problems of multiterminal hypothesis testing multiterminal parameter estimation and multiterminal pattern classification at the zero rate

the lambda context calculus extended version we present the lambda context calculus this simple lambda calculus features variables arranged in a hierarchy of strengths such that substitution of a strong variable does not avoid capture with respect to abstraction by a weaker variable this allows the calculus to express both capture avoiding and capturing substitution instantiation the reduction rules extend the vanilla lambda calculus in a simple and modular way and preserve the look and feel of a standard lambda calculus with explicit substitutions good properties of the lambda calculus are preserved the lamcc is confluent and a natural injection into the lamcc of the untyped lambda calculus exists and preserves strong normalisation we discuss the calculus and its design with full proofs in the presence of the hierarchy of variables functional binding splits into a functional abstraction l lambda and a name binder new we investigate how the components of this calculus interact with each other and with the reduction rules with examples in two more extended case studies we demonstrate how global state can be expressed and how contexts and contextual equivalence can be naturally internalised using function application

f divergence inequalities this paper develops systematic approaches to obtain f divergence inequalities dealing with pairs of probability measures defined on arbitrary alphabets functional domination is one such approach where special emphasis is placed on finding the best possible constant upper bounding a ratio of f divergences another approach used for the derivation of bounds among f divergences relies on moment inequalities and the logarithmic convexity property which results in tight bounds on the relative entropy and bhattacharyya distance in terms of chi 2 divergences a rich variety of bounds are shown to hold under boundedness assumptions on the relative information special attention is devoted to the total variation distance and its relation to the relative information and relative entropy including reverse pinsker inequalities as well as on the e gamma divergence which generalizes the total variation distance pinsker s inequality is extended for this type of f divergence a result which leads to an inequality linking the relative entropy and relative information spectrum integral expressions of the renyi divergence in terms of the relative information spectrum are derived leading to bounds on the renyi divergence in terms of either the variational distance or relative entropy

distributed functional scalar quantization simplified distributed functional scalar quantization dfsq theory provides optimality conditions and predicts performance of data acquisition systems in which a computation on acquired data is desired we address two limitations of previous works prohibitively expensive decoder design and a restriction to source distributions with bounded support we show that a much simpler decoder has equivalent asymptotic performance to the conditional expectation estimator studied previously thus reducing decoder design complexity the simpler decoder features decoupled communication and computation blocks moreover we extend the dfsq framework with the simpler decoder to source distributions with unbounded support finally through simulation results we demonstrate that performance at moderate coding rates is well predicted by the asymptotic analysis and we give new insight on the rate of convergence

nominal rewriting nominal rewriting is based on the observation that if we add support for a equivalence to first order syntax using the nominal set approach then systems with binding including higher order reduction schemes such as l calculus beta reduction can be smoothly represented nominal rewriting maintains a strict distinction between variables of the object language atoms and of the meta language variables or unknowns atoms may be bound by a special abstraction operation but variables cannot be bound giving the framework a pronounced first order character since substitution of terms for variables is not capture avoiding we show how good properties of first order rewriting survive the extension by giving an efficient rewriting algorithm a critical pair lemma and a confluence theorem for orthogonal systems

universal multiparty data exchange and secret key agreement multiple parties observing correlated data seek to recover each other s data and attain omniscience to that end they communicate interactively over a noiseless broadcast channel each bit transmitted over this channel is received by all the parties we give a universal interactive communication protocol termed the recursive data exchange protocol rde which attains omniscience for any sequence of data observed by the parties and provide an individual sequence guarantee of performance as a by product for observations of length n we show the universal rate optimality of rde up to an mathcal o n 1 2 sqrt log n term in a generative setting where the data sequence is independent and identically distributed in time furthermore drawing on the duality between omniscience and secret key agreement due to csiszar and narayan we obtain a universal protocol for generating a multiparty secret key of rate at most mathcal o n 1 2 sqrt log n less than the maximum rate possible a key feature of rde is its recursive structure whereby when a subset a of parties recover each other s data the rates appear as if the parties have been executing the protocol in an alternative model where the parties in a are collocated

on lossy source coding with side information under the erasure distortion measure we consider the wyner ziv and two way source coding problems with the erasure distortion measure we characterize the rate distortion regions for these settings when the source and side information satisfy a positivity condition using our results we show that contrary to recent conjectures in the literature feedback from the decoder to the encoder does not reduce the overall rate required in the wyner ziv setting with erasure distortion when the positivity condition is satisfied finally we extend our techniques to characterize the rate distortion regions of two multiterminal source coding settings the heegard berger setting and the cascade source coding setting when the positivity condition is satisfied

on distributed computing for functions with certain structures the problem of distributed function computation is studied where functions to be computed is not necessarily symbol wise a new method to derive a converse bound for distributed computing is proposed from the structure of functions to be computed information that is inevitably conveyed to the decoder is identified and the bound is derived in terms of the optimal rate needed to send that information the class of informative functions is introduced and for the class of smooth sources the optimal rate for computing those functions is characterized furthermore for i i d sources with joint distribution that may not be full support functions that are composition of symbol wise function and the type of a sequence are considered and the optimal rate for computing those functions is characterized in terms of the hypergraph entropy as a byproduct our method also provides a conceptually simple proof of the known fact that computing a boolean function may require as large rate as reproducing the entire source

common randomness in information theory and cryptography i secret sharing as the first part of a study of problems involving common randomness at distance locations information theoretic models of secret sharing generating a common random key at two terminals without letting an eavesdropper obtain information about this key are considered the concept of key capacity is defined single letter formulas of key capacity are obtained for several models and bounds to key capacity are derived for other models

distributed lossy averaging in this paper an information theoretic formulation of the distributed averaging problem previously studied in computer science and control is presented we assume a network with m nodes each observing a white gaussian noise wgn source the nodes communicate and perform local processing with the goal of computing the average of the sources to within a prescribed mean squared error distortion the network rate distortion function r d for a two node network with correlated gaussian sources is established a general cutset lower bound on r d is established and shown to be achievable to within a factor of 2 via a centralized protocol over a star network a lower bound on the network rate distortion function for distributed weighted sum protocols which is larger in order than the cutset bound by a factor of log m is established an upper bound on the network rate distortion function for gossip base weighted sum protocols which is only log log m larger in order than the lower bound for a complete graph network is established the results suggest that using distributed protocols results in a factor of log m increase in order relative to centralized protocols

converses for secret key agreement and secure computing we consider information theoretic secret key sk agreement and secure function computation by multiple parties observing correlated data with access to an interactive public communication channel our main result is an upper bound on the sk length which is derived using a reduction of binary hypothesis testing to multiparty sk agreement building on this basic result we derive new converses for multiparty sk agreement furthermore we derive converse results for the oblivious transfer problem and the bit commitment problem by relating them to sk agreement finally we derive a necessary condition for the feasibility of secure computation by trusted parties that seek to compute a function of their collective data using an interactive public communication that by itself does not give away the value of the function in many cases we strengthen and improve upon previously known converse bounds our results are single shot and use only the given joint distribution of the correlated observations for the case when the correlated observations consist of independent and identically distributed in time sequences we derive strong versions of previously known converses

secret key generation using correlated sources and channels we study the secret key capacity in a joint source channel coding setup the terminals are connected over a discrete memoryless channel and have access to side information modelled as a pair of discrete memoryless source sequences as our main result we establish the upper and lower bounds on the secret key capacity in the lower bound expression the equivocation terms of the source and channel components are functionally additive even though the coding scheme generates a single secret key by jointly taking into account the source and channel equivocations our bounds coincide thus establishing the capacity when the underlying wiretap channel can be decomposed into a set of independent parallel and reversely degraded channels for the case of parallel gaussian channels and jointly gaussian sources we show that gaussian codebooks achieve the secret key capacity in addition when the eavesdropper also observes a correlated side information sequence we establish the secret key capacity when both the source and channel of the eavesdropper are a degraded version of the legitimate receiver we finally also treat the case when a public discussion channel is available propose a separation based coding scheme and establish its optimality when the channel output symbols of the legitimate receiver and eavesdropper are conditionally independent given the input

security analysis of varepsilon almost dual universal 2 hash functions smoothing of min entropy versus smoothing of renyi entropy of order 2 recently varepsilon almost dual universal2 hash functions have been proposed as a new and wider class of hash functions using this class of hash functions several efficient hash functions were proposed this paper evaluates the security performance when we apply this kind of hash functions we evaluate the security in several kinds of setting based on the l 1 distinguishability criterion and the modified mutual information criterion the obtained evaluation is based on smoothing of renyi entropy of order 2 and or min entropy we clarify the difference between these two methods

an algorithmic introduction to numerical simulation of stochastic differential equations a practical and accessible introduction to numerical methods for stochastic differential equations is given the reader is assumed to be familiar with euler s method for deterministic differential equations and to have at least an intuitive feel for the concept of a random variable however no knowledge of advanced probability theory or stochastic processes is assumed the article is built around 10 matlab programs and the topics covered include stochastic integration the euler maruyama method milstein s method strong and weak convergence linear stability and the stochastic chain rule

on the public communication needed to achieve sk capacity in the multiterminal source model the focus of this paper is on the public communication required for generating a maximal rate secret key sk within the multiterminal source model of csiszar and narayan building on the prior work of tyagi for the two terminal scenario we derive a lower bound on the communication complexity r text sk defined to be the minimum rate of public communication needed to generate a maximal rate sk it is well known that the minimum rate of communication for omniscience denoted by r text co is an upper bound on r text sk for the class of pairwise independent network pin models defined on uniform hypergraphs we show that a certain type mathcal s condition which is verifiable in polynomial time guarantees that our lower bound on r text sk meets the r text co upper bound thus the pin models satisfying our condition are r text sk maximal indicating that the upper bound r text sk le r text co holds with equality this allows us to explicitly evaluate r text sk for such pin models we also give several examples of pin models that satisfy our type mathcal s condition finally we prove that for an arbitrary multiterminal source model a stricter version of our type mathcal s condition implies that communication from all terminals omnivocality is needed for establishing an sk of maximum rate for three terminal source models the converse is also true omnivocality is needed for generating a maximal rate sk only if the strict type mathcal s condition is satisfied however for the source models with four or more terminals counterexamples exist showing that the converse does not hold in general

distributed rate distortion with common components we describe a scheme for rate distortion with distributed encoding in which the sources to be compressed contain a common component we show that this scheme is optimal in some situations and that it strictly improves upon existing schemes which do not make full use of common components this establishes that independent quantization followed by independent binning is not optimal for the two encoder problem with a distortion constraint on one source we also show that independent quantization and binning is suboptimal for the three encoder problem in which the goal is to reproduce one of the sources losslessly this provides a counterexample that is fundamentally different from one provided earlier by korner and marton the proofs rely on the binary analogue of the entropy power inequality and the existence of a rate loss for the binary symmetric wyner ziv problem

computation over gaussian networks with orthogonal components function computation over gaussian networks with orthogonal components is studied for arbitrarily correlated discrete memoryless sources two classes of functions are considered 1 the arithmetic sum function and 2 the type function the arithmetic sum function in this paper is defined as a set of multiple weighted arithmetic sums which includes averaging of the sources and estimating each of the sources as special cases the type or frequency histogram function counts the number of occurrences of each argument which yields various fundamental statistics such as mean variance maximum minimum median and so on the proposed computation coding first abstracts gaussian networks into the corresponding modulo sum multiple access channels via nested lattice codes and linear network coding and then computes the desired function using linear slepian wolf source coding for orthogonal gaussian networks with no broadcast and multiple access components the computation capacity is characterized for a class of networks for gaussian networks with multiple access components but no broadcast an approximate computation capacity is characterized for a class of networks

secret key generation with limited interaction a basic two terminal secret key generation model is considered where the interactive communication rate between the terminals may be limited and in particular may not be enough to achieve the maximum key rate we first prove a multi letter characterization of the key communication rate region where the number of auxiliary random variables depends on the number of rounds of the communication and then provide an equivalent but simpler characterization in terms of concave envelopes in the case of unlimited number of rounds two extreme cases are given special attention first in the regime of very low communication rates the key bits per interaction bit kbib is expressed with a new symmetric strong data processing constant which has a concave envelope characterization analogous to that of the conventional strong data processing constant the symmetric strong data processing constant can be upper bounded by the supremum of the maximal correlation coefficient over a set of distributions which allows us to determine the kbib for binary symmetric sources and conclude in particular that the interactive scheme is not more efficient than the one way scheme at least in the low communication rate regime second a new characterization of the minimum interaction rate needed for achieving the maximum key rate mimk is given and we resolve a conjecture by tyagi regarding the mimk for possibly nonsymmetric binary sources we also propose a new conjecture for binary symmetric sources that the interactive scheme is not more efficient than the one way scheme at any communication rate

cascade triangular and two way source coding with degraded side information at the second user in this paper we consider the cascade and triangular rate distortion problems where the same side information is available at the source node and user 1 and the side information available at user 2 is a degraded version of the side information at the source node and user 1 we characterize the rate distortion region for these problems for the cascade setup we show that at user 1 decoding and rebinning the codeword sent by the source node for user 2 is optimum we then extend our results to the two way cascade and triangular setting where the source node is interested in lossy reconstruction of the side information at user 2 via a rate limited link from user 2 to the source node we characterize the rate distortion regions for these settings complete explicit characterizations for all settings are given in the quadratic gaussian case we conclude with two further extensions a triangular source coding problem with a helper and an extension of our two way cascade setting in the quadratic gaussian case

function computation through a bidirectional relay we consider a function computation problem in a three node wireless network nodes a and b observe two correlated sources x and y respectively and want to compute a function f x y to achieve this nodes a and b send messages to a relay node c at rates r a and r b respectively the relay c then broadcasts a message to a and b at rate r c we allow block coding and study the achievable region of rate triples under both zero error and epsilon error as a preparation we first consider a broadcast network from the relay to a and b a and b have side information x and y respectively the relay node c observes both x and y and broadcasts an encoded message to a and b we want to obtain the optimal broadcast rate such that a and b can recover the function f x y from the received message and their individual side information x and y respectively for this problem we show equivalence between epsilon error and zero error computations this gives a rate characterization for zero error computation as a corollary this also gives a rate characterization for the relay network under zero error for a class of functions called component wise one to one functions when the support set of p xy is full for the relay network the zero error rate region for arbitrary functions is characterized in terms of graph coloring of some suitably defined probabilistic graphs we then give a single letter inner bound to this rate region furthermore we extend the graph theoretic ideas to address the epsilon error problem and obtain a single letter inner bound

two constructions on limits of entropy functions the correspondence between the subvectors of a random vector and their shannon entropies gives rise to an entropy function limits of the entropy functions are closed to convolutions with modular polymatroids and when integer valued also to free expansions the problem of description of the limits of entropy functions is reduced to those limits that correspond to matroids related results on entropy functions are reviewed with regard to polymatroid and matroid theories and perfect and ideal secret sharing

two way source coding with a helper consider the two way rate distortion problem in which a helper sends a common limited rate message to both users based on side information at its disposal we characterize the region of achievable rates and distortions when the markov relation helper user 1 user 2 holds the main insight of the result is that in order to achieve the optimal rate the helper may use a binning scheme as in wyner ziv where the side information at the decoder is the further user namely user 2 we derive these regions explicitly for the gaussian sources with square error distortion analyze a tradeoff between the rate from the helper and the rate from the source and examine a special case where the helper has the freedom to send different messages at different rates to the encoder and the decoder the converse proofs use a technique for verifying markov relations via undirected graphs

cascade and triangular source coding with side information at the first two nodes we consider the cascade and triangular rate distortion problem where side information is known to the source encoder and to the first user but not to the second user we characterize the rate distortion region for these problems as well as some of their extensions for the quadratic gaussian case we show that it is sufficient to consider jointly gaussian distributions which leads to an explicit solution

multi key generation over a cellular model with a helper the problem of simultaneously generating multiple keys for a cellular source model with a helper is investigated in the model considered there are four terminals mathcal x 0 mathcal x 1 mathcal x 2 and mathcal x 3 each of which observes one component of a vector source terminal mathcal x 0 wishes to generate two secret keys k 1 and k 2 respectively with terminals mathcal x 1 and mathcal x 2 under the help of terminal mathcal x 3 all terminals are allowed to communicate over a public channel an eavesdropper is assumed to have access to the public discussion both symmetric and asymmetric key generations are considered in symmetric key generation models model 1a with a trusted helper requires that the two keys are concealed from the eavesdropper and model 1b with an untrusted helper further requires that the two keys are concealed from the helper in addition to the eavesdropper the asymmetric key generation models 2a and 2b are the same as symmetric key generation models 1a and 1b respectively except that the key k 2 is further required to be concealed from terminal mathcal x 1 for all models studied the key capacity region is established by designing a unified achievable strategy to achieve the cut set outer bounds we also study the problem of generating more than two keys and characterize its key capacity region when all the cellular terminals are required to generate independent keys with the base station

rate distortion for correlated sources with partially separated encoders we resolve some open problems concerning the encoding of a pair of correlated sources with respect to a fidelity criterion two encoders are used one encoder observes only the output of the first source while the other is supplied both with the second source output and with partial information about the first source at some predetermined rate a general coding theorem is proved which established that cal s ast a certain region defined in terms of single letter information theoretic quantities is an inner bound to the region of all attainable vectors of rates and distortions for certain cases of interest the converse is proved too thereby establishing the rate distortion region for these cases

higher order rewrite systems and their confluence abstract we study higher order rewrite systems hrss which extend term rewriting to terms hrss can describe computations over terms with bound variables we show that rewriting with hrss is closely related to undirected equational reasoning we define pattern rewrite systems prss as a special case of hrss and extend three confluence results from term rewriting to prss the critical pair lemma by knuth and bendix confluence of rewriting modulo equations a la huet and confluence of orthogonal prss

entropy amplification property and the loss for writing on dirty paper costa s celebrated ldquowriting on dirty paperrdquo wdp shows that the power constrained channel y x s z with gaussian z has the same capacity as the standard awgn channel y x z provided that the ldquointerferencerdquo s no matter how strong it is is known at the transmitter while this ability for perfect interference cancelation is very appealing it relies heavily on the gaussianity of the unknown noise z we construct an example of ldquobadrdquo noise for writing on dirty paper namely ldquodifference set noiserdquo if the interference s is strong then difference set noise limits the wdp capacity to at most 2 bits at the same time like in the awgn case the zero interference capacity grows without bound with the input constraint thus almost 100 of the available capacity is lost in wdp in the presence of difference set noise this high capacity loss is due to the ldquoentropy amplification propertyrdquo eap of noise with an aperiodic probability distribution using the eap and the duality between wdp and wyner ziv source coding we also give an example of dramatic rate loss in quantizing encrypted source

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

on network coding for sum networks a directed acyclic network is considered where all the terminals need to recover the sum of the symbols generated at all the sources we call such a network a sum network it is shown that there exists a solvably and linear solvably equivalent sum network for any multiple unicast network and thus for any directed acyclic communication network it is also shown that there exists a linear solvably equivalent multiple unicast network for every sum network it is shown that for any set of polynomials having integer coefficients there exists a sum network which is scalar linear solvable over a finite field f if and only if the polynomials have a common root in f for any finite or cofinite set of prime numbers a network is constructed which has a vector linear solution of any length if and only if the characteristic of the alphabet field is in the given set the insufficiency of linear net work coding and unachievability of the network coding capacity are proved for sum networks by using similar known results for communication networks under fractional vector linear network coding a sum network and its reverse network are shown to be equivalent however under nonlinear coding it is shown that there exists a solvable sum network whose reverse network is not solvable

secret key capacity for optimally correlated sources under sampling attack the capacity for secret key agreement for permutation invariant and symmetric sources under a sampling attack is investigated the supremum of the normalized secret key capacity is introduced where the supremum is taken over all permutation invariant sources or all symmetric sources and the normalized secret key capacity is the secret key capacity divided by the description length of the symbol it is proved that the supremum of the normalized secret key capacity bound under a sampling attack is close to 1 m for permutation invariant sources and o 1 m for symmetric sources where and m is the number of eve s sources

robust stability and instability of biochemical networks with parametric uncertainty parameter perturbations in dynamical models of biochemical networks affect the qualitative dynamical behaviour observed in the model since this qualitative behaviour is in many cases the key model output used to explain biological function the robustness analysis of the model s behaviour with respect to parametric uncertainty is a crucial step in systems biology research in this paper we develop a new method for robustness analysis of the dynamical behaviour as a first step we provide a characterization of non robust perturbations as a system of polynomial equalities and inequalities in the second step we apply the positivstellensatz and handelman representation of polynomials to check for the non existence of solutions to this system which can be relaxed to solving a linear program thereby a solution to the linear program yields a robustness certificate for the considered dynamical behaviour with these robustness certificates we propose an algorithm to compute a lower robustness bound corresponding to a level of parametric uncertainty up to which no local bifurcations can occur the applicability of the proposed method to biochemical network models is illustrated by analysing the robustness of oscillations in a model of the nf kb signalling pathway the results may be used to define a level of confidence in the observed model behaviour under parametric uncertainty making them valuable for evaluating dynamical models of biological networks

compressing encrypted data achieving optimality and strong secrecy via permutations in a system that performs both encryption and lossy compression the conventional way is to compress first and then encrypt the compressed data this separation approach has been proved to be optimal in certain applications where sensitive information should be protected as early as possible it is preferable to perform the encryption first and then compress the encrypted data which leads to the concept of the reversed system johnson et al proposed an achievable scheme for the reversed system where a modulo sum encryption is followed by a compression using the wyner ziv distributed source coding with side information however in general this reversed system performs worse than the conventional system in the sense that it requires more compression rate and secrecy key rate in this paper we propose a new achievable scheme for the reversed system where the encryption is conducted by a permutation cipher and then the encrypted data is compressed using the optimal rate distortion code the proposed scheme can achieve the optimal compression rate and secret key rate as a result we show that reversing the order of the encryption and compression does not necessarily compromise the performance of an encryption compression system we show that the proposed system attains strong secrecy and the information leakage vanishes exponentially

support recovery for the drift coefficient of high dimensional diffusions consider the problem of learning the drift coefficient of a p dimensional stochastic differential equation from a sample path of length t we assume that the drift is parametrized by a high dimensional vector and study the support recovery problem when both p and t can tend to infinity in particular we prove a general lower bound on the sample complexity t by using a characterization of mutual information as a time integral of conditional variance due to kadota zakai and ziv for linear stochastic differential equations the drift coefficient is parametrized by a p times p matrix that describes which degrees of freedom interact under the dynamics in this case we analyze a ell 1 regularized least squares estimator and prove an upper bound on t that nearly matches the lower bound on specific classes of sparse matrices

a dichotomy of functions f x y of correlated sources x y consider separate encoding of correlated sources x n x l cdots x n y n y l cdots y n for the decoder to reliably reproduce a function f x i y i n i 1 we establish the necessary and sufficient condition for the set of all achievable rates to coincide with the slepian wolf region whenever the probability density p x y is positive for all x y

a dichotomy of functions in distributed coding an information spectral approach the problem of distributed data compression for function computation is considered where 1 the function to be computed is not necessarily symbolwise function and 2 the information source has memory and may not be stationary nor ergodic we introduce the class of smooth sources and give a sufficient condition on functions so that the achievable rate region for computing coincides with the slepian wolf region i e the rate region for reproducing the entire source for any smooth sources moreover for symbolwise functions the necessary and sufficient condition for the coincidence is established our result for the full side information case is a generalization of the result by ahlswede and csiszar to sources with memory our dichotomy theorem is different from han and kobayashi s dichotomy theorem which reveals an effect of memory in distributed function computation all results are given not only for fixed length coding but also for variable length coding in a unified manner furthermore for the full side information case the error probability in the moderate deviation regime is also investigated

secret key agreement general capacity and second order asymptotics we revisit the problem of secret key agreement using interactive public communication for two parties and propose a new secret key agreement protocol the protocol attains the secret key capacity for general observations and attains the second order asymptotic term in the maximum length of a secret key for independent and identically distributed observations in contrast to the previously suggested secret key agreement protocols the proposed protocol uses interactive communication in fact the standard one way communication protocol used prior to this paper fails to attain the asymptotic results above our converse proofs rely on a recently established upper bound for secret key lengths both our lower and upper bounds are derived in a single shot setup and the asymptotic results are obtained as corollaries

duality of a source coding problem and the semi deterministic broadcast channel with rate limited cooperation the wyner ahlswede korner wak empirical coordination problem where the encoders cooperate via a finite capacity one sided link is considered the coordination capacity region is derived by combining several source coding techniques such as wyner ziv coding binning and superposition coding furthermore a semi deterministic sd broadcast channel bc with one sided decoder cooperation is considered duality principles relating the two problems are presented and the capacity region for the sd bc setting is derived the direct part follows from an achievable region for a general bc that is tight for the sd scenario a converse is established by using telescoping identities the sd bc is shown to be operationally equivalent to a class of relay bcs and the correspondence between their capacity regions is established the capacity region of the sd bc is transformed into an equivalent region that is shown to be dual to the admissible region of the wak problem in the sense that the information measures defining the corner points of both regions coincide achievability and converse proofs for the equivalent region are provided for the converse we use a probabilistic construction of auxiliary random variables that depends on the distribution induced by the codebook several examples illustrate the results

some results on distributed source coding for interactive function computation a two terminal interactive distributed source coding problem with alternating messages for function computation at both locations is studied for any number of messages a computable characterization of the rate region is provided in terms of single letter information measures while interaction is useless in terms of the minimum sum rate for lossless source reproduction at one or both locations the gains can be arbitrarily large for function computation even when the sources are independent for a class of sources and functions interaction is shown to be useless even with infinite messages when a function has to be computed at only one location but is shown to be useful if functions have to be computed at both locations for computing the boolean and function of two independent bernoulli sources at both locations an achievable infinite message sum rate with infinitesimal rate messages is derived in terms of a 2 d definite integral and a rate allocation curve the benefit of interaction is highlighted in multiterminal function computation problem through examples for networks with a star topology multiple rounds of interactive coding is shown to decrease the scaling law of the total network rate by an order of magnitude as the network grows

channel coding and lossy source coding using a generator of constrained random numbers stochastic encoders for channel coding and lossy source coding are introduced with a rate close to the fundamental limits where the only restriction is that the channel input alphabet and the reproduction alphabet of the lossy source code are finite random numbers which satisfy a condition specified by a function and its value are used to construct stochastic encoders the proof of the theorems is based on the hash property of an ensemble of functions where the results are extended to general channels sources and alternative formulas are introduced for channel capacity and the rate distortion region since an ensemble of sparse matrices has a hash property we can construct a code by using sparse matrices

interactive source coding for function computation in collocated networks a problem of interactive function computation in a collocated network is studied in a distributed block source coding framework with the goal of computing samples of a desired function of sources at the sink the source nodes exchange messages through a sequence of error free broadcasts for any function of independent sources a computable characterization of the set of all feasible message coding rates the rate region is derived in terms of single letter information measures in the limit as the number of messages tends to infinity the infinite message minimum sum rate viewed as a functional of the joint source probability mass function is characterized as the least element of a partially ordered family of functionals having certain convex geometric properties this characterization leads to a family of lower bounds for the infinite message minimum sum rate and a simple criterion to test the optimality of any achievable infinite message sum rate an iterative algorithm for evaluating the infinite message minimum sum rate functional is proposed and is demonstrated through an example of computing the minimum function of three bernoulli sources based on the characterizations of the rate regions it is shown that when computing symmetric functions of binary sources the sink will inevitably learn certain additional information that is not demanded in computing the function this conceptual understanding leads to new improved bounds for the minimum sum rate the new bounds are shown to be orderwise better than those based on cut sets as the network scales the scaling law of the minimum sum rate is explored for different classes of symmetric functions and source parameters

distributed scalar quantization for computing high resolution analysis and extensions communication of quantized information is frequently followed by a computation we consider situations of distributed functional scalar quantization distributed scalar quantization of possibly correlated sources followed by centralized computation of a function under smoothness conditions on the sources and function companding scalar quantizer designs are developed to minimize mean squared error mse of the computed function as the quantizer resolution is allowed to grow striking improvements over quantizers designed without consideration of the function are possible and are larger in the entropy constrained setting than in the fixed rate setting as extensions to the basic analysis we characterize a large class of functions for which regular quantization suffices consider certain functions for which asymptotic optimality is achieved without arbitrarily fine quantization and allow limited collaboration between source encoders in the entropy constrained setting a single bit per sample communicated between encoders can have an arbitrarily large effect on functional distortion in contrast such communication has very little effect in the fixed rate setting

pnl to hol from the logic of nominal sets to the logic of higher order functions permissive nominal logic pnl extends first order predicate logic with term formers that can bind names in their arguments it takes a semantics in permissive nominal sets in pnl the quantifier or l binder are just term formers satisfying axioms and their denotation is functions on nominal atoms abstraction then we have higher order logic hol and its models in ordinary i e zermelo fraenkel sets the denotation of or l is functions on full or partial function spaces this raises the following question how are these two models of binding connected what translation is possible between pnl and hol and between nominal sets and functions we exhibit a translation of pnl into hol and from models of pnl to certain models of hol it is natural but also partial we translate a restricted subsystem of full pnl to hol the extra part which does not translate is the symmetry properties of nominal sets with respect to permutations to use a little nominal jargon we can translate names and binding but not their nominal equivariance properties this seems reasonable since hol and ordinary sets are not equivariant thus viewed through this translation pnl and hol and their models do different things but they enjoy non trivial and rich subsystems which are isomorphic

the infinite message limit of two terminal interactive source coding a two terminal interactive function computation problem with alternating messages is studied within the framework of distributed block source coding theory for any finite number of messages a single letter characterization of the sum rate distortion function was established in previous works using standard information theoretic techniques this however does not provide a satisfactory characterization of the infinite message limit which is a new unexplored dimension for asymptotic analysis in distributed block source coding involving potentially an infinite number of infinitesimal rate messages in this paper the infinite message sum rate distortion function viewed as a functional of the joint source distribution and the distortion levels is characterized as the least element of a partially ordered family of functionals having certain convex geometric properties the new characterization does not involve evaluating the infinite message limit of a finite message sum rate distortion expression this characterization leads to a family of lower bounds for the infinite message sum rate distortion expression and a simple criterion to test the optimality of any achievable infinite message sum rate distortion expression the new convex geometric characterization is used to develop an iterative algorithm for evaluating any finite message sum rate distortion function it is also used to construct the first examples which demonstrate that for lossy source reproduction two messages can strictly improve the one message wyner ziv rate distortion function settling an unresolved question from a 1985 paper it is shown that a single backward message of arbitrarily small rate can lead to an arbitrarily large gain in the sum rate

estimation with a helper who knows the interference we consider the problem of estimating a signal corrupted by independent interference with the assistance of a cost constrained helper who knows the interference causally or noncausally when the interference is known causally we characterize the minimum distortion incurred in estimating the desired signal in the noncausal case we present a general achievable scheme for discrete memoryless systems and novel lower bounds on the distortion for the binary and gaussian settings our gaussian setting coincides with that of assisted interference suppression introduced by grover and sahai our lower bound for this setting is based on the relation recently established by verdu between divergence and minimum mean squared error we illustrate with a few examples that this lower bound can improve on those previously developed our bounds also allow us to characterize the optimal distortion in several interesting regimes moreover we show that causal and noncausal estimation are not equivalent for this problem finally we consider the case where the desired signal is also available at the helper we develop new lower bounds for this setting that improve on those previously developed and characterize the optimal distortion up to a constant multiplicative factor for some regimes of interest

secret key and private key constructions for simple multiterminal source models we propose an approach for constructing secret and private keys based on the long known slepian wolf code due to wyner for correlated sources connected by a virtual additive noise channel our work is motivated by results of csiszar and narayan which highlight innate connections between secrecy generation by multiple terminals that observe correlated source signals and slepian wolf near lossless data compression explicit procedures for such constructions and their substantiation are provided the performance of low density parity check channel codes in devising a new class of secret keys is examined

secret key agreement by public discussion from common information the problem of generating a shared secret key s by two parties knowing dependent random variables x and y respectively but not sharing a secret key initially is considered an enemy who knows the random variable z jointly distributed with x and y according to some probability distribution p sub xyz can also receive all messages exchanged by the two parties over a public channel the goal of a protocol is that the enemy obtains at most a negligible amount of information about s upper bounds on h s as a function of p sub xyz are presented lower bounds on the rate h s n as n to infinity are derived for the case in which x x sub 1 x sub n y y sub 1 y sub n and z z sub 1 z sub n result from n independent executions of a random experiment generating x sub i y sub i and z sub i for i 1 n it is shown that such a secret key agreement is possible for a scenario in which all three parties receive the output of a binary symmetric source over independent binary symmetric channels even when the enemy s channel is superior to the other two channels

demystifying fixed k nearest neighbor information estimators estimating mutual information from independent identically distributed samples drawn from an unknown joint density function is a basic statistical problem of broad interest with multitudinous applications the most popular estimator is the one proposed by kraskov stogbauer and grassberger ksg in 2004 and is nonparametric and based on the distances of each sample to its k mathrm th nearest neighboring sample where k is a fixed small integer despite of its widespread use part of scientific software packages theoretical properties of this estimator have been largely unexplored in this paper we demonstrate that the estimator is consistent and also identify an upper bound on the rate of convergence of the ell 2 error as a function of a number of samples we argue that the performance benefits of the ksg estimator stems from a curious correlation boosting effect and build on this intuition to modify the ksg estimator in novel ways to construct a superior estimator as a by product of our investigations we obtain nearly tight rates of convergence of the ell 2 error of the well known fixed k nearest neighbor estimator of differential entropy by kozachenko and leonenko

common randomness in information theory and cryptography ii cr capacity for pt i see ibid vol 39 p 1121 1993 the common randomness cr capacity of a two terminal model is defined as the maximum rate of common randomness that the terminals can generate using resources specified by the given model we determine cr capacity for several models including those whose statistics depend on unknown parameters the cr capacity is shown to be achievable robustly by common randomness of nearly uniform distribution no matter what the unknown parameters are our cr capacity results are relevant for the problem of identification capacity and also yield a new result on the regular transmission capacity of arbitrarily varying channels with feedback

perfect omniscience perfect secrecy and steiner tree packing we consider perfect secret key generation for a pairwise independent network model in which every pair of terminals share a random binary string with the strings shared by distinct terminal pairs being mutually independent the terminals are then allowed to communicate interactively over a public noiseless channel of unlimited capacity all the terminals as well as an eavesdropper observe this communication the objective is to generate a perfect secret key shared by a given set of terminals at the largest rate possible and concealed from the eavesdropper first we show how the notion of perfect omniscience plays a central role in characterizing perfect secret key capacity second a multigraph representation of the underlying secrecy model leads us to an efficient algorithm for perfect secret key generation based on maximal steiner tree packing this algorithm attains capacity when all the terminals seek to share a key and in general attains at least half the capacity third when a single helper terminal assists the remaining user terminals in generating a perfect secret key we give necessary and sufficient conditions for the optimality of the algorithm also a weak helper is shown to be sufficient for optimality

secret key agreement over non coherent block fading channels with public discussion motivated by recent interest in physical layer secret key generation over wireless fading channels we study the non coherent secret key generation capacity of a block fading wireless channel with channel reciprocity and bi directional two way communication we assume a non coherent main channel i e the realization of channel gains on the main channel is not known to any terminal the eavesdropper is assumed to have both perfect channel state information of its own channel and orthogonal observations from the forward and backward channels as our main result we establish new upper and lower bounds on the secret key generation capacity with public discussion which are structurally similar the upper bound can be expressed as a sum of three terms one of the terms arises due to channel reciprocity while the other two terms correspond to communication on the forward and backward channels respectively in the limit of long coherence period the contribution from channel reciprocity vanishes to zero whereas the other terms prevail the lower bound is based on a separation based scheme in each coherence block we use the first symbol for training while the remainder of the coherence block is devoted to source emulation i e to generate correlated sources between the terminals the lower bound expression also consists of three terms and admits an interpretation similar to the upper bound expression for rayleigh fading channels in the high signal to noise ratio snr regime the gap between the upper and lower bounds is shown to vanish inversely with the coherence period numerical results indicate significant performance gains over training only schemes even for moderate values of snr and small coherence periods

equivalence of certain entropy contraction coefficients abstract the entropy contraction coefficient associated with the convex function g and column stochastic matrix a is defined as g a sup x y h g ax ay h g x y where h g x y denotes the relative g entropy of the probability vectors x and y we show that for each fixed a all entropy contraction coefficients associated with convex operator functions are equivalent in particular log a w 2 a a where the subscripts log and w 2 refer respectively to the usual logarithmic relative entropy and to the relative g entropy for the function g w w 2 we also give a useful new characterization of w 2 a

quasi structured codes for multi terminal communications a new class of structured codes called quasi group codes qgcs is introduced a qgc is a subset of a group code in contrast with the group codes qgcs are not closed under group addition the parameters of the qgc can be chosen such that the size of mathcal c mathcal c is equal to any number between mathcal c and mathcal c 2 we analyze the performance of a specific class of qgcs this class of qgcs is constructed by assigning single letter distributions to the indices of the codewords in a group code then the qgc is defined as the set of codewords whose index is in the typical set corresponding to these single letter distributions the asymptotic performance limits of this class of qgcs are characterized using single letter information quantities corresponding covering and packing bounds are derived it is shown that the point to point channel capacity and optimal rate distortion function are achievable using qgcs coding strategies based on qgcs are introduced for three fundamental multi terminal problems the korner marton problem for modulo prime power sums computation over the multiple access channel mac and mac with distributed states for each problem a single letter achievable rate region is derived it is shown through examples that the coding strategies improve upon the previous strategies based on the unstructured codes linear codes and group codes

lattices for distributed source coding jointly gaussian sources and reconstruction of a linear function consider a pair of correlated gaussian sources x 1 x 2 two separate encoders observe the two components and communicate compressed versions of their observations to a common decoder the decoder is interested in reconstructing a linear combination of x 1 and x 2 to within a mean square distortion of d we obtain an inner bound to the optimal rate distortion region for this problem a portion of this inner bound is achieved by a scheme that reconstructs the linear function directly rather than reconstructing the individual components x 1 and x 2 first this results in a better rate region for certain parameter values our coding scheme relies on lattice coding techniques in contrast to more prevalent random coding arguments used to demonstrate achievable rate regions in information theory we then consider the case of linear reconstruction of k sources and provide an inner bound to the optimal rate distortion region some parts of the inner bound are achieved using the following coding structure lattice vector quantization followed by ldquocorrelatedrdquo lattice structured binning

on channel capacity per unit cost memoryless communication channels with arbitrary alphabets where each input symbol is assigned a cost are considered the maximum number of bits that can be transmitted reliably through the channel per unit cost is studied it is shown that if the input alphabet contains a zero cost symbol then the capacity per unit cost admits a simple expression as the maximum normalized divergence between two conditional output distributions the direct part of this coding theorem admits a constructive proof via stein s lemma on the asymptotic error probability of binary hypothesis tests single user multiple access and interference channels are studied

modeling and simulating chemical reactions many students are familiar with the idea of modeling chemical reactions in terms of ordinary differential equations however these deterministic reaction rate equations are really a certain large scale limit of a sequence of finer scale probabilistic models in studying this hierarchy of models students can be exposed to a range of modern ideas in applied and computational mathematics this article introduces some of the basic concepts in an accessible manner and points to some challenges that currently occupy researchers in this area short downloadable matlab codes are listed and described

key generation algorithms for pairwise independent networks based on graphical models we consider two secret key generation problems under a pairwise independent network model and propose low complexity key generation schemes in a framework that connects our problems to network flow problems in graphs our schemes have two components 1 local key generation and 2 global key propagation in the local key generation we use point to point source coding with side information to establish pairwise keys from which we construct a graph with the capacity of each edge being the key rate of the corresponding point to point local key in the global key propagation depending on the particular problem secret keys are delivered to users in the network using various network flow algorithms in particular in the first problem in which one is required to generate a group key for a group of users in the network we propose a network coding based global key propagation approach this approach has a low complexity and has a better performance than the existing approach in the second problem in which one is required to generate multiple keys simultaneously for different pairs of users we propose a multicommodity flow based global key propagation approach we show that the proposed approach is optimal for the case of generating two keys for the general case of generating more than two keys we show that the sum rate of the proposed scheme is larger than an upper bound characterized in this paper divided by a constant

causal state communication the problem of state communication over a discrete memoryless channel with discrete memoryless state is studied when the state information is available strictly causally at the encoder it is shown that block markov encoding in which the encoder communicates a description of the state sequence in the previous block by incorporating side information about the state sequence at the decoder yields the minimum state estimation error when the same channel is used to send additional independent information at the expense of a higher channel state estimation error the optimal tradeoff between the rate of the independent information and the state estimation error is characterized via the capacity distortion function it is shown that any optimal tradeoff pair can be achieved via rate splitting these coding theorems are then extended optimally to the case of causal channel state information at the encoder using the shannon strategy

how many queries will resolve common randomness a set of m terminals observing correlated signals communicate interactively to generate common randomness for a given subset of them knowing only the communication how many direct queries of the value of the common randomness will resolve it a general upper bound valid for arbitrary signal alphabets is developed for the number of such queries by using a query strategy that applies to all common randomness and associated communication when the underlying signals are independent and identically distributed repetitions of m correlated random variables the number of queries can be exponential in signal length for this case the mentioned upper bound is tight and leads to a single letter formula for the largest query exponent which coincides with the secret key capacity of a corresponding multiterminal source model in fact the upper bound constitutes a strong converse for the optimum query exponent and implies also a new strong converse for secret key capacity a key tool estimating the size of a large probability set in terms of renyi entropy is interpreted separately too as a lossless block coding result for general sources as a particularization it yields the classic result for a discrete memoryless source

worst case interactive communication ii two messages are not optimal for pt i see ibid vol 36 no 5 p 1111 26 1990 the author defines the chromatic decomposition number of a hypergraph and shows that under general conditions it determines the two message complexity this result is then used to provide that two messages are not optimal protocols complexities and the characteristic hypergraph of x y are defined the playoffs problem is described although similar in appearance to the league problem given in an example it is shown that its two message complexity is about twice as high as its three message complexity the author proves a high lower bound on the chromatic decomposition number of the playoffs problem s characteristic hypergraph showing that the problem has a high two message complexity and that allowing more than two messages may decrease the number of transmitted bits by a factor of two a technique that improves the lower bound for the chromatic decomposition number of the playoffs problem is described however this improved lower bound does not suffice to increase the provable gap between two and three message complexities