multiterminal source coding with high resolution we consider separate encoding and joint decoding of correlated continuous information sources subject to a difference distortion measure we first derive a multiterminal extension of the shannon lower bound for the rate region then we show that this shannon outer bound is asymptotically tight for small distortions these results imply that the loss in the sum of the coding rates due to the separation of the encoders vanishes in the limit of high resolution furthermore lattice quantizers followed by slepian wolf lossless encoding are asymptotically optimal we also investigate the high resolution rate region in the remote coding case where the encoders observe only noisy versions of the sources for the quadratic gaussian case we establish a separation result to the effect that multiterminal coding aimed at reconstructing the noisy sources subject to the rate constraints followed by estimation of the remote sources from these reconstructions is optimal under certain regularity conditions on the structure of the coding scheme

singularity free adaptive pole placement using periodic controllers a globally convergent adaptive control scheme for nonminimum phase systems is presented the approach is based on a particular periodic pole placement controller an ad hoc parameter estimate correction procedure is used to prevent any possible singularities in the control law computation furthermore a lower bound on the estimated model controllability is ensured the a priori knowledge on the system is reduced to the process order

a note on fuzzy group theorems abstract rf groups and pf groups are completely different in form we prove in this paper that they are equivalent in essence

indirect and direct gaussian distributed source coding problems we consider the distributed source coding system of l correlated gaussian sources y l l 1 2 ldots l which are noisy observations of correlated gaussian remote sources x k k 1 2 ldots k we assume that y l t y 1 y 2 ldots y l is an observation of the source vector x k t x 1 x 2 ldots x k having the form y l ax k n l where a is a l times k matrix and n l t n 1 n 2 ldots n l is a vector of l independent gaussian random variables also independent of x k in this system l correlated gaussian observations are separately compressed by l encoders and sent to the information processing center we study the remote source coding problem where the decoder at the center attempts to reconstruct the remote source x k we consider three distortion criteria based on the covariance matrix of the estimation error on x k for each of those three criteria we derive explicit inner and outer bounds of the rate distortion region next in the case of k l and a i l we study the multiterminal source coding problem where the decoder wishes to reconstruct the observation y l x l n l to investigate this problem we shall establish a result that provides a strong connection between the remote source coding problem and multiterminal source coding problem using this result we derive several new partial solutions to the multiterminal source coding problem

distributed source coding using abelian group codes a new achievable rate distortion region a distributed source coding problem with a joint distortion criterion that depends on the sources and the reconstruction is considered in this work while the prevalent trend in information theory has been to prove achievability results using shannon s random coding arguments using structured random codes offer rate gains over unstructured random codes for many problems motivated by this a new achievable rate distortion region an inner bound to the performance limit is presented for this problem for discrete memoryless sources based on good structured random nested codes built over abelian groups for certain sources and distortion functions the new rate region is shown to be strictly bigger than the berger tung rate region which has been the best known achievable rate region for this problem till now this is done using numerical plots achievable rates for single user source coding using abelian group codes are also obtained as a corollary of the main coding theorem it is shown that nested linear codes achieve the shannon rate distortion function in the arbitrary discrete memoryless case

network compression worst case analysis we study the problem of communicating a distributed correlated memoryless source over a memoryless network from source nodes to destination nodes under quadratic distortion constraints we establish the following two complementary results 1 for an arbitrary memoryless network among all distributed memoryless sources of a given correlation gaussian sources are least compressible that is they admit the smallest set of achievable distortion tuples and 2 for any memoryless source to be communicated over a memoryless additive noise network among all noise processes of a given correlation gaussian noise admits the smallest achievable set of distortion tuples we establish these results constructively by showing how schemes for the corresponding gaussian problems can be applied to achieve similar performance for source or noise distributions that are not necessarily gaussian but have the same covariance

multiterminal source channel communication over an orthogonal multiple access channel we consider the problem of multiterminal source channel communication where a number of distributed and possibly correlated sources are transmitted through an orthogonal multiple access channel to a common destination we provide a characterization of the optimal tradeoff between the transmission cost gamma and the distortion vector d as measured against individual sources our approach consists of two steps 1 a multiple letter characterization of the rate distortion region of the multiterminal source coding and 2 a source channel separation theorem ensuring that all achievable pairs of gamma d can be obtained by combining the rate distortion region and the orthogonal multiple access channel capacity region as a corollary we determine the optimal power and distortion tradeoff in a quadratic gaussian sensor network under orthogonal multiple access and show that separate source and channel coding strictly outperforms the uncoded amplify forward transmission and is in fact optimal in this case this result is in sharp contrast to the case of nonorthogonal multiple access for which separate source and channel coding is not only suboptimal but also strictly inferior to uncoded transmission

the ceo problem multiterminal source coding we consider a new problem in multiterminal source coding motivated by the following decentralized communication estimation task a firm s chief executive officer ceo is interested in the data sequence x t sub t 1 sup spl infin which cannot be observed directly perhaps because it represents tactical decisions by a competing firm the ceo deploys a team of l agents who observe independently corrupted versions of x t sub t 1 sup spl infin because x t is only one among many pressing matters to which the ceo must attend the combined data rate at which the agents may communicate information about their observations to the ceo is limited to say r bits per second if the agents were permitted to confer and pool their data then in the limit as l spl rarr spl infin they usually would be able to smooth out their independent observation noises entirely then they could use their r bits per second to provide the ceo with a representation of x t with fidelity d r where d spl middot is the distortion rate function of x t in particular with such data pooling d can be made arbitrarily small if r exceeds the entropy rate h of x t suppose however that the agents are not permitted to convene agent i having to send data based solely on his own noisy observations y sub i t we show that then there does not exist a finite value of r for which even infinitely many agents can make d arbitrarily small furthermore in this isolated agents case we determine the asymptotic behavior of the minimal error frequency in the limit as l and then r tend to infinity

broadcasting correlated gaussians we study the transmission of a memoryless bivariate gaussian source over an average power constrained one to two gaussian broadcast channel the transmitter observes the source and describes it to the two receivers by means of an average power constrained signal each receiver observes the transmitted signal corrupted by a different additive white gaussian noise and wishes to estimate the source component intended for it receiver 1 wishes to estimate the first source component and receiver 2 wishes to estimate the second our interest is in the pairs of expected squared error distortions that are simultaneously achievable at the two receivers we prove that an uncoded transmission scheme that sends a linear combination of the source components achieves the optimal power versus distortion trade off whenever the signal to noise ratio is below a certain threshold the threshold is a function of the source correlation and the distortion at the receiver with the weaker noise

sending a bivariate gaussian source over a gaussian mac with feedback we study the power versus distortion tradeoff for the transmission of a memoryless bivariate gaussian source over a two to one gaussian multiple access channel with perfect causal feedback in this problem each of two separate transmitters observes a different component of a memoryless bivariate gaussian source as well as the feedback from the channel output of the previous time instants based on the observed source sequence and the feedback each transmitter then describes its source component to the common receiver via an average power constrained gaussian multiple access channel from the resulting channel output the receiver wishes to reconstruct each source component with the least possible expected squared error distortion we study the set of distortion pairs that can be achieved by the receiver on the two source components we present sufficient conditions and necessary conditions for the achievability of a distortion pair these conditions are expressed in terms of the source correlation and of the signal to noise ratio snr of the channel in several cases the necessary conditions and sufficient conditions are shown to agree in particular we show that if the channel snr is below a certain threshold then an uncoded transmission scheme that ignores the feedback is optimal thus below this snr threshold feedback is useless we also derive the optimal high snr asymptotics

sending a bivariate gaussian over a gaussian mac we study the power versus distortion tradeoff for the distributed transmission of a memoryless bivariate gaussian source over a two to one average power limited gaussian multiple access channel in this problem each of two separate transmitters observes a different component of a memoryless bivariate gaussian source the two transmitters then describe their source component to a common receiver via an average power constrained gaussian multiple access channel from the output of the multiple access channel the receiver wishes to reconstruct each source component with the least possible expected squared error distortion our interest is in characterizing the distortion pairs that are simultaneously achievable on the two source components we focus on the equal bandwidth case where the source rate in source symbols per second is equal to the channel rate in channel uses per second we present sufficient conditions and necessary conditions for the achievability of a distortion pair these conditions are expressed as a function of the channel signal to noise ratio snr and of the source correlation in several cases the necessary conditions and sufficient conditions are shown to agree in particular we show that if the channel snr is below a certain threshold then an uncoded transmission scheme is optimal moreover we introduce a source channel vector quantizer scheme which is asymptotically optimal as the snr tends to infinity

the non regular ceo problem we consider the ceo problem for non regular source distributions such as uniform or truncated gaussian a group of agents observe independently corrupted versions of data and transmit coded versions over rate limited links to a ceo the ceo then estimates the underlying data based on the received coded observations agents are not allowed to convene before transmitting their observations this formulation is motivated by the practical problem of a firm s ceo estimating non regular beliefs about a sequence of events before acting on them agents observations are modeled as jointly distributed with the underlying data through a given conditional probability density function we study the asymptotic behavior of the minimum achievable mean squared error distortion at the ceo in the limit when the number of agents l and the sum rate r tend to infinity we establish a 1 r 2 convergence of the distortion an intermediate regime of performance between the exponential behavior in discrete ceo problems berger zhang and viswanathan 1996 and the 1 r behavior in gaussian ceo problems viswanathan and berger 1997 achievability is proved by a layered architecture with scalar quantization distributed entropy coding and midrange estimation the converse is proved using the bayesian chazan zakai ziv bound

a cell decomposition of the space of real hankel matrices of rank n and some applications abstract we construct a cellular decomposition of the space of all infinite real hankel matrices of rank n it induces cell decompositions on each manifold h m of hankel matrices of fixed rank m n the relationship between the cellular decomposition of h n and the decomposition of rat n into continued fraction cells is determined finally the cell decomposition is applied to derive some topological properties of the manifolds h n and rat n

distributed rate distortion with common components we describe a scheme for rate distortion with distributed encoding in which the sources to be compressed contain a common component we show that this scheme is optimal in some situations and that it strictly improves upon existing schemes which do not make full use of common components this establishes that independent quantization followed by independent binning is not optimal for the two encoder problem with a distortion constraint on one source we also show that independent quantization and binning is suboptimal for the three encoder problem in which the goal is to reproduce one of the sources losslessly this provides a counterexample that is fundamentally different from one provided earlier by korner and marton the proofs rely on the binary analogue of the entropy power inequality and the existence of a rate loss for the binary symmetric wyner ziv problem

vector gaussian two terminal source coding we derive a lower bound on each supporting line of the rate region of the vector gaussian two terminal ceo problem which is a special case of the indirect vector gaussian two terminal source coding problem the key technical ingredient is a new extremal inequality it is shown that the lower bound coincides with the berger tung upper bound in the high resolution regime similar results are derived for the direct vector gaussian two terminal source coding problem

the gaussian many help one distributed source coding problem jointly gaussian memoryless sources are observed at n distinct terminals the goal is to efficiently encode the observations in a distributed fashion so as to enable reconstruction of any one of the observations say the first one at the decoder subject to a quadratic fidelity criterion our main result is a precise characterization of the rate distortion region when the covariance matrix of the sources satisfies a tree structure condition in this situation a natural analog digital separation scheme optimally trades off the distributed quantization rate tuples and the distortion in the reconstruction each encoder consists of a point to point gaussian vector quantizer followed by a slepian wolf binning encoder we also provide a partial converse that suggests that the tree structure condition is fundamental

on the sum rate loss of quadratic gaussian multiterminal source coding this work studies the sum rate loss of quadratic gaussian multiterminal source coding i e the difference between the minimum sum rates of distributed encoding and joint encoding both with joint decoding of correlated gaussian sources subject to mse distortion constraints on individual sources it is shown that under the nondegraded assumption i e all target distortions are simultaneously achievable by a gaussian berger tung scheme the supremum of the sum rate loss of distributed encoding over joint encoding of l jointly gaussian sources increases almost linearly in the number of sources l with an asymptotic slope of 0 1083 bit per sample per source as l goes to infinity this result is obtained even though we currently do not have the full knowledge of the minimum sum rate for the distributed encoding case the main idea is to upper bound the minimum sum rate of multiterminal source coding by that achieved by parallel gaussian test channels while lower bounding the minimum sum rate of joint encoding by a reverse water filling solution to a relaxed joint encoding problem of the same set of gaussian sources with a sum distortion constraint that equals the sum of the individual target distortions we show that under the nondegraded assumption the supremum difference between the upper bound for distributed encoding and the lower bound for joint encoding is achieved in the bi eigen equal variance with equal distortion case in which both bounds are known to be tight

rate region of the quadratic gaussian two encoder source coding problem we determine the rate region of the quadratic gaussian two encoder source coding problem this rate region is achieved by a simple architecture that separates the analog and digital aspects of the compression furthermore this architecture requires higher rates to send a gaussian source than it does to send any other source with the same covariance our techniques can also be used to determine the sum rate of some generalizations of this classical problem our approach involves coupling the problem to a quadratic gaussian ldquoceo problem rdquo

universal quantization for separate encodings and joint decoding of correlated sources we consider the multi user lossy source coding problem for continuous alphabet sources in a previous work ziv proposed a single user universal coding scheme which uses uniform quantization with dither followed by a lossless source encoder entropy coder in this paper we generalize ziv s scheme to the multi user setting for this generalized universal scheme upper bounds are derived on the redundancies defined as the differences between the actual rates and the closest corresponding rates on the boundary of the rate region for the mean square error distortion measure it is shown that this scheme can achieve redundancies of no more than 0 754 b per sample for each user these bounds are obtained without the knowledge of the multi user rate region which is an open problem in general as a direct consequence of these results the inner and outer bounds on the rate distortion achievable region are obtained

analog transmission of correlated sources over fading simo multiple access channels joint source channel coding for discrete time analog sources is an appealing transmission approach because of its extremely low delay and complexity when the users access the channel orthogonally analog transmission of correlated information over fading multiple access channels macs using modulo like mappings provides better performance than uncoded transmission in this paper we propose a simplified decoder for modulo mappings in possibly non orthogonal mac scenarios with a single antenna users and a multiple antenna receiver sphere decoding is investigated to reduce the computational complexity when the number of users is large in addition affordable strategies are proposed to optimize the mapping parameters according to the channel conditions and the source correlation the obtained results show that the use of modulo mappings is suitable when the number of antennas at the receiver is larger than the number of users and for high correlation between user data

lattice based robust distributed source coding for three correlated sources the problem of robust distributed source coding for three correlated sources is studied in this work a lattice based scheme is proposed and the analysis of its performance is provided in the high resolution regime special attention is paid to the degenerate case where the three sources are gaussian and identical in this case our scheme is shown to achieve within an asymptotic gap of 0 069 bits in terms of rate per description from the information theoretic limit of quadratic symmetric gaussian multiple description coding with central and individual decoders when the side distortion and the ratio between the central and side distortions both approach 0

connectivity properties of spaces of partial realizations abstract this paper addresses the problem of determining the number of connected components of the space of real scalar m n hankel matrices of rank n for n

a new sufficient condition for sum rate tightness in quadratic gaussian multiterminal source coding this paper considers the quadratic gaussian multiterminal mt source coding problem and provides a new sufficient condition for the berger tung bt sum rate bound to be tight the converse proof utilizes a set of virtual remote sources given which the observed sources are block independent with a maximum block size of 2 the given mt source coding problem is then related to a set of two terminal problems with matrix distortion constraints for which a new lower bound on the sum rate is given by formulating a convex optimization problem over all distortion matrices a sufficient condition is derived for the optimal bt scheme to satisfy the subgradient based karush kuhn tucker condition the subset of the quadratic gaussian mt problem satisfying our new sufficient condition subsumes all previously known tight cases and our proof technique opens a new direction for more general partial solutions

distributed source coding of correlated gaussian remote sources we consider the distributed source coding system for l correlated memoryless gaussian remote sources specified with l correlated gaussian random variables xi i 1 2 l we deal with the case where each of l distributed encoders cannot directly observe the source xi but its noisy version yi xi ni here ni i 1 2 l are independent additive l gaussian noises also independent of xi i 1 2 l on this coding system the determination problem of the rate distortion region remains open in general in this paper we derive explicit outer and inner bounds of the rate distortion region we further find an explicit sufficient condition for those two bounds to match we also study the sum rate part of the rate distortion region when the correlation has some symmetrical property we derive an explicit upper bound of the sum rate part from the inner bounds of the rate distortion region on a lower bound we derive a new explicit bound by making full use of the symmetrical property of the correlation we further derive an explicit necessary and sufficient condition for the lower bound to coincide with the upper bound

on network functional compression in this thesis we consider different aspects of the functional compression problem in functional compression the computation of a function or some functions of sources is desired at the receiver s the rate region of this problem has been considered in the literature under certain restrictive assumptions in chapter 2 of this thesis we consider this problem for an arbitrary tree network and asymptotically lossless computations in particular for one stage tree networks we compute a rate region and for an arbitrary tree network we derive a rate lower bound based on the graph entropy we introduce a new condition on colorings of source random variables characteristic graphs called the coloring connectivity condition c c c we show that unlike the condition mentioned in doshi et al this condition is necessary and sufficient for any achievable coding scheme based on colorings we also show that unlike entropy graph entropy does not satisfy the chain rule for one stage trees with correlated sources and general trees with independent sources we propose a modularized coding scheme based on graph colorings to perform arbitrarily closely to the derived rate lower bound we show that in a general tree network case with independent sources to achieve the rate lower bound intermediate nodes should perform some computations however for a family of functions and random variables called chain rule proper sets it is sufficient to have intermediate nodes act like relays to perform arbitrarily closely to the rate lower bound in chapter 3 of this thesis we consider a multi functional version of this problem with side information where the receiver wants to compute several functions with different side information random variables and zero distortion our results are applicable to the case with several receivers computing different desired functions we define a new concept named multi functional graph entropy which is an extension of graph entropy defined by k6rner we show that the minimum achievable rate for this problem is equal to conditional multi functional graph entropy of the source random variable given the side information we also propose a coding scheme based on graph colorings to achieve this rate in these proposed coding schemes one needs to compute the minimum entropy coloring a coloring random variable which minimizes the entropy of a characteristic graph in general finding this coloring is an np hard problem however in chapter 4 we show that depending on the characteristic graph s structure there are some interesting cases where finding the minimum entropy coloring is not np hard but tractable and practical in one of these cases we show that by having a non zero joint probability condition on random variables distributions for any desired function finding the minimum entropy coloring can be solved in polynomial time in another case we show that if the desired function is a quantization function this problem is also tractable we also consider this problem in a general

secret key generation for correlated gaussian sources secret key generation by multiple terminals is considered based on their observations of jointly distributed gaussian signals followed by public communication among themselves exploiting an inherent connection between secrecy generation and lossy data compression two main contributions are made the first is a characterization of strong secret key capacity and entails a converse proof technique that is valid for real valued and not necessarily gaussian as well as finite valued signals the capacity formula acquires a simple form when the terminals observe symmetrically correlated jointly gaussian signals for the latter setup with two terminals considering schemes that involve quantization at one terminal the best rate of an achievable secret key is characterized as a function of quantization rate secret key capacity is attained as the quantization rate tends to infinity structured codes are shown to attain the optimum tradeoff between secret key rate and quantization rate constituting our second main contribution

the bruhat decomposition of finite hankel matrices abstract the bruhat decomposition of the space of n n hankel matrices is derived from the bruhat decomposition of the general linear group some new results concerning the hankel minors map h det h1 det hn are obtained det hi denotes the i th principal minor of h the adherence order of the bruhat decomposition is determined in the complex case

a strong entropy power inequality when one of the random summands is gaussian we sharpen the entropy power inequality epi in terms of the strong data processing function for gaussian channels among other consequences this strong epi generalizes the vector extension of costa s epi to non gaussian channels in a precise sense this leads to a new reverse epi and as a corollary sharpens stam s uncertainty principle relating entropy power and fisher information or equivalently gross logarithmic sobolev inequality applications to network information theory are also given including a short self contained proof of the rate region for the two encoder quadratic gaussian source coding problem and a new outer bound for the one sided gaussian interference channel

an improved outer bound for multiterminal source coding we prove a new outer bound on the rate distortion region for the multiterminal source coding problem this bound subsumes the best outer bound in the literature and improves upon it strictly in some cases the improved bound enables us to obtain a new conclusive result for the binary erasure version of the ceo problem the bound recovers many of the converse results that have been established for special cases of the problem including the recent one for the gaussian two encoder problem

lattices for distributed source coding jointly gaussian sources and reconstruction of a linear function consider a pair of correlated gaussian sources x 1 x 2 two separate encoders observe the two components and communicate compressed versions of their observations to a common decoder the decoder is interested in reconstructing a linear combination of x 1 and x 2 to within a mean square distortion of d we obtain an inner bound to the optimal rate distortion region for this problem a portion of this inner bound is achieved by a scheme that reconstructs the linear function directly rather than reconstructing the individual components x 1 and x 2 first this results in a better rate region for certain parameter values our coding scheme relies on lattice coding techniques in contrast to more prevalent random coding arguments used to demonstrate achievable rate regions in information theory we then consider the case of linear reconstruction of k sources and provide an inner bound to the optimal rate distortion region some parts of the inner bound are achieved using the following coding structure lattice vector quantization followed by ldquocorrelatedrdquo lattice structured binning

length functions and fuzzy groups fuzzy sets were introduced by zadeh 7 in 1965 and fuzzy groups by rosenfeld 4 in 1971 fuzzy groups were also considered by anthony and sherwood i in 1979 and in 1981 the theory was developed by das 2 as in the paper of das 2 we also show how a fuzzy subgroup of a group g is determined by a chain of subgroups of g however we wish to point out that fuzzy subgroups are constructed in a way which is analogous to that of constructing certain length functions of g wilkens s 61 thus in section 3 we consider the theory of length functions and we show the connection between certain length functions and fuzzy subgroups of a group in section 4 we apply the structural results of section 3 on finite solvable groups and we gain a generalization of a theorem of das 2 our notation is standard however we use the symbol e to denote the identity element of a group g and i denotes a real number we start with a preliminary section where we briefly analyse the definition of a fuzzy subgroup given by rosenfeld 4 naturally we use this definition of a fuzzy subgroup throughout the paper

rate distortion for lossy in network linear function computation and consensus distortion accumulation and sequential reverse water filling we consider the problem of distributed lossy linear function computation in a tree network we examine two cases 1 data aggregation only one sink node computes and 2 consensus all nodes compute the same function by quantifying the accumulation of information loss in distributed computing we obtain fundamental limits on network computation rate as a function of incremental distortions and hence incremental loss of information along the edges of the network the above characterization based on quantifying distortion accumulation offers an improvement over classical cut set type techniques which are based on overall distortions instead of incremental distortions this quantification of information loss qualitatively resembles information dissipation in cascaded channels 2 surprisingly this accumulation effect of distortion happens even at infinite blocklength combining this observation with an inequality on the dominance of mean square quantities over relative entropy quantities we obtain outer bounds on the rate distortion function that are tighter than classical cut set bounds by a difference which can be arbitrarily large in both data aggregation and consensus we also obtain inner bounds on the optimal rate using random gaussian coding which differ from the outer bounds by mathcal o sqrt d where d is the overall distortion the obtained inner and outer bounds can provide insights on rate bit allocations for both the data aggregation problem and the consensus problem we show that for tree networks the rate allocation results have a mathematical structure similar to classical reverse water filling for parallel gaussian sources apart from data aggregation and distributed consensus the distortion accumulation analysis framework is also applicable in large scale data summarization through histograms and linear sketching e g word counting tasks for document summarization

on a theorem of hermite and hurwitz abstract the hermite hurwitz theorem computes the degree over r of a real rational function in terms of the signature of an associated quadratic form known today as the hankel matrix of this formula which hermite was led to by his work on the problem of representing integers as sums of squares gave rise to striking applications in the theory of equations and in the stability theory of ordinary differential equations in this paper this theorem and various generalizations to the matrix valued case are discussed and described in terms of signature formulae these include its relation to stability theory and the matrix hermite hurwitz theorem of bitmead anderson as applied to questions of circuit synthesis this also includes a global form of hormander s signature formula for the maslov index of a rational loop in a lagrangian grassmannian due to byrnes and duncan and applications to the topology of spaces of rational matrix valued functions following the work of brockett byrnes and duncan this includes in particular a topological proof of the matrix hermite hurwitz theorem

deterministic annealing based optimization for zero delay source channel coding in networks this paper studies the problem of global optimization of zero delay source channel codes that map between the source space and the channel space under a given transmission power constraint and for the mean square error distortion particularly we focus on two well known network settings the wyner ziv setting where only a decoder has access to side information and the distributed setting where independent encoders transmit over independent channels to a central decoder prior work derived the necessary conditions for optimality of the encoder and decoder mappings along with a greedy optimization algorithm that imposes these conditions iteratively in conjunction with the heuristic noisy channel relaxation method to mitigate poor local minima while noisy channel relaxation is arguably effective in simple settings it fails to provide accurate global optimization in more complicated settings considered in this paper we propose a powerful nonconvex optimization method based on the concept of deterministic annealing which is derived from information theoretic principles and was successfully employed in several problems including vector quantization classification and regression we present comparative numerical results that show strict superiority of the proposed method over greedy optimization methods as well as prior approaches in literature

distributed compression of linear functions partial sum rate tightness and gap to optimal sum rate we consider the problem of distributed compression of the difference z y1 cy2 of two jointly gaussian sources y1 and y2 under an mse distortion constraint d on z the rate region for this problem is unknown if the correlation coefficient and the weighting factor c satisfy c 0 inspired by ahlswede and han s scheme for the problem of distributed compression of the modulo 2 sum of two binary sources we first propose a hybrid random structured coding scheme that is capable of saving the sum rate over both the random quantize and bin qb coding scheme and krithivasan and pradhan s structured lattice coding scheme the main idea is to use a random coding component in the first layer to adjust the source correlation so that the structured coding component in the second layer can be more efficient with the outputs from the first layer as decoder side information we then provide a new sum rate lower bound for the problem in hand by connecting it to the gaussian two terminal source coding problem with covariance matrix distortion constraint our lower bound not only improves existing bounds in many cases but also allows us to prove sum rate tightness of the qb scheme when c is either relatively small or large and d is larger than some threshold furthermore our lower bound enables us to show that our new hybrid scheme performs within two b s from the optimal sum rate for all values of c and d

the wyner ziv problem with multiple sources this paper provides bounds on the rate distortion region for the distributed compression scenario where two or more sources are compressed separately for a decoder that has access to side information conclusive rate distortion results are found for the case where the sources are conditionally independent given the side information

rate region of the gaussian scalar help vector source coding problem we determine the rate region of the gaussian one helper source coding problem in which the helper observes a scalar the main encoder observes a vector and the distortion constraint is a positive semidefinite upper bound on the error covariance matrix of the main source the rate region is achieved by a gaussian achievable scheme we introduce a novel outer bounding technique to establish the converse our approach is to create a reduced dimensional problem by projecting the main source and the distortion constraint in certain directions determined by the optimal gaussian scheme we also provide an outer bound to the rate region of the more general problem in which there are distortion constraints on both sources this outer bound is partially tight in general and completely tight in some nontrivial cases

bruhat canonical form for linear systems based on a generalization of the classical bruhat factorization of nonsingular matrices to arbitrary rectangular matrices a new canonical form for state space equivalence of controllable and observable linear systems is introduced the proposed canonical form is shown to be closely related to a canonical form due to bosgra and van der weiden moreover in the single input single output case and up to minor details the proposed normal form is equivalent to the continued fraction canonical form introduced by kalman connections to a cell decomposition by fuhrmann and krishnaprasad are discussed as well discrete invariants appearing in the bruhat canonical form are shown to be invariants for restricted output feedback equivalence

duality of a source coding problem and the semi deterministic broadcast channel with rate limited cooperation the wyner ahlswede korner wak empirical coordination problem where the encoders cooperate via a finite capacity one sided link is considered the coordination capacity region is derived by combining several source coding techniques such as wyner ziv coding binning and superposition coding furthermore a semi deterministic sd broadcast channel bc with one sided decoder cooperation is considered duality principles relating the two problems are presented and the capacity region for the sd bc setting is derived the direct part follows from an achievable region for a general bc that is tight for the sd scenario a converse is established by using telescoping identities the sd bc is shown to be operationally equivalent to a class of relay bcs and the correspondence between their capacity regions is established the capacity region of the sd bc is transformed into an equivalent region that is shown to be dual to the admissible region of the wak problem in the sense that the information measures defining the corner points of both regions coincide achievability and converse proofs for the equivalent region are provided for the converse we use a probabilistic construction of auxiliary random variables that depends on the distribution induced by the codebook several examples illustrate the results

rate distortion theory for gaussian multiterminal source coding systems with several side informations at the decoder in this paper we consider the separate coding problem for l 1 correlated gaussian memoryless sources we deal with the case where l sources work as partial side information at the decoder for the reconstruction of the remaining source the determination problem of the rate distortion region for this system is the so called many help one problem and it has been known as a highly challenging problem for almost 20 years in this paper we give a partial solution to this problem we determine the rate distortion region in the case where the l sources working as partial side information are conditionally independent if the remaining source we wish to reconstruct is given the additive white gaussian noise ceo problem is a special case of this we also discuss the relation of the result previous results of ours

two terminal video coding following recent works on the rate region of the quadratic gaussian two terminal source coding problem and limit approaching code designs this paper examines multiterminal source coding of two correlated i e stereo video sequences to save the sum rate over independent coding of both sequences two multiterminal video coding schemes are proposed in the first scheme the left sequence of the stereo pair is coded by h 264 avc and used at the joint decoder to facilitate wyner ziv coding of the right video sequence the first i frame of the right sequence is successively coded by h 264 avc intracoding and wyner ziv coding an efficient stereo matching algorithm based on loopy belief propagation is then adopted at the decoder to produce pixel level disparity maps between the corresponding frames of the two decoded video sequences on the fly based on the disparity maps side information for both motion vectors and motion compensated residual frames of the right sequence are generated at the decoder before wyner ziv encoding in the second scheme source splitting is employed on top of classic and wyner ziv coding for compression of both i frames to allow flexible rate allocation between the two sequences experiments with both schemes on stereo video sequences using h 264 avc ldpc codes for slepian wolf coding of the motion vectors and scalar quantization in conjunction with ldpc codes for wyner ziv coding of the residual coefficients give a slightly lower sum rate than separate h 264 avc coding of both sequences at the same video quality

vector gaussian multiterminal source coding we derive an outer bound of the rate region of the vector gaussian l terminal ceo problem by establishing a lower bound on each supporting hyperplane of the rate region to this end we prove a new extremal inequality by exploiting the connection between differential entropy and fisher information as well as some fundamental estimation theoretic inequalities it is shown that the outer bound matches the berger tung inner bound in the high resolution regime we then derive a lower bound on each supporting hyperplane of the rate region of the direct vector gaussian l terminal source coding problem by coupling it with the ceo problem through a limiting argument the tightness of this lower bound in the high resolution regime and the weak dependence regime is also proved

vector gaussian multiple description with individual and central receivers t multiple descriptions of a vector gaussian source for individual and central receivers are investigated the sum rate of the descriptions with covariance distortion measure constraints in a positive semidefinite ordering is exactly characterized for two descriptions the entire rate region is characterized the key component of the solution is a novel information theoretic inequality that is used to lower bound the achievable multiple description rates jointly gaussian descriptions are optimal in achieving the limiting rates we also show the robustness of this description scheme the distortions achieved are no larger when used to describe any non gaussian source with the same covariance matrix

information rates of densely sampled data distributed vector quantization and scalar quantization with transforms for gaussian sources this paper establishes rates attainable by several lossy schemes for coding a continuous parameter source to a specified mean squared error distortion based on sampling at asymptotically large rates first a densely sampled spatiotemporal stationary gaussian source is distributively encoded the berger tung bound to the distributed rate distortion function and three convergence theorems are used to obtain an upper bound expressed in terms of the source spectral density to the smallest attainable rate at asymptotically large sampling rates the bound is tighter than that recently obtained by kashyap both indicate that with ideal distributed lossy coding dense sensor networks can efficiently sense and convey a field in contrast to the negative result obtained by marco for encoders based on scalar quantization and slepian wolf distributed lossless coding the second scheme is transform coding with scalar coefficient quantization a new generalized transform coding analysis as well as the aforementioned convergence theorems is used to find the smallest attainable rate at asymptotically large sampling rates in terms of the source spectral density and the operational rate distortion function of the family of quantizers which in contrast to previous analyses need not be convex the result shows that when a transform is used scalar quantization need not cause the poor performance found by marco as a corollary the final result pursues an approach originally proposed by berger to show that the inverse water pouring formula for the rate distortion function can be attained at high sampling rates by transform coding with ideal vector quantization to encode the coefficients also established in the paper are relations between operational rate distortion and distortion rate functions for a continuous parameter source and those for the discrete parameter source that results from sampling

on source coding for distributed temperature sensing with shift invariant geometries we study the source coding problem in sensor networks deployed to monitor the evolution of spatio temporal temperature distributions the sensors sample the temperature field quantize the samples and transmit the encoded samples through digital channels to some central unit which computes an estimate of the original temperature field our analysis is based on the heat kernel s spectral properties which are induced by the physics of heat diffusion we determine rate distortion functions for various source coding schemes in particular we compare centralized coding independent coding berger tung coding and predictive quantization

energy distortion tradeoffs in gaussian joint source channel coding problems the information theoretic notion of energy efficiency is studied in the context of various joint source channel coding problems the minimum transmission energy e d required to communicate a source over a noisy channel so that it can be reconstructed within a target distortion d is analyzed unlike the traditional joint source channel coding formalisms no restrictions are imposed on the number of channel uses per source sample for single source memoryless point to point channels e d is shown to be equal to the product of the minimum energy per bit ebmin of the channel and the rate distortion function r d of the source regardless of whether channel output feedback is available at the transmitter the primary focus is on gaussian sources and channels affected by additive white gaussian noise under quadratic distortion criteria with or without perfect channel output feedback in particular for two correlated gaussian sources communicated over a gaussian multiple access channel inner and outer bounds on the energy distortion region are obtained which coincide in special cases for symmetric channels the difference between the upper and lower bounds on energy is shown to be at most a constant even when the lower bound goes to infinity as d 0 it is also shown that simple uncoded transmission schemes perform better than the separation based schemes in many different regimes both with and without feedback

fuzzy subgroups some characterizations the notion of a fuzzy set was first introduced in 1965 by zadeh 12 since then the theory of fuzzy sets has gone through remarkably rapid strides with well over 4000 papers by now see for example the bibliography 3 several textbooks for example 2 and 6 and an international journal solely devoted to it s the theory has found wideranging applications in such diverse fields as automata control theory decision theory and social behavior pattern studies to name just a few for an interesting expository account see rosenfeld lo and also the book review 7 in 9 rosenfeld introduced the notion of a fuzzy group and showed that many concepts of group theory can be extended in an elementary manner to develop the theory of fuzzy groups in particular he characterized all the fuzzy subgroups of cyclic groups of prime order 9 proposition 5 101 fuzzy groups were further investigated by das in 1 where he characterized all the fuzzy subgroups of finite cyclic groups l theorem 4 21 in this connection he introduced the notion of level subgroups of a fuzzy subgroup which is based on the notion of a level subset introduced earlier by zadeh for a finite group das showed that these level subgroups of a fuzzy subgroup form a chain 1 corollary 3 11 and also constructed a fuzzy subgroup whose chain of level subgroups is maximal in the following special cases when the group is i a direct product of cyclic groups of prime orders ii abelian l theorem 3 41 we analyze the level subgroups of a fuzzy subgroup in more detail and investigate whether the family of level subgroups of a fuzzy subgroup deter

sending a bivariate gaussian source over a gaussian mac with unidirectional conferencing encoders we consider the transmission of a memoryless bivariate gaussian source over a two user additive gaussian multiple access channel with unidirectional conferencing encoders here prior to each transmission block encoder 1 which observes the first source component is allowed to communicate with encoder 2 which observes the second source component via a unidirectional noise free bit pipe of given capacity the main results of this paper are sufficient conditions and a necessary condition for the achievability of a distortion pair expressed as a function of the channel sf snr and of the source correlation the main sufficient condition is obtained by an extension of the vector quantizer scheme suggested by lapidoth tinguely for the case without conferencing to the case with unidirectional conference in the high sf snr regime and when the capacity of the conference channel is unlimited these necessary and sufficient conditions are shown to agree we evaluate the precise high sf snr asymptotics for a subset of distortion pairs when the capacity of the conference channel is unlimited in which case we show that a separation based scheme attains these optimal distortion pairs however with symmetric average power constraints and fixed conferencing capacity at high sf snr the latter separation based scheme is shown to be suboptimal

distributed scalar quantization for computing high resolution analysis and extensions communication of quantized information is frequently followed by a computation we consider situations of distributed functional scalar quantization distributed scalar quantization of possibly correlated sources followed by centralized computation of a function under smoothness conditions on the sources and function companding scalar quantizer designs are developed to minimize mean squared error mse of the computed function as the quantizer resolution is allowed to grow striking improvements over quantizers designed without consideration of the function are possible and are larger in the entropy constrained setting than in the fixed rate setting as extensions to the basic analysis we characterize a large class of functions for which regular quantization suffices consider certain functions for which asymptotic optimality is achieved without arbitrarily fine quantization and allow limited collaboration between source encoders in the entropy constrained setting a single bit per sample communicated between encoders can have an arbitrarily large effect on functional distortion in contrast such communication has very little effect in the fixed rate setting

multiterminal source coding under logarithmic loss we consider the classical two encoder multiterminal source coding problem where distortion is measured under logarithmic loss we provide a single letter description of the achievable rate distortion region for all discrete memoryless sources with finite alphabets by doing so we also give the rate distortion region for the m encoder ceo problem also under logarithmic loss several applications and examples are given

generalized gaussian multiterminal source coding the symmetric case consider a generalized multiterminal source coding system where binom ell m encoders each observing a distinct size m subset of ell ell geq 2 zero mean unit variance exchangeable gaussian sources with correlation coefficient rho compress their observations in such a way that a joint decoder can reconstruct the sources within a prescribed mean squared error distortion based on the compressed data the optimal rate distortion performance of this system was previously known only for the two extreme cases m ell the centralized case and m 1 the distributed case and except when rho 0 the centralized system can achieve strictly lower compression rates than the distributed system under all non trivial distortion constraints somewhat surprisingly it is established in the present paper that the optimal rate distortion performance of the afore described generalized multiterminal source coding system with m geq 2 coincides with that of the centralized system for all distortions when rho leq 0 and for distortions below an explicit positive threshold depending on m when rho 0 moreover when rho 0 the minimum achievable rate of generalized multiterminal source coding subject to an arbitrary positive distortion constraint d is shown to be within a finite gap depending on m and d from its centralized counterpart in the large ell limit except for possibly the critical distortion d 1 rho

the sum rate bound for a new class of quadratic gaussian multiterminal source coding problems in this paper we show tightness of the berger tung bt sum rate bound for a new class of quadratic gaussian multiterminal mt source coding problems dubbed bi eigen equal variance with equal distortion beev ed where the l l source covariance matrix has equal diagonal elements with two distinct eigenvalues and the l target distortions are equal let k k l be the number of repetitions of the larger eigenvalue the beev covariance structure allows us to connect k i i d virtual gaussian sources with the l given mt sources via an l k semiorthogonal transform whose rows have equal euclidean norm plus additive i i d gaussian noises resulting in the two sets of sources being mutually conditional i i d by relating the given mt source coding problem to a generalized gaussian ceo problem with the k virtual sources as remote sources and the l mt sources as observations we obtain a lower bound on the mt sum rate and show its achievability by bt schemes under the equal distortion constraints our beev ed class of quadratic gaussian mt source coding problems subsumes both the positive symmetric case considered by wagner et al and the negative symmetric case other examples including a subclass of sources with be circulant symmetric covariance matrices and equal distortion constraints are also provided to highlight tightness of the sum rate bound