second order and moderate deviations asymptotics for successive refinement we derive the optimal second order coding region and moderate deviations constant for successive refinement source coding with a joint excess distortion probability constraint we consider two scenarios 1 a discrete memoryless source dms and arbitrary distortion measures at the decoders and 2 a gaussian memoryless source gms and quadratic distortion measures at the decoders for a dms with arbitrary distortion measures we prove an achievable second order coding region using type covering lemmas by kanlis and narayan and by no ingber and weissman we prove the converse using the perturbation approach by gu and effros when the dms is successively refinable the expressions for the second order coding region and the moderate deviations constant are simplified and easily computable for this case we also obtain new insights on the second order behavior compared with the scenario where separate excess distortion proabilities are considered for example we describe a dms for which the optimal second order region transitions from being characterizable by a bivariate gaussian to a univariate gaussian as the distortion levels are varied we then consider a gms with quadratic distortion measures to prove the direct part we make use of the sphere covering theorem by verger gaugry together with appropriately defined gaussian type classes to prove the converse we generalize kostina and verdu s one shot converse bound for point to point lossy source coding we remark that this proof is applicable to general successively refinable sources in the proofs of the moderate deviations results for both scenarios we follow a strategy similar to that for the second order asymptotics and use the moderate deviations principle

hypothesis testing with multiterminal data compression the multiterminal hypothesis testing h xy against h xy is considered where x n x n and y n y n are separately encoded at rates r 1 and r 2 respectively the problem is to determine the minimum beta n of the second kind of error probability under the condition that the first kind of error probability alpha n leq epsilon for a prescribed 0 a good lower bound theta l r 1 r 2 on the power exponent theta r 1 r 2 epsilon lim inf n rightarrow infty 1 n log beta n is given and several interesting properties are revealed the lower bound is tighter than that of ahlswede and csiszar furthermore in the special case of testing against independence this bound turns out to coincide with that given by them the main arguments are devoted to the special case with r 2 infty corresponding to full side information for y n y n in particular the compact solution is established to the complete data compression cases which are useful in statistics from the practical point of view

detection of signals by information theoretic criteria a new approach is presented to the problem of detecting the number of signals in a multichannel time series based on the application of the information theoretic criteria for model selection introduced by akaike aic and by schwartz and rissanen mdl unlike the conventional hypothesis testing based approach the new approach does not requite any subjective threshold settings the number of signals is obtained merely by minimizing the aic or the mdl criteria simulation results that illustrate the performance of the new method for the detection of the number of signals received by a sensor array are presented

second order asymptotics for the gaussian mac with degraded message sets this paper studies the second order asymptotics of the gaussian multiple access channel with degraded message sets for a fixed average error probability varepsilon in 0 1 and an arbitrary point on the boundary of the capacity region we characterize the speed of convergence of rate pairs that converge to that boundary point for codes that have asymptotic error probability no larger than varepsilon as a stepping stone to this local notion of the second order asymptotics we study a global notion and establish relationships between the two we provide a numerical example to illustrate how the angle of approach to a boundary point affects the second order coding rate this is the first conclusive characterization of the second order asymptotics of a network information theory problem in which the capacity region is not a polygon

exponential strong converse for content identification with lossy recovery we revisit the high dimensional content identification with lossy recovery problem tuncel and gunduz 2014 and establish an exponential strong converse theorem as a corollary of the exponential strong converse theorem we derive an upper bound on the joint identification error and excess distortion exponent for the problem our main results can be specialized to the biometrical identification problem willems 2003 and the content identification problem tuncel 2009 since these two problems are both special cases of the content identification with lossy recovery problem we leverage the information spectrum method introduced by oohama and adapt the strong converse techniques therein to be applicable to the problem at hand

fast iterative solution of stabilised stokes systems part i using simple diagonal preconditioners mixed finite element approximation of the classical stokes problem describing slow viscous incompressible flow gives rise to symmetric indefinite systems for the discrete velocity and pressure variables iterative solution of such indefinite systems is feasible and is an attractive approach for large problems the use of stabilisation methods for convenient but unstable mixed elements introduces stabilisation parameters we show how these can be chosen to obtain rapid iterative convergence we propose a conjugate gradient like method the method of preconditioned conjugate residuals which is applicable to symmetric indefinite problems describe the effects of stabilisation on the algebraic structure of the discrete stokes operator and derive estimates of the eigenvalue spectrum of this operator on which the convergence rate of the iteration depends here we discuss the simple case of diagonal preconditioning our results apply to both locally and globally stabilised mixed elements as well as to elements which are inherently stable we demonstrate that convergence rates comparable to that achieved using the diagonally scaled conjugate gradient method applied to the discrete laplacian are approachable for the stokes problem

iterative substructuring methods for spectral element discretizations of elliptic systems ii mixed methods for linear elasticity and stokes flow iterative substructuring methods are introduced and analyzed for saddle point problems with a penalty term two examples of saddle point problems are considered the mixed formulation of the linear elasticity system and the generalized stokes system in three dimensions these problems are discretized with mixed spectral element methods the resulting stiffness matrices are symmetric and indefinite the interior unknowns of each element are first implicitly eliminated by using exact local solvers the resulting saddle point schur complement is solved with a krylov space method with block preconditioners the velocity block can be approximated by a domain decomposition method e g of wire basket type which is constructed from a local solver for each face of the elements and a coarse solver related to the wire basket of the elements the condition number of the preconditioned operator is independent of the number of spectral elements and is bounded from above by the product of the square of the logarithm of the spectral degree and the inverse of the discrete inf sup constant of the problem

on the dispersions of the gel fand pinsker channel and dirty paper coding this paper studies the second order coding rates for memoryless channels with a state sequence known non causally at the encoder in the case of finite alphabets an achievability result is obtained using constant composition random coding and by using a small fraction of the block to transmit the empirical distribution of the state sequence for error probabilities less than 0 5 it is shown that the second order rate improves on an existing one based on independent and identically distributed random coding in the gaussian case dirty paper coding with an almost sure power constraint an achievability result is obtained using random coding over the surface of a sphere and using a small fraction of the block to transmit a quantized description of the state power it is shown that the second order asymptotics are identical to the single user gaussian channel of the same input power without a state

multiple emitter location and signal parameter estimation processing the signals received on an array of sensors for the location of the emitter is of great enough interest to have been treated under many special case assumptions the general problem considers sensors with arbitrary locations and arbitrary directional characteristics gain phase polarization in a noise interference environment of arbitrary covariance matrix this report is concerned first with the multiple emitter aspect of this problem and second with the generality of solution a description is given of the multiple signal classification music algorithm which provides asymptotically unbiased estimates of 1 number of incident wavefronts present 2 directions of arrival doa or emitter locations 3 strengths and cross correlations among the incident waveforms 4 noise interference strength examples and comparisons with methods based on maximum likelihood ml and maximum entropy me as well as conventional beamforming are included an example of its use as a multiple frequency estimator operating on time series is included

a finite element method for the compressible stokes equations a linearized steady state compressible viscous navier stokes system is considered a finite element method is formulated the unique solvability and stability of the finite element solution follow from a theorem for an abstract formulation it is proved that when the subspaces for velocity and pressure satisfy the inf sup condition associated with the incompressible stokes system the finite element method is uniquely solvable an error estimate is obtained for the numerical approximation

nonasymptotic and second order achievability bounds for coding with side information we present a novel nonasymptotic or finite blocklength achievability bounds for three side information problems in network information theory these include 1 the wyner ahlswede korner wak problem of almost lossless source coding with rate limited side information 2 the wyner ziv wz problem of lossy source coding with side information at the decoder and 3 the gel fand pinsker gp problem of channel coding with noncausal state information available at the encoder the bounds are proved using ideas from channel simulation and channel resolvability our bounds for all three problems improve on all previous nonasymptotic bounds on the error probability of the wak wz and gp problems in particular those derived by verdu using our novel nonasymptotic bounds we recover the general formulas for the optimal rates of these side information problems finally we also present achievable second order coding rates by applying the multidimensional berry esseen theorem to our new nonasymptotic bounds numerical results show that the second order coding rates obtained using our nonasymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds

on the conditional smooth renyi entropy and its applications in guessing and source coding a novel definition of the conditional smooth renyi entropy which is different from that of renner and wolf is introduced it is shown that our definition of the conditional smooth renyi entropy is appropriate for providing lower and upper bounds on the optimal guessing moment in a guessing problem where the guesser is allowed to stop guessing and declare an error further a general formula for the optimal guessing exponent is presented in particular a single letterized formula for a mixture of i i d sources is obtained it is also shown that our definition is appropriate to characterize the optimal exponential moment of the codeword length in the problem of source coding with common side information available at the encoder and decoder under a constraint on the probability of a decoding error

sharp bounds for mutual covering a fundamental tool in network information theory is the covering lemma which lower bounds the probability that there exists a pair of random variables among a given number of independently generated candidates falling within a given set we use a weighted sum trick and talagrand s concentration inequality to prove new mutual covering bounds we identify two interesting applications 1 when the probability of the set under the given joint distribution is bounded away from 0 and 1 the covering probability converges to 1 doubly exponentially fast in the blocklength which implies that the covering lemma does not induce penalties on the error exponents in the applications to coding theorems and 2 using hall s marriage lemma we show that the maximum difference between the probability of the set under the joint distribution and the covering probability equals half the minimum total variation distance between the joint distribution and any distribution that can be simulated by selecting a pair from the candidates thus we use the mutual covering bound to derive the exact error exponent in the joint distribution simulation problem in both applications the determination of the exact exponential or double exponential behavior relies crucially on the sharp concentration inequality used in the proof of the mutual covering lemma

exact maximum likelihood parameter estimation of superimposed exponential signals in noise a unified framework for the exact maximum likelihood estimation of the parameters of superimposed exponential signals in noise encompassing both the time series and the array problems is presented an exact expression for the ml criterion is derived in terms of the linear prediction polynomial of the signal and an iterative algorithm for the maximization of this criterion is presented the algorithm is equally applicable in the case of signal coherence in the array problem simulation shows the estimator to be capable of providing more accurate frequency estimates than currently existing techniques the algorithm is similar to those independently derived by kumaresan et al in addition to its practical value the present formulation is used to interpret previous methods such as prony s pisarenko s and modifications thereof

a penalized finite element method for a compressible stokes system the linearized steady state compressible viscous navier stokes equations with constant coefficients are considered in this paper a penalty method has been applied to the continuity equation a finite element formulation is constructed by applying a streamline diffusion method to the penalized continuity equation and applying the usual galerkin method to the momentum equations the resulting finite element formulation is uniquely solvable without requiring any compatibility condition on the subspaces for velocity and pressure an error estimate is obtained

distributed channel synthesis two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel in which a stochastic channel output is generated based on a compressed description of the channel input wyner s common information is the minimum description rate needed however when common randomness independent of the input is available the necessary description rate reduces to shannon s mutual information this paper characterizes the optimal tradeoff between the amount of common randomness used and the required rate of description we also include a number of related derivations including the effect of limited local randomness rate requirements for secrecy applications to game theory and new insights into common information duality our proof makes use of a soft covering lemma known in the literature for its role in quantifying the resolvability of a channel the direct proof achievability constructs a feasible joint distribution over all parts of the system using a soft covering from which the behavior of the encoder and decoder is inferred with no explicit reference to joint typicality or binning of auxiliary interest this paper also generalizes and strengthens this soft covering tool

achievability proof via output statistics of random binning this paper introduces a new and ubiquitous framework for establishing achievability results in network information theory problems the framework uses random binning arguments and is based on a duality between channel and source coding problems furthermore the framework uses pmf approximation arguments instead of counting and typicality this allows for proving coordination and strong secrecy problems where certain statistical conditions on the distribution of random variables need to be satisfied these statistical conditions include independence between messages and eavesdropper s observations in secrecy problems and closeness to a certain distribution usually i i d distribution in coordination problems one important feature of the framework is to enable one to add an eavesdropper and obtain a result on the secrecy rates for free we make a case for generality of the framework by studying examples in a variety of settings including channel coding lossy source coding joint source channel coding coordination strong secrecy feedback and relaying in particular by investigating the framework for the lossy source coding problem over broadcast channel it is shown that the new framework provides a simple alternative scheme to the hybrid coding scheme in addition new results on secrecy rate region under strong secrecy criterion of wiretap broadcast channel and wiretap relay channel are derived in a set of accompanied papers we have shown the usefulness of the framework to establish achievability results for coordination problems including interactive channel simulation coordination via relay and channel simulation via another channel

analysis of new augmented lagrangian formulations for mixed finite element schemes two new augmented lagrangian formulations for mixed finite element schemes are presented the methods lead in some cases to an improvement in the order of the approximation an error analysis is provided together with some interesting examples of applications

the likelihood encoder for lossy compression a likelihood encoder is studied in the context of lossy source compression the analysis of the likelihood encoder is based on the soft covering lemma it is demonstrated that the use of a likelihood encoder together with the soft covering lemma yields simple achievability proofs for classical source coding problems the cases of the point to point rate distortion function the rate distortion function with side information at the decoder i e the wyner ziv problem and the multi terminal source coding inner bound i e the berger tung problem are examined in this paper furthermore a non asymptotic analysis is used for the point to point case to examine the upper bound on the excess distortion provided by this method the likelihood encoder is also related to a recent alternative technique using the properties of random binning

simple effective computation of principal eigenvectors and their eigenvalues and application to high resolution estimation of frequencies we present the results of an investigation of the prony lanczos p l method 14 38 and the power method 39 for simple computation of approximations to a few eigenvectors and eigenvalues of a hermitian matrix we are motivated by realization of high resolution signal processing in an integrated circuit the computational speeds of the above methods are analyzed they are completely dependent on the speed of a matrix vector product operation if only a few eigenvalues or eigenvectors are needed the suggested methods can substitute for methods of the linpack or eispack subroutine libraries the accuracies of the suggested methods are evaluated using matrices formed from simulated data consisting of two sinusoids plus gaussian noise comparisons are made to the corresponding eigenvalues and eigenvectors obtained using linpack as a reference also the accuracies of frequency estimates obtained from the eigenvectors are compared

fast iterative solution of stabilised stokes systems part ii using general block preconditioners mixed finite element approximation of the classical stokes problem describing slow viscous incompressible flow gives rise to symmetric indefinite systems for the discrete velocity and pressure variables iterative solution of such indefinite systems is feasible and is an attractive approach for large problems part i of this work described a conjugate gradient like method the method of preconditioned conjugate residuals which is applicable to symmetric indefinite problems a j wathen and d j silvester siam j numer anal 30 1993 pp 630 649 using simple arguments estimates for the eigenvalue distribution of the discrete stokes operator on which the convergence rate of the iteration depends are easily derived part i discussed the important case of diagonal preconditioning scaling this paper considers the more general class of block preconditioners where the partitioning into blocks corresponds to the natural partitioning into the velocity and pressure variables it is shown that provid

dual mixed finite element methods for the elasticity problem with lagrange multipliers we study a dual mixed formulation of the elasticity system in a polygonal domain of the plane with mixed boundary conditions and its numerical approximation the essential neumann boundary conditions or traction boundary condition are imposed using a discontinuous lagrange multiplier corresponding to the trace of the displacement field moreover a strain tensor is introduced as a new unknown and its symmetry is relaxed also by the use of a lagrange multiplier the rotation the singular behaviour of the solution requires us to use refined meshes to restore optimal rates of convergence uniform error estimates in the lame coefficient l are obtained for large l the hybridization of the problem is performed and numerical tests are presented confirming our theoretical results

a class of iterative methods for solving saddle point problems we consider the numerical solution of indefinite systems of linear equations arising in the calculation of saddle points we are mainly concerned with sparse systems of this type resulting from certain discretizations of partial differential equations we present an iterative method involving two levels of iteration similar in some respects to the uzawa algorithm we relate the rates of convergence of the outer and inner iterations proving that under natural hypotheses the outer iteration achieves the rate of convergence of the inner iteration the technique is applied to finite element approximations of the stokes equations

esprit a subspace rotation approach to estimation of parameters of cisoids in noise the application of a subspace invariance approach esprit to the estimation of parameters frequencies and powers of cisoids in noise is described esprit exploits an underlying rotational invariance of signal subspaces spanned by two temporally displaced data sets the new approach has several advantages including improved resolution over pisarenko s technique for harmonic retrieval

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

an algorithm for pole zero modeling and spectral analysis an explicit connection between fitting exponential models and pole zero models to observed data is made the fitting problem is formulated as a constrained nonlinear minimization problem this problem is then solved using a simplified iterative algorithm the algorithm is applied to simulated data and the performance of the algorithm is compared to previous results

discrete lossy gray wyner revisited second order asymptotics large and moderate deviations in this paper we revisit the discrete lossy gray wyner problem in particular we derive its optimal second order coding rate region its error exponent reliability function and its moderate deviations constant under mild conditions on the source to obtain the second order asymptotics we extend some ideas from watanabe s work in particular we leverage the properties of an appropriate generalization of the conditional distortion tilted information density which was first introduced by kostina and verdu the converse part uses a perturbation argument by gu and effros in their strong converse proof of the discrete gray wyner problem the achievability part uses two novel elements 1 a generalization of various type covering lemmas and 2 the uniform continuity of the conditional rate distortion function in both the source joint distribution and the distortion level to obtain the error exponent for the achievability part we use the same generalized type covering lemma and for the converse we use the strong converse together with a change of measure technique finally to obtain the moderate deviations constant we apply the moderate deviations theorem to probabilities defined in terms of information spectrum quantities

an iterative perturbation method for the pressure equation in the simulation of miscible displacement in porous media the miscible displacement problem in porous media is modeled by a nonlinear coupled system of two partial differential equations the pressure velocity equation and the concentration equation an iterative perturbation procedure is proposed and analyzed for the pressure velocity equation which is capable of producing as accurate a velocity approximation as the mixed finite element method and which requires the solution of symmetric positive definite linear systems only the velocity variable is involved in the linear systems and the pressure variable is obtained by substitution trivially applying perturbation methods can only give an error o epsilon while our iterative scheme can improve the error to o epsilon m at the m th iteration level where epsilon is a small positive number thus the convergence rate of our iterative procedure is o epsilon and consequently a small number of iterations is required theoretical convergence analysis and numerical experiments are presented to show the efficiency and accuracy of our method

the statistical performance of the music and the minimum norm algorithms in resolving plane waves in noise this paper presents an asymptotic statistical analysis of the null spectra of two eigen assisted methods music 1 and minimum norm 2 for resolving independent closely spaced plane waves in noise particular attention is paid to the average deviation of the null spectra from zero at the true angles of arrival for the plane waves these deviations are expressed as functions of signal to noise ratios number of array elements angular separation of emitters and the number of snapshots in the case of music an approximate expression is derived for the resolution threshold of two plane waves with equal power in noise this result is validated by monte carlo simulations

an information spectrum approach to weak variable length source coding with side information this paper studies variable length vl source coding of general sources with side information novel one shot coding bounds for slepian wolf sw coding which give nonasymptotic tradeoff between the error probability and the codeword length of vl sw coding are established one shot results are applied to asymptotic analysis and a general formula for the optimal coding rate achievable by weakly lossless vl sw coding i e vl sw coding with vanishing error probability is derived our general formula reveals how the encoder side information and or vl coding improve the optimal coding rate in the general setting in addition it is shown that if the encoder side information is useless in weakly lossless vl coding then it is also useless even in the case where the error probability may be positive asymptotically

preconditioning in h div and applications we consider the solution of the system of linear algebraic equations which arises from the finite element discretization of boundary value problems associated to the differential operator i grad div the natural setting for such problems is in the hilbert space h div and the variational formulation is based on the inner product in h div we show how to construct preconditioners for these equations using both domain decomposition and multigrid techniques these preconditioners are shown to be spectrally equivalent to the inverse of the operator as a consequence they may be used to precondition iterative methods so that any given error reduction may be achieved in a finite number of iterations with the number independent of the mesh discretization we describe applications of these results to the efficient solution of mixed and least squares finite element approximations of elliptic boundary value problems

implementation of adaptive array algorithms some new efficient and numerically stable algorithms for the recursive solution of matrix problems arising in optimal beam forming and direction finding are described and analyzed the matrix problems considered are systems of linear equations and spectral decomposition while recursive solution procedures based on the matrix inversion lemma may be unstable ours are stable furthermore these algorithms are extremely fast

simulation of random processes and rate distortion theory we study the randomness necessary for the simulation of a random process with given distributions on terms of the finite precision resolvability of the process finite precision resolvability is defined as the minimal random bit rate required by the simulator as a function of the accuracy with which the distributions are replicated the accuracy is quantified by means of various measures variational distance divergence orstein 1973 prohorov 1956 and related measures of distance between the distributions of random process in the case of ornstein prohorov and other distances of the kantorovich vasershtein type we show that the finite precision resolvability is equal to the rate distortion function with a fidelity criterion derived from the accuracy measure this connection leads to new results on nonstationary rate distortion theory in the case of variational distance the resolvability of stationary ergodic processes is shown to equal entropy rate regardless of the allowed accuracy in the case of normalized divergence explicit expressions for finite precision resolvability are obtained in many cases of interest and connections with data compression with minimum probability of block error are shown

smoothing brascamp lieb inequalities and strong converses of coding theorems the brascamp lieb inequality in functional analysis can be viewed as a measure of the uncorrelatedness of a joint probability distribution we define the smooth brascamp lieb bl divergence as the infimum of the best constant in the brascamp lieb inequality under a perturbation of the joint probability distribution an information spectrum upper bound on the smooth bl divergence is proved using properties of the subgradient of a certain convex functional in particular in the i i d setting such an infimum converges to the best constant in a certain mutual information inequality we then derive new single shot converse bounds for the omniscient helper common randomness generation problem and the gray wyner source coding problem in terms of the smooth bl divergence where the proof relies on the functional formulation of the brascamp lieb inequality exact second order rates are thus obtained in the stationary memoryless and nonvanishing error setting these offer rare instances of strong converses second order converses for continuous sources when the rate region involves auxiliary random variables

parameter estimation of superimposed signals using the em algorithm a computationally efficient algorithm for parameter estimation of superimposed signals based on the two step iterative em estimate and maximize with an e step and an m step algorithm is developed the idea is to decompose the observed data into their signal components and then to estimate the parameters of each signal component separately the algorithm iterates back and forth using the current parameter estimates to decompose the observed data better and thus increase the likelihood of the next parameter estimates the application of the algorithm to the multipath time delay and multiple source location estimation problems is considered

on the dispersions of three network information theory problems we analyze the dispersions of distributed lossless source coding the slepian wolf problem the multiple access channel and the asymmetric broadcast channel for the two encoder slepian wolf problem we introduce a quantity known as the entropy dispersion matrix which is analogous to the scalar dispersions that have gained interest recently we prove a global dispersion result that can be expressed in terms of this entropy dispersion matrix and provides intuition on the approximate rate losses at a given blocklength and error probability to gain better intuition about the rate at which the nonasymptotic rate region converges to the slepian wolf boundary we define and characterize two operational dispersions 1 the local dispersion and 2 the weighted sum rate dispersion the former represents the rate of convergence to a point on the slepian wolf boundary whereas the latter represents the fastest rate for which a weighted sum of the two rates converges to its asymptotic fundamental limit interestingly when we approach either of the two corner points the local dispersion is characterized not by a univariate gaussian but a bivariate one as well as a subset of off diagonal elements of the aforementioned entropy dispersion matrix finally we demonstrate the versatility of our achievability proof technique by providing inner bounds for the multiple access channel and the asymmetric broadcast channel in terms of dispersion matrices all our proofs are unified by a so called vector rate redundancy theorem which is proved using the multidimensional berry esseen theorem

random number conversion and locc conversion via restricted storage we consider random number conversion rnc through random number storage with restricted size we clarify the relation between the performance of rnc and the size of storage in the framework of the first and second order asymptotics and derive their rate regions then we show that the results for rnc with restricted storage recover those for conventional rnc without storage in the limit of storage size to treat rnc via restricted storage we introduce a new kind of probability distributions named generalized rayleigh normal distributions using the generalized rayleigh normal distributions we can describe the second order asymptotic behavior of rnc via restricted storage in a unified manner as an application to quantum information theory we analyze locc conversion via entanglement storage with restricted size moreover we derive the optimal locc compression rate under a constraint of conversion accuracy

second order region for gray wyner network the coding problem over the gray wyner network is studied from the second order coding rates perspective a tilted information density for this network is introduced in the spirit of kostina verdu and under a certain regularity condition the second order region is characterized in terms of the variance of this tilted information density and the tangent vector of the first order region the second order region is proved by the type method the achievability part is proved by the type covering argument and the converse part is proved by a refinement of the perturbation approach that was used by gu effros to show the strong converse of the gray wyner network this is the first instance that the second order region is characterized for a multi terminal problem where the characterization of the first order region involves an auxiliary random variable

maximum likelihood estimation of the parameters of multiple sinusoids from noisy measurements the problem of estimating the frequencies phases and amplitudes of sinusoidal signals is considered a simplified maximum likelihood gauss newton algorithm which provides asymptotically efficient estimates of these parameters is proposed initial estimates for this algorithm are obtained by a variation of the overdetermined yule walker method and periodogram based procedure use of the maximum likelihood gauss newton algorithm is not however limited to this particular initialization method some other possibilities to get suitable initial estimates are briefly discussed an analytical and numerical study of the shape of the likelihood function associated with the sinusoids in noise process reveals its multimodal structure and clearly sets the importance of the initialization procedure some numerical examples are presented to illustrate the performance of the proposed estimation procedure comparison to the performance corresponding to the cramer rao lower bound is also presented using a simple expression for the asymptotic cramer rao bound covariance matrix derived in the paper

fixed length lossy compression in the finite blocklength regime this paper studies the minimum achievable source coding rate as a function of blocklength n and probability that the distortion exceeds a given level d tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength for stationary memoryless sources with separable distortion the minimum rate achievable is shown to be closely approximated by r d v d n q 1 where r d is the rate distortion function v d is the rate dispersion a characteristic of the source which measures its stochastic variability and q 1 is the inverse of the standard gaussian complementary cumulative distribution function

on the rate of convergence in the multivariate clt berry esseen theorems are proved in the multidimensional central limit theorem without using fourier methods an effective and simple estimate of the error in the clt for sums and convex sets using stein s method and induction is derived furthermore the error in the clt for multivariate functions of independent random elements is estimated extending results of van zwet and friedrich to the multivariate case

maximum likelihood localization of multiple sources by alternating projection an algorithm referred to as apm for computing the maximum likelihood estimator of the locations of simple sources in passive sensor arrays is presented the algorithm is equally applicable to the case of coherent signals and to the case of a single snapshot the algorithm is iterative the maximum of the likelihood function is computed by successive approximations the convergence of the algorithm to the global maximum is demonstrated for a variety of scenarios the key to this global convergence is the initialization scheme

performance analysis of spatial smoothing with interpolated arrays the interpolated spatial smoothing algorithm is a computationally efficient method for estimating the directions of arrival doas of signals some of which may be perfectly correlated it extends the spatial smoothing method to arbitrary array geometries a statistical performance analysis of the algorithm is presented closed form expressions for the covariance matrix of the doa estimation errors are derived using a perturbation analysis evaluating these expressions for specific cases and comparing them to the cramer rao lower bound for the doa estimates provides insight into the statistical efficiency of this algorithm the formulas for the error covariance are quite general and can be specialized to provide results for other doa estimation algorithms as well

a tight upper bound for the third order asymptotics for most discrete memoryless channels this paper shows that the logarithm of the e error capacity average error probability for n uses of a discrete memoryless channel dmc is upper bounded by the normal approximation plus a third order term that does not exceed 1 2 logn o 1 if the e dispersion of the channel is positive this matches a lower bound by y polyanskiy 2010 for dmcs with positive reverse dispersion if the e dispersion vanishes the logarithm of the e error capacity is upper bounded by n times the capacity plus a constant term except for a small class of dmcs and e 1 2

analysis of the inexact uzawa algorithm for saddle point problems in this paper we consider the so called inexact uzawa algorithm for iteratively solving linear block saddle point problems such saddle point problems arise for example in finite element and finite difference discretizations of stokes equations the equations of elasticity and mixed finite element discretization of second order problems we consider both the linear and nonlinear variants of the inexact uzawa iteration we show that the linear method always converges as long as the preconditioners defining the algorithm are properly scaled bounds for the rate of convergence are provided in terms of the rate of convergence for the preconditioned uzawa algorithm and the reduction factor corresponding to the preconditioner for the upper left hand block in the case of nonlinear iteration the inexact uzawa algorithm is shown to converge provided that the nonlinear process approximating the inverse of the upper left hand block is of sufficient accuracy bounds for the nonlinear iteration are given in terms of this accuracy parameter and the rate of convergence of the preconditioned linear uzawa algorithm applications to the stokes equations and mixed finite element discretization of second order elliptic problems are discussed and finally the results of numerical experiments involving the algorithms are presented

second order noiseless source coding theorems shannon s celebrated source coding theorem can be viewed as a one sided law of large numbers we formulate second order noiseless source coding theorems for the deviation of the codeword lengths from the entropy for a class of sources that includes markov chains we prove a one sided central limit theorem and a law of the iterated logarithm

on the performance of signal subspace processing part i narrow band systems this paper presents an analytical evaluation of detection determination of the number of sources and estimation performances of narrow band signal subspace processing for multiple source direction finding the probabilities of underestimating and overestimating the number of sources are derived under asymptotic conditions and around the threshold regions in terms of the choice of a penalty function and signal noise and array parameters for the cases of at most two closely spaced sources in the spatially white noise a scalar measure is introduced for the evaluation of the quality of the estimated signal subspace based on the statistics of this measure performance thresholds are demonstrated for the signal to noise ratio angle separation and correlation between two equipowered sources

entanglement assisted capacity of a quantum channel and the reverse shannon theorem the entanglement assisted classical capacity of a noisy quantum channel c sub e is the amount of information per channel use that can be sent over the channel in the limit of many uses of the channel assuming that the sender and receiver have access to the resource of shared quantum entanglement which may be used up by the communication protocol we show that the capacity c sub e is given by an expression parallel to that for the capacity of a purely classical channel i e the maximum over channel inputs spl rho of the entropy of the channel input plus the entropy of the channel output minus their joint entropy the latter being defined as the entropy of an entangled purification of spl rho after half of it has passed through the channel we calculate entanglement assisted capacities for two interesting quantum channels the qubit amplitude damping channel and the bosonic channel with amplification attenuation and gaussian noise we discuss how many independent parameters are required to completely characterize the asymptotic behavior of a general quantum channel alone or in the presence of ancillary resources such as prior entanglement in the classical analog of entanglement assisted communication communication over a discrete memoryless channel dmc between parties who share prior random information we show that one parameter is sufficient i e that in the presence of prior shared random information all dmcs of equal capacity can simulate one another with unit asymptotic efficiency

writing on dirty paper corresp a channel with output y x s z is examined the state s sim n 0 qi and the noise z sim n 0 ni are multivariate gaussian random variables i is the identity matrix the input x in r n satisfies the power constraint l n sum i 1 n x i 2 leq p if s is unknown to both transmitter and receiver then the capacity is frac 1 2 ln 1 p n q nats per channel use however if the state s is known to the encoder the capacity is shown to be c ast frac 1 2 ln 1 p n independent of q this is also the capacity of a standard gaussian channel with signal to noise power ratio p n therefore the state s does not affect the capacity of the channel even though s is unknown to the receiver it is shown that the optimal transmitter adapts its signal to the state s rather than attempting to cancel it

reliability in source coding with side information we study error exponents for source coding with side information both achievable exponents and converse bounds are obtained for the following two cases lossless source coding with coded information and lossy source coding with full side information wyner ziv these results recover and extend several existing results on source coding error exponents and are tight in some circumstances our bounds have a natural interpretation as a two player game between nature and the code designer with nature seeking to minimize the exponent and the code designer seeking to maximize it in the wyner ziv problem our analysis exposes a tension in the choice of test channel with the optimal test channel balancing two competing error events the gaussian and binary erasure cases are examined in detail

non asymptotic converse bounds and refined asymptotics for two source coding problems in this paper we revisit two multi terminal lossy source coding problems the lossy source coding problem with side information available at the encoder and one of the two decoders which we term as the kaspi problem kaspi 1994 and the multiple description coding problem with one semi deterministic distortion measure which we refer to as the fu yeung problem fu and yeung 2002 for the kaspi problem we first present the properties of optimal test channels subsequently we generalize the notion of the distortion tilted information density for the lossy source coding problem to the kaspi problem and prove a non asymptotic converse bound using the properties of optimal test channels and the well defined distortion tilted information density finally for discrete memoryless sources we derive refined asymptotics which includes the second order large and moderate deviations asymptotics in the converse proof of second order asymptotics we apply the berry esseen theorem to the derived non asymptotic converse bound the achievability proof follows by first proving a type covering lemma tailored to the kaspi problem then properly taylor expanding the well defined distortion tilted information densities and finally applying the berry esseen theorem we then generalize the methods used in the kaspi problem to the fu yeung problem as a result we obtain the properties of optimal test channels for the minimum sum rate function a non asymptotic converse bound and refined asymptotics for discrete memoryless sources since the successive refinement problem is a special case of the fu yeung problem as a by product we obtain a non asymptotic converse bound for the successive refinement problem which is a strict generalization of the non asymptotic converse bound for successively refinable sources zhou tan and motani 2017

a general formula for channel capacity a formula for the capacity of arbitrary single user channels without feedback not necessarily information stable stationary etc is proved capacity is shown to equal the supremum over all input processes of the input output inf information rate defined as the liminf in probability of the normalized information density the key to this result is a new converse approach based on a simple new lower bound on the error probability of m ary hypothesis tests among equiprobable hypotheses a necessary and sufficient condition for the validity of the strong converse is given as well as general expressions for spl epsiv capacity

a preconditioning technique for indefinite systems resulting from mixed approximations of elliptic problems this paper provides a preconditioned iterative technique for the solution of saddle point problems these problems typically arise in the numerical approximation of partial differential equations by lagrange multiplier techniques and or mixed methods the saddle point problem is reformulated as a symmetric positive definite system which is then solved by conjugate gradient iteration applications to the equations of elasticity and stokes are discussed and the results of numerical experiments are given

a rate of convergence result for a universal d semifaithful code the problem of optimal rate universal coding is considered in the context of rate distortion theory a d semifaithful universal coding scheme for discrete memoryless sources is given the main result is a refined covering lemma based on the random coding argument and the method of types the average codelength of the code is shown to approach its lower bound the rate distortion function at a rate o n sup 1 log n and this is conjectured to be optimal based on a result of a j pilc 1968 issues of constructiveness and universality are addressed

music maximum likelihood and cramer rao bound the performance of the music and ml methods is studied and their statistical efficiency is analyzed the cramer rao bound crb for the estimation problems is derived and some useful properties of the crb covariance matrix are established the relationship between the music and ml estimators is investigated as well a numerical study is reported of the statistical efficiency of the music estimator for the problem of finding the directions of two plane waves using a uniform linear array an exact description of the results is included

multilevel schwarz methods for elliptic problems with discontinuous coefficients in three dimensions multilevel schwarz methods are developed for a conforming finite element approximation of second order elliptic problems we focus on problems in three dimensions with possibly large jumps in the coefficients across the interface separating the subregions we establish a condition number estimate for the iterative operator which is independent of the coefficients and grows at most as the square of the number of levels we also characterize a class of distributions of the coefficients called quasi monotone for which the weighted l 2 projection is stable and for which we can use the standard piecewise linear functions as a coarse space in this case we obtain optimal methods i e bounds which are independent of the number of levels and subregions we also design and analyze multilevel methods with new coarse spaces given by simple explicit formulas we consider nonuniform meshes and conclude by an analysis of multilevel iterative substructuring methods

second order asymptotics in fixed length source coding and intrinsic randomness there is a difference between the optimal rates of fixed length source coding and intrinsic randomness when we care about the second order asymptotics we prove this difference for general information sources and then investigate independent and identically distributed i i d random variables and markovian variables as examples the difference is demonstrated through an investigation of the second order asymptotic behavior of the rates a universal fixed length source code attaining the second order optimal rate is also proposed the difference between the rates of fixed length source coding and intrinsic randomness proves that the outputs of fixed length source codes are not uniformly distributed

general nonasymptotic and asymptotic formulas in channel resolvability and identification capacity and their application to the wiretap channel several nonasymptotic formulas are established in channel resolvability and identification capacity and they are applied to the wiretap channel by using these formulas the epsi capacities of the above three problems are considered in the most general setting where no structural assumptions such as the stationary memoryless property are made on a channel as a result we solve an open problem proposed by han and verduacute moreover we obtain lower bounds of the exponents of error probability and the wiretapper s information in the wiretap channel

iterative schemes for mixed finite element methods with applications to elasticity and compressible flow problems iterative schemes for mixed finite element methods are proposed and analyzed in two abstract formulations the first one has applications to elliptic equations and incompressible fluid flow problems while the second has applications to linear elasticity and compressible stokes problems these schemes are constructed through iteratively penalizing the mixed finite element scheme of which iterated penalty method and augmented lagrangian method are special cases convergence theorems are demonstrated in abstract formulations in hilbert spaces and applications to individual physical problems are considered as examples theoretical analysis and computational experiments both show that the proposed schemes have very fast convergence a few iterations are normally enough to reduce the iterative error to a prescribed precision numerical examples with continuous and discontinuous coefficients are presented

on the expectation of the product of four matrix valued gaussian random variables the formula for the expectation of the product of four scalar real gaussian random variables is generalized to matrix valued real or complex gaussian random variables as an application of the extended formula a simple derivation is presented of the covariance matrix of instrumental variable estimates of parameters in multivariable regression models

capacity and random coding exponents for channel coding with side information capacity formulas and random coding exponents are derived for a generalized family of gel fand pinsker coding problems these exponents yield asymptotic upper bounds on the achievable log probability of error in our model information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence and the channel is selected by a hypothetical adversary partial information about the state sequence is available to the encoder adversary and decoder the design of the transmitter is subject to a cost constraint two families of channels are considered 1 compound discrete memoryless channels cdmc and 2 channels with arbitrary memory subject to an additive cost constraint or more generally to a hard constraint on the conditional type of the channel output given the input both problems are closely connected the random coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder which may be thought of as an empirical generalized maximum a posteriori decoder for channels with arbitrary memory the random coding exponents are larger than their cdmc counterparts applications of this study include watermarking data hiding communication in presence of partially known interferers and problems such as broadcast channels all of which involve the fundamental idea of binning

optimality of high resolution array processing using the eigensystem approach in the classical approach to underwater passive listening the medium is sampled in a convenient number of look directions from which the signals are estimated in order to build an image of the noise field in contrast a modern trend is to consider the noise field as a global entity depending on few parameters to be estimated simultaneously in a gaussian context it is worthwhile to consider the application of likelihood methods in order to derive a detection test for the number of sources and estimators for their locations and spectral levels this paper aims to compute such estimators when the wavefront shapes are not assumed known a priori this justifies results previously found using the asymptotical properties of the eigenvalue eigenvector decomposition of the estimated spectral density matrix of the sensor signals they have led to a variety of high resolution array processing methods more specifically a covariance matrix test for equality of the smallest eigenvalues is presented for source detection for source localization a best fit method and a test of orthogonality between the smallest eigenvectors and the source vectors are discussed

channel simulation with quantum side information we study and solve the problem of classical channel simulation with quantum side information at the receiver this is a generalization of both the classical reverse shannon theorem and the classical quantum slepian wolf problem the optimal noiseless communication rate is found to be reduced from the mutual information between the channel input and output by the holevo information between the channel output and the quantum side information our main theorem has two important corollaries the first is a quantum generalization of the wyner ziv problem rate distortion theory with quantum side information the second is an alternative proof of the tradeoff between classical communication and common randomness distilled from a quantum state the fully quantum generalization of the problem considered is quantum state redistribution here the sender and receiver share a mixed quantum state and the sender wants to transfer part of her state to the receiver using entanglement and quantum communication we present outer and inner bounds on the achievable rate pairs

on gradient type optimization method utilizing mixed finite element approximation for optimal boundary control problem governed by bi harmonic equation in this paper a numerical method based on mixed finite element method for optimal boundary control problem governed by bi harmonic equation is presented the system of optimality equations consisting of state and costate function is derived based on the system a gradient type optimization method using a mixed finite approximation for solving the optimal boundary control is developed in our algorithm the trace of vortex function of costate equation on boundary plays a key role finally a local error analysis at every iteration for this gradient type optimization method is given

information spectrum approach to second order coding rate in channel coding in this paper second order coding rate of channel coding is discussed for general sequence of channels the optimum second order transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method we apply this result to the discrete memoryless case the discrete memoryless case with a cost constraint the additive markovian case and the gaussian channel case with an energy constraint we also clarify that the gallager bound does not give the optimum evaluation in the second order coding rate

analysis of some mixed elements for the stokes problem abstract in this paper we discuss some mixed finite element methods related to the reduced integration penalty method for solving the stokes problem we prove optimal order error estimates for bilinear constant and biquadratic bilinear velocity pressure finite element solutions the result for the biquadratic bilinear element is new while that for the bilinear constant element improves the convergence analysis of johnson and pitkaranta 1982 in the degenerate case when the penalty parameter is set to be zero our results reduce to some related known results proved in by brezzi and fortin 1991 for the bilinear constant element and bercovier and pironneau 1979 for the biquadratic bilinear element our theoretical results are consistent with the numerical results reported by carey and krishnan 1982 and oden et al 1982

a case where interference does not affect the channel dispersion in 1975 carleial presented a special case of an interference channel called the very strong interference regime in which the interference does not reduce the capacity of the constituent point to point gaussian channels in this paper we show that in the strictly very strong interference regime the dispersions are similarly unaffected more precisely in this paper we characterize the second order coding rates of the gaussian interference channel in the strictly very strong interference regime in other words we characterize the speed of convergence of rates of optimal block codes toward a boundary point of the rectangular capacity region these second order coding rates are expressed in terms of the average probability of error and variances of appropriately defined information densities which coincide with the dispersion of the single user gaussian channel this allows us to conclude that the dispersions are unaffected by interference in this channel model