designing structured tight frames via an alternating projection method tight frames also known as general welch bound equality sequences generalize orthonormal systems numerous applications including communications coding and sparse approximation require finite dimensional tight frames that possess additional structural properties this paper proposes an alternating projection method that is versatile enough to solve a huge class of inverse eigenvalue problems ieps which includes the frame design problem to apply this method one needs only to solve a matrix nearness problem that arises naturally from the design specifications therefore it is the fast and easy to develop versions of the algorithm that target new design problems alternating projection will often succeed even if algebraic constructions are unavailable to demonstrate that alternating projection is an effective tool for frame design the paper studies some important structural properties in detail first it addresses the most basic design problem constructing tight frames with prescribed vector norms then it discusses equiangular tight frames which are natural dictionaries for sparse approximation finally it examines tight frames whose individual vectors have low peak to average power ratio par which is a valuable property for code division multiple access cdma applications numerical experiments show that the proposed algorithm succeeds in each of these three cases the appendices investigate the convergence properties of the algorithm

on the limiting distribution of lempel ziv 78 redundancy for memoryless sources we study the lempel ziv 78 algorithm and show that its normalized redundancy rate tends to a gaussian distribution for memoryless sources we accomplish it by extending findings from our 1995 paper in particular by presenting a new simplified proof of the central limit theorem clt for the number of phrases in the lz 78 algorithm we first analyze the asymptotic behavior of the total path length in the associated digital search tree built from independent sequences then a renewal theory type argument yields clt for lz 78 scheme here we extend our analysis of lz 78 algorithm to present new results on the convergence of moments moderate and large deviations and clt for the normalized redundancy in particular we confirm that the average redundancy rate decays as 1 log n and we find that the variance is of order 1 n where n is the length of the text

liouvillian solutions of n th order homogeneous linear differential equations be a linear differential operator with coefficients in f a finite algebraic extension of q x we shall show that one can find in a finite number of steps a basis for the vector space of liouvillian solutions of l y 0 i e those solutions which can be built up from the rational functions by algebraic operations taking exponentials and by integration see section 2 for a precise definition in particular we show how to decide if all solution of l y 0 are liouvillian or if there are any solutions which are liouvillian our algorithm in conjunction with the algorithm in 8 also allows one to determine the algebraic relationships among the liouvillian solutions of l y 0 and in particular determine if all solutions are algebraic c f 111 for second order linear homogeneous equations over q x the problem of determining if all solutions are algebraic functions was considered by fuchs klein and schwarz but none of these mathematicians seems to have presented a complete decision procedure building on the work of klein baldassarri and dwork 21 have given such a procedure baldassarri 11 has extended this to consider linear homogeneous equations whose coefficients are algebraic functions for third order equations painleve and boulanger gave a procedure which in effect reduced the problem of finding algebraic solutions of l y 0 to the problem of effectively bounding the torsion of the jacobian variety of a given curve a complete procedure for deciding if all solutions of an n th order homo

maximin joint optimization of transmitting code and receiving filter in radar and communications in this paper we conduct the joint design of transmitting sequence s and receiving filters subject to the peak to average ratio par constraint in radar and communications applications we consider optimizing the worst case performance and the resulting optimization problem takes a maximin format we propose two algorithms based on the mm majorization minimization or minorization maximization method as opposed to the traditional epigraph based smooth reformulation on top of that both algorithms are guaranteed to converge to a b oulingand stationary solution and b stationarity is the appropriate stationarity condition for problems with a nonconvex constraint set the proposed algorithms successively solve a series of simple convex problems that enjoy low computational complexity numerical simulations have shown that the proposed algorithms empirically achieve slightly higher objective values and converge faster in terms of cpu time than the existing methods

optimization of generalized mean square error in signal processing and communication abstract two matrix optimization problems are analyzed these problems arise in signal processing and communication in the first problem the trace of the mean square error matrix is minimized subject to a power constraint the solution is the training sequence which yields the best estimate of a communication channel the solution is expressed in terms of the eigenvalues and eigenvectors of correlation and covariance matrices describing the communication and an unknown permutation our analysis exhibits the optimal permutation when the power is either very large or very small based on the structure of the optimal permutation in these limiting cases we propose a small class of permutations to focus on when computing the optimal permutation for arbitrary power in numerical experiments with randomly generated matrices the optimal solution is contained in the proposed permutation class with high probability the second problem is connected with the optimization of the sum capacity of a communication channel the second problem which is obtained from the first by replacing the trace operator in the objective function by the determinant minimizes the product of eigenvalues while the first problem minimizes the sum of eigenvalues for small values of the power both problems have the same solution as the power increases the solutions are different since the permutation matrix appearing in the solution of the trace problem is not present in the solution of the determinant problem for large power the ordering of the eigenvectors in the solution of the trace problem is the opposite of the ordering in the determinant problem

optimum sequence multisets for synchronous code division multiple access channels it is shown that the sum capacity of the symbol synchronous code division multiple access channel with equal average input energy constraints is maximized precisely by those spreading sequence multisets that meet welch s lower bound on total squared correlation it is further shown that the symmetric capacity of the channel determined by these same sequence multisets is equal to the sum capacity

counting markov types balanced matrices and eulerian graphs the method of types is one of the most popular techniques in information theory and combinatorics two sequences of equal length have the same type if they have identical empirical distributions in this paper we focus on markov types that is sequences generated by a markov source of order one we note that sequences having the same markov type share the same so called balanced frequency matrix that counts the number of distinct pairs of symbols we enumerate the number of markov types for sequences of length over an alphabet of size this turns out to be asymptotically equivalent to estimating the number of the balanced frequency matrices the number of integer solutions of a system of linear diophantine equations and the number of connected eulerian multigraphs for fixed we prove that the number of markov types is asymptotically equal to d m nm2 m m2 m where we give an integral representation for d m for m we conclude that asymptotically the number of types is equivalent to 2m3m 2em2 m2m22m m 2 nm2 m provided that m o n1 4 these findings are derived by analytical techniques ranging from analytic combinatorics to multidimensional generating functions to the saddle point method

generalized hypergeometric functions and rational curves on calabi yau complete intersections in toric varieties we formulate general conjectures about the relationship between the a model connection on the cohomology of ad dimensional calabi yau complete intersectionv ofr hypersurfacesv1 vr in a toric varietyp and the system of differential operators annihilating the special generalized hypergeometric series 0 constructed from the fan using this generalized hypergeometric series we propose conjectural mirrorsv ofv and the canonicalq coordinates on the moduli spaces of calabi yau manifolds

the method of types information theory the method of types is one of the key technical tools in shannon theory and this tool is valuable also in other fields in this paper some key applications are presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method and a wide selection of further applications are surveyed these range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems while the method of types is suitable primarily for discrete memoryless models its extensions to certain models with memory are also discussed

optimal signature design for spread spectrum steganography for any given host image or group of host images and any block transform domain of interest we find the signature vector that when used for spread spectrum ss message embedding maximizes the signal to interference plus noise ratio sinr at the output of the corresponding maximum sinr linear filter we establish that under a colored gaussian assumption on the transform domain host data the same derived signature minimizes host distortion for any target message recovery error rate and maximizes the shannon capacity of the covert steganographic link then we derive jointly optimal signature and linear processor designs for ss embedding in linearly modified transform domain host data and demonstrate orders of magnitude improvement over current ss steganographic practices optimized multisignature multimessage embedding in the same host data is studied as well

cdma codeword optimization interference avoidance and convergence via class warfare interference avoidance has been shown to reduce total square correlation tsc for given ensembles of user signature waveforms codewords in a synchronous code division multiple access cdma system in all experiments we have conducted sequential application of interference avoidance produces an optimal codeword set when starting from randomly chosen initial codewords here we provide the first formal proof of convergence to optimal codeword ensembles for greedy interference avoidance algorithms augmented by a technique called class warfare whereby users which reside in more heavily loaded areas of the signal space purposely interfere with attack the reception of users in less crowded areas coordination of deliberate interference by a complete class of aggrieved user is also sometimes necessary such attacks and subsequent codeword adjustment by attacked users are shown to strictly decrease tsc along the way we also show using linear algebra and a variant of stochastic ordering equivalence between minimization of tsc and maximization of sum capacity

bandwidth constrained signature waveforms and walsh signal space receivers for synchronous cdma systems synchronous cdma systems whose transmission bandwidth is quantified through the fractional out of band energy fobe constraint are considered either a conventional matched filter mf receiver or a minimum mean square error mmse receiver is employed for users data detection the total squared correlation tsc and the total mean square error tmse are proposed as the performance parameters for the mf and mmse receivers respectively these parameters need to be minimized in order to maximize the signal to interference ratios sirs at the receivers outputs for a given fobe bandwidth constraint the sets of signature waveforms that minimize either tsc or tmse are obtained from the prolate spheroidal wave functions pswfs furthermore if the number of users is the size of a hadamard matrix then optimal signature waveforms can be obtained to maximize the individual sir for every user due to the complicated nature of the pswfs simplified mf and mmse receivers based on the walsh signal space are developed

optimal sequences for cdma under colored noise a schur saddle function property we consider direct sequence code division multiple access ds cdma modeling interference from users communicating with neighboring base stations by additive colored noise we consider two types of receiver structures first we consider the information theoretically optimal receiver and use the sum capacity of the channel as our performance measure second we consider the linear minimum mean square error lmmse receiver and use the signal to interference ratio sir of the estimate of the symbol transmitted as our performance measure our main result is a constructive characterization of the possible performance in both these scenarios a central contribution of this characterization is the derivation of a qualitative feature of the optimal performance measure in both the scenarios studied we show that the sum capacity is a saddle function it is convex in the additive noise covariances and concave in the user received powers in the linear receiver case we show that the mini average power required to meet a set of target performance requirements of the users is a saddle function it is convex in the additive noise covariances and concave in the set of performance requirements

interval algorithm for random number generation the problem of generating a random number with an arbitrary probability distribution by using a general biased m coin is studied an efficient and very simple algorithm based on the successive refinement of partitions of the unit interval 0 1 which we call the interval algorithm is proposed a fairly tight evaluation on the efficiency is given generalizations of the interval algorithm to the following cases are investigated 1 output sequence is independent and identically distributed i i d 2 output sequence is markov 3 input sequence is markov 4 input sequence and output sequence are both subject to arbitrary stochastic processes

optimal sequences and sum capacity of synchronous cdma systems the sum capacity of a multiuser synchronous cdma system is completely characterized in the general case of asymmetric user power constraints this solves the open problem posed by rupf and massey see ibid vol 40 p 1261 6 1994 which had solved the equal power constraint case we identify the signature sequences with real components that achieve sum capacity and indicate a simple recursive algorithm to construct them

optimal orthogonal carriers and sum sinr sum capacity of the multiple access vector channel for the general correlated multiple access vector channel we derive algebraically the sum sinr signal to interference plus noise ratio optimal orthonormal set of carriers for any given set of carrier amplitudes if in addition the correlated multiple access channel is assumed gaussian the sum capacity optimal orthonormal carrier set design and the jointly sum capacity optimal orthonormal carrier and power assignment for a given power budget follow readily

estimating the number of states of a finite state source the problem of estimating the number of states of a finite alphabet finite state source is investigated an estimator is developed that asymptotically attains the minimum probability of understanding the number of states among all estimators with a prescribed exponential decay rate of overestimation probability the proposed estimator relies on the lempel ziv data compression algorithm in an intuitively appealing manner

maximizing the spectral efficiency of coded cdma under successive decoding we investigate the spectral efficiency achievable by random synchronous code division multiple access cdma with quaternary phase shift keying qpsk modulation and binary error control codes in the large system limit where the number of users the spreading factor and the code block length go to infinity for given codes we maximize spectral efficiency assuming a minimum mean square error mmse successive stripping decoder for the cases of equal rate and equal power users in both cases the maximization of spectral efficiency can be formulated as a linear program and admits a simple closed form solution that can be readily interpreted in terms of power and rate control we provide examples of the proposed optimization methods based on off the shelf low density parity check ldpc codes and we investigate by simulation the performance of practical systems with finite code block length

new bounds and optimal binary signature sets part ii aperiodic total squared correlation we derive new bounds on the aperiodic total squared correlation atsc of binary antipodal signature sets for any number of signatures k and any signature length l we then present optimal designs that achieve the new bounds for several k l cases as interesting arguably side results we show that individual maximal merit factor sequences for example barker sequences are single user atsc optimal while neither the familiar gold nor the kasami set designs are atsc optimal in general the atsc optimal signature set designs provided in this work are in this sense better suited for asynchronous and or multipath code division multiplexing applications

stability of distributed power and signature sequence control for cdma systems a game theoretic framework the problem of power control and signature sequence adaptation in code division multiple access cdma systems is studied under a game theoretic framework each user tries to maximize his own utility function which may be different from others sufficient conditions for the existence of an equilibrium point in this multi objective optimization problem are identified the methodology for analyzing this class of problems is illustrated by examples

universal delay limited simulation universal delay limited simulation of an unknown information source of a certain parametric family e g the family of memoryless sources or markov sources of a given order given a training sequence from that source and a stream of independent and uniformly distributed bits is considered the goal of universal simulation is that the probability law of the generated sequence be identical to that of the training sequence with minimum mutual information between the random processes generating both sequences in the delay limited setting the simulation algorithm generates a random sequence sequentially by delivering one symbol for each training symbol that is made available after a given initial delay whereas the random bits are assumed to be available on demand in this paper the optimal universal delay limited simulation scheme is characterized for broad parametric families and the mutual information achieved by the proposed scheme is analyzed the results are extended to a setting of variable delay

maximal degeneracy points of gkz systems motivated by mirror symmetry we study certain integral representations of solutions to the gel fand kapranov zelevinsky gkz hypergeometric system some of these solutions arise as period integrals for calabi yau manifolds in mirror symmetry we prove that for a suitable compactification of the parameter space there exists certain special boundary points which we called maximal degeneracy points at which all but one solutions become singular 3 2 96 email hosono sci toyama u ac jp email lian max math brandeis edu email yau math harvard edu

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

ensuring convergence of the mmse iteration for interference avoidance to the global optimum viswanath and anantharam 1999 characterize the sum capacity of multiaccess vector channels for a given number of users received powers spreading gain and noise covariance matrix in a code division multiple access cdma system viswanath and anantharam present a combinatorial algorithm to generate a set of signature sequences that achieves the maximum sum capacity these sets also minimize a performance measure called generalized total square correlation tsc sub g ulukus and yates 2001 propose an iterative algorithm suitable for distributed implementation at each step one signature sequence is replaced by its linear minimum mean square error mmse filter this algorithm results in a decrease of tsc sub g at each step the mmse iteration has fixed points not only at the optimal configurations which attain the global minimum tsc sub g but also at other configurations which are suboptimal the authors of claim that simulations show that when starting with random sequences the algorithm converges to optimum sets of sequences but they give no formal proof we show that the tsc sub g function has no local minima in the sense that given any suboptimal set of sequences there exist arbitrarily close sets with lower tsc sub g therefore only the optimal sets are stable fixed points of the mmse iteration we define a noisy version of the mmse iteration as follows after replacing all the signature sequences one at a time by their linear mmse filter we add a bounded random noise to all the sequences using our observation about the tsc sub g function we can prove that if we choose the bound on the noise adequately making it decrease to zero the noisy mmse iteration converges to the set of optimal configurations with probability one for any initial set of sequences

singularities of the hypergeometric system associated with a monomial curve we compute using d module restrictions the slopes of the irregular hypergeometric system associated with a monomial curve we also study rational solutions and reducibility of such systems

minimum total squared correlation quaternary signature sets new bounds and optimal designs we derive new bounds on the total squared correlation tsc of quaternary quadriphase signature sequence sets for all lengths l and set sizes k then for all k l we design minimum tsc optimal sets that meet the new bounds with equality direct numerical comparison with the tsc value of the recently obtained optimal binary sets shows under what k l realizations gains are materialized by moving from the binary to the quaternary code division multiplexing alphabet on the other hand comparison with the welch tsc value for real complex field sets shows that arguably not much is to be gained by raising the alphabet size above four for any k l the sum capacity as well as the maximum squared correlation and total asymptotic efficiency of minimum tsc quaternary sets is also evaluated in closed form and contrasted against the sum capacity of minimum tsc optimal binary and real complex sets

redundancy of the lempel ziv incremental parsing rule the lempel ziv codes are universal variable to fixed length codes that have become virtually standard in practical lossless data compression for any given source output string from a markov or unifilar source we upper bound the difference between the number of binary digits needed to encode the string and the self information of the string we use this result to demonstrate that for unifilar or markov sources the redundancy of encoding the first n letters of the source output with the lempel ziv incremental parsing rule lz 78 the welch modification lzw or a new variant is o ln n sup 1 and we upper bound the exact form of convergence we conclude by considering the relationship between the code length and the empirical entropy associated with a string

finite step algorithms for constructing optimal cdma signature sequences a description of optimal sequences for direct spread code division multiple access ds cdma is a byproduct of recent characterizations of the sum capacity this paper restates the sequence design problem as an inverse singular value problem and shows that the problem can be solved with finite step algorithms from matrix theory it proposes a new one sided algorithm that is numerically stable and faster than previous methods

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

simulation of random processes and rate distortion theory we study the randomness necessary for the simulation of a random process with given distributions on terms of the finite precision resolvability of the process finite precision resolvability is defined as the minimal random bit rate required by the simulator as a function of the accuracy with which the distributions are replicated the accuracy is quantified by means of various measures variational distance divergence orstein 1973 prohorov 1956 and related measures of distance between the distributions of random process in the case of ornstein prohorov and other distances of the kantorovich vasershtein type we show that the finite precision resolvability is equal to the rate distortion function with a fidelity criterion derived from the accuracy measure this connection leads to new results on nonstationary rate distortion theory in the case of variational distance the resolvability of stationary ergodic processes is shown to equal entropy rate regardless of the allowed accuracy in the case of normalized divergence explicit expressions for finite precision resolvability are obtained in many cases of interest and connections with data compression with minimum probability of block error are shown

a parametric texture model based on joint statistics of complex wavelet coefficients we present a universal statistical model for texture images in the context of an overcomplete complex wavelet transform the model is parameterized by a set of statistics computed on pairs of coefficients corresponding to basis functions at adjacent spatial locations orientations and scales we develop an efficient algorithm for synthesizing random images subject to these constraints by iteratively projecting onto the set of images satisfying each constraint and we use this to test the perceptual validity of the model in particular we demonstrate the necessity of subgroups of the parameter set by showing examples of texture synthesis that fail when those parameters are removed from the set we also demonstrate the power of our model by successfully synthesizing examples drawn from a diverse collection of artificial and natural textures

rms bandwidth constrained signature waveforms that maximize the total capacity of pam synchronous cdma channels optimum time limited signal sets of equal and unequal energies are obtained under root mean square rms bandwidth constraints the total capacity and the total asymptotic efficiency of the pam synchronous gaussian cdma psg cdma channel are considered as the optimality criteria the latter measure is monotonic with the determinant of the correlation matrix r and the former is monotonic with det i spl sigma sup 2 r where spl sigma sup 2 represents the noise level average as well as maximum rms bandwidth constraints are considered in the equal energy case and the energy weighted rms bandwidth constraint is considered for unequal energy signals for the equal energy problem signal sets are found that simultaneously optimize the total asymptotic efficiency under both average and maximum rms bandwidth constraints for the total capacity measure such simultaneously optimal signal sets are also obtained albeit under the restriction that the number of signals n be a hadamard matrix dimension when the hadamard dimension is in particular a power of two we obtain optimum signal sets that are shown to yield equal optimum multiuser detector asymptotic efficiencies for all users of an uncoded psg cdma channel unequal energy signal sets are also found under an energy weighted rms bandwidth constraint for both optimality criteria

a sequential algorithm for the universal coding of finite memory sources the estimation and universal compression of discrete sources are considered and a sequential algorithm for the universal coding of finite memory sources attaining asymptotically minimum redundancy is presented the algorithm performs an online estimation of the source states and uses an arithmetic code

universal prediction of individual sequences the problem of predicting the next outcome of an individual binary sequence using finite memory is considered the finite state predictability of an infinite sequence is defined as the minimum fraction of prediction errors that can be made by any finite state fs predictor it is proven that this fs predictability can be achieved by universal sequential prediction schemes an efficient prediction procedure based on the incremental parsing procedure of the lempel ziv data compression algorithm is shown to achieve asymptotically the fs predictability some relations between compressibility and predictability are discussed and the predictability is proposed as an additional measure of the complexity of a sequence

gevrey solutions of irregular hypergeometric systems in two variables we describe the gevrey series solutions at singular points of the irregular hypergeometric system gkz system associated with an affine plane monomial curve we also describe the irregularity complex of such a system with respect to its singular support and in particular we prove using elementary methods that this irregularity complex is a perverse sheaf as assured by a theorem of z mebkhout

the a hypergeometric system associated with a monomial curve introduction inthis paper we make a detailed analysis of the hypergeometric system or gkz system associated with a monomial curve and integral hence resonant exponents we describe all rational solutions and show in theorem 1 10 that they are in fact laurent polynomials we also show that for any exponent there are at most two linearly independent laurent solutions and that the upper bound is reached if and only if the curve is not arithmetically cohen macaulay we then construct for all integral parameters a basis of local solutions in terms of the roots of the generic univariate polynomial 0 5 associated with we also determine in theorem 3 7 the holonomic rank r for all z 2 and show that d r d 1 where d is the degree of the curve moreover the value d 1 is attained only for those exponents for which there are two linearly independent rational solutions and therefore r d for all if and only if the curve is arithmetically cohen macaulay inorder to place these results intheir appropriate con text we recall the defin ition

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

on the average redundancy rate of the lempel ziv code in this paper we settle a long standing open problem concerning the average redundancy r sub n of the lempel ziv 78 lz78 code we prove that for a memoryless source the average redundancy rate attains asymptotically er sub n a spl delta n log n o log log n log sup 2 n where a is an explicitly given constant that depends on source characteristics and spl delta x is a fluctuating function with a small amplitude we also derive the leading term for the kth moment of the number of phrases we conclude by conjecturing a precise formula on the expected redundancy for a markovian source the main result of this paper is a consequence of the second order properties of the lempel ziv algorithm obtained by jacquet and szpankowski 1995 these findings have been established by analytical techniques of the precise analysis of algorithms we give a brief survey of these results since they are interesting in their own right and shed some light on the probabilistic behavior of pattern matching based data compression

markov types and minimax redundancy for markov sources redundancy of universal codes for a class of sources determines by how much the actual code length exceeds the optimal code length in the minimax scenario one designs the best code for the worst source within the class such minimax redundancy comes in two flavors average minimax or worst case minimax we study the worst case minimax redundancy of universal block codes for markovian sources of any order we prove that the maximal minimax redundancy for markov sources of order r is asymptotically equal to 1 2m sup r m 1 log sub 2 n log sub 2 a sub m sup r lnlnm sup 1 m 1 lnm o 1 where n is the length of a source sequence m is the size of the alphabet and a sub m sup r is an explicit constant e g we find that for a binary alphabet m 2 and markov of order r 1 the constant a sub 2 sup 1 16 spl middot g spl ap 14 655449504 where g is the catalan number unlike previous attempts we view the redundancy problem as an asymptotic evaluation of certain sums over a set of matrices representing markov types the enumeration of markov types is accomplished by reducing it to counting eulerian paths in a multigraph in particular we propose exact and asymptotic formulas for the number of strings of a given markov type all of these findings are obtained by analytic and combinatorial tools of analysis of algorithms

twice universal simulation of markov sources and individual sequences the problem of universal simulation given a training sequence is studied both in a stochastic setting and for individual sequences in the stochastic setting the training sequence is assumed to be emitted by a markov source of unknown order extending previous work where the order is assumed known and leading to the notion of twice universal simulation a simulation scheme which partitions the set of sequences of a given length into classes is proposed for this setting and shown to be asymptotically optimal this partition extends the notion of type classes to the twice universal setting in the individual sequence scenario the same simulation scheme is shown to generate sequences which are statistically similar in a strong sense to the training sequence for statistics of any order while essentially maximizing the uncertainty on the output

gkz generalized hypergeometric systems in mirror symmetry of calabi yau hypersurfaces we present a detailed study of the generalized hypergeometric system introduced by gel fand kapranov and zelevinski gkz hypergeometric system in the context of toric geometry gkz systems arise naturally in the moduli theory of calabi yau toric varieties and play an important role in applications of the mirror symmetry we find that the grobner basis for the so called toric ideal determines a finite set of differential operators for the local solutions of the gkz system at the special point called the large radius limit we find a close relationship between the principal parts of the operators in the gkz system and the intersection ring of a toric variety as applications we analyze general three dimensional hypersurfaces of fermat and non fermat types with hodge numbers up toh 1 1 3 we also find and analyze several non landau ginzburg models which are related to singular models

type classes of context trees it is well known that a tree model does not always admit a finite state machine fsm representation with the same minimal number of parameters therefore known characterizations of type classes for fsms do not apply in general to tree models in this paper the type class of a sequence with respect to a given context tree is studied an exact formula is derived for the size of the class extending whittle s formula for type classes with respect to fsms the derivation is more intricate than in the fsm case since some basic properties of fsm types do not hold in general for tree types the derivation also yields an efficient enumeration of the tree type class a formula for the number of type classes with respect to is also derived the formula is asymptotically tight up to a multiplicative constant and also extends the corresponding result for fsms the asymptotic behavior of the number of type classes and of the size of a class is expressed in terms of the so called minimal canonical extension of t a tree that is generally larger than but smaller than its fsm closure

performance of lz algorithms on individual sequences the performance of three versions of the lempel ziv 1976 algorithm on individual sequences is investigated it is shown that as the restart length goes to infinity each compresses an individual sequence as well as any block to variable finite state information lossless algorithm and that the same conclusion holds for sliding window lz as the window width goes to infinity examples are given showing that an infinite memory version outperforms such finite memory forms and that such finite memory forms can compress more than the ziv 1978 entropy which is the best compression attainable by finite state block to block codes that have vanishing probability of error

universal modeling and coding the problems arising in the modeling and coding of strings for compression purposes are discussed the notion of an information source that simplifies and sharpens the traditional one is axiomatized and adaptive and nonadaptive models are defined with a measure of complexity assigned to the models a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all a general class of so called first in first out fifo arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models because the coding parameters are the probabilities that define the model their design is easy and the application of the code is straightforward even with adaptively changing source models

sum capacity and tsc bounds in collaborative multibase wireless systems we consider a wireless system with base stations which collaborate and derive bounds on sum capacity and total squared correlation for uniform channels between users and bases the correspondence also investigates structural properties which must be satisfied by user transmit covariance matrices at the optimal sum capacity total squared correlation tsc point and shows that for multibase systems maximizing sum capacity and minimizing tsc are in general not equivalent problems

comments on symmetric capacity and signal design for l out of k symbol synchronous cdma gaussian channels for original paper see a a alsugair and r s cheng ibid vol 41 p 1072 1082 july 1995 we show that the signature sequences set that maximizes the lower bound on symmetric capacity derived in the above paper also maximizes the sum capacity of the gaussian synchronous code division multiple access cdma channel this is done by relying on the equivalence of the eigenvalues of energy weighted correlation matrices this result establishes the missing link between the above paper and the sum capacity maximization result by viswanath et al 1999

perfectly secure encryption of individual sequences in analogy to the well known notion of finite state compressibility of individual sequences due to lempel and ziv we define a similar notion of finite state encryptability of an individual plain text sequence as the minimum asymptotic key rate that must be consumed by finite state encrypters so as to guarantee perfect secrecy in a well defined sense our main basic result is that the finite state encryptability is equal to the finite state compressibility for every individual sequence this is in parallelism to shannon s classical probabilistic counterpart result asserting that the minimum required key rate is equal to the entropy rate of the source however the redundancy defined as the gap between the upper bound direct part and the lower bound converse part in the encryption problem turns out to decay at a different rate in fact much slower than the analogous redundancy associated with the compression problem we also extend our main theorem in several directions allowing 1 availability of side information si at the encrypter decrypter eavesdropper 2 lossy reconstruction at the decrypter and 3 the combination of both lossy reconstruction and si in the spirit of the wyner ziv problem

duality and monodromy reducibility of a hypergeometric systems we study hypergeometric systems ha in the sense of gelfand kapranov and zelevinsky under two aspects the structure of their holonomically dual system and reducibility of their rank module we prove in the first part that rank jumping parameters always correspond to reducible systems we show further that the property of being reducible is invariant modulo the lattice and obtain as a special instance a theorem of alicia dickenstein and timur sadykov on reducibility of mellin systems in the second part we study a conjecture of nobuki takayama which states that the holonomic dual of ha is of the form ha for suitable we prove the conjecture for all matrices a and generic parameter exhibit an example that shows that in general the conjecture cannot hold and present a refined version of the conjecture questions on both duality and reducibility have been quite difficult to answer with classical methods this paper may be seen as an example of the usefulness and scope of applications of the homological tools for a hypergeometric systems developed in matusevich et al j amer math soc 18 919 941 2005

linear relations among algebraic solutions of differential equations abstract for any univariate polynomial with coefficients in a differential field of characteristic zero and any integer q there exists an associated nonzero linear ordinary differential equation lode with the following two properties each term of the lode lies in the differential field generated by the rational numbers and the coefficients of the polynomial and the q th power of each root of the polynomial is a solution of this lode this lode is called a q th power resolvent of the polynomial we will show how one can get a resolvent for the logarithmic derivative of the roots of a polynomial from the th power resolvent of the polynomial where is an indeterminate that takes the place of q we will demonstrate some simple relations among the algebraic and differential equations for the roots and their logarithmic derivatives we will also prove several theorems regarding linear relations of roots of a polynomial over constants or the coefficient field of the polynomial depending upon the nondifferential galois group finally we will use a differential resolvent to solve the riccati equation

the error exponent for the noiseless encoding of finite ergodic markov sources a new approach to the classical fixed length noiseless source coding problem is proposed for the case of finite ergodic markov sources this approach is based on simple counting arguments the central notion of markov type a set containing all the source sequences having the same transition counts from letter to letter is introduced and the cardinality of such a set is evaluated via graph theoretical tools the error exponent is shown to be a weighted average of informational divergences and the universal character of the result is stressed as a corollary the classical source coding theorem determining the achievable rates is rederived

universal coding with minimum probability of codeword length overflow lossless block to variable length source coding is studied for finite state finite alphabet sources the aim is to minimize the probability that the normalized length of the codeword will exceed a given threshold b subject to the kraft inequality it is shown that the lempel ziv algorithm 1978 asymptotically attains the optimal performance in the sense just defined independently of the source and the value of b for the subclass of unifilar markov sources faster convergence to the asymptotic optimum performance can be accomplished by using the minimum description length universal code for this subclass it is demonstrated that these universal codes are also nearly optimal in the sense of minimizing buffer overflow probability and asymptotically optimal in a competitive sense

compression of two dimensional data distortion free compressibility of individual pictures i e two dimensional arrays of data by finite state encoders is investigated for every individual infinite picture i a quantity rho i is defined called the compressibility of i which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for i by any finite state information lossless encoder this is demonstrated by means of a constructive coding theorem and its converse that apart from their asymptotic significance might also provide useful criteria for finite and practical data compression tasks the proposed picture compressibility is also shown to possess the properties that one would expect and require of a suitably defined concept of two dimensional entropy for arbitrary probabilistic ensembles of infinite pictures while the definition of rho i allows the use of different machines for different pictures the constructive coding theorem leads to a universal compression scheme that is asymptotically optimal for every picture the results are readily extendable to data arrays of any finite dimension

on the ergodic capacity as a function of the correlation properties in systems with multiple transmit antennas without csi at the transmitter in this letter the impact of correlation of the transmit antennas of a multiple input single output miso system with no channel state information csi at the transmitter and perfect csi at the receiver is analyzed we show that the ergodic capacity for the single user miso system is schur concave with respect to the vector with eigenvalues of the channel covariance matrix i e the more correlation that exists between the transmit antennas the less is the achievable capacity furthermore the capacity loss for fully correlated transmit antennas in comparison with the uncorrelated case is derived the results for the ergodic capacity are compared with the impact of correlation on the outage probability the relationship between correlation properties and outage probability is more complicated than the relationship between the correlation properties and the ergodic capacity it is shown that the outage probability is schur convex in the high signal to noise ratio snr regime and schur concave in the low snr regime

asynchronous cdma systems with random spreading part i fundamental limits spectral efficiency for asynchronous code division multiple access cdma with random spreading is calculated in the large system limit allowing for arbitrary chip waveforms and frequency flat fading signal to interference and noise ratios sinrs for suboptimal receivers such as the linear minimum mean square error mmse detectors are derived the approach is general and optionally allows even for statistics obtained by undersampling the received signal all performance measures are given as a function of the chip waveform and the delay distribution of the users in the large system limit it turns out that synchronizing users on a chip level impairs performance for all chip waveforms with bandwidth greater than the nyquist bandwidth e g positive roll off factors for example with the pulse shaping demanded in the umts standard user synchronization reduces spectral efficiency up to 12 at 10 db normalized signal to noise ratio the benefits of asynchronism stem from the finding that the excess bandwidth of chip waveforms actually spans additional dimensions in signal space if and only if the users are desynchronized at chip level the analysis of linear mmse detectors shows that the limiting interference effects can be decoupled both in the user domain and in the frequency domain such that the concept of effective interference spectral density arises this generalizes and refines tse and hanly s concept of effective interference in part ii the analysis is extended to any linear detector that admits a representation as multistage detector and guidelines for the design of low complexity multistage detectors with universal weights are provided

types of markov fields and tilings the method of types is one of the most popular techniques in information theory and combinatorics however thus far the method has been mostly applied to 1 d markov processes and it has not been thoroughly studied for general markov fields markov fields over a finite alphabet of size m ge 2 can be viewed as models for multidimensional systems with local interactions the locality of these interactions is represented by a shape mathcal s while its marking by symbols of the underlying alphabet is called a tile two assignments in a markov field have the same type if they have the same empirical distribution i e if they have the same number of tiles of a given type our goal is to study the growth of the number of possible markov field types in either a d dimensional box of lengths n 1 ldots n d or its cyclic counterpart a d dimensional torus we relate this question to the enumeration of nonnegative integer solutions of a large system of diophantine linear equations called the conservation laws we view a markov type as a vector in a d m mathcal s dimensional space and count the number of such vectors satisfying the conservation laws which turns out to be the number of integer points in a certain polytope for the torus this polytope is of dimension mu d 1 mathrm rk mathrm mathrm c where mathrm rk mathrm mathrm c is the number of linearly independent conservation laws mathrm mathrm c this provides an upper bound on the number of types then we construct a matching lower bound leading to the conclusion that the number of types in the torus markov field is theta n mu where n n 1 ldots n d these results are derived by geometric tools including ideas of discrete and convex multidimensional geometry

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

universal simulation with fidelity criteria we consider the problem of universal simulation of a memoryless source with some partial extensions to markov sources based on a training sequence emitted from the source the objective is to maximize the conditional entropy of the simulated sequence given the training sequence subject to a certain distance constraint between the probability distribution of the output sequence and the probability distribution of the input training sequence we derive for several distance criteria single letter expressions for the maximum attainable conditional entropy as well as corresponding universal simulation schemes that asymptotically attain these maxima

a universal data compression system a universal data compression algorithm is described which is capable of compressing long strings generated by a finitely generated source with a near optimum per symbol length without prior knowledge of the source this class of sources may be viewed as a generalization of markov sources to random fields moreover the algorithm does not require a working storage much larger than that needed to describe the source generating parameters

multiuser constrained water pouring for continuous time overloaded gaussian multiple access channels in this paper signature waveforms and signal powers are jointly optimized for multiuser communications over strictly bandlimited continuous time overloaded channels corrupted by additive white gaussian noise awgn the total signal power is minimized subject to general asymmetric signal to interference plus noise ratio sinr constraints at the output of linear minimum mean squared error lmmse receivers using vectorized fourier transform vft technique the optimal solutions as well as a necessary and sufficient condition for the existence of a feasible solution are derived in the frequency domain it turns out that every optimal solution performs the same as the optimal frequency division multiple access fdma system a geometric procedure called multiuser constrained water pouring is developed to construct every possible optimal solution of which profiles of signal power and equivalent bandwidth are respectively the same as those of signal power and physical bandwidth of the optimal fdma system it is shown that orthogonal signature waveforms are assigned to oversized users and continuous time equivalents of generalized welch bound equality cte gwbe sequences are assigned to non oversized users

on quasi orthogonal signatures for cdma systems sum capacity optimal signatures in synchronous code division multiple access cdma systems are functions of the codebook length as well as the number of active users a new signature set must be assigned every time the number of active users changes this correspondence considers signature sets that are less sensitive to changes in the number of active users equiangular signature sequences are proven to solve a certain max min signal to interference plus noise problem which results from their interference invariance unions of orthonormal bases have subsets that come close to satisfying the welch bound bounds on the maximum number of bases with minimum maximum correlation are derived and a new construction algorithm is provided connections are made between these signature design problems grassmannian line packing frame theory and algebraic geometry

power minimization for cdma under colored noise rate constrained power minimization pmin over a code division multiple access cdma channel with correlated noise is studied pmin is shown to be an instance of a separable convex optimization problem subject to linear ascending constraints pmin is further reduced to a dual problem of sumrate maximization rmax the results highlight the underlying unity between pmin rmax and a problem closely related to pmin but with linear receiver constraints subsequently conceptually simple sequence design algorithms are proposed to explicitly identify an assignment of sequences and powers that solve pmin the algorithms yield an upper bound of 2n 1 on the number of distinct sequences where n is the processing gain the sequences generated using the proposed algorithms are in general real valued if a rate splitting and multi dimensional cdma approach is allowed the upper bound reduces to n distinct sequences in which case the sequences can form an orthogonal set and be binary plusmn1 valued

multirate zero forcing tx rx design for mimo channel under ber constraints this paper is concerned with communicating parallel independent streams of data possibly of different rates through a multiple input multiple output channel each data stream is required to satisfy a bit error rate ber constraint this scenario arises in multimedia communications we find the linear zero forcing transmitter receiver tx rx matrix pair that requires the minimum transmitter power to achieve the ber constraints it is well known that the tx rx pair can be chosen to partition the matrix channel into scalar subchannels corresponding to singular values and different data streams can be transmitted through different subchannels this design is suboptimum in general the optimum design has the following structure the set of input data streams is partitioned into groups called signal groups the set of subchannels is conformably partitioned into subchannel groups each signal group is collectively transmitted through its assigned subchannel group with each signal subchannel group pair is associated a unitary matrix that mixes the signals at the transmitter and unmixes them at the receiver we derive a simple algorithm to determine the optimum signal and subchannel groups we then use a known algorithm from majorization theory to determine the unitary matrices using these we completely specify the optimum tx rx pair

hypergeometric functions for projective toric curves abstract we produce a decomposition of the parameter space of the a hypergeometric system associated to a projective monomial curve as a union of an arrangement of lines and its complement in such a way that the analytic behavior of the solutions of the system is explicitly controlled within each term of the union

steepest descent optimization of cdma signatures for multiple correlated sources with applications to joint source channel coding an approach to transmitting multiple correlated sources over a gaussian multiple access channel gmac using code division multiple access cdma is investigated a block coordinate descent bcd algorithm is presented which can be used to optimize cdma signatures for a given source covariance matrix under individual transmitter power constraints this algorithm can be used to design non orthogonal cdma signatures for gmacs with arbitrary input alphabets fading and colored noise while the main focus has been the design of signatures which minimize the mean square error mse of linear decoding the algorithm can also be used to design signatures which maximize the input output mutual information of the channel this paper investigates the uncoded transmission of gaussian sources and the coded transmission of binary sources over a gmac using joint source channel jsc optimized cdma signatures under constraints on transmitter power and channel bandwidth scenarios relevant to wireless sensor networks by using simple coding schemes which only rely on linear multi user detection and independent channel decoding of a large number of sources it is demonstrated that cdma based jsc coding can exploit inter source correlation to achieve lower distortion gaussian sources or higher rates binary sources compared to conventional separate source channel ssc coding over the same channel bandwidth

high snr performance analysis of blind minimum output energy receivers for large ds cdma systems high signal to noise ratio snr properties of the minimum output energy moe receiver and the capon channel estimation technique are analyzed in the case of large direct sequence code division multiple access ds cdma systems it is shown that if the noise power tends to zero and the system load is less than one then the signal to interference plus noise ratio sinr of the moe receiver tends to infinity and the capon channel estimate converges to a scaled version of the channel vector of the user of interest for the case when the system load is larger than one an asymptotic sinr expression is obtained furthermore expressions for the efficiency and asymptotic efficiency of the moe receiver are derived simple approximations of and upper lower bounds on these expressions are found and the effects of the system physical parameters on the performance of the moe receiver are discussed