parameter selection for wyner ziv coding of laplacian sources with additive laplacian or gaussian innovation a large number of practical coding scenarios deal with sources such as transform coefficients that can be well modeled as laplacians for regular coding of such sources samples are often quantized by a family of uniform quantizers possibly with a deadzone and then entropy coded for the wyner ziv coding problem when correlated side information is available at the decoder the side information can be modeled as obtained by independent additive laplacian or gaussian innovation on the source this paper deals with optimal choice of parameters for practical wyner ziv coding in such scenarios using the same quantizer family as in the regular codec to cover a range of rate distortion tradeoffs given the variances of the source and additive noise we propose and analyze a general encoding model based on multilevel coset codes that combines source coding and channel coding and show that at practical block lengths and code complexities not pure channel coding but a hybrid combination of source coding and channel coding with right parameters provide optimal rate distortion performance we also provide a framework for on the fly parameter choice based on nonparametric representation of a set of seed functions for use in scenarios where variances are estimated during encoding a good insight in the optimal parameter selection mechanism is essential for building practical distributed codecs

infinite dimensional primitive linearly compact lie superalgebras we classify open maximal subalgebras of all infinite dimensional linearly compact simple lie superalgebras this is applied to the classification of infinite dimensional lie superalgebras of vector fields acting transitively and primitively in a formal neighborhood of a point of a finite dimensional supermanifold

modeling schizophrenic behavior using general mixture components this article proposes a novel and general mixture component model the features of which include a hierarchical structure with random effects mixture components characterized by anova like linear regressions and mixing mechanisms governed by logistic regressions the model was developed as a consequence of attending to long standing psychological theory about schizophrenic behavior scientifically revealing results are obtained by fitting the model to a data set concerning nonschizophrenic and schizophrenic eye tracking behavior under different conditions included are descriptions of the algorithms for model fitting specifically the ecm secm algorithms for large sample modal inference and the gibbs sampler for simulating the posterior distribution for guidance on model comparison and selection we use posterior predictive check distributions to obtain posterior predictive p values for likelihood ratio statistics which do not have asymptotic chi 2 reference distributions these posterior predictive p values suggest that all the mixture components in our model are necessary the final model is selected using a combination of scientific parsimony the posterior predictive p values and the posterior distributions of relevant parameters

algebraic vs physical n 6 3 algebras in our previous paper we classified linearly compact algebraic simple n 6 3 algebras in the present paper we classify their physical counterparts which actually appear in the n 6 supersymmetric 3 dimensional chern simons theories

on improved bounds on the decoding error probability of block codes over interleaved fading channels with applications to turbo like codes we derive here improved upper bounds on the decoding error probability of block codes which are transmitted over fully interleaved rician fading channels coherently detected and maximum likelihood ml decoded we assume that the fading coefficients during each symbol are statistically independent due to a perfect channel interleaver and that perfect estimates of these fading coefficients are provided to the receiver the improved upper bounds on the block and bit error probabilities are derived for fully interleaved fading channels with various orders of space diversity and are found by generalizing some previously introduced upper bounds for the binary input additive white gaussian nose awgn channel the advantage of these bounds over the ubiquitous union bound is demonstrated for some ensembles of turbo codes and low density parity check ldpc codes and it is especially pronounced in a portion of the rate region exceeding the cutoff rate our generalization of the duman and salehi bound duman and salehi 1998 duman 1998 which is based on certain variations of gallager s 1965 bounding technique is demonstrated to be the tightest reported upper bound we therefore apply it to calculate numerically upper bounds on the thresholds of some ensembles of turbo like codes referring to the optimal ml decoding for certain ensembles of uniformly interleaved turbo codes the upper bounds derived here also indicate good match with computer simulation results of efficient iterative decoding algorithms

on the error exponents of improved tangential sphere bounds the performance of maximum likelihood ml decoded binary linear block codes over the additive white gaussian noise awgn channel is addressed via the tangential sphere bound tsb and two of its recent improved versions the correspondence is focused on the derivation of the error exponents of these bounds although it was shown that some recent improvements of the tsb tighten this bound for finite length codes it is demonstrated in this correspondence that their error exponents coincide for an arbitrary ensemble of binary linear block codes the common value of these error exponents is explicitly expressed in terms of the asymptotic growth rate of the average distance spectrum

performance bounds for erasure list and decision feedback schemes with linear block codes a message independence property and some new performance upper bounds are derived in this work for erasure list and decision feedback schemes with linear block codes transmitted over memoryless symmetric channels similar to the classical work of forney this work is focused on the derivation of some gallager type bounds on the achievable tradeoffs for these coding schemes where the main novelty is the suitability of the bounds for both random and structured linear block codes or code ensembles the bounds are applicable to finite length codes and to the asymptotic case of infinite block length and they are applied to low density parity check code ensembles

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

efficient erasure correcting codes we introduce a simple erasure recovery algorithm for codes derived from cascades of sparse bipartite graphs and analyze the algorithm by analyzing a corresponding discrete time random process as a result we obtain a simple criterion involving the fractions of nodes of different degrees on both sides of the graph which is necessary and sufficient for the decoding process to finish successfully with high probability by carefully designing these graphs we can construct for any given rate r and any given real number spl epsiv a family of linear codes of rate r which can be encoded in time proportional to ln 1 spl epsiv times their block length n furthermore a codeword can be recovered with high probability from a portion of its entries of length 1 spl epsiv rn or more the recovery algorithm also runs in time proportional to n ln 1 spl epsiv our algorithms have been implemented and work well in practice various implementation issues are discussed

list decoding of polar codes we describe a successive cancellation list decoder for polar codes which is a generalization of the classic successive cancellation decoder of arikan in the proposed list decoder l decoding paths are considered concurrently at each decoding stage where l is an integer parameter at the end of the decoding process the most likely among the l paths is selected as the single codeword at the decoder output simulations show that the resulting performance is very close to that of maximum likelihood decoding even for moderate values of l alternatively if a genie is allowed to pick the transmitted codeword from the list the results are comparable with the performance of current state of the art ldpc codes we show that such a genie can be easily implemented using simple crc precoding the specific list decoding algorithm that achieves this performance doubles the number of decoding paths for each information bit and then uses a pruning procedure to discard all but the l most likely paths however straightforward implementation of this algorithm requires omega l n 2 time which is in stark contrast with the o n log n complexity of the original successive cancellation decoder in this paper we utilize the structure of polar codes along with certain algorithmic transformations in order to overcome this problem we devise an efficient numerically stable implementation of the proposed list decoder that takes only o l n log n time and o l n space

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

tightened upper bounds on the ml decoding error probability of binary linear block codes the performance of maximum likelihood ml decoded binary linear block codes is addressed via the derivation of tightened upper bounds on their decoding error probability the upper bounds on the block and bit error probabilities are valid for any memoryless binary input and output symmetric communication channel and their effectiveness is exemplified for various ensembles of turbo like codes over the additive white gaussian noise awgn channel an expurgation of the distance spectrum of binary linear block codes further tightens the resulting upper bounds

codes on finite geometries new algebraic methods for constructing codes based on hyperplanes of two different dimensions in finite geometries are presented the new construction methods result in a class of multistep majority logic decodable codes and three classes of low density parity check ldpc codes decoding methods for the class of majority logic decodable codes and a class of codes that perform well with iterative decoding in spite of having many cycles of length 4 in their tanner graphs are presented most of the codes constructed can be either put in cyclic or quasi cyclic form and hence their encoding can be implemented with linear shift registers

construction of low density parity check codes based on balanced incomplete block designs this correspondence presents a method for constructing structured regular low density parity check ldpc codes based on a special type of combinatoric designs known as balance incomplete block designs codes constructed by this method have girths at least 6 and they perform well with iterative decoding furthermore several classes of these codes are quasi cyclic and hence their encoding can be implemented with simple feedback shift registers

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

bounds on the decoding error probability of binary linear codes via their spectra bounds on the error probability of maximum likelihood decoding of a binary linear code are considered the bounds derived use the weight spectrum of the code and they are tighter than the conventional union bound in the case of large noise in the channel the bounds derived are applied to a code with an average spectrum and the result is compared to the random coding exponent the author shows that the bound considered for the binary symmetrical channel case coincides asymptotically with the random coding bound for the case of awgn channel the author shows that berlekamp s 1980 tangential bound can be improved but even this improved bound does not coincide with the random coding bound although it can be very close to it

on the frame error rate of concatenated turbo codes turbo codes with long frame lengths are usually constructed using a randomly chosen interleaver statistically this guarantees excellent bit error rate ber performance but also generates a certain number of low weight codewords resulting in the appearance of an error floor in the ber curve several methods including using an outer code have been proposed to improve the error floor region of the ber curve we study the effect of an outer bch code on the frame error rate fer of turbo codes we show that additional coding gain is possible not only in the error floor region but also in the waterfall region also the outer code improves the iterative app decoder by providing a stopping criterion and alleviating convergence problems with this method we obtain codes whose performance is within 0 6 db of the sphere packing bound at an fer of 10 sup 6

three algebras triple systems and 3 graded lie superalgebras the three algebras used by bagger and lambert in n 6 theories of abjm type are in one to one correspondence with a certain type of lie superalgebras we show that the description of three algebras as generalized jordan triple systems naturally leads to this correspondence furthermore we show that simple three algebras correspond to simple lie superalgebras and vice versa this gives a classification of simple three algebras from the well known classification of simple lie superalgebras

accumulate repeat accumulate codes capacity achieving ensembles of systematic codes for the erasure channel with bounded complexity this paper introduces ensembles of systematic accumulate repeat accumulate ara codes which asymptotically achieve capacity on the binary erasure channel bec with bounded complexity per information bit of encoding and decoding it also introduces symmetry properties which play a central role in the construction of new capacity achieving ensembles for the bec the results here improve on the tradeoff between performance and complexity provided by previous constructions of capacity achieving code ensembles defined on graphs the superiority of ara codes with moderate to large block length is exemplified by computer simulations which compare their performance with those of previously reported capacity achieving ensembles of low density parity check ldpc and irregular repeat accumulate ira codes ara codes also have the advantage of being systematic

parity check density versus performance of binary linear block codes over memoryless symmetric channels we derive lower bounds on the density of parity check matrices of binary linear codes which are used over memoryless binary input output symmetric mbios channels the bounds are expressed in terms of the gap between the rate of these codes for which reliable communications is achievable and the channel capacity they are valid for every sequence of binary linear block codes if there exists a decoding algorithm under which the average bit error probability vanishes for every mbios channel we construct a sequence of ensembles of regular low density parity check ldpc codes so that an upper bound on the asymptotic density of their parity check matrices scales similarly to the lower bound the tightness of the lower bound is demonstrated for the binary erasure channel by analyzing a sequence of ensembles of right regular ldpc codes which was introduced by shokrollahi and which is known to achieve the capacity of this channel under iterative message passing decoding we show that this sequence of ensembles is asymptotically optimal in a sense to be defined in this paper strengthening a result of shokrollahi finally we derive lower bounds on the bit error probability and on the gap to capacity for binary linear block codes which are represented by bipartite graphs and study their performance limitations over mbios channels the latter bounds provide a quantitative measure for the number of cycles of bipartite graphs which represent good error correction codes

convex clustering via l1 fusion penalization summary we study the large sample behaviour of a convex clustering framework which minimizes the sample within cluster sum of squares under an l1 fusion constraint on the cluster centroids this recently proposed approach has been gaining in popularity however its asymptotic properties have remained mostly unknown our analysis is based on a novel representation of the sample clustering procedure as a sequence of cluster splits determined by a sequence of maximization problems we use this representation to provide a simple and intuitive formulation for the population clustering procedure we then demonstrate that the sample procedure consistently estimates its population analogue and we derive the corresponding rates of convergence the proof conducts a careful simultaneous analysis of a collection of m estimation problems whose cardinality grows together with the sample size on the basis of the new perspectives gained from the asymptotic investigation we propose a key post processing modification of the original clustering framework we show both theoretically and empirically that the resulting approach can be successfully used to estimate the number of clusters in the population using simulated data we compare the proposed method with existing number of clusters and modality assessment approaches and obtain encouraging results we also demonstrate the applicability of our clustering method to the detection of cellular subpopulations in a single cell virology study

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

algebraic construction of quasi cyclic ldpc codes for the awgn and erasure channels this paper is concerned with construction of quasi cyclic qc low density parity check ldpc codes for three different types of channels the additive white gaussian noise the binary random erasure and the binary burst erasure channels two algebraic methods for systematic construction of qc ldpc codes are presented codes constructed perform well over all three types of channels

the error probability of generalized perfect codes via the meta converse we introduce a definition of perfect and quasi perfect codes for discrete symmetric channels based on the packing and covering properties of generalized spheres whose shape is tilted using an auxiliary probability measure this notion generalizes previous definitions of perfect and quasi perfect codes and encompasses maximum distance separable codes the error probability of these codes whenever they exist is shown to coincide with the estimate provided by the meta converse lower bound we illustrate how the proposed definition naturally extends to cover almost lossless source channel coding and lossy compression

superpotentials for superconformal chern simons theories from representation theory these notes provide a detailed account of the universal structure of superpotentials defining a large class of superconformal chern simons theories with matter many of which appear as the low energy descriptions of multiple m2 brane configurations the amount of superconformal symmetry in the chern simons matter theory determines the minimum amount of global symmetry that the associated quartic superpotential must realize which in turn restricts the matter superfield representations our analysis clarifies the necessary representation theoretic data which guarantees a particular amount of superconformal symmetry thereby we shall recover all the examples of m2 brane effective field theories that have appeared in the recent literature the results are based on a refinement of the unitary representation theory of lie algebras to the case when the lie algebra admits an ad invariant inner product the types of representation singled out by the superconformal symmetry turn out to be intimately associated with triple systems admitting embedding lie super algebras and we obtain a number of new results about these triple systems which might be of independent interest in particular we prove that any metric 3 lie algebra embeds into a real metric 3 graded lie superalgebra in such a way that the 3 bracket is given by a nested lie bracket

on universal properties of capacity approaching ldpc code ensembles this paper is focused on the derivation of some universal properties of capacity approaching low density parity check ldpc code ensembles whose transmission takes place over memoryless binary input output symmetric mbios channels properties of the degree distributions graphical complexity and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information theoretic bounds these bounds are expressed in terms of the target block bit error probability and the gap in rate to capacity most of the bounds are general for any decoding algorithm and some others are proved under belief propagation bp decoding proving these bounds under a certain decoding algorithm validates them automatically also under any suboptimal decoding algorithm a proper modification of these bounds makes them universal for the set of all mbios channels which exhibit a given capacity bounds on the degree distributions and graphical complexity apply to finite length ldpc codes and to the asymptotic case of an infinite block length the bounds are compared with capacity approaching ldpc code ensembles under bp decoding and they are shown to be informative and are easy to calculate finally some interesting open problems are considered

ldpc codes and convolutional codes with equal structural delay a comparison we compare convolutional codes and ldpc codes with respect to their decoding performance and their structural delay which is the inevitable delay solely depending on the structural properties of the coding scheme besides the decoding performance the data delay caused by the channel code is of great importance as this is a crucial factor for many applications convolutional codes are known to show a good performance while imposing only a very low latency on the data ldpc codes yield superior decoding performance but impose a larger delay due to the block structure the results obtained by comparison will also be related to theoretical limits obtained from random coding and the sphere packing bound it will be shown that convolutional codes are still the first choice for applications for which a very low data delay is required and the bit error rate is the considered performance criterion however if one focuses on a low signal to noise ratio or if the obtained frame error rate is the base for comparison ldpc codes compare favorably

finite length analysis of low density parity check codes on the binary erasure channel in this paper we are concerned with the finite length analysis of low density parity check ldpc codes when used over the binary erasure channel bec the main result is an expression for the exact average bit and block erasure probability for a given regular ensemble of ldpc codes when decoded iteratively we also give expressions for upper bounds on the average bit and block erasure probability for regular ldpc ensembles and the standard random ensemble under maximum likelihood ml decoding finally we present what we consider to be the most important open problems in this area

influence based diagnostics for finite mixture models summary in the fitting of finite mixtures of distributions to empirical data it is often felt necessary to exclude certain observations in order to achieve a satisfactory fit this is particularly true of fisheries lengthfrequency data where the object is to model a discrete number of age classes by a mixture of normal components and the presence of outliers in the sample can have a large effect on the parameter estimates and the fit this paper develops diagnostic tools to measure the effect of individual observations on the parameter estimates and the fit in order to make the data rejection decision of the data analyst less ad hoc specifically we show how to compute the parameter influence curves and introduce a statistic similar to cook s distance used in regression diagnostics

bayesian winner take all reconstruction of intermediate views from stereoscopic images this paper presents a new algorithm for the reconstruction of intermediate views from a pair of still stereoscopic images the algorithm is designed to address the issue of blur caused by linear filtering often employed in such reconstruction the proposed algorithm is block based and to reconstruct the intermediate views employs nonlinear disparity compensated filtering by means of a winner take all strategy the reconstructed image is modeled as a tiling by fixed size blocks coming from various positions disparity compensation of either the left or right images while the tiling map itself is modeled by a binary decision field in addition to that an observation model relating the left and right images via a disparity field and a disparity field model are used all models are probabilistic and are combined into a maximum a posteriori probability criterion the intermediate intensities disparities and the binary decision field are estimated jointly using the expectation maximization algorithm the new approach is compared experimentally on complex natural images with a reference block based algorithm employing linear filtering although the improvements are localized and often subtle they demonstrate that a high quality intermediate view reconstruction for complex scenes is feasible

refinement of the sphere packing bound asymmetric channels we provide a refinement of the sphere packing bound for constant composition codes over asymmetric discrete memoryless channels that improves the subexponential factor in front of the exponent the order of our subexponential factor is n 0 5 1 e r for any 0 where r is the left derivative of the sphere packing exponent at rate r and n is the blocklength

sphere packing bounds revisited for moderate block lengths the main reference of this paper is the sphere packing bound of 1967 sp67 derived by shannon gallager and berlekamp it offers a lower bound on the decoding error probability over a very large variety of channels if it has failed so far to provide any usable material in practical implementation of telecommunication systems it is due to an original focus on asymptotic results making it inapplicable for moderate code lengths and to the difficulty of the involved methods which makes the derivation of sp67 quite hermetic and uninspiring for further research the purpose of this paper is two fold 1 to stir up some renewed interest in the topic on which shannon concluded his career in information theory thanks to a qualitative rather than technical review of the derivation of sp67 introduced by a review of the simpler sphere packing bound derived by shannon in 1959 2 to prove the practical interest of sp67 by extending its field of application to continuous output channels and particularly the additive white gaussian noise awgn channel used with any particular modulation scheme and by improving its lower bound for the moderate code length case so that it becomes the best lower bound for most iteratively decodable codes turbo codes low density parity check ldpc codes repeat accumulate ra codes etc of usual lengths

the error probability of m ary psk block coded modulation schemes a tight upper bound on the decoding error probability is derived for block coded modulation structures where an m ary phase shift keying m psk signal constellation is employed this bound called a tangential sphere bound is tight for very low as well as for high signal to noise ratios snrs berlekamp s tangential union bound previously derived for binary codes can be derived for an m psk block coded modulation structure as well however it is proven that our tangential sphere bound is tighter than berlekamp s 1980 tangential bound for particular schemes it is shown that for low snrs our bound is considerably tighter than the tangential bound as one of the examples a multistage decoder is considered

coding in the finite blocklength regime bounds based on laplace integrals and their asymptotic approximations in this paper we provide new compact integral expressions and associated simple asymptotic approximations for converse and achievability bounds in the finite blocklength regime the chosen converse and random coding union bounds were taken from the recent work of polyanskyi poor verdu and are investigated under parallel awgn channels the awgn channel the bi awgn channel and the bsc the technique we use which is a generalization of some recent results available from the literature is to map the probabilities of interest into a laplace integral and then solve or approximate the integral by use of a steepest descent technique the proposed results are particularly useful for short packet lengths where the normal approximation may provide unreliable results

the analysis of repeated measures data on schizophrenic reaction times using mixture models reaction times for schizophrenic individuals in a simple visual tracking experiment can be substantially more variable than for non schizophrenic individuals current psychological theory suggests that at least some of this extra variability arises from an attentional lapse that delays some but not all of each schizophrenic s reaction times based on this theory we pursue models in which measurements from non schizophrenics arise from a normal linear model with a separate mean for each individual whereas measurements from schizophrenics arise from a mixture of i a component analogous to the distribution of response times for non schizophrenics and ii a mean shifted component we fit four mixture models within this framework where the distinctions between models arise from assumptions about the variance of the shifted observations and the exchangeability of schizophrenic individuals some of these models can be fit by maximum likelihood using the em algorithm and all can be fit using the ecm algorithm where the covariance matrices associated with the parameters are calculated by the sem and secm algorithms respectively bayesian model monitoring using posterior predictive checks is invoked to discard models that fail to reproduce certain observed features of the data and to stimulate the development of better models

performance bounds for nonbinary linear block codes over memoryless symmetric channels the performance of nonbinary linear block codes is studied in this paper via the derivation of new upper bounds on the block error probability under maximum likelihood ml decoding the transmission of these codes is assumed to take place over a memoryless and symmetric channel the new bounds which are based on the gallager bounds and their variations are applied to the gallager ensembles of nonbinary and regular low density parity check ldpc codes these upper bounds are also compared with sphere packing lower bounds this study indicates that the new upper bounds are useful for the performance evaluation of coded communication systems which incorporate nonbinary coding techniques

using em to obtain asymptotic variance covariance matrices the sem algorithm abstract the expectation maximization em algorithm is a popular and often remarkably simple method for maximum likelihood estimation in incomplete data problems one criticism of em in practice is that asymptotic variance covariance matrices for parameters e g standard errors are not automatic byproducts as they are when using some other methods such as newton raphson in this article we define and illustrate a procedure that obtains numerically stable asymptotic variance covariance matrices using only the code for computing the complete data variance covariance matrix the code for em itself and code for standard matrix operations the basic idea is to use the fact that the rate of convergence of em is governed by the fractions of missing information to find the increased variability due to missing information to add to the complete data variance covariance matrix we call this supplemented em algorithm the sem algorithm theory and particular examples reinforce the conclusion that the sem alg

estimation and hypothesis testing in finite mixture models summary finite mixture models are a useful class of models for application to data when sample sizes are not large and the number of underlying densities is in question likelihood ratio tests based on joint maximum likelihood estimation of the mixing parameter x and the parameter of the underlying densities 0 are problematical our approach places a prior distribution on x and estimates 0 by maximizing the likelihood of the data given 0 with x integrated out advantages of this approach computational issues using the em algorithm and directions for further work are discussed the technique is applied to two examples

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

maximum likelihood estimation for mixed continuous and categorical data with missing values summary maximum likelihood procedures for analysing mixed continuous and categorical data with missing values are presented the general location model of olkin tate 1961 and extensions introduced by krzanowski 1980 1982 form the basis for our methods maximum likelihood estimation with incomplete data is achieved by an application of the em algorithm dempster laird rubin 1977 special cases of the algorithm include orchard woodbury s 1972 algorithm for incomplete normal samples fuchs s 1982 algorithms for log linear modelling of partially classified contingency tables and day s 1969 algorithm for multivariate normal mixtures applications include a imputation of missing values b logistic regression and discriminant analysis with missing predictors and unclassified observations c linear regression with missing continuous and categorical predictors and d parametric cluster analysis with incomplete data methods are illustrated using data from the st louis risk research project some key word8 cluster analysis discriminant analysis em algorithm incomplete data linear regression logistic regression log linear model mixture model

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

an improved sphere packing bound for finite length codes over symmetric memoryless channels this paper derives an improved sphere packing isp bound for finite length error correcting codes whose transmission takes place over symmetric memoryless channels and the codes are decoded with an arbitrary list decoder we first review classical results i e the 1959 sphere packing sp59 bound of shannon for the gaussian channel and the 1967 sphere packing sp67 bound of shannon et al for discrete memoryless channels an improvement on the sp67 bound as suggested by valembois and fossorier is also discussed these concepts are used for the derivation of a new lower bound on the error probability of list decoding referred to as the isp bound which is uniformly tighter than the sp67 bound and its improved version the isp bound is applicable to symmetric memoryless channels and some of its applications are presented its tightness under maximum likelihood ml decoding is studied by comparing the isp bound to previously reported upper and lower bounds on the ml decoding error probability and also to computer simulations of iteratively decoded turbo like codes this paper also presents a technique which performs the entire calculation of the sp59 bound in the logarithmic domain thus facilitating the exact calculation of this bound for moderate to large block lengths without the need for the asymptotic approximations provided by shannon

several observations on symplectic hamiltonian and skew hamiltonian matrices abstract we prove a hamiltonian skew hamiltonian version of the classical theorem relating strict equivalence and t congruence between pencils of complex symmetric or skew symmetric matrices then we give a pure symplectic variant of the recent result of xu concerning the singular value decomposition of a conjugate symplectic matrix finally we discuss implications that can be derived from veselic s result on definite pairs of hermitian matrices for the skew hamiltonian situation

finite dimensional bounds on bbz _m and binary ldpc codes with belief propagation decoders this paper focuses on finite dimensional upper and lower bounds on decodable thresholds of zopfm and binary low density parity check ldpc codes assuming belief propagation decoding on memoryless channels a concrete framework is presented admitting systematic searches for new bounds two noise measures are considered the bhattacharyya noise parameter and the soft bit value for a maximum a posteriori probability map decoder on the uncoded channel for zopf m ldpc codes an iterative m dimensional bound is derived for m ary input symmetric output channels which gives a sufficient stability condition for zopfm ldpc codes and is complemented by a matched necessary stability condition introduced herein applications to coded modulation and to codes with nonequiprobably distributed codewords are also discussed for binary codes two new lower bounds are provided for symmetric channels including a two dimensional iterative bound and a one dimensional noniterative bound the latter of which is the best known bound that is tight for binary symmetric channels bscs and is a strict improvement over the existing bound derived by the channel degradation argument by adopting the reverse channel perspective upper and lower bounds on the decodable bhattacharyya noise parameter are derived for nonsymmetric channels which coincides with the existing bound for symmetric channels

bayesianly justifiable and relevant frequency calculations for the applied statistician a common reaction among applied statisticians is that the bayesian statistician s energies in an applied problem must be directed at the a priori elicitation of one model specification from which an optimal design and all inferences follow automatically by applying bayes s theorem to calculate conditional distributions of unknowns given knowns i feel however that the applied bayesian statistician s tool kit should be more extensive and include tools that may be usefully labeled frequency calculations three types of bayesianly justifiable and relevant frequency calculations are presented using examples to convey their use for the applied statistician

a hierarchical finite mixture model that accommodates zero inflated counts non independence and heterogeneity a number of mixture modeling approaches assume both normality and independent observations however these two assumptions are at odds with the reality of many data sets which are often characterized by an abundance of zero valued or highly skewed observations as well as observations from biologically related i e non independent subjects we present here a finite mixture model with a zero inflated poisson regression component that may be applied to both types of data this flexible approach allows the use of covariates to model both the poisson mean and rate of zero inflation and can incorporate random effects to accommodate non independent observations we demonstrate the utility of this approach by applying these models to a candidate endophenotype for schizophrenia but the same methods are applicable to other types of data characterized by zero inflation and non independence

capacity achieving ensembles for the binary erasure channel with bounded complexity we present two sequences of ensembles of nonsystematic irregular repeat accumulate ira codes which asymptotically as their block length tends to infinity achieve capacity on the binary erasure channel bec with bounded complexity per information bit this is in contrast to all previous constructions of capacity achieving sequences of ensembles whose complexity grows at least like the log of the inverse of the gap in rate to capacity the new bounded complexity result is achieved by puncturing bits and allowing in this way a sufficient number of state nodes in the tanner graph representing the codes we derive an information theoretic lower bound on the decoding complexity of randomly punctured codes on graphs the bound holds for every memoryless binary input output symmetric mbios channel and is refined for the binary erasure channel

maximin effects in inhomogeneous large scale data large scale data are often characterized by some degree of inhomogeneity as data are either recorded in different time regimes or taken from multiple sources we look at regression models and the effect of randomly changing coefficients where the change is either smoothly in time or some other dimension or even without any such structure fitting varying coefficient models or mixture models can be appropriate solutions but are computationally very demanding and often return more information than necessary if we just ask for a model estimator that shows good predictive properties for all regimes of the data then we are aiming for a simple linear model that is reliable for all possible subsets of the data we propose the concept of maximin effects and a suitable estimator and look at its prediction accuracy from a theoretical point of view in a mixture model with known or unknown group structure under certain circumstances the estimator can be computed orders of magnitudes faster than standard penalized regression estimators making computations on large scale data feasible empirical examples complement the novel methodology and theory

on the evaluation of the polyanskiy poor verdu converse bound for finite block length coding in awgn a tight converse bound to the channel coding rate in the finite block length regime and under additive white gaussian noise conditions was recently proposed by polyanskiy poor verdu ppv the bound is a generalization of a number of other classical results and it was also claimed to be equivalent to shannon s 1959 cone packing bound in this paper we investigate methods for a reliable numerical evaluation of the bound which is troublesome even for not too large values of the block length n by compactly expressing the polyanskiy poor and verdu ppv converse bound in terms of non central chi squared distributions and by evaluating those through an integral expression and a corresponding series expansion which exploit a method proposed by temme as a result a robust evaluation method and new insights on the bound s asymptotics as well as new approximate expressions are obtained