maximum likelihood estimation of a structured covariance matrix with a condition number constraint in this paper we deal with the problem of estimating the disturbance covariance matrix for radar signal processing applications when a limited number of training data is present we determine the maximum likelihood ml estimator of the covariance matrix starting from a set of secondary data assuming a special covariance structure i e the sum of a positive semi definite matrix plus a term proportional to the identity and a condition number upper bound constraint we show that the formulated constrained optimization problem falls within the class of maxdet problems and develop an efficient procedure for its solution in closed form remarkably the computational complexity of the algorithm is of the same order as the eigenvalue decomposition of the sample covariance matrix at the analysis stage we assess the performance of the proposed algorithm in terms of achievable signal to interference plus noise ratio sinr both for a spatial and a doppler processing the results show that interesting sinr improvements with respect to some existing covariance matrix estimation techniques can be achieved

on ordering color maps for lossless predictive coding linear predictive techniques perform poorly when used with color mapped images where pixel values represent indices that point to color values in a look up table reordering the color table however can lead to a lower entropy of prediction errors in this paper we investigate the problem of ordering the color table such that the absolute sum of prediction errors is minimized the problem turns out to be intractable even for the simple case of one dimensional 1 d prediction schemes we give two heuristic solutions for the problem and use them for ordering the color table prior to encoding the image by lossless predictive techniques we demonstrate that significant improvements in actual bit rates can be achieved over dictionary based coding schemes that are commonly employed for color mapped images

a note on the ziv lempel model for compressing individual sequences corresp the ziv lempel compression algorithm is a string matching and parsing approach to data compression the symbolwise equivalent for parsing models has been defined by rissanen and langdon and gives the same ideal codelength at the same cost in coding parameters by describing the context and coding parameter for each symbol an insight is provided into how the ziv lempel method achieves compression this treatment does not employ a probabilistic source for the data string the ziv lempel method effectively counts symbol instances within parsed phrases the coding parameter for each symbolwise context is determined by cumulative count ratios the code string length increase for a symbol y following substring s under the symbolwise equivalent is the log of the ratio of node counts in subtrees s and s cdot y of the ziv lempel parsing tree to demonstrate the symbolwise equivalent of the ziv lempel algorithm we extend the work of rissanen and langdon to incomplete parse trees the result requires the proper handling of the comma when one phrase is the prefix of another phrase

permutable indistinguishability operators perfect vague groups and fuzzy subgroups permutability between t indistinguishability operators is a very interesting property that is related to the compatibility of the operators with algebraic structures it will be shown that the sup t product email protected f of two t indistinguishability operators is also a t indistinguishability operator if and only if e and f are permutable t indistinguishability operators i e email protected email protected e this property will be related to the study of fuzzy subgroups fuzzy normal subgroups and vague groups the aggregation of fuzzy subgroups will also be analyzed

an efficient coding system for long source sequences the elias source coding scheme is modified to permit a source sequence of practically unlimited length to be coded as a single codeword using arithmetic of only limited precision the result is shown to be a nonblock arithmetic code of the first in first out fifo type source symbols are decoded in the same order as they were encoded codeword lengths which are near optimum for the specified statistical properties of the source can be achieved explicit encoding and decoding algorithms are provided which effectively implement the coding scheme applications to data compression and cryptography are suggested

entropy constrained halftoning using multipath tree coding we suggest an optimization based method for halftoning that involves looking ahead before a decision for each binary output pixel is made we first define a mixture distortion criterion that is a combination of a frequency weighted mean square error mse and a measure depending on the distances between minority pixels in the halftone a tree coding approach with the ml algorithm is used for minimizing the distortion criterion to generate a halftone while this approach generates halftones of high quality these halftones are not very amenable to lossless compression we introduce an entropy constraint into the cost function of the tree coding algorithm that optimally trades off between image quality and compression performance in the output halftones

asymptotically tight bounds on the depth of estimated context trees we study the maximum markov model order that can be estimated for an individual input sequence x of length n over a finite alphabet of size alpha for popular estimators of tree models where the estimated order is determined by the depth of a context tree estimate and in the special case of plain markov models where the tree is constrained to be perfect with all leaves at the same depth first we consider penalized maximum likelihood estimators where a context tree hat t is obtained by minimizing a cost of the form log hat p t x f n s t where hat p t x is the ml of x under a model with context tree t s t is the set of leaves of t and f n is an increasing penalization function of n the popular bic estimator is a special case with f n frac alpha 1 2 log n for plain markov models a simple argument yields a known upper bound overline k n o log n on the maximum order that can be estimated for x and we show that in fact this simple bound is not far from tight for general context trees we derive an asymptotic upper bound n 1 2 o 1 on the estimated depth and we exhibit explicit input sequences that asymptotically attain the bound up to a multiplicative constant factor we show that a similar upper bound applies also to mdl estimators based on the kt probability assignment and moreover the same example sequences asymptotically approach the upper bound also in this case

fast converging adaptive detection of doppler shifted range distributed targets a generalized likelihood ratio test glrt for the adaptive detection of a target or targets that are doppler shifted and distributed in range is derived the unknown parameters associated with the hypothesis test are the complex amplitudes in range of the desired target and the unknown covariance matrix of the additive interference which is assumed to be characterized as complex zero mean correlated gaussian random variables the target s or targets complex amplitudes are assumed to be distributed across the entire input data block sensor spl times range the unknown covariance matrix is constrained to have the reasonable form of the identity matrix the internal noise contribution plus an unknown positive semidefinite psdh matrix the external interference contribution it is shown via simulation for a variety of interference scenarios that the new detector has the characteristic of having a bounded constant false alarm rate cfar i e for our problem the probability of false alarm p sub f for a given detection threshold is bounded by the p sub f that results when no external interference is present it is also shown via simulation that the new detector converges relatively fast with respect to the number of sample vectors k necessary in order to achieve a given probability of detection p sub d

an mcmc approach to universal lossy compression of analog sources motivated by the markov chain monte carlo mcmc approach to the compression of discrete sources developed by jalali and weissman we propose a lossy compression algorithm for analog sources that relies on a finite reproduction alphabet which grows with the input length the algorithm achieves in an appropriate asymptotic sense the optimum shannon theoretic tradeoff between rate and distortion universally for stationary ergodic continuous amplitude sources we further propose an mcmc based algorithm that resorts to a reduced reproduction alphabet when such reduction does not prevent achieving the shannon limit the latter algorithm is advantageous due to its reduced complexity and improved rates of convergence when employed on sources with a finite and small optimum reproduction alphabet

redundancy of the context tree weighting method on renewal and markov renewal processes the context tree weighting method ctw is shown to be almost adaptive on the classes of renewal and markov renewal processes up to logarithmic factor ctw achieves the minimax pointwise redundancy described by i csiszaacuter and p shields in ieee trans inf theory vol 42 no 6 pp 2065 2072 nov 1996 this result not only complements previous results on the adaptivity of the context tree weighting method on the relatively small class of all finite context tree sources which encompasses the class of all finite order markov sources it shows that almost minimax redundancy can be achieved on massive classes of sources classes that cannot be smoothly parameterized by subsets of finite dimensional spaces moreover it shows that almost adaptive compression can be achieved in a computationally efficient way on those massive classes while previous adaptivity results for ctw could rely on the fact that any markov source is a finite context tree source this is no longer the case for renewal sources in order to prove almost adaptivity of ctw over renewal sources it is necessary to establish that ctw carefully balances estimation error and approximation error

a probability ratio approach to approximate binary arithmetic coding we describe an alternative mechanism for approximate binary arithmetic coding the quantity that is approximated is the ratio between the probabilities of the two symbols analysis is given to show that the inefficiency so introduced is less than 0 7 on average and in practice the compression loss is negligible

a new parametric glrt for multichannel adaptive signal detection a parametric generalized likelihood ratio test glrt for multichannel signal detection in spatially and temporally colored disturbance was recently introduced by modeling the disturbance as a multichannel autoregressive ar process the detector however involves a highly nonlinear maximum likelihood estimation procedure which was solved via a two dimensional iterative search method initialized by a suboptimal estimator in this paper we present a simplified glrt along with a new estimator for the problem both the estimator and the glrt are derived in closed form at considerably lower complexity with adequate training data the new glrt achieves a similar detection performance as the original one however for the more interesting case of limited training the original glrt may become inferior due to poor initialization because of its simpler form the new glrt also offers additional insight into the parametric multichannel signal detection problem the performance of the proposed detector is assessed using both a simulated dataset which was generated using multichannel ar models and the kassper dataset a widely used dataset with challenging heterogeneous effects found in real world environments

entropy constrained vector quantization an iterative descent algorithm based on a lagrangian formulation for designing vector quantizers having minimum distortion subject to an entropy constraint is discussed these entropy constrained vector quantizers ecvqs can be used in tandem with variable rate noiseless coding systems to provide locally optimal variable rate block source coding with respect to a fidelity criterion experiments on sampled speech and on synthetic sources with memory indicate that for waveform coding at low rates about 1 bit sample under the squared error distortion measure about 1 6 db improvement in the signal to noise ratio can be expected over the best scalar and lattice quantizers when block entropy coded with block length 4 even greater gains are made over other forms of entropy coded vector quantizers for pattern recognition it is shown that the ecvq algorithm is a generalization of the k means and related algorithms for estimating cluster means in that the ecvq algorithm estimates the prior cluster probabilities as well experiments on multivariate gaussian distributions show that for clustering problems involving classes with widely different priors the ecvq outperforms the k means algorithm in both likelihood and probability of error

glrt based direction detectors in homogeneous noise and subspace interference in this paper we derive and assess decision schemes to discriminate resorting to an array of sensors between the h0 hypothesis that data under test contain disturbance only i e noise plus interference and the h1 hypothesis that they also contain signal components along a direction which is a priori unknown but constrained to belong to a given subspace of the observables the disturbance is modeled in terms of complex normal random vectors plus deterministic interference assumed to belong to a known subspace we assume that a set of noise only secondary data is available which possess the same statistical characterization of noise in the cells under test at the design stage we resort to either the plain generalized likelihood ratio test glrt or the two step glrt based design procedure the performance analysis conducted resorting to simulated data shows that the one step glrt performs better than the detector relying on the two step design procedure when the number of secondary data is comparable to the number of sensors moreover it outperforms a one step glrt based subspace detector when the dimension of the signal subspace is sufficiently high

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

i fuzzy equivalence relations and i fuzzy partitions a t fuzzy equivalence relation is a fuzzy binary relation on a set x which is reflexive symmetric and t transitive for a t norm t recently mesiar et al r mesiar b reusch h thiele fuzzy equivalence relations and fuzzy partitions j multi valued logic soft comput 12 2006 167 181 have generalised the t norm t to any general conjunctor c and investigated the minimal assumptions required on such operations called duality fitting conjunctors such that the fuzzification of the equivalence relation admits any value from the unit interval and also the one one correspondence between the fuzzy equivalence relations and fuzzy partitions is preserved in this work we conduct a similar study by employing a related form of c transitivity viz i transitivity where i is an implicator we show that although every i fuzzy equivalence relation can be shown to be a c fuzzy equivalence relation there exist c fuzzy equivalence relations that are not i fuzzy equivalence relations and hence these concepts are not equivalent most importantly we show that the class of duality fitting implicators i is much richer than the residuals of the duality fitting conjunctors in the study of mesiar et al we also show that the i fuzzy partitions have a constant wise structure

radar detection of distributed targets in homogeneous interference whose inverse covariance structure is defined via unitary invariant functions in this paper we deal with the problem of detecting an extended target embedded in homogeneous gaussian interference with unknown but structured covariance matrix we model the possible target echo from each range bin under test as a deterministic signal with an unknown scaling factor accounting for the target response at the design stage we exploit some a priori knowledge about the operating environment enforcing the inverse interference plus noise covariance matrix to belong to a set described via unitary invariant continuous functions hence we derive the constrained maximum likelihood ml estimates of the unknown parameters under both the h0 and h1 hypotheses and design the generalized likelihood ratio test glrt for the considered decision problem at the analysis stage we assess the performance of the devised glrt for some covariance matrix uncertainty sets of practical relevance both for spatial and doppler processing the results highlight that correct use of the a priori knowledge can lead to a detection performance quite close to the optimum receiver which supposes the perfect knowledge of the interference plus noise covariance matrix

fused trees and some new approaches to source coding the concept of fusion partition for the node set of a tree and the concept of a fused tree corresponding to it are discussed a fused tree requires in principle significantly less memory for storage than the original tree yet allows the recovery of the original hence it makes sense to store a tree in a fused form furthermore if labels from an ordered set c are assigned to the branches of the original tree so that a lexographic alphabetic order on the set of terminal nodes is induced a fused tree still enables the execution of direct and inverse enumeration algorithms a fusion algorithm to produce the completely fused tree for a given tree is presented some approaches to data compression problems related to the algorithms given are discussed

joint estimation of inverse covariance matrices lying in an unknown subspace we consider the problem of joint estimation of inverse covariance matrices lying in an unknown subspace of the linear space of symmetric matrices we perform the estimation using groups of measurements with different covariances assuming the inverse covariances span a low dimensional subspace our aim is to determine this subspace and to exploit this knowledge in order to improve the estimation we develop a novel optimization algorithm discovering and exploiting the underlying low dimensional subspace we provide a computationally efficient algorithm and derive a tight upper performance bound numerical simulations on synthetic and real world data are presented to illustrate the performance benefits of the algorithm

structural analysis of indistinguishability operators and related concepts given an indistinguishability operator e it is possible to obtain the set of extensional fuzzy subsets h e and the respective operators of upper and lower approximations by extensional sets f e and j e reciprocally given any of these objects the initial indistinguishability operator e can be retrieved it is well known that these concepts are in bijection in this paper we will prove that the relation underlying them is a lattice isomorphism we will also consider further operators such as natural means and finally show the robustness of the results with respect to isomorphisms of t norms

subspace based adaptive generalized likelihood ratio detection subspace based adaptive detection performance is examined for the generalized likelihood ratio detector based on wilks spl lambda statistic the problem considered here is detecting the presence of one or more signals of known shape embedded in gaussian distributed noise with unknown covariance structure the data is mapped into a subspace prior to detection the probability of false alarm is independent of the subspace transformation and depends only on subspace dimension the probability of detection depends on the subspace transformation through a nonadaptive signal to noise ratio snr parameter subspace processing results in an snr loss that tends to decrease performance and a gain in statistical stability that tends to increase performance it is shown that the statistical stability effect dominates the snr loss for short data records and subspace detectors can require substantially less snr than full space detectors for equivalent performance a method for designing the subspace transformation to minimize the snr loss is proposed and illustrated through simulations

adaptive array detection of uncertain rank one waveforms adaptive array detection of known within a complex scaling rank one space time waveforms in unknown spatial noise has received considerable attention the two published solutions are the adaptive matched filter and the glrt we expand on this work to consider the case of rank one waveforms that are uncertain i e only partially known more precisely we model the space time steering vector as the kronecker product of two vectors each of which is unknown but is known to lie in a known subspace applications for such a model include detection in the presence of multipath and spectral or polarization diversity in both radar processing and wireless communication using the principle of invariance we construct detectors based on the maximal invariant we show that the snr required to achieve a given detection probability for a given false alarm rate is only weakly impacted by waveform uncertainty thus our detector approaches the performance of earlier detectors which entail known waveforms

knowledge aided parametric adaptive matched filter with automatic combining for covariance estimation in this paper a knowledge aided parametric adap tive matched filter ka pamf is proposed that utilizing both observations including the test and training signals and ap riori knowledge of the spatial covariance matrix unlike existing ka pamf methods the proposed ka pamf is able to automat ically adjust the combining weight of ap rioricovariance matrix thus gaining enhanced robustness against uncertainty in the prior knowledge meanwhile the proposed ka pamf is significantly more efficient than its ka nonparametric counterparts when the amount of training signals is limited one distinct feature of the proposed ka pamf is the inclusion of both the test and training signals for automatic determination of the combining weights for the prior spatial covariance matrix and observations numerical results are presented to demonstrate the effectiveness of the proposed ka pamf especially in the limited training scenarios

partially hidden markov models partially hidden markov models phmm are introduced they differ from the ordinary hmms in that both the transition probabilities of the hidden states and the output probabilities are conditioned on past observations as an illustration they are applied to black and white image compression where the hidden variables may be interpreted as representing noncausal pixels

reduced complexity delayed decision algorithm for context based image processing systems it is well known that the performance of context based image processing systems can be improved by allowing the processor e g an encoder or a denoiser a delay of several samples before making a processing decision often however for such systems traditional delayed decision algorithms can become computationally prohibitive due to the growth in the size of the space of possible solutions in this paper we propose a reduced complexity one pass delayed decision algorithm that systematically reduces the size of the search space while also preserving its structure in particular we apply the proposed algorithm to two examples of adaptive context based image processing systems an image coding system that employs a context based entropy coder and a spatially adaptive image denoising system for these two types of widely used systems we show that the proposed delayed decision search algorithm outperforms instantaneous decision algorithms with only a small increase in complexity we also show that the performance of the proposed algorithm is better than that of other higher complexity delayed decision algorithms

universal prediction this paper consists of an overview on universal prediction from an information theoretic perspective special attention is given to the notion of probability assignment under the self information loss function which is directly related to the theory of universal data compression both the probabilistic setting and the deterministic setting of the universal prediction problem are described with emphasis on the analogy and the differences between results in the two settings

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

on the structure of f indistinguishability operators abstract some properties concerning the structure of f indistinguishability operators are analyzed it is shown that any of such operators is generated by a family of fuzzy subsets this result since it gives the way to construct f indistinguishabilities facilitates new applications of fuzzy relations the links between f indistinguishability operators and a kind of generalized metrics in the unit interval which are also explored are used to define the canonical generators of an f indistinguishability operator that is the fuzzy partition associated with the operator

a dual interpretation for direct binary search and its implications for tone reproduction and texture quality the direct binary search dbs algorithm employs a search heuristic to minimize the mean squared perceptually filtered error between the halftone and continuous tone original images based on an efficient method for evaluating the effect on the mean squared error of trial changes to the halftone image we show that dbs also minimizes in a pointwise sense the absolute error under the same visual model but at twice the viewing distance associated with the mean squared error metric this dual interpretation sheds light on the convergence properties of the algorithm and clearly explains the tone bias that has long been observed with halftoning algorithms of this type it also demonstrates how tone bias and texture quality are linked via the scale parameter the product of printer resolution and viewing distance finally we show how the tone bias can be eliminated by tone correcting the continuous tone image prior to halftoning it

estimation and inference by compact coding summary the systematic variation within a set of data as represented by a usual statistical model may be used to encode the data in a more compact form than would be possible if they were considered to be purely random the encoded form has two parts the first states the inferred estimates of the unknown parameters in the model the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure in its strict form it has great generality and several nice properties but is computationally infeasible an approximate form is developed and its relation to other methods is explored

design and analysis of vector color error diffusion halftoning systems traditional error diffusion halftoning is a high quality method for producing binary images from digital grayscale images error diffusion shapes the quantization noise power into the high frequency regions where the human eye is the least sensitive error diffusion may be extended to color images by using error filters with matrix valued coefficients to take into account the correlation among color planes for vector color error diffusion we propose three contributions first we analyze vector color error diffusion based on a new matrix gain model for the quantizer which linearizes vector error diffusion the model predicts the key characteristics of color error diffusion esp image sharpening and noise shaping the proposed model includes linear gain models for the quantizer by ardalan and paulos 1987 and by kite et al 1997 as special cases second based on our model we optimize the noise shaping behavior of color error diffusion by designing error filters that are optimum with respect to any given linear spatially invariant model of the human visual system our approach allows the error filter to have matrix valued coefficients and diffuse quantization error across color channels in an opponent color representation thus the noise is shaped into frequency regions of reduced human color sensitivity to obtain the optimal filter we derive a matrix version of the yule walker equations which we solve by using a gradient descent algorithm finally we show that the vector error filter has a parallel implementation as a polyphase filterbank

adaptive double subspace signal detection in gaussian background part i homogeneous environments in this two part paper we consider the problem of adaptive multidimensional multichannel signal detection in gaussian noise with unknown covariance matrix the test data primary data is assumed as a collection of sample vectors arranged as the columns of a rectangular data array the rows and columns of the signal matrix are both assumed to lie in known subspaces but with unknown coordinates due to this feature of the signal structure we name this kind of signal as the double subspace signal part i of this paper focuses on the adaptive detection in homogeneous environments while part ii deals with the adaptive detection in partially homogeneous environments precisely in this part we derive the generalized likelihood ratio test glrt rao test wald test as well as their two step variations in homogeneous environments three types of spectral norm tests snts are also introduced all these detectors are shown to possess the constant false alarm rate cfar property moreover we discuss the differences between them and show how they work another contribution is that we investigate various special cases of these detectors remarkably some of them are well known existing detectors while some others are still new at the stage of performance evaluation conducted by monte carlo simulations both matched and mismatched signals are dealt with for each case more than one scenario is considered

tree structured method for lut inverse halftoning and for image halftoning the authors previously proposed a look up table lut based method for inverse halftoning of images the lut for inverse halftoning is obtained from the histogram gathered from a few sample halftone images and corresponding original images many of the entries in the lut are unused because the corresponding binary patterns hardly occur in commonly encountered halftones these are called nonexistent patterns in this paper we propose a tree structure which will reduce the storage requirements of an lut by avoiding nonexistent patterns we demonstrate the performance on error diffused images and ordered dither images then we introduce lut based halftoning and tree structured lut tlut halftoning even though the tlut method is more complex than lut halftoning it produces better halftones and requires much less storage than lut halftoning we demonstrate how the error diffusion characteristics can be achieved with this method afterwards our algorithm is trained on halftones obtained by direct binary search dbs the complexity of tlut halftoning is higher than the error diffusion algorithm but much lower than the dbs algorithm also the halftone quality of tlut halftoning increases if the size of the tlut gets bigger thus the halftone image quality between error diffusion and dbs will be achieved depending on the size of the tree structure in the tlut algorithm

weakly convergent nonparametric forecasting of stationary time series the conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past several schemes are known for constructing pointwise consistent estimates but they all demand prohibitive amounts of input data we consider real valued time series and construct conditional distribution estimates that make much more efficient use of the input data the estimates are consistent in a weak sense and the question whether they are pointwise consistent is still open for finite alphabet processes one may rely on a universal data compression scheme like the lempel ziv 1978 algorithm to construct conditional probability mass function estimates that are consistent in expected information divergence consistency in this strong sense cannot be attained in a universal sense for all stationary processes with values in an infinite alphabet but weak consistency can some applications of the estimates to on line forecasting regression and classification are discussed

a new class of b w halftoning algorithms a new class of dithering algorithms for black and white b w images is presented the basic idea behind the technique is to divide the image into small blocks and minimize the distortion between the original continuous tone image and its low pass filtered halftone this corresponds to a quadratic programming problem with linear constraints which is solved via standard optimization techniques examples of b w halftone images obtained by this technique are compared to halftones obtained via existing dithering algorithms

predicting a binary sequence almost as well as the optimal biased coin we apply the exponential weight algorithm introduced and littlestone and warmuth 26 and by vovk 35 to the problem of predicting a binary sequence almost as well as the best biased coin we first show that for the case of the logarithmic loss the derived algorithm is equivalent to the bayes algorithm with jeffrey s prior that was studied by xie and barron 38 under probabilistic assumptions we derive a uniform bound on the regret which holds for any sequence we also show that if the empirical distribution of the sequence is bounded away from 0 and from 1 then as the length of the sequence increases to infinity the difference between this bound and a corresponding bound on the average case regret of the same algorithm which is asymptotically optimal in that case is only 1 2 we show that this gap of 1 2 is necessary by calculating the regret of the min max optimal algorithm for this problem and showing that the asymptotic upper bound is tight we also study the application of this algorithm to the square loss and show that the algorithm that is derived in this case is different from the bayes algorithm and is better than it for prediction in the worst case

efficient homophonic coding homophonic coding or homophonic substitution is referred to as a technique that contributes to reliability of the secret key cipher systems its main goal is to convert the plaintext into a sequence of completely random equiprobable and independent code letters in solving this problem three characteristics are to be considered i redundancy defined as the difference between the mean codeword length and the source entropy ii an average number of random bits used in encoding and iii complexity of the encoder and decoder measured by memory size in bits and computation time in bit operations a class of homophonic codes is suggested for which both the redundancy and the average number of random bits can be made as small as required with nonexponential growth of memory size and roughly logarithmic growth of computation time

diffuse multipath exploitation for adaptive radar detection we deal with the problem of detecting point like targets in diffuse multipath environments modeling the target echo as the superposition of a deterministic signal with an unknown scaling factor due to the direct path plus a zero mean complex circular symmetric gaussian random vector with an unknown covariance matrix accounting for the echoes from the glistening surface we devise a constrained generalized likelihood ratio test glrt for the resulting hypothesis testing problem enforcing the primary data covariance matrix due to both interference and multipath echoes to belong to a neighborhood of the secondary data sample covariance matrix remarkably the proposed decision scheme ensures the desirable constant false alarm rate cfar property with respect to the unknown parameters of the interference the performance assessment conducted on simulated data in terms of detection probability also in comparison with existing solutions highlights the effectiveness of the new approach to cope with diffuse multipath phenomena

transitivity of fuzzy relations under discretization fuzzy transitivity is a key property for many fuzzy relational structures such as fuzzy preorders and equivalences theoretical models for practical problems which are based on fuzzy relations make use of continuous scales mostly the unit interval 0 1 practical implementation of these models though involves their discretization into finite scales which generally results in some loss of transitivity in this paper we study if there are any transitivity preserving discretization strategies also we evaluate the loss of transitivity in some commonly used discretization approaches

compaction of ordered dithered images with arithmetic coding ordered dither is considered to be a simple and effective method among all halftoning techniques in this paper compaction of ordered dithered images using arithmetic coding is studied a preprocessor referred to as pixel interleaving i e grouping pixels with similar dithering thresholds is employed in such a way that dithered images can be efficiently coded with the jbig1 code and high compressibility can be achieved experimental results reveal that the four pixel interleaving achieves the best compression performance

look up table lut method for inverse halftoning in this paper we propose look up table lut based methods for inverse halftoning of images the lut for inverse halftoning is obtained from the histogram gathered from a few sample halftone images and corresponding original images the method is extremely fast no filtering is required and the psnr and visual image quality achieved is comparable to the best methods known for inverse halftoning the lut inverse halftoning method does not depend on the specific properties of the halftoning method and can be applied to any halftoning method then an algorithm for template selection for lut inverse halftoning is introduced we demonstrate the performance of the lut inverse halftoning algorithm on error diffused images and ordered dithered images we also extend lut inverse halftoning to color halftones

performance analysis of grammar based codes revisited the compression performance of grammar based codes is revisited from a new perspective previously the compression performance of grammar based codes was evaluated against that of the best arithmetic coding algorithm with finite contexts in this correspondence we first define semifinite state sources and finite order semi markov sources based on the definitions of semifinite state sources and finite order semi markov sources and the idea of run length encoding rle we then extend traditional rle algorithms to context based rle algorithms rle algorithms with k contexts and rle algorithms of order k where k is a nonnegative integer for each individual sequence x let r sup sub sr k x and r sup sub sr k x be the best compression rate given by rle algorithms with k contexts and by rle algorithms of order k respectively it is proved that for any x r sup sub sr k is no greater than the best compression rate among all arithmetic coding algorithms with k contexts furthermore it is shown that there exist stationary ergodic semi markov sources for which the best rle algorithms without any context outperform the best arithmetic coding algorithms with any finite number of contexts finally we show that the worst case redundancies of grammar based codes against r sup sub sr k x and r sup sub sr k x among all length n individual sequences x from a finite alphabet are upper bounded by d sub 1 loglogn logn and d sub 2 loglogn logn respectively where d sub 1 and d sub 2 are constants this redundancy result is stronger than all previous corresponding results

a sequential algorithm for the universal coding of finite memory sources the estimation and universal compression of discrete sources are considered and a sequential algorithm for the universal coding of finite memory sources attaining asymptotically minimum redundancy is presented the algorithm performs an online estimation of the source states and uses an arithmetic code

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

a double adaptive file compression algorithm we describe a one pass compression scheme which presumes no statistical properties of the data being compressed the model structure adaptively selects a subset of first order markov contexts based on an estimate of the candidate context s popularity the probability distributions for the unselected lumped first order contexts are made the same reducing cost over a full first order markov model symbol repetitions are handled in special secondorder markov contexts the statistics for each symbol are adaptively determined by an extension of earlier work

type classes of context trees it is well known that a tree model does not always admit a finite state machine fsm representation with the same minimal number of parameters therefore known characterizations of type classes for fsms do not apply in general to tree models in this paper the type class of a sequence with respect to a given context tree is studied an exact formula is derived for the size of the class extending whittle s formula for type classes with respect to fsms the derivation is more intricate than in the fsm case since some basic properties of fsm types do not hold in general for tree types the derivation also yields an efficient enumeration of the tree type class a formula for the number of type classes with respect to is also derived the formula is asymptotically tight up to a multiplicative constant and also extends the corresponding result for fsms the asymptotic behavior of the number of type classes and of the size of a class is expressed in terms of the so called minimal canonical extension of t a tree that is generally larger than but smaller than its fsm closure

adaptive detection of range distributed targets a modified generalized likelihood ratio test mglrt for the adaptive detection of a target or targets that are distributed in range is derived the unknown parameters associated with the hypothesis test are the complex amplitudes in range of the desired target and the unknown covariance matrix of the additive interference which is assumed to be characterized as complex zero mean correlated gaussian random variables the target s or targets complex amplitudes are assumed to be distributed across the entire input data block sensor spl times range results on probabilities of false alarm and detection are derived and a bounded constant false alarm rate cfar detector is developed simulation results are presented it is shown that the derived mglrt of range distributed targets is much more effective in detecting targets distributed in range than an m out of k detector which is cascaded with a single point target kelly 1986 detector

universal modeling and coding the problems arising in the modeling and coding of strings for compression purposes are discussed the notion of an information source that simplifies and sharpens the traditional one is axiomatized and adaptive and nonadaptive models are defined with a measure of complexity assigned to the models a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all a general class of so called first in first out fifo arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models because the coding parameters are the probabilities that define the model their design is easy and the application of the code is straightforward even with adaptively changing source models

adaptive sequential prediction of multidimensional signals with applications to lossless image coding we investigate the problem of designing adaptive sequential linear predictors for the class of piecewise autoregressive multidimensional signals and adopt an approach of minimum description length mdl to determine the order of the predictor and the support on which the predictor operates the design objective is to strike a balance between the bias and variance of the prediction errors in the mdl criterion the predictor design problem is particularly interesting and challenging for multidimensional signals e g images and videos because of the increased degree of freedom in choosing the predictor support our main result is a new technique of sequentializing a multidimensional signal into a sequence of nested contexts of increasing order to facilitate the mdl search for the order and the support shape of the predictor and the sequentialization is made adaptive on a sample by sample basis the proposed mdl based adaptive predictor is applied to lossless image coding and its performance is empirically established to be the best among all the results that have been published till present

relations in fuzzy class theory this paper studies fuzzy relations in the graded framework of fuzzy class theory fct this includes i rephrasing existing work on graded properties of binary fuzzy relations in the framework of fuzzy class theory and ii generalizing existing crisp results on fuzzy relations to the graded framework our particular aim is to demonstrate that fuzzy class theory is a powerful and easy to use instrument for handling fuzzified properties of fuzzy relations this paper does not rephrase the whole theory of fuzzy relations instead it provides an illustrative introduction showing some representative results with a strong emphasis on fuzzy preorders and fuzzy equivalence relations

progressive image coding using trellis coded quantization in this work we present coding techniques that enable progressive transmission when trellis coded quantization tcq is applied to wavelet coefficients a method for approximately inverting tcq in the absence of least significant bits is developed results are presented using different rate allocation strategies and different entropy coders the proposed wavelet tcq coder yields excellent coding efficiency while supporting progressive modes analogous to those available in jpeg

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

tree searched chain coding for subpixel reconstruction of planar curves coding schemes for the quantization of line drawings that outperform basic chain codes are investigated first subpixel accuracy reconstruction is obtained by a simple linear filtering of the chain code points which achieves a factor of three to four reduction in average reconstruction distortion for smooth curves second the basic chain encoding schemes are generalized to a multipath tree searched encoding scheme a variation of the m l algorithm is used to maintain m contending chain code paths in storage and choose the best path from these over a wide variety of source curves tree searched chain coding results in nearly an order of magnitude reduction in average reconstruction distortion over smoother chain codes the performance improvement for curves is obtained with only a slight increase in bit rate over basic chain codes

maximum entropy as a special case of the minimum description length criterion corresp the maximum entropy me and maximum likelihood ml criteria are the bases for two approaches to statistical inference problems a new criterion called the minimum description length mdl has been recently introduced this criterion generalizes the ml method so it can be applied to more general situations e g when the number of parameters is unknown it is shown that me is also a special case of the mdl criterion maximizing the entropy subject to some constraints on the underlying probability function is identical to minimizing the code length required to represent all possible i i d realizations of the random variable such that the sample frequencies or histogram satisfy those given constraints

a comparison of enumerative and adaptive codes the problem of noiselessly encoding a message when prior statistics are known is considered the close relationship of arithmetic and enumerative coding for this problem is shown by computing explicit arithmetic coding probabilities for various enumerative coding examples this enables a comparison to be made of the coding efficiency of markov models and enumerative codes as well as a new coding scheme intermediate between the two these codes are then extended to messages whose statistics are not known em a priori two adaptive codes are described for this problem whose coding efficiency is upper bounded by the extended enumerative codes on some practical examples the adaptive codes perform significantly better than the nonadaptive ones

how to use expert advice we analyze algorithms that predict a binary value by combining the predictions of several prediction strategies called experts our analysis is for worst case situations i e we make no assumptions about the way the sequence of bits to be predicted is generated we measure the performance of the algorithm by the difference between the expected number of mistakes it makes on the bit sequence and the expected number of mistakes made by the best expert on this sequence where the expectation is taken with respect to the randomization in the predictins we show that the minimum achievable difference is on the order of the square root of the number of mistakes of the best expert and we give efficient algorithms that achieve this our upper and lower bounds have matching leading constants in most cases we then show how this leads to certain kinds of pattern recognition learning algorithms with performance bounds that improve on the best results currently know in this context we also compare our analysis to the case in which log loss is used instead of the expected number of mistakes

inferring decision trees using the minimum description length principle abstract we explore the use of rissanen s minimum description length principle for the construction of decision trees empirical results comparing this approach to other methods are given

glrt based adaptive detection algorithms for range spread targets we address adaptive detection of a range spread target or targets embedded in gaussian noise with unknown covariance matrix to this end we assume that cells referred to in the following as secondary data that are free of signal components are available those secondary data are supposed to possess either the same covariance matrix or the same structure of the covariance matrix of the cells under test in this context we design detectors relying on the generalized likelihood ratio test glrt and on a two step glrt based design procedure remarkably both criteria lead to receivers ensuring the constant false alarm rate cfar property with respect to the unknown quantities a thorough performance assessment of the proposed detection strategies together with the evaluation of their processing cost highlights that the two step design procedure is to be preferred with respect to the plain glrt in fact the former leads to detectors that achieve satisfactory performance under several situations of practical interest and are simpler to implement than those designed resorting to the latter

new asymptotic bounds and improvements on the lempel ziv data compression algorithm new results which follow from a data compression algorithm proposed by a lempel and j ziv 1977 are presented the authors describe the original lempel ziv algorithm and present an improved lempel ziv data compression algorithm they state and prove some asymptotic bounds on the compression ratio that the improved algorithm achieves for completeness they describe the lempel ziv welch data compression algorithm presented by t a welch 1984 a software implementation for both the original and the improved lempel ziv algorithms is given and the algorithms are compared with each other as well as with the lempel ziv welch algorithm

complexity of strings in the class of markov sources shannon s self information of a string is generalized to its complexity relative to the class of finite state machine fsm defined sources unlike an earlier generalization the new one is valid for both short and long strings the definition is justified in part by a theorem stating that asymptotically the mean complexity provides a tight lower bound for the mean length of all so called regular codes this also generalizes shannon s noiseless coding theorem for a large subclass of fsm sources a simple algorithm is described for computing the complexity

universal portfolios with side information we present a sequential investment algorithm the spl mu weighted universal portfolio with side information which achieves to first order in the exponent the same wealth as the best side information dependent investment strategy the best state constant rebalanced portfolio determined in hindsight from observed market and side information outcomes this is an individual sequence result which shows the difference between the exponential growth wealth of the best state constant rebalanced portfolio and the universal portfolio with side information is uniformly less than d 2n log n 1 k n log 2 for every stock market and side information sequence and for all time n here d k m 1 is the number of degrees of freedom in the state constant rebalanced portfolio with k states of side information and m stocks the proof of this result establishes a close connection between universal investment and universal data compression

an o n semipredictive universal encoder via the bwt we provide an o n algorithm for a nonsequential semipredictive encoder whose pointwise redundancy with respect to any unbounded depth tree source is o 1 bits per state above rissanen s lower bound this is achieved by using the burrows wheeler transform bwt an invertible permutation transform that has been suggested for lossless data compression first we use the bwt only as an efficient computational tool for pruning context trees and encode the input sequence rather than the bwt output second we estimate the minimum description length mdl source by incorporating suffix tree methods to construct the unbounded depth context tree that corresponds to the input sequence in o n time third we point out that a variety of previous source coding methods required superlinear complexity for determining which tree source state generated each of the symbols of the input we show how backtracking from the bwt output to the input sequence enables to solve this problem in o n worst case complexity

on line adaptive canonical prefix coding with bounded compression loss semistatic minimum redundancy prefix mrp coding is fast compared with rival coding methods but requires two passes during encoding its adaptive counterpart dynamic huffman coding requires only one pass over the input message for encoding and decoding and is asymptotically efficient dynamic huffman 1952 coding is however notoriously slow in practice by removing the restriction that the code used for each message symbol must have minimum redundancy and thereby admitting some compression loss it is possible to improve the speed of adaptive mrp coding this paper presents a controlled method for trading compression loss for coding speed by approximating symbol frequencies with a geometric distribution the result is an adaptive mrp coder that is asymptotically efficient and also fast in practice