maximum likelihood estimation of functionals of discrete distributions we consider the problem of estimating functionals of discrete distributions and focus on a tight up to universal multiplicative constants for each specific functional nonasymptotic analysis of the worst case squared error risk of widely used estimators we apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional namely their bias we explicitly characterize the worst case squared error risk incurred by the maximum likelihood estimator mle in estimating the shannon entropy h p sum i 1 s p i ln p i and the power sum f alpha p sum i 1 s p i alpha alpha 0 up to universal multiplicative constants for each fixed functional for any alphabet size s leq infty and sample size n for which the risk may vanish as a corollary for shannon entropy estimation we show that it is necessary and sufficient to have n gg s observations for the mle to be consistent in addition we establish that it is necessary and sufficient to consider n gg s 1 alpha samples for the mle to consistently estimate f alpha p 0 the minimax rate optimal estimators for both problems require s ln s and s 1 alpha ln s samples which implies that the mle has a strictly sub optimal sample complexity when 1 we show that the worst case squared error rate of convergence for the mle is n 2 alpha 1 for infinite alphabet size while the minimax squared error rate is n ln n 2 alpha 1 when alpha geq 3 2 the mle achieves the minimax optimal rate n 1 regardless of the alphabet size as an application of the general theory we analyze the dirichlet prior smoothing techniques for shannon entropy estimation in this context one approach is to plug in the dirichlet prior smoothed distribution into the entropy functional while the other one is to calculate the bayes estimator for entropy under the dirichlet prior for squared error which is the conditional expectation we show that in general such estimators do not improve over the maximum likelihood estimator no matter how we tune the parameters in the dirichlet prior this approach cannot achieve the minimax rates in entropy estimation the performance of the minimax rate optimal estimator with n samples is essentially at least as good as that of dirichlet smoothed entropy estimators with n ln n samples

uniform estimates for the finite time ruin probability in the dependent renewal risk model abstract this paper investigates the finite time ruin probability in the dependent renewal risk model where the claim sizes are independent and identically distributed random variables with strongly subexponential tails and the interarrival times are negatively dependent we establish an asymptotic estimate which holds uniformly for the time horizon varying in the positive half line

precise rates in the law of the logarithm for negatively associated random variables let x n n 1 be a strictly stationary sequence of negatively associated random variables with mean zero and finite variance set s email protected k 1 nx k m n max k n s k n 1 suppose s 2 ex 1 email protected k 2 ex 1x k we study the precise rates of a kind of weighted infinite series of p m n email protected snlogn and p s n email protected snlogn as email protected 0 and p m email protected email protected email protected 2n8logn as email protected the results are related to the convergence rates of the law of the logarithm and the chung type law of the logarithm

laguerre domain adaptive filters using a tapped delay line as an adaptive fitter the complexity of the filter increases with increasing correlation length of input and reference signal the author seeks simple adaptive filter structures such that for a long correlation length only a minor complexity of the filter is needed he considers adaptive mechanisms governed by an exponentially weighted squared error criterion laguerre domain adaptive filters are introduced which lead to a tapped iir filter line these filters contain a discount factor as a free variable which makes it possible to set the memory and the number of adaptive coefficients independently convergence properties of the proposed adaptive filters are discussed

exponential stability of general tracking algorithms tracking and adaptation algorithms are from a formal point of view nonlinear systems which depend on stochastic variables in a fairly complicated way the analysis of such algorithms is thus quite complicated a first step is to establish the exponential stability of these systems this is of interest in its own right and a prerequisite for the practical use of the algorithm it is also a necessary starting point to analyze the performance in terms of tracking and adaptation because that is how close the estimated parameters are to the time varying true ones in this paper we establish some general conditions for the exponential stability of a wide and common class of tracking algorithms this includes least mean squares recursive least squares and kalman filter based adaptation algorithms we show how stability of an averaged linear and deterministic equation and stability of the actual algorithm are linked to each other under weak conditions on the involved stochastic processes we also give explicit conditions for exponential stability of the most common algorithms the tracking performance of the algorithms is studied in a companion paper

directional dependence of random vectors in this paper we define and study several properties of the concept of multivariate directional dependence and study the new concept in terms of copulas and provide new measures of multivariate association based on directional dependence finally we show the relationships of directional dependence with other dependence concepts such as association and conditional dependence

asymptotics and uniform asymptotics for finite time and infinite time absolute ruin probabilities in a dependent compound renewal risk model abstract in this paper we consider a dependent compound renewal risk model with constant premium rate and interest rate where the individual claim sizes are widely orthant dependent and the claim number has a distribution belonging to the intersection among the maximum domain of attraction of the gumbel distribution the subexponential class and the rapidly varying class in such a dependent compound renewal or poisson risk model we obtain the asymptotics and uniform asymptotics for the finite time and infinite time absolute ruin probabilities to this end we investigate the tail behavior of the random sum with some widely orthant dependent summands and the random number in the maximum domain of attraction of the gumbel distribution

tail behavior of random sums of negatively associated increments abstract let x x k k 1 be a sequence of negatively associated random variables with a common distribution function and finite expectation and let be a nonnegative integer valued random variable independent of x k k 1 in this paper we give unified form for the asymptotic behavior of the random sums s k 1 x k in the case of lim x p x x p x c 0 the results extend the earlier results of aleskeviciene et al 2008 2

some concepts of negative dependence abstract the theory of positive dependence notions cannot yield useful results for some widely used distributions such as the multinomial dirichlet and the multivariate hypergeometric some conditions of negative dependence that are satisfied by these distributions and which have practical meaning are introduced preservation results for some of these concepts are derived useful inequalities for some widely used distributions are obtained results of mallows 1969 that apply to the multinomial distributions are extended to more distributions examples are listed author

solving optimization problems with diseconomies of scale via decoupling we present a new framework for solving optimization problems with a diseconomy of scale in such problems our goal is to minimize the cost of resources used to perform a certain task the cost of resources grows superlinearly as xq q 1 with the amount x of resources used we define a novel linear programming relaxation for such problems and then show that the integrality gap of the relaxation is aq where aq is the q th moment of the poisson random variable with parameter 1 using our framework we obtain approximation algorithms for the minimum energy efficient routing minimum degree balanced spanning tree load balancing on unrelated parallel machines and unrelated parallel machine scheduling with nonlinear functions of completion times problems our analysis relies on the decoupling inequality for nonnegative random variables the inequality states that ie1nxi q cq ie1nyi q where xi are independent nonnegative random variables yi are possibly dependent nonnegative random variables and each yi has the same distribution as xi the inequality was proved by de la pena in 1990 de la pena ibragimov and sharakhmetov showed that cq 2 for qi 1 2 and cq aq1 q for q 2 we show that the optimal constant is cqe aq1 q for any q 1 we then prove a more general inequality for every convex function v elv ie1nxi r elv p ie1n yi r and for every concave function el ie1n xi r el p ie1n yi r where p is a poisson random variable with parameter 1 independent of the random variables yi

system identification with generalized orthonormal basis functions a least squares identification method is studied that estimates a finite number of expansion coefficients in the series expansion of a transfer function where the expansion is in terms of recently introduced generalized basis functions the basis functions are orthogonal in h2 and generalize the pulse laguerre and kautz bases one of their important properties is that when chosen properly they can substantially increase the speed of convergence of the series expansion this leads to accurate approximate models with only a few coefficients to be estimated explicit bounds are derived for the bias and variance errors that occur in parameter estimates as well as in the resulting transfer function estimates

system identification using kautz models in this paper the problem of approximating a linear time invariant stable system by a finite weighted sum of given exponentials is considered system identification schemes using laguerre models are extended and generalized to kautz models which correspond to representations using several different possible complex exponentials in particular linear regression methods to estimate this sort of model from measured data are analyzed the advantages of the proposed approach are the simplicity of the resulting identification scheme and the capability of modeling resonant systems using few parameters the subsequent analysis is based on the result that the corresponding linear regression normal equations have a block toeplitz structure several results on transfer function estimation are extended to discrete kautz models for example asymptotic frequency domain variance expressions

a generalized orthonormal basis for linear dynamical systems in many areas of signal system and control theory orthogonal functions play an important role in issues of analysis and design in this paper it is shown that there exist orthogonal functions that in a natural way are generated by stable linear dynamical systems and that compose an orthonormal basis for the signal space l sub 2sup n to this end use is made of balanced realizations of inner transfer functions the orthogonal functions can be considered as generalizations of for example the pulse functions laguerre functions and kautz functions and give rise to an alternative series expansion of rational transfer functions it is shown how we can exploit these generalized basis functions to increase the speed of convergence in a series expansion i e to obtain a good approximation by retaining only a finite number of expansion coefficients consequences for identification of expansion coefficients are analyzed and a bound is formulated on the error that is made when approximating a system by a finite number of expansion coefficients

approximations for finite time ruin probability in a dependent discrete time risk model with cmc simulations consider a discrete time risk model in which the insurer is allowed to invest its wealth into a risk free or a risky portfolio under a certain regulation then the insurer is said to be exposed to a stochastic economic environment that contains two kinds of risks the insurance risk and financial risk within period i the net insurance loss is denoted by xi and the stochastic discount factor from time i to zero is denoted by i for any integer n assume that x1 xn form a sequence of pairwise asymptotically independent but not necessarily identically distributed real valued random variables with distributions f1 fn respectively 1 2 n are another sequence of arbitrarily dependent positive random variables and the two sequences are mutually independent under the assumption that the average distribution n1i 1nfi is dominatedly varying tailed and some moment conditions on i i 1 n we derive a weakly equivalent formula for the finite time ruin probability we demonstrate our obtained results through a crude monte carlo simulation with asymptotics

negative association of random variables with applications random variables x 1 cdots x k are said to be negatively associated na if for every pair of disjoint subsets a 1 a 2 of 1 2 cdots k operatorname cov lbrack f x 1 i in a 1 g x j j in a 2 rbrack leq 0 for all nondecreasing functions f g the basic properties of negative association are derived especially useful is the property that nondecreasing functions of mutually exclusive subsets of na random variables are na this property is shown not to hold for several other types of negative dependence recently proposed one consequence is the inequality p x i leq x i i 1 cdots k leq prod k 1p x i leq x i for na random variables x 1 cdots x k and the dual inequality resulting from reversing the inequalities inside the square brackets in another application it is shown that negatively correlated normal random variables are na other na distributions are the a multinomial b convolution of unlike multinomials c multivariate hypergeometric d dirichlet and e dirichlet compound multinomial negative association is shown to arise in situations where the probability measure is permutation invariant applications of this are considered for sampling without replacement as well as for certain multiple ranking and selection procedures in a somewhat striking example na and positive association representing quite strong opposing types of dependence are shown to exist side by side in models of categorical data analysis

association of normal random variables and slepian s inequality abstract a simple extension of slepian s inequality is given which implies the original inequality and the result that positively correlated normal variables are associated author

posterior convergence given the mean for various applications one wants to know the asymptotic behavior of x the posterior density of a parameter given the mean x of the data rather than the full data set here we show that x is asymptotically normal in an l 1 sense and we identify the mean of the limiting normal and its asymptotic variance the main results are first proved assuming that x 1 x n are independent and identical suitable modifications to obtain results for the nonidentical case are given separately our results may be used to construct approximate hpd highest posterior density sets for the parameter which is of use in the statistical theory of standardized educational tests they may also be used to show the covariance between two test items conditioned on the mean is asymptotically nonpositive this has implications for constructing tests of item independence

ricci flow einstein metrics and space forms the main results in this paper are 1 ricci pinched stable riemannian metrics can be deformed to einstein metrics through the ricci flow of r hamilton 2 suitably negatively pinched riemannian manifolds can be deformed to hyperbolic space forms through ricci flow and 3 l 2 pinched riemannian manifolds can be deformed to space forms through ricci flow

tail probability of a random sum with a heavy tailed random number and dependent summands abstract let k k 1 be a sequence of widely orthant dependent random variables with common distribution f satisfying e 0 let be a nonnegative integer valued random variable in this paper we discuss the tail probabilities of random sums s n 1 n when the random number has a heavier tail than the summands i e p x p x 0 as x under some additional technical conditions we prove that if has a consistently varying tail then s has a consistently varying tail and p s x p x e on the other hand the converse problem is also equally interesting we prove that if s has a consistently varying tail then has a consistently varying tail and that p s x p x e still holds in particular the random number is not necessarily assumed to be independent of the summands k k 1 in theorem 3 1 and theorem 3 3 finally some applications to the asymptotic behavior of the finite time ruin probabilities in some insurance risk models are given

a system identification algorithm using orthogonal functions an adaptive filter adf structure is proposed for applications in which large order adfs are required it is based on modeling the impulse response of the system to be identified as a linear combination of a set of discrete legendre orthogonal functions the proposed adaptive filter structure has desirable stability features and a unimodal mean square error surface as well as a modular structure that permits an easy increase of the filter order without changing the previous stages computer simulations in which the proposed structure is used to identify actual acoustic echo path impulse responses show that the legendre adf has better convergence performance than the transversal adf when identifying systems with long impulse response

frequency domain analysis of tracking and noise performance of adaptive algorithms in this paper an analysis of the tracking and noise performance of several adaptive algorithms is carried out for the case of model structures with fixed pole positions such structures have previously been proposed as an efficient generalization of the common fir model structure the focus of this work is to analyze the associated tradeoff between noise sensitivity and tracking ability in the frequency domain by illustrating how it is influenced by such things as input and noise spectral densities step size and in particular the choice of the fixed pole locations the latter influence is not described by preexisting analysis but is shown here to be amenable to attack by a particular class of orthonormal bases

a strong approximation theorem for positively dependent gaussian sequences and its applications abstract under the polynomial decay rates and finite second moment a strong approximation theorem for partial sums of linear positive quadrant dependent gaussian sequences is derived as applications the law of the iterated logarithm the chung type law of the iterated logarithm and the almost sure central limit theorem are obtained with minimal conditions

minimal two spheres and the topology of manifolds with positive curvature on totally isotropic two planes there has been much interest among differential geometers in finding relationships between curvature and topology of riemannian manifolds for the most part efforts have been directed towards understanding the implications of positive or negative sectional curvature ricci curvature or scalar curvature but there are other hypotheses on the curvature which also deserve investigation in this article we will consider the topological implications of a new curvature assumption positive curvature on totally isotropic two planes and we will prove via the sacks uhlenbeck theory of minimal two spheres that a compact simply connected riemannian manifold of dimension at least four with positive curvature on totally isotropic two planes is homeomorphic to a sphere as corollaries we will obtain a proof via minimal two spheres for riemannian manifolds of dimension at least four of the berger klingenberg rauch toponogov sphere theorem strengthened to pointwise quarter pinching as well as a proof that every simply connected compact riemannian manifold with positive curvature operators is homeomorphic to a sphere let m be an n dimensional riemannian manifold with tangent space tpm at the point p e m recall that the curvature operator at p is the self adjoint

globally convergent adaptive iir filters based on fixed pole locations a new class of adaptive filters dubbed fixed pole adaptive filters fpaf s is introduced these adaptive filters have infinite impulse responses yet their adaptation exhibits provable global convergence good filter performance with a relatively small number of adapted parameters is permitted by the new filter structure thus reducing the computational burden needed to implement adaptive filters the implementation and computational complexity of the fpaf is described and its modeling capabilities are determined excitation conditions on the filter input are established that guarantee global convergence of a standard set of adaptive algorithms some methods are described for selecting the fixed pole locations based on a priori information regarding the operating environment of the adaptive filter the fpaf is tailored to applications by such a procedure enabling improved performance in examples the fpaf is shown to achieve a smaller minimum mean square error given an equal number of adapted parameters in comparison with adaptive fir filters and adaptive filters based on laguerre and kautz models

system identification using laguerre models the traditional approach of expanding transfer functions and noise models in the delay operator to obtain linear in the parameters predictor models leads to approximations of very high order in cases of rapid sampling and or dispersion in time constants by using prior information about the time constants of the system more appropriate expansions related to laguerre networks are introduced and analyzed it is shown that the model order can be reduced compared to arx fir ar modeling by using laguerre models furthermore the numerical accuracy of the corresponding linear regression estimation problem is improved by a suitable choice of the laguerre parameter consistency error bounds persistence of excitation conditions and asymptotic statistical properties are investigated this analysis is based on the result that the covariance matrix of the regression vector of a laguerre model has a toeplitz structure

the law of the iterated logarithm for positively dependent random variables the levy s type maximal inequality is a key to establish the law of the iterated logarithm for associated random variables unfortunately this type inequality cannot be obtained for a generalization of association i e linear positive quadrant dependence because of their special dependence structure the purpose of this paper is to provide a different approach to obtain a law of the iterated logarithm for a sequence of linear positive quadrant dependent random variables

exact rates in log law for positively associated random variables abstract let x n n 1 be a strictly stationary sequence of positively associated random variables with mean zero and finite variance set s n k 1 n x k m n max k n s k n 1 suppose 2 e x 1 2 2 k 2 e x 1 x k in this paper we study the exact convergence rates of a kind of weighted infinite series of p m n e n log n p s n e n log n and i s n e n log n as e 0 respectively

convolution equivalence and distributions of random sums a serious gap in the proof of pakes s paper on the convolution equivalence of infinitely divisible distributions on the line is completely closed it completes the real analytic approach to sgibnev s theorem then the convolution equivalence of random sums of iid random variables is discussed some of the results are applied to random walks and levy processes in particular results of bertoin and doney and of korshunov on the distribution tail of the supremum of a random walk are improved finally an extension of rogozin s theorem is proved

a max norm constrained minimization approach to 1 bit matrix completion we consider in this paper the problem of noisy 1 bit matrix completion under a general non uniform sampling distribution using the max norm as a convex relaxation for the rank a max norm constrained maximum likelihood estimate is introduced and studied the rate of convergence for the estimate is obtained information theoretical methods are used to establish a minimax lower bound under the general sampling model the minimax upper and lower bounds together yield the optimal rate of convergence for the frobenius norm loss computational algorithms and numerical performance are also discussed

a unifying construction of orthonormal bases for system identification this paper develops a general and very simple construction for complete orthonormal bases for system identification this construction provides a unifying formulation of many previously studied orthonormal bases since the common fir and recently popular laguerre and two parameter kautz model structures are restrictive special cases of the construction presented here however in contrast to these special cases the basis vectors in the unifying construction of this paper can have arbitrary placement of pole position according to the prior information the user wishes to inject results characterizing the completeness of the bases and the accuracy properties of models estimated using the bases are provided

frequency domain tracking characteristics of adaptive algorithms the problem of tracking time varying linear systems is discussed the focus is on the model quality in terms of the mean square error mse between the true momentary transfer function and the estimated one this mse is thus a function of frequency the exact expression for the mse is complicated but simple expressions that are asymptotic in the model order are developed for model structures of finite impulse response fir character simulations verify that these simple expressions are quite reliable and insightful even for moderate model orders expressions are developed for three basic adaptation algorithms recursive identification algorithms viz the least mean squares algorithm the recursive least squares algorithm with exponential forgetting and a tracking algorithm based on the kalman filter the results apply both to slowly time varying systems and to the model recovery after an abrupt change in the system dynamics

performance analysis of general tracking algorithms a general family of tracking algorithms for linear regression models is studied it includes the familiar least mean square gradient approach recursive least squares and kalman filter based estimators the exact expressions for the quality of the obtained estimates are complicated approximate and easy to use expressions for the covariance matrix of the parameter tracking error are developed these are applicable over the whole time interval including the transient and the approximation error can be explicitly calculated

brief new identification approaches for disturbed models recently much research has been conducted in the field of identification of the linear models in general these methods use a time domain estimate or a frequency domain estimate in this paper the time domain estimate and the frequency domain estimate were combined to identify the autoregressive exogenous noise arx interference model the concept of a general prediction error criterion is introduced for the time domain estimate an optimal frequency estimation is introduced for the frequency domain estimate a new identification method called the empirical frequency domain optimal parameter estimate is proposed for disturbed systems it is fully applied and developed for the output error model and a specific case or the arx model the algorithm theoretically provides the globally optimum frequency domain estimate of the model some simulations are included to illustrate the new identification method

normal approximation for associated point processes via stein s method with applications to determinantal point processes abstract we use stein s method to provide non asymptotic l 1 bounds also known as wasserstein bounds to the normal for functionals of associated point processes as for supporting tools we use the connection between association and mixing properties that was recently uncovered by 29 we apply our main results to determinantal point processes which are known to be negatively associated a potential application to determinantal point processes in the laguerre gaussian family is also presented

an almost sure central limit theorem for products of sums of partial sums under association abstract let x n n 1 be a strictly stationary positively or negatively associated sequence of positive random variables with e x 1 0 and var x 1 2 denote s n i 1 n x i t n i 1 n s i and the coefficient of variation under suitable conditions we show that x lim n 1 log n k 1 n 1 k i 2 k j 1 k t j k k 1 k 1 1 k x f x a s where 1 2 1 2 2 j 2 cov x 1 x j f is the distribution function of the random variables e 10 3 n and n is a standard normal random variable

asymptotic properties of black box identification of transfer functions the problem of estimating the transfer function of a linear stochastic system is considered the transfer function is parametrized as a black box and no given order is chosen a priori this means that the model orders may increase to infinity when the number of observed data tends to infinity the consistency and convergence properties of the resulting transfer function estimates are investigated asymptotic expressions for the variances and distributions of these estimates are also derived for the case that the model orders increase it is shown that the variance of the transfer function estimate at a certain frequency is asymptotically given by the noise to signal ratio at that frequency mulliplied by the model order to number of data points ratio