distribution estimation consistent in total variation and in two types of information divergence the problem of the nonparametric estimation of a probability distribution is considered from three viewpoints the consistency in total variation the consistency in information divergence and consistency in reversed order information divergence these types of consistencies are relatively strong criteria of convergence and a probability distribution cannot be consistently estimated in either type of convergence without any restrictions on the class of probability distributions allowed histogram based estimators of distribution are presented which under certain conditions converge in total variation in information divergence and in reversed order information divergence to the unknown probability distribution some a priori information about the true probability distribution is assumed in each case as the concept of consistency in information divergence is stronger than that of convergence in total variation additional assumptions are imposed in the cases of informational divergences

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

the use of defect correction to refine the eigenelements of compact integral operators we show how the principle of defect correction can be applied to an integral eigenvalue problem we present as particular cases of this principle the two important methods which are known as the two level multigrid and iterative refinement

a modified kolmogorov smirnov test sensitive to tail alternatives it is well known that the kolmogorov smirnov k s test exhibits poor sensitivity to deviations from the hypothesized distribution that occur in the tails a modified version of the k s test is introduced that is more sensitive than the k s test to deviations in the tails the finite and infinite sample distribution along with the consistency properties of the proposed test are studied tables of critical values are provided for two versions of the test one sensitive to heavy tail alternatives and one sensitive to light tail alternatives and the finite sample properties of these two versions of the test are investigated

improving the accuracy of computed eigenvalues and eigenvectors this paper describes a computational method for improving the accuracy of a given eigenvalue and its associated eigenvector the method is analogous to iterative improvement for the solution of linear systems an iterative algorithm using working precision arithmetic is applied to increase the accuracy of the eigenpair the only extended precision computation is the residual calculation the method is related to inverse iteration and to newton s method applied to the eigenvalue problem

on the relation between upper central quotients and lower central series of a group let h be a group with a normal subgroup n contained in the upper central subgroup zch in this article we study the influence of the quotient group g h n on the lower central subgroup yc lh in particular for any finite group g we give bounds on the order and exponent of yc l h for g equal to a dihedral group or quaternion group or extra special group we list all possible groups that can arise as yc lh our proofs involve i the baer invariants of g ii the schur multiplier m l g of g relative to a normal subgroup l and iii the nonabelian tensor product of groups some results on the nonabelian tensor product may be of independent interest

asymptotic optimality of data driven neyman s tests for uniformity data driven neyman s tests resulting from a combination of neyman s smooth tests for uniformity and schwarz s selection procedure are investigated asymptotic intermediate efficiency of those tests with respect to the neyman pearson test is shown to be 1 for a large set of converging alternatives the result shows that data driven neyman s tests contrary to classical goodness of fit tests are indeed omnibus tests adapting well to the data at hand

components of pearson s phi squared distance measure for the k sample problem abstract a general approach to the k sample problem for continuous random variables based on a generalization of pearson s classical 2 measure is presented tests are developed by considering an analysis of variance type of orthogonal decomposition of a distance measure for a k population goodness to fit hypothesis estimates of some of these components are shown to provide test statistics for the hypothesis that the k populations have a common distribution several classical parametric and nonparametric tests are found to be quadratic functions of these component estimates in addition a new class of test statistic is derived that possesses aspects of both parametric and nonparametric methods

cramer type large deviations for linear combinations of order statistics let t n 1 1 cnx be a linear combination of order statistics and put tn t e t vvar tn sufficient conditions on the cn and on the moments of the underlying distribution are established under which the ratio p t x 1 d x tends to 1 either uniformly in the range a c x c cavihh a 2 0 c 0 moderate deviation theorem or uniformly in the range a c x c o n a 2 0 cramer type large deviation theorem the proof relies on helmers approximation method and on the corresponding results for u statistics

refinement methods of newton type for approximate eigenelements of integral operators we present in the context of newton type methods six iterative techniques to refine approximate eigenelements of compact linear operators defined on a complex banach space these methods are compared among themselves both theoretically and numerically computational experiments are reported

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

testing goodness of fit based on a roughness measure abstract a test for the one sample goodness of fit problem is proposed the test is based on a distance that measures the difference in terms of roughness between the underlying density function and the hypothesized density function one advantage of using a roughness measure is high power in detecting high frequency alternatives and densities with sharp features the test statistic that estimates the distance is derived from the viewpoint of kernel density estimation and a testing procedure is developed based on the asymptotic distribution of the test statistic the proposed test is compared to the kolmogorov smirnov test

maximum entropy and conditional probability it is well known that maximum entropy distributions subject to appropriate moment constraints arise in physics and mathematics in an attempt to find a physical reason for the appearance of maximum entropy distributions the following theorem is offered the conditional distribution of x l given the empirical observation 1 n sum n i l h x i alpha where x 1 x 2 cdots are independent identically distributed random variables with common density g converges to f lambda x e lambda t h x g x suitably normalized where lambda is chosen to satisfy int f lambda x h x dx alpha thus the conditional distribution of a given random variable x is the normalized product of the maximum entropy distribution and the initial distribution this distribution is the maximum entropy distribution when g is uniform the proof of this and related results relies heavily on the work of zabell and lanford

gmm estimation of a maximum entropy distribution with interval data we develop a generalized method of moments gmm estimator for the distribution of a variable where summary statistics are available only for intervals of the random variable without individual data one cannot calculate the weighting matrix for the gmm estimator instead we propose a simulated weighting matrix based on a first step consistent estimate when the functional form of the underlying distribution is unknown we estimate it using a simple yet flexible maximum entropy density our monte carlo simulations show that the proposed maximum entropy density is able to approximate various distributions extremely well the two step gmm estimator with a simulated weighting matrix improves the efficiency of the one step gmm considerably we use this method to estimate the u s income distribution and compare these results with those based on the underlying raw income data

data driven version of neyman s smooth test of fit abstract neyman s smooth test for testing uniformity is a recognized goodness of fit procedure as stated by lariccia the test can be viewed as a compromise between omnibus test procedures with generally low power in all directions and procedures whose power is focused in the direction of a specific alternative the basic idea behind this test is to embed the null density into say a k dimensional exponential family and then to construct an asymptotically optimal test for the parametric testing problem the resulting procedure is neyman s test with k components the most difficult problem related with using this test is the choice of k recommendations in statistical literature are sometimes confusing some authors advocate a small number of components whereas others show that in some situations a larger number of components is profitable all existing suggestions concerning how to select k exploit in fact some preliminary knowledge about a possible alternative in this article a new data driven met

joint fixed rate universal lossy coding and identification of continuous alphabetmemoryless sources the problem of joint universal source coding and density estimation is considered in the setting of fixed rate lossy coding of continuous alphabet memoryless sources for a wide class of bounded distortion measures it is shown that any compactly parametrized family of rd valued independent and identically distributed i i d sources with absolutely continuous distributions satisfying appropriate smoothness and vapnik chervonenkis vc learnability conditions admits a joint scheme for universal lossy block coding and parameter estimation such that when the block length n tends to infinity the overhead per letter rate and the distortion redundancies converge to zero as o n 1 log n and o radicn 1 log n respectively moreover the active source can be determined at the decoder up to a ball of radius o radicn 1 log n in variational distance asymptotically almost surely the system has finite memory length equal to the block length and can be thought of as blockwise application of a time invariant nonlinear filter with initial conditions determined from the previous block comparisons are presented with several existing schemes for universal vector quantization which do not include parameter estimation explicitly and an extension to unbounded distortion measures is outlined finally finite mixture classes and exponential families are given as explicit examples of parametric sources admitting joint universal compression and modeling schemes of the kind studied here

exponential series estimator of multivariate densities abstract we present an exponential series estimator ese of multivariate densities which has an appealing information theoretic interpretation for a d dimensional random variable x with density p 0 the ese takes the form p x exp i 1 0 m 1 i d 0 m d i i x where i are some real valued linearly independent functions defined on the support of p 0 we derive the convergence rate of the ese in terms of the kullback leibler information criterion the integrated squared error and some other metrics we also derive its almost sure uniform convergence rate we then establish the asymptotic normality of p we undertake two sets of monte carlo experiments the first experiment examines the ese performance using mixtures of multivariate normal densities the second estimates copula density functions the results demonstrate the efficacy of the ese an empirical application on the joint distributions of stock returns is presented

on pro p groups with potent filtrations abstract in this note we prove that if g is pf group of finite exponent then the exponent of the second homology group h 2 g m divides the exponent of g for every profinite trivial z g module m we introduce the notion of the exponential rank of a pro p group and find a bound for the exponential rank of a pf group

spectral refinements of newton type for eigenelements of bounded linear operators a theoretical framework is developed for constructing spectral refinement schemes for a simple eigenelement which achieve arbitrarily high rates of convergence while keeping the computational cost at a minimum the new approach is illustrated by considering a newton type iteration scheme numerical results are given by considering a model problem

nonparametric likelihood ratio model selection tests between parametric likelihood and moment condition models we propose a nonparametric likelihood ratio testing procedure for choosing between a parametric likelihood model and a moment condition model when both models could be misspecified our procedure is based on comparing the kullback leibler information criterion klic between the parametric model and moment condition model we construct the klic for the parametric model using the difference between the parametric log likelihood and a sieve nonparametric estimate of population entropy and obtain the klic for the moment model using the empirical likelihood statistic we also consider multiple 2 model comparison tests when all the competing models could be misspecified and some models are parametric while others are moment based we evaluate the performance of our tests in a monte carlo study and apply the tests to an example from industrial organization

convergence rates for bayesian density estimation of infinite dimensional exponential families we study the rate of convergence of posterior distributions in density estimation problems for log densities in periodic sobolev classes characterized by a smoothness parameter p the posterior expected density provides a nonparametric estimation procedure attaining the optimal minimax rate of convergence under hellinger loss if the posterior distribution achieves the optimal rate over certain uniformity classes a prior on the density class of interest is induced by a prior on the coefficients of the trigonometric series expansion of the log density we show that when p is known the posterior distribution of a gaussian prior achieves the optimal rate provided the prior variances die off sufficiently rapidly for a mixture of normal distributions the mixing weights on the dimension of the exponential family are assumed to be bounded below by an exponentially decreasing sequence to avoid the use of infinite bases we develop priors that cut off the series at a sample size dependent truncation point when the degree of smoothness is unknown a finite mixture of normal priors indexed by the smoothness parameter which is also assigned a prior produces the best rate a rate adaptive estimator is derived

adaptive estimation of the intensity of inhomogeneous poisson processes via concentration inequalities in this paper we establish oracle inequalities for penalized projection estimators of the intensity of an inhomogeneous poisson process we study consequently the adaptive properties of penalized projection estimators at first we provide lower bounds for the minimax risk over various sets of smoothness for the intensity and then we prove that our estimators achieve these lower bounds up to some constants the crucial tools to obtain the oracle inequalities are new concentration inequalities for suprema of integral functionals of poisson processes which are analogous to talagrand s inequalities for empirical processes

schur multipliers and power endomorphisms of groups abstract we obtain new bounds for the exponent of the schur multiplier of a given p group we prove that the exponent of the schur multiplier can be bounded by a function depending only on the exponent of a given group as a consequence we show that the exponent of the schur multiplier of any group of exponent four divides eight and that this bound is best possible the notion of the exponential rank of a p group is introduced we show that powerful p groups have exponential rank either zero or one

asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity consider a sample of size n from a regular exponential family in p n dimensions let hat theta n denote the maximum likelihood estimator and consider the case where p n tends to infinity with n and where theta n is a sequence of parameter values in r p n moment conditions are provided under which hat theta n theta n o p sqrt p n n and hat theta n theta n overline x n o p p n n where overline x n is the sample mean the latter result provides normal approximation results when p 2 n n rightarrow 0 it is shown by example that even for a single coordinate of hat theta n theta n p 2 n n rightarrow 0 may be needed for normal approximation however if p 3 2 n n rightarrow 0 the likelihood ratio test statistic lambda for a simple hypothesis has a chi square approximation in the sense that 2 log lambda p n sqrt 2p n rightarrow d mathscr n 0 1

the unitary cover of a finite group and the exponent of the schur multiplier abstract for a finite group we introduce a particular central extension the unitary cover having minimal exponent among those satisfying the projective lifting property we obtain new bounds for the exponent of the schur multiplier relating to subnormal series and we discover new families for which the bound is the exponent of the group finally we show that unitary covers are controlled by the zelmanov solution of the restricted burnside problem for 2 generator groups

consistency and monte carlo simulation of a data driven version of smooth goodness of fit tests the data driven method of selecting the number of components in neyman s smooth test for uniformity introduced by ledwina is extended the resulting tests consist of a combination of schwarz s bayesian information criterion bic procedure and smooth tests the upper bound of the dimension of the exponential families in applying schwarz s rule is allowed to grow with the number of observations to infinity simulation results show that the data driven version of neyman s test performs very well for a wide range of alternatives and is competitive with other recently introduced data driven procedures it is shown that the data driven smooth tests are consistent against essentially all alternatives in proving consistency new results on schwarz s selection rule are derived which may be of independent interest

data driven smooth tests when the hypothesis is composite in recent years several authors have recommended smooth tests for testing goodness of fit however the number of components in the smooth test statistic should be chosen well otherwise considerable loss of power may occur schwarz s selection rule provides one such good choice earlier results on simple null hypotheses are extended here to composite hypotheses which tend to be of more practical interest for general composite hypotheses consistency of the data driven smooth tests holds at essentially any alternative monte carlo experiments on testing exponentiality and normality show that the data driven version of neyman s test compares well to other even specialized tests over a wide range of alternatives

data driven smooth tests for composite hypotheses the classical problem of testing goodness of fit of a parametric family is reconsidered a new test for this problem is proposed and investigated the new test statistic is a combination of the smooth test statistic and schwarz s selection rule more precisely as the sample size increases an increasing family of exponential models describing departures from the null model is introduced and schwarz s selection rule is presented to select among them schwarz s rule provides the right dimension given by the data while the smooth test in the right dimension finishes the job theoretical properties of the selection rules are derived under null and alternative hypotheses they imply consistency of data driven smooth tests for composite hypotheses at essentially any alternative

a test for asymmetry associated with the hodges lehmann estimator abstract one version of the hodges lehmann estimator median xi xj 2 1 i j n is also the minimum distance estimator derived from a particular cramer von mises distance this distance evaluated at that is the minimized distance provides a natural statistic for testing asymmetry of the underlying distribution

testing goodness of fit with multinomial data abstract several new test procedures are proposed for assessing the goodness of fit of a postulated multinomial distribution the new tests are neyman smooth type tests with orders selected adaptively from the data they are shown through fourier and large sample analyses to provide potential improvements over classical methods in terms of their ability to detect certain types of alternatives simulation results and a real example illustrate the finite sample validity of the large sample theory and the practical utility of the proposed methods

optimal calibration for multiple testing against local inhomogeneity in higher dimension based on two independent samples x 1 x m and x m 1 x n drawn from multivariate distributions with unknown lebesgue densities p and q respectively we propose an exact multiple test in order to identify simultaneously regions of significant deviations between p and q the construction is built from randomized nearest neighbor statistics it does not require any preliminary information about the multivariate densities such as compact support strict positivity or smoothness and shape properties the properly adjusted multiple testing procedure is shown to be sharp optimal for typical arrangements of the observation values which appear with probability close to one the proof relies on a new coupling bernstein type exponential inequality reflecting the non subgaussian tail behavior of a combinatorial process for power investigation of the proposed method a reparametrized minimax set up is introduced reducing the composite hypothesis p q to a simple one with the multivariate mixed density m n p 1 m n q as infinite dimensional nuisance parameter within this framework the test is shown to be spatially and sharply asymptotically adaptive with respect to uniform loss on isotropic holder classes the exact minimax risk asymptotics are obtained in terms of solutions of the optimal recovery

cramer type large deviations for simple linear rank statistics a cramer type large deviation theorem for simple linear rank statistics is obtained range 0 x 0 n 1 mathord left vphantom 1 6 right kern nulldelimiterspace 6 the method of proof consists in approximating the simple linear rank statistic by a sum of independent uniformly bounded random variables and then applying a cramer type large deviation theorem on this sum

nonparametric density estimation the l 1 view differentiation of integrals consistency lower bounds for rates of convergence rates of convergence in l1 the automatic kernel estimate l1 and pointwise convergence estimates related to the kernel estimate and the histogram estimate simulation inequalities and random variate generation the transformed kernel estimate applications in discrimination operations on density estimates estimators based on orthogonal series index

local asymptotics for linear rank statistics with estimated score functions on etudie des versions modifiees de statistiques de rang lineaires s n avec une fonction de score pour le probleme de test de l aleatoire a deux echantillons

a simple multivariate test for one sided alternatives abstract a simple test for multivariate normal data is proposed that has good power for alternatives where the mean vector tends to be positive the test rejects if the quadratic form of the sample mean vector exceeds its 2 critical value and the sum of the elements of the mean vector exceeds zero the proposed test is shown to have type i error rate equal to whether or not the covariance matrix is known tight bounds on the power of the proposed test are provided and the test is compared to three likelihood ratio tests

efficient and adaptive nonparametric test for the two sample problem the notion of efficient test for a euclidean parameter in a semiparametric model was introduced by stein proc third berkeley symp math statist probab 1 1956 187 195 such tests are locally most powerful for a wide class of infinite dimensional nuisance parameters the first formal application of this notion to a suitably parametrized two sample problem was provided by hajek ann math statist 33 1962 1124 1147 however this and subsequent solutions appear to be not well suited for practical applications this article aims to show that an adaptive two sample test introduced recently by janic wroblewska and ledwina scand j statist 27 2000 281 297 is locally most powerful under a more realistic setting

two modified wilcoxon tests for symmetry about an unknown location parameter summary two modifications of the wilcoxon test for symmetry about the sample median are proposed and their large sample distribution theory derived their pitman efficacies against contamination in the right tail are obtained both the asymptotic results and a monte carlo study show that the procedures are more powerful than the wilcoxon test about the median against many asymmetric alternatives while preserving its robustness of level shown by antille kersting 1977

a class of rank test procedures for censored survival data summary a class of linear rank statistics is proposed for the k sample problem with rightcensored survival data the class contains as special cases the log rank test mantel 1966 cox 1972 and a test essentially equivalent to peto peto s 1972 generalization of the wilcoxon test martingale theory is used to establish asymptotic normality of test statistics under the null hypotheses considered and to derive expressions for asymptotic relative efficiencies under contiguous sequences of alternative hypotheses a class of distributions is presented which corresponds to the class of rank statistics in the sense that for each distribution there is a statistic with some optimal properties for detecting location alternatives from that distribution some monte carlo results are displayed which present small sample behaviour

test of significance based on wavelet thresholding and neyman s truncation abstract traditional nonparametric tests such as the kolmogorov smirnov test and the cramer von mises test are based on the empirical distribution functions although these tests possess root n consistency they effectively use only information contained in the low frequencies this leads to low power in detecting fine features such as sharp and short aberrants as well as global features such as high frequency alternations the drawback can be repaired via smoothing based test statistics in this article we propose two such kind of test statistics based on the wavelet thresholding and the neyman truncation we provide extensive evidence to demonstrate that the proposed tests have higher power in detecting sharp peaks and high frequency alternations while maintaining the same capability in detecting smooth alternative densities as the traditional tests similar conclusions can be made for two sample nonparametric tests of distribution functions in that case the traditional linear rank tests such as th

distribution free tests of independence in high dimensions we consider the testing of mutual independence among all entries in a formula see text dimensional random vector based on formula see text independent observations we study two families of distribution free test statistics which include kendall s tau and spearman s rho as important examples we show that under the null hypothesis the test statistics of these two families converge weakly to gumbel distributions and we propose tests that control the type i error in the high dimensional setting where formula see text we further show that the two tests are rate optimal in terms of power against sparse alternatives and that they outperform competitors in simulations especially when formula see text is large

sieve maximum likelihood estimation of the spatial autoregressive tobit model abstract this paper extends the ml estimation of a spatial autoregressive tobit model under normal disturbances in xu and lee 2015b journal of econometrics to distribution free estimation we examine the sieve mle of the model where the disturbances are i i d with an unknown distribution we show that the spatial autoregressive process with tobit censoring and related variables are spatial near epoch dependent ned a related contribution is that we develop some exponential inequalities for spatial ned random fields with these inequalities we establish the consistency of the estimator asymptotic distributions of structural parameters of the model are derived from a functional central limit theorem and projection simulations show that the sieve mle can improve the finite sample performance upon misspecified normal mles as an empirical application we examine the school district income surtax rates in iowa our results show that the spatial spillover effects are significant but they may be overestimated if disturbances are restricted to be normally distributed

approximation theoretic derivation of logarithmic entropy principles for inverse problems and unique extension of the maximum entropy method to incorporate prior knowledge the minimum distance approach for reconstructing a positive function based on knowledge of finitely many linear functional values is examined two important classes of directed distances for signal processing and statistical inference are discussed by imposing conditions analogous to those satisfied by linear projections in hilbert space two logarithmic entropy principles are derived one of these involves the itakura saito distortion measure of communication theory and uniquely extends burg s maximum entropy method to incorporate prior knowledge the other uses the kullback leibler distance of statistics

catching up faster by switching sooner a predictive approach to adaptive estimation with an application to the aic bic dilemma summary prediction and estimation based on bayesian model selection and model averaging and derived methods such as the bayesian information criterion bic do not always converge at the fastest possible rate we identify the catch up phenomenon as a novel explanation for the slow convergence of bayesian methods which inspires a modification of the bayesian predictive distribution called the switch distribution when used as an adaptive estimator the switch distribution does achieve optimal cumulative risk convergence rates in non parametric density estimation and gaussian regression problems we show that the minimax cumulative risk is obtained under very weak conditions and without knowledge of the underlying degree of smoothness unlike other adaptive model selection procedures such as the akaike information criterion aic and leave one out cross validation bic and bayes factor model selection are typically statistically consistent we show that this property is retained by the switch distribution which thus solves the aic bic dilemma for cumulative risk the switch distribution has an efficient implementation we compare its performance with aic bic and bayesian model selection and averaging on a regression problem with simulated data

on the choice of a model to fit data from an exponential family let x 1 cdots x n be iid observations coming from an exponential family the problem of interest is this given a finite number of models m j smoothly curved manifolds in mathbb r k choose the best model to fit the observations with some penalty for choosing models with dimensions which are too large a result of schwarz is made more specific and is extended to the case where the models are curved manifolds if s y n j is up to a constant c n independent of the model the log of the posterior probability of the j th model where the sample mean y n 1 n sum n i 1 x i has been replaced by y schwarz suggested an asymptotic expansion of s y n j whose leading terms are gamma y n j n sup psi in m j cap theta y psi b psi frac 1 2 k j log n in the case where the models are affine subspaces of mathbb r k we establish a similar asymptotic expansion including the next term with uniform bounds for y in a compact neighborhood of nabla b theta where theta is the true value of the parameter we suggest a criterion for the choice of the best model that consists of maximizing the three leading terms in the expansion s y n j we show that the criterion gives the correct model with probabilities p n theta rightarrow 1 as n rightarrow infty

approximation of density functions by sequences of exponential families 1 introduction consider the estimation of a probability density func tion p x defined on a bounded interval we approximate the logarithm of the density by a basis function expansion consisting of polynomials splines or trigonometric series the expansion yields a regular exponential family within which we estimate the density by the method of maximum likelihood this method of density estimation arises by application of the principle of maxi mum entropy or minimum relative entropy subject to empirical constraints we show that if the logarithm of the density has r square integrable deriva tives f idr log p12 ox then the sequence of density estimators p converges to p in the sense of relative entropy kullback leibler fp log p pn at rate opr 1 m2r m n as m x c and m2 n 0 in the spline and trigonometric cases and m3 n o 0 in the polynomial case where m is the dimension of the family and n is the sample size boundary conditions are assumed for the density in the trigonometric case this convergence rate specializes to op r n 2r 2r 1 by setting m nl 2r 1 when the log density is known to have degree of smoothness at least r analogous convergence results for the relative entropy are shown to hold in general for any class of log density functions and sequence of finite dimensional linear spaces having l2 and l approximation properties the approximation of log densities using polynomials has previously been considered by neyman 1937 to define alternatives for goodness of fit tests by good 1963 as an application of the method of maximum entropy or minimum relative entropy by crain 1974 1976a b 1977 who demonstrates existence and consistency of the maximum likelihood estimator and by mead and

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

maximum entropy in the problem of moments the maximum entropy approach to the solution of underdetermined inverse problems is studied in detail in the context of the classical moment problem in important special cases such as the hausdorff moment problem we establish necessary and sufficient conditions for the existence of a maximum entropy solution and examine the convergence of the resulting sequence of approximations a number of explicit illustrations are presented in addition to some elementary examples we analyze the maximum entropy reconstruction of the density of states in harmonic solids and of dynamic correlation functions in quantum spin systems we also briefly indicate possible applications to the lee yang theory of ising models to the summation of divergent series and so on the general conclusion is that maximum entropy provides a valuable approximation scheme a serious competitor of traditional pade like procedures

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

on the stability of singular finite rank methods the solution of the operator equation lambda bf mathcal k x y where bf mathcal k is a bounded linear operator of finite rank on a banach space x and lambda is a nonzero scalar can be reduced to the solution of a system of linear equations lambda bf tilde mathcal k mathcal k alpha beta where bf tilde mathcal k is a matrix a recent result of whitley for the case where lambda is in the resolvent set of bf mathcal k is extended to include the case where lambda is a nonzero spectral value of bf mathcal k if lambda is a nonzero semisimple eigenvalue of bf mathcal k then the stability of a particular solution of the linear system is related to the stability of a particular solution of the operator equation bounds for the condition number of an equivalent nonsingular linear system are given these results are applied to the implementation of some iterative refinement schemes for approximating a nonzero simple eigenvalue o

data driven rank tests for independence we introduce new rank tests for testing independence the new testing procedures are sensitive not only for grade linear correlation but also for grade correlations of higher order polynomials the number of polynomials involved is determined by the data model selection is combined with application of the score test in the selected model whereas well known tests as spearman s test or hoeffding s test may completely break down for alternatives that are dependent but have low grade linear correlation the new tests have greater power stability monte carlo results clearly show this behavior theoretical support is obtained by proving consistency of the new tests

combined rank tests for the two sample problem with randomly censored data abstract in the two sample problem under random censorship we consider the uncensored observations and the censored ones as two separate groups for each group a suitable rank statistic is obtained and these two are then combined to a final one this idea which was first employed in akritas 1983 is shown here to produce tests that a closely resemble the ordinary rank tests for the uncensored case and do not require the calculation of the kaplan meier estimator b are comparatively easy to apply and to understand and c allow results on asymptotic normality to follow simply from standard results for the uncensored case it is shown that the loss due to using two separate rankings rather than one complete ranking is asymptotically negligible the optimal score function for each of the two separate rank statistics is seen to depend on the censoring distribution whereas in akritas 1983 no assumption on the form of the censoring distribution was made unrestricted adaptation here we pursue rest

accelerated spectral refinement with applications to integral operators accelerated refinement procedures are developed for the resolution of nearly degenerate simple spectra associated with large discrete linear systems error expressions are obtained the spectral analysis of integral operators is discussed and a numerical example of resolving an ill conditioned spectrum is presented

neyman type smooth tests for location scale families summary the smooth tests of neyman 1937 are modified to permit composite alternatives and a choice of orthonormal functions in their construction these orthonormal functions may be chosen to improve detection of particular alternatives the tests are easy to apply and have useful orthogonal components a test for exponentiality is examined and shown to perform well against its competitors

large sample inference for log spline models let f be a continuous and positive unknown density on a known compact interval mathscr y let f denote the distribution function of f and let q f 1 denote its quantile function a finite parameter exponential family model based on b splines is constructed maximum likelihood estimation of the parameters of the model based on a random sample of size n from f yields estimates hat f f and hat q of f f and q respectively under mild conditions if the number of parameters tends to infinity in a suitable manner as n rightarrow infty these estimates achieve the optimal rate of convergence the asymptotic behavior of the corresponding confidence bounds is also investigated in particular it is shown that the standard errors of hat f and hat q are asymptotically equal to those of the usual empirical distribution function and empirical quantile function

a bound on the financial value of information it is shown that each bit of information at most doubles the resulting wealth in the general stock market setup this information bound on the growth of wealth is actually attained for certain probability distributions on the market investigated by j kelly 1956 the bound is shown to be a special case of the result that the increase in exponential growth of wealth achieved with true knowledge of the stock market distribution f over that achieved with incorrect knowledge g is bounded above by the entropy of f relative to g

a geometric interpretation of inferences based on ranks in the linear model abstract four different approaches based on ranks to testing hypotheses are unified through the geometry of the linear model the various tests are identified with different but algebraically equivalent forms of the classical f test small sample differences are investigated via a monte carlo study using both rank and signed rank tests

data driven rank tests for classes of tail alternatives tail alternatives describe the frequent occurrence of a non constant shift in the two sample problem with a shift function increasing in the tail the classes of shift functions can be built up using legendre polynomials it is important to rightly choose the number of polynomials involved here this choice is based on the data using a modification of schwarz s selection rule given the data driven choice of the model appropriate rank tests are applied simulations show that the new data driven rank tests work very well while other tests for detecting shift alternatives as wilcoxon s test may completely break down for important classes of tail alternatives the new tests have high and stable power the new tests have also higher power than data driven rank tests for the unconstrained two sample problem theoretical support is obtained by proving consistency of the new tests against very large classes of alternatives including all common tail alternatives a simple but accurate approximation of the null distribution makes application of the new tests easy

vanishing shortcoming and asymptotic relative efficiency the shortcoming of a test is the difference between the maximal attainable power and the power of the test under consideration vanishing shortcoming when the number of observations tends to infinity is therefore an optimality property of a test other familiar optimality criteria are based on the asymptotic relative efficiency of the test the relations between these optimality criteria are investigated it turns out that vanishing shortcoming is seemingly slightly stronger than first order efficiency but in regular cases there is equivalence the results are in particular applied on tests for goodness of fit

nonparametric maximum likelihood estimation by the method of sieves maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space for example the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an operatorname iid sample the maximum of the likelihood is not attained by any density in this example as in many other examples the parameter space positive functions with area one is too big but the likelihood method can often be salvaged if we first maximize over a constrained subspace of the parameter space and then relax the constraint as the sample size grows this is grenander s method of sieves application of the method sometimes leads to new estimators for familiar problems or to a new motivation for an already well studied technique we will establish some general consistency results for the method and then we will focus on three applications

galton s test as a linear rank test with estimated scores and its local asymptotic efficiency based on rank estimators of b we consider an easy but rough and inconsistent rank estimator of b the resulting rank test turns out to be a generalization of galton s test a formula for local asymptotic power under arbitrary local alternatives is derived which allows for comparison of galton s test with every linear rank test for various types of alternatives the galton test is compared with the optimal linear rank test and with the wilcoxon test in order to get an impression of the validity of extrapolation to finite sample sizes we included a monte carlo study under the same types of fixed alternatives

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

testing for additivity in nonparametric regression additive models are one means of assuaging the curse of dimensionality when nonparametric smoothing methods are used to estimate multivariable regression functions it is important to have methods for testing the fit of such models especially in high dimensions where visual assessment of fit becomes difficult new tests of additivity are proposed in this paper that derive from fourier series estimators with data driven smoothing parameters other tests related to the classical tukey test for additivity are also considered while the new tests are consistent against essentially any smooth alternative to additivity the tukey type tests are found to be inconsistent in certain situations asymptotic power of both varieties of tests is studied under local alternatives that tend toward additivity at a parametric rate and small sample power comparisons are carried out by means of a simulation study

test statistics derived as components of pearson s phi squared distance measure abstract a general approach to the comparison of two absolutely continuous distributions is presented based on pearson s 2 measure although no simple estimates are available for 2 it can be decomposed into components all of which are easily estimated asymptotic distribution theory is derived for the component estimates and the form of the components is considered for several standard problems it is demonstrated that the components of 2 approach provides a unified approach to the development of tests for a variety of problems and that it is useful for generating new test procedures the components of 2 approach is based on what is termed a comparison density that is given two absolutely continuous distributions f and h with densities f and h the comparison density is defined as d u f h 1 u h h 1 u where h 1 is the h quantile function since d must be uniform under the hypothesis f h an overall measure of the validity of the hypothesis is provided by 2 1 0 d u 1 2 du by u

an omnibus consistent adaptive percentile modified wilcoxon rank sum test with applications in gene expression studies we present an adaptive percentile modified wilcoxon rank sum test for the two sample problem the test is basically a wilcoxon rank sum test applied on a fraction of the sample observations and the fraction is adaptively determined by the sample observations most of the theory is developed under a location shift model but we demonstrate that the test is also meaningful for testing against more general alternatives the test may be particularly useful for the analysis of massive datasets in which quasi automatic hypothesis testing is required we investigate the power characteristics of the new test in a simulation study and we apply the test to a microarray experiment on colorectal cancer these empirical studies demonstrate that the new test has good overall power and that it succeeds better in finding differentially expressed genes as compared to other popular tests we conclude that the new nonparametric test is widely applicable and that its power is comparable to the power of the baumgartner weis schindler test

data driven k sample tests a new solution of the general nonparametric k sample problem for independent continuous random variables is developed our approach to solving the nonparametric problem is via introducing a net of semiparametric models solving the testing problem for members of this approximating net and then combining the resulting statistics via model selection rules which mimic the rule proposed recently in inglot and ledwina 2006 we call such a solution a data driven test since we in fact use a class of model selection criteria we also obtain a class of k sample data driven tests the class is flexible and allows us to construct omnibus as well as more specialized solutions simulations show that the new omnibus test has power comparable to existing k sample tests in case of changes of location or scale and is more powerful for much more complex changes a variant of the new solution focused on detecting high frequency alternations is also briefly discussed some theoretical support is given this article is a