Empirical distribution function
In statistics, an empirical distribution function (commonly also called an empirical Cumulative Distribution Function, eCDF) is the distribution function associated with the empirical measure of a sample. This cumulative distribution function is a step function that jumps up by at each of the data points. Its value at any specified value of the measured variable is the fraction of observations of the measured variable that are less than or equal to the specified value. The empirical distribution function is an estimate of the cumulative distribution function that generated the points in the sample. It converges with probability  1 to that underlying distribution, according to the Glivenko–Cantelli theorem. A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function. Definition. Let be independent, identically distributed real random variables with the common cumulative distribution function . Then the empirical distribution function is defined as where formula_2 is the indicator of event . For a fixed , the indicator formula_3 is a Bernoulli random variable with parameter ; hence formula_4 is a binomial random variable with mean and variance . This implies that formula_5 is an unbiased estimator for . However, in some textbooks , the definition is given as formula_6 Mean. The mean of the empirical distribution is an unbiased estimator of the mean of the population distribution. formula_7 which is more commonly denoted formula_8 Variance. The variance of the empirical distribution times formula_9 is an unbiased estimator of the variance of the population distribution. formula_10 Mean squared error. The mean squared error for the empirical distribution is as follows. formula_11 Where formula_12 is an estimator and formula_13 an unknown parameter Quantiles. For any real number formula_14 the notation formula_15 (read “ceiling of a”) denotes the least integer greater than or equal to formula_14 . For any real number a, the notation formula_17 (read “floor of a”) denotes the greatest integer less than or equal to formula_14. If formula_19 is not an integer, then the formula_20-th quantile is unique and is equal to formula_21 If formula_19 is an integer, then the formula_20-th quantile is not unique and is any real number formula_24 such that formula_25 Empirical median. If formula_26 is odd, then the empirical median is the number formula_27 If formula_26 is even, then the empirical median is the number formula_29 Asymptotic properties. Since the ratio approaches 1 as goes to infinity, the  asymptotic properties of the two definitions that are given above are the same. By the strong law of large numbers, the estimator formula_30 converges to as almost surely, for every value of : thus the estimator formula_30 is consistent. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over : The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution formula_30 and the  assumed true cumulative distribution function . Other norm functions may be reasonably used here instead of the sup-norm. For example, the L-norm gives rise to the Cramér–von Mises statistic. The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that "pointwise", formula_30 has asymptotically normal distribution with the standard formula_36 rate of convergence: This result is extended by the Donsker’s theorem, which asserts that the "empirical process" formula_38, viewed as a function indexed by formula_39, converges in distribution in the Skorokhod space formula_40 to the mean-zero Gaussian process formula_41 , where is the standard Brownian bridge. The covariance structure of this Gaussian process is The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding: Alternatively, the rate of convergence of formula_44 can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality provides bound on the tail probabilities of formula_45: In fact, Kolmogorov has shown that if the cumulative distribution function is continuous, then the expression formula_45 converges in distribution  to formula_48, which has the Kolmogorov distribution that does not depend on the form of . Another result, which follows from the law of the iterated logarithm, is that and Confidence intervals. As per Dvoretzky–Kiefer–Wolfowitz inequality the interval that contains the true CDF, formula_51, with probability formula_52 is specified as As per the above bounds, we can plot the Empirical CDF, CDF and Confidence intervals for different distributions by using any one of the Statistical implementations. Following is the syntax from Statsmodel for plotting empirical distribution. Statistical implementation. A non-exhaustive list of software implementations of Empirical Distribution function  includes: