Error exponents in hypothesis testing
In statistical hypothesis testing, the error exponent of a hypothesis testing procedure is the rate at which the probabilities of Type I and Type II decay exponentially with the size of the sample used in the test. For example, if the probability of error formula_1 of a test decays as formula_2, where formula_3 is the sample size, the error exponent is formula_4. Formally, the error exponent of a test is defined as the limiting value of the ratio of the negative logarithm of the error probability to the sample size for large sample sizes: formula_5 . Error exponents for different hypothesis tests are computed using Sanov's theorem and other results from large deviations theory. Error exponents in binary hypothesis testing. Consider a binary hypothesis testing problem in which observations are modeled as independent and identically distributed random variables under each hypothesis. Let formula_6 denote the observations. Let formula_7 denote the probability density function of each observation formula_8 under the null hypothesis formula_9 and let formula_10 denote the probability density function of each observation formula_8 under the alternate hypothesis formula_12. In this case there are two possible error events. Error of type 1, also called false  positive, occurs when the null hypothesis is true and it is wrongly rejected. Error of type 2, also called false negative, occurs when the alternate hypothesis is true and null hypothesis is not rejected. The probability of type 1 error is denoted formula_13 and the probability of type 2 error is denoted formula_14. Optimal error exponent for Neyman–Pearson testing. In the Neyman–Pearson version of binary hypothesis testing, one is interested in minimizing the probability of type 2 error formula_15 subject to the constraint that the probability of type 1 error formula_16 is less than or equal to a  pre-specified level formula_17. In this setting, the optimal testing procedure is a likelihood-ratio test. Furthermore, the optimal test guarantees that the type 2 error probability decays exponentially in the sample size formula_3 according to formula_19. The error exponent formula_20 is the Kullback–Leibler divergence between the probability distributions of the observations under the two hypotheses. This exponent is also referred to as the Chernoff–Stein lemma exponent. Optimal error exponent for average error probability in Bayesian hypothesis testing. In the Bayesian version of binary hypothesis testing one is interested in minimizing the average error probability under both hypothesis , assuming a prior probability of occurrence on each hypothesis. Let formula_21 denote the prior probability of hypothesis formula_22. In this case the average error probability is given by formula_23. In this setting again a likelihood ratio test is optimal and the optimal error decays as formula_24 where formula_25 represents the Chernoff-information between the two distributions defined as formula_26.