page_title: Beta distribution
section_title: Jeffreys' prior probability (Beta(1/2,1/2) for a Bernoulli or for a binomial distribution)
hierarchical_section_title: Beta distribution / Statistical Inference / Bayesian inference / Jeffreys' prior probability (Beta(1/2,1/2) for a Bernoulli or for a binomial distribution)
caption_attribution_description: English: Jeffreys prior probability for the beta distribution - J. Rodal
context_page_description: In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval [0, 1] parameterized by two positive shape parameters, denoted by α and β, that appear as exponents of the random variable and control the shape of the distribution. The generalization to multiple variables is called a Dirichlet distribution.
  The beta distribution has been applied to model the behavior of random variables limited to intervals of finite length in a wide variety of disciplines.
  In Bayesian inference, the beta distribution is the conjugate prior probability distribution for the Bernoulli, binomial, negative binomial and geometric distributions. The beta distribution is a suitable model for the random behavior of percentages and proportions.
  The formulation of the beta distribution discussed here is also known as the beta distribution of the first kind, whereas beta distribution of the second kind is an alternative name for the beta prime distribution.
context_section_description: Harold Jeffreys proposed to use an uninformative prior probability measure that should be invariant under reparameterization: proportional to the square root of the determinant of Fisher's information matrix.  For the Bernoulli distribution, this can be shown as follows: for a coin that is "heads" with probability p ∈ [0, 1] and is "tails" with probability 1 − p, for a given (H,T) ∈ {(0,1), (1,0)} the probability is pᴴ(1 − p)ᵀ.  Since T = 1 − H, the Bernoulli distribution is pᴴ(1 − p)¹ ⁻ ᴴ. Considering p as the only parameter, it follows that the log likelihood for the Bernoulli distribution is
  The Fisher information matrix has only one component (it is a scalar, because there is only one parameter: p), therefore:
  Similarly, for the Binomial distribution with n Bernoulli trials, it can be shown that
  Thus, for the Bernoulli, and Binomial distributions, Jeffreys prior is proportional to , which happens to be proportional to a beta distribution with domain variable x = p, and shape parameters α = β = 1/2, the arcsine distribution:
  It will be shown in the next section that the normalizing constant for Jeffreys prior is immaterial to the final result because the normalizing constant cancels out in Bayes theorem for the posterior probability.  Hence Beta(1/2,1/2) is used as the Jeffreys prior for both Bernoulli and binomial distributions. As shown in the next section, when using this expression as a prior probability times the likelihood in Bayes theorem, the posterior probability turns out to be a beta distribution. It is important to realize, however, that Jeffreys prior is proportional to  for the Bernoulli and binomial distribution, but not for the beta distribution.  Jeffreys prior for the beta distribution is given by the determinant of Fisher's information for the beta distribution, which, as shown in the section titled "Fisher information matrix"  is a function of the trigamma function ψ₁ of shape parameters α and β as follows:
  As previously discussed, Jeffreys prior for the Bernoulli and binomial distributions is proportional to the arcsine distribution Beta(1/2,1/2), a one-dimensional curve that looks like a basin as a function of the parameter p of the Bernoulli and binomial distributions. The walls of the basin are formed by p approaching the singularities at the ends p → 0 and p → 1, where Beta(1/2,1/2) approaches infinity. Jeffreys prior for the beta distribution is a 2-dimensional surface (embedded in a three-dimensional space) that looks like a basin with only two of its walls meeting at the corner α = β = 0 (and missing the other two walls) as a function of the shape parameters α and β of the beta distribution. The two adjoining walls of this 2-dimensional surface are formed by the shape parameters α and β approaching the singularities (of the trigamma function) at α, β → 0. It has no walls for α, β → ∞ because in this case the determinant of Fisher's information matrix for the beta distribution approaches zero.
  It will be shown in the next section that Jeffreys prior probability results in posterior probabilities (when multiplied by the binomial likelihood function) that are intermediate between the posterior probability results of the Haldane and Bayes prior probabilities.
  Jeffreys prior may be difficult to obtain analytically, and for some cases it just doesn't exist (even for simple distribution functions like the asymmetric triangular distribution). Berger, Bernardo and Sun, in a 2009 paper  defined a reference prior probability distribution that (unlike Jeffreys prior) exists for the asymmetric triangular distribution. They cannot obtain a closed-form expression for their reference prior, but numerical calculations show it to be nearly perfectly fitted by the (proper) prior
  where θ is the vertex variable for the asymmetric triangular distribution with support [0, 1] (corresponding to the following parameter values in Wikipedia's article on the triangular distribution: vertex c = θ, left end a = 0,and right end b = 1). Berger et al. also give a heuristic argument that Beta(1/2,1/2) could indeed be the exa
caption: Jeffreys prior probability for the beta distribution: the square root of the determinant of Fisher's information matrix: 
    
      
        
          
            
              
                det
                (
                
                  
                    I
                  
                
                (
                α
                ,
                β
                )
                )
              
            
            =
            
              
                
                  ψ
                  
                    1
                  
                
                (
                α
                )
                
                  ψ
                  
                    1
                  
                
                (
                β
                )
                −
                (
                
                  ψ
                  
                    1
                  
                
                (
                α
                )
                +
                
                  ψ
                  
                    1
                  
                
                (
                β
                )
                )
                
                  ψ
                  
                    1
                  
                
                (
                α
                +
                β
                )
              
            
          
        
      
      {\displaystyle \scriptstyle {\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}={\sqrt {\psi _{1}(\alpha )\psi _{1}(\beta )-(\psi _{1}(\alpha )+\psi _{1}(\beta ))\psi _{1}(\alpha +\beta )}}}
    
   is a function of the trigamma function ψ1 of shape parameters α, β
exif: {}