page_title: Kullback–Leibler divergence
section_title: Relationship to available work
hierarchical_section_title: Kullback–Leibler divergence / Relationship to available work
caption_attribution_description: This is a pressure versus volume (PV) plot of the work available from a mole of Argon gas with respect to an ambient reservoir at standard temperature and pressure. In engineering parlance these are contours of fixed exergy difference, while in statistical mechanics they are contours of constant Kullback-Leibler divergence times ambient temperature.
context_page_description: In mathematical statistics, the Kullback–Leibler divergence is a measure of how one probability distribution is different from a second, reference probability distribution. Applications include characterizing the relative entropy in information systems, randomness in continuous time-series, and information gain when comparing statistical models of inference. In contrast to variation of information, it is a distribution-wise asymmetric measure and thus does not qualify as a statistical metric of spread - it also does not satisfy the triangle inequality. In the simple case, a Kullback–Leibler divergence of 0 indicates that the two distributions in question are identical. In simplified terms, it is a measure of surprise, with diverse applications such as applied statistics, fluid mechanics, neuroscience and machine learning.
context_section_description: Surprisals add where probabilities multiply. The surprisal for an event of probability  is defined as . If  is  then surprisal is in nats, bits, or  so that, for instance, there are  bits of surprisal for landing all "heads" on a toss of  coins.
  Best-guess states (e.g. for atoms in a gas) are inferred by maximizing the average surprisal  (entropy) for a given set of control parameters (like pressure  or volume ). This constrained entropy maximization, both classically and quantum mechanically, minimizes Gibbs availability in entropy units  where  is a constrained multiplicity or partition function.
  When temperature  is fixed, free energy () is also minimized. Thus if  and number of molecules  are constant, the Helmholtz free energy  (where  is energy) is minimized as a system "equilibrates." If  and  are held constant (say during processes in your body), the Gibbs free energy  is minimized instead. The change in free energy under these conditions is a measure of available work that might be done in the process. Thus available work for an ideal gas at constant temperature  and pressure  is  where  and  (see also Gibbs inequality).
  More generally the work available relative to some ambient is obtained by multiplying ambient temperature  by Kullback–Leibler divergence or net surprisal  defined as the average value of  where  is the probability of a given state under ambient conditions. For instance, the work available in equilibrating a monatomic ideal gas to ambient values of  and  is thus , where Kullback–Leibler divergence
  The resulting contours of constant Kullback–Leibler divergence, shown at right for a mole of Argon at standard temperature and pressure, for example put limits on the conversion of hot to cold as in flame-powered air-conditioning or in the unpowered device to convert boiling-water to ice-water discussed here. Thus Kullback–Leibler divergence measures thermodynamic availability in bits.
caption: Pressure versus volume plot of available work from a mole of argon gas relative to ambient, calculated as 
    
      
        
          
            T
            
              o
            
          
        
      
      {\displaystyle T_{o}}
    
   times the Kullback–Leibler divergence.
exif: {}