# Geometry of maximum-entropy proofs: stationary points, convexity, Legendre transforms, exponential families

P.G.L. Porta Mana

► To cite this version:

P.G.L. Porta Mana. Geometry of maximum-entropy proofs: stationary points, convexity, Legendre transforms, exponential families. 2017. hal-01540184

HAL Id: hal-01540184

https://hal.science/hal-01540184

Preprint submitted on 12 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Geometry of maximum-entropy proofs:
stationary points, convexity,
Legendre transforms, exponential families

P.G.L. Porta Mana <pgl@portamana.org>

Independent researcher, Italy

12 June 2017

Most proofs of the traditional maximum-entropy formulae (Jaynes 1963; Sivia 2006 ch. 5)

$$p = \frac{\exp(lE + \ln q)}{Z(l)} \qquad Z(l) = \sum \exp(lE + \ln q) \qquad \frac{\partial \ln Z}{\partial l} = \epsilon,$$

with or without $q$ terms, rely on the method of Lagrange multipliers, and many of them also show that the constrained maximization of the Shannon entropy is equivalent to the minimization of a “potential function” via a Lagrange transform. Examples are Mead & Papanicolaou’s (1984 § II) and Jaynes’s (2003 § 11.6) proofs.

This note is a geometric commentary on such proofs. Its purpose is to help you visualize some of the geometric structures involved and to explain more in detail why we end up *minimizing* a potential function to *maximize* the entropy, and how Lagrange transforms emerge. A synopsis of the main functions involved in the proof and of their very different properties is given at the end, together with a brief discussion of exponential families of probabilities, which appear in the proof.

I assume that you have some familiarity with the maximum-entropy method and the standard proof of its formulae, as in the references above. The geometric commentary is formulated in terms of the *relative* Shannon entropy (Jaynes 1963 § 4.c; Kullback 2006; Good 1983), also called negative discrimination information (Kullback 1978 ch. 3), with respect to a reference distribution, because only this formulation is coordinate-independent in the continuum limit (Hobson et al. 1973; Good 1983).

**1 Notation** I hope you will indulge me in using vector-covector notation, which produces compact multidimensional formulae. A covector maps a vector into a scalar; it can be represented as a row matrix, and a vector as a column matrix. Here Latin letters will represent vectors; Greek, covectors. Juxtaposition of a vector and a covector always means their contraction or row-column multiplication, irrespective of their order; e.g.
$\epsilon l = l \epsilon = \sum_n \epsilon^n l_n$. I also use the convention that the logarithm maps vectors into covectors elementwise, and vice versa for the exponential; for example $\ln p := (\ln p_1, \ln p_2, \dots)$ and is a covector. Finally, *convex* means U-shaped, and *concave* means ∩-shaped.

**2 Geometry of the proof** We have $K$ states $k \in \{1, \dots, K\}$ and $N$ observables $n \in \{1, \dots, N\}$. A measurement of the $n$th observable when the state is $k$ gives the value $E^{nk}$. These values are grouped into the operator $E = (E^{nk})$ which maps vectors to covectors. The distribution of probabilities $p = (p_k)$, a vector, expresses our uncertainty about the actual state. The expectations for the $N$ observables under our state of uncertainty are $Ep$. The $K$-dimensional covector $\Sigma = (1, \dots, 1)$ and allows us very suggestively to write $\Sigma p = \sum_k p_k$.

We want to choose a probability distribution for which the expectations are constrained to values $\epsilon = (\epsilon_n)$, represented by a covector. But the constraints $Ep = \epsilon$ don't select a unique $p$ if $N < K - 1$. We therefore ask that that the distribution meet an additional requirement that makes it unique: $p$ must maximize the relative Shannon entropy $H(p) := -p \ln p + p \ln q$, under those constraints, with respect to a reference distribution $q$. We denote this unique distribution $p_\epsilon$, and the corresponding maximum value of the entropy $S(\epsilon) := H(p_\epsilon)$; it's called the *Gibbs entropy*. The Shannon and Gibbs entropies are different functions, of completely different quantities.

To find the constrained maximum of the relative Shannon entropy we use the method of Lagrange multipliers (Boyd et al. 2009 ch. 5). I warmly recommend Rockafellar's (1993) brilliant review of the various meanings of Lagrange multipliers, which offers plenty of geometric insights.

With scalar $u$ and vector $l = (l_n)$, define the function of $(p, u, l)$ with parameter $\epsilon$

$$L_{\epsilon}(p, u, l) := -p \ln p + p \ln q + u (\Sigma p - 1) + l (Ep - \epsilon), \quad (1)$$

usually called *Lagrangian* (Fang et al. 1997; Boyd et al. 2009). It's defined on the $(K + N + 1)$-dimensional manifold $\mathbf{R}_{\ge 0}^K \times \mathbf{R}^{N+1}$, our "base manifold". Proofs of the maximum-entropy formulae with linear constraints show that the *Lagrangian* $L_{\epsilon}(p, u, l)$ has a unique saddle point $(p_{\epsilon}, u_{\epsilon}, l_{\epsilon})$, and the saddle-point coordinate $p_{\epsilon}$ is the maximum-entropy solution.

Figure 1 depicts our constrained maximization problem in the simplest nontrivial case: $K = 3, N = 1$. Our base manifold has therefore 5 dimensions. The 3-dimensional space represented in the figure is a section of our base manifold, obtained by selecting specific values of the coordinates $p_3$ and $u$ as functions – (3a) and (4b)–(4d) below – of the remaining ones.
<figure><img src="image_2.png" /></figure>

$K = 3, N = 1$. Equations $\Sigma \mathbf{p} - 1 = 0$ (3b) and $u = 1 - \ln Z(l)$ (4b) are identically satisfied.

**Black dot:** saddle point ($p_{\epsilon}, u_{\epsilon}, l_{\epsilon}$).

**Blue surfaces:** contours of $L_{\epsilon}(p, u, l)$ (1).

**Thick purplish-red curve:** submanifold $-\ln p + \ln q + lE + u\Sigma = 0$ (3a) or $p = \exp(lE + \ln q)/Z(l)$ (4a); its intersection with the blue surfaces are the “contours” (points) of $G_{\epsilon}(l)$, eq. (5).

**Vertical bluish-green plane:** submanifold $Ep - \epsilon = 0$ (3c).

**Horizontal greenish-yellow plane:** submanifold $E \exp(lE + \ln q)/Z(l) = \epsilon$ (4c), which is the plane $l = l_{\epsilon}$.

<caption>Figure 1 (Colour-blind-friendly palette by Tol 2012)</caption>
The **blue surfaces** in fig. 1 are the contour levels of the Lagrangian (1). Their saddle is clearly visible. The saddle point is the **black dot**. Note that a section roughly parallel to the *l* axis is made within the figure for clearer visibility of the surfaces and curves involved.

The value of the Lagrangian $L_\epsilon$ at the saddle point is the value of the Gibbs entropy:

$$L_{\epsilon}(p_{\epsilon}, u_{\epsilon}, l_{\epsilon}) \equiv S(\epsilon). \qquad (2)$$

It's easy to check this: the constraint terms in (1) vanish at the saddle-point and what remains is the maximized Shannon entropy, that is, the Gibbs entropy.

Our saddle-point problem reduces, thanks to the continuous differentiability of the Lagrangian, to the system of three vector equations

$$\partial L_{\epsilon} / \partial p = 0: \quad -\ln p + \ln q + lE + u\Sigma = 0, \qquad (3a)$$

$$\partial L_{\epsilon} / \partial u = 0: \quad \Sigma p - 1 = 0, \qquad (3b)$$

$$\partial L_{\epsilon} / \partial l = 0: \quad Ep - \epsilon = 0. \qquad (3c)$$

They are the implicit equations of three submanifolds in our $(K + N + 1)$-dimensional base manifold. The first is curved, $(N + 1)$-dimensional. The second is flat, $(K + N)$-dimensional, and contains the simplex of normalized probability distributions; the 3-dimensional space depicted in fig. 1 is the intersection of our 5-dimensional base manifold with this submanifold, and therefore eq. (3b) is identically satisfied in the figure. The third submanifold is flat, $(K + 1)$-dimensional; it is the **vertical bluish-green plane** in the figure. The saddle point is the intersection of these three submanifolds.

The vector equations above can be recombined and written as a system of three new vector equations:

$$p = \frac{1}{Z(l)} \exp(lE + \ln q), \qquad (4a)$$

$$u = 1 - \ln Z(l), \qquad (4b)$$

$$\frac{1}{Z(l)} E \exp(lE + \ln q) = \epsilon, \qquad (4c)$$

$$\text{with } Z(l) := \Sigma \exp(lE + \ln q). \qquad (4d)$$

The first is the parametric equation of a curved $(N + 1)$-dimensional submanifold, which is also submanifold of (3b). The second is the parametric equation of a curved $(K + N)$-dimensional submanifold; the 3-dimensional
space of fig. 1 is the intersection of this submanifold, besides (3b), with our 5-dimensional base manifold; eq. (4b) is thus also identically satisfied in the figure. The third equation in the system above is the implicit equation of a flat (K + 1) dimensional submanifold. This equation determines a unique value $l_ϵ$, so it is simply the equation of the (K + 1)-plane $l = l_ϵ$, the **horizontal greenish-yellow plane** in fig. 1. These three submanifolds (4), are distinct from the previous three (3), but they also intersect at the saddle point. The function $\ln Z(l)$ defined in (4d) is called *normalization function* or *partition function*.

Note that we could have formulated our constrained-maximization problem by imposing the normalization (3b) from the very beginning, for example defining $p_K = 1 - p_1 - p_2 - \dots$. The multiplier $u$ and eqs (3b), (4b) wouldn't have appeared, and our base manifold would have been (K + N - 1)-dimensional. Figure 1 can also be interpreted this way.

The system (3a) & (3b) is equivalent to the system (4a) & (4b), as can be verified by substitution; that is, the N-dimensional intersection submanifold of the first pair is also the intersection of the second pair. This submanifold is the **thick purplish-red curve** in the figure. Projected onto the simplex of probability distributions – that is, disregarding the $u$ and $l$ dimensions – this submanifold is an subset of the latter called an *exponential family*. Exponential families are briefly discussed in § 4. The saddle point is the intersection of this N-dimensional submanifold with either the (K + 1)-plane (3c) or the (K + 1)-plane (4c).

The saddle point ($p_ϵ, u_ϵ, l_ϵ$) could therefore be found by finding the root $l = l_ϵ$ of eq. (4c), and substituting this root in ($p(l), u(l)$), the parametric form of eqs (4a) & (4b). But it turns out that we do not need to solve eq. (4c). There's an interesting development.

Looking at fig. 1 we notice that the *N-dimensional manifold* ($p(l), u(l)$) *extends from the pommel to the cantle of the saddle* (if the lower part is the cantle the horse is rearing). This was a priori not necessary: by construction this submanifold must pass through the saddle point, but it could have done so by going from the pommel down to the flaps of the saddle. It couldn't have made a U-turn back to the pommel, however, because that implies the presence of a wedge, whereas our manifold is continuously differentiable.

This placement of the *N*-dimensional submanifold implies that if we ride it we see the values of the Lagrangian decrease until we reach the saddle point, and then increase again. *The saddle point is therefore the minimum of the Lagrangian restricted to the *N*-dimensional submanifold.* This is
true for the $N = 1$ case of our figure, but it generalizes to larger $N$; here's how.

Consider the restriction of the Lagrangian $L_\epsilon$, eq. (1), to the $N$-dimensional submanifold. This restriction, denoted $\Gamma_\epsilon$, is usually called the potential function. It's by construction a function of $l$ alone, from eqs (4a) and (4b):

$$ \Gamma_{\epsilon}(l) := L_{\epsilon}[p(l), u(l), l] \equiv \ln Z(l) - l\epsilon. \qquad (5) $$

In fig. 1 the intersections of the blue surfaces with the thick purplish-red curve are the “contours” – just points in this case – of this function. The Hessian matrix of its second derivatives has non-negative eigenvalues: a simple calculation and a look at (4) reveal that this is in fact the covariance matrix of the observable $E$:

$$ \frac{\partial^2 \Gamma_{\epsilon}}{\partial l_n \partial l_m} \equiv \frac{\partial^2 \ln Z}{\partial l_n \partial l_m} = \sum_k E^{nk} E^{mk} p_k(l) - \sum_k E^{nk} p_k(l) \sum_k E^{mk} p_k(l), \quad (6) $$

and covariance matrices have non-negative eigenvalues (Feller 1971 § III.5). The potential function $\Gamma_\epsilon$ is therefore *convex* – strictly so, without flat regions, owing to the differential properties of the logarithm. Calculation of its unique minimum by derivation leads to eq. (4c). The conclusion is that the potential function $\Gamma_\epsilon$ is convex in $l$, and the saddle point of $L_\epsilon$ in the $(K+N+1)$-dimensional base manifold is the unique minimum of $\Gamma_\epsilon$ in the $N$-submanifold.

This is the geometric reason why the constrained-maximization problem in $K$ dimensions for the Shannon entropy $H(p)$ can be transformed into an unconstrained-minimization problem in $N$ dimensions for the potential function $\ln Z(l) - l\epsilon$. The latter is usually called the *dual problem* (Fang et al. 1997; Boyd et al. 2009). This fact is enormously useful for numerical computations: it allows us to use convex optimization techniques (Press et al. 2007) to find the extremizing Lagrange multipliers $l_\epsilon$ and thence the distribution $p_\epsilon$ and the Gibbs entropy $S(\epsilon) := H(p_\epsilon)$. See Rockafellar's (1993) insightful discussion in this respect too.

But the geometry of this extremization problem has further surprises.

The Gibbs entropy $S(\epsilon)$ is, from its definition, equal to $L_\epsilon(p_\epsilon, u_\epsilon, l_\epsilon) \equiv \Gamma_\epsilon(l_\epsilon) \equiv \ln Z(l_\epsilon) - l_\epsilon \epsilon$, which is the unique minimum of $\Gamma_\epsilon(l)$. We can write this as

$$ S(\epsilon) = \inf_{l} [\ln Z(l) - l\epsilon]. \qquad (7) $$

This formula is the proper definition of the negative *Legendre transform* of the normalization function $\ln Z(l)$, that is, its negative convex conjugate
(Fenchel 1949). This means that the Gibbs entropy $S(\epsilon)$ is a concave function of $\epsilon$. The concavity of the Gibbs entropy is an important property, completely distinct from the concavity of the Shannon entropy $H(p)$. I said "proper" Legendre transform because the physics literature often defines the latter without the extremization indicated by "inf" or "sup". Such simplified definition breaks down if the transformed function is not strictly convex – which may happen in important physical situations. See Wightman's illuminating and pedagogical discussion (Wightman 1979 pp. xxiv–xxix).

From the involutive property of the Legendre transform (see refs above), the normalization function $\ln Z(l)$ is the negative Legendre transform of the negative Gibbs entropy:

$$ \ln Z(l) = \inf_{\epsilon} [-S(\epsilon) - \epsilon l]. \qquad (8) $$

The appearance of these Lagrange transforms has many important connections with statistical mechanics; for interesting recent developments see Chomaz & al.'s (2006; 2005) reviews. I refrain from speaking about the relationship between the maximum-entropy method: it's subtle and already too often oversimplified in the literature.

**3 Synopsis of the main functions** It's important to keep the main functions involved in the proof well-distinct from one another:

* The Shannon entropy $H(p)$ is a function of the probability distribution $p$. It's concave in $p$. It isn't the Legendre transform of anything.

* The Gibbs entropy

$$ S(\epsilon) = \sup_{p}^{E_p = \epsilon} H(p) = \inf_{l} [\ln \Sigma \exp(lE + \ln q) - l\epsilon] = \inf_{l} [\ln Z(l) - l\epsilon] $$

is a function of the expectation values $\epsilon$. It's concave in $\epsilon$. It's the constrained maximum of the relative Shannon entropy, the unconstrained minimum of the potential function, and the negative Legendre transform of the normalization function.

* The normalization function, partition function, or free entropy

$$ \ln Z(l) = \ln \Sigma \exp(lE + \ln q) = \inf_{\epsilon} [-S(\epsilon) - \epsilon l] $$

is a function of the Lagrange multipliers $l$. It's convex in $l$. It's the negative Legendre transform of the negative Gibbs entropy.

* The potential function $\Gamma_{\epsilon}(l) = \ln Z(l) - l\epsilon$ is a function of the Lagrange multipliers $l$ with a parametric dependence on the expectation values $\epsilon$. It's convex in $l$. It isn't the Legendre transform of anything. Not to be confused with the Gibbs entropy.
**4 Exponential families** The maximum-entropy method is essentially a function that maps a set of observables, a set of observable constraints, and a reference distribution to a probability distribution: $(E, \epsilon, q) \mapsto p$. From this point of view all other quantities appearing in its proof and formulae are just auxiliary quantities – including the Lagrange multipliers $l$.

But the parametric submanifold of probabilities $p(l)$, eq. (4a):

$$p(l) = \frac{1}{Z(l)} \exp(lE + \ln q), \quad Z(l) := \sum \exp(lE + \ln q), \quad (9)$$

has a meaning and an importance of its own, outside of the maximum-entropy method. It is an example of *exponential family*. Exponential families are particular submanifolds of a simplex of probability distributions characterized by an exponential parametric form like the above or more general (see below for references).

The figures on the right show an example of 2-dimensional exponential family for the case with four states, $K = 4$, two observables, $N = 2$, having values

$$(E^{nk}) = \begin{pmatrix} E^{1k} \\ E^{2k} \end{pmatrix} = \begin{pmatrix} 0 & 1 & 2 & 3 \\ 1 & 1 & 0 & 0 \end{pmatrix}, \quad (10)$$

and a uniform reference distribution $q$. The **black lines** are the edges of the **simplex of probability distributions $p$, a tetrahedron.** The **exponential-family submanifold** is the **yellow surface**, the same in both figures.

In the top figure the **reddish-purple curves** are equally-spaced $(l_1, l_2)$ coordinates for the parametrization in terms of the Lagrange multipliers, $l := (l_1, l_2) \in \mathbb{R}^2$. In the bottom figure the **green curves** are equally-spaced $(\epsilon_1, \epsilon_2)$ coordinates for the parametrization in terms of the expectations, $\epsilon := (\epsilon_1, \epsilon_2) \in [0, 3] \times [0, 1]$.

<figure><img src="image_3.png" /></figure>
The relation between these two coordinate systems, eq. (3c), is highly non-linear. The *l* coordinates have the advantage of parametrizing *p*(*l*) in closed form, eq. (4a), but are uncongenial to the convex structure of the simplex of probability distributions; their non-compact range must in fact cover a compact set. The *ε* coordinates are clearly more congenial to the convex structure of the probability simplex, but they do not lend themselves to a parametrization *p*(*ε*) in closed form. Both *l*- and *ε*-parametrizations are therefore important.

The Bernoulli, Poisson, exponential, normal distributions belong to exponential families. See Bernardo & Smith (2000 ch. 4, esp. § 4.5.3) for a thorough discussion of these families, Barndorff-Nielsen (2014) for their relation with Lagrange transforms, Dawid (2013) for a broader context. Exponential families appear in the probability calculus when we assume that a particular fixed set of quantities from some measurements is all we need to make inferences about other similar measurements, a condition called sufficiency (Bernardo et al. 2000 ch. 4, esp. § 4.5.3; Barndorff-Nielsen 2014; Dawid 2013; Andersen 1970). The fact that they appear in the maximum-entropy formulae thus suggests a relation between maximum-entropy and sufficiency. A recent work which I don't fully understand (Porta Mana 2017) argues, however, that this relation has some downsides, and maximum-entropy distributions are best related to so-called exchangeable models (Bernardo et al. 2000 § 4.3).

**Thanks** I owe the inspiration for writing this note to Moritz Helias, Tobias Kühn, and Vahid Rostami. I cordially thank Tobias for detecting some deficiencies in an early draft. It goes without saying that any deficiencies that may remain are therefore *his* fault. . . right?

**Bibliography**

Andersen, E. B. (1970): *Sufficiency and exponential families for discrete sample spaces*. J. Am. Stat. Assoc. **65**<sup>331</sup>, 1248–1255.

Barndorff-Nielsen, O. E. (2014): *Information and Exponential Families: In Statistical Theory*, reprint. (Wiley, New York). First publ. 1978.

Bernardo, J.-M., Smith, A. F. (2000): *Bayesian Theory*, reprint. (Wiley, New York). First publ. 1994.

Boyd, S., Vandenberghe, L. (2009): *Convex Optimization*, 7th printing with corrections. (Cambridge University Press, Cambridge). http://www.stanford.edu/~boyd/cvxbo ok/. First publ. 2004.

Chomaz, Ph., Gulminelli, F. (2005): *Phase transition in small system*. Nucl. Phys. A **749**, 3–13. arXiv:cond-mat/0410426.

— (2006): *The challenges of finite-system statistical mechanics*. Eur. Phys. J. A **30**<sup>1</sup>, 317–331.
Damien, P., Dellaportas, P., Polson, N. G., Stephens, D. A., eds. (2013): *Bayesian Theory and Applications*. (Oxford University Press, Oxford).

Dawid, A. P. (2013): *Exchangeability and its ramifications*. In: (Damien, Dellaportas, Polson, Stephens 2013), ch. 2, 19–29.

Fang, S.-C., Rajasekera, J. R., Tsao, H.-S. J. (1997): *Entropy Optimization and Mathematical Programming*, reprint. (Springer, New York).

Feller, W. (1971): *An Introduction to Probability Theory and Its Applications*. Vol. II, 2nd ed. (Wiley, New York). First publ. 1966.

Fenchel, W. (1949): *On conjugate convex functions*. Can. J. Math. **1**, 73–77.

Ford, K. W., ed. (1963): *Statistical Physics*. (Benjamin, New York).

Good, I. J. (1983): *Review: The Maximum Entropy Formalism*. J. Am. Stat. Assoc. **78**<sup>384</sup>, 987–989.

Hobson, A., Cheng, B.-K. (1973): *A comparison of the Shannon and Kullback information measures*. J. Stat. Phys. **7**<sup>4</sup>, 301–310.

Israel, R. B. (1979): *Convexity in the Theory of Lattice Gases*. (Princeton University Press, Princeton). With an Introduction by Arthur S. Wightman.

Jaynes, E. T. (1963): *Information theory and statistical mechanics*. In: (Ford 1963), 181–218. Repr. in (Jaynes 1989), ch. 4, 39–76. http://bayes.wustl.edu/etj/node1.html.

— (1989): *E. T. Jaynes: Papers on Probability, Statistics and Statistical Physics*, reprint. (Kluwer, Dordrecht). Ed. by R. D. Rosenkrantz. First publ. 1983.

— (2003): *Probability Theory: The Logic of Science*. (Cambridge University Press, Cambridge). Ed. by G. Larry Bretthorst; http://omega.albany.edu:8008/JaynesBook.html, http://omega.albany.edu:8008/JaynesBookPdf.html, http://www-biba.inria.fr/Jaynes/prob.html. First publ. 1994.

Kullback, S. (1978): *Information Theory and Statistics*. (Dover, New York). Republ. with a new preface and corrections and additions by the author. First publ. 1959.

— (2006): *Information, Kullback*. Encyclopedia of Statistical Sciences 2nd ed. 5. First publ. 1982.

Mead, L. R., Papanicolaou, N. (1984): *Maximum entropy in the problem of moments*. J. Math. Phys. **25**<sup>8</sup>, 2404–2417. http://bayes.wustl.edu/Manual/MeadPapanicolaou.pdf.

Porta Mana, P. G. L. (2017): *Maximum-entropy from the probability calculus: exchangeability, sufficiency*. arXiv:1706.02561.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., Flannery, B. P. (2007): *Numerical Recipes: The Art of Scientific Computing*, 3rd ed. (Cambridge University Press, Cambridge). First publ. 1988.

Rockafellar, R. T. (1993): *Lagrange multipliers and optimality*. SIAM Rev. **35**<sup>2</sup>, 183–238. http://www.math.washington.edu/~rtr/papers.html.

Sivia, D. S. (2006): *Data Analysis: A Bayesian Tutorial*, 2nd ed. (Oxford University Press, Oxford). Written with J. Skilling. First publ. 1996.

Tol, P. (2012): *Colour schemes*. Tech. rep. SRON/EPS/TN/09-002. (SRON Netherlands Institute for Space Research, Utrecht). http://www.sron.nl/~pault/. First publ. 2009.

Wightman, A. S. (1979): *Convexity and the notion of equilibrium state in thermodynamics and statistical mechanics*. In: (Israel 1979), Introduction, ix–lxxxv.