数理解析研究所講究録980 巻 1997 年 56-65

56

Detailed Balance and Free Energy. Streater,Dept. of Mathematics,
King’s College London,
Strand$\mathrm{R}.\mathrm{F}$$\mathrm{W}\mathrm{C}2\mathrm{R}2\mathrm{L}\mathrm{s}$October 27, 1993
Abstract
We forlnulate the notion of detailed balance at a given beta for a
discrete Markov chain, and show that it implies that one time-step in. this is defined on the set
the chain is a contractioll in tlue normof states as tlle dual to tlle
on the random variables given by theclassical KM state.
Key words:
ellergy; relative entropy$||\cdot||_{\beta)}$$\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{n}\mathrm{u}$$\mathrm{S}$$\mathrm{R}\cdot \mathrm{e}\mathrm{e}$1Introduction and Notation

Suppose that is a countable space, with points labelled by integers $i,j\ldots$ .
Denote by the (abelian) algebra of bounded real-valued random variablesonwith $\sup$ norm, and by
the dual space (of continuous linear
maps,). The positive, normalised elements ofare the probability
measures on , also called the states on . A typical state will be denotedby ; they form
convex set. In fact,
is a simplex, whose
extreme points, the corners, are the Dirac measures on the points of , thus
for any subset,$\Omega$$A$$f$$\Omega$$A^{*}$$Aarrow \mathrm{R}$$A^{*}$$A$$\Omega$$p$$\mathrm{t}1_{1}\mathrm{e}$$\Sigma(\Omega)$$\Sigma(\Omega)$$\Omega$$\Omega_{0}\subseteq\Omega$$\delta_{j}(\Omega_{0})$Any probability measure$p\in\Sigma$$p= \sum_{i}p_{j}\delta j$

$=$

1

if

$=$

$0$

otherwise(1)(2)$j\in\Omega_{0}$can be written uniquely as a sum
where$p_{j}\geq 0,$$\sum_{i}p_{j}=1$

.

(3)

57

will be called the components of . The duality between $A$ andThe
reduces on to the expectation:$p$$p_{j}$$A^{*}$$\Sigma$$p\in\Sigma,$$\langle p, f\rangle=\sum_{j\in\Omega}pjf_{j}$Let: $A\neg$
defined by
$T$

$A$

be a linear map. Its adjoint,$\langle T^{*}p, f\rangle=\langle p, Tf\rangle$

In order forstochastic$\tau*$$\mathrm{m}\mathrm{a}_{1^{)}}$

to map, that is$\Sigma$$f\in A$(4)

, maps$\tau*$$p\in A^{*},$.

$f\in A$

$A^{*}$

toand is$A^{*}$.

(5)

to itself it is necessary and sufficient tllat

$T$

be a

1. If $f\geq 0$ , then $Tf\geq 0$
2. $T1=1$ .
define a bounded function on , andThe components of an element of$A$
so give us an element of . The components of the Dirac measure defineform a dual basis as wea function that we shall denote by . Thenis determined
map,
then
. If is a stochastic
vary andby its matrix elements$A^{*}$$\Omega$$\delta_{i}$$\triangle_{i}$$i$$\delta_{i},$$\triangle_{j}$$T$$j:\langle\delta_{i}, \triangle_{j}\rangle=\delta_{ij}$$\tau*$$T_{ij}^{*}=\langle T^{*}\delta_{j}, \triangle_{i}\rangle=\langle\delta_{j}, T\triangle i\rangle$

Clearly,$T_{ij}^{*}\geq 0$.

(6). Also$=$$\langle T^{*}\delta_{i}, \sum_{i}\triangle i\rangle=\langle T^{*}\delta_{j}, 1\rangle$$\sum_{i\in\Omega}T_{ij}^{*}$

$=$

(7)
(8)

$\langle\delta_{j}, T1\rangle=\langle\delta_{j}, 1\rangle=1$

is a Markov matrix in the usual sense.
for all . Thus
is a positive random variable, not necSuppose now that :isessarily bounded, interpreted as the energy. We shall assume thatsummable, so that the partition function$j$$T_{ij}^{*}$$\mathcal{E}$$\Omegaarrow \mathrm{R}$$e^{-\beta \mathcal{E}}$(9)$Z_{\beta}= \sum_{\omega}e^{-\beta}\mathcal{E}(\omega)$

is finite. A Markov matrix
balance” at beta if$T_{ij}^{*},$$i,j\in\Omega$, is usually said to obey “detailed$\beta$$T_{ij}^{*}=e^{-\beta(\epsilon_{i}\mathcal{E})}-j\tau ji*$$(i,j\in\Omega)$

.

(10)

58

If $\beta=0$ Eqn.(10) says thatis symmetric. For some applications ofMarkov chains such as neural nets there is no good reason for
to besymmetric, even for $\beta=0$ . In this paper we suggest a generalisation of (10)which avoids the symmetry, but which nevertheless enables us to derive
thermodynamics. We show that the Gibbs state is a fixed point of the chain
if and only if generalised detailed balance holds. We use the “dual KMS
norm” coming from the Gibbs state and show that the Gibbs state is thestate of minimum norm, and thatis a contraction. We obtain an estimate
for the loss in free energy at each time-step in terms of this norm. Finally,a short proof of the second law of thermodynamics for a quantum Markov
chain is presented.
$\tau*$$\tau*$$\tau*$

2

The Detailed Balance ConditionLet
be the subalgebra of functions on zero except at a finite numberof points, and for $f,$ $g\in A$ define the scalar product$A_{c}$$\Omega$$\langle f,g\rangle_{\beta}=Z_{\beta}^{-1}\sum_{i\in\Omega}e-\beta \mathcal{E}_{i}$

figi

(11)

$\cdot$This idea is borrowed from the theory of quantum statistical mechanics, [1].
In this paper, the importance of the analyticity of the Green’s functions
found by Kubo, and by Martin and Schwinger, is emphasized. So we callthe corresponding scale of norms(12)$||f||_{\beta}=(Z_{\beta}^{-1} \sum e^{-}|j\beta \mathcal{E}_{j}f(j)|^{2})1/2$

the KMS norms. Let
be tlle completion ofin the metric given by.
The dual spaces of these Banach spaces are the completions of Span,of real measures of compact support, in the topology defined by the dual
$A_{\beta}$$A_{c}$$||\cdot||$$\Sigma(\Omega)_{c}$

norms
(13)

$||p||_{-\beta}=(Z_{\beta} \sum_{j}e|\beta \mathcal{E}_{r}2)^{/2}p_{j}|1$

We shall see that these are the natural norms governing the convergence ofstates under isothermal conditions, rather than the-norms usually used
for measures. Convergence in the KMS-normwith $\beta>0$ is a muchstronger notion thanconvergence since the rising exponential
forces$L^{1}$$||\cdot||_{-\beta}$$L^{1}$$e^{\beta \mathcal{E}_{j}}$59

the high-energy tail of the probability to converge very fast, and captures
some of the information in the theory of large deviations.
. Let beis finite, the unit function lies in
Since we assulne that.
, and denote by
its adjoint relative toa bounded operator on
space.
aHilbert,which isacts onNote that$T$$A_{\beta}$$Z_{\beta}$$T^{(\beta)}$$\langle\cdot, .\rangle$$A_{\beta}$$T^{(\beta)}$$A_{\beta}$

obeys detailed balance (relative toDefinition 1 We say that
maps.
both
stochastic
and
are
if
$T:A_{\beta}arrow A_{\beta}$$\mathcal{E})$$T^{(\beta)}$$T$

In particular, if satisfies detailed balance, then $T1=T^{(\beta)}1=1$ . We note
. We have an easy lemma:lies comfortably inthat the Gibbs measure$T$$A_{\beta}^{*}$$p_{\beta}$

Lemma 2 A stochastic matrix
a fixed point of .

$T$

obeys detailed balance if and only if$p_{\beta}$is$\tau*$For the proof, note that if$\langle 1, f\rangle_{\beta}$

$=$

$=$

$T^{*}p_{\beta}=p_{\beta}$, then we have$\langle p_{\beta}, f\rangle=\langle\tau_{p_{\beta}}^{*}, f\rangle$$\langle p_{\beta}, Tf\rangle=\langle 1, Tf\rangle_{\beta}=\langle\tau^{(\beta)}1, f\rangle_{\beta}$, we get $T^{(\beta)}1=1$ as required.
is dense infor all $f\in A_{c}$ . Hence, as
The converse is obtained by working backwards.
So our forlnulation of detailed balance is the most general that could
to the thermal
lead to the convergence of the Markov chain. At $\beta=0$ , the definition (1) reduces to the condition that is
state
. At
bistochastic. This is enough to ensure that entropy is increased by$\beta>0$ , we shall get the corresponding result, that
reduces the free energy.
$A_{c}$$A_{\beta}$$\{(T^{*})^{n}p\}_{n}=0,1\ldots$$T$$p_{\beta}$$\tau*$$\tau*$

Theorem 3 The unique probability measure with the smallest norm
is the Gibbs state .

$||p||\beta$$p_{\beta}$

This is proved by using a Lagrange multiplier
We now come to the main result of this section.

$\lambda\sum_{j}p_{j}$to fix the$L^{1}$

norm.

.
is a contraction in
Theorem 4 If obeys detailed balance, thenactingpoint
irreducible,
the
is
then
is
in
additionofonly. fixed
If
.
then
on
and if$\tau*$$T$$T$$p\neq p_{\beta}$$A_{\beta}^{*}$$l^{j}\beta$$||T^{*}p||_{-\beta}<||p||_{-\beta}$

irreducible means that for any
Note:length $i_{1}=i,$ $i_{2}i_{3}\ldots i_{N}=j$ such that$T$$||\cdot||_{-\beta}$$\tau*$$\dot{i},j\in\Omega$$T_{i_{k},i_{k}+1}^{*}>0$

there is a chain of finite.
, $k=1,$$\ldots N$

60

Proof.
Obviously for any sum over a finite subset:$\Omega_{0}\subseteq\Omega$

we have for any$p,$ $q\in$$\Sigma(\Omega)$(14)$\sum_{i,j}z_{\beta}\tau*eji-\beta \mathcal{E}i(^{\beta\epsilon_{J}\epsilon_{;_{p_{i}}}}eqj-e^{\beta}\mathrm{I}2\geq 0$

and this inequality persists in the limit asincreases toif the terms
converge. Let $q=T^{*}p$ . The first term is the sum of non-negative terms$\Omega$$\Omega_{0}$.

$Z_{\beta} \sum_{i,j}T*\mathcal{E},-\mathcal{E}_{j}ejie^{-\beta()}q\beta \mathcal{E}_{j}2j=z_{\beta}\sum_{i,j}T^{()_{e}\mathcal{E}}\beta jqjij\beta\cdot 2$

By detailed balance the sum of. So the first term reduces to$T_{ji}^{\langle\beta)}$over converges to 1 as$i$$\Omega_{0}$increases to$\Omega$$Z_{\beta} \sum_{j}e^{\beta}\epsilon jq_{j}^{2}=||T^{*}p||_{-\beta}^{2}$.

The cross-terms are also of one sign,$-2Z_{\beta} \sum_{ij}T^{*}jie^{\beta\epsilon_{j}}qjp_{i}$and the sum over$i$is$\sum_{i\in\Omega}\tau_{j}*pii=q_{j}=(T^{*}p)_{j}$The$\mathrm{c}\mathrm{r}\mathrm{o}\mathrm{s}\mathrm{S}- \mathrm{t}\mathrm{e}\Gamma \mathrm{n}\mathrm{l}\mathrm{s}$.

thus give us$-2||\tau_{P}^{*2}||-\beta$

The last term is exactly$||p||_{-\beta}^{2}$, since$\tau*$.

is stochastic. We conclude that$-||T^{*}p||2+-\beta||p||2\geq-\beta \mathrm{o}$as desired. For the strict inequality we follow [3]. We get a proper contraction unless$e^{\beta_{\mathrm{c}}^{c_{J}}}qj=e^{\beta \mathcal{E}_{i}}p_{i}$Starting with $i=0$ and taking

for all$\mathcal{E}_{0}=0$

$i,$

$j$

with

we get$q_{j}=e^{-\beta \mathcal{E}_{j}}p0$$T_{ji}^{*}>0$.

61

for all$j$linked to

by one step. But then we get$0$$q_{j’}=e^{-\beta(\prime}\mathcal{E}_{j}-\mathcal{E}_{j})_{e}-\beta\epsilon_{j}p0$

for all $j’$ linked to$j$by one step, so$q_{j’}=e^{-\beta\epsilon_{g}\prime}p0$

for all $j’$ linked to$0$by two steps. In this way we get$q_{k}=e^{-\beta \mathcal{E}_{k}}p0$

for all .

$p_{k}=e^{-\beta^{c_{k}}}q_{0}$for all$k$

Similarly we get

so both

3

$p$

and

$\mathrm{r}_{\mathit{1}}$are$p_{\beta}$$k$.

The Free-energy Theorem

a stateDefinition 5 Let us say$C\geq 1$ such thatexiststhere
if$tl_{l\mathit{0}}t$$p\in\Sigma(\Omega)$$p(j)\leq CZ_{\beta}^{-1\beta \mathcal{E}_{j}}e^{-}$

for allis close to equilibrium $(C,\beta)$$j\in\Omega$.

This includes all Gibbs states of beta greater than , all states with a maxwith $f\in A$ .
imum energy, and all states of the form$\beta$$fp_{\beta}$

Lemma 6 If is close to equilibrium $(C,\beta)$ andis close to equilibrium $(C,\beta)$ .
, then
$p$

For,

$T$

obeys detailed balance$T^{*}p$$(\beta)$$\tau*$

being positivity preserving,$T^{*}(cz_{\beta}^{-1}e^{-}p)\beta\epsilon_{-}(j)\geq 0$

Since$\tau*$

leaves$p\beta$.

fixed, we get$CZ_{\beta}^{-1-\beta}e\mathcal{E}J-(T^{*}p)j\geq 0$

which proves the lemma. The next theorem relates the relative entropy to
the KMS-norm, and is proved ill [2].

Theorem 7 If is close to $equilibriu’ n(C,\beta)$ then the relative entropy obeys$p$$S(p_{\beta}|p)= \sum p\beta(j)(\log p\beta(ij\mathrm{I}-\log p_{j})\geq(2C^{2})^{-1}||p-p\beta||^{2}-\beta\cdot$

(15)

62

This says that for a sequence of states close to equilibrium, if the relative
entropy converges to zero, then the states converge to equilibrium in.
We now give an estimate for the decrease in free energy under a map obeying
detailed balance, following [3] where a special case is proved.
$||\cdot||_{-\beta}$

Theorem 8) $ose$$Su_{l^{)}I}$

that

$T$

obeys detailed balance$(\beta)$and let$F(p)=\beta\langle p, \mathcal{E}\rangle-S(p)$denote the free energy, defined on a subset of$F(T^{*}p)\leq F(lJ)$

If $furtller_{P}$ is close to equilibrium ($C,\beta\grave{)}$$\Omega$. Then.

, then$F(T^{*}p)-F(p)\leq-(2C)^{-1}(||I^{y||_{-\beta}-}2||T^{*}p||2-\beta)$.

Our proof uses theLemma 9 Let $s(x)=-x\log x$ and$x\neq x_{j},j=1\ldots n$ we ltave$x= \sum\lambda_{j}x_{j},$$\lambda_{j}\geq 0,$$\sum\lambda_{j}=1$. Then if$\sum\lambda_{j}s(X_{j})=S(X)-\frac{1}{2}\sum\lambda_{j(}X_{j}-X)^{2}/\xi j$

where $x_{j}<\xi_{j}<x$ or $x<\xi_{j}<x_{j}$ .
Proof.
By Taylor’s theorem with Lagrange remainder,$s(x_{i})=s(x)+(x_{i}-x)[-1-\log X]+(x_{i}-x)^{2}(-1/(2\xi_{i})$ .

Multiply byand sum; we get the lemma, since $\sum\lambda_{i}(x_{i}-X)=0$ .
Try this lemma with$\lambda_{i}$$\lambda_{i}=T_{ji}^{*}e^{-}\beta(\mathcal{E}i-\epsilon_{j})=T_{j}^{(\beta)}i$for each , and with$j$$x_{i}=e^{\beta \mathcal{E}_{i}}$. Then$x= \sum\tau_{ii}^{*}e^{-}\dot{l}\beta\epsilon i+\beta \mathcal{E}_{j}ep_{i}\beta\epsilon i=e^{\beta \mathcal{E}_{j_{(\mathit{1}j}}}$63

where $q=T^{*}p$ . So for each

$j$

$=$

$- \sum_{i}T_{j}^{*}ie^{-\beta(}.-\mathcal{E}j\rangle(^{\beta}\epsilon.\mathcal{E}_{1pi}.(\log pi+\beta \mathcal{E}_{i}))e$$-e^{-\beta \mathcal{E}_{j}}qj(\log qj+\beta \mathcal{E}_{j})$$- \sum_{i}T_{j}^{*}ie^{-\beta(}\mathcal{E}|.-\epsilon j)(e^{\beta \mathcal{E}_{i}}pi-eq\beta\epsilon_{j)^{2}j/}(2\xi_{ii})$

Cancel.

$e^{\beta^{c_{j}}}$.

to get$T_{ji}^{*}(-I^{)}i\log p_{i}-\beta pi\mathcal{E}i)$$=$$-q_{j}\log q_{j}-\beta q_{j}\mathcal{E}j$$\sum_{1}$.

$(e^{\beta \mathcal{E}_{i}}pi-e^{\beta}q\mathcal{E}_{j)j)}2/(2\xi ij\cdot$$\sum_{i}T_{ji}^{*}e^{-\beta}\epsilon|$

Now either$e^{\beta \mathcal{E}_{i}}p_{i}<\xi_{ij}<e^{\beta \mathcal{E}_{j}}q_{j}<Z_{\beta}^{-1}C$or.

$e^{\beta \mathcal{E}_{\mathrm{J}}}q_{j}<\xi_{ij}<e^{\beta \mathcal{E}_{\mathrm{i}}}p_{i}<Z_{\beta}^{-1}C$

In either case, $1/\xi_{ij}>Z_{\beta}C^{-1}.$ Sumlning over$S(p)-\beta\langle p, \mathcal{E}\rangle$$\leq$$j$

we get$S(q)-\beta\langle q, \mathcal{E}\rangle$$\frac{Z_{\beta}}{2C}\sum_{ij}T^{*}jie^{-}\beta \mathcal{E}_{i}(e^{\beta \mathcal{E}_{i}}p_{iqj}-e^{\beta})^{2}\mathcal{E}_{j}$,which gives tlle result, in view of the proof of theorem(4).
is irreducible, thenIt follows that if is close to equilibrium and$narrow\infty$
classical, discreteas
in KMS-norm. Our theorem istime version of the deep result of Majewski [4], who shows that a continuous
stochastic semigroup on a-algebra, obeying a suitable detailed-Segal Hilbertbalance condition, is a contraction in thespace given by the KMS state. It is remarkable that he does not needto assume complete positivity. Majewski’s version of “detailed balance”
expresses microscopic reversibility in quantum mechanics, and is similar todynanimical
the formulation of Agarwal [5]. A useful account ofsemigroups can be found in [6].
$T$

$p$

$T^{*n}parrow p_{\beta}$$\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{n}\mathrm{t}_{\mathrm{U}\mathrm{n}1}$$\mathrm{t}\mathrm{l}\iota \mathrm{e}$$C^{*}$$\mathrm{G}\mathrm{e}\mathrm{l}\mathrm{f}\mathrm{a}\mathrm{n}\mathrm{d}_{- \mathrm{N}\mathrm{a}}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{r}\mathrm{k}$.antum$\mathrm{q}\mathrm{u}$4An Easy Result on Quantum Entropy

There is a parallel theory in quantum mechanics, and our notation has been-algebra
designed to bring out the similarities. The observables form a$C^{*}$

64, make up
and the states, the positive normalised linear maps :. A stochastis map is a linear map
a convex subset of the dual space$T:Aarrow A$ which takes positive operators to positive operators, and takes
takes to . We say that is bistochastic
1 to 1. It then follows that
if it is stochastic and trace-preserving, which makes sense on the class of
trace-class operators. For a state in a suitable domain we can define the.
quantum entropy of a by
I. Amah [7] has noticed that we can reduce some questions in quantum
statistical dynamics to the corresponding classical question.

$A$$\rho$$Aarrow \mathrm{R}$$A^{*}$$\Sigma$$\tau*$$\Sigma$$T$$\Sigma$$\rho$$S(\rho)=-\mathrm{T}\mathrm{r}(\rho\log\rho)$$\rho$

be a
Theorem 10 Let
Then for any normal state$T$$bi_{St_{oC}}l\iota asti_{C}$$\rho$

map on a represented$C^{*}$-algebra.

we have$S(T^{*}\rho)\geq S(\rho)$.

Proof.
are of trace class; letandThe states andare one-dimensionalandbe their spectral resolutions. We take it thatprojections, and that multiplicity is accounted for by repetition. Since bothcan be chosen mutually orthogonal. The same goesare self-adjoint, the. Define the infinite matrix
for the$\rho$$\sigma=T^{*}\rho$$\rho=\sum\rho_{i}P_{i}$$P_{i}$$\sigma=\sum_{j}\sigma_{j}Q_{j}$$Q_{j}$$P_{i}$$Q_{j}$$T_{ij}=\mathrm{T}\mathrm{r}(Q_{i}(\tau Pj))$

.

(16)

The point is that this is a Markov matrix: $\sum_{j}T_{ij}=$ Tr $(Q_{i}(T \sum_{jj}P))=$. Its elements are non-negative, since
is a
is a density matrix. In the same way, we show
positive operator, andis bistochastic in the quantum sense, thenis bistochastic.
that if. Therefore the entropies ofMoreover, it is immediate thatand obey the inequality $S(T^{*}\rho)\geq S(\rho)$ by appealing to the classicalentropy can also be estimated using a classicalresult. The gain inestimate.
$TP_{j}$$\mathrm{T}\mathrm{r}(Q_{i}\tau 1)=\mathrm{T}\mathrm{r}(Q_{i})=1$$Q_{i}$$T$$[T_{ij}]$$\sigma_{i}=\sum_{j}\tau_{ij\rho_{j}}$$\rho$$\sigma$$\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{u}\ln$5Conclusions

We have advocated the use of the dual KMS-norms for studying the convergence of Markov chains obeying detailed balance. Such an operator is shown
to be a contraction in this norm. The same proof works for a generalisationof the notion of detailed balance, which reduces to bistochasticity when beta
is zero. The concept of a state’s being close to equilibrium is formulated,

65

and it is shown that this condition is stable under one time-step. The definition gives a restriction on the class of initial states for which convergenceof the chain holds in the KMS-norm. The condition always holds if is
finite, and for all states in the salne representation space as the Gibbs statein question.
$\Omega$

References[1] R. Haag, N.M. IIugenholtz and M. Winnick, On the equilibrium states, Commun. Math. Phys., 5, 215-236
in quantum statistical(1967). See G.L. Sewell, Quantum Theory of Collective Phenomena,, 1986.
Clarendon Press,$\mathrm{m}\mathrm{e}\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{l}\dot{\mathrm{u}}\mathrm{c}\mathrm{S}$$\mathrm{O}\mathrm{x}\mathrm{f}_{0}1\backslash \mathrm{d}$[2] R.F. Streater, Entropy and KMS-Norms; Proceedings of the 1993
Madeira Advanced Study Institute “Stochastic Analysis and Applications to Physics”, (L.Streit, Editor), Kluwer, 1994.
[3] S.Koseki, Thesis, Ph.D., King’s College London University, 1993.

[4] W.A. Majewski, Dynamical Semigroups in the Algebraic Formulation of
Statistical Mechanics, Fortschritte der Physik, 25, 89-133, (1984). See
also W.A. Majewski, Journ. of Statistical Physics, 55417 (1989).

[5] G.S. Agarwal, Open Quantum Markovian Systems and Microreversibility; Zeits. fur Physik 258,409-422, 1973.
[6] R. Alicki and K. Lendl, Quantum Dynamical Semigroups and Applications. Lecture Notes in Physics, 286, 1987.
[7] I. Amah, M.Sc. Thesis, Imperial College, London University, (1993).