Lecture Notes on Statistical Mechanics & Thermodynamics
Jared Kaplan

Department of Physics and Astronomy, Johns Hopkins University

Abstract
Various lecture notes to be supplemented by other course materials.

Contents
1 What Are We Studying?

3

2 Energy in Thermal Physics
2.1 Temperature and Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Heat and Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55
7

3 Entropy and the 2nd Law
3.1 State Counting and Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Two Coupled Paramagnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Another Example: ‘Einstein Solid’ . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Ideal Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11
11
15
16
19
21

4 Interactions and Temperature
4.1 What is Temperature? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Entropy and Heat Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Paramagnets – Everything You Know Is Wrong . . . . . . . . . . . . . . . . . . . .
4.4 Mechanical Equilibrium and Pressure . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Diﬀusive Equilibrium and Chemical Potential . . . . . . . . . . . . . . . . . . . . .
4.6 Extensive and Intensive Quantities . . . . . . . . . . . . . . . . . . . . . . . . . . .

23
23
25
27
29
31
33

5 Engines and Refrigerators
5.1 Refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Real Heat Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Real Refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33
35
36
38

6 Thermodynamic Potentials

39

7 Partition Functions and Boltzmann Statistics
7.1 Boltzmann Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Average or Expectation Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 Equipartition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.4 Maxwell Speed Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.5 Partition Function and Free Energy . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.6 From Laplace Transforms to Legendre Transforms . . . . . . . . . . . . . . . . . .

45
45
47
50
51
52
53

8 Entropy and Information
8.1 Entropy in Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2 Choice, Uncertainty, and Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.3 Other Properties of Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.4 A More Sophisticated Derivation of Boltzmann Factors . . . . . . . . . . . . . . . .
8.5 Limits on Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55
55
56
58
59
59

1

9 Transport, In Brief
9.1 Heat Conduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.2 Viscosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.3 Diﬀusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60
61
62
63

10 Quantum Statistical Mechanics
10.1 Partition Functions for Composite Systems . . . . . . . . . . . . . . . . . . . . . . .
10.2 Gibbs Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.3 Bosons and Fermions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.4 Degenerate Fermi Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.5 Bosons and Blackbody Radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.6 Debye Theory of Solids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.7 Bose-Einstein Condensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.8 Density Matrices and Classical/Quantum Probability . . . . . . . . . . . . . . . . .

64
64
66
68
70
76
80
83
85

11 Phase Transitions
11.1 Clausius-Clapeyron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 Non-Ideal Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.3 Phase Transitions of Mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.4 Chemical Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.5 Dilute Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.6 Critical Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85
86
87
89
90
92
94

12 Interactions
97
12.1 Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
12.2 Interacting Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Basic Info
This is an advanced undergraduate course on thermodynamics and statistical physics. Topics
include basics of temperature, heat, and work; state-counting, probability, entropy, and the second
law; partition functions and the Boltzmann distribution; applications to engines, refrigerators, andcomputation; phase transitions; basic quantum statistical mechanics and some advanced topics.
The course will use Schroeder’s An Introduction to Thermal Physics as a textbook. We will follow
the book fairly closely, though there will also be some extra supplemental material on information
theory, theoretical limits of computation, and other more advanced topics. If you’re not planningto go on to graduate school and become a physicist, my best guess is that the subject matter andstyle of thinking in this course may be the most important/useful in the physics major, as it has
signiﬁcant applications in other ﬁelds, such as economics and computer science.

2

1

What Are We Studying?

This won’t all make sense yet, but it gives you an outline of central ideas in this course.
We’re interested in the approximate or coarse-grained properties of very large systems. You may
have heard that chemists deﬁne NA = 6.02 × 1023 as a mole, which is the number of atoms in asmall amount of stuﬀ (ie 12 grams of Carbon-12). Compare this to all the grains of sand on all of
the world’s beaches ∼ 1018 , the number of stars in our galaxy ∼ 1011 , or the number of stars in thevisible universe ∼ 1022 . Atoms are small and numerous, and we’ll be taking advantage of that fact!
Here are some examples of systems we’ll study:• We’ll talk a lot about gases. Why? Because gases have a large number of degrees of freedom,so statistics applies to them, but they’re simple enough that we can deduce their properties bylooking at one particle at a time. They also have some historical importance. But that’s it.
• Magnetization – are magnetic dipoles in a substance aligned, anti-aligned, random? Onceagain, if they don’t interact, its easy, and that’s part of why we study them!
• Metals. Why and how are metals all roughly the same (answer: they’re basically just gases ofelectrons, but note that quantum mechanics matters a lot for their behavior).
• Phase Transitions, eg gas/liquid/solid. This is obviously a coarse-grained concept that
requires many constituents to even make sense (how could you have a liquid consisting of only
one molecule?). More interesting examples include metal/superconductor, ﬂuid/superﬂuid,
gas/plasma, aligned/random magnetization, conﬁned quarks/free quarks, Higgs/UnHiggsed,
and a huge number of other examples studied by physicists.
• Chemical reactions – when, how, how fast do they occur, and why?
Another important idea is universality – in many cases, systems with very diﬀerent underlying
physics behave the same way, because statistically, ie on average, they end up being very similar.
For example, magnetization and the critical point of water both have the same physical description!
Related ideas have been incredibly important in contemporary physics, as the same tools are usedby condensed matter and high-energy theorists; eg superconductivity and the Higgs mechanism areessentially the same thing.
Historically physicists ﬁgured out a lot about thermodynamics without understanding statisticalmechanics. Note that
• Thermodynamics deals with Temperature, Heat, Work, Entropy, Energy, etc as rather abstract
macroscopic concepts obeying certain rules or ‘laws’. It works, but it’s hard to explain why
without appealing to...
• Statistical Mechanics gets into the details of the physics of speciﬁc systems and makes statisticalpredictions about what will happen. If the system isn’t too complicated, you can directly
derive thermo from stat mech. So statistical mechanics provides a much more complete
picture, and is easier to understand, but at the same time the analysis involved in stat mech ismore complicated. It was discovered by Maxwell, Boltzmann, Gibbs, and many others, whoprovided a great deal of evidence for the existence of atoms themselves along the way.
3

A Very Brief Introduction
So what principles do we use to develop stat mech?
• Systems tend to be in typical states, ie they tend to be in and evolve towards the most likely
states. This is the second law of thermodynamics (from a stat mech viewpoint). Note
S ≡ kB log N(1.0.1)is the deﬁnition of entropy, where N is the number of accessible states, and kB is a pointlessconstant included for historical reasons. (You should really think of entropy as a pure number,
simply the log of the number of accessible states.)
For example, say I’m a bank robber, and I’m looking for a challenge, so instead of stealing $100
bills I hijack a huge truck full of quarters. But I end up crashing it on the freeway, and 106 quarters
ﬂy everywhere, and come to rest on the pavement. How many do you expect will be heads vs tails?
The kind of reasoning you just (intuitively) employed is exactly the sort of thinking we use in statmech. You should learn basic statistical intuition well, as its very useful both in physics, and in therest of science, engineering, and economics. It’s very likely the most universally important thing you
can learn well as a physics undergraduate.
Is there any more to stat mech than entropy? Not much! Only that
• The laws of physics have to be maintained. Mostly this means that energy is conserved.
This course will largely be an exploration of the consequences of basic statistics + physics, which
will largely mean counting + energy conservation.
Say I have 200 coins, and I restrict myself to coin conﬁgurations with the same number of heads
and tails. Now I start with the heads in one pile and the tails in the other (100 each). I play a gamewhere I start randomly ﬂipping a head and a tail over simultaneously. After a while, will the two
100-coin piles still be mostly heads and mostly tails? Or will each pile have roughly half heads andhalf tails? This is a pretty good toy model for heat transfer and entropy growth.
Connections with Other Fields
The methods of this course are useful in a broad range of ﬁelds in and outside of physics:• In stat mech we learn how to sum or average over distributions of states. We do somethingquite similar in Quantum Mechanics. In fact, there are deep and precise connections between
the partition function in stat mech and the path integral in quantum mechanics.
• Information Theory concerns how much information we can store, encode, and transmit, andis also built on the idea of entropy (often called Shannon Entropy in that context).
• Because gases are easy, engines use them, and they were important to the development ofthermo and stat mech, we’ll talk somewhat about how thermodynamics places fundamental
limitations on the eﬃciency of engines. There are similar constraints on the eﬃciency ofcomputation, which are just as easy to understand (though a bit less pessimistic).
4

• Machine Learning systems are largely built on the same conceptual foundations. In particular,‘learning’ in ML is essentially just statistical regression, and diﬀerences between ML models
are often measured with a version of entropy. More importantly, even when ML systems are
deterministic algorithms, they are so complicated and have so many parameters that we use
statistical reasoning to understand them – this is identical to the situation in stat mech, where
we describe deterministic underlying processes using probability and statistics. (It’s also what
we do when we talk about probability in forecasting, eg when we say there’s a 30% probability
someone will win an election.)

2

Energy in Thermal Physics

2.1

Temperature and Energy

Temperature
Temperature is an intrinsic property of both of two systems that are in equilibrium, ie that are incontact much longer than a relaxation time.
• Contact can be physical, or it can be established through radiation, or via mutual conduct
with some other substance/ﬂuid. But heat must be exchanged for temperature...
• Other kinds of equilibrium might be energy = thermal, volume = mechanical, particles= diﬀusive.
• Objects that give up energy are at a higher temperature.
So more speciﬁcally, objects at higher temperature spontaneously give up heat energy to objects atlower temperature. From the point of view of Mechanics or E&M, this is weird! Nothing like this istrue of momentum, charge, force, ie they aren’t spontaneously emitted and absorbed!?
• Operationally, use a thermometer. These are usually made of substances that expand whenhotter.
• Scale is basically arbitrary for now. Just mark some points and see how much expansion
happens for a single material! Celsius from boiling water vs freezing.
• Diﬀerent materials (eg mercury vs metal) expand a bit diﬀerently, so precise measurements
require care and more thought. But in many cases expansion is linear in temperature, so we
get same approximate results.
Hmm, are any materials more interesting than others for this purpose?
What if we use a gas to measure temperature?
• Operationally, a gas can keep on shrinking. In fact, at some temperature its volume or pressure
will go to zero!
• This temperature is called absolute zero, and deﬁnes the zero point of the absolute temperature scale.
5Ideal Gas and the Energy-Temperature Relation
Gases are the harmonic oscillator of thermodynamics – the reason is that we want to study a large
number of very simple (approximately independent!) degrees of freedom in a fairly intuitive context.
Everyone breathes and has played with a balloon.
Low-density gases are nearly ideal, and turn out to satisfy (eg experiments demonstrate this)P V = N kT

(2.1.1)

where k is Boltzmann’s constant (which is sort of arbitrary and meaningless, as we’ll see, but wehave to keep it around for historical reasons).
• Note that Chemists use Avogadro’s number NA = 6.02 × 1023 and nR = N k where n is molar
density. This is also totally arbitrary, but it hints at something very important – there arevery big numbers ﬂoating around!
• Of course the ideal gas law is an approximation, but it’s a good approximation that’s importantto understand.
Why are we talking about this? Because the relationship between temperature and energy
is simple for an ideal gas. Given• the ideal gas law• the relationship between pressure and energy
we can derive the relationship between temperature and energy.
Consider a molecule of gas and piston with area A controlling a volume V = LA. Let’s imagine
its only a single molecule, as that’s equivalent to many totally non-interacting molecules. The
average pressure is
� x
m Δv
F¯x,onpistonF¯x,onmoleculeΔt(2.1.2)
P̄ =
=−
=−
A
A
A
Its natural to use the double-round-trip timeδt = 2Lvx(2.1.3)Note thatΔvx = −2vx

(2.1.4)m (−2vx )mvx2=
VA 2L/vx(2.1.5)

So we ﬁnd thatP̄ = −

6

The two factors of vx come from the momentum transfer and from the time between collisions.
If we have N molecules, the total pressure is the sum, so we can writeP̄ V = N mv¯x2(2.1.6)

So far we haven’t made any assumptions. But if we assume the ideal gas law, we learn that11
kT = mv¯x2
2
2(2.1.7)or the average total kinetic energy is31 �
kT = m v¯x2 + v¯y 2 + v¯z2
(2.1.8)2
2
Cool! We have related energy to temperature, by assuming the (empirical for now) ideal gas law.
We can also get the RMS speed of the molecules as
3kT(2.1.9)m
Is the sum or average of the squares of numbers equal to the sum squared? So this isn’t the average
speed, but it’s pretty close. (What does it being close tell us about the shape of the distribution of
speeds?)Our result is true for ideal gases. The derivation breaks down at very low temperatures, whenthe molecules get very close together, though actually the result is still true (just not our argument).
We’ll discuss this later in the course.
v̄ 2 =

Equipartition of Energy
The equipartition theorem, which we’ll prove later in the course, says that all ‘quadratic’ degrees of
freedom have average energy = 12 kT . This means kinetic energies and harmonic oscillator potential
energies:• Atoms in molecules can rotate, and their bonds can vibrate.
• Solids have 3 DoF per molecular motion and 3 DoF for the potential energy in bonds.
• Due to QM, some motions can be frozen out, so that they don’t contribute.
This is all useful for understanding how much heat energy various substances can absorb.

2.2

Heat and Work

Book says that we can’t deﬁne energy... actually, we can deﬁne it as the conserved quantity associatedwith time translation symmetry, for those who know about Noether’s theorem.
Recall that temperature is a measure of an object’s tendency to spontaneously give up
energy... temperature often increases with energy, but not always, and it’s important not to confuse
temperature with energy or heat.
In thermodynamics we usually discuss energy in the form of heat or work.
7• Heat is the spontaneous ﬂow of energy due to temperature diﬀerences.
• Work is any other transfer of energy... you do work by pushing a piston, stirring a cup, runningcurrent through a resistor or electric motor. In these cases the internal energy of these systems
will increase, and they will usually ‘get hotter’, but we don’t call this heat.
• Note that these are both descriptions of energy transfers, not total energies in a system.
These aren’t the everyday usages of these terms.
If I consider shining a ﬂashlight with an incandescent bulb on something, I’m honestly not clear
if that’s heat or work – it’s work since it’s intentional (and the electrical ‘heating’ of the ﬁlament is
deﬁnitely work), but when the ﬂashlight emits light, it’s actually ‘spontaneous’ thermal radiations,so the classiﬁcation is unclear.
The change in the internal energy of a system is
ΔU = Q + W(2.2.1)and this is just energy conservation, sometimes called the ﬁrst law of thermodynamics.
• Note these are the amounts of heat and work entering the system.
• Note that we often want to talk about inﬁnitessimals, but note that ‘dQ’ and ‘dW ’ aren’t thechange in anything, since heat and work ﬂow in or out, they aren’t properties of the systemitself.
Processes of heat transfer are often categorized as• Conduction is by molecular contact• Radiation is by emission of electromagnetic waves• Convection is the bulk motion of a gas or liquid
Compression WorkFor compression work we have
~ → F Δx
W = F~ · dx

(2.2.2)for a simple piston. If compression happens slowly, so the gas stays in equilibrium, then I can replace
W = P AΔx = −P ΔV(2.2.3)This is called quasistatic compression. To leave this regime you need to move the piston at a speedof order the speed of sound or more (than you can make shock waves).

8That was for inﬁnitesimal changes, but for larger changes we have
Z Vf
W =−
P (V )dV(2.2.4)Vi

as long as the entire process is quasistatic.
Now let’s consider two types of compressions/expansions of an ideal gas, where we hold diﬀerent
quantities constant. These are• isothermal – holds temperature constant• adiabatic – no heat ﬂow• Which do you think is slower??
For an isotherm of an ideal gas we have
P =

N kT
V

with T ﬁxed, and so it’s easy to compute
Z Vf
N kT
Vi
W =−
dV = N kT log
V
Vf
Vi(2.2.5)(2.2.6)as the work done on the gas. If it contracts the work done on the gas is positive, if it expands thework done on the gas is negative, or in other words it does work on the environment (ie on thepiston). How much heat ﬂows? We just have
Q = ΔU − W = −W

(2.2.7)so heat is equal to work.
What if no heat is exchanged? By equipartition we have
U=

f
N kT
2

(2.2.8)where f is the number of degrees of freedom per molecule. If f is constant then
dU =

f
N kdT
2

(2.2.9)so that we havef
N kdT = −P dV
2(2.2.10)

f dT
dV
=−
2 T
V

(2.2.11)which along with ideal gas gives9

so that
V T f /2 = constant(2.2.12)V γ P = constant(2.2.13)orwhere γ = 1 + f2 is the adiabatic exponent.
Heat Capacities
Heat capacity is the amount of heat needed to raise its temperature. The speciﬁc heat capacity
is the heat capacity per unit mass.
As stated, speciﬁc heat capacity is ambiguous because Q isn’t a state function. That isQ
ΔU − W=(2.2.14)ΔT
ΔTIt’s possible that U only depends on T (but likely not!), but W deﬁnitely depends on the process.
C=• Most obvious choice is W = 0, which usually means heat capacity at constant volume


∂U
CV =
(2.2.15)∂T V
This could also be called the ‘energy capacity’.
• Objects tend to expand when heated, and so its easier to use constant pressure






ΔU + P ΔV
∂U
∂V
CP =
+P
=
∂T P
∂T P
ΔT
P

(2.2.16)

For gases this distinction is important.
Interesting and fun to try to compute CV and CP theoretically.
When the equipartition theorem applies and f is constant, we ﬁnd


∂U
∂
f
f
CV =
=
N kT = N k
∂T
∂T 2
2

(2.2.17)

For a monatomic gas have f = 3, while gases with rotational and vibrational DoF have (strictly)
larger f . So CV allows us to measure f .
Constant pressure? For an ideal gas the only diﬀerence is the addition of∂V∂ N kTNk==(2.2.18)
∂T P
∂T P
P
so that
CP = CV + N k
Interestingly CP is independent of P , because for larger P the gas just expands less.
10(2.2.19)

Phase Transitions and Latent Heat
Sometimes you can put heat in without increasing the temperature at all! This is the situation
where there’s Latent Heat, and usually occurs near a ﬁrst order phase transition. We say that theQmL≡(2.2.20)is the latent heat per mass needed to accomplish the transformation. Conventionally this is atconstant pressure, as its otherwise ambiguous. Note it takes 80 cal/g to melt water and 540 cal/g toboil it (note it takes 100 cal/g to heat water from freezing to boiling).
Enthalpy
Since we often work at constant pressure, it’d be convenient if we could avoid constantly having toinclude compression-expansion work. We can do this be deﬁning the enthalpy
H = U + PV

(2.2.21)

This is the total energy we’d need to create the system out of nothing at constantpressure. Or if you annihilate the system, you could get H back having the atmosphere contract!
It’s useful because
ΔH = ΔU + P ΔV
= Q + Wotherwhere Wother is non-compression work. If no other form of work, ΔH = Q. Means


∂H
CP =
∂T P

(2.2.22)(2.2.23)

Could all CP the “enthalpy capacity” while CV is “energy capacity”. No need for any heat at all,since could just be ‘other’ work (eg a microwave oven).
For example, boiling a mole of water at 1 atmosphere takes 40, 660 J, but via ideal gas
P V = RT ≈ 3100 J(2.2.24)at room temperature, so this means 8% of the energy put in went into pushing the atmosphere away.

3

Entropy and the 2nd Law

3.1

State Counting and Probability

Why do irreversible processes seem to occur, given that the laws of physics are reversible? The
answer is that irreversible processes are overwhelmingly probable.
11

Two-State Systems, Microstates, and Macrostates
Let’s go back to our bank robber scenario, taking it slow.
If I have a penny, a nickel, and a dime and a ﬂip them, there are 8 possible outcomes. If the coins
are fair, each has 1/8 probability. Note that there are 3 ways of getting 2 heads and a tail, so theprobability of that outcome (2 heads and a tail) is 3/8. But if I care about whether the tail comesfrom the penny, nickel, or dime, then I might want to discriminate among those three outcomes.
This sort of choice of what to discriminate, what to care about, or what I can actually distinguishplays an important role in stat mech. Note that
• A microstate is one of these 8 diﬀerent outcomes.
• A macrostate ignores or ‘coarse-grains’ some of the information. So for example just giving
information about how many heads there were, and ignoring which coin gave which H/Toutcome, deﬁnes a macrostate.
• The number of microstates in a macrostate is called the multiplicity of the macrostate. We
often use the symbol Ω for the multiplicity.
If we label macrostates by the number of heads n, then for n = 0, 1, 2, 3 we ﬁndp(n) =

Ω(n)1 3 3 1
= , , ,Ω(all)
8 8 8 8(3.1.1)

How many total microstates are there if we have N coins? It’s 2N , as each can be H or T. What are
the multiplicities of the macrostates with n heads with N = 100? Well note thatΩ(0) = 1,

Ω(1) = 100,

Ω(2) =

100 × 99,2

Ω(3) =

100 × 99 × 98
3×2

(3.1.2)

In general we haveΩ(n) =N!
n!(N − n)!

(3.1.3)

which is N choose n. So this gives us a direct way to compute the probability of ﬁnding a given
number of heads, even in our bank robbery scenario where N = 106 .
We can immediately make this relevant to physics by studying the simplest conceivable magnetic
system, a two-state paramagnet. Here each individual magnetic dipole has only two states(because of QM), so it can only point up or down, and furthermore each dipole behaves as though
its magnetic moment is independent of the others (they interact very weakly... in fact, in our
approximation, they don’t interact at all). Now H/T just becomes ↑ / ↓, so a microstate is· · · ↑↑↑↓↑↓↓↑↓↓↑ · · ·(3.1.4)

while a macrostate is speciﬁed by the total magnetization, which is just N↑ − N↓ = 2N↑ − N , andmultiplicity
Ω(N↑ ) =N!
N↑ !N↓ !
12(3.1.5)

This really becomes physics if we apply an external magnetic ﬁeld, so that the system has anenergy corresponding with how many little dipoles are aligned or anti-aligned with the external ﬁeld.
Silly Comments on Large Numbers
We saw that NA = 6.02 × 1023 . But if we have that many spins, and each can be up or down, then...
we get really large numbers! Let’s note what’s worth keeping track of here, eg consider
1023 + 50(3.1.6)or
23

8973497 × 1010

(3.1.7)

With very very large numbers, units don’t even matter! Be very careful though, because in many
calculations the very very large, or just very large numbers cancel out, and so you really do have tocarefully keep track of pre-factors.
Important Math
Now we will derive the special case of a very universal result, which provides information about theaggregate distribution of many independent random events.
Since N is large, N ! is really large. How big? One way to approximate it is to note thatN ! = elog N ! = e

PN

n=1 log n(3.1.8)dx log x = N (log N − 1)

(3.1.9)Now we can approximate
N
X
n=1

Z N
log n ≈
0which says thatN ! ∼ N N e−N(3.1.10)

This is the most basic part of Stirling’s Approximation to the factorial. The full result is√(3.1.11)
N ! ≈ N N e−N 2πN
You can derive this more precise result by taking the large N limit of the exact formulaZ ∞N! =xN e−x dx
013(3.1.12)That was a warm-up to considering the distribution of H/T, or spins in a paramagnet.
N!
n!(N − n)!
N N e−N≈nn e−n (N − n)N −n en−NΩ(n) =(3.1.13)

Now if we write n = xN then this becomesNN
xxN N xN (1 − x)N (1−x) N N (1−x)
N1≈xx (1 − x)(1−x)Ω(n) ≈≈ eN (−x log x−(1−x) log(1−x))(3.1.14)

The prefactor of N in the exponent means that this is very, very sharply peaked at its maximum,
which is at x = 12 . Writing x = 12 + y we have4
−x log x − (1 − x) log(1 − x) = log 2 − 2y 2 − y 4 + · · ·3(3.1.15)so that


1+y
2
Ω n=
N
≈ 2N e−2N y2≈ 2N e−2(2n−N )2N

(3.1.16)This is a special case of the central limit theorem, which is arguably the most important resultin probability and statistics. The distribution we have found is the Gaussian Distribution, orthe ‘bell curve’. Its key feature is its variance, which says intuitively thatn−√N
. few × N
2

(3.1.17)√
where here N is the standard deviation. Thus we expect that 2nN−N . √1N  1.
This means that if we ﬂip a million fair coins, we expect the diﬀerence between the number ofheads and the number of tails to be of order 1000, and not of order 10, 000. And if the diﬀerence
was only, say, 3, then we might be suspicious that the numbers are being faked.
Paramagnets and Random Walks
More relevantly for physics, this tells us that if we have ∼ 1023 dipoles in a paramagnet (recall
Avogadro’s number), with their dipole moments randomly√ arranged, then the total contribution
to the magnetic moment we should expect is of order ∼ 1023 ≈ 3 × 1011 . That seems like a big
14

number, but it’s a very small fraction of the total, namely ∼ 10−11 fraction. Thus the diﬀerence
between up and down spins is minute.
We have also largely solved the problem of a (1-dimensional) random walk. Note that ‘heads’
and ‘tails’ can be viewed as steps to the right or left. Then the total nH − nT is just the numberof steps taken to the√ right. This means that if we take N random steps, we should expect to be a
distance of order N from our starting point. This is also an important, classic result.

3.2

Two Coupled Paramagnets

With two coupled paramagnets, we can study heat ﬂow and irreversible processes.
But to have a notion of energy, we need to introduce a background magnetic ﬁeld, so that ↑ hasenergy µ and ↓ has energy −µ. Then if we have 100 magnetic moments in a Paramagnet, the energyof the system can range from−100µ, −98µ, −96µ, · · · , 0, · · · 98µ, 100µ(3.2.1)

Note that there’s a unique conﬁguration with minimum and maximum energy, whereas there are100!
conﬁgurations with energy 0.
50!50!
So let’s assume that we have two paramagnets, which can exchange energy between them byﬂipping some of the dipoles.
• We will treat the two paramagnets as weakly coupled, so that energy ﬂows between themslowly and smoothly, ie energy exchange within each paramagnet is much faster than between
the two. We’ll call these energies UA and UB .
• Utotal = UA + UB is ﬁxed, and we refer to the macrostate just via UA and UB . Note that the
number of spins in each paramagnet NA and NB are ﬁxed.
The total number of states is just
Ωtot = ΩA ΩB = Ω(NA , UA )Ω(NB , UB )(3.2.2)

Now we will introduce the key assumption of statistical mechanics:• In an isolated system in thermal equilibrium, all accessible (ie possible) microstates are equallyprobable.
A microstate might be inaccessible because it has the wrong energy, or the wrong values of conserved
charges. Otherwise it’s accessible.
Microscopically, if we can go from X → Y , we expect we can also transition from Y → X (thisis the principle of detailed balance). Really there are so many microstates that we can’t possible‘visit’ all of them, so the real assumption is that the microstates we do see are a representative
sample of all the microstates. Sometimes this is called the ‘ergodic hypothesis’ – averaging overtime is the same as averaging over all accessible states. This isn’t obvious, but should seem plausible– we’re assuming the transitions are ‘suﬃciently random’.
15

For simplicity, let’s imagine that the total energy is UA + UB = 0, and the two paramagnets
have NA = NB = N dipoles each. How many conﬁgurations are there with all spins up in one
paramagnet, and all spins down in the other? Just one! But with n up spins in the ﬁrst paramagnet
and N − n up spins in the second, there are(2N )!
(N )!
×Ωtot = Ω(N, (N − n)µ)Ω(N, (n − N )µ) =n!(N − n)! n!(N − n)!

2
(N )!
=
n!(N − n)!

(3.2.3)Thus the number of states is vastly larger when n = N . In fact, we can directly apply our analysis
of combinatorics√ to conclude that the number of states has an approximately Gaussian distribution
with width N .
Note that this is mathematically identical to the coin ﬂipping example from the ﬁrst lecture.

3.3

Another Example: ‘Einstein Solid’

The book likes to use a simple toy model where a ‘solid’ is represented as a system of N identicalbut separate harmonic oscillators. They are treated as quantum oscillators, which means that they
have energy1E=+ n ~ω
(3.3.1)
2
for some non-negative integer n. And actually we’ll drop the 1/2, as only energy diﬀerences are
relevant. Also, since all energies are multiples of ~ω, we’ll just refer to this as an ‘energy unit’ andnot keep track of it.
This wasn’t Einstein’s greatest achievement by any means, and it does not describe real solids.
But it’s an easy toy example to play with, as we’ll see. More realistic models would require fancier
combinatorics.
If the total ES has E units of energy and N oscillators, then the number of states isΩ(N, E) =(E + N − 1)!
E!(N − 1)!

(3.3.2)

Let’s brieﬂy check this for N = 3, then we have
Ω(3, 0) = 1, Ω(3, 1) = 3, Ω(3, 2) = 6, Ω(3, 3) = 10, · · ·(3.3.3)

We can prove our general formula by writing a conﬁguration as•| • •|| • • • •| • | • •||| • •(3.3.4)

where • denote units of energy and | denote transitions between oscillators. So this expression has
E = 12 and N = 9. Thus we just have a string with N − 1 of the | and E of the •, we can put the |and • anywhere, and they’re indistinguishable.
16

If this reasoning isn’t obvious, ﬁrst consider how to count a string of N digits, say N = 7. A
conﬁguration might be
4561327(3.3.5)

7 × 6 × 5 × 4 × 3 × 2 × 1 = 7!

(3.3.6)and so it’s clear that there are

conﬁgurations. But in the case of the • and |, there are only two symbols, so we divide to avoid
over-counting identical conﬁgurations.
Two Coupled Einstein Solids
The formula for counting conﬁgurations for an Einstein solid is very (mathematically) similar tothat for counting states of a paramagnet. But it’s slightly diﬀerent in its dependence on energy andnumber of oscillators.
If we ﬁx E = EA + EB for the two ESs, then the number of conﬁgurations is(EA + NA − 1)! (EB + NB − 1)!
×
EB !(NB − 1)!
EA !(NA − 1)!
1(EA + NA − 1)!(E − EA + NB − 1)!
=(NA − 1)!(NB − 1)!
EA !(E − EA )!

Ωtot =(3.3.7)

so we can study how many conﬁgurations there are as a function of EA , where we ﬁx the totalenergy E and the number of oscillators in each ES.
It’s not so easy to get intuition about this formula directly, so let’s approximate it. First we
will just study Ω(E, N ) itself. An interesting approximation is the high-temperature and (mostly)classical limit, where E  N , so that each oscillator will tend to have a lot of energy (that is E/Nis large).
(If we view the ‘pool’ of total energy as very large, then we could assign each oscillator to haveenergy randomly chosen between 0 and 2E/N , so that overall the total energy works out. This
would give us (2E/N )N conﬁgurations, which is very close to the right answer.)
We already did the relevant math. But once again, using Stirling’s formulalog(E + N )!
≈ (E + N ) log(E + N ) − E − N − E log E + E − N log N + N
E!N !
≈ (E + N ) log(E + N ) − E log E − N log N(3.3.8)In the large E/N limit, we haveN
log(E + N ) = log E + log 1 +EN≈ log E +E
17



(3.3.9)

This then gives thatlog Ω ≈ N log

E+N
N(3.3.10)which means thatN

Ω(N, E) ≈ e



E
N

N
(3.3.11)

We can make sense of this by noting that this is roughly equal to the number of excited energy
levels per oscillator, E/N , raised to a power equal to the number of oscillators.
Thus using this approximation, we see that

N 
N
EA A E − EA B
Ωtot ≈(3.3.12)NB
NAThis is maximized as a function of EA when
EA =

NA
E
NA + NB(3.3.13)which is when the two ESs have energies proportional to the number of oscillators in each.
Repeating the math we used previously, we can parameterize
EA =

NA
E+x
NA + NB(3.3.14)

and we ﬁnd that
NB
NA 
E
x
E
x
≈
+
−
N
NA
N
N


B


E
x
E
x
−
+
+ NB log≈ exp NA log
NA
N
NB
N
"
 N

2 #
E
N
Nx
≈
exp −
E
N
2NA NBΩtot(3.3.15)

Thus we see that we have another Gaussian distribution
in x/ρ, where ρ = E/N is the energyqdensity. The exponent becomes large whenever xρ > NANNB . This means that (assuming NA ∼ NB )x/ρ is much much larger than 1, but much much smaller than N .
The punchline is that conﬁgurations that maximize Ω are vastly more numerous than thosethat do not, and that macroscopic ﬂuctuations in the energy are therefore (beyond) astronomicallyunlikely. Speciﬁcally, since N & 1023 in macroscopic systems, we would need to measure energies to
11 sig ﬁgs to see these eﬀects.
The limit where systems grow to inﬁnite size and these approximations become arbitrarily goodis called the thermodynamic limit.
18

3.4

Ideal Gases

All large systems – ie systems with many accessible states or degrees of freedom – have the property
that only a tiny fraction of macrostates have reasonably large probabilities. Let us now apply thisreasoning to the ideal gas. What is the multiplicity of states for an ideal gas?
In fact, in the ideal limit the atoms don’t interact, so... what is the multiplicity of states for asingle atom in a box with volume V ?
If you know QM then you can compute this exactly. Without QM it would seem that there are an
inﬁnite number of possible states, since the atom could be anywhere (it’s location isn’t quantized!).
But we can get the right answer without almost any QM.
• First note that the number of states must be proportional to the volume V .
• But we can’t specify a particle’s state without also specifying its momentum. Thus the numberof states must be proportional to the volume in momentum space, Vp .
Note that the momentum space volume is constrained by the total energy p~2 = 2mU .
We also need a constant of proportionality, but in fact that’s (essentially) why Planck’s constant~ exists! You may have heard that δxδp > h/2, ie this is the uncertainty principle. So we can write
Ω1 =

V Vp
h3

(3.4.1)by dimensional analysis. You can conﬁrm this more precisely if you’re familiar with QM for a freeparticle in a box.
If we have two particles, then there are two added complications• We must decide if the particles are distinguishable or not. For distinguishable particles we’dhave Ω21 , while for indistinguishable particles we’d have 12 Ω21 .
• We need to account for momentum conservation as p~21 + p~22 = 2mU . With more particles wehave a sum over all of their momenta.
For N indistinguishable particles we have
ΩN =

1 VN
× (momentum hypersphere area)
N ! h3N(3.4.2)where the momentum hypersphere is the volume in momentum space, constrained by momentum
conservation.
The surface area of a D-dimensional sphere isA=

2π D/2 D−1
R( D2 − 1)!

19(3.4.3)

We know that R =in gives√
2mU , where U is the total energy of the gas, and that D = 3N . Plugging this

ΩN =

3N −1
1 V N 2π 3N/2
(2mU ) 2
3N
3N
N ! h ( 2 − 1)!

3N
1 π 3N/2 V N≈(2mU ) 2
3N
3N
N ! ( 2 )! h

(3.4.4)

where we threw away some 2s and −1s because they are totally insigniﬁcant. This eliminates thedistinction between ‘area’ and ‘volume’ in momentum space, because momentum space has theenormous dimension 3N . With numbers this big, units don’t really matter!
Note that we can write this formula asΩN = f (N )V N U 3N/2

(3.4.5)so the dependence on Volume and Energy are quite simple.
Interacting Ideal Gases
Let’s imagine that we have two ideal gases in a single box with a partition. Then we just multiply
multiplicities:Ωtot = f (N )2 (VA VB )N (UA UB )3N/2(3.4.6)Since UA + UB = Utot , we see that multiplicity is maximized very sharply by UA = UB = Utot /2. And
the peaks are very, very sharp as a functions of UA – we once again have a Gaussian distribution for
ﬂuctuations about this equilibrium. Though the exact result is the simpler function(x(1 − x))P

(3.4.7)for very, very large P = 3N/2. It should be fairly intuitive that this function is very sharply
maximized at x = 1/2.
If the partition can move, then it will shift around so that
VA = VB = Vtot /2(3.4.8)

as well. This will happen ‘automatically’, ie the pressures on both sides will adjust to push thepartition into this conﬁguration. Once again, the peak is extremely sharp.
Finally, we could imagine NA 6= NB to start, but we could poke a hole in the partition to see
how the molecules move. Here the analysis is a bit more complicated – it requires using Stirling’s
formula again – but once again there’s a very, very sharply peaked distribution.
Note that as a special case, putting all the molecules on one side is 2−N less likely than having
them roughly equally distributed. This case exactly like ﬂipping N coins!

20

3.5

Entropy

We have now seen in many cases that the highest multiplicities macrostates are vastly, vastly more
numerous than low-multiplicity microstates, and that as a result systems tend to evolve towards
these most likely states. Multiplicity tends to increase.
For a variety of reasons, it’s useful to replace the multiplicity with a smaller number, its logarithm,
which is the entropy
S ≡ log Ω(3.5.1)For historical reasons, in physics we actually deﬁne
S ≡ k log Ω(3.5.2)where k is the largely meaningless but historically necessary Boltzmann constant.
Notice that because logarithms turn multiplication into addition, we have
Stot = SA + SB

(3.5.3)

when Ωtot = ΩA × ΩB . This is quite convenient for a variety of reasons. For one thing, the entropy isa much smoother function of other thermodynamic variables (ie it doesn’t have an extremely sharppeak).
Since the logarithm is a monotonic function, the tendency of multiplicity to increase is the samething as saying that entropy tends to increase:δS ≥ 0(3.5.4)That’s the 2nd law of thermodynamics!
Note that the 2nd law picks an arrow of time. That’s a big deal. This arrow of time reallycomes from the fact that initial conditions in the universe had low entropy. (If you start with a low
entropy ﬁnal condition and run physics backwards, entropy would increase as time went backward,
not forward.)
Can you oppose the 2nd law and decrease entropy through intelligence and will? No. You canwork hard to decrease the entropy in one system, but always at the cost of far more entropy creationelsewhere. Maxwell discussed this idea as ‘Maxwell’s Demon’.

21

Entropy of an Ideal Gas
We can compute the entropy of an ideal gas from the multiplicity formula"#3N
1 π 3N/2 V N
S = log(2mU ) 2
3N
N ! ( 3N
h
)!
2




3
3
3N
V
= N k (1 − log N ) + π +1 − log+ log 3 + log 2mU
2
2
2
h
"

3/2 !#
5V 4πmU= Nk+ log
2
N 3N h2"3/2 !#
5
1 4πmu
= Nk
+ log
2
n
3h2

(3.5.5)where u = U/N and n = N/V are the energy density and number density, respectively. Apparently
this is called the Sackur-Tetrode equation, but I didn’t remember that.
What would this result have looked like if we had used distinguishable particles??? The problem
associated with this is called the Gibbs paradox. We’ll talk about extensive vs intensive quantities
more thoroughly later.
Entropy dependence on volume (with ﬁxed U and N ) is simplest, ie just
ΔS = N k log

VfVi(3.5.6)Note that this makes sense in terms of doubling the volume and gas particles being in either the leftor right half of the new container.
This formula applies both to expansion that does work, eg against a piston (where the gas
does work, and heat ﬂows in to replace the lost internal energy) and to free expansion of a gas.
These are quite diﬀerent scenarios, but result in the same change in entropy.
What about increasing the energy U while ﬁxing V and N ? Where did the exponent of 3/2
come from, and why does it take that value?
Entropy of Mixing
Another way to create entropy is to mix two diﬀerent substances together. There are many morestates where they’re all mixed up than when they are separated, so mixing creates entropy.
One way to think about it is if we have gas A in one side of a partition, and gas B on the otherside, then when they mix they each double in volume, so we end up withΔS = (NA + NB )k log 2(3.5.7)in extra entropy. It’s important to note that this only occurs if the two gases are diﬀerent(distinguishable).
22

Note that doubling the number of molecules of the same type of gas in a ﬁxed volume does notdouble the entropy. This is because while out front N → 2N , we also have the log n1 → log 21n , andso the change in the entropy is strictly less than a factor of 2.
(Now you may wonder if the entropy ever goes down; it doesn’t, because for N (X − log N ) to bea decreasing function of N , we must have S < 0, which means a multiplicity less than 1!)
In fact, the very deﬁnition of entropy strongly suggests that S ≥ 0 always.
A Simple Criterion for Reversible Processes
Reversible processes must have
ΔS = 0(3.5.8)because otherwise they’d violate the second law of thermodynamics! Conversely, processes thatcreate entropy ΔS > 0 must be irreversible.
• Free expansion, or processes requiring heat exchange will increase entropy. We’ll see soon thatreversible changes in volume must be quasistatic with W = −P ΔV , and no heat ﬂow.
• From a quantum point of view, these quasistatic processes slowly change the energy levels of
atoms in a gas, but they do not change the occupancy of any energy levels. This is what keeps
multiplicity constant.
• We saw earlier that heat ‘wants’ to ﬂow precisely because heat ﬂow increases entropy(multiplicity). The only way for heat to ﬂow without a change in entropy is if it ﬂows
extremely slowly, so that we can take a limit where multiplicity almost isn’t changing.
To see what we mean by the last point, consider the entropy of two gases separated by a partition.
As a function of the internal energies, the total entropy is
3
Stot = N k log ((Utot − δU )(Utot + δU ))2(3.5.9)So we see that as a function of δU , the multiplicity does not change when δU  Utot . More precisely
∂Stot
=0
∂δU

(3.5.10)at δU = 0. So if we keep δU inﬁnitesimal, we can cause heat to move energy across the partition
without any change in entropy or multiplicity. In fact, this idea will turn out to be important...

4Interactions and Temperature

4.1

What is Temperature?

Early in the course we tentatively deﬁned temperature as a quantity that’s equal when two substances
are in thermal equilibrium. And we know it has something to do with energy. But what is it really?
23

The only important ideas here are entropy/multiplicity tend to increase until we reachequilibrium, so they stop increasing when we’re in equilibrium, and energy can be shared, but
total energy is conserved.
If we divide our total system into sub-systems A and B, then we have UA + UB = Utot is ﬁxed,
and total entropy Stot = SA + SB is maximized. This means that
∂Stot
=0
∂UA
equilibrium(4.1.1)But∂SA ∂SB∂Stot=+∂UA ∂UA
∂UA
∂SA
∂SB
=
−
∂UA ∂UB(4.1.2)where to get the second line we used energy conservation, dUA = −dUB . Thus we see that at
equilibrium
∂SB
∂SA
=
∂UB
∂UA

(4.1.3)So the derivative of the entropy with respect to energy is the thing that’s the same between
substances in thermal equilibrium.
If we look at units, we note that the energy is in the denominator, so it turns out that temperatureis really deﬁned to be


1
∂S
≡
(4.1.4)T∂U N,V
where the subscript means that we’re holding other quantities like the number of particles and thevolume ﬁxed.
Let’s check something basic – does heat ﬂow from larger temperatures to smaller temperatures?
The inﬁnitessimal change in the entropy is


1
1
1
1
δS =
δUA +
δUB =
−
δUA
(4.1.5)TB
TA TB
TA
So we see that to increase the total entropy (as required by the 2nd law of thermodynamics), if
TA < TB then we need to have δUA > 0 – that is, the energy of the low-temperature sub-systemincreases, as expected.
What if we hadn’t known about entropy, but were just using multiplicity? Then we would havefound that
∂ΩA
∂ΩB
∂Ωtot
=
Ω B + ΩA
∂UA
∂UA
∂UA


1 ∂ΩA
1 ∂ΩB
= ΩA ΩB−(4.1.6)
ΩA ∂UA ΩB ∂UB
24

and we would have learned through this computation that entropy was the more natural quantity,rather than multiplicity. I emphasize this to make it clear that entropy isn’t arbitrary, but isactually a natural quantity in this context.
Note that we now have a crank to turn – whenever there’s a quantity X that’s conserved,
we can consider what happens when two systems are brought into contact, and ﬁnd the equilibrium
conﬁguration where S(XA , XB ) is maximized subject to the constraint that XA + XB is constant.
We can do this for X = Volume, Number of Particles, and other quantities.
ExamplesFor an Einstein Solid, we computed S(U ) a while ago and foundS = N k loguU= N k log
+ Nk

N

(4.1.7)

where  is some energy unit in the oscillators. So the temperature is
T =

U
Nk

(4.1.8)This is what the equipartition theorem would predict, since a harmonic oscillator has a kinetic
energy and a potential.
More interestingly, for an monatomic ideal gas we found3
S = N k log V + N k log U + log f (N )2

(4.1.9)3
U = N kT2

(4.1.10)

Thus we ﬁnd thatwhich once again agrees with equipartition.

4.2

Entropy and Heat Capacity

We can use our newfound knowledge of temperature to deﬁne and understand the heat capacity. It’s
important to know what we’re holding constant; at constant volume we deﬁne∂U(4.2.1)CV ≡∂T N,V
which you might also think of as the energy capacity.
For a monatomic ideal gas this is
CV =

∂ 3
3
N kT = N k∂T 22

which can be experimentally veriﬁed, though many systems have more exotic CV .
In general, if you want to compute CV you need to25(4.2.2)

1. Compute S in terms of U and other quantities.
2. Compute T from the derivative of S
3. Solve for U (T ), and then diﬀerentiate it.
We’ll learn a quite diﬀerent and often more eﬃcient route in a few lectures.
Measuring Entropy via CV
We can measure entropy, or at least entropy diﬀerences, using heat. As long as work W = 0, we
have that
dS =

dU
Q
=
T
T

(4.2.3)

So if we can measure T and Q then we know how much the entropy changed!
• If T stays constant than this works even for ﬁnite changes in S.
• If the volume is changing, but the process is quasistatic, then this rule also applies.
A related statement is that
dS =

CV dT
T

(4.2.4)

If the temperature does change, then we can just use CV (T ), by thinking of the changes as tiny
steps. So
Z Tf
CV
ΔS =
dT
(4.2.5)TTi
In some examples CV is approximately constant. For instance if we heat a cup of water from room
temperature to boiling, we have
Z 373
1
373
(4.2.6)
ΔS ≈ (840 J/K)
dT = (840 J/K) log≈ 200J/K ≈ 1.5 × 1025
T
293
293
In fundamental units this is an increase of 1.5 × 1025 , so that the multiplicity increases by a factorof eΔS , which is a truly large number. This is about 2 per molecule.
If we knew CV all the way down to zero, then we could compute
Z Tf
CV
Sf − S(0) =dT(4.2.7)T
0
So what is S(0)? Intuitively, it should be S(0) = 0, so that Ω = 1, the unique lowest energy state.
Unless there isn’t a unique ground state.
26

In practice many systems have residual entropy at (eﬀective) T ≈ 0, due to eg re-arrangements
of molecules in a crystal that cost very, very little energy. There can also be residual entropy fromisotope mixing. (Interestingly Helium actually re-arranges its isotopes at T = 0, as it remains aliquid.)Note that to avoid a divergence in our deﬁnitions, it would seem that
CV → 0 as T → 0(4.2.8)This, and the statement that S(0) = 0, are both sometimes called the third law of thermodynamics.
Apparently ideal gas formulas for heat capacities must be wrong as T → 0, as we have alreadyobserved.
Historic View of Entropy
Historically dS = Q/T was the deﬁnition of entropy used in thermodynamics. It was an importantand useful quantity, even though no one knew what it was! Nevertheless, they got quite far using
ΔS ≥ 0 as the 2nd law of thermodynamics.

4.3

Paramagnets – Everything You Know Is Wrong

Now we will solve the two-state paramagnet model. This is almost identical to the toy model withcoin ﬂipping from the very ﬁrst day of class! However, we’ll see some features that you may ﬁndvery surprising.
The paramagnet is ideal in that it just consists of non-interacting dipoles. Also, we will invoke
quantum mechanics to study quantized dipoles that can only point up or down:· · · ↑↑↑↓↑↓↓↑↓↓↑ · · ·(4.3.1)We’ve already counted the states in this system. So what we need to do is add energy (and energy
conservation). The dipoles don’t interact, so the only energy comes from assuming there’s an external
magnetic ﬁeld, so thatU = µB(N↓ − N↑ ) = µB(N − 2N↑ )(4.3.2)The magnetization is just
M =−

U
B

(4.3.3)so it’s proportional to the energy. Recall the multiplicity of states isN!
N↑ !N↓ !

(4.3.4)

N
U
−
2
2µB

(4.3.5)

Ω(N↑ ) =Note that we can solve for N↑ in terms of U via
N↑ =

27

which means that N↑ = N/2 for U = 0. Where is multiplicity maximized?
We see that as a function of energy U , multiplicity is maximized at U = 0, and it goes down when
we increase or decrease U ! This is very diﬀerent from most other systems, as typically multiplicity
increases with energy, but not here.
Let’s draw the plot of S(U ). In the minimum energy state the graph is very steep, so the systemwants to absorb energy, as is typical for systems at very low energy. But at U = 0 the slope goes tozero, so in fact that system stops wanting to absorb more energy. And then for U > 0 the slope
becomes negative, so the paramagnet wants to spontaneously give up energy!
But remember that

−1
∂S
T ≡
(4.3.6)∂U
Thus when S 0 (U ) = 0, we have T = ∞. And when S 0 (U ) < 0, we have that T is negative.
Note that negative temperatures are larger than positive temperature, insofar as systems at
negative temperature give up energy more freely that systems at any positive temperature. When
|T | small and T < 0, temperature is at its eﬀective maximum.
It’s possible to actually do experiments on systems at negative temperature using nuclear
paramagnets by simply ﬂipping the external magnetic ﬁeld. Nuclear dipoles are useful because they
don’t equilibrate with the rest of the system, so they can be studied in isolation.
Negative temperature can only occur in equilibrium for systems with bounded total energy andﬁnite number of degrees of freedom.
Aside from simply demonstrating a new phenomenon, this example is useful for emphasizingthat entropy is more fundamental than temperature.
Explicit Formulae
It’s straightforward to apply Stirling’s approximation to get an analytic solution for large N .
The entropy is
S ≈ N log N − N↑ log N↑ − (N − N↑ ) log(N − N↑ )To get the temperature we just have


1 ∂S
1
∂S
∂N↑ ∂S===−T∂U N,B∂U ∂N↑
2µB ∂N↑(4.3.7)(4.3.8)Then the last derivative can be evaluated giving
1k
N − U/µB
=
log
T
2µB
N + U/µB(4.3.9)where T and U have opposite signs, as expected. Solving for U gives
U = −N µB tanh

µB
,
kT

M = N µ tanh
28

µB
kT

(4.3.10)

Then the heat capacity at constant magnetic ﬁeld is



2
∂U
µB
1
� 
= Nk
CB =
2
∂T N,BkT
cosh µB
kT(4.3.11)This goes to zero very fast at both low and high temperature, and is maximized at T → ∞ (witheither sign).
All this work wasn’t for nothing. For physical magnets at room temperature, kT ≈ 0.025eV , while
if µ = µB ≈ 5.8 × 10−5 eV /T is the Bohr magneton (the value for an electron), then µB ∼ 10−5 eVeven for strong magnets. Thus µB/(kT )  1. In that limit
tanh x ≈ x(4.3.12)and so we see that the magnetization isM ≈ NµµBkT


(4.3.13)which is called Curie’s law, as it was discovered experimentally by Pierre Curie. At high temperature
this works for all paramagnets, even those with a fundamenatlly more complicated description (egwith more than two states, and some interactions).

4.4

Mechanical Equilibrium and Pressure

Next we would like to consider systems whose volumes can change through interaction. The idea
is that exchange of volumes should be related to pressure through the entropy. We’returning the crank!
So let’s imagine two gases separated by a movable partition. The gases can exchange both energy
and volume, but total energy and volume are ﬁxed.
Total entropy should be maximized, so we expect that
∂Stotal=0∂VA(4.4.1)

By the same analysis that led to temperature, we can re-write this as∂SA ∂SB−=0∂VA ∂VB(4.4.2)since VA + VB = Vtotal is ﬁxed, and the entropy of the full system is the sum of the individual
entropies. Thus we conclude that
∂SB
∂SA
=
∂VB
∂VA

(4.4.3)at equilibrium, and so we should expect this to have a simple relationship with the pressure! Note
this is at ﬁxed energy and ﬁxed number of particles, and we are assuming that we are also in thermal
29equilibrium. If the partition were to move without this assumption, energies would not be ﬁxed, andthings would be more complicated.
Since TA = TB and T × S has units of energy, we can identify the pressure by


∂S
P =T
(4.4.4)∂V U,N
where we are free to multiply by T since we are in thermal equilibrium.
Let’s check this in an example. For an ideal gas we have thatS = N k log V + · · ·(4.4.5)where the ellipsis denotes functions of U and N that are independent of V . So we ﬁnd that
P =T

∂
N kT
(N k log V ) =∂V
V(4.4.6)

So we have derived the ideal gas law! Or alternatively, we have veriﬁed our expression for thepressure in a familiar example.
Thermodynamic Identity
We can summarize the relation between entropy, energy, and volume by a thermodynamic identity.
Let’s consider how S changes if we change U and V a little. This will be




∂S
∂S
dS =
dU +
dV
(4.4.7)∂U V∂V U
where we ﬁx volume when we vary U , and ﬁx U when we vary the volume. We can recognize thesequantities and write
1P
dU + dVT
T(4.4.8)dU = T dS − P dV

(4.4.9)dS =This is most often written as

This is true for any system where T and P make sense, and nothing else is changing (eg number ofparticles). This formula reminds us how to compute everything via partial derivatives.
Heat and Entropy
The thermodynamic identity looks a lot like energy conservation, ie
dU = Q + W
30(4.4.10)where we interpret Q ∼ T dS and W ∼ −P dV . Can we make this precise?
Not always. This is true if the volume changes quasistatically, if there’s no other work done, and
if no other relevant variables are changing. Then we know that W = −P dV , so that
Q = T dS(quasistatic)(4.4.11)This means that ΔS = Q/T , even if there’s work being done. If the temperature changes, we canstill compute
Z
CP(ΔS)P =
dT
(4.4.12)T
if the process is quasistatic.
Also, if Q = 0, then the entropy doesn’t change. In other words
quasistatic + adiabatic = isentropic(4.4.13)

We’ll note later that isentropic processes are interesting.
What about exceptions, which abound? Say we hit a piston really hard. Then the work done on
the gas is
W > −P dV(4.4.14)

because we hit the molecules harder than in a quasistatic process. However, we can choose to only
move the piston inﬁnitesimally, so that the
dS =

dU
P
Q
+ dV >
T
T
T(4.4.15)This is really just a way of saying that we’re doing work on the gas while keeping V constant, and so
U , T , and S will increase. It looks like compression work but it’s ‘other’ work.
A more interesting example is free expansion of the gas into a vacuum. No work is done andno heat ﬂows, but S increases.
It’s easy to create more entropy, with or without work and heat. But we can’t ever decrease it.

4.5

Diﬀusive Equilibrium and Chemical Potential

We’re turning the crank again, this time using conservation of particle number!
Let’s consider two systems that can share particle number. The usual logic tells us that
∂SA
∂SB
=
∂NB
∂NA
in equilibrium, where the partial derivatives are at ﬁxed energy and volume.
We can multiply by −T to get the conventional object chemical potential∂Sµ ≡ −T∂N U,V31(4.5.1)(4.5.2)

This is the quantity that’s the same for two systems in equilibrium. It’s the potential for sharing
particles.
Since we added a minus sign, if two systems aren’t in equilibrium the one with smaller µ gains
particles. This is like temperature. Particles ﬂow towards low chemical potential.
We can generalize the thermodynamic identity as
dS =

1
P
µ
dU + dV − dN
T
VT

(4.5.3)where the minus sign comes from the deﬁnition. ThusdU = T dS − P dV + µdN(4.5.4)

Sometimes we refer to µdN as the ‘chemical work’.
The general thermodynamic identity is useful for remembering the relations between various
thermodynamic quantities. For example note thatdU = µdN
which provides another way to think about µ as


∂U
µ=
∂N S,V

(4.5.5)(4.5.6)This is how much energy changes when we add a particle and keep S, V ﬁxed.
Usually to add particles without changing S, you need to remove energy. But if you need to givethe particle potential energy to add it, this contributes to µ.
Let’s compute µ for an ideal gas. We take!

3/2 !
4πmU
5/2
S = N k log V− log N(4.5.7)3h2
Thus we have!
3/2 !
4πmU
5µ = −T k log V− log N 5/2 + T N k
2
2N
3h
" 
#
3/2
V 4πmU= −kT log
N 3N h2" 3/2 #
V 2πmkT= −kT logNh2where at the end we used U = 32 N kT .
32

(4.5.8)

For gas at room temperature and atmospheric pressure, V /N ≈ 4 × 10−26 m3 , whereas the otherfactor is, for helium, about 10−31 m3 and so the logarithm is 12.7, and µ = −0.32 eV for helium at
300 K, 105 N/m.
Increasing the density of particles increases the chemical potential, as particles don’t ‘want’ tobe there.
So far we used only one particle type, but with many types we have a chemical potential for
each. If two systems are in diﬀusive equilibrium all the chemical potentials must be the same.
So we see that through the idea of maximizing entropy/multiplicity, we have been led to deﬁne
T , P , and µ, and the relationdU = T dS − P dV + µdN(4.5.9)for classical thermodynamics.

4.6

Extensive and Intensive Quantities

There’s an important distinction that we’ve alluded to but haven’t discussed directly, between
extensive and intensive quantities. The distinction is whether the quantity doubles when we
double the amount of stuﬀ, or if it stays the same. So extensive quantities are
V, N, S, U, H, mass

(4.6.1)

T, P, µ, density(4.6.2)extensive × intensive = extensive(4.6.3)

while intensive quantities are

If you multiply note thatfor example volume × density = mass. Conversely, if the ratios of extensive quantities will be
intensive.
Note that you should never add an intensive and an extensive quantity – this is often wrong by
dimensional analysis, but in cases where it’s not, it also doesn’t make sense conceptually.
So intensive/extensive is like a new kind of dimensional analysis that you can use to check what
you’re calculating.

5

Engines and Refrigerators

A heat engine absorbs heat energy and turns it into work. Note that this doesn’t include electricmotors, which turn electricity and mechanical work. Internal combustion engines in cars
oﬃcially don’t absorb heat, rather they produce it from combustion, but we can treat them as
though the heat was coming from outside.
33

Engines can’t convert heat into work perfectly. As we’ll see, the reason is that absorbing heatincreases the entropy of the engine, and if its operating on a cycle, then it must somehow get ridof that entropy before it can cycle again. This leads to ineﬃciency. Speciﬁcally, the engine must
dump waste heat into the environment, and so only some of the heat it initially absorbed can beconverted into work.
We can actually say a lot about this without knowing anything speciﬁc about how the engine
works. We can just represent it with a hot reservoir, the engine itself, and a cold reservoir.
Energy conservation says thatQh = W + Qc

(5.0.1)and that the eﬃciency is
e=

W
Qc
=1−
Qh
Qh

(5.0.2)This already says that eﬃciency is at most 1. We can get a stronger bound using the 2nd law ofthermodynamics.
Note that the entropy extracted from the hot reservoir is
Sh =

Qh
Th

(5.0.3)

while the entropy absorbed by the cold reservoir is
Qc
Tc(5.0.4)

Qh
Qc
≥
Tc
Th

(5.0.5)

Tc
Qc
≥
Qh
Th

(5.0.6)Sc =Thus we must have Sc − Sh ≥ 0 or

but this implies

This then tells us that
e=1−

Qc
Tc
≤1−
Qh
Th

(5.0.7)This is the famous Carnot bound on the eﬃciency of heat engines.
Most engines are signiﬁcantly less eﬃcient than this, because they produce excess entropy, often
through the heat transfer processes themselves. Note that to be maximally eﬃcient, the engine has
to be at Th and Tc during each transfer!
34

Carnot Cycle
The arguments above also suggest how to build a maximally eﬃcient engine – at least in theory.
Let’s see how it would work.
The basic point is that we don’t want to produce excess entropy. This means that• heat transfers must happen exactly at Th and Tc• there shouldn’t be any other heat transfers
This basically ﬁxes the behavior the engine, as pointed out by Sadi Carnot in 1824.
To have heat transfers exactly at Th and Tc , we must expand or contract the gas isothermally.
To otherwise not produce any entropy, we should have adiabatic expansion and contraction for theother parts of the cycle, ie with Q = 0. That’s how we get the gas from Th to Tc and back.
You should do problem 4.5 to check that this all works out, but based on our understanding ofenergy conservation and the 2nd law of thermodynamics, it should actually be ‘obvious’ that this isa maximally eﬃcient engine.
Note that Carnot engines are extremely impractical, because they must operate extremely slowly
to avoid producing excess entropy.
It’s easier to draw the Carnot cycle in T and S space rather than P and V space.

5.1

Refrigerators

Theoretically, a refrigerator is just a heat engine run in reverse.
We deﬁne the coeﬃcient of performance of a refrigerator as
COP =

Qc
Qc
1
=
= Qh
W
Qh − Qc−1
Q(5.1.1)c

We cannot just use the inequality from our discussion of engines, because heat ﬂows in the opposite
direction.
Instead, the entropy dumped into the hot reservoir must be at least as large as that drawn fromthe cold reservoir, which means that
Qh
Qc
≥
Th
Tc

(5.1.2)

Th
Qh
≥
Qc
Tc

(5.1.3)1
Tc=
Th − Tc−1
Tc(5.1.4)so thatThis then tells us that
COP ≤ Th35

So in theory we can refrigerate very, very well when Tc ≈ Tc . This should be intuitive, because inthis limit we need not transfer almost any entropy while transferring heat.
Conversely, if Tc  Th then the performance is very poor. It takes an immense amount of work
to cool something towards absolute zero.
It should also be clear from the way we derived it that the maximally eﬃcient refrigerator would
be a Carnot cycle run in reverse.
Brief Timeline
Carnot came up with ingenius arguments that engines couldn’t be more eﬃcient than the Carnot
cycle, and that they must always produce waste heat. But he didn’t distinguish very clearly between
entropy and heat. Clausius deﬁned entropy (and coined the term) clearly as Q/T , but he didn’tknow what it really was. Boltzmann mostly ﬁgured this out by 1877.

5.2

Real Heat Engines

Let’s discuss some real heat engines.
Internal combustion engines ﬁll a chamber with fuel and air, then combust it, expanding thegas and pushing the piston; then the material vents and is replaced with lower temperature/pressuregas, the piston compresses, and the cycle starts again. This is called the Otto cycle. Notice there’s
no hot reservoir, but combustion behaves the same way.
We can idealize this with ﬁxed volume changes in pressure and adiabatic compression andexpansion operations.
Let’s compute the eﬃciency, recalling that V γ P = constant where γ is the adiabatic exponent,
where γ = 1 + f2 . On the adiabats, no heat is exchanged, so all heat exchanges occur at constantvolume. This means that when the heat is exchanged, no work is done, and so the heat is equal tothe change in internal energy of the gas. Thus we have thatQh =f
N k(T3 − T2 )2

(5.2.1)

Qc =f
N k(T4 − T1 )2

(5.2.2)T3 − T4 − T2 + T1
T4 − T1=1−
T3 − T2
T3 − T2(5.2.3)andso that the eﬃciency ise=

It turns out that this simpliﬁes nicely, using the fact that on adiabats
T3 V2γ−1 = T4 V1γ−1(5.2.4)T1 V1γ−1 = T2 V2γ−1(5.2.5)and36

This leads to
e=1−
where r =

 γ−1V2
V1

rT3 − rT2
=1−r
T3 − T2(5.2.6). So in other words, the overall eﬃciency is

e=1−

V2
V1

γ−1
(5.2.7)

For air γ = 7/5 and we might have V2 /V1 = 8, so that the eﬃciency is around 56%.
Recall that T V γ−1 is ﬁxed on an adiabat, so to compare this eﬃciency with Carnot we can solve
for V ratios in terms of T ratios, and we ﬁnd
e=1−

T4
T1
=1−
T3
T2

(5.2.8)which doesn’t involve the extreme temperatures, so it less eﬃcient than Carnot. In practice real
gasoline engines are 20-30% eﬃcient.
You can get greater eﬃciency with higher compression ratio, but the fuel will pre-ignite once
it gets too hot. We can take advantage of this using diesel engines, which simply use heat fromcompression to ignite the mixture. They spray the fuel/air mixture as the piston starts to moveoutward, meaning that (to some approximation) they have a constant volume and a constant pressure
piece in their cycles, which means computing their eﬃciency is more complicated. They ultimately
have higher eﬃciency (around 40%) using very high compression ratios, which are limited by not
melting the engine.

Steam Engines
See the tables and diagrams in the book.
These use a Rankine cycle, and go outside the ideal gas regime, as the steam condenses!
Here water gets pumped in, heats into steam and pushes a turbine (as it cools and decreases inpressure), then is condensed back into water.
We can’t compute the eﬃciency from ﬁrst principles. But under constant pressure conditions,the heat absorbed is equal to the change in enthalpy H. So t we can write the eﬃciency as
e=1−

H4 − H1
H4 − H1Qc=1−≈1−
H3 − H2
H3 − H1
Qh(5.2.9)where the last approximation is good because the pump adds very little energy to the water, and
the P V term is small for liquids.
There are tables called ‘steam tables’ that simply list the enthalpies and entropies of steam, water,
and water-steam mixtures in various conditions. We need the entropies because for the steam+water
mixture, we only get H by using the fact that 3 → 4 is an adiabat, so S3 = S4 , allowing us to use
the S values of the tables to get H.
37

5.3

Real Refrigerators

Real refrigerators follow a cycle much like the inverse of the Rankine cycle. The working substancechanges back and forth between a liquid and a gas. That liquid-gas transition (boiling point) mustbe much lower in a refrigerator though, because we want to get to lower temperatures.
A variety of ﬂuids have been used, including carbon dioxide, ammonia (dangerous), Freon, andnow HFC-134a.
The PV diagram involves ﬁrst from 1 the gas is compressed adiabatically, raising P and T . Thenfrom 2 to 3 it gives up heat and liqueﬁes in the condenser. Then from 3 to 4 it passes througha throttling valve – a narrow opening – emerging on the other side at much lower pressure andtemperature. Finally it absorbs heat and turns back into a gas in the evaporator.
We can write the COP using enthalpies
Qc
H1 − H4
COP ==(5.3.1)H2 − H3 − H1 + H4
Qh − Qc
Enthalpies at 1, 2, 3 can be looked up, with point 2 we can assume S is constant during compression.
We can understand point 4 by thinking more about throttling.
Throttling
Throttling, or the Joule-Thomson process involves pushing a ﬂuid through a tiny hole or plug froma region with pressure Pi to one with pressure Pf . Initial and ﬁnal volumes are Vi and Vf . There is
no heat ﬂow, so
Uf − Ui = Q + W = 0 + Wlef t + Wright(5.3.2)

Uf − Ui = Pi Vi − Pf Vf(5.3.3)

Uf + Pf Vf = Ui + Pi Vi(5.3.4)

which givesoror just conservation of enthalpy.
The purpose is to cool the ﬂuid below the cold reservoir temperature, so it can absorb heat. This
wouldn’t work if the ﬂuid were an ideal gas, since H = f +2
N kT .
2
But in a dense gas or liquid, the energy contains a potential term which decreases (due toattractive forces) when the molecules get close together. So
H = Upot + Ukin + P V(5.3.5)and so as the gas molecules get further apart, Upot increases, and the kinetic energy and temperature
tend to decrease.
If we use H3 = H4 in refrigeration, then we have
H1 − H3
COP =(5.3.6)
H2 − H1and so we just need to look up the enthalpies.
38

Cooling MORE
See the book for some fun reading.
• You can make dry ice by liquifying CO2 at higher pressure, then when you throttle you get
dry ice frost residue.
• Making liquid Nitrogen is more diﬃcult. You can just throttle for a while, but you need tostart at higher pressure.
• Then you need to do it more eﬃciently, so you can use a heat exchanger to cool one gas asanother is throttled.
However, throttling isn’t good enough for hydrogen or helium, as these gases become hotter
when throttled. This is because attractive interactions between gas molecules are very weak, so
interactions are dominated by positive potential energies (repulsion). Hydrogen and Helium wereﬁrst liquiﬁed in 1898 and 1908 respectively.
One can use further evaporative cooling to get from 4.2 K (boiling point of helium at 1 atm) toabout 1 K. To go even further need
• A helium dilution refrigerator, where 3 He evaporates into a liquid bath with 4 He. We use the3
He to take heat from the 4 He. This can get us from 1K to a few milliK.
• Magnetic cooling using
M = N µ tanh

µB
kT

(5.3.7)

where we start with mostly up spins, and then smoothly decrease B, so that T also decreases
as the population of up and down spins stays the same. This is analogous to cooling of an
ideal gas by adiabatic expansion, following isothermal compression. We can’t get to 0K thisway due to dipole-dipole interactions, ie an imperfect paramagnet.
• There’s also laser cooling.

6

Thermodynamic Potentials

Forces push us towards minima of energies. Probability pushes us towards maximization of entropy.
Are there thermodynamic potentials mixing U and S that combine to tell us how a system willtend to change?
And a related question – we have seen that enthalpy H is useful when we are in an environment
with constant pressure. What if instead we’re connected to a reservoir at constant T ? Perhaps
there’s a natural way to account for heat transfer to and from this reservoir, just as H automaticallyaccounts for work due to the ambient pressure?

39

Let’s assume the environment acts as a reservoir of energy, so that it can absorb or release
unlimited energy without any change in its temperature. If SR is the reservoir entropy, thendStotal = dS + dSR ≥ 0(6.0.1)by the 2nd law.
We would like to eliminate the reservoir. For this, let’s use the thermodynamic identity for the
reservoir
dSR =

1
P
µ
dUR + dVR − dNR
T
T
T(6.0.2)and let’s hold VR and NR ﬁxed, so only energy can be exchanged. Then we can write
dStotal = dS +

1
dUR
T

(6.0.3)since TR = T . But energy is conserved, so we have thatdStotal = dS −111
dU = − (dU − T dS) = − dF
T
TT

(6.0.4)

where we deﬁne the Helmholtz free energy as
F = U − TS

(6.0.5)

We have deﬁned it with a minus sign because now we have thatdF ≤ 0(6.0.6)

which means that F can only decrease. So it’s a kind of ‘potential’ such that we can only move tolower and lower values of F at ﬁxed T, V, N .
And now we can just ignore the reservoir and account for the 2nd law using F . The system will
just try to minimize F .
We can play the same game letting the volume of the system change, and working at constantpressure, so thatdStotal = dS −1P11
dU − dV = − (dU − T dS + P dV ) = − dG
T
T
T
T

(6.0.7)

where
G = U − TS + PV

(6.0.8)and we work at constant T and constant P . Thus in this case its the Gibbs free energy that must
decrease.
We can summarize this as40• At constant U and V , S increases.
• At constant T and V , F tends to decrease.
• At constant T and P , G tends to decrease.
All three assume no particles enter or leave, though one can also make even more potentials if we
consider diﬀusion of particles.
We can think about this directly by just noting that in
F ≡ U − TS

(6.0.9)S ‘wants’ to increase, and (one may think that) U ‘wants’ to decrease. The latter is only because it
can give energy to the environment, and thereby increase the total entropy of the universe.
Wait what? Here what’s happening is that if U decreases without S increasing, then this meansthat the entropy of the system isn’t changing – but the entropy of the environment will deﬁnitelychange by dU. So the fact that objects ‘want’ to roll down hills (and then not roll back up again!)Tultimately comes from the 2nd law of thermodynamics.
Note also that the desire to give away energy isn’t very great if T is large, since then the system
may not want to do this and risk ‘losing valuable entropy’. This is one way of seeing that at hightemperature, our system will have a lot of energy.
Reasoning for the Gibbs free energy
G ≡ U + PV − TS

(6.0.10)is similar, except that the environment can also ‘take’ volume from the system.
Why Free Energy?
We can also interpret F and G as analogues to H – these are the amount of energy needed to createa system, minus the heat-energy that we can get for free from the environment. So for example in
F = U − TS

(6.0.11)

we see that we can get heat Q = T ΔS = T S from the environment for free when we make thesystem.
Conversely, F is the amount of energy that comes out as work if we annihilate the system,
because we have to dump some heat Q = T S into the environment just to get rid of the systems
entropy. So the available or ‘free’ energy is F .
Above we meant all work, but if we’re in a constant pressure environment then we get some workfor free, and so in that context we can (combining H and F , in some sense) deﬁne
G = U − TS + PV

(6.0.12)which is the systems energy, minus the heat term, plus the work the constant P atmosphere willautomatically do.
41

We call U, H, F, G thermodynamic potentials, which diﬀer by adding P V or −T S. They are all
useful for considering changes in the system, and changes in the relevant potential. Note that
ΔF = ΔU − T ΔS = Q + W − T ΔS

(6.0.13)

If no new entropy is created then Q = T ΔS, otherwise Q < T ΔS, and so we have
ΔF ≤ W

(6.0.14)at constant T . This includes all work done on the system. Similarly, we ﬁnd that
ΔG ≤ Wother(6.0.15)at constant T, P , where Wother includes work done other than by pressure.
Changes in H, F, G are very useful to know, and so are tabulated by chemists and engineers. We
rarely talk about their absolute values, since that would include all of the energy (even E = mc2
energy), so instead these quantities are usually viewed as changes from a convenient reference point.

More Thermodynamic Identities and Legendre Transforms
We learned the identity
dU = T dS − P dV + µdN(6.0.16)

In this sense we are automatically thinking about U as a function of S, V, N , since its these quantities
that we’re directly varying.
We can use this plus the deﬁnitions of H, F, G to write down a ton of other thermodynamic
identities. The math that we use is the ‘Legendre transform’. Given a convex function f (y), it
makes it possible to eﬀectively ‘switch variables’ and treat f 0 (y) = x as the independent variable.
See 0806.1147 for a nice discussion.
Say we have a function so that its diﬀerential isdf = xdy(6.0.17)g = xy − f(6.0.18)

dg = ydx + xdy − df = ydx(6.0.19)

Then if we consider a new function

then we have

So now we can instead use our new function g(x) in place of f (y).
For example, perhaps instead of using the entropy S(E) as a function of E, we would instead∂S
like to make ∂E= T1 , ie the temperature, the independent variable. That way we can use T as our‘knob’ instead of E, perhaps because we can control T but not E.
42

Although it may seem a bit mysterious – and the number of possibilities is rather mystifying –we can apply these ideas very directly. For example since
H = U + PV

(6.0.20)

dH = dU + P dV + V dP

(6.0.21)

we haveand so we can write
dH = T dS − P dV + µdN + P dV + V dP
= T dS + µdN + V dP

(6.0.22)so we can think of the enthalpy as a function of S, N, P . We traded V for P and switched fromenergy U to enthalpy H.
We can really go to town and do this for all the thermodynamic potentials. For example, for F
we ﬁnddF = −SdT − P dV + µdN(6.0.23)and this tells us how F changes as we change T, V, N , and also that F is naturally dependent on
those variables. We also havedG = −SdT + V dP + µdN(6.0.24)

and we get other identities from it.

Gibbs Free Energy and Chemical Potential
Combining the previous results with the idea of intensive/extensive variables, we can derive a cutefact about G.
Recall that


∂G
µ=
(6.0.25)∂N T,P
This says that adding particles at ﬁxed T, P simply increases G by µ.
You might think that µ changes as you change N , but this can’t happen with T, P ﬁxed, because
G is an extensive quantity that must grow proportionally to µ. Thus we see thatG = Nµan amazingly simple relation, which provides a new interpretation for G and µ.

43(6.0.26)

Our argument was subtle – why doesn’t it apply to


∂F
µ=
∂N T,V

(6.0.27)

The reason is that with ﬁxed V , as you change N the system becomes more and more dense. So an
intensive quantity is changing as you change N , namely the density N/V . It was crucial in the caseof G that all ﬁxed quantities were intensive.
P
With more types of particles, we just have G = i Ni µi . Note though that µi for a mixture are
not equal to µi for a pure substance.
We can use this to get a formula for µ for an ideal gas. Using the fact that


∂G
V =
(6.0.28)∂P T,N
we see that
∂µ
1 ∂G
V
=
=
∂P
N ∂P
N

(6.0.29)

But by the ideal gas law this is kT /P . So we can integrate to get Pµ(T, P ) − µ(T, P0 ) = kT log
P0(6.0.30)

for any reference P0 , usually atmospheric pressure.
This formula also applies to each species independently in a mixture, if P is the partial pressureof that species. This works because ideal gases are non-interacting – that’s essentially what makes
them ideal.

Example Application of G – Electrolysis & Batteries
Let’s consider the electrolysis process
1
H2 O → H2 + O2
2

(6.0.31)

We’ll assume we start with a mole of water.
Reference tables say that ΔH = 286 kJ for this reaction. It turns out P ΔV = 4 kJ to push theatmosphere away, the remaining 282 kJ remains in the system. But do we have to supply all of the
286 kJ as work, or can some enter as heat?
For this we need to look at the entropies for a mole of each species
SH2 O = 70J/K,

SH2 = 131J/K,

SO2 = 205J/K(6.0.32)

So during the reaction we have an entropy change of1
SH2 + SO2 − SH2 O
244(6.0.33)

and so the maximum amount of heat that can enter the system is
T ΔS = (298K)(163J/k) = 49kJ

(6.0.34)

So the electrical work we need to do is 237 kJ. Thi sis the ΔG of the Gibbs free energy:
ΔG = ΔH − T ΔS

(6.0.35)

Standard tables include this information. If you perform the reaction in reverse, you can get ΔG ofenergy out.
The same reasoning applies to batteries. For example lead-acid cells in car batteries run thereactionP b + P bO2 + 4H + + 2SO42− → 2P bSO4 + 2H2 O(6.0.36)

Tables say ΔG = −390 kJ/mol in standard conditions, so per mole of metallic lead we get 390 kJ
from the battery.
Note that ΔH = −312 kJ/mol for this reaction, so we actually get extra energy from heatabsorbed from the environment! When we charge the battery and run the reaction in reverse, wehave to put the extra 78 kJ of heat back into the environment.
You can also compute the battery voltage if you know how many electrons get pushed around
per reaction. For this reaction 2 electrons get pushed through the circuit, so the electrical work perelectron is
390kJ= 3.24 × 10−19 J = 2.02eV
2 × NA

(6.0.37)

A volt is the voltage needed to give an electron 1 eV, so the voltage is 2.02 V. Car batteries have six
cells to get to 12 V.

7

Partition Functions and Boltzmann Statistics

Now we will develop some more powerful tools for studying general systems.
Speciﬁcally, we will obtain a formula for the probability of a ﬁnding any system in a given
microstate, assuming that the system is in equilibrium with a reservoir of temperature T .

7.1

Boltzmann Factors

We’ll consider an analysis that applies to any system. But since Hydrogen atoms tend to be familiar,
we can phrase the analysis in terms of a single electron in some hydrogen atom orbital.
A central point is that we are not considering a truly isolated system, but one that it’s contact
with a ‘thermal bath’, ie some other system with a great deal of energy and entropy, and ﬁxed
temperature T .
Let’s simplify by considering the ratio of probabilities to be in two speciﬁc microstates of interest,
s1 and s2 , with energies E(s1 ), E(s2 ) and probabilities P (s1 ), P (s2 ). How can we ﬁnd the ratio?
45

We know that for an isolated system, all accessible microstates are equally probable. Our atom
isn’t isolated, but the atom + reservoir system is isolated. We expect that we’re equally likely to
ﬁnd this combined system in any microstate.
The reservoir will have ΩR (s1 ) available microstates when the atom is in s1 , and ΩR (s2 ) microstates
when the atom is in s2 . These will be diﬀerent because the reservoir has more or less energy, andthus it has access to more or less states. Since all states in total are equally likely, the ratio of
probabilities must beΩR (s2 )P (s2 )=
ΩR (s1 )P (s1 )

(7.1.1)

So we just need to write the right-side in a more convenient form that doesn’t make reference to thereservoir.
Note that something a bit surprising happened – because multiplicity depends so strongly onstates and energies, changing our single atom’s state just a bit actually had a signiﬁcant eﬀect on
the multiplicity of a giant reservoir of energy. This contrasts with most questions in physics, where
the state of a single atom has almost no relevance to the behavior of a large reservoir.
But that’s easy by noting that
P (s2 )eSR (s2 )= S (s1 ) = eSR (s2 )−SR (s1 )P (s1 )
e R

(7.1.2)

Now we can use the thermodynamic identitydSR =1(dUR + P dVR − µdNR )T(7.1.3)

We can write all of these changes in terms of properties of the atom!
We know dNR really is zero here, because reservoir particle number isn’t changing. We will set
dVR = 0, even though in principle it may change, because the change in it is very tiny (eg a million
times smaller than the term we’ll keep).
This then leads to
SR (s2 ) − SR (s1 ) =1
1[UR (s2 ) − UR (s1 )] = − [E(s2 ) − E(s1 )]TT(7.1.4)

Notice this is another situation where deﬁning the natural quantity as β = 1/T might have been
more convenient. Anyway, we ﬁnd thatE(s2 )P (s2 )e− kT
= E(s1 )P (s1 )
e− kT

(7.1.5)So the ratio strongly suggests that we can simply deﬁne a universal Boltzmann factor ofe−E(s)/kT
as the proportionality factor for any given state s with energy E(s).
46(7.1.6)

Note that probabilities aren’t equal to this factor because they also have to add up to 1! And so
we need to divide by the sum of all Boltzmann factors, including all states. We call that sum overthese factors
X E(s)
Z=e− kT(7.1.7)s

the partition function. The true probabilities are
P (s) =

1 − E(s)e kTZ(7.1.8)This is perhaps the most important fact or formula in statistical mechanics.
The partition function Z itself is extremely useful. Notice that while it doesn’t depend on any one
state (since it depends on all states and energies), it does depend on the temperature. It essentiallycounts the eﬀective number of states that are accessible; for that interpretation we just set E = 0
for the ground state.
Notice that when we think about statistical mechanics using Boltzmann factors, Temperatureplays a more fundamental role.
Example - Hydrogen in the Sun
The sun’s atmosphere has a temperature around 5800 K. The relative probability of ﬁnding an atom
in a its ﬁrst excited state as compared to the ground state ise−(E2 −E1 )/kT

(7.1.9)with δE = 10.2 eV, and we have kT = 0.5 eV (note that an eV is about 10, 000 K). So the ratio of
probabilities is about e−20.4 ≈ 1.4 × 10−9 . Since there are four excited states, the portion in the ﬁrst
excited state is about about 5 × 10−9 .
Atoms in the atmosphere of the sun can be absorbed if they can induce a transition between
states. A hydrogen atom in the ﬁrst excited state can transition up in energy to give the Balmer
series, which give missing wavelengths in sunlight. Some other lines are also missing due to otherkinds of atoms, but these transition from their ground state. So there must be way more hydrogenthan other atoms in the sun’s atmosphere!

7.2

Average or Expectation Values

We saw that the probability to be in a given state is
P (s) =e−βE(s)Z

47(7.2.1)Given the probability to be in any given state, we can easily (in principle) compute the average or
expectation value for any quantity f (s). Formally this means thatXf¯ =P (s)f (s)s1 X −βE(s)
=
e
f (s)
Z s
P −βE(s)
f (s)
se
P
=
−βE(s)
se(7.2.2)For example, the simplest thing we might want to know about is the average energy itself, ie
f (s) = E(s). Thus we have thatP −βE(s)E(s)
seĒ = P
(7.2.3)−βE(s)
sefor the average energy.
The average energy is actually easier to compute because
Ē = −

1 ∂Z
Z ∂β(7.2.4)

as we can easily check by explicit computation. That’s why I wrote the partition function in theway I did.
Note that expectation values are additive, so the average total energy of two objects is thesum of their individual average energies. So if you have a collection of identical particles
U = N Ē(7.2.5)This means that in many cases, working atom-by-atom is no diﬀerent than studying the wholesystem (if we only care about averages).
7.2.1

Paramagnets

For paramagnets, there’s just up and down dipole moment, so
Z = e−β(−µB) + e−β(+µB) = eβ(µB) + e−β(µB)= 2 cosh(βµB)(7.2.6)and so we can easily compute the up and down probabilities. The average energy isĒ = −1 ∂Z= −µB tanh(βµB)
Z ∂βwhich is what we found before. We also get the Magnetization right.
48(7.2.7)

7.2.2

Rotation

A more interesting example is the rotational motion of a diatomic molecule.
In QM, rotations are quantized with total angular momentum j~. The energy levels are
E(j) = j(j + 1)

(7.2.8)for some ﬁxed energy  inversely proportional to the molecules moment of inertia. The number ofenergy levels at angular momentum j is 2j + 1. (Here we are assuming the two ends of the molecule
are non-identical.)Given this structure, we can immediately write the partition function∞XZ =(2j + 1)e−E(j)/kT

=

j=0
∞
X



(2j + 1)e−j(j+1) kT(7.2.9)j=0

Unfortunately we can’t do the sum in closed form, but we can approximate it.
Note that for a CO molecule,  = 0.00024 eV, so that /k = 2.8 K. Usually we’re interested inmuch larger temperatures, so that /kT  1. This is the classical limit, where the quantum spacing
doesn’t matter much. Then we have
Z ∞

kT
Z≈
(2j + 1)e−j(j+1) kT dj =(7.2.10)0when kT  . As one might expect, the partition function increases with temperature. Now we ﬁndĒ = −∂ log Z
= kT
∂β(7.2.11)

Diﬀerentiating with respect to T gives the contribution to the heat capacity per molecule
CV ⊃

∂Ē
=k
∂T

(7.2.12)

as expected for 2 rotational DoF. Note that this only holds for kT  ; actually the heat capacity
goes to zero at small T .
If the molecules are made of indistinguishable atoms like H2 or O2 , then turning it around gives
back the same molecule, and there are half as many rotatational degrees of freedom. Thus we have
Z=

kT
2

(7.2.13)

The extra 1/2 cancels out when we compute the average energy, so it has no eﬀect on heat capacity.
At low temperatures we need to more carefully consider QM to get the right answer.
49

7.2.3

Very Quantum System

What if we have a very quantum system, so that energy level splittings are much larger than kT ?
Then we haveZ ≈ 1 + e−βE1(7.2.14)

where E1 is the lowest excited state, and we are assuming En > E1  kT . Note that the expectationof the energy is
E¯ = 0 + E1 e−βE1 = E1 e−βE1(7.2.15)

where we have set the energy of the ground state to 0 as our normalization.
We also have thatE1∂Ē
E1 e− kT=∂T
kT 2

(7.2.16)

It may appear that as T → 0 this blows up, but actually it goes to zero very rapidly, because
exponentials dominate power-laws. For comparison note that xn e−x → 0 as x → ∞, this is just thex → 1/x version of that statement.
7.2.4

This is Powerful

Anytime we know the energy levels of a system, we can do statistical mechanics. This is usuallymuch more eﬃcient than using combinatorics to count states, then determine S(E), and then ﬁgure
out T -dependence from there.

7.3

Equipartition

We have discussed the equipartition theorem... now let’s derive it.
Recall that it doesn’t apply to all systems, but just to quadratic degrees of freedom with energies
like
E = kq 2(7.3.1)

where q = x as in a harmonic oscillator, or q = p as in the kinetic energy term. We’ll just treat q as
the thing that parameterizes states, so that diﬀerent q corresponds to a diﬀerent independent state.
This leads to
X
X
2
Z =
e−βE(q) =
e−βkq
q

Z ∞
≈

q
2

e−βkq dq−∞50

(7.3.2)

Really the latter should be the deﬁnition in the classical limit, but we might also imagine startingwith a QM description with discreteq and smearing it out. Anyway, we can evaluate the integral by√a change of variables to x = βkq so thatZ ∞1
2
(7.3.3)Z≈√e−x dxβk −∞The integral is just some number, so we’re free to writeCZ=√βfor some new number C. Note that C =q.
Now the average energy is

pπ 1
k Δq

Ē = −(7.3.4)

where Δq indicates the possible QM discreteness of∂ log Z
1
= kT
∂β
2

(7.3.5)which is the equipartition theorem. Notice that k and Δq drop out. But that only works because
we assumed Δq was very small, in particular so that
kqΔq, kΔq 2  kT(7.3.6)This is just the statement that we can take the classical limit, because energy level spacings are
small.

7.4

Maxwell Speed Distribution

As another application of Boltzmann factors, we can determine the distributions of speeds in a gas.
From equipartition we already know thatr
3kT
vrms =(7.4.1)m
but that’s only an average (or more precisely, a moment of the distribution). With Boltzmann
factors we can just write down the distribution itself.
Here we are really computing a probability density, which gives a probability P (~v )d3~v or P (s)ds
for speed. So the integral gives the probability that v lies in some interval. Here P (~v ) or P (s)is a probability distribution function. Note that its only P (s)ds that has the right units forprobability.
By the way, the moments of the distribution would beZ ∞ds sn P (s)(7.4.2)051

where this is the nth moment. The vrms is the square root of the 2nd moment.
The Boltzmann factor for a gas molecule is the simplem~v2

e− 2kT

(7.4.3)using the usual formula for kinetic energy. But if we want to compute the probability distribution
for speed P (s), we need to account for the dimensionality of space. This means that we have
Z
m~
v2
m~
v2
P (s) ∝
d2~v e− 2kT = 4πs2 e− 2kT(7.4.4)|v|=s

We used proportional to because we need to normalize. For that we need to compute the constant
N such that
Z ∞
m~
v2
N =
ds 4πs2 e− 2kT
0


= 4π

2kT
m

3/2 Z ∞2x2 e−x dx(7.4.5)0Thus we have thatm~v2
14πs2 e− 2kTN m 3/2m~v2=
4πs2 e− 2kT
2πkT

P (v) =

(7.4.6)This is the Maxwell distribution for the speeds of molecules. It may look complicated, but its just a
Boltzmann factor times a geometric factor.
Note that this distribution is very asymmetric, insofar as it’s parabolic at small v, where its
geometry-dominated, but exponential (Gaussian) at large v, where it falls oﬀ exponentially fast.
Note that the most common, mean, and average speeds are all diﬀerent. We haver
2kT
(7.4.7)
vmax =
m
The mean speed (1st moment) is
r

Z ∞
v̄ =

ds sP (s) =
0

8kT
πm

(7.4.8)

So we have vmax < v̄ < vrms .

7.5

Partition Function and Free Energy

We began the course focused on the multiplicity Ω and S = log Ω, which tends to increase. These
are the fundamental quantities for an isolated system.
52

For a system at temperature T (in contact with a reservoir), the partition function Z(T ) is
the fundamental quantity. We know that Z(T ) is proportional to the number of accessible states.
Since Z(T ) should (thus) tend to increase, its logarithm will tend to increase too. This suggests a
relationship with −F , where F is the Helmholtz free energy. In fact
Z = e−F/kT

F = −kT log Z,(7.5.1)

One could even take this as the deﬁnition of F . Instead, we’ll derive it from our other deﬁnition
F = U − TS

(7.5.2)

To derive the relationship with Z, note that


∂F
F −U
= −S =∂T V,N
T(7.5.3)This is a diﬀerential equation for F . Let’s show that −kT log Z satisﬁes this diﬀerential equation,
and with the same initial condition.
Letting F̃ = −kT log Z, we have that∂F̃∂
= −k log Z − kT
log Z
∂T
∂T

(7.5.4)∂U
log Z =
∂T
kT 2

(7.5.5)but

This then tells us that F˜ obeys the correct diﬀerential equation.
They also agree at T = 0, since both F and F̃ will be 1, as the system will just be in its groundstate. So they are the same thing.
One reason why this formula is so useful is that
S=−

∂F
∂F
∂F
, P =−
, µ=
∂T
∂V
∂N

(7.5.6)

where we ﬁx the T, V, N we aren’t diﬀerentiating with respect to. So from Z we can get all of thesequantities.

7.6

From Laplace Transforms to Legendre Transforms

Let’s consider another derivation/explanation of the relationship between F and Z.
Note that eS(E) is the multiplicity of states at energy E. ThusXZ =e−E(s)/kT
Zs ∞
=

E

dE eS(E)− kT

0

53(7.6.1)

From this perspective, the statement thatF = −kT log Z(7.6.2)is equivalent to the statement that we can just view
Z ∞
E
Z =
dE eS(E)− kT
0U
S(U )− kT≈ e(7.6.3)That is, somehow we have simply forgotten about the integral! What’s going on? Have we made a
mistake?
The reason why this is alright is that the function
E

eS(E)− kT(7.6.4)¯ Its so sharply peaked that its integral iswill be very, very sharply peaked at the energy U = E.
simply dominated by the value at the peak. This ‘sharply peaked’ behavior is exactly what we
spent the ﬁrst couple of weeks of class going on and on about. So
F = −kT log Z = U − T S(7.6.5)is actually true because we are taking averages in the thermodynamic limit.
This provides a new perspective on thermodynamic potentials. That is, if we work in the
microcanonical ensemble, that is the ensemble of states with ﬁxed energy U , then its natural to
study the multiplicity
Ω(U ) = eS(U )(7.6.6)and expect that systems in equilibrium will maximize this quantity. But we can introduce atemperature by ‘averaging over states’ weighted by a Boltzmann factor. This puts us in the
canonical ensemble, the ensemble of states of all energies with ﬁxed average energy. And it’s
simply given by taking the Laplace transform with respect to energy
Z ∞Z(β) =
dE e−βE eS(E)(7.6.7)0

where β = 1/kT as usual. Notice that we have traded a function of energy U for a function oftemperature T , just as in the Legendre transform discussion. This isn’t a coincidence!
The statement that this is dominated by its maximum also implies that the canonical andmicrocanonical ensembles are the same thing in the thermodynamic limit. Further, it tells usthat since−kT log Z = U − T S(U )(7.6.8)

we learn that the Legendre transform is just the thermodynamic limit of the Laplacetransform. This is a useful way to think about Legendre transforms in thermodynamics.
We can obtain all the other thermodynamic potentials, and all of the other Legendre transforms,
in exactly this way.
54

8

Entropy and Information

8.1

Entropy in Information Theory

We have been thinking of entropy as simply
S = log Ω(8.1.1)

But for a variety of reasons, it’s more useful to think of entropy as a function of the probability
distribution rather than as depending on a number of states.
If we have Ω states available, and we believe the probability of being in any of those states is
uniform, then this probability will be1
Ωfor any of the states i = 1, · · · , Ω. So we see thatpi =(8.1.2)

S = − log p(8.1.3)for a uniform distribution with all pi = p = 1/Ω. But notice that this can also be viewed asXS≡−pi log pi(8.1.4)isince all pi are equal, and their sum is 1 (since probabilities are normalized). We have not yet
demonstrated it for distributions that are not uniform, but this will turn out to be the most usefulgeneral notion of entropy. It can also be applied for continuous probability distributions, whereZ
S ≡ − dx p(x) log p(x)(8.1.5)Note that 0 log 0 = 0 (this is what we ﬁnd from a limit).
This arises in information theory as follows. Say we have a long sequence of 0s and 1s, ie· · · 011100101000011 · · ·(8.1.6)

If this is truly a fully random sequence, then the number of possibilities is 2N if it has length N ,and the entropy is just the log of this quantity. This also means that the entropy per bit is1
log 2N = log 2(8.1.7)N
But what if we know that in fact 0s occur with probability p, and 1s with probability 1 − p? Fora long sequence, we’ll have roughly pN of the 0s, so this means that the number of such messages
will beSbit =N!
NN≈(pN )!((1 − p)N )!
(pN )pN (N − pN )N −pNN1== eN Sbit
pp (1 − p)1−p55(8.1.8)

where we see that
Sbit = −p log p − (1 − p) log(1 − p)(8.1.9)So the entropy S that we’ve studied in statistical mechanics, ie the log of the number of possible
sequences, naturally turns into the deﬁnition in terms of probability distributions discussed above.
This has to do with information theory because N S quantiﬁes the amount of information
we actually gain by observing the sequence. Notice that as Warren Weaver wrote in an initial
popularization of Shannon’s ideas:• “The word information in communication theory is not related to what you do say, but towhat you could say. That is, information is a measure of one’s freedom of choice when one
selects a message.”

8.2

Choice, Uncertainty, and Entropy

Shannon showed in his original paper that entropy has a certain nice property. It characterizes how
much ‘choice’ or ‘uncertainty’ or ‘possibility’ there is in a certain random process. This should seemsensible given that we just showed that entropy counts the amount of information that a sequencecan carry.
Let’s imagine that there are a set of possible events that can occur with probabilities p1 , · · · , pn .
We want a quantity that measures how much ‘possibility’ there is in these events. For example, if
p1 = 1 and the others are 0, then we’d expect there’s no ‘possibility’. While if n  1 and the pi areuniform, then ‘almost anything can happen’. We want a quantitate measure of this idea.
We will show that entropy is the unique quantity H(pi ) with the following natural properties:
1. H is continuous in the pi .
2. If all pi are equal, so that all pi = 1/n, then H should increase with n. That is, having moreequally likely options increases the amount of ‘possibility’ or ‘choice’.
3. If the probabilities can be broken down into a series of events, then H must be a weighted
sum of the individual values of H. For example, the probabilities for events A, B, C


1 1 1
{A, B, C} =, ,(8.2.1)2 3 6
can be rewritten as the process{A, B or C} =

1 1
,
2 2




then {B, C} =

2 1
,
3 3

We are requiring that such a situation follows the rule






1 1 1
1 1
1
2 1
H
, ,
=H
,
+ H
,
2 3 6
2 2
2
3 3
56


(8.2.2)(8.2.3)

We will show that up to a positive constant factor, the entropy S is the only quantity with theseproperties.
Before we try to give a formal argument, let’s see why the logarithm shows up. Consider






1 1 1 1
1 1
1
1 1
H
, , ,
= H
,
+2 H
,
4 4 4 4
2 2
2
2 2


1 1
= 2H
,
(8.2.4)
2 2
SimilarlyH11,··· ,
8
8




= 3H

1 1
,
2 2


(8.2.5)and so this is the origin of the logarithm. To really prove it, we will take two fractions, and notethat s1n ≈ t1m for suﬃciently large n and m. The only other point we then need is that we canapproximate any other numbers using long, large trees.
Here is the formal proof. Let11A(n) ≡ H,··· ,(8.2.6)nn
for n equally likely possibilities. By using an exponential tree we can decompose a choice of sm
equally likely possibilities into a series of m choices among s possibilities. So
A(sm ) = mA(s)(8.2.7)

We have the same relation for some t and n, ieA(tn ) = nA(t)(8.2.8)

By taking arbitrarily large n we can ﬁnd an n, m with sm ≤ tn < sm+1 , and so by taking thelogarithm and re-arranging we can write
m
log t
m 1
≤
≤
+
n
log s
n
n

(8.2.9)m log t
−
<
n
log s

(8.2.10)mA(s) ≤ nA(t) < (m + 1)A(s)(8.2.11)which means that we can makeUsing monotonicity we have that57

So we also ﬁnd thatm A(t)−<nA(s)(8.2.12)and so we conclude via these relations that
A(t) = K log t(8.2.13)for some positive constant K, since we can make  arbitrarily small.
But now by continuity we are essentially done, because we can approximate any set of probabilities
pi arbitrarily well by using a very ﬁne tree of equal probabilities.
Notice that the logarithm was picked out by our assumption 3 about the way that H applies tothe tree decomposition of a sequence of events.

8.3

Other Properties of Entropy

These properties all make sense in the context of information theory, where S counts the uncertainty
or possibility.
1. S = 0 only if there’s a single outcome.
2. S is maximized by the uniform distribution.
3. Suppose there are two events, x and y, where at each event n and m possibilities can occur,
respectively. The full joint entropy isX
S(x, y) =p(i, j) log p(i, j)(8.3.1)i,jwhereas there are partial entropies
S(x) =X

p(i, j) log

X

i,j

j

X

p(i, j) log

X

S(y) =i,jp(i, j)p(i, j)(8.3.2)i

Then we have the inequality
S(x, y) ≤ S(x) + S(y)(8.3.3)

with equality only if the two events are independent, so that p(i, j) = p(i)p(j).
4. Any change in the probabilities that makes them more uniform increases the entropy. Morequantitatively, if we perform an operation
p0i = aij pjPP(8.3.4)

where i aij = j aij = 1 and aij > 0, then S increases. This means that if x and y arecorrelated, the total entropy is always less than or equal to the sum of the entropies from x
and y individually.
58

8.4

A More Sophisticated Derivation of Boltzmann Factors

Boltzmann factors can be derived in another simple and principled way – they are the probabilities
that maximize the entropy given the constraint that the average energy is held ﬁxed.
We can formalize this with some Lagrange multipliers, as follows.
We want to ﬁx the expectation value of the energy and the total probability while maximizing
the entropy. We can write this maximization problem using a function (Lagrangian)
!
!
X
X
X
pi Ei + νpi − 1(8.4.1)L=−
pi log pi + β hEi −iiiwhere we are maximizing/extremizing L with respect to the pi and β, ν; the latter are Lagrange
multipliers.
Varying gives the two constraints along with− log pi − 1 − βEi + ν = 0(8.4.2)pi = eν−1−βEi(8.4.3)

which implies thatNow ν just sets the total sum of the pi while β is determined by the average energy itself. So we havere-derived Boltzmann factors in a diﬀerent way. This derivation also makes it clear what abstract
assumptions are important in arriving at e−βE .

8.5

Limits on Computation

In practice, computation has a number of ineﬃciencies. Despite decades of progress, my impressionis that most of the waste is still due to electrical resistance and heat dissipation. We’ll perform a
relevant estimate in a moment. But what are the ultimate limits on computation?
We have seen several times that irreversible processes are those that create entropy. Conversely,
for a process to be reversible, it must not create any entropy.
However, computers tend to create entropy, and thus waste, for what may be a surprising reason– they erase information. For example, say we add two numbers, eg
58 + 23 = 81

(8.5.1)We started out with information representing both 58 and 23. Typically this would be stored as aninteger, and for example a 16 bit integer has information, or entropy, 16 log 2. But at the end of thecomputation, we don’t remember what we started with, rather we just know the answer. Thus wehave created an entropy
S = 2 × (16 log 2) − (16 log 2) = 16 log 2
through the process of erasure!
59(8.5.2)

Since our computer will certainly be working at ﬁnite temperature, eg room temperature, wewill be forced by the laws of thermodynamics to create heatQ = kB T (16 log 2) ≈ 5 × 10−20 Joules(8.5.3)

Clearly this isn’t very signiﬁcant for one addition, but it’s interesting as its the fundamental limit.
Furthermore, computers today are very powerful. For example, it has been estimated that while
training AlphaGo Zero, roughly
1023 ﬂoat point operations(8.5.4)

were performed. Depending on whether they were using 8, 16, or 32 bit ﬂoating point numbers (let’s
assume the last), this meant that erasure accounted for
Q = kB T (32 log 2) × 1023 ∼ 8000 Joules(8.5.5)of heat. That’s actually a macroscopic quantity, and the laws of thermodynamics say that it’s
impossible to do better with irreversible computation!
But note that this isn’t most of the heat. For example, a currently state of the art GPU like theNvidia Tesla V100 draw about 250 Watts of power and perform at max about 1014 ﬂop/s. Thismeans their theoretical minimum power draw is
Q = kB T (16 log 2) × 1014 f lop/s ∼ 10−5 Watts(8.5.6)

Thus state of the art GPUs are still tens of millions of times less eﬃcient than the theoreticalminimum. We’re much, much further from the theoretical limits of computation than we are fromthe theoretical limits of heat engine eﬃciency.
In principle we can do even better through reversible computation. After all, there’s no reason to
make erasures. For example, when adding we could perform an operation mapping(x, y) → (x, x + y)(8.5.7)(58, 23) → (58, 81)(8.5.8)for example

so that no information is erased. In this case, we could in principle perform any computation we like
without producing any waste heat at all. But we need to keep all of the input information aroundto avoid creating entropy and using up energy.

9Transport, In Brief

Let’s get back to real physics and discuss transport. This is a fun exercise. We’ll study the transportof heat (energy), of momentum, and of particles.
60

9.1

Heat Conduction

We’ll talk about radiation later, in the quantum stat mech section. And discussing convection isbasically hopeless, as ﬂuid motions and mixings can be very complicated. But we can try to discuss
conduction using some simple models and physical intuition.
The idea is roughly that molecules are directly bumping into each other and transferring energy.
But this can also mean that energy is carried by lattice vibrations in a solid, or by the motion offree electrons (good electrical conductors tend to have good heat conductivity too).
We can start by just making a simple guess that
ΔT δtAδx(9.1.1)

dQ
dT
∝A
dt
dx

(9.1.2)Q∝or

The overall constant is called the thermal conductivity of a given ﬁxed material. So
dQ
dT= −kt Adt
dx

(9.1.3)

which is called the Fourier heat conduction law.
Thermal conductivities of common materials vary by four orders of magnitude, in W/(mK): air
0.026, wood 0.08, water 0.6, glass 0.8, iron 80, copper 400. In common household situations thinlayers of air are very important as insulation, and can be more important than glass itself.
Heat Conductivity of an Ideal Gas
Let’s derive the simplest example of this, the conductivity for an ideal.
An important relevant parameter is the mean free path, which is the distance a molecule cango before interacting with another molecule. It’s typically much larger than the distance between
molecules in a dilute gas. Let’s estimate it.
Collisions happen when molecules get within 2r of each other, where r is some rough estimateof the ‘size’ the molecule. Really this is related to the cross section for interaction. Anyway,
molecules collide when the volume swept out
π(2r)2 `(9.1.4)is of order the volume per molecule, ieVN(9.1.5)V
4πr2 N

(9.1.6)π(2r)2 ` ≈
or
`≈

61

for the mean free path `. This is an approximation in all sorts of ways, but it’s roughly correct.
In air the volume per particle is, using the ideal gas approximation, aboutVkT≈≈ 4 × 10−26 m3
NP(9.1.7)at room temperature. If the radius of nitrogen or oxygen molecules is about 1.5 angstroms, thenthe mean free path is about 150 nm, which is about 40 times greater than the average separation
between air molecules. The average collision time isΔ̄t =`≈ 3 × 10−10 sv̄

(9.1.8)so this gives another relevant quantity. Apparently they’re colliding very frequently!
Now let’s study heat conduction. Imagine a surface across which we are studying heat ﬂow. Theenergy crossing this surface will be about UL /2 from the left and UR /2 from the right, if we study atime Δt and a thickness away from the surface of order `. This leads to a net heat ﬂow
111dT
Q = (U1 − U2 ) = − CV (T2 − T1 ) = − CV `
2
2
2
dx

(9.1.9)This leads to a prediction for the thermal conductivity as
kt =

CV
CV `
=
`v̄
2AΔt
2V

(9.1.10)in terms of the MFP and average velocity. For an ideal gas
fP
CV
=
V
2T

(9.1.11)

where f is the number of active DoF per molecule. √This suggests√ a general scaling law that since v̄ ≈ T and ` is independent of temperature, we
expect kT ∝ T . This has been conﬁrmed for a wide variety of gases.

9.2

Viscosity

Another quantity that you may have heard of is viscosity. It concerns the spread of momentum
through a ﬂuid.
To understand this, we want to think of a picture of a ﬂuid ﬂowing in the x̂ direction, but wherethe velocity of the ﬂuid changes in the z direction. This could lead to chaotic turbulence, but let’s
assume that’s not the case, so that we have laminar ﬂow.
In almost all cases, as one might guess intuitively, ﬂuids tend to resist this shearing, diﬀerential
ﬂow. This resistance is called viscosity. More viscosity means more resistance to this diﬀerential.
It’s not hard to guess what the viscous force is proportional to. We’d expect that
Fx
dux
∝
A
dz
62

(9.2.1)and the proportionality constant is the viscosity, ie
dux
Fx
∝η
A
dz

(9.2.2)for coeﬃcient of viscosity η. Although Fx /A has units of pressure, it isn’t a pressure, rather its
shear stress.
Viscosities vary a great deal with both material and temperature. Gases have lower viscosities
than ﬂuids. And ideal gases have viscosity independent of pressure but increasing with temperature.
This can be explained because viscosity depends on momentum density in the gas, the mean freepath, and the average thermal speed. The ﬁrst two depend on the density, but this cancels between
them – a denser gas carries more momentum, but it carries it less far! But since molecules
move√
faster at higher temperature, more momentum gets transported, and so in a gas η ∝ T just as forheat conductivity.
In a liquid, viscosity actually decreases with temperature because the molecules don’t stick toeach other as much or as well at higher temperature, and this ‘sticking’ is the primary cause of
liquid viscosity. Note that in eﬀect η = ∞ for a solid, where ‘sticking’ dominates.

9.3

Diﬀusion

Now let’s talk about the spread of particles from high concentration to low concentration.
Let’s imagine the density n of particles increases uniformly in the x̂ direction. The ﬂux is the~ Once again we’d guess thatnet number of particles crossing a surface, we’ll write it as J.
Jx = −Ddndx(9.3.1)

where D is the diﬀusion coeﬃcient, which depends on both what’s diﬀusing and what it’s diﬀusing
through. Diﬀusion for large molecules is slower than for small molecules, and diﬀusion is much
faster through gases than through liquids. Values ranging from 10−11 to 10−5 m2 /s can be found.
Diﬀusion coeﬃcients increase with temperature, because molecules move faster.
Diﬀusion is a very ineﬃcient way for particles to travel. If I make a rough estimate for dye inwater that I want roughly all the dye to move through a glass of water then
N
N/V
=D
AΔt
δx

(9.3.2)

V δx
δx2
≈
AD
D

(9.3.3)and V = Aδx, then i have thatΔt ≈for a glass of water with δx ≈ 0.1 m and a D ≈ 10−9 m2 /s, it would take 107 seconds, or about 4
months!
Real mixing happens through convection and turbulence, which is much, much faster, but also
more complicated.
6310

Quantum Statistical Mechanics

Let’s get back to our main topic and (oﬃcially) begin to discuss quantum statistical mechanics.

10.1

Partition Functions for Composite Systems

We’d like to understand what happens if we combine several diﬀerent systems together – what is thepartition function for the total system, in terms of the partition functions for the constituents?
The simplest and most obvious case is when we have two non-interacting distinguishable systems.
Then we have
X
Ztotal =e−βE1 (s1 ) e−βE2 (s2 ) = Z1 Z2(10.1.1)
s1 ,s2This follows because every state of the total systems is (s1 , s2 ), the Cartesian product. And
furthermore, there is no relation or indistinguishability issue between states in the ﬁrst and second
system.
I put this discussion under the Quantum Stat Mech heading because, in many cases, the systems
are indistinguishable. In that case the state stotal = (s1 , s2 ) = (s2 , s1 ), ie these are the same state.
We can approximate the partition function for that case by1
Ztotal ≈ Z1 Z2
2(10.1.2)

but this is only approximate, because we have failed to account correctly for s1 = s2 , in which case
we don’t need the 1/2. In the limit that most energy levels are unoccupied, this isn’t a big problem.
We’ll ﬁx it soon.
These expressions have the obvious generalizations
Ztotal =N
Y

Zi(10.1.3)

i=1

for distinguishable and
N

1 Y
Zi
Ztotal =N ! i=1(10.1.4)

for indistinguishable particles or sub-systems.
Ideal Gas from Boltzmann Statistics
The natural application of our approximations above is to ideal gases. We still haven’t computed Z
for an ideal gas!

64

The total partition function is
Ztotal =

1 N
Z
N! 1

(10.1.5)

in terms of the partition function for a single gas molecule, Z1 . So we just have to compute that.
To get Z1 we need to add up the Boltzmann factors for all of the microstates of a single moleculeof gas. This combines into
Z1 = Ztr Zint

(10.1.6)

where the ﬁrst is the kinetic energy from translational DoF, and the latter are internal states suchas vibrations and rotations. There can also be electronic internal states due to degeneracies in the
ground states of the electrons in a molecule. For example, oxygen molecules have 3-fold degenerate
ground states.
Let’s just focus on Ztr . To do this correctly we need quantum mechanics.
This isn’t a QM course, so I’m mostly just going to tell you the result. A particle in a 1-d box
must have momentum
pn =

h
n
2L

(10.1.7)

as it must be a standing wave (with Dirichlet boundary conditions). So the energy levels are
p2n
h2 n2
=
En =
2m
8mL2

(10.1.8)

From the energies, we can write down the partition function
Z1d =X −β h2 n2
e 8mL2(10.1.9)nUnless the box is extremely small, or T is extremely small we can approximate this by the integral
Z ∞
h2 n2
L
dne−β 8mL2 =(10.1.10)Z1d ≈h√2πmkB T0

where we can deﬁne a quantum length scale`Q = √h
2πmkB T

(10.1.11)

It’s roughly the de Broglie wavelength of a particle of mass m with kinetic energy kT . At room
temperature for nitrogen, this length is ∼ 10−11 meters.
The partition function in 3d is justZ3d = Z13d65(10.1.12)

because the x, y, z motions are all independent. This gives
Z3d =

V
`3Q

(10.1.13)

So the total partition function for an ideal gas is
1
Z≈
N!

V
Zint
`3Q

!N
(10.1.14)

where we don’t know Zint and it depends on the kind of molecule.
We can now compute the expected energy. Note thatlog Z = N (log V + log Zint − log N − 3 log `Q + 1)

(10.1.15)

and we can ﬁnd
∂
log Z
∂β

(10.1.16)

3
U = N kT + Uint2

(10.1.17)

U =−This gives

which conﬁrms equipartition once again, plus the interaction term. The heat capacity is
3∂Uint
CV = N k +∂T2(10.1.18)

We can compute the other properties of our gas using
F = −kT log Z = Fint − N kT (log V − log N − 3 log `Q + 1)

(10.1.19)

For example we can ﬁnd the pressure, entropy, and chemical potential this way. We will once again
recover expressions we’ve seen earlier in the semester.

10.2

Gibbs Factors

We can modify our original derivation of Boltzmann factors to account for the possibility thatparticles are exchanged between the reservoir and the system. The result are Gibbs factors andthe Grand Canonical Ensemble. The latter means that we ﬁx Temperature and Chemical
Potential. Whereas in the microcanonical ensemble we ﬁxed energy and particle number, and in
the canonical ensemble we ﬁxed temperature and particle number.
Gibbs factors will be useful soon for quantum statistical physics because they allow us to reorganize our thinking in terms of the number of particles in each state, rather than in terms of statesas given by a conﬁguration of a ﬁxed number of particles.
66

Recall that
P (s2 )= eSR (s2 )−SR (s1 )P (s1 )

(10.2.1)

Now we need to use
dSR =1(dUR + P dVR − µdNR )T(10.2.2)

1(E(s2 ) − E(s1 ) − µN (s2 ) + µN (s1 ))T(10.2.3)

to ﬁnd that
SR (s2 ) − SR (s1 ) = −where E and N refer to the system. Thus we ﬁnd that the Gibbs factor ise−(E−µN )/(kB T )(10.2.4)

and so the probabilities are
P (s) =

1 −(E−µN )/(kB T )eZ(10.2.5)

Xe−(E(s)−µN (s))/(kB T )(10.2.6)

where
Z=

s

where the sum includes all states, ranging over all values of E and all N . This is the promised
grand partition function or grand canonical distribution. In the presence of more types ofparticles, we have chemical potentials and numbers for each.
Carbon Monoxide Poisoning
Carbon monoxide poisoning illustrates the use of the grand canonical ensemble.
We can idealize hemoglobin as having two states, occupied and unoccupied (by oxygen). So
Z = 1 + e−(−µ)/kT(10.2.7)

where  = −0.7 eV. The chemical potential µ is high in the lungs, where oxygen is abundant, but
lower in cells where oxygen is needed. Near the lungs the partial pressure of oxygen is about 0.2
atm, and so the chemical potential is!
V Zintµ = −kT log≈ −0.6eV(10.2.8)
N `3Q
So at body temperature 310 K we ﬁnd that the Gibbs factor ise−(−µ)/kT ≈ 40
67

(10.2.9)

and so the probability of occupation by oxygen is 98%.
But things change in the presence of carbon monoxide, CO. Then there are three possible states00

Z = 1 + e−(−µ)/kT + e−( −µ )/kT(10.2.10)

The CO molecule will be less abundant. If it’s 100 times less abundant, thenδµ = kT log 100 ≈ 0.12 eV(10.2.11)

But CO binds more tightly to hemoglobin, so that 0 ≈ −0.85 eV. In total this means
00e−( −µ )/kT ≈ 120(10.2.12)

which means that the oxygen occupation probability sinks to 25%. This is why CO is poisonous.

10.3

Bosons and Fermions

Now let’s correct our mistake, where we took
Z≈

1 N
Z
N! 1

(10.3.1)

This was approximate because it ignores what happens when two particles are in the same state.
In fact, there are two types of particles, fermions and bosons. We can put as many identicalbosons as we like in the same state, but due to the Pauli exclusion principle, identical fermionscannot be in the same state. This has extremely important implications for statistical physics. Notethat bosons always have integer spin, and fermions always have half-integer spin.
Note that this idea is irrelevant in the classical limit, when Z1  N , or when
V /N  `3Q

(10.3.2)

When this is violated, the wavefunctions of the particles start to overlap, and quantum eﬀects
become important. Systems that are quantum are either very dense or very cold, since we recall thath
`Q = √
2πmkB T

(10.3.3)

We can picture this as a situation where the particle’s individual wavefunctions start to overlap. We
can’t separate them (make the wavefunctions more narrow) without giving them a lot of momentum,
increasing the energy and temperature of the gas.
Distribution Functions
We introduced the Gibbs ensemble to make computing the distribution functions easier. The idea is
to focus on individual, speciﬁc states that (some number of ) particles can be.
68

If we have a given single-particle state, then the probability of it being occupied by n particles is
P (n) =

1 − n (−µ)e kTZ(10.3.4)

So we can use this to determine the Z for fermions and bosons.
In the case of fermions, the only occupancy allowed is n = 0, 1. That means that for any given
state, we simply have1

Z = 1 + e− kT (−µ)(10.3.5)

This allows us to compute the occupancyn̄ = 0P (0) + P (1) =

1
e

1
+ kT
(−µ)

+1

(10.3.6)

Note the crucial sign ﬂip in the denominator! This is the Fermi-Dirac distribution. The F-D
distribution vanishes when   µ and goes to 1 when   µ. Draw it!
In the case of bosons all values of n are allowed. That means that
Z =

∞
X

n

e− kT (−µ)

n=0

=

1
1

1 − e− kT (−µ)(10.3.7)

by summing the geometric series. Notice that crucially the signs in the denominator here are
diﬀerent.
Let’s look at the expected occupancy. This is
n̄ =

∞
X

nP (n)

n=0

= −

∂
log Z
∂x

(10.3.8)1( − µ). By computing directly with this formula we ﬁnd
where x = kT

n̄ =

1
1
+ kT
(−µ)

e

−1

(10.3.9)

where once again we emphasize that the signs here are crucial! This is the Bose-Einstein
distribution.
Notice that the B-E distribution goes to inﬁnity as  → µ. This means that in that limit, thestate achieves inﬁnite occupancy! In the case of fermions this was impossible because occupancy can
only be 0 or 1. This divergence is Bose condensation. Draw it!

69

Let’s compare these results to the classical limit, where we’d have a Boltzmann distribution.
In the Boltzmann distributionn̄ = e−(−µ)/kT  1(10.3.10)

Draw it! I wrote the last inequality because the classical limit obtains when we don’t have many
particles in the same state. In this limit, it’s ﬁne to simply ignore large occupancies. Note that thisis equal to both the F-D and B-E distributions in the desired limit. The classical limit is low
occupancy.
In the rest of this section we will apply these ideas to simple systems, treating ‘quantum gases’.
That means that we can pretend that the systems are governed by single-particle physics. Thisapplies to electrons in metals, neutrons in neutron stars, atoms in a ﬂuid at low temperature, photons
in a hot oven, and even ‘phonons’, the quantized sound (vibrations) in a solid. We’ll determine µ in
most cases indirectly, using the total number of particles.

10.4

Degenerate Fermi Gases

An extremely important application of quantum stat mech is to gases of fermions, or fermions at
low temperature. Most importantly, this theory describes most of the properties of metals, which
are just ‘gases’ of free electrons. This also applies to nuclei, white dwarves, and neutron stars.
Here low temperature means ‘quantum’, not ‘low compared to room temperature’. That is we
mean

3
h
V
3
≈ (4 nm)3(10.4.1). `Q = √N
2πmkT
where the last is for an electron at room temperature. The reason metals are at ‘low temperatures’is because there is roughly one conduction electron per atom, and the atomic spacing are roughly
0.2 nm. So we’re far outside the limit of Boltzmann statistics, and in fact closer to the limit T = 0.
So let’s start with that.
Zero Temperature
Recall the F-D distributionn̄F D =

1
e

1
(−µ)
kT

+1

(10.4.2)

In the limit that T = 0 this is a step function. All states with  < µ are occupied, and all others areunoccupied.
We callF ≡ µ(T = 0)

(10.4.3)

the Fermi Energy. In this low-temperature state of aﬀairs the fermion gas is said to be ‘degenerate’.
70How can we determine F ? It can be ﬁxed if we know how many electrons are actually present.
If we imagine adding electrons to an empty box, then they simply ﬁll the states from lowest energy
up. To add one more electron you need F = µ of energy, and our understanding that


∂U
µ=
(10.4.4)∂N S,V
makes perfect sense. Note that adding an electron doesn’t change S, since we’re at zero temperatureand the state is unique!
To compute F we’ll assume that the electrons are free particles in a box of volume V = L3 . This
isn’t a great approximation because of interactions with ions in the lattice, but we’ll neglect them
anyway.
As previously seen, the allowed wavefunctions are just sines and cosines, with momenta
h
p~ =
~n
(10.4.5)
2L
where ~n = (nx , ny , nz ). Energies are therefore
h2 2~n(10.4.6)
8mL2
To visualize the allowed states, its easiest to draw them in ~n-space, where only positive integer
coordinates are allowed. Each point describes two states after accounting for electron spins. Energies
are proportional to ~n2 , so states start at the origin and expand outward to ﬁll availabilities.
The Fermi energy is just=

h2 n2max(10.4.7)8mL2
We can compute this by computing the total volume of the interior region in the space, which is
F =

1 4 3
πn3maxN = 2 × × πnmax =3
8 3(10.4.8)

Thus we ﬁnd that
h2
F =
8m



3N
πV

2/3
(10.4.9)

Is this intensive of extensive? What does that mean? (Shape of ‘box’ is irrelevant.)
We can compute the average energy per electron by computing the total energy and dividing.
That is
Z
1
h2 2
U = 2
d3 n
~n
8
8mL2
Z nmaxh2 2=dn(πn2 )~n8mL20πh2 n5max=
40mL2
3(10.4.10)
=
N F
5
71

where the 3/5 is a geometrical factor that would be diﬀerent in more or fewer spatial dimensions.
1
Note that F ∼ few eV, which is much much larger than kT ≈ 40
eV. This is the same as
the comparison between the quantum volume and the average volume per particle. The Fermi
temperature isF
TF ≡& 104 K
(10.4.11)kB
which is hypothetical insofar as metals would liquefy before reaching this temperature.
Using the standard or thermodynamic deﬁnition of the pressure we can ﬁnd
∂U
∂V
∂
= −
∂V

P = −

=

3 h2
N
5 8m



3N
πV

2/3 !

2N F
2U
=
5V
3V

(10.4.12)

which is the degeneracy pressure. Electrons don’t want to be compressed, as they want space!
(Remember this has nothing to do with electrostatic repulsion.)
The degeneracy pressure is enormous, of order 109 N/m2 , but it isn’t measurable, as it is canceledby the electrostatic forces that pulled the electrons into the metal in the ﬁrst place. The bulk
modulus is measureable, and its just


∂P
10 U
B = −V=(10.4.13)∂V T
9 VThis quantity is also large, but its not completely canceled by electrostatic forces, and (very roughly)accords with experiment.
Small Temperatures
The distribution of electrons barely changes when TF  T > 0, but we need to study ﬁnite
temperature to see how metals change with temperature – for example, to compute the heat
capacity!
Usually particles want to acquire a thermal energy kT , but in a degenerate Fermi gas they can’t,
because most states are occupied. So it’s only the state near the Fermi surface that can become
excited. Furthermore, it’s only states within kT of the Fermi surface that can be excited. So thenumber of excitable electrons is proportional to T .
This suggests that
δU ∝ (N kT )(kT ) ∼ N (kT )2(10.4.14)

By dimensional analysis then, since the only energy scale around is F , we should haveδU ≈ N(kT )2
F
72(10.4.15)

This implies thatCV ∝ NkT
F(10.4.16)

This agrees with experiments. Note that it also matches what we expect from the 3rd law ofthermodynamics, namely that CV → 0 as T → 0.
Density of States
A natural object of study is the density of states, which is the number of single-particle states
per unit of energy. For a Fermi gas note that
=

h2 2
~n ,
8mL2

|~n| =

L√8m
h(10.4.17)

so that
L
dn =
h

r

2m
d


This means that we can write integrals like
Z nmaxh2 2U =dn(πn2 )
n
8mL2
0
" 
3/2 #Z F√
π 8mL2
=
d 

2
2
h
0

(10.4.18)(10.4.19)

where the quantity in square brackets is the density of statesg() =πV √3N √(8m)3/2 3  = 3/2 2
h2F(10.4.20)

The total energy is the integral of the number of states at a given energy times that energy g().
Note that this expression does not depend on N except through V ; in the last formula N cancels.
The density of states is a useful concept in a variety of systems. It doesn’t depend on statisticalmechanics, rather determining it is just a quantum mechanics problem.
At zero temperature we haveZ FN=g()d
(10.4.21)
0but at higher temperature we have to multiply by the non-trivial occupancy, which is really theright way to think about zero temperature as well
Z ∞
Z ∞
g()
d
(10.4.22)
N=g()n̄F D ()d =1
e kT (−µ) + 1
0073

similarly for the energyZ ∞Z ∞g()n̄F D ()d =

U=
0

0

g()
e

1
(−µ)
kT

+1

d

(10.4.23)

The zero temperature limit is just a special case.
Note that at T > 0 the chemical potential changes. Its determined by the fact that the integral
expression for N must remain ﬁxed, and since g() is larger for larger , the chemical potential mustchange to compensate. In fact it must decrease slightly. We can compute it by doing the integrals
and determining µ(T ). Then we can determine U (T ) as well.
Sommerfeld Expansion – Evaluating Integrals at Low Temperature
Unfortunately, we can’t do the integrals for N or U explicitly and analytically. But we canapproximate them in the limit
kB T  Fwhich we have seen is a very good approximation.
To get started, we can write√Z ∞
N = g0d
1e kT (−µ) + 1
0(10.4.24)(10.4.25)

The only interesting region in the integral is when  ≈ µ, as otherwise the n̄F D factor is simply 1 or
0. So to isolate this region, we can integrate by parts, writing


Z ∞
∞
2 3/2
2
d¯
nF D
3/2

N = g0  n̄F D () + g0−d
(10.4.26)
3
3
d
0
0
The boundary term vanishes, and so what we’re left with is nicer, because d¯ndF D is basically zero
except near  ≈ µ.
We will now start writing
x=

−µ
kT

(10.4.27)

to simplify the algebra. In these terms we see that
nF D
d¯
1
ex
=
d
kT (ex + 1)2(10.4.28)

So we need to evaluate the integral
Z ∞
2
1
ex
N =
g0
3/2 dx + 1)2
kT
(e
3
Z0 ∞
2
ex
=
g0
3/2 dx
x + 1)2
(e
3
−µ/kT
74

(10.4.29)

So far this was exact, but now we will start making approximations.
First of all, note that µ/kT  1 and that the integrand falls oﬀ exponentially when −x  1.
This means that we can approximate
Z ∞2ex
N ≈ g0(10.4.30)
3/2 dx
x + 1)23(e−∞
up to exponentially small corrections. Then we can expand3
3
3/2 = µ3/2 + ( − µ) µ1/2 + ( − µ)2 µ−1/2 + · · ·2
8(10.4.31)

So we’re left with2
N ≈ g0
3

Z ∞



ex
3
3 (kT )2 2
1/2
3/2
µ + kT µ x +x + · · · dxx22
8 µ1/2−∞ (e + 1)

(10.4.32)

Now the integral can be performed term-by-term. This provides a series expansion of the result,while neglecting some exponentially small corrections.
The ﬁrst term is


Z ∞
Z ∞
ex
1
dx = 1(10.4.33)
dx =
−∂x x
x
2
e + 1)−∞ (e + 1)−∞
The second term vanishes because
Z ∞
Z ∞
xex
x
dx
=
dx = 0x2x−x + 1)−∞ (e + 1)−∞ (e + 1)(esince the integrand is odd. Finally, the integral
Z ∞x 2 ex
π2
dx
=
x
2
3
−∞ (e + 1)

(10.4.34)(10.4.35)

though it’s harder to evaluate.
Assembling the pieces, we ﬁnd that
π 2 (kT )2
2
3/2
N =
g0 µ
1+
+ ···
3
8 µ2
 3/2 µπ 2 (kT )2
= N
1+
+ ···
8 µ2
F

(10.4.36)

This shows that µ/F ≈ 1, and so we can use that in the second term to solve and ﬁndµπ 2 (kT )2
≈1−
+ ···
F
12 2F75(10.4.37)

so the chemical potential decreases a bit as T increases.
We can evaluate the energy integrals using the same tricks, with the result that
3 µ5/2 3π 2 (kT )2
U = N 3/2 +
N
+ ···
5 F
8
F

(10.4.38)

We can plug in for µ to ﬁnd that3π 2 (kT )2
U = N F + N
+ ···
5
4
F

(10.4.39)

so that we have the energy in terms of T . As expected, it increases with temperature with theanticipated parametric dependence.

10.5

Bosons and Blackbody Radiation

Now we will consider what we expect to ﬁnd for electromagnetic radiation inside a box.
Classically, this radiation takes the form of excitations of the electromagnetic ﬁeld. Without
quantum mechanics, there is no notion of occupancy, and instead each degree of freedom isapproximately quadratic. This means that we have 12 kT of energy per DoF. But there are inﬁnitely
many modes at short distances, leading to an inﬁnite energy! This is called the ultraviolet catastrophe.
Quantum Mechanics was invented, in part, to solve this problem.
Planck Distribution
Instead of equipartition, each mode of the electromagnetic ﬁeld, which is its own harmonic oscillator,
can only have energy
En = n~ω(10.5.1)

where the values of ω are ﬁxed by the boundary conditions of the box. This n is the number ofphotons!
The partition function isZ=

1
1 − e−β~ω

(10.5.2)

for the Bose-Einstein distribution. So the average energy in an oscillator isĒ = −d~ω
log Z = β~ωdβe−1(10.5.3)

The number of photons is just given by n̄BE . But in the context of photons, its often called the
Planck distribution.
This solves the ultraviolet catastrophe, by suppressing the high energy contributions exponentially.
This really required quantization, so that high energy modes are ‘turned oﬀ’.
76

Note that for photons µ = 0. At equilibrium this is required by the fact that photons can be
freely created or destroyed, ie∂F(10.5.4)= 0 = µγ
∂N T,VThis explains why µ didn’t appear in the Planck distribution.
Photons in a Box
We would like to know the total number and energy of photons inside a box.
As usual, the momentum of these photons is
pm =

hm
2L

(10.5.5)

but since they are relativistic, the energy formula is now
 = pc =hcm
2L(10.5.6)

This is also just what we get from their frequencies.
The energy in a mode is just the occupancy times the energy. This is
U =2

hc|m|
1
hc|m|
2L e 2LkT − 1
m ,m ,mXxy(10.5.7)z

Converting to an integral, we have
Z ∞

π hcm
1
dm m2
hcm
2
L e 2LkT − 1
0
Z ∞
3
πhcm
1
=
dm
hcm
2L e 2LkT − 10

U =(10.5.8)

We can then change variables to  = hcm
and use L3 = V to ﬁnd
2L
U
=
V

Z ∞
d
0

8π3
1
3
/kT
(hc) e−1(10.5.9)

the integrand is the spectrum, or energy density per unit photon energyu() =

8π3
1
3
/kT
(hc) e−1(10.5.10)

The number density is just u/. The spectrum peaks at  ≈ 2.8kT ; the 3 comes from living in
3-dimensions.
77

To evaluate the integral, its useful to change variables to x = /kT , so that all of the physicalquantities come out of the integral, giving
ZU
8π(kT )4 ∞ x3=
dx
(10.5.11)V(hc)3ex − 10This means that one can determine the temperature of an oven by letting a bit of radiation leak out
and looking at its color.
Total Energy
Doing the last integral givesU
8π 5 (kT )4=
V
15(hc)3(10.5.12)

Note that this only depend on kT , as the other constants are just there to make up units. Since its
an energy density, pure dimensional analysis could have told us that it had to scale as (kT )4 .
Numerically, the result is small. At a typical oven temperature of 460 K, the energy per unit
volume is 3.5 × 10−5 J/m3 . This is much smaller than the thermal energy of the air inside the oven...
because there are more air molecules than photons. And that’s because the photon wavelengths are
smaller than the typical separations between air molecules at this temperature.
Entropy
We can determine the entropy by computing CV and then integrating CTV dT . We have that
CV =∂U
= 4aT 3∂T V

(10.5.13)

5 4

8π k
where a = 15(hc)
3 . Since our determinations were fully quantum, this works down to T = 0, whichmeans that
Z T
CV
4
(10.5.14)
S(T ) =
dt = aT 3t
30

The total number of photons scales the same way, but with a diﬀerent coeﬃcient.
CMB
The Cosmic Microwave Background is the most interesting photon gas; it’s at about 2.73 K. So its
spectrum peaks at 6.6 × 10−4 eV, with wavelength around in a millimeter, in the far infrared.
The CMB has far less energy than ordinary matter in the universe, but it has far more entropy –about 109 units of S per cubic meter.

78

Photons Emission Power
Now that we have understood ‘photons in a box’, we would like to see how radiation will be emitted
from a hot body. A natural and classical starting point is to ask what happens if you poke a hole inthe box of photons.
Since all light travels at the same speed, the spectrum of emitted radiation will be the same as
the spectrum in the box. To compute the amount of radiation that escapes, we just need to do somegeometry.
To get out through the hole, radiation needs to have once been in a hemisphere of radius R fromthe hole. The only tricky question is how much of the radiation at a given point on this hemispheregoes out through a hole with area A. At an angle θ from the perpendicular the area A looks like it
has area
Aef f = A cos θ(10.5.15)

So the fraction of radiation that’s pointed in the correct direction, and thus gets out is
Z π/2

Aef f
4πR2
0
Z π/2
A
A
=
dθ sin θ cos θ =240dθ2πR2 sin θ(10.5.16)

Other than that, the rate is just given by the speed of light times the energy density, so we ﬁndcAU
2π 5 A(kT )4=4V
15h3 c2(10.5.17)

In fact, this is the famous power per unit area blackbody emission formula
P =A

2π 5 k 4 4
T = AσT 4
3
2
15h c

(10.5.18)

where σ is known as the Stefan-Boltzmann constant, with value σ = 5.67 × 10−8 mW
The
2K4 .
dependence on the fourth power of the temperature was discovered by Stefan empirically in 1879.
Universality of Black Body Spectrum
This rule actually applies to emission from any non-reﬂecting surface, and is referred to as blackbody
radiation. The argument that the hole and blackbody are the same is simply to put them in contact
with each other via radiation, and demand that since they’re in equilibrium, to avoid violating the
2nd law of thermodynamics, they must emit in the same way. Note that by adding ﬁlters, this
implies the full spectra must be the same.
The same argument implies that the emissivity e of the material at any given wavelength mustbe such that 1 − e is its reﬂection coeﬃcient. So good reﬂectors must be poor emitters, and viceversa. In this case the spectrum will be modiﬁed, since e generally depends on wavelength.
79

Sun and Earth
We can use basic properties of the Earth and Sun to make some interesting deductions.
The Earth receives 1370 W/m2 from the sun (known as the solar constant). The earth’s 150
million km from the sun. This tells us the sun’s total luminosity is 4 × 1026 Watts.
The sun’s radius is a little over 100 times earth’s, or 7 × 108 m. So its surface area is 6 × 1018 m2 .
From this information, if we assume an emissivity of 1, we can ﬁnd thatT =luminosityσA

1/4= 5800 K(10.5.19)

This means that the sun’s spectrum should peak at
 = 2.82kT = 1.41 eV(10.5.20)

which is in the near infrared. This is testable and agrees with experiment. This is close to visible
red light, so we get a lot of the sun’s energy in the visible spectrum. Note that ‘peak’ and ‘average’
aren’t quite the same here. (Perhaps in some rough sense evolution predicts that we should be able
to see light near the sun’s peak?)
We can also easily estimate the Earth’s equilibrium temperature, assuming its emission andabsorption are balanced. If the power emitted is equal to the power absorbed and emissivity is 1
then(solar constant)πR2 = 4πR2 σT 4

(10.5.21)

T ≈ 280 K(10.5.22)

which gives

This is extremely close to the measured average temperature of 288 K.
Earth isn’t a perfect blackbody, as clouds (and other stuﬀ) reﬂect about 30% of the light, sothat the earth’s predicted temperature is 255 K. Emissivity and absorption don’t balance because
Earth emits in Infrared and mostly absorbs in near IR and visible.
However, the earth’s surface is warmer than the upper atmosphere, so while 255 K applies to theatmosphere, the surface is warmer due to the greenhouse eﬀect. If we treat the atmosphere as a thinlayer intermediate between the earth’s surface, we get an extra factor of 2 in power on the surface,which brings the temperature back up by 21/4 to 303 K. This is a bit high, but roughly right again.

10.6

Debye Theory of Solids

Now we get to see why I was complaining about the Einstein model early in the course.
The point is that the vibrations in a solid are sound waves, and so they have a range of wavelengths
and corresponding frequencies, with λf = cs . So it doesn’t make sense to treat them as copies ofthe same harmonic oscillator.
80

In fact, the ‘phonons’ of sound are extremely similar to the photons of light, except that phonons
travel at cs , have 3 polarizations, can have diﬀerent speeds for diﬀerent polarizations (and evendirections!), and cannot have wavelength smaller than (twice) the atomic spacing.
All this means that we have a simple relation
 = hf =

hcs
|~n|
2L

(10.6.1)

and these modes have a Bose-Einstein distributionn̄BE =

1(10.6.2)e/(kT ) − 1

with vanishing chemical potential, as phonons can be created and destroyed.
The energy is also the usual formulaX
U =3n̄BE ()(10.6.3)

nx ,ny ,nzThe only complication is that we need to decide
what range of sum or integration to do for the ~n.
√Formally we should sum ni from 1 to 3 N . But that would be annoying, so let’s pretend thatinstead we have a spehere in n-space with
nmax =

6Nπ1/3(10.6.4)

so that the total volume in n-space is the same. This approximation is exact at both low and hightemperatures! Why?
Given this approximation, we can easily convert our sum into spherical coordinates and computethe total average energy. This is
Z nmax
4πn2

U = 3
dn

8 e kT − 10Z nmax
3π
hcs
n3
=
dn
(10.6.5)hcs n
2 0
2L e 2LkT−1Don’t be fooled – this is almost exactly the same as our treatment of photons.
We can change variables to
x=

hcs
n
2LkT

(10.6.6)

This means that we are integrating up to
hcs
hcs
xmax =
nmax =
2LkT
2kT
81



6N
πV

1/3
=

TD
T

(10.6.7)

where
hcs
TD =
2k



6N
πV

1/3
(10.6.8)

is the Debye Temperature.
The integral for energy becomes
9N kT 4
U=
TD3

Z TD /T0x3dxex − 1

We can just integrate numerically, but what about physics?
In the limit that T  TD , the upper limit of the integral is very small, so 3
Z TD /TZ TD /T
x3
1 TD
2
dx ≈
x dx ≈
ex − 1
3 T
00(10.6.9)(10.6.10)

so that we ﬁnd
U = 3N kT(10.6.11)

in agreement with equipartition and also the Einstein model.
In the opposite limit of T  TD the upper limit doesn’t matter, as its almost inﬁnity, and weﬁndU=

3π 4 N k 4
T
5TD3(10.6.12)

which means that phonons are behaving exactly like photons, and physics is dominated by the
number of excited modes.
This means that at low temperature
12π 4 N k 3
CV =
T
5TD3

(10.6.13)

and this T 3 rule beautifully agrees with experimental data. Though for metals, to really match, weneed to also include the electron contribution.
Some Comments on Debye
Let’s compare this to the Fermi energy
h2
F =
8me



3N
πV

82

2/3
(10.6.14)

for electrons. Notice that the power-law dependence on the density is diﬀerent because electrons are
non-relativistic. In particular, we have 1/3D
cs me V∝
(10.6.15)hN
FThis shows that if V /N doesn’t vary all that much, the main thing determining D is the speed ofsound cs . If we had time to cultivate a deeper understanding of materials, we could try to estimate
if we should expect this quantity to be order one.
You should also note that for T  TD we might expect that solids will melt. Why? If not, itmeans that the harmonic oscillator potentials have signiﬁcant depth, so we can excite many, many
modes without the molecules escaping.

10.7

Bose-Einstein Condensation

So far we have only treated bosons where µ = 0, so that particles can be created and destroyed. But
what about bosonic atoms, where N is ﬁxed? In that case we’ll have to follow the same procedure
as for Fermions. But we get to learn about B-E condensation, another rather shocking consequenceof quantum mechanics.
Note that in the limit T → 0 this is obvious – the bosons will all be in their ground state, whichis the same state for all N atoms. That this state has energy
0 =

3h2
h2
(1
+
1
+
1)
=
8mL2
8mL2

(10.7.1)

which is a very small energy if L is macroscopic.
Let’s determine µ in this limit. We see that
N0 =

1
e

0 −µ
kT

−1

≈

kT
0 − µ(10.7.2)

This means that µ = 0 at T = 0, and just a bit larger for small, positive T . But the question is,
how low must T be for N0 to be large?
We determine µ by ﬁxing N , and it’s usually easiest to think of this as
Z ∞1N=d g() −µ(10.7.3)
e kT − 10Here g() is the usual density of states. For spin-zero bosons in a box, this is identical to the case ofelectrons (ignoring a 2 for spin) so that

3/2
√
2
2πm
g() = √V(10.7.4)h2πNote that this is an approximation assuming a discretum of states, which isn’t a good approximation,
really, for very small energies.
83

As we’ve seen before, we cannot do the integral analytically.
Let’s see what happens if we just choose µ = 0. In that case, using our usual x = /kT typechange of variables we have
3/2 Z ∞ √
2
2πmkT
xdx
N=√
(10.7.5)
V
2
x
h
e −1
π
0
Numerically this givesN = 2.612

2πmkT
h2

3/2
V

(10.7.6)

This really just means that there’s a particular Tc for which it’s true that µ = 0, so that
 2/3h2
N
kTc = 0.527
2πm V(10.7.7)

At T > Tc we know that µ < 0 so that the total N is ﬁxed. But what about at T < Tc ?
Actually, our integral representation breaks down at T < Tc as the discreteness of the states
becomes important. The integral does correctly represent the contribution of high energy states, but
fails to account for the few states near the bottom of the spectrum.
This suggests that

3/2
2πmkT
V
(10.7.8)
Nexcited ≈ 2.612
h2
for T < Tc . (This doesn’t really account for low-lying states very close to the ground state.)Thus we have learned that at temperature T > Tc , the chemical potential is negative and all
atoms are in excited states. But at T < Tc , the chemical potential µ ≈ 0 and so 3/2T
Nexcited ≈ N(10.7.9)Tc
and so the rest of the atoms are in the ground state with"
N0 = N − Nexcited ≈ N 1 −TTc3/2 #(10.7.10)

The accumulation of atoms in the ground state is Bose-Einstein condensation, and Tc is called
the condensation temperature.
Notice that as one would expect, this occurs when quantum mechanics becomes important, sothat the quantum length

3
V
h
3
(10.7.11)≈ `Q = √
N
2πmkT
84

So condensation occurs when the wavefunctions begin to overlap quite a bit.
Notice that having many bosons helps – that is for ﬁxed volume
h2
kTc = 0.527
2πm



N
V

2/3

∝ N 2/3 0(10.7.12)

We do not need a kT of order the ground state energy to get all of the bosons into the ground state!
Achieving BEC
BEC with cold atoms was ﬁrst achieved in 1995, using a condensate of about 1000 rubidium-87
atoms. This involved laser cooling and trapping. By 1999 BEC was also achieved with sodium,
lithium, hydrogen.
Superﬂuids are also BECs, but in that case interactions among the atoms are important for
the interesting superﬂuidic properties. The famous example is helium-4, which has a superﬂuid
component below 2.17 K.
Note that superconductivity is a kind of BEC condensation of pairs of electrons, called Cooper
pairs. Helium-3 has a similar pairing-based superﬂuidic behavior at extremely low temperatures.
Why? Entropy
With distinguishable particles, the ground state’s not particularly favored until kT ∼ 0 . But withindistinguishable particles, the counting changes dramatically.
The number of ways of arranging N indistinguishable particles in Z1 1-particle states is(N + Z1 )!
∼ eZ1
N !Z1 !



N
Z1

Z1
(10.7.13)

for Z1  N . Whereas in the distinguishable case we just have Z1N states, which is far, far larger. So
in the QM case with Bosons, at low temperatures we have very small relative multiplicity or
entropy, as these states will be suppressed by e−U/kT ∼ e−N . So we quickly transition to a BEC.

10.8

Density Matrices and Classical/Quantum Probability

As a last comment on quantum statistical mechanics, I want to mention how we actually combine
quantum mechanics with statistics, and to emphasize that statistical probabilities and quantum
probabilities are totally diﬀerent.

11

Phase Transitions

A phase transition is a discontinuous change in the properties of a substance as its environment(pressure, temperature, chemical potential, magnetic ﬁeld, composition or ‘doping’) only change85

inﬁnitesimally. The diﬀerent forms of a substance are called phases. A diagram showing theequilibrium phases as a function of environmental variables is called a phase diagram.
On your problem set you’ll show (if you do all the problems) that BEC is a phase transition.
Phase structure of water has a triple point where all meet, and a critical point where waterand steam become indistinguishable (374 C and 221 bars for water). No ice/liquid critical point,
though long molecules can form liquid crystal phase where molecules move but stay oriented. There
are many phases of ice at high pressure. Metastable phases are possible via super-cooling/heatingetc.
The pressure at which a gas can coexist with its solid or liquid phase is called the vapor
pressure. At T = 0.01 Celsius and P = 0.006 bar all three phases can coexist at the triple point.
At low pressure ice sublimates directly into vapor.
We’ve seen sublimation of CO2 , this is because the triple point is above atmospheric pressure, at
5.2 bars. Note pressure lowers the melting temperature of ice, which is unusual (because ice is lessdense than water).
There are also phase transitions between ﬂuid/superﬂuid and to superconductivity (eg in thetemperature and magnetic ﬁeld plane). Ferromagnets can have domains of up or down magnetization,and there’s a critical point at B = 0 and T = Tc called the Curie point.
Phase transitions have been traditionally classiﬁed as ‘ﬁrst order’ or ‘second order’ depending onwhether the Gibbs energy or its various derivatives are discontinuous; we usually just say ﬁrst orderor ‘continuous’.
Diamond and Graphite
Graphite is more stable than Diamond in the atmosphere, as the Gibbs free energy of a mole ofdiamond is 2900 J higher than that of graphite. Recall
G = U − TS + PV
To analyze what happens under other conditions, we can usefully note that




∂G
∂G
= V,
= −S
∂P T,N∂T P,N(11.0.1)(11.0.2)

Since diamond has a smaller volume than graphite, as pressure increases its G doesn’t increase as
fast as that of graphite, so at 15k atmospheres of pressure diamond becomes more stable. Thus
diamonds must form deep in the earth – deeper than about 50 km.
Graphite has more entropy, so at high temperatures graphite is even more stable compared to
Diamond, and so at higher temperatures Diamonds are more likely to convert to graphite.
In general, many properties of phase transitions follow from thinking carefully about the behaviorof the free energy.

11.1

Clausius-Clapeyron

Given the relations for derivatives of G, it’s easy to see how phase boundaries must depend on P
and T .
86

At the boundary we must have that the two phases have G1 = G2 . But this also means thatdG1 = dG2

(11.1.1)

along the phase boundary, which means that−S1 dT + V1 dP = −S2 dT + V2 dP(11.1.2)dP
S1 − S2=dT
V1 − V2(11.1.3)

so thatand this determines the slope of the phase boundary (eg between a liquid and gas transition).
We can re-write this in terms of the latent heat L via
L = T (S1 − S2 )(11.1.4)

and use the change in volume ΔV = V1 − V2 so that
dP
L
=
dT
T ΔV

(11.1.5)

This is the Clausius-Clapeyron relation. We can use it to trace out phase boundaries.

11.2

Non-Ideal Gases

Gases become ideal in the limit of very low density, so that N/V → 0. Thus it’s natural to treat
this as a small parameter and expand in it. For this purpose, motivated by the ideal gas law, it’snatural to write
 3
 2
P
N
N
N
=
+ B2 (T )+ B3 (T )+ ···(11.2.1)
kTVVV
We can actually try to compute the functions Bn (T ) by doing what’s called a ‘cluster expansion’ in
the interactions of the gas molecules.
But for now, let’s just see what we get if we include the ﬁrst correction. A particular equationof state that’s of this form, and which keeps only the ﬁrst two terms, is the famous va der Waals
equation of stateN2
P + a 2 (V − N b) = N kT
(11.2.2)
V
or
P =

N kT
N2
−a 2
V − NbV
87(11.2.3)

for constants a and b. This formula is just an approximation! Note that the absence of higher orderterms means that we are neglecting eﬀects from interactions among many molecules at once, which
are certainly important at high density.
We can also write the vdW equation more elegantly in terms of purely intensive quantities as
P =

kT
1
−a 2
v−b
v

(11.2.4)

where v = V /N . This makes it clear that the total amount of stuﬀ doesn’t matter!
The modiﬁcation by b is easy to understand, as it means that the ﬂuid cannot be compressed
down to zero volume. Thus b is a kind of minimum volume occupied by a molecule. It’s actuallywell approximated by the cube of the average width of a molecule, eg 4 angstroms cubed for water.
The a term accounts for short-range attractive forces between molecules. Note that it scales
like N 2 /V 2 , so it corresponds to forces on all the molecules from the molecules, and it eﬀectively
decreases the pressure (for a > 0), so it’s attractive. The constant a varies a lot between substances,in fact by more than two orders of magnitude, depending on how much molecules interact.
Now let’s investigate the implications of the van der Waals equation. We’ll actually be focusedon the qualitative implications. Let’s plot pressure as a function of volume at ﬁxed T . At large T
curves are smooth, but at small T they’re more complicated, and have a local minimum! How can agas’s pressure decrease when its volume decreases!?
To see what really happens, let’s compute G. We havedG = −SdT + V dP + µdN
At ﬁxed T and N this is just dG = V dP so




∂G
∂P
=V
∂V T,N∂V T,N
From the van der Waals equation we can compute derivatives of P , giving


∂G
2aN 2
N kT V=−
2
V(V − N b)2∂V T,N

(11.2.5)(11.2.6)(11.2.7)

Integrating then gives
G = −N kT log(V − N b) +2aN 2
N 2 bkT−+ c(T )V − Nb
V(11.2.8)

where c(T ) is some constant that can depend on T but not V .
Note that the thermodynamically stable state has minimum G. This is important because at agiven value of pressure P , there can be more than one value of G. We see this by plotting G and P
parametrically as functions of V . This shows the volume changes abruptly at a gas-liquid transition!
We can determine P at the phase transition directly, but there’s another cute, famous method.
Since the total change in G is zero across the transition, we have

I
I 
I
∂G
0 = dG =dP = V dP(11.2.9)∂P T,N88

We can compute this last quantity from the PV diagram. This is called the Maxwell construction.
This is a bit non-rigorous, since we’re using unstable states/phases to get the result, but it works.
Repeating the analysis shows where the liquid gas transition occurs for a variety diﬀerent
temperatures. The corresponding pressure is the vapor pressure where the transition occurs. At
the vapor pressure at a ﬁxed volume, liquid and gas can coexist.
At high temperature, there is no phase boundary! Thus there’s a critical temperature Tc anda corresponding Pc and Vc (N ) at which the phase transition disappears. At the critical point, liquids
and gases become identical! The use of the volume in all of this is rather incidental. Recall thatG = N µ, and so we can rephrase everything purely in terms of intensive quantities.
Note though that while van der Waals works qualitatively, it fails quantitatively. This isn’t
surprising, since it ignores multi-molecule interactions.

11.3

Phase Transitions of Mixtures

Phase transitions become more complicated when we consider mixtures of diﬀerent kinds of particles.
Essentially, this is due to the entropy of mixing. As usual, we should study the Gibbs free energy
G = U + PV − TS(11.3.1)

In the unmixed case, with substances A and B, we would expect
G = (1 − x)GA + xGB(11.3.2)

where x is the fraction and GA/B are the Gibbs free energies of unmixed substances.
But if we account for entropy of mixing, then instead of a straight line, Smix will be largest whenx ≈ 1/2, so that G will be smaller than this straight line suggests. Instead G will be concave up as
a function of x.
We can roughly approximate
Smixing ≈ −N (x log x + (1 − x) log(1 − x))(11.3.3)

where N is the total number of molecules. This is correct for ideal gases, and for liquids or solids
where the molecules of each substance are roughly the same. This leads to
G = (1 − x)GA + xGB − N kB T (x log x + (1 − x) log(1 − x))(11.3.4)

This is the result for an ideal mixture. Liquid and solid mixtures are usually far from ideal, but
this is a good starting point.
Since the derivative of G wrt x at x = 0 and x = 1 is actually inﬁnite, equilibrium phases almost
always contain impurities.
Even non-ideal mixtures tend to behave this way, unless there’s some non-trivial energy or U
dependence in mixing, eg with oil and water, where the molecules are only attracted to their own
kind. In that case the G of mixing can actually be concave down – that’s why oil and water don’tmix, at least at low T . Note that at higher T there’s a competition between the energy eﬀect and89

the entropy eﬀect. Even at very low T , the entropy dominates near x = 0 or x = 1, so oil and water
will have some ‘impurities’.
Note that concave-down G(x) simply indicates an instability where the mixture will split up
into two unmixed substances. In this region we say that the two phases are immiscible or have asolubility gap.
So we end up with two stable mixtures at small x > 0 and large x < 1, and unmixed substances
between. Solids can have solubility gaps too.
You can read about a variety of more complicated applications of these ideas in the text.

11.4

Chemical Equilibrium

Chemical reactions almost never go to completion. For example consider
H2 O ↔ H + + OH −(11.4.1)

Even if one conﬁguration of chemicals is more stable (water), every once in a while there’s enoughenergy to cause the reaction to go the other way.
As usual, the way to think about this quantitatively is in terms of the Gibbs free energy
G = U − TS + PV

(11.4.2)

and to consider the chemical reaction as a system of particles that will react until they reach anequilibrium. We end up breaking up a few H2 O molecuels because although it costs energy, theentropy increase makes up for that. Of course this depends on temperature though!
We can plot G as a function of x, where 1 − x is the fraction of H2 O. Without any mixing wewould expect G(x) to be a straight line, and since U is lower for water, we’d expect x = 0. However,
due to the entropy of mixing, equilibrium will have x > 0 – recall that G0 (x) is inﬁnite at x = 0!
Plot this.
We can characterize the equilibrium condition by the fact that the slope of G(x) is zero. Thismeans thatX
0 = dG =µi dNi(11.4.3)iwhere we assumed T, P are ﬁxed. The sum on the right runs over all species. But we know thatdNH2 O = −1, dNH + = −1, dNOH − = 1(11.4.4)

because atoms are conserved. So this implies thatµH2 O = µH + + µOH −(11.4.5)

at equilibrium. Since the chemical potentials are a function of species concentration, this determines
the equilibrium concentrations!
90

The equilibrium condition is always the same as the reaction itself, with names of chemicals
replaced by their chemical potentials. For instance with
N2 + 3H2 ↔ 2N H3(11.4.6)µN2 + 3µH2 ↔ 2µN H3(11.4.7)

we would have

simply by noting how the various dNi must change.
Note that these results apply beyond ‘Chemistry’, for instance they also apply to the reactions
about nuclei in the early universe!
If the substances involved in reaction are all ideal gases, then recall that we can write theirchemical potentials asµi = µi0 + kT log

Pi
P0(11.4.8)

where µi0 is the chemical potential in a ﬁxed ‘standard’ state when its partial pressure is P0 . So for
example, for the last chemical reaction we can write
PN2 PH3 2
ΔG0
N H3
N2
H2
kT log
=
2µ
−
µ
−
3µ
=
(11.4.9)000N
PN2 H3 P02
which means thatΔGPN2 PH3 2− N kT0=e=K
PN2 H3 P02(11.4.10)

where K is the equilibrium constant. This equation is called the law of mass action by chemists.
Even if you don’t know K a priori, you can use this equation to determine what happens if you add
reactants to a reaction.
For this particular reaction – Nitrogen ﬁxing – we have to use high temperatures and very highpressures – this was invented by Fritz Haber, and it revolutionized the production of fertilizers andexplosives.
Ionization of Hydrogen
For the simple reaction
H ↔p+ewe can compute everything from ﬁrst principles. We have the equilibrium condition


PH P0
kT log= µp0 + µe0 − µH0
Pp P e91(11.4.11)(11.4.12)

We can treat all of these species as structureless monatomic gases, so that we can use"3/2 #
kT 2πmkTµ = −kT log
Ph2(11.4.13)

The only additional issues is that we need to subtract the ionization energy of I = 13.6 eV from µH
0 .
Since mp ≈ mH we have"
3/2 #
kT 2πmkT
I
PH P0
− log
= kT log−(11.4.14)
2P0hkT
P p PeThen with a bit of algebra we learn that
Pp
kT
=
PH
Pe



2πmkT
h2

3/2

e−I/kT(11.4.15)

which is called the Saha equation. Note that Pe /kT = Ne /V . If we plug in numbers for thesurface of the sun this givesPp. 10−4PH(11.4.16)

so less than one atom in ten thousand are ionized. Note though that this is much, much larger thanthe Boltzmann factor by itself!

11.5

Dilute Solutions

When studying mixtures, there’s a natural, ‘simplest possible’ limit that we can take, where one
substance dominates. In this case, molecules of the other substances are always isolated from eachother, as their immediate neighbors are molecules of the dominant substance. This is the limit ofdilute solutions, and the dominant substance is called the solvent, while the other components are
solutes.
In this case we can think of the solutes as almost like an ‘ideal gas’, since they are approximately
non-interacting. It’s the kind of problem we know how to solve!
So we need to compute the Gibbs free energy and chemical potentials. If the solvent is A, thento start with, with no solute, we have
G = NA µ0 (T, P )(11.5.1)

Now we imagine adding B molecules, and G changes by
dG = dU + P dV − T dS

(11.5.2)

The dU and P dV terms don’t depend on NA , rather they only depend on how B molecules interact
with their A-molecule neighbors. But T dS depends on NB due to the freedom we have in placing
the B molecules. The number of choices is proportional to NA , so we get one termdS ⊃ k log NA
92(11.5.3)

But we also need to account for the redundancy because B molecules are identical. The result is
G = NA µ0 (T, P ) + NB f (T, P ) − NB kT log NA + kT log NB !
= NA µ0 (T, P ) + NB f (T, P ) − NB kT log NA + NB kT (log NB − 1)

(11.5.4)

where f (T, P ) doesn’t depend on NA . It is what accounts for the interaction of B molecules with
the A molecules that surround them. This expression is valid as long as NB  NA .
The chemical potentials follow from


∂G
NB
µA =
= µ0 (T, P ) −kT(11.5.5)
NA∂NA T,P,NB
and

µB =

∂G
∂NB


= f (T, P ) + kT log
T,P,NA

NB
NA(11.5.6)

Adding solute reduces µA and increases µB . Also, these chemical potentials depend only on theintensive ration NB /NA .
Osmotic Pressure
If we have pure solvent on one side of a barrier permeable only by solvent, and solute-solvent solution
on the other side, then solvent will want to move across the barrier to further dilute the solution.
This is called osmosis.
To prevent osmosis, we could add additional pressure to the solution side. This is the osmoticpressure. It’s determined by equality of solvent chemical potential. So we need
µ0 −

NB
∂µ0
kT = µ0 + δP∂PNA

(11.5.7)

Note that
V
∂µ0
=
∂P
N

(11.5.8)

V
NB
=
kT
NA
NA

(11.5.9)

NB kT
V(11.5.10)

at ﬁxed T, N because G = N µ, and so
δP
or
δP =
is the osmotic pressure.
93

Boiling and Freezing Points
The boiling and freezing points of a solution are altered by the presence of the solute. Often we canassume the solute never evaporates, eg if we consider salt in water. Intuitively, the boiling point
may increase (and the freezing point decrease) because the eﬀective liquid surface area is decreased
due to the solute ‘taking up space’ near the surface and preventing liquid molecules from moving.
In the case of freezing, the solute takes up space for building solid bonds.
Our theorizing above provides a quantitative treatment of these eﬀects. For boiling we haveµ0 (T, P ) −NB
kT = µgas (T, P )NA(11.5.11)

and so the NB /NA fraction of solute makes the liquid chemical potential smaller, so that higher
temperatures are needed for boiling.
If we hold the pressure ﬁxed and vary the temperature from the pure boiling point T0 we ﬁndµ0 (T0 , P ) + (T − T0 )∂µgas∂µ0 NB−kT = µgas (T, P ) + (T − T0 )
∂T
∂T
NA

(11.5.12)

and we have that
∂µ
S
=−
∂T
N

(11.5.13)

so that−(T − T0 )Sliquid − Sgas
NA


=

NB
kT
NA

(11.5.14)

The diﬀerence in entropies between gas and liquid is L/T0 , where L is the latent heat of vaporization,so we see thatN kT02L(11.5.15)

P
NB
=1−
NA
P0

(11.5.16)

T − T0 =where we treat T ≈ T0 on the RHS.
A similar treatment of pressure shows thatfor the boiling transition – we need to be at lower pressure for boiling to occur.

11.6

Critical Points

Now let us further discuss the critical point. We will see that critical points are universal.
Note that both
dP
= 0,
dV

d2 P
=0
dV 2
94

(11.6.1)

at the critical point, which helps us to easily identify it (eg using the van der Waals equation). A
cute way to say this is to write the vdW equation using intensive variables as
P v 3 − (P b + kT )v 2 + av − ab = 0(11.6.2)

The critical point occurs when all three roots of the cubic coincide, so thatpc (v − vc )3 = 0(11.6.3)

and if we expand this out we can identify that
kT =

8a
a
, vc = 3b, Pc =
27b
27b2

(11.6.4)

Law of Corresponding States
We can eliminate a, b completely if we re-write the vdW equation in terms of Tc , vc , and Pc .
Furthermore, we can then write the equation in dimensionless variables
Tv
P
T¯ = , v̄ = , P¯ =
vc
Pc
Tc

(11.6.5)

so that the vdW equation becomes
P̄ =

8 T̄
3
− 2
3 v̄ − 1/3 v̄

(11.6.6)

This is often called the law of corresponding states. The idea being that all gases/liquids correspondwith each other once expressed in these variables. This extends the extreme equivalence amonggases implied by the ideal gas law to the vdW equation.
Furthermore, since Tc , vc , pc only depend on a, b, there must be a relation among them, it is that
pc vc
3
=
kT
8

(11.6.7)

Thus all gases described by the vdW equation must obey this relation! Of course general gases
depend on more than the two parameters a, b, since vdW required truncating the expansion in thedensity.
This universality has been conﬁrmed in experiments, which show that all gases, no matter theirmolecular makeup, behave very similarly in the vicinity of the critical point. Let’s explore that.
Approaching the Critical Point
It turns out that the most interesting question is to ask how various quantities such as v, T, P behave
as we approach the critical point. We’re now going to do a naive analysis that will turn out to be
wrong for interesting reasons.
95For T < Tc so that T¯ < 1, the vdW equation has two solutions to
P̄ =

8 T̄
3
− 2
3 v̄ − 1/3 v̄

(11.6.8)

corresponding to a vgas and vliquid . This means that we can writeT̄ =(3v̄l − 1)(3v̄g − 1)(v̄l + v̄g )8v̄g2 v̄l2

(11.6.9)

We can expand near the critical point by expanding in vg − vl , whcih givesT̄ ≈ 1 −1(v̄g − v̄l )2
16(11.6.10)orv̄g − v̄l ∝p

Tc − T(11.6.11)

so this tells us how the diﬀerence in molecular volumes varies near the critical temperature.
We can answer a similar question about pressure with no work, giving
P − Pc ∝ (v − vc )3(11.6.12)

since the ﬁrst and second derivatives of P wrt volume at Pc vanish.
As a third example, let’s consider the compressibility
κ= −

1 ∂v
v ∂P T(11.6.13)

We know that at Tc the derivative of pressure wrt volume vanishes, so we must have that∂v
= −a(T − Tc )∂P T,vc(11.6.14)

which we can invert to learn that
κ∝1
T − Tc(11.6.15)

Note that to discover these last two facts, we didn’t even need the vdW equation. Rather, we
assumed that the scaling behavior near the critical point was analytic, ie it has a nice Taylor series
expansion.
Actually, our results are quite wrong, even the ones that didn’t rely on vdW!
In fact, all gas-liquid transitions seem to universally behave as
vg − vl ∝ (Tc − T )0.32
p − pc ∝ (v − vc )4.8
1
κ ∝(T − Tc )1.2
96

(11.6.16)

which diﬀer signiﬁcantly from our predictions. In particular, these are non-analytic, ie they don’t
have a nice Taylor series expansion about the critical point.
We already saw a hint of why life’s more complicated at the critical point – there are large
ﬂuctuations, for example in the density. The fact that the compressibility
−

1 ∂v
1
∝
v ∂P T(T − Tc )γ(11.6.17)(where we though γ = 1, and in fact γ = 1.2) means that the gas/liquid is becoming arbitrarily
compressible at the critical point. But this means that there will be huge, local ﬂuctuations in thedensity. In fact one can show that locally
Δn2
∂hni
1 ∂v
= −kT
n
∂v P,T v ∂P N,T(11.6.18)

so that ﬂuctuations in density are in fact diverging. This also means that we can’t use an equationof state that only accounts for average pressure, volume, and density.
Understanding how to account for this is the subject of ‘Critical Phenomena’, with close links to
Quantum Field Theory, Conformal Field Theory, and the ‘Renormalization Group’. Perhaps they’ll
be treated by a future year-long version of this course. But for now we’ll move on to study a speciﬁcmodel that shows some interesting related features.

12Interactions

Let’s study a few cases where we can include the eﬀects of interactions.

12.1

Ising Model

The Ising Model is one of the most important non-trivial models in physics. It can be interpreted aseither a system of interacting spins or as a lattice gas.
As a Magnetic System
The kind of model that we’ve already studied has energy or Hamiltonian
EB = −BNX

si(12.1.1)

i=1

where si are the spins, an d we can only havesi = ±1This is our familiar paramagnet.
97(12.1.2)

The Ising model adds an additional complication, a coupling between nearest neighbor spins, sothat
E = −JXs i sj − B

N
X

si

(12.1.3)

i=1

hiji

where the notation hiji implies that we only sum over the nearest neighbors. How this actuallyworks depends on the dimension – one can study the Ising model in 1, 2, or more dimensions. We
can also consider diﬀerent lattice shapes. We often use q to denote the number of nearest neighbors.
If J > 0 then neighboring spins prefer to be aligned, and the model acts as a ferromagnet; whenJ < 0 the spins prefer to be anti-aligned and we have an anti-ferromagnet.
We can thus describe the Ising model via a partition functionXZ=
e−βE[si ](12.1.4)
si

The B and J > 0 will make the spins want to align, but temperature causes them to want toﬂuctuate. The natural observable to study is the magnetization
m=

1 ∂
log Z
N β ∂B(12.1.5)

As a Lattice Gas
With the same math we can say diﬀerent words and view the model as a lattice gas. Since theparticles have hard cores, only one can sit at each site, so the sites are either full, ni = 1, or empty,
ni = 0. Then we can further add a reward in energy for them to sit on neighboring sites, so thatXX
E = −4Jni nj − µni(12.1.6)ihijiwhere µ is the chemical potential, which determines overall particle number. This Hamiltonian isthe same as above with si = 2ni − 1.
Mean Field Theory
The Ising model can be solved in d = 1 and, when B = 0, in d = 2; the latter is either very
complicated or requires very fancy techniques. But we can try to say something about its behavior
by approximating the eﬀect of the interactions by an averaging procedure.
The idea is to write each spin assi = (si − m) + m(12.1.7)

where m is the average spin throughout the lattice. Then the neighboring interactions aresi sj = (si − m)(sj − m) + m(sj − m) + m(si − m) + m2
98

(12.1.8)

The approximation comes in when we assume that the ﬂuctuations of spins away from the average
are small. This means we treat(si − m)(sj − m) → 0(12.1.9)(si − m)2 = 0(12.1.10)

but note that this does’t mean thatas this latter statement is never true, since we will sum over si = ±1. The former statement ispossibly true because si 6= sj . In this approximation the energy simpliﬁes greatly toXXm(si + sj ) − more2 − Bsi
Emf t = −J
i

hiji

=

1
JN qm2 − (Jqm + B)
2

X

si

(12.1.11)i

where N q/2 is the number of nearest neighbor pairs. So the mean ﬁeld approximation has removed
the interactions. Instead we have
Bef f = B + Jqm(12.1.12)

so that the spins see an extra contribution to the magnetic ﬁeld set by the mean ﬁeld of theirneighbors.
Now we are back in paramagnet territory, and so we can work out that the partition function isZ = e−βJN qm22

2−β JN 2qm

= e(e−βBef f + eβBef f )N
2N coshN (βBef f )(12.1.13)

However, this is a function of m... yet m is the average spin, so this can’t be correct unless it predicts
the correct value of m. We resolve this issue by computing the magnetization using m and solving
for it, giving
m = tanh(βB + βJqm)(12.1.14)

We can understand the behavior by making plots of these functions.
Vanishing B
Perhaps the most interesting case is B = 0. The behavior depends on the value of βJq, as can benoted from the expansion of tanh.
If βJq < 1, then the only solution is m = 0. So for T > Jq, ie at high temperatures, the averagemagnetization vanishes. Thermal ﬂuctuations dominate and the spins do not align.
99However if T < Jq, ie at low temperatures, we have a solution m = 0 as well as magnetizedsolutions with m = ±m0 . The latter correspond to phases where the spins are aligning. It turns outthat m = 0 is unstable, in a way analogous to the unstable solutions of the van der Waals equation.
The critical temperature separating the phases is
kTc = Jq

(12.1.15)

and at this temperature, the magnetization abruptly turns oﬀ.
At ﬁnite B, the magnetization is never zero, but instead always aligns with B; it smoothly shuts
oﬀ as T → ∞, but is ﬁnite for any ﬁnite T . Though in fact at small T there are other solutions thatare unstable or meta-stable.
But wait... are these results even correct??? Mean Field Theory was an uncontrolled approximation. It turns out that MFT is qualitatively correct for d ≥ 2 (in terms of the phase structure),
but is totally wrong for an Ising chain, in d = 1 – in that case there is no phase transition at ﬁnite
temperature. It’s amusing to note that MFT is exact for d → ∞.
Free Energy
We can understand the phase structure of the Ising model by computing the free energy. In the
MFT approximation it is11N
F = − log Z = JN qm2 − log(2 cosh(βBef f ))β2
β(12.1.16)

Demanding that F be in a local minimum requires dF/dm = 0, which requires thatm = tanh(βBef f )(12.1.17)

To determine which solutions dominate, we just calculate F (m). For example when B = 0 and
T < Tc we have the possibilities m = 0, ±m0 , and we see that F (m = 0) = 0 while F (±m0 ) < 0, sothat the aligned phases dominate.
Usually we can only deﬁne F in equilibrium. But we can take a maverick approach and pretendthat F (T, B, m) actually makes sense away from equilibrium, and see what it says about various
phases with diﬀerent values of m. This is the Landau theory of phases. It is the beginning of a richsubject...

12.2

Interacting Gases

Maybe someday.

100