September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

PrefaceEver since I heard the word “entropy” for the ﬁrst time, I was
fascinated with its mysterious nature. I vividly recall my ﬁrst
encounter with entropy and with the Second Law of Thermodynamics. It was more than forty years ago. I remember the hall,the lecturer, even the place where I sat; in the ﬁrst row, facing
the podium where the lecturer stood.
The lecturer was explaining Carnot’s cycle, the efﬁciencyof heat engines, the various formulations of the Second Law
and ﬁnally introducing the intriguing and mysterious quantity, named Entropy. I was puzzled and bewildered. Until thatmoment, the lecturer had been discussing concepts that werefamiliar to us; heat, work, energy and temperature. Suddenly, acompletely new word, never heard before and carrying a completely new concept, was being introduced. I waited patiently to
ask something, though I was not sure what the question would
be. What is this thing called entropy and why does it always
increase? Is it something we can see, touch or feel with any of oursenses? Upon ﬁnishing her exposition, the lecturer interjected,“If you do not understand the Second Law, do not be discouraged. You are in good company. You will not be able to understand it at this stage, but you will understand it when you study
statistical thermodynamics next year.” With these concludingixSeptember 22, 2009

x

13:20

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁed

remarks, she had freed herself from any further explanation of
the Second Law. The atmosphere was charged with mystery.
I, as well as some of those present during the lecture were left
tongue-tied, our intense craving for understanding the Second
Law unsatisﬁed.
Years later, I realized that the lecturer was right in claimingthat statistical mechanics harbors the clues to the understanding of entropy, and that without statistical mechanics, there is
no way one can understand what lies beneath the concept ofentropy and the Second Law. However, at that time, we all suspected that the lecturer had chosen an elegant way of avoiding
any embarrassing questions she could not answer. We therefore
accepted her advice, albeit grudgingly.
That year, we were trained to calculate the entropy changes
in many processes, from ideal gas expansion, to mixing of gases,
to transfer of heat from a hot to a cold body, and many otherspontaneous processes. We honed our skills in calculations ofentropy changes, but we did not really capture the essence of themeaning of entropy. We did the calculations with professional
dexterity, pretending that entropy is just another technical quantity, but deep inside we felt that entropy was left ensconced ina thick air of mystery.
What is that thing called entropy? We knew it was deﬁned
in terms of heat transferred (reversibly) divided by the absolutetemperature, but it was neither heat nor temperature. Why is it
always increasing, what fuel does it use to propel itself upwards?
We were used to conservation laws, laws that are conceived as
more “natural.” Matter or energy cannot be produced out of
nothing but entropy seems to defy our common sense. How cana physical quantity inexorably keep “producing” more of itself
without any apparent feeding source?
I recall hearing in one of the lectures in physical chemistry, that the entropy of solvation of argon in water is large

September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexi

and negative.1 The reason given was that argon increases the
structure of water. Increase of structure was tantamount toincrease of order. Entropy was loosely associated with disorder.
Hence, that was supposed to explain the decrease of entropy.
In that class, our lecturer explained that entropy of a system
can decrease when that system is coupled with another system(like a thermostat) and that the law of ever-increasing entropy is
only valid in an isolated system — a system that does not interact with its surroundings. That fact only deepened the mystery.
Not only do we not know the source which supplies the fuel forthe ever-increasing entropy, but no source is permitted, in principle, no feeding mechanism and no provision for any suppliesof anything from the outside. Besides, how is it that “structure”
and “order” have crept into the discussion of entropy, a conceptthat was deﬁned in terms of heat and temperature?
A year later, we were taught statistical mechanics and alongside we learnt the relationship between entropy and the numberof states, the famous Boltzmann relationship which is carved on
Ludwig Boltzmann’s tombstone in Vienna.2 Boltzmann’s relationship provided an interpretation of entropy in terms of disorder; the ever-increasing entropy, being interpreted as nature’s
way of proceeding from order to disorder. But why should a system go from order to disorder? Order and disorder are intangible
concepts, whereas entropy was deﬁned in terms of heat and temperature. The mystery of the perpetual increment of disorder inthe system did not resolve the mystery of entropy.
I taught thermodynamics and statistical mechanics for many
years. During those years, I came to realize that the mystery associated with the Second Law can never be removed
within classical thermodynamics (better referred to as the1 This was another fascinating topic that was eventually chosen for my PhD thesis.
2 A picture is shown on the dedication page of this book.

September 22, 2009

xii

13:20

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁed

non-atomistic formulation of the Second Law; see Chapter 1).
On the other hand, looking at the Second Law from the molecular point of view, I realized that there was no mystery at all.
I believe that the turning point in my own understanding ofentropy, hence also in my ability to explain it to my students
came when I was writing an article on the entropy of mixing
and the entropy of assimilation. It was only then that I felt I
could penetrate the haze enveloping entropy and the Second
Law. It dawned on me (during writing that article) how two key
features of the atomic theory of matter were crucial in dispersing
the last remains of the clouds hovering above entropy; the large(unimaginably large) numbers and the indistinguishability of theparticles constituting matter.
Once the haze dissipated, everything became crystal clear.
Not only clear, but in fact obvious; entropy’s behavior which
was once quite difﬁcult to understand, was reduced to a simple
matter of common sense.
Moreover, I suddenly realized that one does not need toknow any statistical mechanics to understand the Second Law.
This might sound contradictory, having just claimed that statistical mechanics harbors the clues to understanding the Second
Law. What I discovered was that, all one needs is the atomistic formulation of entropy, and nothing more from statistical mechanics. This ﬁnding formed a compelling motivation forwriting this book which is addressed to anyone who has neverheard of statistical mechanics.
While writing this book, I asked myself several times atexactly what point in time I decided that this book was worth
writing. I think there were three such points.
First, was the recognition of the crucial and the indispensablefacts that matter is composed of a huge number of particles, andthat these particles are indistinguishable from each other. These
facts have been well-known and well-recognized for almost aSeptember 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexiii

century, but it seems to me that they were not well emphasizedby authors who wrote on the Second Law.
The second point was while I was reading the two books by
Brian Greene.3 In discussing the entropy and the Second Law,
Greene wrote4 :“Among the features of common experience that haveresisted complete explanation is one that taps into thedeepest unresolved mysteries in modern physics.”
I could not believe that Greene, who has explained so brilliantly and in simple words so many difﬁcult concepts in modern
physics, could write these words.
The third point has more to do with aesthetics than substance. After all, I have been teaching statistical thermodynamics and the Second Law for many years, and even using dice
games to illustrate what goes on in spontaneous processes. However, I always found the correspondence between the dice changing faces, and the particles rushing to occupy all the accessible
space in an expansion process, logically and perhaps aesthetically unsatisfactory. As you shall see in Chapter 7, I made thecorrespondence between dice and particles, and between theoutcomes of tossing dice and the locations of the particles. This
correspondence is correct. You can always name a particle ina right compartment as an R–particle and a particle in the left
compartment as an L–particle. However, it was only when I
was writing the article on the entropy of mixing and entropy ofassimilation, that I “discovered” a different process for which
this correspondence could be made more “natural” and moresatisfying. The process referred to is deassimilation. It is a spontaneous process where the change in entropy is due solely to
3 Greene, B. (1999, 2004).
4 Greene, B. (2004), p. 12.

September 22, 2009

13:20

xiv

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁed

the particles acquiring new identity. The correspondence wasnow between a die and a particle, and between the identity ofthe outcome of throwing a die, and the identity of the particle.
I found this correspondence more aesthetically gratifying, thus
making the correspondence between the dice-game and the realprocess of deassimilation a perfect one and worth publishing.
In this book, I have deliberately avoided a technical style ofwriting. Instead of teaching you what entropy is, how it changes,
and most importantly why it changes in one direction, I will
simply guide you so that you can “discover” the Second Law
and obtain the satisfaction of unveiling the mystery surrounding
entropy for yourself.
Most of the time, we shall be engaged in playing, or imagining playing, simple games with dice. Starting with one die, thentwo dice, then ten, a hundred or a thousand, you will be building
up your skills in analyzing what goes on. You will ﬁnd out whatis that thing that changes with time (or with the number of steps
in each game), and how and why it changes. By the time you get
to a large number of dice, you will be able to extrapolate withease whatever you have learned from a small number of dice, toa system of a huge number of dice.
After experiencing the workings of the Second Law in thedice world, and achieving full understanding of what goes on,
there is one last step that I shall help you with in Chapter 7.
There, we shall translate everything we have learned from thedice world into the real experimental world. Once you havegrasped the evolution of the dice games, you will be able tounderstand the Second Law of thermodynamics.
I have written this book having in mind a reader who knows
nothing of science and mathematics. The only prerequisite forreading this book is plain common sense, and a strong will toapply it.

September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexv

One caveat before you go on reading the book; “common
sense” does not mean easy or effortless reading!
There are two “skills” that you have to develop. The ﬁrst
is to train yourself to think in terms of big numbers, fantastically big numbers, inconceivably big numbers and beyond. I
will help you with that in Chapter 2. The second is a little more
subtle. You have to learn how to distinguish between a speciﬁc
event (or state or conﬁguration) and a dim event (or a state or
conﬁguration). Do not be intimidated by these technical sounding terms.5 You will have ample examples to familiarize yourself
with them. They are indispensable for understanding the Second
Law. If you have any doubts about your ability to understandthis book, I will suggest that you take a simple test.
Go directly to the end of Chapter 2 (Sections 2.7 and 2.8).
There, you shall ﬁnd two quizzes. They are speciﬁcally designed
to test your understanding of the concepts of “speciﬁc” and“dim.”
If you answer all the questions correctly, then I can assure
you that you will understand the entire book easily.
If you cannot answer the questions, or if you tried but got
wrong answers, do not be discouraged. Look at my answers tothese questions. If you feel comfortable with my answers eventhough you could not answer the questions yourself, I believe
you can read and understand the book, but you will need a little
more effort.
If you do not know the answers to the questions, and evenafter reading my answers, you feel lost, I still do not think that
understanding the book is beyond your capacity. I would suggestthat you read Chapter 2 carefully and train yourself in thinking
5 In statistical mechanics, these terms correspond to microstates and macrostates. In

most of the book, we shall be playing with dice; and dice are always macroscopic.
That is why I chose the terms “speciﬁc” and “dim” instead.

September 22, 2009

13:20

xvi

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁedprobabilistically. If you need more help, you are welcome towrite to me and I promise to do my best to help.
Again, do not feel intimidated by the word “probabilistically.” If you are not surprised that you did not win the one
million prize in the lottery, although you habitually buy tickets,
you have been thinking “probabilistically.” Let me tell you a little story to make you comfortable with this formidable soundingword.
My father used to buy one lottery ticket every weekend foralmost sixty years. He was sure that someone “up there” favored
him and would bestow upon him the grand prize. I repeatedly
tried to explain to him that his chances of winning the grandprize were very slim, in fact, less than one hundredth of one
percent. But all my attempts to explain to him his odds fell on
deaf ears. Sometimes he would get seven or eight matching numbers (out of ten; ten matches being the winning combination).
He would scornfully criticize me for not being able to see theclear and unequivocal “signs” he was receiving from Him. He
was sure he was on the right track to winning. From week toweek, his hopes would wax and wane according to the numberof matches he got, or better yet, according to the kind of signs hebelieved he was receiving from Him. Close to his demise, at theage of 96, he told me that he was very much disappointed andbitter as he felt betrayed and disfavored by the deity in whom
he had believed all his life. I was saddened to realize that he did
not, and perhaps could not, think probabilistically!
If you have never heard of the Second Law, or of entropy, you
can read the brief, non-mathematical description of various formulations and manifestations of the Second Law in Chapter 1.
In Chapter 2, I have presented some basic elements of probability and information theory that you might need in order toexpress your ﬁndings in probabilistic terms. You should realize that the fundamentals of both probability and information

September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexvii

theory are based on nothing more than sheer common sense.
You need not have any background in mathematics, physics
or chemistry. The only things you need to know are: how tocount (mathematics!), that matter is composed of atoms andmolecules (physics and chemistry!), and that atoms are indistinguishable, (this is advanced physics!). All these are explainedin non-mathematical terms in Chapter 2. From Chapters 3–5,
we shall be playing games with a varying number of dice. You
watch what goes on, and make your conclusions. We shall haveplenty of occasions to “experience” the Second Law with allof our ﬁve senses. This reﬂects in a miniscule way the immense
variety of manifestations of the Second Law in the real physicalworld. In Chapter 6, we shall summarize our ﬁndings. We shalldo that in terms that will be easy to translate into the language ofa real experiment. Chapter 7 is devoted to describing two simple
experiments involving increase in entropy; all you have to do is
to make the correspondence between the number of dice, andthe number of particles in a box, between different outcomesof tossing a die, and the different states of the particles. Onceyou have made this correspondence, you can easily implement
all that you have learned from the dice-game to understand theSecond Law in the real world.
By the time you ﬁnish reading Chapter 7, you will understandwhat entropy is and how and why it behaves in an apparently
capricious way. You will see that there is no mystery at all in its
behavior; it simply follows the rules of common sense.
By understanding the two speciﬁc processes discussed in
Chapter 7, you will clearly see how the Second Law works.
Of course, there are many more processes that are “driven”
by the Second Law. It is not always a simple, straightforward
matter to show how the Second Law works in these processes.
For this, you need to know some mathematics. There are many
more, very complex processes where we believe that the SecondSeptember 22, 2009

13:20

xviii

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁed

Law has its say, but there is, as yet, no mathematical proof of
how it does that. Biological processes are far too complicatedfor a systematic molecular analysis. Although I am well awarethat many authors do use the Second Law in conjunction withvarious aspects of life, I believe that at this stage, it is utterly
premature. I fully agree with Morowitz6 who wrote: “The useof thermodynamics in biology has a long history of confusion.”In the last chapter, I have added some personal reﬂections
and speculations. These are by no means universally accepted
views and you are welcome to criticize whatever I say there. My
email address is given below.
My overall objective in writing this book is to help you
answer two questions that are associated with the Second Law.
One is: What is entropy? The second is: Why does it change in
only one direction — in apparent deﬁance of the time-symmetryof other laws of physics?
The second question is the more important one. It is the heart
and core of the mystery associated with the Second Law. I hope
to convince you that:
1. The Second Law is basically a law of probability.
2. The laws of probability are basically the laws of common
sense.
3. It follows from (1) and (2) that the Second Law is basicallya law of common sense — nothing more.
I admit, of course, that statements (1) and (2) have beenstated many times by many authors. The ﬁrst is implied in
Boltzmann’s formulation of the Second Law. The second has
been expressed by Laplace, one of the founders of probability theory. Certainly, I cannot claim to be the ﬁrst to makethese statements. Perhaps I can claim that the relationship of
6 Morowitz (1992) page 69.

September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexix“basicality” is a transitive relationship, i.e., that statement (3)follows from (1) and (2), is original.
The ﬁrst question is about the meaning of entropy. For
almost a hundred years, scientists speculated on this question.
Entropy was interpreted as measuring disorder, mixed-upness,disorganization, chaos, uncertainty, ignorance, missing information and more. To the best of my knowledge, the debate isstill on going. Even in recent books, important scientists express
diametrically opposing views. In Chapter 8, I will spell out indetails my views on this question. Here I will brieﬂy commentthat entropy can be made identical, both formally and conceptually, with a speciﬁc measure of information. This is a far fromuniversally accepted view. The gist of the difﬁculty in accepting
this identity is that entropy is a physically measurable quantity
having units of energy divided by temperature and is therefore
an objective quantity. Information however, is viewed as a nebulous dimensionless quantity expressing some kind of human
attribute such as knowledge, ignorance or uncertainty, hence, ahighly subjective quantity.7
In spite of the apparent irreconcilability between an objective and a subjective entity, I claim that entropy is information.
Whether either one of these is objective or subjective is a question that encroaches on philosophy or metaphysics. My view isthat both are objective quantities. But if you think one is subjective, you will have to concede that the second must be subjective
too.
There is trade-off in order to achieve this identity. We needto redeﬁne temperature in units of energy. This will require thesacriﬁce of the Boltzmann constant, which should have been
expunged from the vocabulary of physics. It will bring a few
other beneﬁts to statistical mechanics. For the purpose of this
7 More on this aspect of entropy may be found in Ben–Naim (2007).

September 22, 2009

xx

13:20

SPI-B439 Entropy Demystiﬁed

fm

Entropy Demystiﬁed

book, absence of the Boltzmann constant will automaticallymake entropy dimensionless and identical with a measure information. This will, once and for all, “exorcise” the mystery outof entropy!
To the reader of this book, I dare to promise the following:
1. If you have ever learned about entropy and been mystiﬁedby it, I promise to unmystify you.
2. If you have never heard and never been mystiﬁed by entropy,
I promise you immunity from any future mystiﬁcation.
3. If you are somewhere in between the two, someone who has
heard, but never learned, about entropy, if you heard people
talking about the deep mystery surrounding entropy, then I
promise you that by reading this book, you should be puzzled
and mystiﬁed! Not by entropy, not by the Second Law, butby the whole ballyhoo about the “mystery” of entropy!
4. Finally, if you read this book carefully and diligently and do
the small assignments scattered throughout the book, youwill feel the joy of discovering and understanding somethingwhich has eluded understanding for many years. You should
also feel a deep sense of satisfaction in understanding “oneof the deepest, unsolved mysteries in modern physics.”8

Acknowledgements
I want to express my sincerest thanks and appreciation to
all those who were willing to read either parts, or the wholemanuscript and offered comments and criticism.
I would like ﬁrst of all to thank my friends and colleagues,
Azriel Levy and Andres Santos for their meticulous reading andchecking of the entire manuscript. They have saved me from
8 Greene, B. (2004).

September 22, 2009

13:20

SPI-B439 Entropy Demystiﬁed

fm

Prefacexxi

what could have been embarrassing errors I did not, or could notdetect. Thanks are also due to Shalom Baer, Jacob Bekenstein,
Art Henn, Jeffrey Gordon, Ken Harris, Marco Pretti, SamuelSattath and Nico van der Vegt, who read parts of the manuscriptand sent me valuable comments. Finally, I would like to express
my sincere thanks to my lifetime partner Ruby for her patiencein struggling with my messy handwriting and correcting andre-correcting the manuscript. Without her gracious help thismanuscript could not have seen the light of publication. The
book’s planning had a long period of incubation. The actual
writing of the book began in Burgos, Spain and it was completed in La Jolla, California, USA.
Arieh Ben-Naim
Department of Physical Chemistry
The Hebrew University
Jerusalem, Israel
Email: arieh@fh.huji.ac.il and ariehbennaim@yahoo.com

P.S. Just in case you wonder about the meaning of the little ﬁgures at the end ofeach chapter, let me tell you that since I undertook the responsibility of explaining
to you the Second Law, I decided to do a little espionage on your progress. I placedthese icons so that I can monitor your progress in grasping the Second Law. You arewelcome to compare your state of understanding with my assessment. If you do notagree, let me know and I will do my best to help.

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

1
Introduction, and a Short History of theSecond Law of Thermodynamics

In this chapter, I shall present some important milestones in
the history of the Second Law of Thermodynamics. I shall alsopresent a few formulations of the Second Law in a descriptive
manner. In doing so, I necessarily sacriﬁce precision. The important point here is not to teach you the Second Law, but to give
you a qualitative description of the types of phenomena which
led the scientists of the nineteenth century to formulate the Second Law.
There are many formulations of the Second Law of Thermodynamics. We shall group all these into two conceptuallydifferent classes: Non-Atomistic and Atomistic.

1.1. The Non-Atomistic Formulation of the Second Law1
Traditionally, the birth of the Second Law is associated with thename Sadi Carnot (1796–1832). Although Carnot himself did
1 By “non-atomistic” formulation, I mean the discussion of the Second Law without

any reference to the atomic constituency of matter. Sometimes, it is also said that
this formulation views matter as a continuum. The important point to stress here isthat these formulations use only macroscopically observable or measurable quantities
without any reference to the atomic constituency of matter. It does not imply that theformulation applies to non-atomistic or continuous matter. As we shall see later, werematter really non-atomistic or continuous, the Second Law would not have existed.
1

September 18, 2009

2

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

Level Two

Level OneT1 0˚CT1 0˚C

Insulator

Insulator

Reservoir

Insulator

Reservoir
T1 0˚CT2 100˚C

I
State 1IIState 2

Fig. (1.1)

IIIState 3

IV
State 4

State 1Heat engine.

not formulate the Second Law,2 his work laid the foundations
on which the Second Law was formulated a few years later byClausius and Kelvin.
Carnot was interested in heat engines, more speciﬁcally, inthe efﬁciency of heat engines. Let me describe the simplest ofsuch an engine (Fig. (1.1)). Suppose you have a vessel of volume
V containing any ﬂuid, a gas or a liquid. The upper part ofthe vessel is sealed by a movable piston. This system is referred
to as a heat engine. The vessel is initially in State 1, thermallyinsulated, and has a temperature T1 , say 0◦ C. In the ﬁrst step of
the operation of this engine (Step I), we place a weight on thepiston. The gas will be compressed somewhat. The new state isState 2. Next, we attach the vessel to a heat reservoir (Step II).
The heat reservoir is simply a very large body at a constanttemperature, say T2 = 100◦ C. When the vessel is attached to theheat reservoir, thermal energy will ﬂow from the heat reservoir
to the engine. For simplicity, we assume that the heat reservoir
is immense compared with the size of the system or the engine.
In Fig. (1.1), the heat reservoir is shown only at the bottom of
the engine. Actually it should surround the entire engine. This
2 This is the majority opinion. Some authors do refer to Carnot as the “inventor” or

the “discoverer” of the Second Law.

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

3

ensures that after equilibrium is reached, the system will havethe same temperature, T2 , as that of the reservoir, and though
the reservoir has “lost” some energy, its temperature will benearly unchanged. As the gas (or the liquid) in the engine heats
up, it expands, thereby pushing the movable piston upwards.
At this step, the engine did some useful work: lifting a weightplaced on the piston from level one to a higher level, two. The
new state is State 3. Up to this point, the engine has absorbed
some quantity of energy in the form of heat that was transferredfrom the reservoir to the gas, thereby enabling the engine to dosome work by lifting the weight (which in turn could rotate
the wheels of a train, or produce electricity, etc.). Removing
the weight, Step III, might cause a further expansion of the gas.
The ﬁnal state is State 4.
If we want to convert this device into an engine that repeatedly does useful work, like lifting weights (from level one tolevel two), we need to operate it in a complete cycle. To do this,we need to bring the system back to its initial state, i.e., cool
the engine to its initial temperature T1 . This can be achieved byattaching the vessel to a heat reservoir or to a thermostat, attemperature T1 = 0◦ C, Step IV (again, we assume that the heat
reservoir is much larger compared with our system such thatits temperature is nearly unaffected while it is attached to theengine). The engine will cool to its initial temperature T1 , andif we take away the weight, we shall return to the initial state
and the cycle can start again.
This is not the so-called Carnot cycle. Nevertheless, it has all
the elements of a heat engine, doing work by operating between
the two temperatures, T1 and T2 .
The net effect of the repeated cycles is that heat, or thermal energy, is pumped into the engine from a body at a hightemperature T2 = 100◦ C; work is done by lifting a weight
and another amount of thermal energy is pumped out from theengine into a body at lower temperature T1 = 0◦ C. The Carnot

September 18, 2009

4

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

cycle is different in some details. The most important differenceis that all the processes are done very gradually and very slowly.3
We shall not be interested in these details here.
Carnot was interested in the efﬁciency of such an engine
operating between two temperatures under some ideal conditions (e.g. mass-less piston, no friction, no heat loss, etc.).
At the time of the publication of Carnot’s work in 1824,4 it
was believed that heat is a kind of ﬂuid referred to as caloric.
Carnot was mainly interested in the limits on the efﬁciency ofheat engines. He found out that the limiting efﬁciency depends
only on the ratio of the temperatures between which the engine
operates, and not on the substance (i.e., which gas or liquid)that is used in the engine. Later, it was shown that the efﬁciency of Carnot’s idealized engine could not be surpassed byany other engine. This laid the cornerstone for the formulationof the Second Law and paved the way for the appearance of the
new term “entropy.”
It was William Thomson (1824–1907), later known as Lord
Kelvin, who ﬁrst formulated the Second Law of Thermodynamics. Basically, Kelvin’s formulation states that there could be noengine, which when operating in cycles, the sole effect of which
is pumping energy from one reservoir of heat and completely
converting it into work.
Although such an engine would not have contradicted theFirst Law of Thermodynamics (the law of conservation of thetotal energy), it did impose a limitation on the amount of workthat can be done by operating an engine between two heat reservoirs at different temperatures.
3 Technically, the processes are said to be carried out in a quasi-static manner. Sometimes, this is also referred to as a reversible process. The latter term is, however, also
used for another type of process where entropy does not change. Therefore, the term
quasi-static process is more appropriate and preferable.
4 “Reﬂections on the motive power of ﬁre and on machines ﬁtted to develop thispower,” by Sadi Carnot (1824).

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

5

In simple terms, recognizing that heat is a form of energy, theSecond Law of Thermodynamics is a statement that it is impossible to convert heat (thermal energy) completely into work
(though the other way is possible, i.e., work can be converted
completely into heat, for example, stirring of a ﬂuid by a magnetic stirrer, or mechanically turning a wheel in a ﬂuid). This
impossibility is sometimes stated as “a perpetual motion of thesecond kind is impossible.” If such a “perpetual motion” waspossible, one could use the huge reservoir of thermal energy ofthe oceans to propel a ship, leaving a tail of slightly cooler water
behind it. Unfortunately, this is impossible.
Another formulation of the Second Law of Thermodynamics was later given by Rudolf Clausius (1822–1888). Basically,
Clausius’ formulation is what every one of us has observed; heatalways ﬂows from a body at a high temperature (hence is cooled)to a body at a lower temperature (which is heated up). We neverobserve the reverse of this process occurring spontaneously.
Clausius’ formulation states that no process exists, such that its
net effect is only the transfer of heat from a cold to a hot body.
Of course we can achieve this direction of heat ﬂow by doing
work on the ﬂuid (which is how refrigeration is achieved). What
Clausius claimed was that the process of heat transferred from a
hot to a cold body when brought in contact, which we observeto occur spontaneously, can never be observed in the reverse
direction. This is shown schematically in Fig. (1.2), where twobodies initially isolated are brought into thermal contact.
While the two formulations of Kelvin and Clausius are different, they are in fact equivalent. This is not immediately apparent.

T2

100 C

T1

0C

T
Fig. (1.2)

50 C

September 18, 2009

6

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

Fig. (1.3)

Fig. (1.4)

However, a simple argument can be employed to prove theirequivalency, as any elementary textbook of thermodynamics
will show.
There are many other formulations or manifestations of theSecond Law of Thermodynamics. For instance, a gas in a conﬁned volume V, if allowed to expand by removing the partition,
will always proceed in one direction (Fig. (1.3)).5 The gas will
expand to ﬁll the entire new volume, say 2V. We never see aspontaneous reversal of this process, i.e., gas occupying volume
2V will never spontaneously converge to occupy a smaller volume, say V.
There are more processes which all of us are familiar with,which proceed in one way, never in the reverse direction, such as
the processes depicted in Figs. (1.2), (1.3), (1.4) and (1.5). Heatﬂows from a high to a low temperature; material ﬂows froma high to a low concentration; two gases mix spontaneously;
and a small amount of colored ink dropped into a glass ofwater will spontaneously mix with the liquid until the water
5 The Second Law may also be formulated in terms of the spontaneous expansion ofa gas. It can also be shown that this, as well as other formulations, is equivalent to
the Clausius and Kelvin formulations.

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

7

Fig. (1.5)

is homogeneously colored (Fig. (1.5)). We never see the reverseof these processes.
All these processes have one thing in common. They proceed
in one direction, never proceeding spontaneously in the reverse
direction. But it is far from clear that all these processes are
driven by a common law of nature. It was Clausius who saw thegeneral principle that is common in all these processes. Recall
that Clausius’ formulation of the Second Law is nothing but astatement of what everyone of us is familiar with. The greatnessof Clausius’ achievement was his outstanding prescience that
all of these spontaneous processes are governed by one law,and that there is one quantity that governs the direction of theunfolding of events, a quantity that always changes in one direction in a spontaneous process. This was likened to a one-way
arrow or a vector that is directed in one direction along the timeaxis. Clausius introduced the new term entropy. In choosing theword “entropy,” Clausius wrote:6“I prefer going to the ancient languages for the namesof important scientiﬁc quantities, so that they mean thesame thing in all living tongues. I propose, accordingly, tocall S the entropy of a body, after the Greek word ‘transformation.’ I have designedly coined the word entropy
to be similar to energy, for these two quantities are so
6 Quoted by Cooper (1968).

September 18, 2009

8

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

analogous in their physical signiﬁcance, that an analogyof denominations seems to me helpful.”
In the Merriam-Webster Collegiate Dictionary (2003),“entropy” is deﬁned as: “change, literary turn, a measure of
the unavailable energy in a closed thermodynamic system… ameasure of the system’s degree of order…”
As we shall be discussing in Chapter 8, the term entropy inthe sense that was meant by Clausius is an inadequate term.
However, at the time it was coined, the molecular meaning ofentropy was not known nor understood. In fact, as we shall see
later, “entropy” is not the “transformation” (nor the “change”
nor the “turn”). It is something else that transforms or changes
or evolves in time.
With the new concept of entropy one could proclaim the general overarching formulation of the Second Law. In any spontaneous process occurring in an isolated system, the entropy neverdecreases. This formulation, which is very general, embracing
many processes, sowed the seed of the mystery associated withthe concept of entropy, the mystery involving a quantity that
does not subscribe to a conservation law.
We are used to conservation laws in physics. This makes
sense:7 material is not created from nothing, energy is not given
to us free. We tend to conceive of a conservation law as “understandable” as something that “makes sense.” But how can aquantity increase indeﬁnitely and why? What fuels that unrelenting, ever-ascending climb? It is not surprising that the Second Law and entropy were shrouded in mystery. Indeed, within
the context of the macroscopic theory of matter, the Second
Law of Thermodynamics is unexplainable. It could have stayed
7 Here we use the term “makes sense” in the sense that it is a common experience andnot necessarily a consequence of logical reasoning.

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

9

a mystery forever had the atomic theory of matter not been discovered and gained the acceptance of the scientiﬁc community.
Thus, with the macroscopic formulation we reach a dead end in
our understanding of the Second Law of Thermodynamics.

1.2. The Atomistic Formulation of the Second Law
Before the development of the kinetic theory of heat (which
relied on the recognition of the atomistic theory of matter), thermodynamics was applied without any reference to the composition of matter — as if matter were a continuum. Within thisapproach there was no further interpretation of entropy. That
in itself is not unusual. Any law of physics reaches a dead end
when we have to accept it as it is, without any further understanding. Furthermore, the Second Law was formulated as an
absolute law — entropy always increases in a spontaneous process in an isolated system. This is not different from any otherlaw, e.g. Newton’s laws are always obeyed — no exceptions.8
A huge stride forward in our understanding of entropy andof the Second Law of Thermodynamics, was made possible
following Boltzmann’s statistical interpretation of entropy —the famous relationship between entropy and the total numberof microstates of a system characterized macroscopically by agiven energy, volume, and number of particles. Take a look at
the cover illustration or at the picture of Boltzmann’s statue.
Ludwig Boltzmann (1844–1906),9 along with Maxwell andmany others, developed what is now known as the kinetic theory of gases, or the kinetic theory of heat. This not only led
to the identiﬁcation of temperature, which we can feel with
8 “Always” in the realm of phenomena that were studied at that time, and which are

now referred to as classical mechanics.
9 For a fascinating story of Boltzmann’s biography, see Broda (1983), Lindley (2001),
and Cercignani (2003).

September 18, 2009

15:21

10

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

our sense of touch, with the motions of the particles constituting matter, but also to the interpretation of entropy interms of the number of states that that are accessible to thesystem.
The atomistic formulation of entropy was introduced by
Boltzmann in two stages. Boltzmann ﬁrst deﬁned a quantity hedenoted as H, and showed that as a result of molecular collisions
and a few other assumptions, this quantity always decreases
and reaches a minimum at equilibirium. Boltzmann called his
theorem “the minimum theorem”, which later became famousas Boltzmann’s H-theorem (published in 1872). Furthermore,
Boltzmann showed that a system of particles starting with any
distribution of molecular velocities will reach thermal equilibrium. At that point, H attains its minimum and the resulting
velocity distribution will necessarily be the so-called Maxwell
distribution of the velocities (see also Chapter 7).
At that time, the atomistic theory of matter had not yet been
established nor universally accepted. Although the idea of the“atom” was in the minds of scientists for over two thousand
years, there was no compelling evidence for its existence. Nevertheless, the kinetic theory of heat did explain the pressure andtemperature of the gas. But what about entropy, the quantitythat Clausius introduced without any reference to the molecular composition ofmatter?
Boltzmann noticed that his H-quantity behaved similarly toentropy. One needs only to redeﬁne entropy simply as the negative value of H, to get a quantity that always increases withtime, and that remains constant once the system reaches thermal equilibrium.
Boltzmann’s H-theorem drew criticisms not only frompeople like Ernst Mach (1838–1916) and Wilhelm Ostwald

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

11(1853–1932), who did not believe that atoms existed, but also
from his colleagues and close friends.10
The gist of the criticisms (known as the reversibility objectionor the reversibility paradox), is the seeming conﬂict between theso-called time-reversal11 or time symmetry of the Newtonian’s
equations of motion, and the time asymmetry of the behaviorof Boltzmann’s H-quantity. This conﬂict between the reversibility of the molecular motion, and the irreversibility of the Hquantity was a profound one, and could not be reconciled. How
can one derive a quantity that distinguishes between the pastand the future (i.e. always increasing with time), from equations of motions that are indifferent and do not care for the
past and future? Newton’s equations can be used to predict theevolution of the particles into the past as well as into the future.
Woven into the H-Theorem were arguments from both mechanics and probability, one is deterministic and time symmetric,
while the other is stochastic and time asymmetric. This conﬂict
seems to consist of a fatal ﬂaw in the Boltzmann H-theorem.
It was suspected that either something was wrong with the Htheorem, or perhaps even with the very assumption of the atomistic nature of matter. This was clearly a setback for Boltzmann’s
H-theorem and perhaps a (temporary) victory for the
non-atomists.
Boltzmann’s reaction to the reversibility objection was thatthe H-theorem holds most of the time, but in very rare cases,
10 For instance, Loschmidt wrote in 1876 that the Second Law cannot be a result of

purely mechanical principle.
11 It should be noted as Greene (2004) emphasized that “time-reversal symmetry” is
not about time itself being reversed or ”running” backwards. Instead, time reversal isconcerned with whether events that happen in time in one particular temporal order
can also happen in the reverse order. A more appropriate phrase might be “event
reversal or process reversal”.

September 18, 2009

15:21

12

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

it can go the other way, i.e. H might increase, or the entropy
might decrease with time.
This was untenable. The (non-atomistic) Second Law ofThermodynamics, like any other laws of physics, was conceived
and proclaimed as being absolute — no room for exceptions,
not even rare exceptions. No one had ever observed violation of
the Second Law. As there are no exceptions to Newton’s equations of motion,12 there should be no exceptions to the Second
Law, not even in rare cases. The Second Law must be absoluteand inviolable. At this stage, there were two seemingly different views of the Second Law. On the one hand, there was theclassical, non-atomistic and absolute law as formulated by Clausius and Kelvin encapsulated in the statement that entropy neverdecreases in an isolated system. On the other hand, there was theatomistic formulation of Boltzmann which claimed that entropy
increases “most of the time” but there are exceptions, albeit
very rare exceptions. Boltzmann proclaimed that entropy could
decrease — that it was not an impossibility, but only improbable.13 However, since all observations seem to support theabsolute nature of the Second Law, it looked as if Boltzmann
suffered a defeat, and along with that, the atomistic view ofmatter.
In spite of this criticism, Boltzmann did not back down. He
reformulated his views on entropy. Instead of the H-theorem
which had one leg in the ﬁeld of mechanics, and the otherin the realm of probability, Boltzmann anchored both legs
ﬁrmly on the grounds of probability. This was a radically
12 Within classical mechanics.
13 As we shall see in Chapters 7 and 8, the admitted non-absoluteness of the atom-ists’ formulation of the Second Law is, in fact, more absolute than the proclaimed
absoluteness of the non-atomists’ formulation. On this matter, Poincare commented:“…to see heat pass from a cold body to a warm one, it will not be necessary to havethe acute vision, the intelligence, and the dexterity of Maxwell’s demon; it will sufﬁce
to have a little patience” quoted by Leff and Rex (1990).

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

13

new and foreign way of reasoning in physics. Probability, atthat time, was not part of physics (it was not even a part ofmathematics). Boltzmann proclaimed that entropy, or rather
atomistic-entropy, is equal to the logarithm of the total number of arrangements of a system. In this bold new formulation,
there were no traces of the equations of motion of the particles. It looks as if it is an ad-hoc new deﬁnition of a quantity,
devoid of any physics at all, purely a matter of counting thenumber of possibilities, the number of states or the number of
conﬁgurations. This atomistic entropy had built-in provisions
for exceptions, allowing entropy to decrease, albeit with anextremely low probability. At that time, the exceptions allowed
by Boltzmann’s formulation seemed to weaken the validity of
his formulation compared with the absolute and inviolable nonatomist formulation of the Second Law. In Chapter 8, I shallreturn to this point arguing that, in fact, the built-in provision
for exceptions strengthens rather than weakens the atomistic
formulation.
There seemed to be a state of stagnation as a result of thetwo irreconcilable views of the Second Law. It was not until
the atomic theory of matter had gained full acceptance that the
Boltzmann formulation won the upper hand. Unfortunately, thiscame only after Boltzmann’s death in 1906.
A year earlier, a seminal theoretical paper published by Einstein on the Brownian motion provided the lead to the victoryof the atomistic view of matter. At ﬁrst sight, this theory seems
to have nothing to do with the Second Law.
Brownian motion was observed by the English botanist
Robert Brown (1773–1858). The phenomenon is very simple:
tiny particles, such as pollen particles, are observed to move atseemingly random fashion when suspended in water. It was initially believed that this incessant motion was due to some tinyliving organism, propelling themselves in the liquid. However,September 18, 2009

15:21

14

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

Brown and others showed later that the same phenomenon
occurs with inanimate, inorganic particles, sprinkled into aliquid.
Albert Einstein (1879–1955) was the ﬁrst to propose a theory
for this so-called Brownian motion.14 Einstein believed in theatomic composition of matter and was also a staunch supporterof Boltzmann.15 He maintained that if there are very large numbers of atoms or molecules jittering randomly in a liquid, there
must also be ﬂuctuations. When tiny particles are immersed in aliquid (tiny compared to macroscopic size, but still large enoughcompared to the molecular dimensions of the molecules comprising the liquid), they will be “bombarded” randomly by themolecules of the liquid. However, once in a while there will be
assymetries in this bombardment of the suspended particles, asa result of which the tiny particles will be moving one way or
the other in a zigzag manner.
In 1905 Einstein published as part of his doctoral dissertation, a theory of these random motions.16 Once his theory
was corroborated by experimentalists [notably by Jean Perrin(1870–1942)], the acceptance of the atomistic view became
inevitable. Classical thermodynamics, based on the continuous
nature of matter, does not have room for ﬂuctuations. Indeed,
ﬂuctuations in a macroscopic system are extremely small. That
is why we do not observe ﬂuctuation in a macroscopic piece ofmatter. But with the tiny Brownian particles, the ﬂuctuations
14 It is interesting to note that the founders of the kinetic theory of gases such as
Maxwell, Clausius and Boltzmann never published anything to explain the Brownian
motion.
15 It is interesting to note that Einstein, who lauded Boltzmann for his probabilistic view of entropy, could not accept the probabilistic interpretation of quantum
mechanics.
16 A well-narrated story of Einstein’ theory of Brownian motion may be found in John
Rigden (2005). A thorough and authoritative discussion of the theory of Brownian
motion, including a historical background, has been published by Robert Mazo
(2002).

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

15

are magniﬁed and rendered observable. With the acceptance of
the atomic composition of matter also came the acceptance ofBoltzmann’s expression for entropy. It should be noted that thisformulation of entropy stood fast and was not affected or modiﬁed by the two great revolutions that took place in physics earlyin the 20th century: quantum mechanics and relativity.17 The
door to understanding entropy was now wide open.
The association of entropy with the number of conﬁgurations and probabilities was now unassailable from the pointof view of the dynamics of the particles. Yet, it was not easily
understood and accepted, especially at the time when probability was still not part of physics.
Almost at the same time that Boltzmann published his viewson the Second Law, Willard Gibbs (1839–1903) developed thestatistical mechanical theory of matter based on a purely statistical or probabilistic approach. The overwhelming success of
Gibbs’ approach, though based on probabilistic postulates,18
has given us the assurance that a system of a very large numberof particles, though ultimately governed by the laws of motion,
will behave in a random and chaotic manner, and that the lawsof probability will prevail.
The mere relationship between entropy and the number ofstates in a system is not enough to explain the behavior of
17 Perhaps, it should be noted that within the recent theories of black holes, people

speak about the “generalized Second Law of Thermodynamics” [Bekenstein (1980)].
It seems to me that this generalization does not affect Boltzmann’s formula for theentropy.
18 Today, any book on physics, in particular, statistical mechanics, takes for granted
the atomic structure of matter. It is interesting to note in Fowler and Guggenheim’sbook on Statistical Thermodynamics (ﬁrst published in 1939, and reprinted in 1956),
one of the ﬁrst assumptions is: “Assumption 1: The atomistic constitution of matter.” They add the comment that “Today, this hardly ranks as an assumption but it
is relevant to start by recalling that it is made, since any reference to atomic constitutions is foreign to classical thermodynamics.” Today, no modern book on statisticalmechanics makes that assumption explicitly. It is a universally accepted fact.

September 18, 2009

15:21

16

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

entropy. One must supplement this relationship with three critically important facts and assumptions. First, that there is a huge
number of particles and an even “huger” number of microstates.
Second, that all these states are equally likely i.e. have equal
probability of occurrence, hence are equally likely to be visitedby the system. Third, and most importantly, that at equilibrium, the number of microstates that are consistent with (orbelonging to) the macrostate that we actually observe, is almost
equal to the total number of possible microstates. We shallcome back to these aspects of a physical system in Chapters 6
and 7.
With these further assumptions that would crystallize into
a ﬁrm theory of statistical thermodynamics, the atomistic formulation of entropy has gained a decisive victory. The nonatomistic formulation of the Second Law is still being taught
and applied successfully. There is nothing wrong with it except
for the fact that it does not, and in principle cannot reveal thesecrets ensconced in the concept of entropy.
Boltzmann’s heuristic relation between entropy and the logarithm of the total number of states19 did open the door to anunderstanding of the meaning of entropy. However, one needs
to take further steps to penetrate the haze and dispel the mysterysurrounding entropy.
There are several routes to achieve this end. I shall discuss thetwo main routes. One is based on the interpretation of entropy interms of the extent of disorder in a system;20 the second involves

19 For simplicity and concreteness, think of N particles distributed in M cells. A fulldescription of the state of the system is a detailed speciﬁcation of which particle is inwhich cell.
20 The association of entropy with disorder is probably due to Bridgman (1941;1953).
Guggenheim (1949) suggested the term “spread” to describe the spread over a large
number of possible quantum states. A thorough discussion of this aspect is given by
Denbigh and Denbigh (1985).

September 18, 2009

15:21

SPI-B439 Entropy Demystiﬁed

ch01

Introduction, and a Short History of the Second Law of Thermodynamics

17

the interpretation of entropy in terms of the missing informationon the system.21
The ﬁrst, the older and more popular route, has its origin in
Boltzmann’s own interpretation of entropy: a large number ofstates can be conceived of as having a large degree of disorder.
This has led to the common statement of the Second Law ofThermodynamics that “Nature’s way is to proceed from order
to disorder.”In my opinion, although the order-disorder interpretation ofentropy is intuitively clear in many examples, it is not always
valid. In a qualitative way, it can answer the question of what is
the thing that changes in some spontaneous processes, but notin all. However, it does not offer any answer to the question of
why entropy always increases.
The second route, though less popular among scientists is, inmy opinion, the superior one. First, because information is a better, quantitative and objectively deﬁned quantity, whereas orderand disorder are less well-deﬁned quantities. Second, information, or rather the missing information, can be used to answer
the questions of what is the thing that changes in any spontaneous process. Information is a familiar word; like energy,
force or work, it does not conjure up mystery. The measure of
information is deﬁned precisely within information theory. Thisquantity retains its basic meaning of information with which weare familiar in everyday usage. This is not the case when we use
the concept of “disorder” to describe what is the thing thatchanges. We shall further discuss this aspect in Chapters 7 and8. Information in itself does not provide an answer to the question of why entropy changes in this particular way. However,information unlike disorder, is deﬁned in terms of probabilities
21 Information theory was developed independently of thermodynamics by Claude

Shannon in 1948. It was later realized that Shannon’s informational measure is identical (up to a constant that determines the units) with Boltzmann’s entropy.

September 18, 2009

15:21

18

SPI-B439 Entropy Demystiﬁed

ch01

Entropy Demystiﬁed

and as we shall see, probabilities hold the clues to answering thequestion “why.”
For these reasons, we shall devote the next chapter to familiarizing ourselves with some basic notions of probability andinformation. We shall do that in a very qualitative manner sothat anyone with or without a scientiﬁc background can follow
the arguments. All you need is sheer common sense. Once you
acquire familiarity with these concepts, the mystery surrounding entropy and the Second Law will disappear, and you will
be able to answer both the questions: “What is the thing that ischanging?” and “Why is it changing in this particular manner?”September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

8
Reﬂections on the Status of theSecond Law of Thermodynamics
as a Law of Physics

If you have followed me so far and have reached this last chapter,
you must feel comfortable with the concept of entropy and with
the Second Law. If you throw a pair of (real) dice many times,
and ﬁnd that the sum = 7 appears on the average more thanany other sums, you should not be surprised. If you throw one
hundred simpliﬁed dice (with “0” and “1”), you should not be
puzzled to ﬁnd out that the sum of the outcomes will almost
always be about 50. If you throw a million simpliﬁed dice, you
should not be mystiﬁed to ﬁnd out that you will “never” getthe sum = 0 or the sum = 1,000,000. You know that bothof these results are possible outcomes, but they are so rare thatyou can play all your life and will not witness even once thatparticular result. You will not be mystiﬁed because you havethought about that and your common sense tells you that eventswith high probability will be observed more frequently, while
events with extremely low probability will “never” occur.
If you have never heard of the atomic constituency of matterand you watch a colored gas initially contained in one compartment of a vessel ﬂowing and ﬁlling up the two compartments ofthe vessel, as shown in Fig. (8.1a); or two compartments withtwo different gases, say yellow and blue, transformed into a
blend of homogenous green, as shown in Fig. (8.1b); or a hot
186

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

187

a

b

c

T2

100 C

T1

0C

T

50 C

Fig. (8.1)

body at a temperature say T2 = 100◦ C, when brought into contact with a cold body, say at T1 = 0◦ C, cooled to a temperaturesomewhere in between T1 and T2 , as shown in Fig. (8.1c), you
should be mystiﬁed. Why did the blue gas ﬂow from one chamber to ﬁll the other chamber? Why were the two colored gases
transformed into a single color? Why did the temperatures of
the two bodies change into a single temperature? What are thehidden forces that propelled all these phenomena, and always inthese directions and never in the opposite directions? Indeed, foras long as the atomic theory of matter was not discovered andaccepted,1 all of these phenomena were shrouded in mystery.
Mystery might not be the right word. Perhaps “puzzlement”
will describe the situation better. The only reason for you to be
puzzled is that you do not have any understanding of why these
phenomena happen in the particular direction. But that is thesame for any law of physics. Once you accept the law as a fact,
1 By “discovered and accepted,” I mean “not yet discovered and accepted.” If matter
did not consist of atoms and molecules, then there would have been no mysterynone of the phenomena would have occurred. The Second Law as formulated withinclassical thermodynamics would not have existed at all.

September 18, 2009

15:22

188

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

you will feel that it is natural, and that it makes sense.2 The sameis true for the Second Law; the fact that these processes are so
common in daily life, means that they are slowly and gradually
being perceived as “natural” and “make sense.”If, however, you know that a gas consists of some 1023 atoms
or molecules, jittering and colliding incessantly millions of times
a second, then you know that the laws of probability will prevail,and that there is no mystery. There is no mystery in all theseprocesses as much as there is no mystery in failing to win the“one million” prize in the last lottery.
I would like to believe that even if you encountered the words“entropy” and the “Second Law” for the ﬁrst time in this book,
you would be puzzled as to why the word “mystery” was associated with these terms at all. You will have no more reasonsto cringe upon hearing the word “entropy,” or to be puzzled
by that unseen “force” that pushes the gas from one side to theother. There is also no need for you to continue reading thisbook. My mission of explaining the “mysteries of the Second
Law” has ended on the last pages of Chapter 7, where you havereached a full understanding of the Second Law.
In this chapter, I take the liberty to express some personal
reﬂections on the Second Law. Some of my views are not necessarily universally agreed upon. Nevertheless, I have ventured
into expressing these views and taking the risk of eliciting thecriticism of scientists whose views might be different and perhaps more correct than mine.
In this chapter, I shall raise some questions and shall try
to answer them. I will begin with a relatively innocent question: “Why has the Second Law been shrouded in mystery forso long?” Is it because it contains a seed of conﬂict between
2 Here, “makes sense” is used in the sense of being a common and familiar experience,

not in the logical sense.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

189

the time-reversal symmetry of the equations of motion, and theobserved irreversibility of natural processes? Then I shall discussa few other questions, the answers to which are still controversial. Is entropy really a measure of “disorder,” and what doesorder or disorder of a system mean? How has “information”
invaded a “territory” that used to harbor only physically measurable entities? Is the Second Law intimately associated withthe arrow of time? What is the “status” of the Second Law ofThermodynamics vis a vis other laws of nature? Is it also possible that one day science will do away with the Second Lawof Thermodynamics as it will be deemed a redundancy, a relicof the pre-atomistic view of matter that does not further enrich
our knowledge of how nature works?

8.1. What is the Source of the Mystery?
In my opinion, there are several reasons which gave rise to themystery enveloping the Second Law. The ﬁrst, and perhaps thesimplest reason for the mystery is the very word “entropy.”
Everyone is familiar with concepts like force, work, energy andthe like. When you learn physics, you encounter the same words,
although sometimes they have quite different meanings than theones you are used to in everyday life. The amount of “work”
that I have expended in writing this book is not measured in thesame units of work (or energy) that are used in physics. Likewise, the “force” exerted on a politician to push for a speciﬁc
law or a bill is not the same as the force used in physics. Nevertheless, the precise concepts of work and force as deﬁned inphysics retain some of the qualitative ﬂavor of the meaning ofthese words as used in daily life. Therefore, it is not difﬁcult
to accommodate the new and more precise meaning conferredon familiar concepts such as force, energy or work. When you
encounter, for the ﬁrst time, a new word such as “entropy,”

September 18, 2009

15:22

190

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

it conjures up an air of mystery; it has a strange and uneasy
effect on you. If you are not a student of physics or chemistry,and by chance hear scientists talking about “entropy,” you willcertainly feel that this concept is beyond you and a fortiori so,when you hear the scientists themselves referring to “entropy”
as a mystery.
Leon Cooper (1968), right after quoting Clausius’ explanation of his reasons for the choice of the word “entropy,”
comments3“By doing this, rather than extracting a name from thebody of the current language (say: lost heat), he succeeded in coining a word that meant the same thing toeverybody: nothing.”
I generally agree with Cooper’s comment but I have tworeservations about it. First, the word “entropy” is unfortunately a misleading word. This is clearly different than meaning“nothing.” Open any dictionary and you will ﬁnd: “Entropy —Ancient Greek change, literary turn.” Clearly, the concept ofentropy is not “transformation,” nor “change,” nor “turn.” As
we have seen, entropy as deﬁned in either the non-atomistic orthe atomistic formulation of the Second Law is something thatchanges. But it is not the “transformation” that is transformed,
nor the “change” that is changing, and certainly not the “turn”
that is evolving.
My second reservation concerns the casual suggestion madeby Cooper that “lost heat” could have been more appropriate. Of course, “lost heat” is a more meaningful term than“entropy.” It is also in accordance with the universal meaning
3 See Chapter 1, page 7. We again quote from Clausius’ writing on the choice of theword “entropy.” Clausius says: “I propose, accordingly, to call S the entropy of abody after the Greek word “transformation.”September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

191

assigned to entropy as a “measure of the unavailable energy.4
I will revert to this meaning assigned to entropy in Section 8.3
below.
Besides the unfamiliarity with a new concept that creates an
air of mystery, there is a second reason for the mystery. Thevery fact that many authors writing on entropy say that entropy
is a mystery, makes entropy a mystery. This is true for writersof popular science as well as writers of serious textbooks onthermodynamics.
Take for example a very recent book, brilliantly written forthe layman by Brian Greene. He writes5 :“And among the features of common experience thathave resisted complete explanation is one that taps into
the deepest unresolved mysteries in modern physics,
the mystery that the great British physicist, Sir Arthur
Eddington called the arrow of time.”
On the next pages of the book, Greene explains the behavior of entropy using the pages of Tolstoy’s epic novel War andPeace. There are many more ways that the pages of the said
novel can fall out of order, but only one (or two) ways to put
them in order.
It seems to me that the above quoted sentence contributes toperpetuating the mystery that is no longer there. In a few moresentences, Greene could have easily explained “entropy,” as heexplained so many other concepts of modern physics. Yet to me,
it is odd that he writes: “…the deepest unresolved mysteries inmodern physics,” when I believe he should instead have written:“Today, the mystery associated with the Second Law no longer
exists.” There are many authors who wrote on the Second Law
4 Merriam Webster’s Collegiate Dictionary (2004).
5 Greene (2004).

September 18, 2009

15:22

192

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

with the intention of explaining it, but in fact ended up propagating the mystery.6
Here is a classical example. Atkins’ book on The Second
Law starts with the following words7 :“No other part of science has contributed as much to
the liberation of the human spirit as the Second Law ofThermodynamics. Yet, at the same time, few other partsof science are held to be recondite. Mention of the Second
Law raises visions of lumbering steam engines, intricate
mathematics, and inﬁnitely incomprehensible entropy.”
What should one make of these opening sentences? I definitely do not agree with all the three quoted sentences. The
ﬁrst sentence is ambiguous. I failed to understand what theSecond Law has got to do with “liberating the human spirit.”
However, my point here is not to argue with Atkins views on
the Second Law. I quote these opening sentences from Atkins’
book to demonstrate how each contributes to propagating themystery. The ﬁrst sentence elicits great expectations from theSecond Law and presumably encourages you to read the book.
However, these expectations are largely frustrated as you go on
reading the book. The next two sentences are explicitly discouraging — “an inﬁnitely incomprehensible entropy” does not whetyour appetite to even try to taste this dish. In many textbookson thermodynamics, the authors spend a lot of time discussing
different manifestations of the Second Law, but very little onwhat is common to all these manifestations. Instead of selecting
one or two simple examples of processes that are manifestationsof the Second Law, the authors present a very large number of
6 An exception is Gamov’s book One, Two, Three Inﬁnity that opens a section withthe title The Mysterious Entropy but ends it with: “and as you see, there is nothing
in it to frighten you.”
7 Atkins (1984).

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

193

examples, some of which are too complicated to comprehend.
Reading all these, you cannot see the forest for the trees.8
In Chapter 7, we have discussed two relatively simple examples that demonstrate the workings of the Second Law. In eachof these examples only one parameter changes. In the ﬁrst, thechange we observed was in the locational information, i.e., particles that are initially conﬁned to a smaller volume, disperse
and ﬁll a larger volume. In the second example, the identitiesof the particles were changed. In the experiment on heat transfer from a hot to a cold body, it is the distribution of velocities
that was changed. There are, of course, more complicated processes that involve changes in many parameters (or degrees of
freedom). Sometimes, it is difﬁcult to enumerate all of them. For
instance, the processes that occur following the splattering of an
egg involve changes of location, identities of molecules, distribution of velocities, orientations and internal rotations within themolecules. All of these complicate the description of the process,but the principle of the Second Law is the same. To understandthe principle, it is enough to focus on one simple process, andthe simpler, the better and the easier to understand.
Atkins’ book devotes a whole chapter to “see how the Second Law accounts for the emergence of the intricately ordered
forms characteristic of life.”9 In my opinion, this promise is notdelivered. I have read Atkins’ entire book, cover-to-cover, and I
failed to “see how the Second Law accounts for the emergenceof the intricately ordered forms characteristic of life.”
These kinds of promises contribute to the frustration of thereaders and discourage them from getting to grips with theSecond Law.
8 It is interesting to note that “entropy” and “the Second Law” feature in the titlesof scores of books (see some titles of books in the bibliography). To the best of myknowledge, no other single law of physics has enjoyed that treat.
9 Atkins (1984).

September 18, 2009

15:22

194

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

Life phenomena involve extremely complicated processes.
Everyone “knows,” scientists as well as non-scientists, that life
is a complex phenomenon, many aspects of which, involving themind and consciousness, are still not well understood. Therefore,
discussing life in a book which is supposed to explain the Second
Law leaves the reader with the impression that entropy, like life,
is hopelessly difﬁcult to understand and very mysterious.
It is true that many scientists believe that all aspects of life,
including consciousness, are ultimately under the control of thelaws of physics and chemistry, and that there is no such separate entity as the mind which does not succumb to the laws ofphysics. I personally believe that this is true. However, this contention is still far from being proven and understood. It might
be the case that some aspects of life will require extension of thepresently known laws of physics and chemistry, as was cogently
argued by Penrose.10 Therefore, in my opinion, it is premature
to discuss life as just another example, fascinating as it may be,within the context of explaining the Second Law.
There are more serious reasons for the mystery that has
befogged entropy. For over a century, the Second Law was formulated in thermodynamic terms and even after the moleculartheory of matter has been established, the Second Law is still
being taught in thermodynamics, employing macroscopic terms.
This approach inevitably leads down a blind alley. Indeed, as
my ﬁrst lecturer correctly proclaimed (see Preface), there is nohope of understanding the Second Law within thermodynamics.
To reach the light, you must go through the tunnels of statistical thermodynamics, i.e., the formulation of the Second Law interms of a huge number of indistinguishable particles. If you go
through the various different formulations of the Second Law
within classical thermodynamics, you can prove the equivalence
10 Penrose (1989, 1994).

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

195

of one formulation to some other formulations; you can showthat the entropy that drives one process, say the expansion ofa gas, is the same entropy that drives another process, say themixing of two different gases. It is somewhat more difﬁcult toshow that it is also the same entropy that drives a chemical reaction, or mixing of two liquids. It is impossible to prove that it is
the same entropy that causes the mess created by the splatteringof an egg (yet we do assume that it is the same entropy and that
one day, when the tools of statistical thermodynamics shall havebeen more powerful, we will be able to prove it). However, nomatter how many examples you work out and prove that they
are driven by the inexorably and the ever-increasing entropy,
you will reach a blind alley. You can never understand what
the underlying source of this one-way ascent of the entropy is.
Thermodynamics does not reveal to you the underlying molecular events.
Had the atomic theory of matter not been discovered andaccepted,11 we would have never been able to explain the Second
Law; it would have forever remained a mystery.
That was the situation at the end of the nineteenth century
and at the beginning of the twentieth century. Although the
kinetic theory of heat had succeeded in explaining the pressure,temperature, and eventually also the entropy in terms of the
motions of atoms and molecules, these theories were considered
to be hypotheses. Important and inﬂuential scientists such as
Ostwald and Mach thought that the concept of the atom, andthe theories based on its existence, should not be part of physics.
Indeed, they had a point. As long as no one had “seen” the atoms
directly or indirectly, their incorporation in any theory of matter
was considered speculative.

11 See footnote 1, page 187.

September 18, 2009

15:22

196

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

The situation changed dramatically at the beginning of thetwentieth century. It was Einstein who contributed decisively todefeating the aether, and also paved the way for the atomists’victory. The acceptance of Boltzmann’s molecular interpretationof entropy became inevitable (see Chapter 1).
But how come the mystery still did not vanish with theembracing of Boltzmann’s interpretation of entropy? True, thedoor was then widely open to a full understanding of the waysof entropy and yet the mystery persisted.
I am not sure I know the full answer to this question.
But I do know why, in my own experience, the mystery hasremained ﬂoating in the air for a long time. The reason, I believe,
involves the unsettled controversy which arose from the association of entropy with “disorder,” with “missing information”
and with the “arrow of time.” I shall discuss each of theseseparately.

8.2. The Association of Entropy with “Disorder”
The association of entropy with disorder is perhaps the oldestof the three, and has its roots in Boltzmann’s interpretation ofentropy. Order and disorder are vague and highly subjective
concepts, and although it is true that in many cases, increase inentropy can be correlated with increase in disorder, the statement that “nature’s way is to go from order to disorder” isthe same as saying that “nature’s way is to go from low to highentropy.” It does not explain why disorder increases in a spontaneous process. There is no law of nature that states that systems
tend to evolve from order to disorder.
In fact, it is not true that, in general, a system evolves fromorder to disorder. My objection to the association of entropy
with disorder is mainly that order and disorder are not welldeﬁned, and are very fuzzy concepts. They are very subjective,

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

197

sometimes ambiguous, and at times totally misleading. Consider
the following examples:
In Fig. (8.2) we have three systems. On the left hand side,
we have N atoms of gas in volume V. In the second, some of
the N atoms occupy a larger volume 2V. In the third, the Natoms are spread evenly in the entire volume 2V. Take a look.
Can you tell which of the three systems is the more orderedone? Well, one can argue that the system on the left, where
the N atoms are gathering in one half of the volume, is moreordered than the system on the right, where N atoms are spreadin the entire volume. That is plausible when we associate entropy
with missing information (see below), but regarding order, I
personally do not see either of the systems in the ﬁgures to be
more ordered, or more disordered, than the other.
Consider next the two systems depicted in Fig. (8.3):
In the left system, we have N blue particles in one boxof volume V and N red particles in another box of the same
volume V. In the right, we have all the atoms mixed up in thesame volume V. Now, which is more ordered? In my view, theleft side is more ordered — all the blues and all the reds are

Fig. (8.2)

Fig. (8.3)

September 18, 2009

15:22

198

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

separated in two different boxes. On the right-hand side, they
are mixed up in one box. “Mixed-up” is certainly a disorderedstate, colloquially speaking. In fact, even Gibbs himself used theword “mix-upness” to describe entropy. Yet, one can prove thatthe two systems mentioned above have equal entropy. The association of mixing with increase in disorder, and hence increasein entropy, is therefore only an illusion. The trouble with theconcept of order and disorder is that they are not well-deﬁned
quantities — “order” as much as “structure” and “beauty” arein the eyes of the beholder!
I am not aware of any precise deﬁnition of order and disorderthat can be used to validate the interpretation of entropy interms of the extent of disorder. There is one exception, however.
Callen (1985), in his book on thermodynamics, writes (p. 380):“In fact, the conceptual framework of “information theory” erected by Claude Shannon, in the late 1940s, provides a basis for interpretation of the entropy in terms of
Shannon’s measure of disorder.”And further, on the next page, Callen concludes:“For closed system the entropy corresponds to Shannon’s
quantitative measure of the maximum possible disorder in the distribution of the system over its permissible
microstates.”
I have taught thermodynamics for many years and used
Callen’s book as a textbook. It is an excellent textbook. However, with all due respect to Callen and to his book, I must saythat Callen misleads the reader with these statements. I havecarefully read Shannon’s article “The Mathematical Theoryof Communication,” word-for-word and cover-to-cover, andfound out that Shannon neither deﬁned nor referred to “disorder.” In my opinion, Callen is fudging with the deﬁnition ofdisorder in the quoted statement and in the rest of that chapter.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

199

What for? To “legitimize” the usage of disorder in interpreting entropy. That clearly is not in accord with Shannon’s writings. What Callen refers to as Shannon’s deﬁnition of disorder
is in fact Shannon’s deﬁnition of information. In my opinion,
Callen’s re-deﬁnition of information in terms of disorder does
not help to achieve the goal of explaining entropy. As we haveseen in Chapters 2 and 6, the concept of information originatedfrom a qualitative and highly subjective concept, has been transformed into a quantitative and objective measure in the hands of
Shannon. As we have also seen, the distilled concept of “information” also retains the meaning of information as we use itin everyday life. That is not so for disorder. Of course, one candeﬁne disorder as Callen has, precisely by using Shannon’s definition of information. Unfortunately, this deﬁnition of “disorder” does not have, in general, the meaning of disorder as we
use the word in our daily lives, and has been demonstrated in
the examples above.12
To conclude this section, I would say that increase in disorder (or any of the equivalent words) can sometimes, but notalways, be associated with increase in entropy. On the other
hand, “information” can always be associated with entropy,
and therefore it is superior to disorder.
8.3. The Association of Entropy with Missing Information
Ever since Shannon put forward his deﬁnition of the concept
of information, it has been found to be very useful in interpreting entropy.13 In my opinion, the concept of missing information has not only contributed to our understanding of what is
12 Furthermore, Shannon has built up the measure of information, or uncertainty, by

requiring that this measure fulﬁll a few conditions. These conditions are plausible for
information, but not for disorder. For further reading on this aspect of entropy see
Ben-Naim (2007).
13 See Tribus (1961) and Jaynes (1983) both dealing with the informational theoreticalinterpretation of entropy.

September 18, 2009

15:22

200

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

the thing that changes (which is called entropy), but it has alsobrought us closer to the last and ﬁnal step of understanding
entropy’s behavior as nothing but common sense. This view,
however, is not universal.
On this matter, Callen (1983, page 384) writes:“There is a school of thermodynamics who view thermodynamics as a subjective science of prediction.”
In a paragraph preceding the discussion of entropy as disorder, Callen writes:“The concept of probability has two distinct interpretations in common usage. ‘Objective probability’ refers to afrequency, or a fractional occurrence; the assertion that‘the probability of newborn infants being male is slightly
less than one half’ is a statement about census data. ‘Subjective probability’ is a measure of expectation based onless than optimum information. The (subjective) probability of a particular yet unborn child being male, asassessed by a physician, depends upon that physician’s
knowledge of the parents’ family histories, upon accumulating data on maternal hormone levels, upon the increasing clarity of ultrasound images, and ﬁnally upon an educated, but still subjective, guess.”
As I have explained in Chapter 2 (in the section on “Conditional probabilities and subjective probability”), my views differ
from Callen’s in a fundamental sense. Both examples given by
Callen could be subjective or objective depending on the given
condition or on the given relevant knowledge.
I have quoted Callen’s paragraph above to show that hisargument favoring “disorder” is essentially fallacious. I believe
Callen has misapplied probabilistic argument to deem information “subjective” and to advocate in favor of “disorder,” which
in his view is “objective.”September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

201

An extraterrestial visitor, who has no information on therecorded gender of newborn infants, would have no idea what
the probabilities for a male or female are, and his assignmentof probabilities would be totally subjective. On the other hand,
given the same information and the same knowledge, including the frequencies of boys and girls, the reliability of all thestatistical medical records, his assignment of probabilities will
inevitably be objective.
It is unfortunate and perhaps even ironic that Callen dismisses “information” as subjective, while at the same timeembracing Shannon’s deﬁnition of information, but renaming
it as disorder. By doing that, he actually replaces a well-deﬁned,
quantitative and objective quantity with a more subjective concept of disorder. Had Callen not used Shannon’s deﬁnition of
information, the concept of disorder would have remained an
undeﬁned, qualitative and highly subjective quantity.
In my view, it does not make any difference if you refer toinformation or to disorder, as subjective or objective. What matters is that order and disorder are not well-deﬁned, scientiﬁc concepts. On the other hand, information is a well-deﬁned scientiﬁcquantity, as much as a point or a line is scientiﬁc in geometry,or the mass or charge of a particle is scientiﬁc in physics.
Ilya Prigogine (1997) in his recent book End of Certainty
quotes Murray Gell-Mann (1994), saying:“Entropy and information are very closely related.
In fact, entropy can be regarded as a measure of ignorance. When it is known only that a system is in a given
macrostate, the entropy of the macrostate measures thedegree of ignorance the microstate is in by counting thenumber of bits of additional information needed to specify
it, with all the microstates treated as equally probable.”14
14 The microstates and macrostates referred to here are what we call speciﬁc anddim-conﬁgurations, or states, or events.

September 18, 2009

15:22

202

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

I fully agree with this quotation by Gell-Mann, yet Ilya
Prigogine, commenting on this very paragraph, writes:“We believe that these arguments are untenable. They
imply that it is our own ignorance, our coarse graining,that leads to the second law.”
Untenable? Why?
The reason for these two diametrically contradictory viewsby two great Nobel prize winners lies in the misunderstandingof the concept of information.
In my opinion, Gell-Mann is not only right in his statement,
but he is also careful to say “entropy can be regarded as a measure of ignorance… Entropy … measures the degree of ignorance.” He does not say “our own ignorance,” as misinterpretedby Prigogine.
Indeed, information, as we have seen in Chapter 2, is a measure that is there in the system (or in the game of Chapter 2).
Within “information theory,” “information” is not a subjective
quantity. Gell-Mann uses the term “ignorance” as a synonymof “lack of information.” As such, ignorance is also an objective quantity that belongs to the system and it is not the same as“our own ignorance,” which might or might not be an objective
quantity.
The misinterpretation of the informational-theoretical interpretation of entropy as a subjective concept is quite common.
I will quote one more paragraph from Atkins’ preface from thebook The Second Law.15“ I have deliberately omitted reference to the relation
between information theory and entropy. There is thedanger, it seems to me, of giving the impression thatentropy requires the existence of some cognizant entity
capable of possessing “in formation” or of being to some
15 Atkins (1984).

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

203

degree “ignorant.” It is then only a small step to the presumption that entropy is all in the mind, and hence is an
aspect of the observer.”Atkins’ rejection of the informational interpretation ofentropy on grounds that this “analogy” might lead to the “presumption that entropy is all in the mind,” is ironic. Instead, heuses the terms “disorder” and “disorganized,” etc., which in myview are concepts that are far more “in the mind.”
The fact is that there is not only an “analogy” between
entropy and information; the two concepts can also be madeidentical.
It should be stressed again that the interpretation of entropy
as a measure of information cannot be used to explain the Second Law of Thermodynamics. The statement that entropy is anever-increasing quantity in a spontaneous process (in an isolated
system) is not explained by saying that this is “nature’s way of
increasing disorder,” or “nature’s way of increasing ignorance.”
All these are possible descriptions of the thing that changes ina spontaneous process. As a description, “information” is evenmore appropriate than the term “entropy” itself in describing thething that changes.
Before ending this section on entropy and information, I
should mention a nagging problem that has hindered the acceptance of the interpretation of entropy as information. We recall
that entropy was deﬁned as a quantity of heat divided by temperature. As such, it has the units of energy divided by K (i.e.,
Joules over K or J/K, K being the units of the absolute temperature in Kelvin scale). These two are tangible, measurable andwell-deﬁned concepts. How is it that “information,” which is adimensionless quantity,16 a number that has nothing to do witheither energy or temperature, could be associated with entropy,
16 I used here “dimensionless” as unit-less or lack of units.

September 18, 2009

15:22

204

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

a quantity that has been deﬁned in terms of energy and temperature? I believe that this is a very valid point of concern which
deserves some further examination. In fact, even Shannon himself recognized that his measure of information becomes identical with entropy only when it is multiplied by a constant k(now known as the Boltzmann constant), which has the units ofenergy divided by temperature. This in itself does not help muchin proving that the two apparently very different concepts are
identical. I believe there is a deeper reason for the difﬁculty ofidentifying entropy with information. I will elaborate on thisissue on two levels.
First, note that in the process depicted in Fig. (8.1c), thechange in entropy does involve some quantity of heat transferred
as well as the temperature. But this is only one example of aspontaneous process. Consider the expansion of an ideal gas inFig. (8.1a) or the mixing of two ideal gases in Fig. (8.1b). In
both cases, the entropy increases. However, in both cases, there
is no change in energy, no heat transfer, and no involvementof temperature. If you carry out these two processes for ideal
gas in an isolated condition, then the entropy change will be
ﬁxed, independent of the temperature at which the process has
been carried out and obviously no heat transfer from one bodyto another is involved. These examples are only suggestive thatentropy change does not necessarily involve units of energy andtemperature.
The second point is perhaps on a deeper level. The units ofentropy (J/K) are not only unnecessary for entropy, but theyshould not be used to express entropy at all. The involvementof energy and temperature in the original deﬁnition of entropy
is a historical accident, a relic of the pre-atomistic era of thermodynamics.
Recall that temperature was deﬁned earlier than entropy andearlier than the kinetic theory of heat. Kelvin introduced theSeptember 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

205

absolute scale of temperature in 1854. Maxwell published his
paper on the molecular distribution of velocities in 1859. This
has led to the identiﬁcation of temperature with the mean kinetic
energy of atoms or molecules in the gas.17 Once the identiﬁcation of temperature as a measure of the average kinetic energy of
the atoms had been conﬁrmed and accepted, there was no reason
to keep the old units of K. One should redeﬁne a new absolutetemperature, denoting it tentatively as T, deﬁned by T = kT.
The new temperature T would have the units of energy and there
should be no need for the Boltzmann constant k.18 The equation
for the entropy would simply be S = ln W, and entropy would
be rendered dimensionless!19
Had the kinetic theory of gases preceded Carnot, Clausius
and Kelvin, the change in entropy would still have been deﬁned
as energy divided by temperature. But then this ratio would havebeen dimensionless. This will not only simplify Boltzmann’s formula for entropy, but will also facilitate the identiﬁcation of thethermodynamic entropy with Shannon’s information.
In (1930), G. N. Lewis wrote:“Gain in entropy always means loss of information andnothing more.”This is an almost prophetic statement made eighteen years
before information theory was born. Lewis’ statement left no2

17 This identity has the form (for atomic particles of mass m) 3kT = mν  where T is22
the absolute temperature and ν2 , the average of the squared velocity of the atoms,

and k, the same k appearing on Boltzmann’s tombstone.
18 In doing so, the relation 3kT/2 = mν2 /2 will become simpler 3T/2 = mν2 /2.

The gas constant R in the equation of state for ideal gases would be changed into
Avogadro number NAV = 6.022 × 1023 and the equation state of one mole of an
ideal gas will read: PV = NAV T, instead of PV = RT.
19 Boltzmann’s formula assumes that we know what conﬁgurations to count in W.
To the best of my knowledge, this equation is not challenged within non-relativistic
thermodynamics. In the case of Black-Hole entropy, it is not really known if thisrelation is valid. I owe this comment to Jacob Bekenstein.

September 18, 2009

15:22

206

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

doubt that he considered entropy as conceptually identical toinformation.
Shannon (1948) has shown that entropy is formally identical with information. There is a story20 that John von Neumannadvised Claude Shannon to use the term “entropy” when discussing information because:“No one knows what entropy really is, so in a debate
you will always have the advantage.”
Thus, without entering into the controversy about the question of the subjectivity or objectivity of information, whatever it
is, I believe that entropy can be made identical, both conceptually and formally, to information. The identiﬁcation of the twois rendered possible by redeﬁning temperature in terms of unitsof energy.21 This would automatically expunge the Boltzmann
constant (k) from the vocabulary of physics. It will simplify the
Boltzmann formula for entropy, and it will remove the stumbling block that has hindered the acceptance of entropy as information for over a hundred years. It is also time to change not
only the units of entropy to make it dimensionless,22 but theterm “entropy” altogether. Entropy, as it is now recognized,
does not mean “transformation,” or “change,” or “turn.” Itdoes mean information. Why not replace the term that means“nothing” as Cooper noted, and does not even convey the meaning it was meant to convey when selected by Clausius? Why
not replace it with a simple, familiar, meaningful, and precisely
20 Tribus, M. and McIrvine, E. C. (1971), Energy and Information, Scientiﬁc Amer-ican, 225, pp. 179–188.
21 As is effectively done in many ﬁelds of Physics.
22 Note that the entropy would still be an extensive quantity, i.e., it would be pro-portional to the size of the system.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

207

deﬁned term “information?” This will not only remove muchof the mystery associated with the unfamiliar word entropy, butwill also ease the acceptance of John Wheeler’s view to “regard
the physical world as made of information, with energy andmatter as incidentals.”23
Before concluding this section, I owe you an explanation of
my second reservation regarding Cooper’s comment cited on
page 190.
I agree that “lost heat” could be better than “entropy.” However, both the terms “lost heat,” and the more common term
“unavailable energy,” are applied to T  S (i.e., the product of
the temperature with the change in entropy), and not to thechange of entropy itself. The frequent association of entropy
with “lost heat” or “unavailable energy” is due to the fact that
it is the entropy that carries the energy units. However, if one
deﬁnes temperature in terms of units of energy, then entropy
becomes dimensionless. Therefore, when forming the productT  S, it is the temperature that carries the burden of the unitsof energy. This will facilitate the interpretation of T  S (notthe change in entropy) as either “lost heat” or “unavailable
energy.”I should also add one further comment on nomenclature.
Brillouin (1962) has suggested to refer to “information” as“neg-entropy.” This amounts to replacing a simple, familiar andinformative term with a vague and essentially misleading term.
Instead, I would suggest replacing entropy with either “neginformation,” “missing information,” or “uncertainty.”
Finally, it should be said that even when we identify entropy
with information, there is one very important difference between
the thermodynamic information (entropy) and Shannon’s information, which is used in communications or in any other branch
23 Quoted by Jacob Bekenstein (2003).

September 18, 2009

15:22

208

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁedof science. It is the huge difference in order of magnitudes
between the two.24
As we have seen, the association between entropy and probability not only removes the mystery, but also reduces the Second
Law to mere common sense. Perhaps it is ironic that the atomic
view of matter that has led to a full understanding of entropy
had initially created a new and apparently deeper mystery. This
brings us to the next question.

8.4. Is the Second Law Intimately Associated with theArrow of Time?
Every day, we see numerous processes apparently occurring in
one direction, from the mixing of two gases, to the decaying of adead plant or animal. We never observe the reverse of these phenomena. It is almost natural to feel that this direction of occurrence of the events is in the right direction, consistent with thedirection of time. Here is what Greene writes on this matter25 :“We take for granted that there is a direction in the way
things unfold in time. Eggs break, but do not unbreak;candles melt, but they don’t unmelt; memories are of thepast, never of the future; people age, they don’t unage.”
However, Greene adds: “The accepted laws of Physics
show no such asymmetry, each direction in time, forward
24 A binary question gives you one bit (binary-unit) of information. A typical book,contains about one million bits. All the printed material in the world is estimated tocontain about 1015 bits. In statistical mechanics, we deal with information on theorder of 1023 and more bits. One can deﬁne information in units of cents, or dollars,or euros. If it costs one cent to buy one bit of information, then it would cost one
million cents to buy the information contained in a typical book. The information
contained in one gram of water, all the money in the world, will not sufﬁce to buy!
25 Greene (2004) page 13.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

209

and backward, is treated by the laws without distinction,and that’s the origin of a huge puzzle.”
Indeed it is! For almost a century, physicists were puzzled
by the apparent conﬂict between the Second Law of Thermodynamics and the laws of dynamics.26 As Brian Greene puts it,“Not only do known laws (of physics) fail to tell us why we see
events unfold in only one order, they also tell us that, in theory,
events can fold in the reverse order. The crucial question is Why
don’t we ever see such things? No one has actually witnessed a
splattered egg un-splattering, and if those laws treat splattering
and un-splattering equally, why does one event happen while its
reverse never does?”Ever since Waddington associated the Second Law of Thermodynamics with the arrow of time, scientists have endeavored
to reconcile this apparent paradox. The equations of motion are
symmetrical with respect to going forward or backward in time.
Nothing in the equations of motion suggests the possibility ofa change in one direction and forbids a change in the opposite
direction. On the other hand, many processes we see every day
do proceed in one direction and are never observed to occur inthe opposite direction. But is the Second Law really associatedwith the arrow of time?
The classical answer given to this question is that if you areshown a movie played backwards, you will immediately recognize, even if not told, that the movie is going backwards. You
will recognize, for instance, that a splattered egg scattered on theﬂoor, suddenly and spontaneously collects itself into the piecesof the broken egg shell, the broken egg shell then becoming
26 Here, we refer to either the classical (Newtonian) or the quantum mechanical lawsof dynamics. These are time-symmetric. There are phenomena involving elementary
particles that are not time-reversible. However, no one believes that these are theroots of the second law. I owe this comment to Jacob Bekenstein.

September 18, 2009

15:22

210

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

whole again, and the egg ﬂying upward and landing intact on
the table. If you see that kind of movie, you will smile and invariably conclude that the movie is going backwards. Why? Because
you know that this kind of process cannot proceed in this direction in time.
But what if you actually sit in the kitchen one day, look at a
splattered egg scattered on the ﬂoor, and suddenly the egg getsback to its unbroken state, and then jumps back on top of thetable?
Fantastic as it might sound, your association of the processof the splattering of the egg with the arrow of time is so strongthat you will not believe what your eyes see, and you will probably look around to see if someone is playing a trick on you
by running the ﬁlm you are acting in backwards. Or, if you
understand the Second Law, you might tell yourself that you
are fortunate to observe a real process, in the correct directionof time, a process that is extremely rare but not impossible.
This is exactly the conclusion reached by the physicist inGeorge Gamov’s book Mr. Tompkin’s Adventure in Wonderland.27 When he saw his glass of whisky, suddenly and spontaneously, boiling in its upper part, with ice cubes forming on
the lower part, the professor knew that this process, thoughextremely rare, can actually occur. He might have been puzzled
to observe such a rare event, but he did not look for someone
playing backwards the “movie” he was acting in. Here is thatlovely paragraph from Gamov’s book:“ The liquid in the glass was covered with violently bursting bubbles, and a thin cloud of steam was rising slowly
toward the ceiling. It was particularly odd, however, thatthe drink was boiling only in a comparatively small area
27 Gamov (1940, 1999).

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

211

around the ice cube. The rest of the drink was still quite
cold.
‘Think of it!’ went on the professor in an awed, trembling
voice. ‘Here, I was telling you about statistical ﬂuctuations in the law of entropy when we actually see one! Bysome incredible chance, possibly for the ﬁrst time since
the earth began, the faster molecules have all grouped
themselves accidentally on one part of the surface of thewater and the water has begun to boil by itself.
In the billions of years to come, we will still, probably,
be the only people who ever had the chance to observe
this extraordinary phenomenon. He watched the drink,which was now slowly cooling down. ‘What a stroke ofluck!’ he breathed happily.”
Our association of the spontaneously occurring events withthe arrow of time is, however, a mere illusion. An illusion createdby the fact that in our lifetime we have never seen even one process that unfolds in the “opposite” direction. The association of
the spontaneous, natural occurrence of processes with the arrowof time is almost always valid – almost, but not absolutely always.
George Gamov, in his delightful book Mr Tompkins in Wonderland, attempted to explain the difﬁcult-to-accept results ofthe theories of relativity and quantum mechanics by narrating
the adventures of Mr. Tompkins in a world where one can actually see and experience the difﬁcult-to-accept results. He tried toimagine how the world would look if the speed of light was muchslower than 300,000,000 meters per second, or conversely, how
the world would appear to someone travelling at velocities nearto the speed of light. In this world, one could observe phenomena that are almost never experienced in the real world.

September 18, 2009

15:22

212

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

Similarly, one can imagine a world where Planck’s constant (h) is very large and experience all kinds of incredible phenomena such as, for example, your car effortlessly penetrating awall (tunneling effect), and similar phenomena which are neverexperienced in the real world where we live.
To borrow from Gamov’s imagination, we can imagine aworld where people will be living for a very long time, many30
times the age of the universe, say 1010 years.28
In such a world, when performing the experiment with gas
expansion, or with mixing of gases, we should see something
like what we have observed in the system of 10 dice. If we startwith all particles in one box, we shall ﬁrst observe expansion
and the particles will ﬁll the entire volume of the system. But“once in a while” we will also observe visits to the originalstate. How often? If we live for an extremely long time, say
301010 years, and the gas consists of some 1023 particles, thenwe should observe visits to the original state many times in our
lifetime. If you watch a ﬁlm of the expanding gas, running forward or backward, you will not be able to tell the difference.
You will have no sense of some phenomena being more “natural” than others, and there should not be a sense of the “arrowof time” associated with the increase (or occasionally decrease)of entropy. Thus, the fact that we do not observe the unsplattering of an egg or unmixing of two gases is not because there
is a conﬂict between the Second Law of Thermodynamics andthe equations of motion or the laws of dynamics. There is nosuch conﬂict. If we live “long enough” we shall be able observe
28 Perhaps, we should note here that as far as it is known, there is no law of nature that

limits the longevity of people or of any living system. There might be however, somefundamental symmetry laws that preclude that. But this could be true also for the
speed of light and Planck constant. If that is true, then none of Gamov’s imaginations
could be realized in any “world” where the speed of light or Planck’s constant would
have different values.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

213

all these reverse processes! The connection between the arrowof time and the Second Law is not absolute, only “temporary,”
for a mere few billion years.
It should be added that in the context of the association of
the Second Law with the arrow of time, some authors invoke ourhuman experience that distinguishes the past from the future. It
is true that we remember events from the past, never from thefuture. We also feel that we can affect or inﬂuence events in the
future, but never events in the past. I fully share these experiences. The only question I have is what have these experiences
to do with the Second Law or with any law of physics?
This brings me to the next question.

8.5. Is the Second Law of Thermodynamics a Law
of Physics?
Most textbooks on statistical mechanics emphasize that the Second Law is not absolute; there are exceptions. Though extremely
rare, entropy can go downwand “once in a while.”
Noting this aspect of the Second Law, Greene (2004) writes
that the Second Law “is not a law in the conventional sense.”
Like any law of nature, the Second Law was founded on experimental grounds. Its formulation in terms of the increasing
entropy encapsulates, in a very succinct way, the common feature of a huge number of observations. In its thermodynamic
formulation or, rather, in the non-atomistic formulation, theSecond Law does not allow exceptions. Like any other law ofphysics, it proclaims a law that is absolute, with no exceptions. However, once we have grasped the Second Law fromthe molecular point of view, we realize that there can be exceptions. Though rare, extremely rare, entropy can go the otherway. The Second Law is thus recognized as not absolute, hence
Greene’s comments that it is not a law in the “conventional

September 18, 2009

15:22

214

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

sense.” Greene’s statement leaves us with the impression that
the Second Law is somewhat “weaker” than the conventionallaws of physics. It seems to be “less absolute” than the otherlaws of physics.
But what is a law in the conventional sense? Is Newton’s law
of inertia absolute? Is the constancy of the speed of light absolute? Can we really claim that any law of physics is absolute? Weknow that these laws have been observed during a few thousand
years in which events have been recorded. We can extrapolate
to millions or billions of years by examining geological records
or radiations emitted from the time near the Big Bang, but wecannot claim that these laws have always been the same, or will
always be the same in the future, and that no exceptions will befound. All we can say is that within a few millions or billions ofyears, it is unlikely that we shall ﬁnd exceptions to these laws.
In fact, there is neither theoretical nor experimental reason tobelieve that any law of physics is absolute.
From this point of view, the second law is indeed “not a law
in the conventional sense,” not in a weaker sense, as alluded toby Greene, but in a stronger sense.
The fact that we admit the existence of exceptions to theSecond Law makes it “weaker” than other laws of physics only
when the other laws are proclaimed to be valid in an absolutesense. However, recognizing the extreme rarity of the exceptions
to the Second Law makes it not only stronger but the strongest
among all other laws of physics. For any law of physics, one canargue that no exceptions can be expected within at most some
1010 years. But exceptions to the Second Law can be expected
only once in 1010000000000 or more years.
Thus, the Second Law when formulated within classical(non-atomistic) thermodynamics is an absolute law of physics.
It allows no exceptions. When formulated in terms of molecular
events, violations are permitted. Though it sounds paradoxical,

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

215

the relative “weakness” of the atomistic formulation makes theSecond Law the strongest among other laws of physics, including the Second Law in its thermodynamic (non-atomist) formulation. Putting it differently, the admitted non-absolutenessof the atomistic-Second-Law is in fact more absolute than the
proclaimed absoluteness of the non-atomistic-Second-Law.29
In the context of modern cosmology, people speculate on thegloomy fate of the universe, which ultimately will reach a stateof thermal equilibrium or “thermal death.”
Perhaps not?!
On the other end of the time scale, it has been speculated thatsince entropy always increases, the universe must have started
in the “beginning” with a lowest value of the entropy.
Perhaps not?!
And besides, the last speculation is in direct “conﬂict” with
the Bible:“1. In the beginning God created the heaven and theearth.
2. And the earth was unformed, and void.” Genesis 1:1

The original Hebrew version includes the expression “Tohu
Vavohu,” instead of “unformed” and “void.” The traditional
interpretation of “Tohu Vavohu,” is total chaos, or total disorder, or if you prefer, highest entropy!
Having said these, I would venture a provocative view that
the Second Law of Thermodynamics is neither “weaker” nor
29 Although my knowledge of cosmology is minimal, I believe that what I have saidin this section is applicable also to the “generalized second law,” used in connection
with black hole entropy, see Bekenstein (1980).

September 18, 2009

15:22

216

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed“stronger” than the other laws of physics. It is simply not a lawof physics at all, but rather a statement of pure common sense.
This brings me to the last question.

8.6. Can We Do Away with the Second Law?
If the Second Law of Thermodynamics is nothing but a statement of common sense, do we have to list it and teach it as oneof the laws of Physics? Paraphrasing this question, suppose thatno one had ever formulated the Second Law of Thermodynamics? Could we, by purely logical induction and common sensederive the Second Law? My answer is probably yes, provided
we have also discovered the atomic nature of matter and theimmense number of indistinguishable particles that constituteeach piece of material. I believe that one can go from the bottom up and deduce the Second Law.30 We can certainly do sofor the simple example of expansion of gas or mixing two different gases (as we have done at the end of Chapter 7). If wedevelop highly sophisticated mathematics, we can also predictthe most probable fate of a falling egg.31 All of these predictions
would not rely, however, on the laws of physics but on the lawsof probability, i.e., on the laws of common sense.
You can rightly claim that I could make this “prediction”
because I have beneﬁted from the ﬁndings of Carnot, Clausius,
Kelvin, Boltzmann and others. So it is not a great feat to“predict” a result that you know in advance. This is probably true. So I will rephrase the question in a more intriguing
30 Here, I do not mean one can deduce the Second Law by solving the equationsof motion of particles, but from the statistical behavior of the system. The ﬁrst is
impractical for a system of 1023 particles.
31 Again, I do not mean to predict the behavior of the falling egg by solving theequations of motion of all the particles constituting the egg. However, knowing all
the possible degrees of freedom of all the molecules comprising an egg, we could, inprinciple, predict the most probable fate of a falling egg.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

217

form. Suppose that all these great scientists, who founded theSecond Law, never existed, or that they did exist but never formulated the Second Law. Would science arrive at the Second
Law purely through logical reasoning, presuming the currently
available knowledge of the atomic nature of matter and all therest of physics?
The answer to this question might be NO! Not because one
could not derive the Second Law from the bottom up even if
no top-down derivation has ever existed. It is because science
will ﬁnd it unnecessary to formulate a law of physics based on
purely logical deduction.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

219

References and Suggested Reading
Atkins, P.W. (1984), The Second Law, Scientiﬁc American Books, W. H. Freeman
and Co., New York.
D’ Agostini, G. (2003), Bayesian Reasoning in Data Analysis, A Critical Introduction.
World Scientiﬁc Publ., Singapore.
Barrow, J.D. and Webb, J.K. (2005), Inconstant Constants, Scientiﬁc American, Vol.
292, 32.
Bekenstein, J.D. (1980), Black-Hole Thermodynamics, Physics Today, January 1980
p. 24.
Bekenstein, J.D. (2003), Information in the Holographic Universe, Scientiﬁc American, Aug. 2003 p. 49.
Ben-Naim, A. (1987), Is Mixing a Thermodynamic Process? Am. J. of Phys. 55, 725.
Ben-Naim, A (2006), Molecular Theory of Liquids, Oxford University Press, Oxford.
Ben-Naim, A (2007), Statistical Thermodynamics Based on Information, World
Scientiﬁc, in press.
Bent, H.A. (1965), The Second Law, Oxford-Univ. Press, New York.
Bennett, D.J. (1998), Randomness, Harvard University Press, Cambridge.
Bridgman, P.W. (1941), The Nature of Thermodynamics, Harvard Univ. Press.
Brillouin, L. (1962), Science and Information Theory, Academic Press.
Broda, E. (1983), Ludwig Boltzmann. Man. Physicist. Philosopher. Ox Bow Press,
Woodbridge, Connecticut.
Brush, S.G. (1983), Statistical Physics and the Atomic Theory of Matter, from Boyle
and Newton to Landau and Onsager, Princeton Univ. Press, Princeton.
Callen, H.B. (1985), Thermodynamics and an Introduction to Thermostatics, 2ndedition, John Wiley and Sons, US and Canada.
Carnap, R. (1950), The Logical Foundations of Probability, The University of
Chicago Press, Chicago.
Carnap, R. (1953), What is Probability? Scientiﬁc American, Vol. 189, 128–138.
Cercignani, C. (2003), Ludwig Boltzmann. The Man Who Trusted Atoms, Oxford
Univ. Press, London.
Cooper, L.N. (1968), An Introduction to the Meaning and Structure of Physics,
Harper and Low, New York.
David, F.N. (1962), Games, Gods and Gambling, A History of Probability and Statistical Ideas, Dover Publ., New York.
Denbigh, K.G. and Denbigh, J.S. (1985), Entropy in Relation to Incomplete Knowledge, Cambridge Univ. Press, Cambridge.
Falk, R. (1979), Revision of Probabilities and the Time Axis, Proceedings of theThird International Conference for the Psychology of Mathematics Education,
Warwick, U.K. pp. 64–66.
Fast, J.D. (1962), Entropy, The signiﬁcance of the concept of entropy and its applications in science and technology, Philips Technical Library.
Feller, W. (1950), An Introduction to Probability Theory and its Application, John
Wiley and Sons, New York.
Feynman R. (1996), Feynmann Lectures, Addison Wesley, Reading.
Fowler, R. and Guggenheim, E.A. (1956), Statistical Thermodynamics, Cambridge
Univ. Press, Cambridge.

September 18, 2009

15:22

220

SPI-B439 Entropy Demystiﬁed

ch08

Entropy Demystiﬁed

Gamov, G. (1940), Mr. Tompkins in Wonderland, Cambridge University Press, Cambridge.
Gamov, G. (1947), One, Two, Three…Inﬁnity, Facts and Speculations of Science,
Dover Publ., New York.
Gamov, G. and Stannard, R. (1999), The New World of Mr. Tompkins, Cambridge
University Press, Cambridge
Gatlin, L.L. (1972), Information Theory and the Living System, Columbia University
Press, New York.
Gell-Mann, M. (1994), The Quark and the Jaguar, Little Brown, London.
Gnedenko, B.V. (1962), The Theory of Probability, Chelsea Publishing Co.,New York.
Greene, B. (1999), The Elegant Universe, Norton, New York.
Greene, B. (2004), The Fabric of the Cosmos, Space, Time, and the Texture of Reality,
Alfred A. Knopf.
Greven, A., Keller, G. and Warnecke, G. editors (2003), Entropy, Princeton Univ.
Press, Princeton.
Guggenheim, E.A. (1949), Research 2, 450.
Jaynes, E.T., (1957), Information Theory and Statistical Mechanics, Phys. Rev. 106,620.
Jaynes, E.T. (1965), Gibbs vs. Boltzmann Entropies, American J. of Physics, 33, 391.
Jaynes, E.T. (1983), Papers on Probability Statistics and Statistical Physics, Edited by
R.D. Rosenkrantz, D. Reidel Publishing Co., London.
Jaynes, E.T. (2003), Probability Theory With Applications in Science and Engineering, ed. G.L. Brethhorst, Cambridge Univ. Press, Cambridge.
Katz, A. (1967), Principles of Statistical Mechanics, The Information Theory
Approach, W. H. Freeman and Co., San Francisco.
Kerrich, J.E. (1964), An Experimental Introduction to the Theory of Probability,
Witwatersrand University Press, Johannesburg.
Lebowitz, J.L. (1993), Boltzmann’s Entropy and Time’s Arrow, Physics Today, Sept.
1993, p. 32.
Leff, H.S. and Rex, A.F. (eds) (1990), Maxwell’s Demon: Entropy, Information,
Computing, Adam Hilger, Bristol.
Lewis, G.N. (1930), The Symmetry of Time in Physics, Science, 71, 0569.
Lindley (2001), Boltzmann’s Atom, The Free Press, New York.
Morowitz, H.J. (1992), Beginnings of Cellular Life. Metabolism Recapitulates, Biogenesis, Yale University Press.
Mazo, R.M. (2002), Brownian Motion, Fluctuations, Dynamics and Applications,
Clarendon Press, Oxford.
Nickerson, R.S. (2004), Cognition and Chance, The Psychology of Probabilistic Reasoning, Lawrence Erlbaum Associates, Publishers, London.
Papoulis, A. (1965), Probability, Random Variables and Stochastic Processes,
McGraw Hill Book Comp. New York.
Penrose, R. (1989), The Emperor’s New Mind, Oxford Univ. Press, Oxford.
Penrose, R. (1994), Shadows of the Mind. A Search for the Missing Science of Consciousness. Oxford Univ. Press, Oxford.
Planck, M. (1945), Treatise on Thermodynamics, Dover, New York.

September 18, 2009

15:22

SPI-B439 Entropy Demystiﬁed

ch08

Reﬂections on the Status of the Second Law of Thermodynamics

221

Prigogine, I. (1997), The End of Certainty, Time, Chaos, and the New Laws of
Nature, The Free Press, New York.
Rigden, J.S. (2005), Einstein 1905. The Standard of Greatness, Harvard Univ. Press,
Cambridge.
Schrodinger, E. (1945), What is life? Cambridge, University Press, Cambridge.
Schrodinger, E. (1952), Statistical Thermodynamics, Cambridge U.P., Cambridge.
Shannon, C.E. (1948), The Mathematical Theory of Communication, Bell System
Tech Journal 27, 379, 623; Shannon, C.E. and Weaver, (1949) W. Univ. of Illinois Press, Urbana.
Tribus, M. and McIrvine, E.C. (1971), Entropy and Information, Scientiﬁc American,
225, pp. 179–188.