**12. Knowledge, Information, and Entropy**

The book *John von Neumann and the Foundations of Quantum Physics*
contains a fascinating and informative article written by Eckehart
Kohler entitled "Why von Neumann Rejected Carnap's Dualism of
Information Concept." The topic is precisely the core issue before us:
How is knowledge connected to physics? Kohler illuminates von Neumann's
views on this subject by contrasting them to those of Carnap.

Rudolph Carnap was a distinguished philosopher, and member of the Vienna
Circle. He was in some sense a dualist. He had studied one of the
central problems of philosophy, namely the distinction between
*analytic* statements and *synthetic* statements. (The former are true
or false by virtue of a specified set of rules held in our minds,
whereas the latter are true or false by virtue their concordance with
physical or empirical facts.) His conclusions had led him to the idea
that there are two different domains of truth, one pertaining to logic
and mathematics and the other to physics and the natural sciences. This
led to the claim that there are "Two Concepts of Probability," one
logical the other physical. That conclusion was in line with the fact
that philosophers were then divided between two main schools as to
whether probability should be understood in terms of abstract
idealizations or physical sequences of outcomes of measurements.
Carnap's bifurcations implied a similar division between two different
concepts of information, and of entropy.

In 1952 Carnap was working at the Institute for Advanced Study in
Princeton and about to publish a work on his dualistic theory of
information, according to which epistemological concepts like
information should be treated separately from physics. Von Neumann, in
private discussion, raised objections, and Pauli later wrote a forceful
letter, asserting that "I am quite strongly opposed to the position you
take." Later he adds "I am indeed concerned that the confusion in the
area of the foundations of statistical mechanics not grow further (and I
fear very much that a publication of your work in its present form would
have this effect)."

Carnap's view was in line with the Cartesian separation between a domain
of real objective physical facts and a domain of ideas and concepts. But
von Neumann's view, and also Pauli's, linked the probability that
occurred in physics, in connection with entropy, to *knowledge*, in
direct opposition to Carnap's view that epistemology (considerations
pertaining to knowledge) should be separated from physics. The
opposition of von Neumann and Pauli significantly influenced the
publication of Carnap's book.

This issue of the relationship of knowledge to physics is the central
question before us, and is in fact the core problem of all philosophy
and science. In the earlier chapters I relied upon the basic insight of
the founders of quantum theory, and upon the character of quantum theory
as it is used in actual practice, to justify the key postulate that
Process I is associated with knowing, or feeling. But there is also an
entirely different line of justification of that connection developed in
von Neumann's book, *Mathematical Foundations of Quantum Mechanics*.
This consideration, which strongly influenced his thinking for the
remainder of his life, pertains to the second law of thermodynamics,
which is the assertion that entropy (disorder, defined in a precise way)
never decreases.

There are huge differences in the quantum and classical workings of the
second law. Von Neumann's book discusses in detail the quantum case, and
some of those differences. In one sense there is no nontrivial objective
second law in classical physics: a classical state is supposed to be
objectively well defined, and hence it always has probability one.
Consequently, the entropy is zero at the outset and remains so
forevermore. Normally, however, one adopts some rule of "coarse
graining" that destroys information and hence allows probabilities to be
different from unity, and then embarks upon an endeavor to deduce the
laws of thermodynamics from statistical considerations. Of course, it
can be objected that the subjective act of choosing some particular
coarse graining renders the treatment not completely objective, but that
limited subjective input seems insufficient to warrant the claim that
physical probability is closely tied to knowledge.

The question of the connection of entropy to the *knowledge and actions
of an intelligent being* was, however, raised in a more incisive form by
Maxwell, who imagined a tiny "demon" to be stationed at a small doorway
between two large rooms filled with gas. If this agent could distinguish
different species of gas molecules, or their energies and locations, and
slide a frictionless door open or closed according to which type of
molecule was about to pass, he could easily cause a decrease in entropy
that could be used to do work, and hence to power a perpetual motion
machine, in violation of the second law.

This paradox was examined Leo Szilard, who replaced Maxwell's
intelligent "demon" by a simple idealized (classical) physical mechanism
that consumed no energy beyond the apparent minimum needed to 'recognize
and responded differently to' a two-valued property of the gas molecule.
He found that this rudimentary process of merely 'coming to know and
respond to' the two-valued property transferred entropy from heat baths
to the gaseous system in just the amount needed to preserve the second
law. Evidently nature is arranged so that what we conceive to be the
purely intellectual process of coming to know something, and acting on
the basis of that knowledge, is closely linked to the probabilities that
enter into the constraints upon physical processes associated with
entropy.

Von Neumann describes a version of this idealized experiment. Suppose a
single molecule is contained in a volume V. Suppose an agent comes to
know whether the molecule lies to the left or to the right of the center
line. He is then in the state of being able to order the placement of a
partition/piston at that line and to switch a lever either to the right
or to the left, which restricts the direction in which the piston can
move. This causes the molecule to drive the piston slowly to the right
or to the left, and transfer some of its thermal energy to it. If the
system is in a heat bath then this process extracts from the heat bath
an amount 'log 2' of entropy (in natural units). Thus the *knowledge* of
which half of the volume the molecule was in is converted into a
decrement of "log 2' units of entropy. In von Neumann's words, "we have
exchanged our knowledge for the entropy decrease k log 2." (k is the
natural unit of entropy.)

What this means is this: When we conceive of an increase in the
"knowledge possessed by some agent" we must not imagine that this
knowledge exists in some ethereal kingdom, apart from its physical
representation in the body of the agent. Von Neumann's analysis shows
that the change in knowledge represented by Process I is quantitatively
tied to the probabilities associated with entropy.

Among the many things shown by von Neumann are these two:

(1) The entropy of a system is unaltered when the state of that system
    is evolving solely under the governance of Process II.

(2) The entropy of a system is never decreased by any Process I event.

The first result is analogous to the classical result that if an
objective "probability" were to be assigned to each if a countable set
of possible classical states, and the system were allowed to evolve in
accordance with the classical laws of motion then the entropy of that
system would remain fixed.

The second result is a nontrivial quantum second law of thermodynamics.
Instead of coarse graining one has Process I, which in the simple
'Yes-No' case converts the prior system into one where the question
associated with the projection operator P has a definite answer, but
only the *probability* associated with each possible answer is
specified, not an answer itself.

One sees, therefore, why von Neumann rejected Carnap's attempt to
divorce knowledge from physics: large tracts in his book were devoted to
establishing their marriage. That work demonstrates the quantitative
link between the increment of knowledge or information associated with a
Process I event and the probabilities connected to entropy. This focus
on Process I allowed him to formulate and prove a quantum version of the
second law. Process III (Collapse) gives no such result. In the quantum
universe the rate of increase of entropy would be determined not by some
imaginary and arbitrary coarse graining rule, but by the number and
nature of objectively real Process I events.

Kohler discusses another outstanding problem: the nature of mathematics.
At one time mathematics was imagined to be an abstract resident of some
immaterial Platonic realm, independent in principle from the brains and
activities of those who do it. But many mathematicians and philosophers
now believe that the process of doing mathematics rests in the end on
mathematical intuitions, which are essentially aesthetic evaluations.

Kohler argues that von Neumann held this view. But what is the origin or
source of such aesthetic judgments?

Roger Penrose based his theory of consciousness on the idea that
mathematical insight comes from a Platonic realm. But according to the
present account each such illumination, like any other experience, is
represented in the quantum description of nature as a picking out of an
organized state in which diverse brain processes act together in an
harmonious state of mutual support. A mathematical illumination is a
grasping of an *aesthetic* quality of order in the quantum state of the
agent's brain/body. And every experience of any kind is fundamentally
like this.

This notion that each Process I event is a felt grasping of a state in
which diverse sub-processes act in concert provides a foundation for
answering in a uniform way many outstanding philosophical and scientific
problems. For example, it provides a foundation for a solution to a
basic issue of neuroscience, the so-called "binding problem". It is
known that diverse features of a visual scene, such as color, location,
size, shape, etc. are processed by separate modules located in different
regions of the brain. A Process I event is the representation in the
brain of a grasping of these diverse elements as a stable unified whole.
To achieve maximal organizational impact this event should provide the
conditions for a rapid sequence of re-enactments of itself. Then this
conception of the operation of von Neumann's process I provides an
understanding of the capacity of an agent's thoughts to influence its
bodily behavior, and thence the development of its species in accordance
with the laws of nature. These issues all come down to the problem of
the connection of knowledge to physics, which von Neumann's treatment of
entropy ties to Process I.