From: samodena@csemail.cropsci.ncsu.edu (S. A. Modena)
Newsgroups: bionet.info-theory
Subject: Re: Questions for a thermo person
Date: 10 Oct 92 02:20:19 GMT
Distribution: bionet
Organization: Crop Science Dept., NCSU, Raleigh, NC 27695-7620

In article <1992Oct9.161239.24505@hellgate.utah.edu> tolman%asylum.cs.utah.edu@cs.utah.edu (Kenneth Tolman) writes:

I might as well stick in my $0.02 since it won't muddle this thread
any more than it is.


>>>1) How do you choose parititions?
>>If your question contained a little more information it might map onto a
>>better answer than this.
>
>Alright, here is a little more entropy (information) :)  Please tell if
>the following has correctly interpreted anything correctly, it may not!
>
>Classically,  the entropy of a system is proportional to the log of the
>number of microscopic "complexions" of a given macroscopic state.

To Shannon the communications engineering theorist, the macrostate is
a electromagnetic signal traveling from the emiter to the receiver.

The microscopic complexions are the details of the *spectrum* of the
electromagnetic signal.  The spectrum *width* and the probability of
any particular frequency or combination of frequencies being occupied
at any random instant is a consequence of the encoding system for the
original information content and the randomness (or lack of it) of the
original information content.


>Therefore, we set up a coarse grained system characterized by paritions,
>and the partitions are chosen by a human attempting to understand what
>will happen.  However, if we choose a partition differently the macrostate
>is defined by a different class of microstates.  

Be careful here.  What you really have described with the word "partitions"
is the limit of resolution of your measurement system.  If the partitions
are "closer spaced," you are saying that your measurement resolution is
going up.  It makes no sense to define more partitions that you can
resolve FULLY by your measurement system.....i.e. detector or message
receiver/differentiator/guessor.

Don't be deceived by the single number Shannon Entropy: it is the
AVERAGE number of bits needs to resolve the states population the
state space.  But when the states are "encoded" by VARIABLE length
messages, your measurement device/detector/decoder/decision maker
needs to be able to resolve the number of bits distinguishing the
LONGEST message.............and often, this amounts to (as Shannon
said) being able to wait enough LENGTH OF TIME without blinking
(so to speak) and loosing track of the count.

>............................................Entropy is said to
>go up when we move into indistinguishable macrostates.... which apparently
>depends on our partition set up. 

Nope.  The entropy number reaches it's assymptotic maximum when the
microstates become equiprobable and the entropy number becomes infinity
when there are an infinity of distinguishable states and each have
equiprobable occurance.

On the other hand, there might be an infinity of microstates possible,
but their occurance is sparce, and it is thoroughly impossible to guess
WHICH will constitute that sparce population during the next 10
microstate transitions. Therefore I will be forced to use a description
system to label (or detect) the next ten microstates with a number
of bits that is the inverse of the least probable of any microstate 
you HOPE will appear.  


>................................My contention is that actual entropy goes
>up without any sort of human intervention or understanding, and that it
>should not rely on any partitions at all.... unless of course there is some
>a priori reason for establishing partitions in some particular way.
>

Well, of course, in a finite world, either you lack the inspiration
to build a infinite resolution detector or you lack the money to
build such a detector.  :^)

Steve
---
+------------------------------------------------------------------+
|     In person:  Steve Modena     AB4EL                           |
|     On phone:   (919) 515-5328                                   |
|     At e-mail:  nmodena@unity.ncsu.edu                           | 
|                 samodena@csemail.cropsci.ncsu.edu                |
|                 [ either email address is read each day ]        |
|     By snail:   Crop Sci Dept, Box 7620, NCSU, Raleigh, NC 27695 |
+------------------------------------------------------------------+
         Lighten UP!  It's just a computer doing that to you.
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO