HDTenEightyP/NetNews huggingface dataset:
From: toms@fcs260c2.ncifcrf.gov (Tom Schneider)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 28 Sep 92 22:13:26 GMT
Organization: Frederick Cancer Research and Development Center

In article <1992Sep24.164924.1939@hellgate.utah.edu>
tolman%mindblow.cs.utah.edu@cs.utah.edu (Kenneth Tolman) writes:

|1) Erasing a bit of information requires at least ktLog2 of heat dissipation

|1- (discussion) This result is "highly accepted" by many various authors,
|especially cited and discussed in context of Maxwell's Demon.  It can sort
|of trace its ancestry back to Szilard's work (which appears to be entirely
|unfounded on itself)  Personally, I think that statistical thermodynamics
|is based on a bunch of subjective assumptions, and it will only fully be
|true once it is encorporated into quantum mechanics.  There is lots of talk
|about "expanding the degrees of freedom"- what are the equations for this,
|and how can it occur without there being quantum interaction?

As we have discussed here before, this is plain and simply wrong.  Erasure
requires an energy input.  Consider the case of a coin.  It may store H or T.
The ONLY way to "erase" this is to PUT ENERGY INTO THE COIN.  The literature is
a mess as a result.  Acceptance level has nothing to do with whether they are
right.  The literature on this topic never defines information content.  I
suggest you think carefully about the coin.

Measurements of nucleic-acid binding sites are in no obvious way subjective, so
that is wrong.  The definition of information (Schneider.edmm) shows that
quantum mechanics is not necessary.  "expanding the degrees of freedom" does
not occur usually, as information processing works perfectly well in any
constant dimensional space.

|2) Universal computation can occur with zero loss of information (and
|   conversely, with zero gain of information)

This is a misstatement of the literature.

|2- (discussion) This work was motivated by the first result- if we must
|dissipate heat to erase information, then lets not erase any!  Thus, the
|computation saves its history as it proceeds, and then RUNS backwards
|to fully "take back" everything that happened- nothing erased. Neat idea!

That conclusion is wrong.  Feynman pointed out the correct solution - that heat
must be dissipated for each bit of output.  I add to this that you must
dissipate heat for each bit of input.  What happens in between doesn't matter.
I have two papers in the works on these topics, but if you feel you can defend
the (incorrect!! :-) literature here, I'd be happy to continue.

  Tom Schneider
  National Cancer Institute
  Laboratory of Mathematical Biology
  Frederick, Maryland  21702-1201
  toms@ncifcrf.gov
From: pgh@bnr.co.uk (Peter Hamer)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 29 Sep 92 09:37:39 GMT
Reply-To: pgh@bnr.co.uk (Peter Hamer)
Distribution: bionet
Organization: BNR Europe Limited, London Road, Harlow, Essex, GB

In the referenced article toms@fcs260c2.ncifcrf.gov (Tom Schneider) writes:
>In article <9209250126.AA04548@eureka.buc.edu.au> kjm@buc.edu.au (Kevin Moore) writes:
>
>| Entropy itself is anthropomorphic, in the sense that it depends on how much we
>| know about a system. (Jaynes, in "Gibbs vs. Boltzmann entropies" American
>| Journal of Physics 1965 demonstrates this rather well)
>
>This is wrong (or "not right" for those with weak hearts... :-) - the
>entropy change of a ribosome has nothing whatsoever to do with humans.
>Entropy changes are only anthropomorphic when you are discussing humans!!

Surely, in a fairly weak sense, entropy does have something to do with
the observer, and their beliefs?

* Entropy is defined as a function of probabilities.

* These probabilities are assigned with respect to some hypothesis.

* Hence entropy is not an absolute measure, but conditional on the
  hypothesis used in assigning probabilities to events.

* So the *measured* entropy change of a ribosome would differ between
  observers holding different hypotheses.

... of course, the hypothesis you use to assign probabilities may be
so widely accepted that this is a philosophical point rather than a
practical one.

Regards, Peter
From: pauld@cs.washington.edu (Paul Barton-Davis)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 30 Sep 92 19:04:02 GMT
Organization: Computer Science & Engineering, U. of Washington, Seattle

In article <4000@fcs280s.ncifcrf.gov> toms@fcs260c2.ncifcrf.gov (Tom Schneider) writes:
>If EcoRI finds G and A and A and T and T and C, then it does a Boolean
>operation.  "Computation" is merely large numbers of Boolean
>operations.  

Minor nitpick, Tom. Computation can be *performed* by carrying out large
numbers of boolean operations, but the two are not the same.

Various network-based computing systems can only be described as
performing boolean operations in a symbolic sense - there are no
gates, no boolean logic to their operation (or rather, there *need*
not be), at least while you remain at the level at which computation
is nominally occuring. Summation of inputs to cross a threshold is not
a boolean operation in the sense that conventional electronic circuits
perform them.

-- paul
-- 
There is hardly anything in the world that some man cannot make a
little worse and sell a little cheaper, and the people who consider
price only are this man's lawful prey.
						John Raskin
Newsgroups: bionet.info-theory
From: burchard@horizon.math.utah.edu (Paul Burchard)
Subject: Re: Mead and Conway (Section 9.11)
Organization: University of Minnesota
Distribution: bionet
Date: Tue, 6 Oct 1992 10:54:39 GMT

In article <1992Oct5.234928.10679@hellgate.utah.edu>  
tolman%asylum.cs.utah.edu@cs.utah.edu (Kenneth Tolman) writes:
> I will point out though that many information theory types appear to
> think that Kolmogrov's notion of entropy is a form of "absolute"
> entropy- better than even Shannon's.  Your above example is perhaps why.

Actually, it is not absolute, as it depends on a choice of programming language and a model of computation.  True, there is a certain *asymptotic* equivalence between the standard models of sequential computation.  But if we're only making asymptotic claims, we haven't really gained any moral ground over probability theory.

--
--------------------------------------------------------------------
Paul Burchard	<burchard@geom.umn.edu>
``I'm still learning how to count backwards from infinity...''
--------------------------------------------------------------------
strong compile-time type-checking is the mysterious, elusive nature of the errors which can occur with incorrectly typed data.  While this is true of languages like C++ w
> entropy- better than even Shannon's.  Your above example is perhaps why.

Actually, it is not absolute, as it depends on a choice of programming language and a model of computation.  True, there is a certain *asymptotic* equivalence between the standard models of sequential computation.  But if we're only making asymptotic claims, we haven't really gained any moral ground over probability theory.

--
--------------------------------------------------------------------
Paul Burchard	<burchard@geom.umn.edu>
``I'm still learning how to count backwards from infinity...''
--------------------------------------------------------------------
strong compile-time type-checking is the mysterious, elusive nature of the errors which can occur with incorrectly typed data.  While this is true of languages like C++ which merely provide a front for structure pointers whose type cannot be verified at run time, it is not true of OOP languages with run-time binding which by necessity maintain type info at run time (e.g. in Obj-C, the first instance var of every object is its class).  If I go wild and send messages to the wrong types of objects, I don't ge


t a memory violation or corrupted data, but rather the helpful message

	Button does not respond to selector -wowCheckThis:outMan:

Of course, this still aborts the program, but at least there's some belated type checking going on here.

--
--------------------------------------------------------------------
Paul Burchard	<burchard@geom.umn.edu>
``I'm still learning how to count backwards from infinity...''
--------------------------------------------------------------------
From: tolman%asylum.cs.utah.edu@cs.utah.edu (Kenneth Tolman)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 6 Oct 92 05:49:27 GMT
Distribution: bionet
Organization: University of Utah, CompSci Dept


>If I produce a diskett full of ones and zeros and ask somone to tell me the
>entropy of the information contained within.  I am reasonably sure that most
>well traineed statistitions would find no significant compressability in That
>information if it were generated by some variation of one of my pet random
>number generator functions.
>
>They are essentaly caotic and iterative and tuned to reflect a minimum of
>predictability.
>
>YET, there exists a small set of data (the rnd function with the seed) that
>encodes the whole quantity (some 1.6 M bits).
>
>I am not saying that it is imposible to find the correct function -- which

Actually, it may be impossible to find the entropy of this information.  If
you consider information from the Kolmogrov definition, then the amount
of information is equal to the minimal program to produce it. Thus
the amount of information is equal to that program you suggested.  Suppose
we wanted to FIND that program with no extra knowledge... then we would
have to list all program smaller than your list of numbers, and see if
any of them ever produced the sequence.   This task is impossible for
two reasons:
1)  there are too many, tooo many for even all the atoms in the universe
2)  one would have to solve the halting problem to be sure of the minimal
entropy, because if any shorter sequence was still running, you could not
be sure it might produce your sequence and then halt.  This was first
discussed by Chaitin, and is quite clever.

This is not to say that it IS impossible to find it, it just may be....
how to actually find it is left as an exercise to the reader.

I will point out though that many information theory types appear to
think that Kolmogrov's notion of entropy is a form of "absolute"
entropy- better than even Shannon's.  Your above example is perhaps why.
From: toms@fcs260c2.ncifcrf.gov (Tom Schneider)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 6 Oct 92 17:26:58 GMT
Distribution: bionet
Organization: Frederick Cancer Research and Development Center

In article <9209300553.AA24276@eureka.buc.edu.au> kjm@buc.edu.au (Kevin Moore) writes:

>one observer knows the temperature, and the second knows the temperature
>and magnetisation. The ensemble of the first observer contains the ensemble
>of the second. Both observers see the second law hold, but both calculate
>DIFFERENT entropies for the system. No hypotheses are involved here, merely
>the fact that different information about the system is held by the two
>observers.

Not quite.  In your example, the person who knew less about the system will
eventually figure that they were missing something.  Get the observer out of
your calculations, it messes things up.

  Tom Schneider
  National Cancer Institute
  Laboratory of Mathematical Biology
  Frederick, Maryland  21702-1201
  toms@ncifcrf.gov
From: toms@fcs260c2.ncifcrf.gov (Tom Schneider)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 6 Oct 92 17:15:46 GMT
Distribution: bionet
Organization: Frederick Cancer Research and Development Center

In article <6260@m1.cs.man.ac.uk> mshute@cs.man.ac.uk (Malcolm Shute) writes:

>Tom, I'm disappointed!
>What you said is correct *because* you made an important change in your wording...
>but omitted to remind us what and how important this distinction is.

>A more meaningful question is... in Process-A, by how much does its
>potential energy *change*.

Yes, that is an important point, that we should usually be discussing changes
of state.  However, I'm not sure I agree with your idea that entropy itself is
dependent on people, since I can look up the entropy of copper in CRC - I did
that for the calculation given in Schneider.edmm!  So there is a way to get the
entropy of copper.  I think this is relative to absolute zero though.

  Tom Schneider
  National Cancer Institute
  Laboratory of Mathematical Biology
  Frederick, Maryland  21702-1201
  toms@ncifcrf.gov
Newsgroups: bionet.info-theory
From: tolman%asylum.cs.utah.edu@cs.utah.edu (Kenneth Tolman)
Subject: Re: Mead and Conway (Section 9.11)
Date: 28 Sep 92 18:44:15 MDT
Organization: University of Utah, CompSci Dept

>|1) Erasing a bit of information requires at least ktLog2 of heat dissipation
>As we have discussed here before, this is plain and simply wrong.  Erasure
>requires an energy input.  Consider the case of a coin.  It may store H or T.

Yes, I agree.  The literature is largely wrong.  

>|2) Universal computation can occur with zero loss of information (and
>|   conversely, with zero gain of information)

>This is a misstatement of the literature.

Hmmm... I don't agree with you here.  It is certainly correct that every
bit of the input and of the output must be erased AFTER THE COMPUTATION.
The actual process of computation does not require this.  Fredkin points
out that erasing the input requires mKt (where m is the input length)
Universal computation can occur with zero loss of information.. what happens
after the computation realisitically will require some heat dissipation.
It is conceivable though to run a computation and not erase any of its
results, hence the above statement is correct.  Specifically, it used to
be believed that computation required dissipation at every gate, this is
no longer believed.  

    I would like to look at your Feynmann reference.
He does state in this reference I have here " If anything, <the energy
cost of computation> is kT per bit of the output answer.  But even this,
which might be considered the free energy needed to clear the computation
for further use..."  In this reference he has ommitted the cost of erasing
the input information- which is another incurred cost.  

  It may be actually possible to compress the cost of erasure for both
the input and the output- if these are algorithmically compressible
themselves.  I believe Zurek has talked about this in another context...

  However, I will still maintain that computation can occur without loss
of information... reading the data, changing it, erasing it, erasing the
input may all cost energy, but the computation itself does not.   No
information is created NOR destroyed necessarily- hence the evolution of
organisms could NOT be a purely computational process, if they are "gaining
information".
From: kjm@buc.edu.au (Kevin Moore)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 30 Sep 92 20:53:03 GMT
Distribution: bionet


In article <9209291311.AA19563@genbank.bio.net>, mroussel@alchemy.chem.utoronto.ca (Marc Roussel) writes

> In article <1992Sep29.093739.12409@bnr.co.uk> pgh@bnr.co.uk (Peter Hamer)
writes:
> >Surely, in a fairly weak sense, entropy does have something to do with
> >the observer, and their beliefs?

Not so much their beliefs, as their information about a system. (see
below)

> >
> >* Entropy is defined as a function of probabilities.
> >
> >* These probabilities are assigned with respect to some hypothesis.
> >
> >* Hence entropy is not an absolute measure, but conditional on the
> >  hypothesis used in assigning probabilities to events.

AND on the thermodynamic information available about the system. (see
comments below)

> >
> >* So the *measured* entropy change of a ribosome would differ between
> >  observers holding different hypotheses.
> >
> >... of course, the hypothesis you use to assign probabilities may be
> >so widely accepted that this is a philosophical point rather than a
> >practical one.
> 
>      I hope I don't offend Peter, but this sounds like mumbo-jumbo to
> me.  What hypothesis are you talking about?  When I write down a
> partition function, I make no hypotheses at all.

You do you know. The set of states available to the system is denumerated
using some form of model. (Trivially, e.g. whether you use an Einstein or Debye
model for a crystal, or one which fully describes the density of states
from crystallographic data) The statistical problem remains the same,
however. Two observers with different information about the same system,
using the same physical model will calculate different entropies.
  
						  I just use the most
> basic definition of probability and count up the number of states
> accessible to a system.  

And the set of states which we deem accessible depends on what information
we have about the system. For instance it is entirely concievable that two
observers can have different information about the same system, e.g.
one observer knows the temperature, and the second knows the temperature
and magnetisation. The ensemble of the first observer contains the ensemble
of the second. Both observers see the second law hold, but both calculate
DIFFERENT entropies for the system. No hypotheses are involved here, merely
the fact that different information about the system is held by the two
observers.

				If you see any bias in the way we compute
> partition functions, please say so.  Otherwise, I'm really not sure I
> understand this discussion.

No bias at all. Two observers with the same information about the same
system will deem the same set of states accessible, and therefore calculate
the same entropy. 

cheers,
Kevin.

kjm@eureka.buc.edu.au
From: mshute@cs.man.ac.uk (Malcolm Shute)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 28 Sep 92 14:16:41 GMT
Organization: Dept Computer Science, University of Manchester, U.K.

In article <1992Sep26.195005.10950@hellgate.utah.edu> tolman%mindblow.cs.utah.edu@cs.utah.edu (Kenneth Tolman) writes:
>Keep in mind there are many equivalent models of computation, of which
>the Turing machine is merely the "base model".  Fredkin's model
>basically avoids "erasing squares" by keeping them all around....
>I believe Toffoli once produced a reversible Turing machine, which kept
>the old information around somewhere... I am thinking of building one
>myself, embedded in a reversible cellular automata.

Errr.  At the risk I looking dense... I must confess to still not
understanding this.

If my computer is finding new answers for me, it must put these
answers somewhere.  If it puts them somewhere, it must overwrite
what was in the physical location before.  If it overwrites what
was there before... it must dissipate energy.

Take the following circuit as an example:

                   +----------+
        A ---->----|--      --|---->---- P
                   |   \  /   |
                   |    \/    |
                   |    /\    |
                   |   /  \   |
        B ---->----|--      --|---->---- Q
                   +----------+
                        |
        C ---->---------+---------->---- R

Where the cross-over connection is made as shown if C is high, otherwise
A is routed to P and B is routed to Q if C is low.

Suppose this is part of the logic for a vending machine, and that signal A
has the meaning 'Three 20p coins have been inserted', B means 'A 10p coin
has been inserted' and C means 'A 50p coin has been inserted'.  Signal P
therefore means '60p has been inserted', and can be used to unlock the next
bit of the mechanism.

I never delete the information from the A wire... it either gets routed
to P or to Q... but I *do* delete the information from the P wire
(it either gets a new value from A or from B).

At the very least, the P wire has resistance and capacitance, and therefore
memory.  As you point out, this is *necessary* if the circuit is to be
able to tolerate noise.  Therefore, this machine cannot work in a
dissipationless manner.

You say there *are* systems which can work in the absence of this problem,
but I do not understand how.  (BTW, thanks for the reference to suggested
background reading... I must chase that one, but haven't done so yet).
Surely, I can only get an answer if my machine *stores* the answer where
I expect to find it... and this storage dissipated energy??

--

Malcolm SHUTE.         (The AM Mollusc:   v_@_ )        Disclaimer: all
From: kjm@buc.edu.au (Kevin Moore)
Newsgroups: bionet.info-theory
Subject: Re: Mead and Conway (Section 9.11)
Date: 30 Sep 92 21:16:18 GMT
Distribution: bionet


In article <9209290750.AA01036@genbank.bio.net>, toms@fcs260c2.ncifcrf.gov (Tom Schneider) writes:

> In article <9209250126.AA04548@eureka.buc.edu.au> kjm@buc.edu.au (Kevin Moore) writes:

> | Entropy itself is anthropomorphic, in the sense that it depends on how much we
> | know about a system. (Jaynes, in "Gibbs vs. Boltzmann entropies" American
> | Journal of Physics 1965 demonstrates this rather well)

> This is wrong (or "not right" for those with weak hearts... :-) - the
> entropy change of a ribosome has nothing whatsoever to do with humans.
> Entropy changes are only anthropomorphic when you are discussing humans!!

I said "entropy", not entropy change. Whether entropy changes depend on
which set of parameters define your thermodynamic system requires a bit
more thought. Entropy certainly does, and is not well-defined, even on a
phenomenological level without specification of which parameters define the
thermodynamic state of the system. I suspect that they do, and I'll come
back to you on that one when I've had a proper think about it.

Cheers,
Kevin.

kjm@eureka.buc.edu.au
From: CZJ@CU.NIH.GOV
Newsgroups: bionet.general
Subject: GenBank Farewell
Date: 29 Sep 92 19:13:36 GMT
Distribution: bionet

A GenBank Farewell

Jim Cassatt

Twas the night before GenBank,
And one heard a great plea
For the entry of nucleotides,
Every A, C, G, and T.

The journals were flooded
With sequences and data;
They had to be collected,
Sooner or later.

Then came the Government,
Along with Sir Walter,
To establish a database,
That would limp but not falter

They called in Bilofsky,
Rindone and others,
Found EMBL and Hamm,
With his sisters and brothers.

They struggled and labored
and tried to make do
With a dedicated staff,
But resources were few.

Along came Sir Charles
And the great Genome dream.
These little bits of sequence
Were great clues it would seem.

With the help of the leaders,
Money opened the door,
To manage this resource,
As never before.

New leaders were found in
Mike, Dave B. and Julie,
who joined burks, Paul and Michael
Boy! Were they unruly.

The database flourished;
It was easy to sell,
Along with its partners
DDBJ and EMBL.

Access was expanded,
And BIOSCI grew.
Dave K. was most pleased,
As everyone knew.

In the midst of all this,
A new baby sings,
Brought into this world
By a Pepper of all things

The baby soon grew,
And it soon became apparent
That GenBank would flourish
With the baby as parent.

So with the transfer of GenBank
to NCBI,
We look forward to great things
And bid our goodbye.

Accomplishments are many;
The database did grow,
Beyond all expectations
Of those in the know.

It is readily available
To all who would want it,
At little or no cost
Every word, byte and bit.

To NCBI,
Good luck for the future.
We give you our child
To baby and to nurture.

The new tools look great;
The future is bright.
Happy sequencing to all,
and to all a good night!
From: turcotte@saguenay.IRO.UMontreal.CA (Marcel Turcotte)
Newsgroups: bionet.general
Subject: Looking for 5S rRNA sequences; preferably aligned.
Date: 1 Oct 92 18:47:10 GMT
Organization: Universite de Montreal -- Laboratoire Incognito


Subject says it all !
/MT
--

Marcel TURCOTTE(Etudiant Ph.D.)        Departement Informatique et Recherche 
e-mail:turcotte@IRO.UMontreal.CA        Operationnelle et departement Biochimie
                                       Universite de Montreal
tel. (514) 343-6111 ext.5354           C.P. 6128, Succ A
fax  (514) 343-2210                    Montreal (Quebec) Canada
pavillon principal local G-316         H3C 3J7