From: madavis@nmsu.edu (Mark Davis)
Subject: Re: AND gate puzzle for you
Organization: Computing Research Lab
	<1992Oct8.155813.26894@hellgate.utah.edu> <6411@m1.cs.man.ac.uk>
Distribution: bionet
Date: Wed, 14 Oct 1992 23:40:50 GMT

In article <6411@m1.cs.man.ac.uk> mshute@cs.man.ac.uk (Malcolm Shute) writes:

   In article <1992Oct8.155813.26894@hellgate.utah.edu> tolman%asylum.cs.utah.edu@cs.utah.edu (Kenneth Tolman) writes:
   >This is an interesting analysis!  I do not agree with it, but it feels
   >correct in a way, for why would we undertake computation unless we were
   >given new information?
   We could be doing it because we want to *reveal* information which,
   though already present, is obscurred by a surfeit of other information.

   >What would be the point of computation unless it
   >told us something we did not know?
   Because we were previously unable to see the wood for the trees.

   >If computation lost information at
   >every gate, what the heck would we be doing?
   We'd be taking a chain-saw to all but the information that we wanted, so
   that we could at last have an unobscurred view of the information of interest.

   And it's not just computation which works as a metaphorical chain-saw.
   I believe we are doing something similar when we are designing experiments.
   The art of choosing a suitable experiment for a particular measurement
   is to find some property of the universe which cause the characteristic
   in question to be attenuated least, and all other characteristics to be
   attenuated down into the noise level so that we can just about ignore them.

   For instance, you want to verify Newton's law of cooling...
   so you don't just play a bunsen burner on a patch of open sea, and watch it...
   you close in a measured sample in a flask, you sheild it from drafts,
   you would even have screened it from radio waves if you had any suspicions
   that radio emisions from Alpha Centuria would have an effect above the
   noise level of your experiment.

   On the other hand, if you were interested in measuring the amount of radio
   emision from Alpha Centuria, you'd build yourself a radio telescope, and
   shield it from bunsen burners (and other heat sources), etc.

Ray Solomonoff said something similar:  a scientist is attempting to
develop a theory about a system he or she is observing.  Imagine that
the measurables are coming in as a bit stream.  Now, a purely random
bit stream (and hence a purely random system) would be a bit
stream with no redundancies, and would only be representable as that
bit stream itself.  But the presence of patterns in the bit stream
suggests that there are ways of squeasing the redundancy out to
produce a more compact model.  The optimum theory is the minimal
coding of the stream.  Sort of an informational Occam's Razor.

The same can be said in a different way:  for a dynamics experiment,
the trajectory of an object is expressable as an (n-1) order
polynomial, where n is the number of measurements (i think this
is called AIC, Akaike's Info. Crit.).  Yet a second order polynomial
is perfectly adequate.  Why?  Because there is informational
redundancy in larger expressions which may help account for the
noise in the system, but which adds nothing in terms of the
"essential" properties over a huge set of similar experiments.

I have toyed with the relationships between statistical consequences
of information redundancy and logic systems for some time.  The
problem is that a more compact representation is not a better
representation than others.  A huge number of models can be
proposed which give similar predictive power, with larger, more
unwieldy ones perhaps giving better solutions in certain cases.
But the minimal model is seen as better for some reason, almost
a logical consequence.  And it is not.

mark





--
*************************************************************
***   M. W. Davis,             madavis@NMSU.edu           ***
*************************************************************
***       "Crap and Art still toiling together"           ***
*************************************************************