From: samodena@csemail.cropsci.ncsu.edu (S. A. Modena)
Subject: Re: The Dualism in info theory
Organization: Crop Science Dept., NCSU, Raleigh, NC 27695-7620
Date: Sun, 11 Oct 1992 07:29:35 GMT

In article <1992Oct10.171852.27566@wixer.cactus.org> sparky@wixer.cactus.org (Timothy Sheridan) writes:
>
>I hope i can get this right..

Well, I can't be sure exactly what you're up to: maybe you have a special
way of looking at things.....

>
>In the search for the info content we basically seem to use a reversable
>mapping process that can translate the signal into a code of minimum bits.

Well, probably not the signal, but the messages contained in/transported by
the signal.

There is, under Shannon, absolutely no requirement to map into a "code"
of minimum bits.  The messages (coded one way) can be translated into
another code (not necessarily the final one) of more bits.

>
>An interesting propertyn of these reversable mappings is that the best ones
>--that is the ones that maximally compress the signal -- seem to be more
>complex.

Perhaps you are interested in compression processes?  The entropy analysis
of the type that you all have been talking about, in fact, attempts to
appraise whether the current messages are encoded using redundancy.  If so,
then one may says that the current messages may be recast into a "more"
efficient code in that the total number of bits used to encode the
messages can be reduced without loss of information content.

However, certainly a non-redundant message can be recoded such that
the message will be shorter still, provided that you are willing to
accept some loss of inforamtion content.

>
>Take the entropy of english. if we just map the letters as they come it takes
>log(26+) bits. this direct mapping is obviously not very complex so if we map
>each possible word onto a log(100000) space then theres a reduction in avrg.
>bits/word but the transform is more complex.

Just to get you on track: what would be the point of what you suggest?
State *first* what your purpose is (codes *always* have a purpose).  Then
propose a contrast, if you are asking a question for someone to 
clarify a point for you.  Ok?  :^)

>
>if we asume a set of efficient transforms--ie that dont have extrainoius
>complexity for there results-- then is there a formal relation between the
>size of the transform and its ability to compress or determin minimum
>entropy in the signal?

I think the question is a non-question.  BTW, notice your mispellings.
If you had used a non-redundant encoding of that same long question,
the same number of spelling errors you made might have rendered the
question unintelligible to me.  By virtue of the fact that our common 
language is redundant, I can *correct* your spelling errors *efficiently*.
I am not a mathematician and dislike the use of 'transform', 'formal',
'formal relation[ship]', 'extraneous'.  Why do I dislike them?  Because
they are used in a way that I don't understand, and though they may
be FORMALLY CORRECT and EFFICIENTLY CONCISE, they garble the question for
me.  Axiomatic statements are so efficient that almost only the inventor
understands them....therefore they are efficient coding for a fellow
mathematician, but for the remaining 99.999% of humanity, they are
meaningless jargon.  :^)

>
>There several reasons for why this seems important. since a computer program
>that performs a mapping always requires a a 'computer to run it the question
>arises-"how much of the complexity of the mapping is hidden in the physical
>structure of the computer?". And, can we minimize the functional part of
>the systen so as to perhaps equate the form with the process..?

Again, are you trapping yourself in jargon?  :^)

Let's take something that amounts to the equivalent.  In the movie
"Amadeus," the jealous Saleriani (sp) stands at the foot of Mozart's
deathbed.  For a certain reason, Morzart's wife thrusts orchastration
scores into Saleriani's hands.  As he peruses them, it suddenly dawns
on him that there is not a single markover anywhere!  Mozart heard the
orchestra in his head and his genius guided his hand to score every
instrument perfectly, with no subsequent revision needed upon
reconsideration or upon hearing the orchastra play.

Do Androids Dream of Electric Sheep? (a.k.a. Blade Runner)

How much more(?) complexity(?) was in the hidden structure of Amadeus'
brain versus the hidden struture of Saleriani's brain?  

What has this to do with entropy?  :^)  Complexity might really be
quite another topic!  Who knows?  What transform would we need to
construct such that Saleriani could become Mozart?  What effect
would my somewhat lower level of genius have on the process?
Does what I'm asking make sense?  Who knows?  But these are
*hypothetically* legitimate questions within the Turing Machine framework.
But the Turing Machine is a fiction....Murphy Brown is a fiction,
but Dan Quayle doesn't seem to notice.  The replicants were fictions.
Nonetheless, fictions have lives of their own.  Perhaps one needs
a greater fiction to enforce to lesser fictions?  Was the protagonist
in Blade Runner a replicant?  Unaware of himself?  

Is this making any sense?  See my answer just below.  :^)

>
>Is this making any sense?

not to me, right now, but I might not be on your wavelength...or your ideas
might not be expressed well enough.

>
>can anyone help?

One thing that bothers me about some of the stepwise calculations we've
been treated to is that Shannon, at least, was interested in dynamic
processes.  The static calculations were meaningless and without utility 
in a practical way.  He was interested in the dynamic utility of
understanding entropy, encoding schemes and noise/error.  

That is why I posted that notice about the VLSI seminar, which I hope
to attend, for it's very Shannonesque flavor.  Reread it (you'll
have to fixup your .newsrc file).  The problem stated that VLSI
design methods have become inefficient at high transistor density
because, until now, only the STATIC parameters of the logic gates
have been considered in the silicon compilers.  Now however, submicron,
high density VSLI has become DOMINATED by the COMMUNICATION processes
BETWEEN the gates.  He is saying that the switching transient
characteristics used to be a problem, but now gates can be manufactured
that can run at very high clocks indeed.  It is the interconnections
that are important....the dynamics of signal propagation, signal 
degradation, and, of course, the effect of NOISE.  

Whether analysing the "entropy" of a digital signal or the
"complexity" of English sentences, Shannon was only interested
in their DYNAMIC properties....or rather he was fascinated by
their dynamic behaviors.  He struggled to capture their mystery.

Remember your Physics courses?  First statics; then, dynamics.  Which
has technological utility?  :^)

Steve
---
+------------------------------------------------------------------+
|     In person:  Steve Modena     AB4EL                           |
|     On phone:   (919) 515-5328                                   |
|     At e-mail:  nmodena@unity.ncsu.edu                           | 
|                 samodena@csemail.cropsci.ncsu.edu                |
|                 [ either email address is read each day ]        |
|     By snail:   Crop Sci Dept, Box 7620, NCSU, Raleigh, NC 27695 |
+------------------------------------------------------------------+
         Lighten UP!  It's just a computer doing that to you.
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
         
Newsgroups: bionet.info-theory