HDTenEightyP/NetNews huggingface dataset:
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Learning what COULD be learned
Organization: University of Alberta, Edmonton, Canada
Date: Wed, 22 Jul 1992 12:57:07 GMT

uds@wsu-eng.eng.wayne.edu (Seetamraju Udaybhaskar) writes:

>In article <arms.711759321@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>>
>>If a tree isn't learning the required task, as shown by a lack of
>>further improvement, then you can double the size of the tree for that
>>particular output and try again.  Does this make it clear how the
>>technique of independent trees would be used?

>could you clarify, about how, with the number of inputs constant, the
>size of the tree could be varied ?

Sure.  The use of the term tree is a misnomer if you also include the
inputs, which must be in general connected to several nodes in the
first layer of elements.  Sorry for the terminological problem.
To learn XOR, for example, you could start with at tree of two layers
and synthesize x1* (~ x2) + x2 * (~x1). The input layer fans out to two
nodes each.  Those nodes are ANDs (*) and those ANDs output to an OR.

What I have against non-trees is that there has to be a good reason
for supposing that a node's learned output will serve two or more
specific needs well.  This seems to me to make learning much harder.
But at the input level, there is no learning, so the argument doesn't apply.
Besides, the only way to generate arbitrary functions is with repeated use of
inputs (and complements in the case of ALNs).

Hope that clears it up.

Bill
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: kps2@ellis.uchicago.edu (kenneth paul scholz)
Subject: Re: Neural Nets and Brains
Reply-To: kps2@midway.uchicago.edu
Organization: University of Chicago Computing Organizations
Date: Fri, 24 Jul 1992 16:22:37 GMT



Bill Armstrong wrote:

>In sum: to study the brain, in my opinion, based on ignorance of how
>the brain works, is that one should study a system that works on
>logical signals.  Although ALNs might help in this, they are not
>designed as a brain model at all, and different learning algorithms
>would have to be developed to capture the learning properties of
>neurons.
>
>Are we getting closer?

IMHO a system that works exclusively on logical signals would not
approximate the function of the brain or even a relatively simple
neuronal circuit.  Let me use an example to stress my point, the
monosynaptic reflexes in the spinal cord.  When you touch a hot object,
your sensory neurons fire and synapse directly onto motor neurons.  The
motor neurons activate muscle cells that cause withdrawal from the
hot object.  This form of reflex action could probably be modeled
by a logical NN fairly effectively.  However, you have now 'learned'
that the object is (an can be in the future) hot. This learning
might involve a number of modulatory actions (whatever that means) in 
the brain, which are often not logical processes.  Yes, the end effect
of these modulatory actions *might* be to increase or decrease firing
rate of some neuron, but the 'memory' of the learned event is likely
to be stored as a graded function at some synaptic connection.

Someone earlier quoted a scientist as saying they would like to trace
all the signals through the brain to examine how the signal is processed.
If this was done just by recording action potential frequency, well, yes
you might get something analogous to a Thevenin equivalent circuit, but
you would miss alot.  In particular, you would miss the underlying
mechanisms that the brain uses to alter neuronal activity.  Thus, you
would probably miss many of the mechanisms that the brain uses to 'learn'

-- 
Ken Scholz                    Department of Pharm. and Physiology
kps2@midway.uchicago.edu      Univ. of Chicago
From: lordSnooty@cup.portal.com (Andrew - Palfreyman)
Newsgroups: comp.ai.neural-nets
Subject: Re: Thanks Neural Nets, and Brains?
Date: Fri, 24 Jul 92 19:35:06 PDT
Organization: The Portal System (TM)
  <1992Jul23.222259.9713@hubcap.clemson.edu>
  <1992Jul24.145301.12566@midway.uchicago.edu>

: Benjamin Bryant responded:
: 
: Yes, it is very strange that the most widespread artificial model of
: neural operation uses continuous quantities, and I agree whole-heartedly,
: that more research is needed in the way neuronal firings, (mean-rate),
: encode information in the brain. It would seem that these firing
: frequencies (i.e. spikes/sec or some similar unit) actually encode
: what are our thoughts. It will indeed be a major breakthrough when
: we are actually able to trace these firing rates through living brain
: issue.

Recently I've seen, on TV no less, some fabulous animations produced by
nascent attempts in this field; either from CAT- or PET scans. With the
advent now of more sensitive SQUIDS, the day cannot be far away for real
time monitoring of this fascinating domain.
 
Does anyone have up-to-date information as to where this is heading now?
 --------------------------------------------------------------------------
| lord snooty @the giant | how DARE they cancel Dennis Miller?             |
| poisoned electric head |                               andrew palfreyman |
 --------------------------------------------------------------------------
Newsgroups: comp.ai.neural-nets
From: raghu@morgan.ucs.mun.ca (Raghu B)
Subject: BACK PROPOGATION - DOUBTS?
Keywords: bP
Organization: Memorial University of Newfoundland
Date: Sun, 26 Jul 1992 20:18:03 GMT



	I am using Back Propogation algorithm to train my net. I have
chosen a random number between -1. and 1.0. I am giving a set of input
datas (say 20). I trained that and and the neural output almost matched
the desired output in the case (where the output lies between -1 to 1.)
But in cases where the output is going above 1 my net fails to train
them and it is giving a large error.

	I am giving 3 input values and I wanted 3 output values with
11 hidden layers. I am just using the basic back propogation algorith.
I am not giving any bias values? Correct me if I am wrong. Does the
bias input have any effect in shifting the results or does it give
a faster convergence.  Or does momentum parameter help in anyway
to get convergence.

	I hope you could understand my question? If not do write
back to me. I am not a hi-fi computer engineer. I am just  a basic
mechanical engineer and I could understand most of the terms in
NN field.

	Expecting positive responses from dear netters.


				Raghu,Balasubramanian

				raghu@engr.mun.ca
From: rr2p+@andrew.cmu.edu (Richard Dale Romero)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Nets and Brains
Date: 24 Jul 92 17:37:49 GMT
	<arms.711986585@spedden>
Organization: Carnegie Mellon, Pittsburgh, PA

Excerpts from netnews.comp.ai.neural-nets: 24-Jul-92 Re: Neural Nets and
Brains Bill Armstrong@cs.UAlber (3087)

> Don't you agree that if the brain works on 0-1 signals, then to study
> the brain one could beneficially look at logical systems?

The brain working on 0-1 signals has nothing to do with it being a
logical system :-)

Rick Romero
Carnegie Mellon University
Newsgroups: comp.ai.neural-nets
From: tag@hermes.ecn.purdue.edu (Tony A. Gibbens)
Newsgroups: comp.ai.neural-nets
Subject: local minima
Keywords: local minima, backpropagation, error rate
Date: 30 Jul 92 20:52:52 GMT
Organization: Purdue University Engineering Computer Network


I am hoping that somebody or perhaps multiple individuals 
will respond to my request.   I need a collection of various
problems likely (yes likely) to get stuck in local minima when
trained with backpropagation.   If possible, I would like these
problems to be small or at least manageble.  What I mean to say
is that if possible, I would like the training (and testing) data
sets not to be overwelmingly complex to generate or over 2000 units
(items, data units?) each.  

Please mail me with the responses.  I do not always read this group.  
If you wish, any responses I recieve, I will take the time to post.

thankyou very much 
tag@ecn.purdue.edu (Tony A. Gibbens)
From: scottd@cs.hw.ac.uk (Scott Dunn)
Newsgroups: comp.ai.neural-nets
Subject: Dynamically adding nodes, layers ?
Date: 31 Jul 92 13:27:15 GMT
Organization: Dept of Computer Science, Heriot-Watt University, Scotland

I hope someone can help me here,  I seem to recall reading somewhere
about a learning algorithm which first checks to see if the new pattern
can be learnt without overwriting old info, and if not, then adding new
nodes as required.  I *DEFINITELY* read about this somewhere, unfortunately
I dismissed this as being of little import to what I am trying to do.
However, the world being as it is, this irrelevance has become relevant.
So, if anyone can help me I would be most appreciative.

Cheers,

Scott.
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Neural Nets and Brains
Organization: University of Alberta, Edmonton, Canada
Date: Tue, 28 Jul 1992 19:19:34 GMT

tedwards@src.umd.edu (Thomas Grant Edwards) writes:

>In article <arms.712027061@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>>Thank you.  A nice explanation.  Unfortunately, taking phase and
>>density of pulses into account makes for quite a complex, non-linear
>>system (refractory periods, neurotransmitter release and uptake,...).
>>If the brain works that way, then the simple multiply-add
>>accumulations of backprop are over-simplified, aren't they?  Then
>>maybe both backprop and ALNs are useless for the purposes of brain
>>modelling.

>Yes!

>But they are (more to the point, were) useful in showcasing that
>one can algorithmically string together parallel computational
>units to produce a specific behaviour.  Certainly brain does not use
>backprop or ALNs specifically, but the mathematical insight we gain
>from them helps us to understand what's going on. 

Thanks for your comments.  I would still like to know what the
neurobiologists (or people who study neuron information processing,
whoever they are) think of the idea of using a boolean logic system to
capture the functionality of how a neuron's axon firings depend on the
axon firings of neurons which synapse upon it.  If you look at the
little "mosquito" demo in atree 2.6, you see how ALNs are used to
process 80-bit input vectors to learn to predict malaria (a toy
problem, not a real one). The only difference in using atree for
studying axon firings is that you would have to use more inputs
(thousands) and you would have to represent signals at several input
time steps. It would be interesting if someone had data on axonal
firings in a collection of neurons that one could apply atree to.

Any pointers to people working on this who might be interested?

Bill


--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: tridge@nimbus.anu.edu.au (Andrew Tridgell)
Subject: comp.speech?
Organization: Comp. Sci. Lab., Australian National Uni.
Date: Mon, 27 Jul 92 01:46:17 GMT

I think comp.speech would be very useful. I have hesitated to post speech
specific stuff to comp.ai.* or comp.dsp as I know that most people
using the group would not be interested in very specific speech
discussions.

I think it is particularly appropriate at this stage as many speech models
are appearing which combine such techniques as neural-nets, decision-trees
and statistical models, so relevant questions often would span many 
newsgroups.

I am unfamilar with the process of creating such a group but would be
willing to try if a more experienced person could not be found.

I doubt the comp.speech group would need to be split into sub groups at
this stage.

Perhaps others interested in such a group could send me some e-mail
so we can start to get things organised.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Andrew Tridgell                 CSLab, Research School of Physical Sciences
Andrew.Tridgell@anu.edu.au      Australian National University (x3064)
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
From: henrik@mpci.llnl.gov (Henrik Klagges)
Newsgroups: comp.ai.neural-nets
Subject: Re: ALN vs BP
Date: 24 Jul 92 16:08:11 GMT

tsioutsias-dimitris@CS.YALE.EDU (Dimitris Tsioutsias) writes:

>It seems that after the backprop fans, we have now the ALN ones. Why

You are over-generalizing from too few input-output patterns of the 
form (Bill Armstrong -> ALN yes) here. Count me as a (Henrik Klagges
-> ALN so-so) 8-) !
--

Cheers, Henrik
MPCI at LLNL
IBM Research
From: bernard@srt-poste.fr (Bernard Lemarie )
Newsgroups: comp.ai.neural-nets
Subject: Poggio & Girosi Paper
Date: 23 Jul 92 16:36:41 GMT
Reply-To: bernard@srt-poste.fr
Organization: Service de Recherche Technique de la Poste

I'm looking for Poggio & Girosi: "Extensions of a theory of networks for approximation and Learning: dimensionality reduction and clustering ",Artificial Intelligence Lab. Memo, No 1167, MIT.  What's the best way to get a copy of this?  Is there an address or e-mail at MIT to which 
I can write?  Thanks.
Newsgroups: comp.ai.neural-nets
From: bdbryan@eng.clemson.edu (Ben Bryant)
Subject: Re: Thanks Neural Nets, and Brains?
Reply-To: bdbryan@eng.clemson.edu
Organization: College of Engineering, Clemson Univ.
Date: Thu, 23 Jul 1992 22:22:59 GMT

In article 711907358@spedden, arms@cs.UAlberta.CA (Bill Armstrong) writes:
>bdbryan@eng.clemson.edu (Ben Bryant) writes:
>
>>In article 57397@cc.usu.edu, sl859@cc.usu.edu () writes:
>
>>> 	I do have an additional question.  How extensive has the research
>>>been on neurons of a neural net and the actual neurons of the brain?  I 
>>>am interested mostly out of curiosity, but from what I've read I can only 
>>>deduct that we still haven't got a very good model of the actual workings 
>>>of the brain neurons, let alone how to model them with computers.
>
>>  ... one scientist (I can't remember his name) said that he would
>> one day like to be able to follow the path of firing responses through
>>the brain using some MAJOR HEAVY DUTY processing and some HYPER-SENSITIVE
>>electrical sensors. -I believe when that time comes, this the field of
>>artificial life (Data from Star Trek?) will appear. That was an article in
>>one of the spring `92 issues of PS.
>
>First off, isn't it rather strange that the most widespread artificial
>model of neural operation. the multilayer perceptron, uses continuous
>quantities on its connections, while the dendrites and axons of
>neurons use "zero or one" type action potentials?  Until physiological
>psychologists start studying adaptive logic networks, can anyone
>expect much progress on understanding the brain?

Yes, it is very strange that the most widespread artificial model of
neural operation uses continuous quantities, and I agree whole-heartedly,
that more research is needed in the way neuronal firings, (mean-rate),
encode information in the brain. It would seem that these firing
frequencies (i.e. spikes/sec or some similar unit) actually encode
what are our thoughts. It will indeed be a major breakthrough when
we are actually able to trace these firing rates through living brain
tissue.

>What use the the MAJOR HEAVY DUTY processing if there is no model to
>help understand the operations?  Imagine trying to understand an
>operating system by looking at the flows of bits in a computer system.

Again, I agree whole-heartedly.

>***************************************************
>Prof. William W. Armstrong, Computing Science Dept.
>University of Alberta; Edmonton, Alberta, Canada T6G 2H1
>arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071

Benjamin Bryant
<bdbryan@eng.clemson.edu>


Newsgroups: comp.ai.neural-nets
From: rka@fct.unl.pt (Rafaela Kufahl Azinhal)
Subject: Looking for GENESIS (Genetic Alg. System) from Grefenstette
Organization: Universidade Nova de Lisboa -- Lisbon, Portugal
Distribution: comp.ai.neural-nets
Date: Thu, 30 Jul 1992 15:06:03 GMT


The Handbook of Genetic Algorithms by Lawrence Davis, VNR 1991,
supplies a diskette with two Genetic Algorithm Systems:
. OOGA - an object-oriented Genetic Algorithm system written in Lisp,
and 
. GENESIS - a Genetic Algorithm system written in C by John Grefenstette.

I've installed OOGA with no problems, but the diskette contains bad
sectors while copying GENESIS.

I already emailed a message to gref@aic.nrl.navy.mil, but had no
response.
Does anyone know where and how I can get GENESIS? Please email to my
address.
--
+---------------------------------------------------------------------+
| Rafaela Kufahl Valente Azinhal    | BITNET/Internet: rka@fct.unl.pt |
| Universidade Nova de Lisboa       |          UUCP: rka@unl.uucp     |
| FCT - Dpto. de Informatica        | Fax:   (+351) (1) 295-5641      |
| Quinta da Torre                   | Telex: 14 542 FCTUNL P          |
| 2825 Monte Caparica, PORTUGAL     | Phone: (+351) (1) 295-4464      |
+---------------------------------------------------------------------+
From: demers@cs.ucsd.edu (David DeMers)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Net for Matlab
Date: 22 Jul 92 20:45:43 GMT
Organization: =CSE Dept., U.C. San Diego

In article <1992Jul22.134759.25636@uceng.UC.EDU> chan@uceng.UC.EDU (Alistair Keating Chan) writes:

>Has anyone seen or written a standard backpropgation algorithm
>for Matlab that they would like to and be able to share?

No, but MathWorks sells a Neural Networks Toolbox, just out.  I don't 
know the price.  I saw it demonstrated a couple of months 
ago and it seems pretty spiffy.  Sorry, I don't have a
better or more detailed review than that :-)

However, standard bp is pretty easy to code up, in my opinion.
Forward activation is just matrix multiply, squash, matrix multiply,
squash...
and standard delta rule learning is 
compute delta, update weights, back prop (matrix mult),
compute delta, update weights, back prop...

Dave

-- 
Dave DeMers			 ddemers@UCSD   demers@cs.ucsd.edu
Computer Science & Engineering	C-014		demers%cs@ucsd.bitnet
UC San Diego					...!ucsd!cs!demers
La Jolla, CA 92093-0114	(619) 534-0688, or -8187, FAX: (619) 534-7029
From: dekorte@scrinext.scri.fsu.edu (Stephen L. DeKorte)
Newsgroups: comp.ai.neural-nets
Subject: Re: Speech recognition and Time Warping
Date: 21 Jul 92 19:26:15 GMT

In article <1992Jul10.115834.15707@ucthpx.uct.ac.za> brendt@dip1 (Brendt  
Wohlberg) writes:
..
> and this isn't a neural-nets question (on this subject, why is there no 
> separate group for general pattern recognition -  
comp.ai.pattern-recognition say?)

comp.ai.pattern-recognition sounds like an excelent idea.
Anyone else interested in starting such a news group?

SD
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Body transplants
Date: Sat, 25 Jul 92 16:02:44 GMT
Organization: School of Computer Science, Carnegie Mellon


    From: jim@netlink.cts.com (Jim Bowery)
    
    I just saw a brief part of a show on a researcher who has been
    transplanting the bodies of primates and apparently is now in a
    position to attempt to do so with humans -- or so he believes.
    His claim is that it is actually more difficult to transplant
    limbs than it is an entire body.  Why this would be so or whether
    this guy is off in left field I don't know.  But the show was on
    public broadcasting and delved into the ethical issues quite
    extensively.  The researcher is a devout Catholic.
    
I saw a TV report on this guy as well, though I don't remember his name.
He has swapped some monkeys' heads (or bodies, depending on your point of
view) and kept the subjects alive for some time after that, in the sense
that the brain gets the blood, nutrition, and oxygen it needs.  However,
according to the piece I saw, he had not "yet" made any serious attempt to
make functioning neural connections between the newly-mated parts, so the
resulting animals are totally paralyzed below the neck, and I believe that
breathing is only possible with a respirator.  Needless to say, the animal
rights people take a dim view of the whole thing.

General discussions of this sort of experiment and its morality probably
belong on some other newsgroup than this one (unless there is more work
going on in connecting up the nerves than was reported).

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: marcus@sun-of-smokey.NoSubdomain.NoDomain (Jeff Marcus)
Newsgroups: comp.ai.neural-nets
Subject: Re: need for unique test sets
Date: 23 Jul 92 14:07:31 GMT
Organization: MIT/LCS Spoken Language Systems

In article <1992Jul21.224019.6615@u.washington.edu>,
davisd@milton.u.washington.edu (Daniel Davis) writes:
|> I hope I can clear up this debate with a little specificity.
|> 
|> A couple guys say that all we need is independant sampling, while
|> someone else seems to think that one should not include the training
|> data in the test set.
|> 
|> Independant sampling is in fact all you need, but given the proper
|> context, it is also proper to say that one should not include any of
|> the training data in the test set.
|> 
|> Suppose you take 10000 independant samples. You use 5000 as your
|> training set. You would *not* select your test set from all 10000
|> samples, but instead, only from the 5000 not included in the
|> training
|> set. If you selected test data from a random sampling of all 10000
|> samples, your test data and your training data would no longer be
|> independant. Of the data you have, only the 5000 previously
|> unselected
|> data correspond to data independant of your training set. In this
|> sense, then, one should not include any of the training data in the
|> test data. 
|> 
|> However, it is *not* a problem if it happens that some of the 5000
|> previously unselected data are in fact repeats of the original
|> training data, as it is assumed that the original 10000 were
|> independant samples.
|> 
|> Buy Buy -- Dan Davis
|> Univ. of Washington, Dept. of EE, davisd@u.washington.edu

Exactly. I should have been this specific in making my argument. It might
have saved some bandwidth.
  Jeff
From: zonker@diku.dk (Claus Engdahl)
Subject: Re: Speech recognition and Time Warping
Date: Thu, 23 Jul 1992 10:47:53 GMT
Organization: Department of Computer Science, U of Copenhagen

dekorte@scrinext.scri.fsu.edu (Stephen L. DeKorte) writes:

>In article <1992Jul10.115834.15707@ucthpx.uct.ac.za> brendt@dip1 (Brendt  
>Wohlberg) writes:
>..
>> and this isn't a neural-nets question (on this subject, why is there no 
>> separate group for general pattern recognition -  
>comp.ai.pattern-recognition say?)

>comp.ai.pattern-recognition sounds like an excelent idea.
>Anyone else interested in starting such a news group?


Yes, definitely.






-- 
Claus Engdahl (zonker@diku.dk)
Newsgroups: comp.ai.neural-nets