HDTenEightyP/NetNews huggingface dataset:
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Neural Nets and Brains
Organization: University of Alberta, Edmonton, Canada
Date: Sat, 25 Jul 1992 01:17:41 GMT

clarke@acme.ucf.edu (Thomas Clarke) writes:

>In article <arms.711986585@spedden> arms@cs.UAlberta.CA (Bill Armstrong)  
>writes:
>> ....  My point
>> was that BP nets use continuous signals and the brain doesn't -- an
>> obvious very significant difference.  I was asking why people would
>> expect to understand the brain by studying a system (BP) that is
>> *different* at the most basic level of signalling.
>> 
>> Don't you agree that if the brain works on 0-1 signals, then to study
>> the brain one could beneficially look at logical systems? 

>I believe the brain may well use continuous or analog signals.

>While individual neuron action potentials are on or off, they 
>often occcur in trains of varying frequency.  The time-density of 
>action potentials then forms an analog or continuous variable 
>that may be processed in much the same way as a BP network.  

>In fact, if you posit a central clock in the brain, the phase of 
>the action potentials with respect to the clock provide another 
>possibility for a continuous variable.  Indeed, both phase and 
>density could work together to make the brain a 
>complex-valued neural network :-)  

Thank you.  A nice explanation.  Unfortunately, taking phase and
density of pulses into account makes for quite a complex, non-linear
system (refractory periods, neurotransmitter release and uptake,...).
If the brain works that way, then the simple multiply-add
accumulations of backprop are over-simplified, aren't they?  Then
maybe both backprop and ALNs are useless for the purposes of brain
modelling.
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
From: bill@nsma.arizona.edu (Bill Skaggs)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Nets and Brains
Date: 26 Jul 92 19:54:31 GMT
	<arms.711986585@spedden> <1992Jul25.031126.1722@news.iastate.edu>
	<arms.712073050@spedden>
Organization: ARL Division of Neural Systems, Memory and Aging, University of
	Arizona

arms@cs.UAlberta.CA (Bill Armstrong) writes:

   >Aside from brain modelling using purely boolean signals, there are 
   >other potential uses of ALNs un studying the brain; namely
   >explaining what is happening in nerve cell collections.  If the
   >axons have action potentials, then it would still be possible,
   >even if there are continuous signals at other places in the
   >system, to study the relationships among axon firings using
   >boolean methods.  One would have to relate the firing of axon A to
   >firings of others B, C, ... at previous time instants.  In this
   >way, one would factor out the continuous parts of the operations
   >and concentrate on the functional modelling just based on boolean
   >operations and time delays. 
   >
   >Does anyone know if this has been tried?

Hmm, well.  It's at least been *thought of*.  The problem is that when
there are potentially thousands of inputs to a cell, and the cell
integrates inputs over a hundred or so time steps, it takes way too
much space to represent the operations in Boolean terms.  If you're
trying to get the relationships biologically correct, the only
reasonably efficient way to represent neural operations is in terms of
difference equations.

At a more abstract level, there is a long history of cellular-
automaton style models of neural systems.  I have to say that I'm not
very impressed by their overall success: it seems that such models
almost invariably show strange phenomena (particularly long and
complex cycles) that disappear when the models are made more
realistic.

On the other hand, *probabilistic* cell-aut models have made some
important contributions.  The simple Ising model -- consisting of a
layer of binary units connected to their nearest neighbors and having
a tendency to switch into the same state as the majority of their
nearest neighbors -- has played a very important role in the
development of statistical mechanics (particularly the theory of phase
transitions).  The theory of attractor neural networks owes a great
deal to it and related simple models.

	-- Bill
From: erik@til.til.com (Erik Horstkotte)
Subject: Re: Fuzzy Logic and ANN's
Organization: Togai InfraLogic, Inc.
Date: Tue, 21 Jul 1992 16:46:19 GMT

You might want to check out Dr. Bart Kosko's book:

    Kosko, Bart A., Neural Networks and Fuzzy Systems, Prentice-Hall,
    1990.

He chatters a lot about hybrid fuzzy/NN systems, but more importantly,
he discusses the use of NN learning techniques to train fuzzy expert
systems.  In fact, we (Togai InfraLogic) have a commercial product
available (TILGen) that uses this technique (among others).

We are currently working on a genetic algorithm approach to fuzzy
expert system training that seems to work *much* better.  It will be
incorporated in the next generation of our products.

Some papers (I have no titles, sorry) have been presented over the last
few years on the subject of "self-organizing" fuzzy controllers, that is,
fuzzy expert systems that modify themselves to (hopefully) improve their
performance.  I'm not intimately familiar with the technique, so I'll
let others discuss it.
-- 
Erik Horstkotte, Manager, Software Development, Togai InfraLogic, Inc.
erik@til.til.com, uunet!til!erik - (714) 975-8522
Standard disclaimer: the preceding are my opinions, not TIL's.
Newsgroups: comp.ai.neural-nets
From: ellner@stat.ncsu.edu (Steve Ellner)
Subject: Re: Training Networks on Chaotic Time Series
Originator: ellner@esssse.stat.ncsu.edu
Keywords: chaos, time series, neural nets
Organization: Statistics, NCSU
Date: Thu, 23 Jul 1992 18:09:46 GMT


In article <670@trwacs.fp.trw.com>, erwin@trwacs.fp.trw.com (Harry Erwin) 
reminds us that neural nets performed poorly on chaotic time series
data generated by the logistic map:
 
|> ... the network was clearly learning the training set and not
|> generalizing. This was particularly clear for the more highly chaotic
|> cases since the effectiveness of the network was significantly reduced for
|> a fixed training period. It was also clear that the network "hadn't a
|> clue" for regions of the chaotic process that were not represented in the
|> training set.

   Our experience on similar problems has been better. But, we found that
it was essential to have a cautious criterion for adjusting the network
complexity to match the amount and quality of training data, and 
to be _very_ persistent at seeking best-fit net parameters,  i.e., 
a stringent convergence criterion, and many, many replicate 
attempts at fitting with different starting parameters. (Refs: D. Nychka
et al., J. Royal Statistical Soc. Series B, vol54(2), 399-426 (1992) ; 
A.R. Gallant & H. White, Neural Networks 5,129-138; and Dan McCaffrey's 1991
Ph.D thesis, to appear this Fall in J. Amer. Stat. Assoc.). M. Casdagli 
(Physica D35, 335-356 (1989) ) also found that neural nets were among
the best predictors for chaotic time series (see his Table I).

   I expect that any regression model, not just nets, will have trouble 
"learning" what's going on far from any data that are used to fit the model. 
That may not be a problem if generalization near the training data suffices
for the task at hand. For example a short-term predictor only needs 
to be accurate near the support of the system's invariant measure (since
real trajectories will always lie in that region) and that's where you would
get training data from observing the system running "on its own".

--- Steve Ellner (ellner@stat.ncsu.edu)
From: Daniel.K.R.McCormack@cm.cf.ac.uk (Daniel K R McCormack)
Newsgroups: comp.ai.neural-nets
Subject: Survey data analysis using NN
Date: 22 Jul 92 12:04:43 GMT
Organization: University of Wales College of Cardiff, Cardiff, Wales, UK.

Hello Neural Netters,

	I am looking for referneces on the analysis of trends in 
survey data ( or similar applications ).

	I know BP has been used for credit risk assesment, has any 
other network been used for survey data analysis and does anyone think
a particular network would be more suitable to this application
and why? 

Please reply by email

	Many thanks

		Danny

P.S I will summarise if there is enough interest!
Newsgroups: comp.ai.neural-nets
From: tom@cs.su.oz.au (Thomas James Jones)
Subject: Re: Neocognitron: ftp address
Reply-To: matty@cs.su.oz.au
Organization: Basser Department of Computer Science
Date: Tue, 21 Jul 1992 23:01:59 GMT

In article <1992Jul21.145731.26702@tamsun.tamu.edu>, fs0997@easttexas.tamu.edu (Frank Schnorrenberg 5-9403) writes:
|> 
|> There is an implementation of the simulator AVAILABLE at the following
|> location using anonymous ftp:
|> 
|>                 128.194.15.32 alias tamsun.tamu.edu
|>                 Filename: /pub/neocognitron.Z.tar
			
			/pub/neocognitron.tar.Z

		Is the file name (obviously I guess)

		Thankyou for providing it

	tom

|> 
|> I hope this clears the confusion my mail server created by not 
|> sending notification messages to all the people interested.
|> 
|> If you have comments etc., send e-mail to schnob@cs.tamu.edu.
|> 
|> 
|> The simulator is written in C and comes with a list of references 
|> which are necessary to read to understand the specifics of my
|> implementation. I coded the unsupervised version without (!)
|> C-cell inhibition.
|> 
|> Take a look and if you like it- fine, if not, remember: 
|> It was free!
|> 
|> 		
|> --
|> Frank Schnorrenberg				e-mail:	schnob@cs.tamu.edu
|> Department of Computer Science
|> Texas A&M University
|> College Station, Texas
Newsgroups: comp.ai.neural-nets
From: kawai@Csli.Stanford.EDU (goh kawai - n6uok)
Subject: comp.ai.pattern-recognition (was: Speech recognition and Time Warping)
Organization: speech research and technology, sri international
Date: Tue, 21 Jul 1992 21:33:34 GMT

Brendt Wohlberg suggests:

   > why is there no separate group for general pattern recognition - 
   > comp.ai.pattern-recognition say?

Stephen L. DeKorte suggests:

   > comp.ai.pattern-recognition sounds like an excelent idea.
   > Anyone else interested in starting such a news group?

I am a linguist (a phonetician, to be precise) working on speech
recognition.  I would be very interested in starting a news group, either
for pattern recognition in general or for speech processing in particular.
My colleagues and I recently established a speech processing newsgroup in
Japan -- fj.comp.speech -- and would like to see something similar here on
Usenet.  

----------------- speech technology and research, sri, menlo park, california
--- Goh Kawai --- work:(415)859-2231   fax:(415)859-5984   home:(415)323-7214
----------------- internet: kawai@speech.sri.com      radio: n6uok and jk1zyp
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: need for unique test sets
Organization: University of Alberta, Edmonton, Canada
Date: Tue, 28 Jul 1992 15:54:28 GMT

hcbarth@afterlife.ncsc.mil (Bart Bartholomew) writes:

>	At the moment, we don't have any tools that I know
>of to examine a BP net and understand what's going on except
>for some trivial cases.  I suspect I'd recognize an XOR net
>fairly easily, but there are only four distinct configurations
>for an XOR BP net.  More complicated functions are more difficult
>to analyze.  Not necessarily impossible, but definately more
>difficult.  At this point I expect Prof Armstrong to point
>to the inordinate ease with which an ALN can be analyzed.

Why should I, especially after pointing out that the analysis problem is
provably intractable (NP-complete).  I just agreed with Scott Fahlman on this
on the net yesterday.

If you want to criticize me for something I actually advocate, why
don't you take issue with my ideas for a "safe design methodology"?

>I remain unconvinced, however, of the universal applicability
>and/or superiority of ALNs over BP in all situations.

I would expect you to remain unconvinced, at least until ALN software
and hardware that is actually superior is available.  I do try to
reason about the extreme inefficiency of BP and how ALN superiority can
be achieved though.  People who want to follow the reasoning will do
so; but most won't.  Most of this will remain just that -- reasoning
-- until adequate funding is available for the ALN approach, funding
which is now being plowed into supporting arithmetic BP networks that
have no future at all.


One more question please, that maybe you are in a position to answer
as a result of your (.mil) domain.  I have done contracts for the
Defence Research Establishment Valcartier, which showed that ALN ideas
could be applied to FLIR images (ca. 1980).  I heard from someone
attached to a foreign military research establishment last year that
my stuff was actually quite popular in their country.  That surprised
me a lot, since it didn't seem like the ALN idea had gotten very far.
Do you know of any other non-classified work on ALNs that has been
done for the military, that you could point me to?

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
From: lordSnooty@cup.portal.com (Andrew - Palfreyman)
Newsgroups: comp.ai.neural-nets
Subject: Re: Correctness of NNs
Date: Fri, 24 Jul 92 19:49:57 PDT
Organization: The Portal System (TM)

 Robert B Lambert writes:
[...]
: Neural networks, like fuzzy logic have the advantage of being able to produce
: sensible responses to new inputs. In handwritten OCR for example, reading a 
: character at a time be it by a human, ANN, or some other technique, must 
: give recognition rates of less than 100%. Everyone writes differently and 
: no system could be trained on examples of every human beings writing.
[...]

If you cast your mind back to the time when (oh, 1985/6 maybe) Expert Systems
were the New Kids On The Block, you'll recall that NNs made a competitive
splash by emphasising their "robustness" vis a vis ESs, especially in the
marginal domains of the knowledge base, where ESs would suddenly and often
catastrophically "lose it" while NNs would "gracefully degrade".

These days it seems that all this is now taken for granted. The fuzzy logic/ 
ES meld, however, seems to have reversed this somewhat.

Apologies for the handwaving generalisations.
 --------------------------------------------------------------------------
| lord snooty @the giant | how DARE they cancel Dennis Miller?             |
| poisoned electric head |                               andrew palfreyman |
 --------------------------------------------------------------------------
Newsgroups: comp.ai.neural-nets
From: gsbg9079@uxa.cso.uiuc.edu (G. Scott Braley)
Subject: Help requested for backprop
Organization: University of Illinois at Urbana
Date: Fri, 31 Jul 1992 18:49:30 GMT

I am looking for a "real world" problem to apply backprop to.  Ideally it wouldnt be too complex-my nn software is running on a 386, and I need realistic 
training time.  I don't have any fancy equipment like scanners and such for
OCR or Computer Vision.  I don't know why, but I just can't think of anything!
Any help would be appreciated.

				-Scott

-- 
*****************************************************************************
*  F.B.I.B.M.--The most feared merger  *  gsbg9079@uxa.cso.uiuc.edu         *
*****************************************************************************

Newsgroups: comp.ai.neural-nets
From: lloyd@cs.monash.edu.au (lloyd allison)
Subject: Re: Bayes Classifier & Cluster Analysis software required
Organization: Computer Science, Monash University, Australia
Date: Mon, 27 Jul 1992 02:29:32 GMT

gibsoni@ee470.ee.mcgill.ca ( irwin roger  gibson) writes:
   >Bayesian Classifier & Cluster Analysis
   >=======================================
   >If anyone has or knows of the following types of software and there
   >availability: (ftp or email)
   >Bayesian Classifier & Cluster Analysis (DOS or UNIX)
   >This software is urgently required for use in Neural Network Research.
try:
C. S. Wallace
Classification by minimum message length inference.
Advances in Computing and Information (ICCI-90) Springer-V LNCS 468, 72-81 1990

describes a Bayesian classifier `SNOB' (!)
Written in Fortran, handles thousands of things with dozens of attributes
very quickly.

Contact for SNOB: David Dowe,  dld@cs.monash.edu.au
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Neural Nets and Brains
Date: Fri, 24 Jul 92 17:39:03 GMT
Organization: School of Computer Science, Carnegie Mellon


    From: arms@cs.UAlberta.CA (Bill Armstrong)

    My point
    was that BP nets use continuous signals and the brain doesn't -- an
    obvious very significant difference.  I was asking why people would
    expect to understand the brain by studying a system (BP) that is
    *different* at the most basic level of signalling.
    
Well, your basic premise here is far from obvious and (in my
never-very-humble opinion) probably wrong.  Yes, neurons seem to do most of
their long-distance signalling with discrete pulses, though as others have
pointed out, there are also local connections that seem to use continuous
analog signals of an electrical or chemical nature.  However, even if we
concentrate on the pulse-coded communication, it appears that most of the
information is being carried by pulse-rate encoding -- an analog signal
that happens to be encoded as a train of pulses rather than a DC level.  By
integrating these pulse trains, you get the old familiar
sum-of-weighted-inputs used by BP and related models.  Look for concurrency
in the pulse trains (or introduce some other nonlinearity before the
summation) and you get a quick and dirty probabilistic multiplication of
two input signals.

There may well be some insight to be gained by modeling these systems paths
at the pulse, ion, and time-constant level, but if you want to move to a
more abstract representation, something like BP (or a related continuous
model, like sigma-pi units) seems a much better fit than Boolean logic.
Boolean logic nets fall out of BP as a special case, but I've seen no
reason to believe that the brain uses such special cases exclusively, or
even favors them.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: bill@nsma.arizona.edu (Bill Skaggs)
Newsgroups: comp.ai.neural-nets
Subject: Re: activation dynamics in neurons
Date: 25 Jul 92 02:11:25 GMT
Organization: ARL Division of Neural Systems, Memory and Aging, University of
	Arizona

santulli@europa.informatik.uni-dortmund.de (Marco Santulli) writes:

   >I'm looking for a detailed description of the activation
   >dynamics of the real neuron.
   >
   >Are they equal for every neuron, or is there a differenciation
   >in the various functions of neurons with different acitvities??
   >
   >Please post answers or mail them to: [ . . . ]

This is too big a question to be answered by email or a post.  Look at
Chapter 8 of "Principles of Neural Science" by Kandel, Schwartz, and
Jessel; perhaps the preceding chapters as well.  This will lead you to
more detailed treatments if you want them.

(Oh:  the activation dynamics are *not* the same for every neuron.)

	-- Bill
Newsgroups: comp.ai.neural-nets
From: ice@skynet.uucp (Ice)
Subject: Re: Rule extraction from NN
Organization: Industrial Cybernetics
Date: Tue, 28 Jul 1992 16:25:41 GMT

Hybrid Systems -> Oil and Water

Newsgroups: comp.ai.neural-nets
From: tedwards@src.umd.edu (Thomas Grant Edwards)
Subject: Re: Thanks Neural Nets, and Brains?
Organization: Systems Research Center, Maryversity of Uniland, College Park
Date: Tue, 28 Jul 1992 16:11:14 GMT

In article <675@trwacs.fp.trw.com> erwin@trwacs.fp.trw.com (Harry Erwin) writes:
>>I also believe work on chaos in man-made neural nets may also be a
>>useful tool for understanding chaos in real neural systems.

>Why?
>(What you're essentially saying is that the non-linearity of man-made
>neural nets is similar to the non-linearity of real neural systems.)

Work by Freeman (Neural Networks v.1 277-278) has shown chaotic 
activities in the
rabbit olfactory system.  Examination of this activity has indicated 
the use chaos as a novelty detector, and such dynamics have been
modeled on a computer. Further work
by Tsuda (Neural Networks, v. 5 pp 313-326) show the possibility of
chaos as a dynamic link for memory and memory search.  Of course,
we await the electro-neurophysiology to determine whether Tsuda's
model has any validity.

-Thomas Edwards
From: bill@nsma.arizona.edu (Bill Skaggs)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Nets and Brains
Date: 24 Jul 92 18:17:28 GMT
	<arms.711907358@spedden> <BILL.92Jul23135614@ca3.nsma.arizona.edu>
	<arms.711935064@spedden> <BILL.92Jul23224539@ca3.nsma.arizona.edu>
	<arms.711986585@spedden>
Organization: ARL Division of Neural Systems, Memory and Aging, University of
	Arizona

arms@cs.UAlberta.CA (Bill Armstrong) writes:

   >In sum: to study the brain, in my opinion, based on ignorance of
   >how the brain works, is that one should study a system that works
   >on logical signals.  Although ALNs might help in this, they are
   >not designed as a brain model at all, and different learning
   >algorithms would have to be developed to capture the learning
   >properties of neurons.
   >
   >Are we getting closer?

This seems entirely reasonable to me.

I would like to stress, though, that, while there are good reasons to
pay special attention to binary-neuron networks, there is no need to
*avoid* looking at scalar-neuron networks.  For some subsystems of the
brain, at certain levels of analysis, scalar neurons are actually
preferable.  The oculomotor system is the best example I know of:
when the variables of the system are taken to be firing rates and eye
position angles, the system is very nearly linear (David Robinson is
responsible for working this out).  This very important relationship
would probably have been completely missed had the neurons been
treated as binary units.

	-- Bill