HDTenEightyP/NetNews huggingface dataset:
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: neural nets and generalization (was Why not trees?)
Organization: University of Alberta, Edmonton, Canada
Date: Fri, 24 Jul 1992 15:01:00 GMT

paulking@next.com (Paul King) writes:

>Hidden units are interesting because they occasionally converge
>on an intermediate representation that highlights regularities
>in the behavior of the black box.  These intermediate states can
>then be used as inferred information about internal states of the
>black box, which can be useful as inputs to further networks.

>As an example, consider handwriting recognition.  Mapping input
>drawings to target characters might result in hidden-units that
>correlate with subfeatures in the input drawings.  This ability
>to map input drawings to subfeatures might then benefit a higher-level
>system that maps subfeatures to words, bypassing the character
>representation altogether.  One now has a network that can recognize
>words even if certain letters are incompletely written.

I agree that discovery of subfeatures is useful, but how much extra
interconnection is required -- complete connections to all elements in
the preceding layer?  If you cut down on the number of connections, you
cut the time of executing the net.  Go too far -- and you eliminate the
possibility of finding subfeatures that are shared.

Maybe the only remaining question is how many connections should be set
up initially to do this optimally.

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets,bionet.neuroscience
From: icann@mbfys.kun.nl (Icann '93)
Subject: Icann '93, Announcement and call for papers
Keywords: Neural Networks, conference
Organization: University of Nijmegen
Date: Thu, 30 Jul 1992 09:48:09 GMT


                              ICANN'93
        International Conference on Artificial Neural Networks
                 Amsterdam, September 13-16, 1993

              From Neurobiology to Real World Computing

PRINCIPLES FROM NEUROBIOLOGY
Dynamics of single neurons: Their information processing capabilities
Somato-sensory maps: Self-organization and unsupervised learning
The visual cortex: Massive parallel processing on structured data
The oculomotor system: From sensors to actuators
The hippocampus: An associative memory 

PHYSICAL AND MATHEMATICAL THEORIES
Information theory and statistical inference * Attractor neural networks
* Learning and generalization * Dynamical systems * Selforganizing systems *
Stochastic processes * Novel architectures and learning rules 

COGNITIVE CONNECTIONISM
From neuronic to symbolic representation * Psychological modeling *
Natural language processing 

ROBOTICS
Robot vision * Sensor fusion * Autonomous vehicles * Control of redundant 
manipulators

APPLICATIONS OF NEURAL NETWORKS
Speech recognition * Pattern recognition * Economic modeling and time series 
prediction * Process control * Industrial applications * Software * Hardware

                Plenaries from international authorities 
                Invited papers from leading specialists 
                Selected oral presentations and posters  
                Tutorials 
                Scientific and technical exhibition

                        FIRST CALL FOR PAPERS
                 Submission deadline February 1 1993

PROGRAM COMMITTEE:
Abbott (USA) * Aertzen (Ger) * Aleksander (UK) * Amari (J) * Eckmiller (Ger) 
* Epping (NL) * Fogelman-Soulie (F) * Gielen (NL) * Goser (Ger) * Grossberg 
(USA) * Hertz (DK) * Hinton (CAN) * Hopfield (USA)* Johannesma (NL) *
Jordan (USA) * Kappen (NL) * Kawato (J) * Koenderink (NL) * Kohonen (Fin) * 
Lautrup (DK) *  Martinetz (Ger) * von Seelen (Ger) * Ritter (Ger) * Sporns 
(USA) * Sompolinsky (Isr) * Strube (Ger) * Taylor (UK) * Toulouse (F) * 
Treleaven (UK) * Treves (UK) 

    Organizers: Bert Kappen, Stan Gielen (University of Nijmegen)

    TO RECEIVE FUTURE MAILINGS AND REGISTRATION FORMS, PLEASE MAIL
    YOUR NAME AND ADDRESS TO THE ICANN'93 SECRETARIAT, NOVEP,
    PAULUS POTTERSTRAAT 44, 1071 DB AMSTERDAM, THE NETHERLANDS.
    TEL. 31-20-6714814, FAX 31-20-6628136, E-MAIL icann@mbfys.kun.nl
From: arms@cs.UAlberta.CA (Bill Armstrong)
Newsgroups: comp.ai.neural-nets
Subject: Re: Thanks Neural Nets, and Brains?
Date: 23 Jul 92 16:02:38 GMT
Organization: University of Alberta, Edmonton, Canada

bdbryan@eng.clemson.edu (Ben Bryant) writes:

>In article 57397@cc.usu.edu, sl859@cc.usu.edu () writes:

>> 	I do have an additional question.  How extensive has the research
>>been on neurons of a neural net and the actual neurons of the brain?  I 
>>am interested mostly out of curiosity, but from what I've read I can only 
>>deduct that we still haven't got a very good model of the actual workings 
>>of the brain neurons, let alone how to model them with computers.

>  ... one scientist (I can't remember his name) said that he would
> one day like to be able to follow the path of firing responses through
>the brain using some MAJOR HEAVY DUTY processing and some HYPER-SENSITIVE
>electrical sensors. -I believe when that time comes, this the field of
>artificial life (Data from Star Trek?) will appear. That was an article in
>one of the spring `92 issues of PS.

First off, isn't it rather strange that the most widespread artificial
model of neural operation. the multilayer perceptron, uses continuous
quantities on its connections, while the dendrites and axons of
neurons use "zero or one" type action potentials?  Until physiological
psychologists start studying adaptive logic networks, can anyone
expect much progress on understanding the brain?

What use the the MAJOR HEAVY DUTY processing if there is no model to
help understand the operations?  Imagine trying to understand an
operating system by looking at the flows of bits in a computer system.
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Neural Nets and Brains
Organization: University of Alberta, Edmonton, Canada
Date: Sat, 25 Jul 1992 14:04:10 GMT

honavar@iastate.edu (Vasant G Honavar) writes:

>In article <arms.711986585@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>>
>>My point
>>was that BP nets use continuous signals and the brain doesn't -- an
>>obvious very significant difference.  I was asking why people would
>>expect to understand the brain by studying a system (BP) that is
>>*different* at the most basic level of signalling.
>>
>>Don't you agree that if the brain works on 0-1 signals, then to study
>>the brain one could beneficially look at logical systems? 


>But I would like to point out that it is far from clear that the brain
>works entirely on 0-1 signals.

I didn't mean entirely.  Obviously, there are packets of
neurotransmitter being released that could have a more continous and
cumulative effect.  Not only that, but the synapse possibly acts as a
contiunously-variable delay unit too.

I shall try to chase down what I have heard about action potentials
not only on axons but on dendrites.  Since I don't study the brain or
neurons, I have to depend on what others say.

Aside from brain modelling using purely boolean signals, there are
other potential uses of ALNs un studying the brain; namely explaining
what is happening in nerve cell collections.  If the axons have action
potentials, then it would still be possible, even if there are
continuous signals at other places in the system, to study the
relationships among axon firings using boolean methods.  One would
have to relate the firing of axon A to firings of others B, C, ... at
previous time instants.  In this way, one would factor out the
continuous parts of the operations and concentrate on the functional
modelling just based on boolean operations and time delays.

Does anyone know if this has been tried?
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: tedwards@src.umd.edu (Thomas Grant Edwards)
Subject: Re: Neural Nets and Brains
Organization: Systems Research Center, Maryversity of Uniland, College Park
Date: Tue, 28 Jul 1992 16:32:06 GMT

In article <arms.712027061@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>Thank you.  A nice explanation.  Unfortunately, taking phase and
>density of pulses into account makes for quite a complex, non-linear
>system (refractory periods, neurotransmitter release and uptake,...).
>If the brain works that way, then the simple multiply-add
>accumulations of backprop are over-simplified, aren't they?  Then
>maybe both backprop and ALNs are useless for the purposes of brain
>modelling.

Yes!

But they are (more to the point, were) useful in showcasing that
one can algorithmically string together parallel computational
units to produce a specific behaviour.  Certainly brain does not use
backprop or ALNs specifically, but the mathematical insight we gain
from them helps us to understand what's going on. 

-Thomas Edwards


Newsgroups: comp.ai.neural-nets
From: piccolbo@ghost.dsi.unimi.it (antonio piccolboni)
Subject: Re: What's in a layer ? (was: dumb question on layer enumeration)
Keywords: layer counting, neuron, connection
Organization: Computer Science Dep. - Milan University
Date: Fri, 24 Jul 1992 13:10:35 GMT

 
 osherson@uni2a.unige.ch writes:
 
 >In general, there are three kinds of connections in layered neural networks:
 
 >1.  Interlayer connections: connecting neurons from adjacent layers,
 >2.  Intralayer connections: connecting neurons within the same layer
 >    (including self-connections), and
 >3.  Supralayer connections: connecting neurons from neither the same, nor
 >    adjacent layers; i.e. these connections "skip" at least one layer.
 
 I'd like to shift your attention to a more compelling problem (for me):
 can we train a neural net with connections of the third type by means
 of standard back-propagation? In our experience this one works as far as
 the connection relation ( (a,b) belong to R iff the weight of connection
 from a to b is different from 0) is a directed acyclic graph.
 Is there a formal proof of this? (I think it would consist in proving 
 that the partial derivatives of error with respect to connections weight
 are spatially and timely local) Can we weaken the hypotesys allowing 
 recurrent connections? I know there are particular results (see Frasconi, 
 Gori, Soda: Local feedback multilayered networks, in Neural Computation
 vol 4, 1992 pp.120-130), but what I'm looking for are necessary and sufficient
 conditions for the gradient descent algorithm to be spatially and timely local.
 
 >P.S. My paper entitled "Neural Network Formalization" in the neuroprose
 >     (anonymous ftp 128.146.8.52; pub/neuroprose/fiesler.formalization.ps.Z)
 >     relates to this.
 
 Does this book relate to my problem too? Where can I look?
 Thank you in advance (promise of a summary and apologies for my english
 are too obvious to say)
 
 				Antonio Piccolboni
 


Newsgroups: comp.ai.neural-nets
From: vrkaramc@mtu.edu (Karamchedu)
Subject: Re: Carver Meads VLSI CAD tools
Organization: Michigan Technological University
Date: Mon, 27 Jul 1992 17:34:54 GMT


PLEASE SEND REPLIES TO MY EMAIL ADDRESS:  vrkaramc@mtu.edu

 

I was wondering if any of you people can initiate me into the immensely 
interesting field of Analog VLSI and NEURAL systems ( of course, apart from
suggesting that I should read Carver Mead's text).

Especially I would appreciate if you would enlighten me on the following:


* What kind of behavioral description language is this ?

* I am sure it is different from the conventionals like VHDL or OCTtools. But 
  how different ?  One difference I know is that I know something about the 
  former two but nothing about the Analog VLSI. 

* Any references on the language using which we can obtain the physical 
  layout ? 

* I am working on a particular architecture of Neural Net (Supervised learning)
  and I am in the process of overcoming some of the difficulties in realization
  of the circuit in discrete chip configuration. I have an immanent feeling that  what I should do is to look into this Analog VLSI and Neural CAD tool.
  The combination of the words ANALOG , VLSI and NEURAL somehow seems to me
  to answer many of the variations of the doubts. 

* No doubt, there's a vast reservoir of thought and endless possibilities in
  the Analog VLSI vis a vis Neural Architectures.

* PLEASE REPLY TO MY EMAIL ADDRESS. I AM NOT SURE I WOULD BE SPENDING MUCH TIME
  ON THE USENET. IT WOULD BE PREFERABLE FOR ME TO JUST READ EVERYTHING OFF 
  MY MAILBOX.

Thank you 

Karamchedu Raj 
Dept. of Electrical Engineering.
  
Newsgroups: comp.ai.neural-nets
From: len@retina.mqcs.mq.oz.au (Len Hamey)
Subject: Re: Why not trees?(was dumb question on layer enumeration)
Organization: Macquarie University, School of Mathematics, Physics, Computing and Electronics
Date: Tue, 21 Jul 1992 04:40:06 GMT

In article <arms.711643374@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>If you trace backwards from any single output node in a feedforward
>net with any kind of layer structure, you can construct a tree of
>nodes that will perform exactly the same computation of that output.
>Some nodes may have to be duplicated in this process.  Now suppose
>that were the hardware structure you had chosen in the first place.
>Just as a "man cannot serve two masters", wouldn't the learning task
>be easier for this tree than for the original net, where some nodes
>are shared between outputs, and have to provide outputs that satisfy
>two or more elements in net further on?  If so, then either the tree
>learns better, or can be made smaller (probably a good idea to avoid
>"overtraining").

Yu and Simmons have an interesting paper "Extra Output Biased Learning"
in which they show that providing additional outputs using the same
hidden layer can SPEED UP (!) learning by providing hints that the
hidden layer finds useful.  For example, it is quicker to teach a net
to solve parity and count the number of 1s in the input at the same time
than to simply teach it to solve parity.  This means that if there are
relationships between the outputs a shared hidden layer can learn more
quickly than separate hidden layers for each output.  Of course, one
could easily conceive of examples where shared hidden units have trouble:
when the outputs are totally unrelated to each other.

Yu and Simmons' paper is available in neuroprose.  yu.output-biased.ps.


			Len Hamey
			Lecturer in Computing
			Macquarie University
			NSW 2109 AUSTRALIA
			len@mpce.mq.edu.au
--
	Leonard G. C. Hamey (Len)	len@mpce.mq.edu.au
	Lecturer in Computing		(02)805-8978
	Macquarie University
	NSW 2109 AUSTRALIA
From: hns@ldv.e-technik.tu-muenchen.de (Hans Nikolaus Schaller)
Newsgroups: comp.ai.neural-nets
Subject: Preprint on NN design for combinatorial optimization
Keywords: NN, Hopfield, Optimization, Design Rule
Date: 26 Jul 92 15:56:07 GMT
Distribution: all
Organization: Leibniz-Rechenzentrum, Muenchen (Germany)

There is a preprint available of my paper:

  H. Nikolaus Schaller; A collection of constraint design rules for neural 
  optimization problems; in I. Aleksander, J. Taylor; Artificial Neural
  Networks II; Proc. ICANN'92; 4-7 September; Elsevier 1992

Abstract:
Neural Networks for combinatorial optimization problems are often designed by
trial and error. To aid the network engineer, a collection of simple building
blocks is presented. An example will show how to use this collection for
map-coloring problems.

To obtain a preprint, please mail me your postal address.

H. N. Schaller
hns@ldv.e-technik.tu-muenchen.de
From: sef@sef-pmax.slisp.cs.cmu.edu
Newsgroups: comp.ai.neural-nets
Subject: Re: What's in a layer ? (was: dumb question on layer enumeration)
Date: 24 Jul 92 14:37:26 GMT
Article-I.D.: cs.1992Jul24.143726.50351
Organization: School of Computer Science, Carnegie Mellon


    From: piccolbo@ghost.dsi.unimi.it (antonio piccolboni)
     
     I'd like to shift your attention to a more compelling problem (for me):
     can we train a neural net with connections of the third type by means
     of standard back-propagation? In our experience this one works as far as
     the connection relation ( (a,b) belong to R iff the weight of connection
     from a to b is different from 0) is a directed acyclic graph.
     Is there a formal proof of this? (I think it would consist in proving 
     that the partial derivatives of error with respect to connections weight
     are spatially and timely local).

It's unclear to me exactly what you're trying to prove, but the backprop
works just fine on networks with shortcuts, as long as there are
well-defined output units and some total ordering of the units such that
unit n does not depend on the values of any unit m, where m >= n.  The
usual derivation of the formula for the partials dE/dw does not assume any
kind of layered structure.

By the way, non-zero weights are not required either.  A zero weight will,
in general, evolve into something more useful.  What you don't want to do
is initialize *all* the weights to zero, or to any other common value.
This creates symmetries that, in the absence of noise, can't be broken, so
the net will get stuck.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
Newsgroups: comp.ai.neural-nets
From: mlevin@forte.cs.tufts.edu
Subject: how to get neural net to answer within a set of legal answers?
Organization: Tufts University
Distribution: usa
Date: Thu, 23 Jul 1992 18:23:08 GMT


     Suppose I want to use a neural net to inductively learn the
relationship between some inputs and outputs. Suppose also that the
outputs I want from the net have to be members of a certain set,
specified with a syntactic rule. How do I get the net to output only
"legal" guesses?  More specifically, suppose I want the net to learn a
relationship between sets of real numbers and strings; however, the
strings that it offers up as its guesses need to be "legal" strings -
they need to conform to a certain syntax. How in general do people
working with neural nets get them to only output legal (relative to
some domain) guesses? In case anyone can help me, I want the net to
produce outputs of strings S, where each S is recursively defined as a
real number, the letter "z", or the string "(f S' S'')" where f is one
of a small set of function names, and S' and S'' are other strings
(which may themselves be complex strings etc.). Thus, "7" is a legal
string. So is "(+ (* 8 z) (- z 5.5))".  Basically, a legal string is
any math expression in variable z in infix (LISP-like) notation.  So,
what can I do to have it output only legal formulas? Please reply to
mlevin@presto.cs.tufts.edu. 

Mike Levin
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Learning what COULD be learned
Organization: University of Alberta, Edmonton, Canada
Date: Tue, 21 Jul 1992 22:55:21 GMT

dbk@aber.ac.uk (D B Kell) writes:

>In article <arms.711663417@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>>The impediment to learning of one output by others that are difficult
>>or impossible is closely related to the "Why not trees?" question I
>>am asking.  If you use trees, this harmful interaction can't occur.


>Please elaborate much more explicitly! **HOW** do the trees flag that
>some things are not learnable, others are.

If a tree isn't learning the required task, as shown by a lack of
further improvement, then you can double the size of the tree for that
particular output and try again.  Does this make it clear how the
technique of independent trees would be used?

Deciding whether a given function can be *learned* is a different
issue -- complicated considerably by the question "does it generalize
well?  After all, we are allowing contradictory training points as
well as pattern recognition problems with overlapping
class-conditional probabilities; so what does it mean to learn?
Certainly not just being 100% correct on the training data (and
"overtrained").  Any boolean function is realizable by an ALN; any
continuous function can be approximated to any desired degree of
precision without exhorbitant hardware cost.  Discontinuous functions
can also be produced.  However, it sounds like an intractable, even
ill-defined problem to flag things as "unlearnable".

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: qmdbms@gsusgi2.gsu.edu (Brian Schott)
Subject: WANTED cart-pole (plant) system to balance
Keywords: control system
Organization: Georgia State University
Date: Tue, 21 Jul 1992 11:17:49 GMT

To test controller theory we are presently using a computer simulation of
an inverted-pendulum system, otherwise known as a cart-pole system. The
simulation lacks friction and we wish to test our theory on a physical
system.

Does anyone know of a physical system which is available? 


-- 
Brian Schott/Decision Sciences Dept.             qmdbms@gsusgi2.gsu.edu
College of Business Administration                         404-651-4070
Georgia State University                  interests: approx. reasoning,
Atlanta, Georgia USA 30303-3083     (B=)       decision support systems
From: yuwaraj@ecf.toronto.edu (YUWARAJ Murugathas)
Newsgroups: comp.ai.neural-nets
Subject: ART implementation
Keywords: ART
Date: 23 Jul 92 22:33:39 GMT
Article-I.D.: ecf.Brv5C4.2pI
Organization: University of Toronto, Engineering Computing Facility

I saw the request for an ftp site where an implementation of ART model
(for nn's) could be found. I would verymuch appreciate if anyone 
who saw the reply for the posting ( or is aware of a site) could kindly
notify me ( via e-mail ). Thanks in advance...
P.S. I hope to evaluate the feasibility of using an ART to classify
EMG (Electro Myographic ) signals. This has been attempted before using
backprop .    

e-mail: yuwaraj@ecf.toronto.edu
Murugathas Yuwaraj (Thas)
Engineering Science (Electrical Option)
U of T, Toronto