HDTenEightyP/NetNews huggingface dataset:
From: mwtilden@watmath.waterloo.edu (Mark W. Tilden)
Subject: What are the six different types of human neurons?
Organization: University of Waterloo
Date: Wed, 8 Apr 1992 15:11:18 GMT


During a recent neural-networks lecture, it was pointed out that
comp-sci types only model one type of human neuron, where animal (human)
bodys contain no less than six different types.

I know I could look this up, but what is a net for?  Could someone
please post the names of these six, and, if possible, a short discription
of their physiological locations and function (so far as is known).

Just when I thought we were making progress in neural-nets, Ma Nature
throws us a 6-sided multiplier.  Typical.

Is all.


-- 
Mark Tilden: _-_-_-__--__--_      /(glitch!)  M.F.C.F Hardware Design Lab.
-_-___       |              \  /\/            U of Waterloo. Ont. Can, N2L-3G1
     |__-_-_-|               \/               (519)885-1211 ext. 2454 
"MY OPINIONS, YOU HEAR!? MINE! MINE! MINE! MINE! MINE! AH HAHAHAHAHAHAHAHAHA!!"
From: cb@fantod.xis.xerox.com (Christopher Bader)
Newsgroups: comp.ai.neural-nets
Subject: Re: Back-Propagation
Keywords: Back-Propagation
Date: 9 Apr 92 01:32:19 GMT
Article-I.D.: spectrum.1992Apr9.013219.11917
Organization: Xerox Imaging Systems, Inc.

In article <587@mmf1.ee.QueensU.CA> osmanh@mmf1.ee.QueensU.CA (Hossam Osman) writes:
>
>    I would appreciate if anyone can give answers or discussions to the following two 
>questions:
>
>    1) Does increasing the number of hidden units always improve the mean-square error
>       at the output of feed-forward nets trained with BP ? 
>       That is, the error will not increase after reaching a specific limit of hidden
>       units.
>
>    2) Do nonlinear networks always yield a smaller value (or at least the same value)
>       for the mean-square error when compared to linear nets.
>       That is, if linearity decreases the error, the network will always be capable
>       of working in the linear region of the utilized nonlinearity in order to 
>       decrease the error.
>
>    Again, I would appreciate any response.

1) On the training set, yes.  On an independent test set, no.

2) Yes, because linear nets are simply a special case of non-linear nets.
BUT linear nets have the *significant* advantage of having no local minima.
So they train faster (usually much faster) than non-linear nets (e.g. classic
backprop) and they always arrive at the "best" answer that they are capable of.

					-- Christopher Bader
From: akempka@luke.eecs.wsu.edu (kempka anthony aaron - CS500)
Newsgroups: comp.ai.neural-nets
Subject: Re: (multi-layer + shortcut connections) vs general feedforward net
Date: 13 Apr 92 18:05:16 GMT
Organization: Washington State University

In article <1992Apr13.082917.14846@aar.alcatel-alsthom.fr> moutarde@aar.alcatel-alsthom.fr (Fabien Moutarde) writes:
>
>In fact, I've been wondering myself for a while why most NN designers
>using backprop restrict themselves to the class of multi-layer nets,
>despite the fact that the general form of the algo deals with *any*
>feedforward net.
Me too.
>My guess was that introducing what you call the shortcut
>connections was avoided because it increases the total number of weights,
Quite possible.


>However I would expect that allowing many composed non-linearities
>might make the cost hypersurface (in weight-space) even more complex
>and increase the probability to fall in a local minimum much higher
>than the global minimum. Does anyone has any comment or reference about that?

Since I've had the same question and was unable to find a reference, I'm
studying this area as a chapter in itself in my Master's Thesis. I'm not
completely finished with my experimentation, but I'm comparing training
times and the net's power to generalize using 'standard' feedforward
nets and 'shortcut' connnections using a variety of activation functions
(ie. not just sigmoids).

The shortcut connections seem to add greater dimensionality to the
training set possibly making a hyperplane easier to fit.
ALL of my tests so far (some 2000 test cases) find the shorcut connections
to dramaticaly reduce traing times versus standard feedforward nets
with the same placement of nodes (ie. inputs, hidden layers, outputs).

I've noticed a slight drop in the net's power to generalize with the shortcut
connections using a sigmoid (a drop of .6%), however the other activation
functions I'm using actualy show an INCREASE in the power to generalize when
using shortcut conenctions! 

It was my original deductions that said it would be more likely that
the nets would generalize less because of a sloppy hyperplane fit
due to the higher dimensionality. Although I wasn't real surprised to
discover otherwise.

I need to conclude my experiments to truly verify this phenomena for
a variety of problem types. I'll try to keep tabs on the net news to 
see where this discussion goes. In the mean time, I'm letting my computer
crunch, crunch, crunch on test cases.


Sincerly,

Anthony A. Kempka
==============================================================================
WSU Cpt S. 105  Wrangler/Student Harasser
From: demers@cs.ucsd.edu (David DeMers)
Newsgroups: comp.ai.neural-nets
Subject: Re: Discussion : Input/Output number of NN
Keywords: Back propagation network
Date: 10 Apr 92 00:46:15 GMT
Organization: =CSE Dept., U.C. San Diego

In article <1992Apr9.165826.10034@magnus.acs.ohio-state.edu> tsou@magnus.acs.ohio-state.edu (Po-Yu Tsou) writes:
>Dear Netter:

>I have a question about the numbers of input/output neurons which would
>affect the success of the learning of neuron networks. That is, when
>the input number is LESS than the number of outputs, the Backprop network
>failed to learn such a set of patterns. My conjecture is that the problem
>could be the nonlinear mapping. But I am not sure.

>Can someone help me about this matter? I am willing to see the discussion 
>about it. Or, has any one met the same trouble. If so, please let me know.

When the input dimensionality is less than the output dimensionality,
the mapping cannot be onto.  So the network computes a function
from the input space (let's say of dimensionality n) to an
n-dimensional (or less) submanifold of the output space.  Such functions
are learnable, but it's easier to learn functions which are onto.
I don't have a proof of that last sentence, but it is consistent
with my experience.

For example, I had a problem learning a mapping from real euclidean
3-space to the 3-torus of angles of a robot manipulator.
Representation of the angles as a 6-vector (using sines &
cosines) meant I had a 3-d manifold embedded in 6-space.
Whereas using a representation as a 3-vector of the sine of
the half-angle resulted in representation of the output on
a 3-d manifold.  The latter was far, far easier to approximate
with a NN than the former. (there are a lot of other problems
such as the fact that the mapping is non-unique, but that
is solvable.  See my paper in NIPS 4 for a solution)


I find it helpful to think geometrically about the 
spaces in which the input and
output vectors live.  Assuming that you are trying to learn
a smooth mapping, one hopes that the data in each space is a smooth
manifold, with some non-linear relationship. Topological
properties of the mapping are important, such as location of
critical points, location of open neighborhoods in the
input space which map one-to-one and onto the output space
etc.  

Hope this helps a little.

Dave 

-- 
Dave DeMers			 ddemers@UCSD   demers@cs.ucsd.edu
Computer Science & Engineering	C-014		demers%cs@ucsd.bitnet
UC San Diego					...!ucsd!cs!demers
La Jolla, CA 92093-0114	(619) 534-0688, or -8187, FAX: (619) 534-7029
Newsgroups: comp.ai.neural-nets
From: kkt@philabs.philips.com (Kim-Kiat Tan)
Subject: Re: CMAC Network?
Organization: Philips Laboratories, Briarcliff, New York
Date: Fri, 10 Apr 1992 14:16:24 GMT

In article <1992Apr10.040041.28157@cam.compserv.utas.edu.au> dbenn@leven.appcomp.utas.edu.au (David Benn) writes:
>In article <1992Apr09.125205.206211@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu writes:
>>
>>
>>    Anyone ever hear of a CMAC network?  If anyone knows of sources of
>>    information on this please let me know.  Thanks.
>>    

	Look at the latest issue of IEEE Control Systems Magazine,it's a
special issue on NN.

Xref: sparky comp.ai.neural-nets:1848 bionet.neuroscience:193 bionet.info-theory:407
From: bill@NSMA.AriZonA.EdU (Bill Skaggs)
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
Subject: Re: What are the six different types of human neurons?
Date: 8 Apr 92 23:57:05 GMT
Reply-To: bill@NSMA.AriZonA.EdU (Bill Skaggs)
Organization: Center for Neural Systems, Memory, and Aging

In article <1992Apr8.151118.4780@watmath.waterloo.edu> 
mwtilden@watmath.waterloo.edu (Mark W. Tilden) writes:
>
>During a recent neural-networks lecture, it was pointed out that
>comp-sci types only model one type of human neuron, where animal (human)
>bodys contain no less than six different types.
>
>I know I could look this up, but what is a net for?  Could someone
>please post the names of these six, and, if possible, a short discription
>of their physiological locations and function (so far as is known).
>
  I doubt that you could look this up, because it's nonsense.
Animal bodies contain literally hundreds of different types of
neuron.  Even if you only distinguish on the basis of the
chemical neurotransmitter used, there are still at least several
dozen different types.  The functional consequences of these
differences have hardly begun to be explored by neural network
theorists.

	-- Bill
From: dld@magi.ncsl.nist.gov (Darrin Dimmick X4147)
Newsgroups: comp.ai.neural-nets
Subject: NIST SPECIAL DATABASE 2
Date: 15 Apr 92 14:53:24 GMT
Organization: National Institute of Standards and Technology, Gaithersburg, MD


 NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

 Announces a New Database

 +-----------------------------+
 | "NIST Special Database 2"   | 
 +-----------------------------+

 Structured Forms Reference Set
 (SFRS)

The NIST database of structured forms contains 5,590 full page images of 
simulated tax forms completed using machine print. THERE IS NO REAL TAX DATA IN
THIS DATABASE. The structured forms used in this database are 12 different 
forms from the 1988, IRS 1040 Package X. These include Forms 1040, 2106, 2441, 
4562, and 6251 together with Schedules A, B, C, D, E, F and SE. Eight of these 
forms contain two pages or form faces making a total of 20 form faces 
represented in the database. 

Each image is stored in bi-level black and white raster format. The images in 
this database appear to be real forms prepared by individuals but the images
have been automatically derived and synthesized using a computer and contain no
"real" tax data. The entry field values on the forms have been automatically
generated by a computer in order to make the data available without the danger 
of distributing privileged tax information.

In addition to the images the database includes 5,590 answer files, one for
each image. Each answer file contains an ASCII representation of the data found
in the entry fields on the corresponding image. Image format documentation and
example software are also provided.

The uncompressed database totals approximately 5.9 gigabytes of data.

"NIST Special Database 2" has the following features:
+ 5,590 full-page images
+ 5,590 answer files
+ 12 pixel per millimeter resolution
+ image format documentation and example software

Suitable for automated document processing system research and development, the
database can be used for:
+ algorithm development
+ system training and testing

The system requirements are a 5.25" CD-ROM drive with software to read ISO-9660
format.

If you have any further technical questions please contact:


Darrin L. Dimmick
dld@magi.ncsl.nist.gov
(301)975-4147

If you wish to order the database, please contact:

Standard Reference Data
National Institute of Standards and Technology
221/A323
Gaithersburg, MD 20899
(301)975-2208
(301)926-0416 (FAX)
From: morten@cogsci.ed.ac.uk (Morten Christiansen)
Newsgroups: comp.ai.neural-nets
Subject: Request for nn reference
Keywords: tanks
Date: 9 Apr 92 11:37:00 GMT
Organization: Centre for Cognitive Science, Edinburgh, UK

Hi netters,

I'm a Ph.D. student at the Centre for Cognitive Science at the
University of Edinburgh, Scotland. I am interested in knowledge
representation and neural networks. I'm presently working on a paper
addressing the problems inherent in the kind of correlational
semantics that implicitly underly most work within connectionism. 

In a recent TV program on AI - "Dream Machines" on BBC 2 (UK), 8/12-91
- professor Dreyfus mentioned a (for me) particularly illuminating nn
simulation. The simulation in question demonstrated that it is not a
trivial matter to get a network to pick out the right salient features
(from a human perspective) of a given domain. A network was trained to
recognize tanks hidden in a forest from photos taken from the air. The
network learned successfully to pick out the photos which had tanks on
them from the photos which had merely trees in them. It was also able
to generalize to photos that it had not seen previously during
learning.  To be certain that the network had really learned to
recognize tanks, a new set of photos was taken and presented to the
network. However, this time the network failed completely to
categorize the photos.  Apparently, the network had not learned to
recognize tanks but to distinguish photos taken on a sunny day from
those taken on a cloudy day. A closer examination of the photos used
to train the network showed that all the photos which had tanks in
them were taken on a sunny day, whereas the photos of trees only was
taken on a cloudy day.

The tank example is particularly interesting because the problems
encountered here parallels similar problems within animal concept
research. If anyone is familiar with this particular nn simulation (or
a similar example dealing with equally complex and uninterpreted input
data), I would be very grateful for receiving an eamil with the
reference in question.


			Sincerely,

				Morten Christiansen


Morten Hyllekvist Christiansen			| University of Edinburgh
Phone: +44 (0)31 650 4420			| Centre for Cognitive Science
ARPA:  morten%cogsci.ed.ac.uk@nsfnet-relay.ac.uk| 2 Buccleuch Place
JANET: morten@uk.ac.ed.cogsci                   | GB-Edinburgh EH8 9LW,Scotland



-- 
Morten Hyllekvist Christiansen			| University of Edinburgh
Phone: +44 (0)31 650 4420			| Centre for Cognitive Science
ARPA:  morten%cogsci.ed.ac.uk@nsfnet-relay.ac.uk| 2 Buccleuch Place
JANET: morten@uk.ac.ed.cogsci                   | GB-Edinburgh EH8 9LW,Scotland
Newsgroups: comp.ai.neural-nets
From: minsky@media.mit.edu (Marvin Minsky)
Subject: Re: Request for nn reference
Cc: minsky
Organization: MIT Media Laboratory
Date: Sat, 11 Apr 1992 00:16:53 GMT

In article <2515@keele.keele.ac.uk> cod14@seq1.keele.ac.uk (A.J. Schofield) writes:
>In article <1992Apr10.052002.179851@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu writes:
>> 
>> I think that this story has become the scientific equivalent of an urban
>> legend.
>> 
>I vaguely remember hearing the story about the tank detector from an 
>aquantance BEFORE it was mentioned on the BBC TV program `Dream Machine'. The
>sourse was definately second hand but my contact is based in the UK. I think
>this is posibly a legend but it does serve as a warning that NN have the
>potential to make perverse correlations between input data and desired 
>outputs.
>
>Regards Andrew Schofield

It happened in the mid-1960s at SRI. The machine was a very nice optical
perceptron -- that is, it used large number of optical masks with
photodetectors to make weighted sums of the input picture.  The
machine had achieved a high score and there was a party to celebrate
the event.  In the middle of the party the research director (Charlie
Rosen) came in to sadly announce that the two rolls of film, one with
the pictures of bushes and the other with tanks in the bushes, had
been developed with a large difference in density.  This was a great
research group with many important accomplishments, (including the
Isodata clustering procedure described in chapter 12 of
"Perceptrons").  The group soon moved its center of concern in the
direction of developing vision and planning for the robot Shaky.  I
think I was present at that party, but such is the nature of urban
legends that the story is more compelling than the actuality.  I mean,
the recollection is so striking that I seriously doubt I was actually
there -- but no question that it happened.  Probably Nillsson was
there, too.

  A similar true story around the same time: I was visiting another
early NN theorist, A. Gamba in Genoa, Italy, and his machine had just
learned to distinguish between a collection of scores of genuine Bach
Chorales and another collection written by students at the
conservatory there.  I examined the scores and noticed that there was
much more whitespace in the final measures of all the pictures of one
set.  They went off to check on whether this was what their NN had
discovered.  I never got a report about this, but the paper was never
published.
From: kb@jet.uk (ken blackler)
Newsgroups: comp.ai.neural-nets
Subject: Aspirin 5.0
Summary: Aspirin 5.0 in UK?
Keywords: Neural Nets Aspirin
Date: 13 Apr 92 17:10:02 GMT
Organization: Joint European Torus


If anyone has Aspirin/Migraines version 5.0 in the UK and is willing
to post it to me on a floppy disk(PC or MAC formats, which I can supply!).
Alternatively if it is available on a BBS then I would be very grateful for the 
telephone number.

(We don't have remote  FTP access :-( )

Thanks in advance,

		Ken Blackler
-- 
 ______________________________________   ____________   _____________
|Ken Blackler kb@jet.uk (+44)235 464743| | __________ | |   -Fusion-   |
|JET Joint Undertaking, Abingdon       | |   | |_ |   | |Energy for the|
|Oxfordshire, England. OX14 3EA        | | \_/ |_ |   | | (far) future |
- Disclaimer: Please note that the above is a personal view and should not 
  be construed as an official comment from the JET project.
From: dow@idtg.UUCP (Keith Dow)
Newsgroups: comp.ai.neural-nets
Subject: Questions
Date: 13 Apr 92 23:11:54 GMT
Organization: Integrated Device Technology, Santa Clara


I have a few questions.

1.  What is a good journal to publish results in?

2.  Since a single hidden layer should suffice for all problems,
    why include more?
From: minsky@media.mit.edu (Marvin Minsky)
Subject: Re: Hidden Layers, How many?
Cc: minsky
Organization: MIT Media Laboratory
Date: Wed, 15 Apr 1992 22:45:48 GMT

In article <1992Apr15.213448.20124@elroy.jpl.nasa.gov> sgrenander@nasamail.jpl.nasa.gov (Sven Grenander) writes:
>The following message seems to have gottent truncated. As the questions are
>probably of more than passing interest for "NN-engineers" I would appreciate
>it if you would care to complete what was lost/cut  off.
>
>In article <1992Apr15.145721.10666@news.media.mit.edu>, minsky@media.mit.edu (Marvin Minsky) writes:
>> 

Umm, my "add blank line" patch vanished again,  Hail UNIX!  I think I
simply concluded that Fahlman has pointed out gains in flexibility you get by
including some layer-skipping connections.  The point of the mailing
was, of course, to try to get some more theory of classes of learning
problems.  For example, I keep seeing references to various
"successes" at learning XOR distinctions -- but I never see any (even
empirical) descriptions of learning-time vs. number of variables vs.
numbers of layers and units.  Come on, folks, surely we could be
learning a lot more from all this activity!!!!!

!This line might be erased!

Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: CMAC Network?
Date: Thu, 09 Apr 92 12:52:05 GMT
Organization: School of Computer Science, Carnegie Mellon


    Anyone ever hear of a CMAC network?  If anyone knows of sources of
    information on this please let me know.  Thanks.
    
It's a sort of table-lookup or nearest-neighbors algorithm with
interpolation that has proven to be useful for a number of robotics and
other control problems.  Look for papers by James Albus some years back,
and other work that references these.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: nlonginow@falcon.aamrl.wpafb.af.mil
Newsgroups: comp.ai.neural-nets
Subject: Re: Feature selection of NNs (Take two)
Date: 8 Apr 92 07:31:37 EST
Organization: USAF AL/CFH, WPAFB, Dayton, OH

In article <1992Apr8.053029.15730@noose.ecn.purdue.edu>, muttiah@stable.ecn.purdue.edu (Ranjan S Muttiah) writes:
> Let me restate again what I had originally intended.  Given any n-features
> that have varying degrees of "importance" (and features that could also be 
> context sensitive), how do I cluster them onto an output map that reflects their 
> ... 

I must have missed the original posting of this. Dennis Ruck (Air Force Inst
of Techn) did an excellent paper called "Feature Selection Using a Multilayer
Perceptron", where he rank ordered input features according to their effect
on the learning of the net. Simply put, the relative importance of a given
feature is computed according to the partial derivative of an output feature
with respect to an input feature. This sensitivity analysis yields a good
measure of the relative importance of a set of features, although it says
nothing about their goodness in a more general sense. 

I dont know where the paper was published. Let me know if you need more and
maybe I can scare up the exact ref.

-- 
Nick Longinow
Armstrong Lab
Human Engineering division


"The opinions and views expressed here are strictly my own and do not 
necessarily reflect the official position of either the U.S. Air Force 
or its contractors. "  You should think otherwise ?
From: robert@cs.strath.ac.uk (Robert B Lambert)
Newsgroups: comp.ai.neural-nets
Subject: ftp sites for hand-written characters
Date: 8 Apr 92 08:39:22 GMT
Organization: Comp. Sci. Dept., Strathclyde Univ., Glasgow, Scotland.

Hi.

I am looking for hand-written characters for evaluating the performance of a 
number of network architectures. Are their any sites which have a collection
of hand-written characters obtainable by ftp, and if so how do I gain access.
The size of the individual images are not important.

I will post a summary of sites (if any exist) to this news group.

Thanks in advance,   Robert.

###############################################################################
#  Robert B Lambert           #       E-Mail : robert@cs.strath.ac.uk         #
#  Dept. Computer Science,    #################################################
#  University of Strathclyde, #      Life always gives us an alternative,     #
#  Glasgow, Scotland.         #         there is always the exception!        #
###############################################################################
From: saparici@crc.ac.uk (Dr. S.A.J.R. Aparicio)
Newsgroups: comp.ai.neural-nets
Subject: human neurons
Date: 11 Apr 92 11:01:00 GMT
Reply-To: saparici@crc.ac.uk (Dr. S.A.J.R. Aparicio)
Organization: MRC Human Genome Mapping Project Resource Centre, Harrow, UK


 
Keywords: 


Any standard neurophysiology textbook will contain descriptions of neuronal types.
If you want a text with more connectionist connections have a look in:

"Neural Darwinism" GM Edleman, Basic Books, although you might not agree with everything, the biolgy is
well presented and accurate.

Sam Aparicio
Newsgroups: comp.ai.neural-nets