HDTenEightyP/NetNews huggingface dataset:
From: lordSnooty@cup.portal.com (Andrew - Palfreyman)
Newsgroups: comp.ai.neural-nets
Subject: Re: CMAC Network?
Date: Sun, 12 Apr 92 15:59:31 PDT
Organization: The Portal System (TM)
Distribution: world
  <1992Apr10.040041.28157@cam.compserv.utas.edu.au>
  <1992Apr10.141624.2691@philabs.philips.com>

In article <1992Apr10.040041.28157@cam.compserv.utas.edu.au> dbenn@leven.appcom
p.utas.edu.au (David Benn) writes:
>In article <1992Apr09.125205.206211@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu
 writes:
>>
>>
>>    Anyone ever hear of a CMAC network?  If anyone knows of sources of
>>    information on this please let me know.  Thanks.
>>    
:
:	Look at the latest issue of IEEE Control Systems Magazine,it's a
:special issue on NN.

Check also the article on CMAC in this month's Neural Networks mag.
 --------------------------------------------------------------------------
| lord snooty @the giant | wolfpack, silver, down to the water             |
| poisoned electric head |              andrew palfreyman@cup.portal.com   |
 --------------------------------------------------------------------------
From: kavuri@lips.ecn.purdue.edu (Surya N Kavuri )
Newsgroups: comp.ai.neural-nets
Subject: Re: Back-Propagation
Date: 9 Apr 92 17:39:30 GMT
Organization: Purdue University Engineering Computer Network

In article <1992Apr9.072110.14997@Silence.pi.nctu.edu.tw>, gcp79527@csunix.csie.nctu.edu.tw writes:
> 
>     Increasing the number of hidden units DOES NOT always improve the MSE.
> Below a threshold number(not known exactly in general), MSE will be improved
> if more HUs are added. But given too many HUs (the same as giving too many 
> FREE variables), the nets will be hard to construct a proper mapping between
> input and output.
> 
> Yo-Yau Lin.

  As long as the residuals have an underlying structure (ie., not just white noise)
 shouldn't the addition of extra nodes (say by fixing the previous ones) be useful 
 in fitting the residuals ? 

 On the question of comparison with linear models, neural nets always need not do  better.
 Try fitting a straight line with a network and compare to a linear model.  
 As they say in my home town : "Don't need a Porsche to get your groceries" ;-)


Surya Kavuri
(FIAT LUX)
From: pck@castle.ed.ac.uk (P C Knox)
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
Subject: Re: What are the six different types of human neurons?
Date: 13 Apr 92 13:33:24 GMT
Organization: Edinburgh University

bill@NSMA.AriZonA.EdU (Bill Skaggs) writes:

>In article <1992Apr8.151118.4780@watmath.waterloo.edu> 
>mwtilden@watmath.waterloo.edu (Mark W. Tilden) writes:
>>
>>During a recent neural-networks lecture, it was pointed out that
>>comp-sci types only model one type of human neuron, where animal (human)
>>bodys contain no less than six different types.

>  I doubt that you could look this up, because it's nonsense.
>Animal bodies contain literally hundreds of different types of
>neuron.  Even if you only distinguish on the basis of the
>chemical neurotransmitter used, there are still at least several
>dozen different types.  The functional consequences of these
>differences have hardly begun to be explored by neural network
>theorists.

>	-- Bill
Interestingly I'm sure there was an article I read in a recent TINS
suggesting that there was a lot to be gained to setting up a new branch
of Neuroscience concerning the taxonomy of neurons. This does indicate
the complexities of the subject though. As well as dealing with types of
neuron in vertebrate nervous system, it would deal with invertebrate
neurons.
	Mind you not withstanding all the discussion on this topic,
simple classifications have been atempted, such as classing neurons as
bipolar etc on the basis of structure. The problem is that at the end of
the day such approaches surely amount to little more than what Lord
Rutherford would have called stamp collecting.

Paul C. Knox
Newsgroups: comp.ai.neural-nets
From: sgrenander@nasamail.jpl.nasa.gov (Sven Grenander)
Subject: Any BrainMaker H/W experiences ?
Organization: Jet Propulsion Laboratory
Date: Tue, 7 Apr 1992 16:13:00 GMT

I am currently using the BrainMaker/Mac beta version (S/W NN from California
Scientific Software) on a Mac IIfx. I've been very happy with the
results of this S/W implementation, but need faster training. I am
therefore considering (at $13K + IBM host, it takes a lot of considering !)
getting the BrainMaker IBM-card set which supposedly would give me a
factor of 40 speed-up over the stock Mac IIfx.

I'd be interested to hear what experiences, if any, there have been in
NN-land when using this H/W solution. In particular, does the 40-times
speed-up seem realistic, were there any problems, were there any un-
expected shortcomings ?

Any other suggestions ?

Thanks In Advance

-Sven
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Discussion : Input/Output number of NN
Date: Fri, 10 Apr 92 03:43:05 GMT
Organization: School of Computer Science, Carnegie Mellon

    From: demers@cs.ucsd.edu (David DeMers)
    
    When the input dimensionality is less than the output dimensionality,
    the mapping cannot be onto.  So the network computes a function
    from the input space (let's say of dimensionality n) to an
    n-dimensional (or less) submanifold of the output space.  Such functions
    are learnable, but it's easier to learn functions which are onto.
    I don't have a proof of that last sentence, but it is consistent
    with my experience.
    
Well, that's one way to look at it.  Another way is this: A net with, say,
3 inputs, 6 outputs, and n hidden units can be replaced by six independent
nets with 3 inputs and 1 output.  Training the 3-??-6 net should therefore
be no worse than six times the cost of training the most difficult of the
3-??-1 subnets.  It shouldn't be exponentially harder, or anything like
that.

In fact, you would often find that the six outputs can share internal
features, so producing all six outputs might only be a little bit harder
than training the single hardest output.

I will agree that its easier to come up with some really obscure, tangled,
hard to learn output encoding in 6-space than in 3-space.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu

Newsgroups: comp.ai.neural-nets
From: mdg@magi.ncsl.nist.gov (Mike Garris x2928)
Newsgroups: comp.ai.neural-nets
Subject: New NIST OCR Database
Date: 16 Apr 92 19:09:50 GMT
Organization: National Institute of Standards and Technology, Gaithersburg, MD



           NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

                      Announces a New Database

                  +-----------------------------+
                  |  "NIST Special Database 3"  |
                  +-----------------------------+

          Binary Images of Handwritten Segmented Characters
                              (HWSC)


The NIST database of handwritten segmented characters contains 313,389
isolated character images segmented from the 2,100 full-page images
distributed with "NIST Special Database 1". The database includes the
2,100 pages of binary, black and white, images of hand-printed numerals
and text. This significant new database contains 223,125 digits, 44,951
upper-case, and 45,313 lower-case character images. Each character image
has been centered in a separate 128 by 128 pixel region and has been
assigned a classification which has been manually corrected so that the
error rate of the segmentation and assigned classification is less than
0.1%. The uncompressed database totals approximately 2.75 gigabytes of
image data and includes image format documentation and example software.

"NIST Special Database 3" has the following features:
     + 313,389 isolated character images including classifications
     + 223,125 digits, 44,951 upper-case, and 45,313 lower-case images
     + 2,100 full-page images
     + 12 pixel per millimeter resolution
     + image format documentation and example software

Suitable for automated hand-print recognition research, the database
can be used for:
     + algorithm development
     + system training and testing

The database is a valuable tool for training recognition systems on a
large statistical sample of hand-printed characters. The system
requirements are a 5.25" CD-ROM drive with software to read ISO-9660
format.

If you have any further technical questions please contact:

                         Michael D. Garris
                       mdg@magi.ncsl.nist.gov
                           (301)975-2928

If you wish to order the database, please contact:

                      Standard Reference Data
         National Institute of Standards and Technology
                             221/A323
                      Gaithersburg, MD 20899
                          (301)975-2208
                       (301)926-0416 (FAX)
Newsgroups: comp.ai.neural-nets
From: jdean@galaxy.afit.af.mil (Jeffrey S. Dean)
Subject: Auditory system modelling
Summary: Looking for references on modelling auditory system in NN
Keywords: modelling, auditory, neural nets
Date: Wed, 08 Apr 92 21:18:28 GMT
Organization: Air Force Institute of Technology

I am currently working on my Master's, and would like to try using a NN
to cancel out background noise on an audio signal.  Does anybody have
a good reference on a model of how the biological auditory system performs
this task?  What would be a good NN system to try for this?  (Need a method 
that has not already been overworked and studied)  Any suggestions (aside 
from those painful or physically improbable) will be appreciated.

Jeff Dean

Disclaimer: I work for the Air Force, but I don't speak for it.  My opinions
are my own (or my wife's when she gives them to me).

Newsgroups: comp.ai.neural-nets
From: wuzhibia@iscs.nus.sg
Subject: IJCNN92 student financial surport
Reply-To: WUZHIBIA@NUSDISCS.BITNET
Organization: Dept. of Information Systems and Computer Science, National University of Singapore
Date: Mon, 6 Apr 1992 06:49:08 GMT

I was told that there is a financial surport program for those
Ph.D students whose papers have been accepted by IJCNN92. I don't
know how to apply this kind of surport? Any information will
be appreciated.

Thanks in advance.

Please reply to wuzhibia@iscs.nus.sg

best
zhibiao
Newsgroups: comp.ai.neural-nets
From: c-mt@uglab2.math.utah.edu (Thomas Murphy)
Subject: ai research on symbolic math 
Date: Fri, 10 Apr 1992 20:46:38 GMT
Organization: University of Utah, Dept. of Math
Reply-To: c-mt@math.utah.edu

Greetings,

  AI Expert is interested in doing an article on AI research using
symbolic math packages.  If you have interest and experience please
send me e-mail.

thanks,
murph

-- 
Thomas Murphy           Product Review/Tech Editor        
c-mt@math.utah.edu      Computer Language and AI Expert
(801)581-3905           Miller Freeman Publishing
From: sef@sef-pmax.slisp.cs.cmu.edu
Newsgroups: comp.ai.neural-nets
Subject: Re: Hidden Layers, How many?
Date: 10 Apr 92 01:22:10 GMT
Article-I.D.: cs.1992Apr10.012210.62971
Organization: School of Computer Science, Carnegie Mellon


    From: sgrenander@nasamail.jpl.nasa.gov (Sven Grenander)
    
    I've gotten a number of reminders that I neglected to state the rule of thumb.
    It is almost embarrasingly simple: the average of the number of input and output
    neurons. 
    
I suppose if you're going to make a totally blind guess as to what topology
to use, this is no worse than any other.  At least it will produce
nice-looking diagrams in your paper.  But this rule is wrong most of the
time.  As one counter-example, consider the two-spirals problem: two analog
inputs, one binary output, and ten or fifteen hidden units required, even
if you allow shortcut connections.  Or try doing parity on a 64-32-1
network.

A surprisingly large number of real-world classification problems turn out
to be linearly separable, or nearly so.  So my favorite blind guess is a
net with no hidden units at all, just direct input-output connections.  It
has the advantage of being quick to train.  (And if you find that it doesn't
solve the problem, you can start adding hidden units one at a time...)

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: sef@sef-pmax.slisp.cs.cmu.edu
Newsgroups: comp.ai.neural-nets
Subject: Re: Discussion : Input/Output number of NN
Date: 9 Apr 92 19:28:04 GMT
Organization: School of Computer Science, Carnegie Mellon


    From: tsou@magnus.acs.ohio-state.edu (Po-Yu Tsou)
    I have a question about the numbers of input/output neurons which would
    affect the success of the learning of neuron networks. That is, when
    the input number is LESS than the number of outputs, the Backprop network
    failed to learn such a set of patterns. My conjecture is that the problem
    could be the nonlinear mapping. But I am not sure.
    
Your problem might be many things, but I don't think that it creates any
special problems to have more outputs than inputs.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: francescottp@bvc.edu
Newsgroups: comp.ai.neural-nets
Subject: exit
Date: 10 Apr 92 18:38:17 GMT
Organization: Buena Vista College, Storm Lake, IA


For anyone interested...Larry Punteney at KTIV in Sioux City is looking for 
a couple unpid interns in their sports department this summer.  Anyone 
interested should call Larry at 1-800-234-KTIV.
Newsgroups: comp.ai.neural-nets
From: jwilliam@stsci.EDU (John Williams)
Subject: NN as a Temperature Controller
Keywords: Controller
Reply-To: jwilliam@stsci.EDU (John Williams)
Organization: Space Telescope Science Institute
Date: Wed, 8 Apr 1992 19:55:49 GMT

--
I'm thinking of using an NN as part of a temperature/humidity controller.  Has anyone else
tried this?  I'm interested in hearing what kind of network people think would be most 
appropriate.  I have been thinking of using a counterpropagation feedforward network.
I understand that others have used the conjugate gradient algorithm in conjunction with CPN
for shorter learning times.  Another thought was to try the Kohonen self-organizing maps.

Issues raised include:  How adaptable is the network when a new output is needed?  It is not just a
matter of stabilizing on an output based on varying inputs.  The desired output may also change.
Will the network have to be retrained when desired output conditions or available inputs vary?  Any thoughts on the subject would be welcome.

***********************************************************************
*John Williams - jwilliam@stsci.edu      Happiness is Earth in your      *
*Space Telescope Science Institute       rear view mirror.                      *
***********************************************************************
Newsgroups: comp.ai.neural-nets
Subject: Re: (multi-layer + shortcut connections) vs
From: massimo@cui.unige.ch (DEFRANCESCO Massimo)
Date: 16 Apr 92 00:05:37 +0200
Reply-To: massimo@cui.unige.ch
Distribution: world
Organization: University of Geneva

In article 260661@cs.cmu.edu, sef@sef-pmax.slisp.cs.cmu.edu () writes:
>
>    From: moutarde@aar.alcatel-alsthom.fr (Fabien Moutarde)
>    
>
>    In fact, I've been wondering myself for a while why most NN designers
>    using backprop restrict themselves to the class of multi-layer nets,
>    despite the fact that the general form of the algo deals with *any*
>    feedforward net.
>
>Just tradition, I think.
>

There is an important reason why multi-layer networks are more interesting
than general feedforward networks. I would call it `composability' or
`modularity'.
I'm actually working on a framework (and a simulator) for `first class
neural networks', that is an environment in which neural networks are
first class citizens with functional behaviour:

	1. they can be passed as parameters to functions, 
	   assigned to variables, etc.
	2. they can be returned by functions
	3. They can be `composed'

Actually the best example so far is that of multi-layer networks.
The basic layered network is the `functional layer'. Structurally
it consists of a set of input sites, connections, and a set of 
regular output units. Functionally they can be applied to a set of
(input) values and return the set of values corresponding to a forward
pass in a regular network.
A multi layer network is nothing else than the composition of functional
layers (there is more than just `sequential composition').
Composition takes care also of the learning behaviour. At least 
Backpropagation can be defined locally on a functional layer and
the learning behaviour of the composition is perfectly defined by the 
composition of the learning behaviours!
The advantage: modularity, and the possibility to define complex structures
(SexNet is one _trivial_ example) in a very easy way.

Of course, supra-layer connections (Hi Emile!) -or shortcuts- could speak
against composition and suggest a loss of generality of multi-layer
network WRT general feedforward neural networks. In fact, it is easy
to show that a general feedforward neural network can be transformed
into an `identical' multi-layer network, in the sense that the two 
networks trained with the same parameters, the `same' starting weights
and the same training set will result in two networks which will
give _exactly_ the same outputs for the same inputs.

I have written a research report that describes all that and more. 
I won't put it in neuroprose because it is still `work in progress',
is a condensate of all the work I've been doing during a year
and I'd like to make some papers out of it, but I'd be happy to
send it to anybody who asks me (specify hard-copy or Postcript email).


Massimo (massimo@cui.unige.ch)

---------------------------------------
Research and teaching assistant.
AI group, University of Geneva, Switzerland
From: yoshio@grip.cis.upenn.edu (Yoshio Yamamoto)
Newsgroups: comp.ai.neural-nets
Subject: Re: Backpropagation
Date: 9 Apr 92 22:09:18 GMT
Organization: GRASP Lab, University of Pennsylvania


In article <1992Apr9.173930.17404@noose.ecn.purdue.edu> you write:
>In article <1992Apr9.072110.14997@Silence.pi.nctu.edu.tw>, gcp79527@csunix.csie.nctu.edu.tw writes:
>> 
>>     Increasing the number of hidden units DOES NOT always improve the MSE.
>> Below a threshold number(not known exactly in general), MSE will be improved
>> if more HUs are added. But given too many HUs (the same as giving too many 
>> FREE variables), the nets will be hard to construct a proper mapping between
>> input and output.
>> 
>> Yo-Yau Lin.
>
>  As long as the residuals have an underlying structure (ie., not just white noise)
> shouldn't the addition of extra nodes (say by fixing the previous ones) be useful 
> in fitting the residuals ? 
>

Yes, you're right. Also if an additional mechanism is provided such that
multiple hidden units compete for higher correlation with residual errors
and the winner unit with the maximal correlation is chosen to be installed
in a preexisting network, then it would result in much more efficient
learning and a compact network in terms of the number of hidden units 
installed.

This is exactly what Scott Fahlman's Cascade Correlation Algorithm does. 

Even with his algorithm, however, MSE does not always decrease monotonically 
perhaps due to its approximate nature of Newton method.  But with an
appropriate choice of parameters and unit types, it almost always converges 
to a good solution in a reasonable amount of time.

>
>Surya Kavuri
>(FIAT LUX)


Yoshio Yamamoto

Newsgroups: comp.ai.neural-nets
From: srctran@world.std.com (Gregory Aharonian)
Subject: Re: Current Neural Networks Funding
Organization: The World
Date: Thu, 9 Apr 1992 15:28:35 GMT

    Maybe neural networks are the cold-fusion of the statistics world, and
people are going back to learning and funding statistics.  If you are really
interested, see whose applying for neural network patents.  If there is
any money to be made big time with neural networks (NOT), patents are the
best measure.

Greg Aharonian
From: lordSnooty@cup.portal.com (Andrew - Palfreyman)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neuron Digest V9 #16 (conferences, CFPs)
Date: 9 Apr 92 08:06:32 GMT
Organization: The Portal System (TM)

i have a timeshare in cancun. if interested...
 --------------------------------------------------------------------------
| lord snooty @the giant | wolfpack, silver, down to the water             |
| poisoned electric head |              andrew palfreyman@cup.portal.com   |
 --------------------------------------------------------------------------
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: (multi-layer + shortcut connections) vs
Date: Fri, 17 Apr 92 04:50:29 GMT
Organization: School of Computer Science, Carnegie Mellon

    From: arms@cs.UAlberta.CA (Bill Armstrong)

    What I am in doubt about is whether the speed of BP learning will
    deteriorate as the number of layers increases.  What is the current
    situation with regard to large, deep BP networks or deep compositions,
    do they learn well?  How about 10 or 20 layers?

Backprop net training, in general, slows down dramatically (maybe
exponentially?) as the number of layers increases.  This is partly because
of some trivially solved problems, such as the fact that the derivative of
the sigmoid function has a range from 0.0 to 0.25, so each stage attenuates
the error signal.  But it is mostly due to the mixing and muddling of the
error signal after a few layers.  You end up with a very complex error
surface with few clear, dramatic slopes to follow.  I don't recall seeing a
fully-connected, layered backprop net for any interesting problem that has
more than three hidden layers.

Adding shortcut connections or bottlenecks to the network gives the
interior nodes some clearer signals to follow, and learning becomes
somewhat faster.

If you want to get up to 10 or 20 layers, the only hope is to use some sort
of modular structure (with explicit training of the individual modules) or
an algorithm like Cascade-Correlation that builds up multiple layers by
training one layer at a time.  Truth to tell, I don't think that there are
many problems that really need more than 5-10 layers.  A number of
researchers are playing with versions of Cascor that will not start a new
layer unless it really buys you something.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu

Newsgroups: comp.ai.neural-nets
From: Sylvano Carrasco <scarrasc@sunee.waterloo.edu>
Subject: maze
Originator: scarrasc@sunee.waterloo.edu
Organization: University of Waterloo
Date: Mon, 6 Apr 1992 02:13:54 GMT
Return-Receipt-To: scarrasc@sunee.waterloo.edu

I am looking for information about solving mazes (or path planning)
using neural networks.  I am investigating trying to solve
an N x N maze.  I am not sure if I can do it with something
as simple as back-prop.  I think I will need to use a
recurrent net (output layer feeding back to input).  If
anyone has any papers or references to this topic, I would
be very interested in hearing about it.

Sylvano Carrasco
U. of Waterloo

Newsgroups: comp.ai.neural-nets