HDTenEightyP/NetNews huggingface dataset:
From: strech@cs.tu-berlin.de (Christian Stredicke)
Newsgroups: comp.ai.neural-nets
Subject: Performance of NN simulators
Date: 10 Apr 92 13:32:58 GMT
Distribution: comp
Organization: Technical University of Berlin, Germany

I wonder if it makes sense to build a dedicated NN simulation hardware
to archieve a higher performance. 

 - What performance is the "state of the art"? How many units can be
   simulated? 
 - Which kinds of networks can be simulated? Which algorithm for the
   simulation is being used? 
 - Is the "connection machine" suitable for the simulation task? What
   about building a dedicated "NN simulation processor" and using it
   in a massive parallel system?
 - In what way differes the behaviour of analog VLSI designs from the
   behaviour of simulated NN? Is there a significant difference?

Any comments are welcome.

--
Christian Stredicke
Fachbereich Informatik
Technische Universitaet Berlin
-- 
Christian Stredicke
Fachbereich Informatik
Technische Universitaet Berlin
From: osmanh@mmf1.ee.QueensU.CA (Hossam Osman)
Newsgroups: comp.ai.neural-nets
Subject: Back-Propagation
Keywords: Back-Propagation
Date: 7 Apr 92 22:13:46 GMT
Organization: Dept. of Electrical Engineering, Queen's University, Kingston, Ontario, Canada


    I would appreciate if anyone can give answers or discussions to the following two 
questions:

    1) Does increasing the number of hidden units always improve the mean-square error
       at the output of feed-forward nets trained with BP ? 
       That is, the error will not increase after reaching a specific limit of hidden
       units.

    2) Do nonlinear networks always yield a smaller value (or at least the same value)
       for the mean-square error when compared to linear nets.
       That is, if linearity decreases the error, the network will always be capable
       of working in the linear region of the utilized nonlinearity in order to 
       decrease the error.

    Again, I would appreciate any response.
From: lordSnooty@cup.portal.com (Andrew - Palfreyman)
Newsgroups: comp.ai.neural-nets
Subject: Re: What are the six different types of human neurons?
Date: 9 Apr 92 18:29:45 GMT
Organization: The Portal System (TM)

I was told - on an NN course - that each "type" of mammalian neuron is
distinguished by the specific neurotransmitter employed at the synapse.

I believe there are over 40 known neurotransmitters; of course, more
may await discovery. There is also the possibility of structural
differentiation between different neuron "types", I would have expected.

Conclusion: it's hard to imagine how this panoply of options boils down
to a number as low as six! Comments?
 --------------------------------------------------------------------------
| lord snooty @the giant | wolfpack, silver, down to the water             |
| poisoned electric head |              andrew palfreyman@cup.portal.com   |
 --------------------------------------------------------------------------
From: slcj4@cc.usu.edu
Newsgroups: comp.ai.neural-nets
Subject: Is anyone working on ANN with Ultrasound
Date: 16 Apr 92 11:48:56 MDT
Organization: Utah State University

I have done a rather extensive search and have only found very limited 
refences to the use of neural networks to identify and/or characterize 
images from an rf ultrasound image.

Is anyone working on this type of work or know of some work that has been
done in the past?  My particular intrest is in the backscatter reflection
of an object in medical applications, but I would be very interested in 
any work being done.

Thanks for any help you can provide.

I can post any responces

Mark Parsons
slcj4@cc.usu.edu
Newsgroups: comp.ai.neural-nets
From: sgrenander@nasamail.jpl.nasa.gov (Sven Grenander)
Subject: Re: Hidden Layers, How many?
Organization: Jet Propulsion Laboratory
Date: Wed, 8 Apr 1992 23:12:20 GMT

In article <1992Apr8.212404.12428@thunder.mcrcim.mcgill.edu>, gibsoni@ee.mcgill.ca ( irwin roger  gibson ) writes:
> 
> 					Irwin Gibson
>                  		        Dept. of Electrical Engineering
>                          		McGill University
>                          		Montreal, Quebec CANADA
> 
> 					gibsoni@ee470.ee.mcgill.ca
> 
> 					April 6, 1992
> 
> 
> I have recently acquired the Aspirin/Migraines neural network simulator
> and have complied and run a test network.  My problem, Black Box has 13 inputs
> and one output and does not converge. The nature of the problem is as follows:
> 
> 1] One of the 13 inputs can predict the correct out 70 % of the time.  This
> lone input is currently the only input that humans base decisions on.  The
> other 12 inputs usually over load the average human operator, but are relevant
> inputs. 
> 
> All of the 13 inputs are analog, the one input the operator
> currently uses is threshold at 0 (zero) and converted to binary
> (two state output 0.1 or 0.9)
>  
> EXAMPLE:
> if the analog input is greater than 0, operator takes action I.
> if the analog input is less than 0, operator takes action II.
> 
> My question is how many neurons should be in the hidden layer and how many
> hidden layer should the network have?
> 
> Your assistance in this matter would be greatly appreciated.
> 
> 

Using the rule of thumb supplied with BrainMaker/Mac, it ought to have around
6 hidden neurons in a single hidden layer. I don't know how they (California
Scientific Software) came up with this simple formula, but CPU-months of my own
empirical observations seems to support its general accuracy. Fewer neurons
leads to more rapid learning initially, but usually (very fuzzy statement) not
as good ultimately. More neurons learn better if just tested on the training set,
but if given new facts (never seen before) it will not do as well - seems to
have learned to memorize outlier facts rather than incorporate them into
a braoder generalization. 

To follow up on your question with a question of my own: It is my understanding
that a 1-layer net can theoretically learn anything that an n-layer net can learn
(please correct me if I am wrong). Having experimented with various numbers of
layers I have found no advantage in increasing the number of layers. Are there
any rules of thumb governing more-than-1-layer situations ? 

-Sven



Newsgroups: comp.ai.neural-nets
From: minsky@media.mit.edu (Marvin Minsky)
Subject: Re: Hidden Layers, How many?
Cc: Minsky
Organization: MIT Media Laboratory
Date: Wed, 15 Apr 1992 14:57:21 GMT

In article <0133@calsci.UUCP> al%calsci@gvgpsa.gvg.tek.com (Al Petterson) writes:
>
>In article <1992Apr9.175720.7849@elroy.jpl.nasa.gov>,
>sgrenander@nasamail.jpl.nasa.gov (Sven Grenander) writes:
>>
>> Using the rule of thumb supplied with BrainMaker/Mac, it ought to have
>> around 6 hidden neurons in a single hidden layer.  I don't know how they
>> (California Scientific Software) came up with this simple formula...
>
>Empirical observation.  It obviously depends on the complexity of your
>training set.  We're engineers here, not researchers, so I know that's an
>unsatisfying answer, but it's the best I can do.
>
>> To follow up on your question with a question of my own: It is my
>> understanding that a 1-[hidden]-layer net can theoretically learn anything
>> that an n-[hidden]-layer net can learn (please correct me if I am wrong).
>> Having experimented with various numbers of layers I have found no advantage
>> in increasing the number of layers.  Are there any rules of thumb governing
>> more-than-1-layer situations ?
>
>The key word is "theoretically." Backpropagation isn't perfect.  In some
>instances (such as an output neuron whose value has multiple inflection
>points as one or several input neurons vary linearly) backprop won't train
>with a single hidden layer.
>
>There's no rules of thumb, except: If one hidden layer doesn't converge, or
>requires what seems an excessive number of hidden neurons, try two hidden
>layers.
>
>I've never seen a network benefit from more than two hidden layers.

Interesting issues here.  Chapters 6 and 7 of "Perceptrons" prove
various theorems about how many connections are required in order for
a net with one inner layer to recognize various pattern classes.
Generally, the NN community has not realized that this stuff is the
main concern of the book -- and not the convergence issue.  Despite
what losers like Hecht-Neilsson and others have published, all these
theorems appear still to be true.  ("Flash: General Franco is still
dead!" as they used to say on Saturday Night Live.)  I mention this
mainly because Papert and I are merely entertained by his distortion
of the history, but genuinely annoyed by the accompanying
misrepresentation of the mathematical aspects.

  In any case, a little thought will show that if Pattern P has "order
N" for a one-hidden-layer net, then it will have order of the order of
the k-th root of N for a net with K hidden layers.
  So, as Patterson says, we learn partly through engineering and
partly through theory.  Has anyone tried to extend this theory to
finding the orders -- and then, perhaps , to thus discover some bounds
on the convergences -- of some interesting P's for K-inner-layer nets?
And also of course, as Fahlman points out, for nets like those plus
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: What are the six different types of human neurons?
Date: Thu, 09 Apr 92 03:18:45 GMT
Organization: School of Computer Science, Carnegie Mellon


    During a recent neural-networks lecture, it was pointed out that
    comp-sci types only model one type of human neuron, where animal (human)
    bodys contain no less than six different types.
    
    I know I could look this up, but what is a net for?  Could someone
    please post the names of these six, and, if possible, a short discription
    of their physiological locations and function (so far as is known).
    
Actually, there are seven:

Name		Location		Function
-------------------------------------------------------
Happy		Hypothalamus		Reinforcement learning.

Grumpy		Hypothalamus		Inhibition of Happy cells.

Sneezy		Brain stem		Naso-mucoid reflexes, propagation
					of viruses.

Sleepy		Hypothalamus		Generation of theta waves, detection
					and inhibiton of boring lectures.

Bashful		Limbic system		Fight or flight.  Mostly flight.
					Dilation of facial capillaries.

Doc		Cerebral Cortex		Winning big money on Jeopardy.
					Writing a thesis.  Medicine.

Dopey		Cerbral Cortex		Inhibition of Doc cells, losing
					big money on Jeopardy, motor
					control of ears.

Blitzen		North Pole		Oops, wrong species.
---------------------------------------------------------------------------

But seriously, there are a lot more than six flavors of neurons, depending
on how you want to cluster things.  These differ in the patterns of
connectivity, whether they are inhibitory or excitatory, and in more subtle
ways such as time constants.  The "neurons" in most artificial neural net
models are general enough to model all of these neuron types equally well
at some very high level of abstraction.  Whether that level of abstraction
is too high to yield any useful insight into the behavior of real neural
assemblies is the subject of much debate.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu

Newsgroups: comp.ai.neural-nets
From: saarinen@sp1.csrd.uiuc.edu (Sirpa Saarinen)
Subject: Re: Failure to Overfit
Organization: UIUC Center for Supercomputing Research and Development
Date: Tue, 7 Apr 92 16:12:30 GMT

jim@hydra.maths.unsw.oz.au () writes:

>Sometimes if you train a small net on some data, then train a much
>larger net on the same data, the result is much the same (i.e.,
>much the same function is computed by the trained net). So why
>doesn't the large net, with many free parameters, overfit?
>Is there a tendency for large nets to seek a fit that is good,
>in some sense (low entropy?)? Does anyone understand what is
>going on?

Please mention how many data points you have and how many parameters
there are in the two networks you mention (else it is nearly impossible to 
answer your question)!

Sirpa Saarinen
saarinen@csrd.uiuc.edu
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: (multi-layer + shortcut connections) vs
Date: Thu, 16 Apr 92 06:01:42 GMT
Organization: School of Computer Science, Carnegie Mellon

    From: massimo@cui.unige.ch (DEFRANCESCO Massimo)
    
    There is an important reason why multi-layer networks are more interesting
    than general feedforward networks. I would call it `composability' or
    `modularity'.

Well, I was talking about what is a good network architecture in terms of
power and training speed.  It looks to me like you are talking about
something completely unrelated: what can be described most elegantly when
you implement it in a simulator.

    ...In fact, it is easy
    to show that a general feedforward neural network can be transformed
    into an `identical' multi-layer network, in the sense that the two 
    networks trained with the same parameters, the `same' starting weights
    and the same training set will result in two networks which will
    give _exactly_ the same outputs for the same inputs.

After training for the same length of time?  With only nonlinear units?  I
guess I'm a little slow, because this doesn't look easy to me.  Please
explain.  Or were you allowing yourself to make some subset of the units
linear?  In that case I agree that it's easy, since some of the units can
be turned into "shortcut wires", but then I don't see the point.
    
-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: Daniel.K.R.McCormack@uk.ac.cf.cm (Daniel K R McCormack)
Newsgroups: comp.ai.neural-nets
Subject: more info. on ref
Date: 7 Apr 92 08:50:08 GMT
Organization: University of Wales College of Cardiff, Cardiff, Wales, UK.
X-Mailer: Cardiff Computing Maths PP Mail Open News Gateway

Hi ,	
	There was a ref on the news group for :-

	SEXNET
	B.A.Golomb , D.T.Lawrence , T.J.Senjnowski
	NIPS3

	Unfortunately this is not sufficient infomation for
	our library to obtain this refernce.
	Any extra info would be much appricated.

	Many Thanks

		Danny
Newsgroups: comp.ai.neural-nets
From: holder@cse.uta.edu (Larry Holder)
Subject: Structural Databases
Organization: Computer Science & Engineering, Univ. of Texas at Arlington
Date: Wed, 15 Apr 1992 21:19:48 GMT

We are looking for databases to be used for testing an intelligent
structural data analysis tool.  The database should be structural or a
flat representation of a structural environment (e.g., coordinates of
3d entities).  The database will be converted to a graph
representation for input to the system.  Any pointers to such
databases would be appreciated.

Thank you.

-- 
Larry Holder (holder@cse.uta.edu)

University of Texas at Arlington, Department of Computer Science Engineering
Box 19015, Arlington, TX 76019-0015, Phone: 817-273-2596, FAX: 817-273-2548
Newsgroups: comp.ai.neural-nets
From: stgprao@xing.unocal.com (Richard Ottolini)
Subject: Re: What are the six different types of human neurons?
Originator: stgprao@xing
Organization: Unocal Corporation, Anaheim, California
Date: Thu, 9 Apr 1992 15:00:21 GMT

I recall the thin sheet of the neo-cortex was classified into six layers at
one time.  The original poster may have been referring to this.
Newsgroups: comp.ai.neural-nets
From: dbenn@leven.appcomp.utas.edu.au (David Benn)
Subject: Re: CMAC Network?
Organization: University of Tasmania at Launceston
Date: Fri, 10 Apr 92 04:00:41 GMT

In article <1992Apr09.125205.206211@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu writes:
>
>
>    Anyone ever hear of a CMAC network?  If anyone knows of sources of
>    information on this please let me know.  Thanks.
>    
>It's a sort of table-lookup or nearest-neighbors algorithm with
>interpolation that has proven to be useful for a number of robotics and
>other control problems.  Look for papers by James Albus some years back,
>and other work that references these.
>
>-- Scott
>===========================================================================
>Scott E. Fahlman
>School of Computer Science
>Carnegie Mellon University
>5000 Forbes Avenue
>Pittsburgh, PA 15213
>
>Internet: sef+@cs.cmu.edu
>

Also see James Albus' book: "Brains, Behaviour and Robotics".

Regards...


-- 
David Benn - Applied Computing             "We dance around in a ring and  
University of Tasmania at Launceston        suppose; but the secret sits
dbenn@leven.appcomp.utas.edu.au             in the middle and knows".
                                            - Robert Frost
Newsgroups: comp.ai.neural-nets
From: gibsoni@ee.mcgill.ca ( irwin roger  gibson )
Subject: Hidden Layers, How many?
Originator: gibsoni@honey.EE.McGill.CA
Organization: Undergraduate UNIX Lab, EE, McGill University
Date: Wed, 8 Apr 92 21:24:04 GMT

					Irwin Gibson
                 		        Dept. of Electrical Engineering
                         		McGill University
                         		Montreal, Quebec CANADA

					gibsoni@ee470.ee.mcgill.ca

					April 6, 1992


I have recently acquired the Aspirin/Migraines neural network simulator
and have complied and run a test network.  My problem, Black Box has 13 inputs
and one output and does not converge. The nature of the problem is as follows:

1] One of the 13 inputs can predict the correct out 70 % of the time.  This
lone input is currently the only input that humans base decisions on.  The
other 12 inputs usually over load the average human operator, but are relevant
inputs. 

All of the 13 inputs are analog, the one input the operator
currently uses is threshold at 0 (zero) and converted to binary
(two state output 0.1 or 0.9)
 
EXAMPLE:
if the analog input is greater than 0, operator takes action I.
if the analog input is less than 0, operator takes action II.

My question is how many neurons should be in the hidden layer and how many
hidden layer should the network have?

Your assistance in this matter would be greatly appreciated.
From: bourne-linda@CS.YALE.EDU (Linda Bourne)
Newsgroups: comp.ai.neural-nets
Subject: Alan J. Perlis Symposium
Date: 8 Apr 92 20:05:12 GMT
Article-I.D.: RA.199204082005.AA10806
Organization: Yale CS Mail/News Gateway



ALAN J. PERLIS SYMPOSIUM:  Parallel Computers of the Future

Sponsored by the Department of Computer Science, Yale University

April 23, 1992

Sheffield-Sterling-Strathcona Hall
Room 114
Corner Grove and Prospect Streets
New Haven, Connecticut


Morning Session (9:30-11:00)

	Opening Remarks
		Drew McDermott
		Chairman, Yale CS

	"Dataflow Architectures:  From Monsoon to *T"
		Arvind
		Massachusetts Institute of Technology


Afternoon Session (1:30-4:30)

	"PRAMs:  Theory, Physical Design, and Application"
		Wolfgang Paul
		University of Saarbrucken

	"Will Parallel Architectures Converge?"
		H.T. Kung
		Carnegie-Mellon University



Following the last talk a public reception will be held at the
Department of Computer Science, Arthur K. Watson Hall.

All talks are free and open to the public.

For further information please call (203) 432-1246.
-------
From: steveh@goanna.cs.rmit.oz.au (Steven Hayes)
Newsgroups: comp.ai.neural-nets
Subject: T-C problem - as described by Rumelhart, Hinton and Williams
Date: 7 Apr 92 04:19:33 GMT
Organization: Comp Sci, RMIT, Melbourne, Australia

Howdy,

I've been trying to implement a T-C classifier that is position and
orientation invariant, as described in Chapter 8 of Parallel Distributed
Processing, Volume 1.

I'm using a 5x5 unit input layer, 3x3 hidden layer, and a single output unit.
Each unit in the hidden layer is connected to a 3x3 region of the input layer,
and the weights between 3x3 regions are constrained to the same values.
The weights between the hidden and output units are contrainted to have the
same value also. The hidden units have no bias, but the output unit does.
(See the book for a much more articulate desciption ;-)

I can get this net to converge, training it to recognise 'T' and 'C'
patterns in the upper left coner of the input 'retina', and things are fine.
Now because the weights between 'receptive fields' are constrained, it
should be possible to recognise a T or C pattern _anywhere_ on the 'retina'
of input units. This just doesn't work.

I dont understand why. If anyone has ideas as to why this doesn't work,
pleasde mail me. If anyone wants to play with this themselves, I'll send
them the files (used by the 'bp' software also described by Rumelhart and
McClelland).

Thanks in advance,


-- 

   Steven Hayes                                    |  steveh@cs.rmit.edu.au
   Royal Melbourne Institute of Technology (RMIT)  |  +61 3 660-2693
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Failure to Overfit
Date: Thu, 09 Apr 92 05:54:32 GMT
Organization: School of Computer Science, Carnegie Mellon


    From: tom@minnie.cs.su.OZ.AU (Thomas James Jones)

    |> Please mention how many data points you have and how many parameters
    |> there are in the two networks you mention (else it is nearly impossible to 
    |> answer your question)!
    |> 
    |> Sirpa Saarinen
    |> saarinen@csrd.uiuc.edu
    
    	That is not true. The question stands in general.
    
    	I believe that over-fitting _is_ a problem with neural-networks,
    	especially those trained by back-propogation.  The network,
    	given time, will simply learn to reproduce the input/output
    	pairs and _lose_ the ability to generalize.
    	There are plenty of papers on this.
    
    	tom j. jones

I think that if you actually read some of those papers, you will find that
overfitting is a function of the training set size and number of free
parameters in the network, as well as how long you train the net, plus a
few other things.  So Sirpa Saarinen was right: there's not enough
information here to tell whether overfitting is likely or even possible.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu

Xref: sparky comp.ai.neural-nets:1858 comp.ai:1947
From: wichert@sol.cs.uni-sb.de (Andreas Wichert)
Newsgroups: comp.ai.neural-nets,comp.ai
Subject: speech data
Keywords: MTC, speech data
Date: 9 Apr 92 12:54:50 GMT
Followup-To: comp.ai.neural-nets


I developed a new model of a neuronal net, the MTC model (Map-
 Transformation-Cascade)
To test its ability to classify speech, I am searching an ASCII
speech database of segmented spectrals of small words in any language.
If somone has such a database, that he is willing to share,
I would be very glad.

Thanks in advance for any help you can offer.

Andrzej

My adress: wichert@sol.cs.uni-sb.de
From: kens@castle.ed.ac.uk (K G N Sutherland)
Newsgroups: comp.ai.neural-nets
Subject: Re: more info. on ref
Date: 7 Apr 92 10:24:38 GMT
Organization: Edinburgh University

Daniel.K.R.McCormack@uk.ac.cf.cm (Daniel K R McCormack) writes:

>Hi ,	
>	There was a ref on the news group for :-

>	SEXNET
>	B.A.Golomb , D.T.Lawrence , T.J.Senjnowski
>	NIPS3

>	Unfortunately this is not sufficient infomation for
>	our library to obtain this refernce.
>	Any extra info would be much appricated.

>	Many Thanks

>		Danny

@InCollection{	GolLawSej90,
	Author = { B A Golomb and
		D T Lawrence and
		T J Sejnowski },
	Title = { {SEXNET}: A Neural Network Identifies Sex From Human
		Faces. },
	Booktitle = { Advances In Neural Information Processing Systems
		3 },
	Editor = { R P Lippmann and
		J E Moody and
		D S Touretzky },
	Pages = { 572-577 },
	Publisher = { Morgan Kaufmann },
	Year = 1990	}

Ken
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory