HDTenEightyP/NetNews huggingface dataset:
From: mylene@swanee.ee.uwa.oz.au (Mylene Pijpers)
Subject: Re: Request for nn reference
Organization: Elec Eng, Univ of Western Australia
Date: 14 Apr 92 06:23:56 GMT

In <1992Apr10.052002.179851@cs.cmu.edu> sef@sef-pmax.slisp.cs.cmu.edu writes:

>    From: morten@cogsci.ed.ac.uk (Morten Christiansen)

>    In a recent TV program on AI - "Dream Machines" on BBC 2 (UK), 8/12-91
>    - professor Dreyfus mentioned a (for me) particularly illuminating nn
>    simulation...

>    A closer examination of the photos used
>    to train the network showed that all the photos which had tanks in
>    them were taken on a sunny day, whereas the photos of trees only was
>    taken on a cloudy day.
>    
>I think that this story has become the scientific equivalent of an urban
>legend.  The first time I heard it was in about 1972 or 1973, and the
>technolgy being discredited was perceptrons rather than "neural nets".
>NASA was using perceptrons to discriminate satellite photos with cyclonic
>storms in them from photos without.  They were doing slightly better than
>chance until someone realized that the photos have been processed by
>different labs, and the contrast was different in the two sets.

>I have no idea if the story is true, though I assumed that something like
>this had indeed happened.  But now I wonder.  It would be interesting to
>contact whichever Dreyfus this was (Hubert, Stuart, Richard, Alfred, or
>Zeppo) to see if he has a reference.  It might well be that he heard this
>fron a "friend of a friend".

>BTW, some neural nets do a pretty good job of spotting tanks under some
>conditions.  Your mileage may vary.

My brother had to memorize enemy (Russian?) tanks while he was doing
his military service. Often pictures of very poor quality.
He told me it was not *that* difficult: Tank X has always a black
soldier sitting on top of it, and tank Y is always pictured in the
snow...

Mylene
Newsgroups: comp.ai.neural-nets
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Request for nn reference
Date: Fri, 10 Apr 92 05:20:02 GMT
Organization: School of Computer Science, Carnegie Mellon

    From: morten@cogsci.ed.ac.uk (Morten Christiansen)

    In a recent TV program on AI - "Dream Machines" on BBC 2 (UK), 8/12-91
    - professor Dreyfus mentioned a (for me) particularly illuminating nn
    simulation...

    A closer examination of the photos used
    to train the network showed that all the photos which had tanks in
    them were taken on a sunny day, whereas the photos of trees only was
    taken on a cloudy day.
    
I think that this story has become the scientific equivalent of an urban
legend.  The first time I heard it was in about 1972 or 1973, and the
technolgy being discredited was perceptrons rather than "neural nets".
NASA was using perceptrons to discriminate satellite photos with cyclonic
storms in them from photos without.  They were doing slightly better than
chance until someone realized that the photos have been processed by
different labs, and the contrast was different in the two sets.

I have no idea if the story is true, though I assumed that something like
this had indeed happened.  But now I wonder.  It would be interesting to
contact whichever Dreyfus this was (Hubert, Stuart, Richard, Alfred, or
Zeppo) to see if he has a reference.  It might well be that he heard this
fron a "friend of a friend".

BTW, some neural nets do a pretty good job of spotting tanks under some
conditions.  Your mileage may vary.

-- Scott
===========================================================================
Scott E. Fahlman
Same place as always...
From: smoore@dartmouth.edu (Sean Moore)
Newsgroups: comp.ai.neural-nets
Subject: Re: determining the number of neurons in a hidden layer
Date: 7 Apr 92 12:20:40 GMT
Organization: Dartmouth College, Hanover, NH

In article <1992Apr5.213354.25629@noose.ecn.purdue.edu> tag@hermes.ecn.purdue.edu (Tony A. Gibbens) writes:

   >I have a question I would like to ask of the news group.  

   >When determining the number of neurons needed in a hidden 
   >layer to solve a given problem, has anyone used the 
   >information present in the weight matrix from the hidden 
   >layer to the output layer.  It seems to me that the singulare 
   >value decomposition would yield what neurons were contributing 
   >to the solution.  Also, one could look at the kernal of the
   >weight matrix (from hidden to output) to prune or limit the
   >number of neurons in the hidden layer.  Something like this 
   >is done in control theory and is called model reduction.  

   >If anyone has done this or is doing this, could someone
   >please list the source so that I could read up on it.  

   >Thanks in advance
   >tag (Tony A. Gibbens)

There is an excellent paper by Terence Sanger titled
"Optimal Unsupervised Learning in a Single-Layer Linear Feedforward
Neural Network".  He describes a method to compute the Karhunen-Loeve
Transform of a layer's input information and as a side effect, to
minimize the number of nodes at a given level.  I don't know what
journal it appeared in, but Sanger lists his e-mail address:
	TDS@wheaties.ai.mit.edu

- Sean
From: tom@minnie.cs.su.OZ.AU (Thomas James Jones)
Newsgroups: comp.ai.neural-nets
Subject: Re: Failure to Overfit
Date: 9 Apr 92 23:44:06 GMT

In article <1992Apr7.161230.2156@csrd.uiuc.edu>, saarinen@sp1.csrd.uiuc.edu (Sirpa Saarinen) writes:
|> jim@hydra.maths.unsw.oz.au () writes:
|> 
|> >Sometimes if you train a small net on some data, then train a much
|> >larger net on the same data, the result is much the same (i.e.,
|> >much the same function is computed by the trained net). So why
|> >doesn't the large net, with many free parameters, overfit?
|> >Is there a tendency for large nets to seek a fit that is good,
|> >in some sense (low entropy?)? Does anyone understand what is
|> >going on?
|> 
|> Please mention how many data points you have and how many parameters
|> there are in the two networks you mention (else it is nearly impossible to 
|> answer your question)!
|> 
|> Sirpa Saarinen
|> saarinen@csrd.uiuc.edu

	That is not true. The question stands in general.

	I believe that over-fitting _is_ a problem with neural-networks,
	especially those trained by back-propogation.  The network,
	given time, will simply learn to reproduce the input/output
	pairs and _lose_ the ability to generalize.
	There are plenty of papers on this.

	tom j. jones
From: al@calsci.UUCP (Al Petterson)
Newsgroups: comp.ai.neural-nets
Subject: Re: Hidden Layers, How many?
Date: 15 Apr 92 01:10:18 GMT
Reply-To: al%calsci@gvgpsa.gvg.tek.com (Al Petterson)
Followup-To: comp.ai.neural-nets
Organization: California Scientific Software
News-Software: FSUUCP 1.1 Release 5


In article <1992Apr9.175720.7849@elroy.jpl.nasa.gov>,
sgrenander@nasamail.jpl.nasa.gov (Sven Grenander) writes:
>
> Using the rule of thumb supplied with BrainMaker/Mac, it ought to have
> around 6 hidden neurons in a single hidden layer.  I don't know how they
> (California Scientific Software) came up with this simple formula...

Empirical observation.  It obviously depends on the complexity of your
training set.  We're engineers here, not researchers, so I know that's an
unsatisfying answer, but it's the best I can do.

> To follow up on your question with a question of my own: It is my
> understanding that a 1-[hidden]-layer net can theoretically learn anything
> that an n-[hidden]-layer net can learn (please correct me if I am wrong).
> Having experimented with various numbers of layers I have found no advantage
> in increasing the number of layers.  Are there any rules of thumb governing
> more-than-1-layer situations ?

The key word is "theoretically." Backpropagation isn't perfect.  In some
instances (such as an output neuron whose value has multiple inflection
points as one or several input neurons vary linearly) backprop won't train
with a single hidden layer.

There's no rules of thumb, except: If one hidden layer doesn't converge, or
requires what seems an excessive number of hidden neurons, try two hidden
layers.

I've never seen a network benefit from more than two hidden layers.

> I was also asked for a contact at California Scientific Software (the
> BrainMaker producers).  The guy I usually talk to is Michael Mittman at
> (916) 478-9035, he is at least partially responsible for the Mac
> implementation (still in beta, but quite usable) and can probably transfer
> you to the IBM tech people or the sales people.

The address below is another way to contact us at CSS.

--------
Al Petterson
California Scientific Software
al%calsci@gvgpsa.gvg.tek.com
From: tholozan@cli23da.edf.fr ( Jean-Luc Tholozan stagiaire Corinne)
Newsgroups: comp.ai.neural-nets
Subject: transformation-invariant pattern recognition
Keywords: pattern recognition, object extraction
Date: 16 Apr 92 13:34:21 GMT
Reply-To: tholozan@cli23da.UUCP ( Jean-Luc Tholozan stagiaire Corinne)
Distribution: comp.ai.neural-nets
Organization: Electricite de France, DER, Clamart FRANCE

I am studying the possibility of a 3D pattern recognition of a 3D scene,
which must be invariant to geometrical transformations.

I have 2 main problems:
-I need to recognize every object in a scene, but I do not know how many
objects are in my scene. 
-I need to be able to recognize an object in every geometrical position.

I am not particularly looking for a complete connexionist method, if I
can use ANNs to improve a step of non-connexionist method it will be all
right.

If anybody has an idea or a reference about using ANNs in this kind of
issues email me at tholozan@cli23da.edf.fr

thanks.
Jean-Luc
Newsgroups: comp.ai.neural-nets
From: dtau@mullian.ee.mu.oz.au (David Au)
Subject: OCR data
Organization: Electrical and Electronic - University of Melbourne 
Date: Thu, 16 Apr 1992 05:40:25 GMT

Hi there,

Does anybody know how to obtain the IEEE databases on handprinted characters,
such the ones by Munson and Highleyman? Thanks in advance.

Dave
                        \\|//
David Au                -O-O-    |  ACSnet   : dtau@mullian.ee.mu.oz
Dept of Elec Eng          L      |  internet : dtau@mullian.ee.mu.oz.au
University of Melbourne  \_/     |  uunet    : uunet!munnari!mullian!dtau
Newsgroups: comp.ai.neural-nets
From: dhf@marie.mitre.org (David H. Friedman)
Subject: Re: Kohonent Clustering Algorithms
Organization: Research Computer Facility, MITRE Corporation, Bedford, MA
Date: Thu, 2 Apr 1992 14:46:22 GMT

.  I suppose "kohonent" algorithms have the property of "kohonency"...
From: boris@dehn.mth.pdx.edu (Boris Itkis)
Newsgroups: comp.ai.neural-nets
Subject: Theorem prover
Summary: I am looking for theorem prover
Keywords: expert systems
Date: 9 Apr 92 19:29:00 GMT
Article-I.D.: dehn.boris.702847740

I am looking for theorem prover based on resolution and paramodulation,
written in Turbo Pascal or C (C++) for IBM PC.
If anybody has some info about it, please send me e-mail.

Thanks
Boris
 
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Failure to Overfit
Organization: University of Alberta, Edmonton, Canada
Date: Thu, 9 Apr 1992 16:27:40 GMT

sef@sef-pmax.slisp.cs.cmu.edu writes:


>    From: tom@minnie.cs.su.OZ.AU (Thomas James Jones)

>    	I believe that over-fitting _is_ a problem with neural-networks,
>    	especially those trained by back-propogation.  The network,
>    	given time, will simply learn to reproduce the input/output
>    	pairs and _lose_ the ability to generalize.
>    	There are plenty of papers on this.

>I think that if you actually read some of those papers, you will find that
>overfitting is a function of the training set size and number of free
>parameters in the network, as well as how long you train the net, plus a
>few other things.  So Sirpa Saarinen was right: there's not enough
>information here to tell whether overfitting is likely or even possible.

If we train neural networks with the goal of fitting empirical data,
how can we expect them to come up with different functions we really
want which are not "overfitted"?  The problem doesn't lie in our
inability to specify what we want, because a large enough net with a
dense enough training set derived from the original data could always
do it.  The error is in assuming that neural net training paradigms
will miraculously produce what we want even though we just present the
empirical data and choose a stopping point.  It's obviously an error:
given the same empirical data, there are innumerable ways we could
specify what we want: e.g.  we could say we want the result to be a
K-NN classification, where K is a function of the location in the
input space.  There is no point at which one can stop training that
will work for the whole space.



--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: fittery@arcturus.uucp (Elmer Fittery)
Subject: MIGRAIN/ASPERIN does any body use it?
Organization: Rockwell International
Distribution: ca
Date: Tue, 7 Apr 92 15:44:58 GMT

The Subject says it all.  I am trying to generate interest in using
this tool at work, but so far no cigar.
Newsgroups: comp.ai.neural-nets
From: sgrenander@nasamail.jpl.nasa.gov (Sven Grenander)
Subject: Re: Hidden Layers, How many?
Organization: Jet Propulsion Laboratory
Date: Thu, 9 Apr 1992 17:57:20 GMT

In article <1992Apr8.231220.25892@elroy.jpl.nasa.gov>, sgrenander@nasamail.jpl.nasa.gov (Sven Grenander) writes:
> 
> In article <1992Apr8.212404.12428@thunder.mcrcim.mcgill.edu>, gibsoni@ee.mcgill.ca ( irwin roger  gibson ) writes:
> > 

 ..............

> > 
> > My question is how many neurons should be in the hidden layer and how many
> > hidden layer should the network have?
> > 
> > Your assistance in this matter would be greatly appreciated.
> > 
> > 
> 
> Using the rule of thumb supplied with BrainMaker/Mac, it ought to have around
> 6 hidden neurons in a single hidden layer. I don't know how they (California
> Scientific Software) came up with this simple formula, but CPU-months of my own

 .............

> To follow up on your question with a question of my own: It is my understanding
> that a 1-layer net can theoretically learn anything that an n-layer net can learn
> (please correct me if I am wrong). Having experimented with various numbers of
> layers I have found no advantage in increasing the number of layers. Are there
> any rules of thumb governing more-than-1-layer situations ? 
> 
> -Sven
> 

I've gotten a number of reminders that I neglected to state the rule of thumb.
It is almost embarrasingly simple: the average of the number of input and output
neurons. 

I was also asked for a contact at California Scientific Software (the BrainMaker
producers). The guy I usually talk to is Michael Mittman at (916) 478-9035, he is
at least partially responsible for the Mac implementation (still in beta, but
quite usable) and can probably transfer you to the IBM tech people or the sales
people.

Note: I am not a basic NN researcher, only a NN applications researcher. Other
than privately beta-testing and applying BrainMaker in a private venture I have
no association with California Scientific Software.

-Sven
From: nlonginow@falcon.aamrl.wpafb.af.mil
Newsgroups: comp.ai.neural-nets
Subject: Re: Feature selection of NNs (Take two)
Date: 9 Apr 92 08:51:24 EST
Organization: USAF AL/CFH, WPAFB, Dayton, OH

> 
> I must have missed the original posting of this. Dennis Ruck (Air Force Inst
> of Techn) did an excellent paper called "Feature Selection Using a Multilayer
> Perceptron", where he rank ordered input features according to their effect
> 

	Some people expressed concern as to getting aholt of this paper.
Sorry for being so vague, but I can only supply the authors email address.
If anyone knows of the actual citation, please post it. Email me for the
address if you need it.

-- 
Nick Longinow
Armstrong Lab
Human Engineering division


"The opinions and views expressed here are strictly my own and do not 
necessarily reflect the official position of either the U.S. Air Force 
or its contractors. "  You should think otherwise ?

Newsgroups: comp.ai.neural-nets
From: tsou@magnus.acs.ohio-state.edu (Po-Yu Tsou)
Subject: Discussion : Input/Output number of NN
Keywords: Back propagation network
Organization: The Ohio State University
Date: Thu, 9 Apr 1992 16:58:26 GMT

Dear Netter:

I have a question about the numbers of input/output neurons which would
affect the success of the learning of neuron networks. That is, when
the input number is LESS than the number of outputs, the Backprop network
failed to learn such a set of patterns. My conjecture is that the problem
could be the nonlinear mapping. But I am not sure.

Can someone help me about this matter? I am willing to see the discussion 
about it. Or, has any one met the same trouble. If so, please let me know.

----------------------------------------------------------------------------
Poyu Tsou                                             *   *
The Ohio State University                               :  
E-Mail : p.tsou@ieee.org or Tsou.1@osu.edu           \-----/
----------------------------------------------------------------------------
From: hazem@mmf1.ee.QueensU.CA (Hazem Abbas)
Newsgroups: comp.ai.neural-nets
Subject: TR (University of Toronto)
Date: 13 Apr 92 14:29:43 GMT
Organization: Dept. of Electrical Engineering, Queen's University, Kingston, Ontario, Canada

I am looking for a TR written by Steve Nowlan while he was with the
Connectionist Research Group at University in Toronto, 1990.
I wonder if any body knows the e-mail address od the phone number of
the person at U of T who is in charge of these TR's.
Please reply to
          hazem@eleceng.ee.queensu.ca
Thanx.

--Hazem

*************************************************************************
|  Hazem Abbas                                                          |
|  Queen's University  ............ Tel: (613) 545-6388  (lab)          | 
|  Dept. Electrical Engineering ... Fax: (613) 545-6615                 | 
|  Kingston, Ontario   .............e-mail: hazem@eleceng.ee.queensu.ca | 
|  K7L 3N6, CANADA                                                      | 
*************************************************************************
Newsgroups: comp.ai.neural-nets
From: dms@uno.vis.citri.EDU.AU (David Mcgregor SQUIRE)
Subject: Neural Networks Textbooks
Organization: Computer Science, University of Melbourne, Australia
Date: Fri, 17 Apr 1992 02:01:52 GMT

Fellow netters (pun intended),
								I have recently discovered that I will be
teaching a graduate course in Neural Networks (theory and applications, various varieties etc.) next semester. I have to tell the University Bookroom here
what textbook they should order, so I need to choose one. Unfortuately, I
learn't what I know about networks by reading papers, and have little or no
idea of what textbooks have been written on the subject.

	I would greatly appreciate email on any textbooks anyone knows of,
including the publisher's address (if known).
	
							Thanks in Advance,
											 Squizz

-- 
| David McG. Squire (Squizz),           |  A physicist doing Computer Science. |
| Vision Group, CITRI, 723 Swanston St.,|  For a while. He claims.             |
| 723 Swanston St, Melbourne, AUSTRALIA.|       haven't sold my soul......     |
| Net:       dms@due.vis.citri.EDU.AU   |                 I've pawned it."     |
From: thssrkr@iitmax.iit.edu (Raghunath K Rao)
Newsgroups: comp.ai.neural-nets
Subject: GRADUATE ASSISTANTSHIP. COMPUTER VISION/IMAGE PROC.
Date: 13 Apr 92 18:37:12 GMT
Organization: Illinois Institute of Technology

There is a graduate assistantship available for M.S/Ph.D candidates
for the Fall'92 semester in the Computer Vision and Neural Networks 
Lab, ECE Dept., Illinois Institute of Technology, Chicago. 

Current Research: 
1. Signal Representation using Wavelets
2. Pattern and Object Recognition
3. Image Compression
4. Probabilistic Models of Vision
5. Neural Networks for Image Representation and Recognition 

We are seeking only candidates with excellent qualifications.  
Interested candidates, please reply via EMAIL and send an ASCII resume 
to me at the following address : 

    thssrkr@iitmax.iit.edu

IMPORTANT: GRE and TOEFL (where applicable) scores.



K. Raghunath Rao
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
From: wcalvin@hardy.u.washington.edu (William Calvin)
Subject: Re: Ok, how about 6 different neuron *classes*?
Organization: University of Washington
Date: Sun, 12 Apr 1992 07:35:40 GMT

Classifications of neurons by functional emphasis have been attempted by
various people over the years.  See chapter 29 "Styles of Neuronal
Computation" by W. H. Calvin and K. Graubard in THE NEUROSCIENCES FOURTH
STUDY PROGRAM (MIT Press 1979).

    William H. Calvin   WCalvin@U.Washington.edu
    University of Washington  NJ-15
    Seattle, Washington 98195 FAX:1-206-720-1989

Xref: sparky alt.binaries.sounds.d:761 comp.ai.neural-nets:1930 alt.binaries.sounds.misc:975
From: jaj5c@Virginia.EDU (John Anthony Jordano)
Newsgroups: alt.binaries.sounds.d,comp.ai.neural-nets,alt.binaries.sounds.misc
Subject: Phoneme Libraries?
Summary: Where can I find recorded phonemes
Keywords: neural networks, phonemes, speech recognition
Date: 16 Apr 92 15:44:53 GMT
Article-I.D.: murdoch.1992Apr16.154453.23730
Organization: University of Virginia
Originator: jaj5c@holmes.acc.Virginia.EDU

I'm working on a project where we are attempting to perform speech recognition
of phonemes using a time-delay neural network.  The biggest task we have to do
is simple to segment and classify phonemes for training and testing.  Are there
any libraries out there of segmented phonemes to use for training and testing
speech recognition systems?  If you know of any available via FTP, or if you
have the e-mail address of someone who has worked with speech recognition and
might be willing to share the set of segmented phonemes, please reply to:

   jaj5c@virginia.edu

-- 
John Jordano   jaj5c@virginia.edu   (804) 296-2520
Newsgroups: comp.ai
From: ops@vuse.vanderbilt.edu (Omar Patino-Siliceo)
Subject: Re: Request for reference to intelligent tutoring system
Organization: Computer Center, Vanderbilt University. Nashville, Tenn., U.S.A.
Date: Mon, 13 Apr 1992 14:27:34 GMT

In article <1992Apr13.053549.13859@cc.uow.edu.au> simlau@cc.uow.edu.au (sim kim lau) writes:
>I am doing a project in intelligent tutoring system, any pointer to the
>reference will be much appreciated

``The Book'':

Sleeman, D. and Brown, J.S. Intelligent Tutoring Systems. London,
England: Academic Press (1982).

Plus these other references:

Polson and Richardson, eds. Foundations of Intelligent Tutoring
Systems. Hillsdale, N.J.: Lawrence Erlbaum Associates (1988).

Burns, Parlett, and Redfield, eds. Intelligent Tutoring Systems:
Evolutions in Design. Hillsdale, N.J.: Lawrence Erlbaum Associates
(1991).

Psotka, Massey and  Mutter, eds. Intelligent Tutoring Systems: Lessons
Learned. Hillsdale, N.J.: Lawrence Erlbaum Associates (1988).


<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
     ____   ____               _
    /    \  |   \     Omar Patino-Siliceo       Academic Computing Support
   |      | |    |    Box 1375, Station B       Computer Center
   |      | |__ /     Nashville, TN 37235       Vanderbilt University
    \    /  |  \
   __\  /__ |   \     Tel: (615) 343 1631       ops@vuse.vanderbilt.edu     

     ``Research is what I am doing when I don't know what I'm doing''
                                                  Wernner von Braun
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
From: hampton@tigger.Colorado.EDU (Jeff Hampton)
Newsgroups: comp.ai
Subject: Re: Question Parsing
Keywords: question/answering
Date: 8 Apr 92 18:25:11 GMT
Article-I.D.: colorado.1992Apr8.182511.1312
Organization: University of Colorado, Boulder

I would like any information on Question Parsing also! I'm working on a
question/answering system for a library information system. Any information
would be appreciated. Thanks in advance.

Please e-mail responses to: hampton@tigger.cs.colorado.edu

== Jeff Hampton ==