HDTenEightyP/NetNews huggingface dataset:
From: gcp79527@csunix.csie.nctu.edu.tw ()
Subject: Re: Back-Propagation
Organization: NCTU
Date: Thu, 9 Apr 1992 07:21:10 GMT

osmanh@mmf1.ee.QueensU.CA (Hossam Osman) writes:
: 
:     1) Does increasing the number of hidden units always improve the mean-square error
:        at the output of feed-forward nets trained with BP ? 
:        That is, the error will not increase after reaching a specific limit of hidden
:        units.

    Increasing the number of hidden units DOES NOT always improve the MSE.
Below a threshold number(not known exactly in general), MSE will be improved
if more HUs are added. But given too many HUs (the same as giving too many 
FREE variables), the nets will be hard to construct a proper mapping between
input and output.

Yo-Yau Lin.
From: nicholas@cogsci.ed.ac.uk (Nicholas Chater)
Newsgroups: comp.ai.neural-nets
Subject: weight decay
Date: 13 Apr 92 11:43:53 GMT
Organization: Centre for Cognitive Science, Edinburgh, UK

John Bullinaria and I have been using weight decay in a 
simple feedforward network, trained with back-prop and have
had difficulty in figuring out what the right amount of 
decay is.  Too little and weights get large with a lot
of training, just as with no decay at all; too much and
the weights are small but don't solve the problem. With
some fiddling around, its possible to find a good value for 
a given problem, but it seems that the right value is
problem dependent. 

Questions:

1. is there a sensible value for weight decay which works for most
problems?
2. if not, how should weight decay be determined - are there
for example an ways of adaptively modifying the ratio of weight
decay as the problem progresses? 

Thanks in advance

	Nick Chater
From: cod14@seq1.keele.ac.uk (A.J. Schofield)
Newsgroups: comp.ai.neural-nets
Subject: Re: Request for nn reference
Summary: The legend of tank detection
Date: 10 Apr 92 10:42:37 GMT
Organization: University of Keele, England

In article <1992Apr10.052002.179851@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu writes:
> 
> I think that this story has become the scientific equivalent of an urban
> legend.
> 
I vaguely remember hearing the story about the tank detector from an 
aquantance BEFORE it was mentioned on the BBC TV program `Dream Machine'. The
sourse was definately second hand but my contact is based in the UK. I think
this is posibly a legend but it does serve as a warning that NN have the
potential to make perverse correlations between input data and desired 
outputs.


Regards Andrew Schofield
From: boris@dehn.mth.pdx.edu (Boris Itkis)
Newsgroups: comp.ai.neural-nets
Subject: Theorem prover
Keywords: expert systems
Date: 9 Apr 92 06:27:05 GMT

I need a theorem prover based on resolution and paramodulation,
written in Turbo Pascal or C (C++) ( for IBM PC ).

If anybody has some info about it, pleace, send my e-mail.

                               Thank you
                                          Boris
From: fong@latcs1.lat.oz.au (Simon Fong)
Newsgroups: comp.ai.neural-nets
Subject: NN for characters recognition
Date: 15 Apr 92 07:35:49 GMT
Organization: Comp Sci, La Trobe Uni, Australia
Originator: fong@latcs2.lat.oz.au



Hi,

Could anyone please recommend some reference on using neural-nets for 
characters recognition?
Thanks.....

regards,
Simon Fong
La Trobe University,
Bundoora, Australia.
<email>: fong@latcs1.OZ.AU
<phone>: 457-3871
From: sef@sef-pmax.slisp.cs.cmu.edu
Newsgroups: comp.ai.neural-nets
Subject: Re: more info. on ref
Date: 7 Apr 92 15:19:16 GMT
Organization: School of Computer Science, Carnegie Mellon


    From: Daniel.K.R.McCormack@uk.ac.cf.cm (Daniel K R McCormack)
    
    	SEXNET
    	B.A.Golomb , D.T.Lawrence , T.J.Senjnowski
    	NIPS3
    
    	Unfortunately this is not sufficient infomation for
    	our library to obtain this refernce.

The full reference is:

B. A. Golomb, D. T. Lawrence, and T. J. Sejnowski (1992) "Sexnet: A Neural
Network Identifies Sex from Human Faces" in Advances in Neural Information
Processing Systems 3, R. P. Lippmann, John E. Moody, and D. S. Touretzky
(eds.), Morgan Kaufmann Publishers, San Mateo CA, ISSN 1049-5258, ISBN
1-55860-184-8.

If there are connectionists working at your institution, your library
should carry *all* the NIPS proceedings.

I find myself not posting as many references as I might because it's too
time-consuming to type in the whole thing and I can never seem to get away
with any abbreviation in this forum.  Too many people with too many diverse
backgrounds, I guess.  I'm not sure what the solution is.  Maybe we need a
FAQ (that's "frequently asked questions") list with a glossary of short
naems and long names for all the major conferences and journals.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: whsu@panther.ecn.purdue.edu (William Hsu)
Newsgroups: comp.ai.neural-nets
Subject: Database
Date: 9 Apr 92 13:37:17 GMT
Article-I.D.: noose.1992Apr9.133717.8942
Organization: Purdue University Engineering Computer Network

Is there some real life database on classification e.g. medical diagnosis.
Would someone who has tried classifying interesting data sets share
that with me. Thanks.
--
William Hsu Department of Electrical Engineering, Purdue University 
West Lafayette, IN 47907.    	Tel : (317) 494-3471.
Internet:	whsu@panther.ecn.purdue.edu
uucp: ...!{decvax,hplabs,ihnp4,linus,rutgers,seismo}!pur-ee!whsu
From: inyup@python.icsl.ucla.edu (Inyup Kang)
Newsgroups: comp.ai.neural-nets
Subject: Asynchronous system and Neural network
Date: 6 Apr 92 21:42:20 GMT
Organization: UCLA Integrated Circuits & Systems Laboratory

Hi, folks.

I'm going to do research on the asynchronous VLSI design.
Then, I have a generic questions on the operation of the world.
My questions are as follows:
0. The definition of the "asynchronous" and "synchronous"?
   (The easiest one.)
1. Is the world operated in asynchronous manner or synchronous?
   (My basic philosophy push it to the asynchronous,
    In other words, there is no absolute clock determining
    the behavior of the world. If there is some clock, it's
    purely local one. Need verification or rationale!)
2. Are the animal systems(e.g. human) asynchronous or synchronous?
   (My guess; the system of having life is purely asynchronous.
   For example, the thinking mechanism and the behavior of the neural
   cell is purely asynchronous.) 
3. Let the following the model of the electric(or signal) processing;

	 real world ---> processing ---> real world
            ^        ^               ^
	    |	     |               |
	    |	     |               |
	    |	   sampling       post-processing
(async or sync?) (synchronous)    (synchronous)

    Then, if we use the asynchronous design in implement the processing block,
    the conversion or synchronization must be required in post-processing.
    (It's might be the bad news for the asynchronous mania.)
    My question is how about the data processing unit(computer) interacting
    only with the human(supposedly the asynchronous one)?
    (It's might be the good news for some guys.)

How do you think about these?

---
 Inyup Kang // inyup@icsl.ucla.edu, ..!(uunet,ucbvax)!icsl.ucla.edu!inyup 
--
 Inyup Kang // inyup@icsl.ucla.edu, ..!(uunet,ucbvax)!icsl.ucla.edu!inyup 
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
From: edstrom@hsc.ucalgary.ca (John Edstrom)
Subject: Re: What are the six different types of human neurons?
Date: Thu, 09 Apr 1992 14:53:02 GMT
Organization: Neuroscience Division, U of Calgary School of Medicine, Calgary, Alberta, Canada

In article <1992Apr8.235705.26412@organpipe.uug.arizona.edu> bill@NSMA.AriZonA.EdU (Bill Skaggs) writes:
>In article <1992Apr8.151118.4780@watmath.waterloo.edu> 
>mwtilden@watmath.waterloo.edu (Mark W. Tilden) writes:
>>
>>During a recent neural-networks lecture, it was pointed out that
>>comp-sci types only model one type of human neuron, where animal (human)
>>bodys contain no less than six different types.
>>

I'd be interested in hearing more about this.  What criteria were used
to define these classes?  What was the context of the talk?

>>I know I could look this up, but what is a net for?  Could someone
>>please post the names of these six, and, if possible, a short discription
>>of their physiological locations and function (so far as is known).
>>
>  I doubt that you could look this up, because it's nonsense.
>Animal bodies contain literally hundreds of different types of
>neuron.  Even if you only distinguish on the basis of the
>chemical neurotransmitter used, there are still at least several
>dozen different types.  The functional consequences of these
>differences have hardly begun to be explored by neural network
>theorists.
>
>	-- Bill

Amen!  If there only were six types we'd probably know everything by
now. :-)

 Except for specail restricted purposes (retina physiolgy, cerebellum
physiology, etc.) the notion of a few types of neurons or standard
circuitry has no credibility among neuroscientists.


John Edstrom | edstrom @ elmer.hsc.ucalgary.ca

--
 RM 2104, HSc Building,  Div. Neuroscience
 U. Calgary School of Medicine,  3330 Hospital Drive NW
 Calgary, Alberta       T2N 3Y4
 (403) 220 4493  (wk)
From: rapaport@acsu.buffalo.edu (William J. Rapaport)
Newsgroups: comp.ai.neural-nets
Subject: Computer Science position at SUNY Buffalo
Date: 6 Apr 92 16:51:52 GMT
Organization: State University of New York at Buffalo/Comp Sci


             STATE UNIVERSITY OF NEW YORK AT BUFFALO

                 Department of Computer Science
                   Temporary Faculty Position

We anticipate an opening for a one-year visiting faculty position
for  the  coming  year at a competitive salary for a new Ph.D. or
for a more senior person who wants  to  supplement  a  sabbatical
salary.  The successful candidate will teach one course or gradu-
ate seminar each semester. I would welcome you bringing  this  to
the  attention  of your finishing doctoral students or colleagues
who might be interested.  Please ask anyone interested to  submit
a curriculum vitae, and to arrange to have four letters of refer-
ence  sent  directly  from  their  referees,  to  me  by   e-mail
(selman@cs.buffalo.edu) or by mail to

                  Alan L. Selman
                  Department of Computer Science
                  226 Bell Hall
                  State University of New York at Buffalo
                  Buffalo, NY 14260

The Department currently  has  15  tenure-track  faculty, 12  ad-
ditional    faculty,   and   125   graduate   students.   Primary
research  areas  include:  artificial  intelligence,   complexity
theory,   computer vision, numerical linear algebra, parallel al-
gorithms, programming languages,  systems  and  VLSI.  Department
members   are   actively  engaged  in  interdisciplinary research
programs in the Advanced Scientific  Computing   Graduate  Group,
the  Cognitive Science Center, the Vision Graduate Group, and the
NSF National Center for  Geographic  Information  and   Analysis.
Departmental  computing  facilities include a network of worksta-
tions, hypercubes, Symbolics, an Encore   Multimax   and  several
image processing/graphics systems.
Newsgroups: comp.ai.neural-nets
From: gcp79527@csunix.csie.nctu.edu.tw ()
Subject: Re: Back-Propagation
Organization: NCTU
Date: Fri, 10 Apr 1992 01:46:19 GMT

kavuri@lips.ecn.purdue.edu (Surya N Kavuri) writes:
: In article <1992Apr9.072110.14997@Silence.pi.nctu.edu.tw>, gcp79527@csunix.csie.nctu.edu.tw writes:
: > 
: >     Increasing the number of hidden units DOES NOT always improve the MSE.
: > Below a threshold number(not known exactly in general), MSE will be improved
: > if more HUs are added. But given too many HUs (the same as giving too many 
: > FREE variables), the nets will be hard to construct a proper mapping between
: > input and output.
: > 
: > Yo-Yau Lin.
: 
:   As long as the residuals have an underlying structure (ie., not just white noise)
:  shouldn't the addition of extra nodes (say by fixing the previous ones) be useful 
:  in fitting the residuals ? 
: 

    There are some training method based on this _add HU(s) on error_ 
during training, CASCOR is one of these.  But for "increasing the number of 
HUs", I mean "before training an ANN, we use MORE HUs."

    There is a local minimum problem in MLP. A nolinear(?) net with gradient 
search may (someone said that he often gets what he want) sufer from this.
So, if we fix the previous learned weights, and then add extra nodes, it seems
that it will not escape from this local minimum.  Then, does this lead to 
a net of good generalization? If we start from one HU and add extra node(s) 
during training, I think it is much like Principal Component Analysis, 
doing this way may yield a better result.

:  On the question of comparison with linear models, neural nets always need not do  better.
:  Try fitting a straight line with a network and compare to a linear model.  
:  As they say in my home town : "Don't need a Porsche to get your groceries" ;-)

I agree.


Yo-Yau Lin.
From: moutarde@aar.alcatel-alsthom.fr (Fabien Moutarde)
Newsgroups: comp.ai.neural-nets
Subject: (multi-layer + shortcut connections) vs general feedforward net
Date: 13 Apr 92 08:29:17 GMT
Organization: Alcatel-Alsthom Recherche ( France )


In article <1992Mar29.053049.267141@cs.cmu.edu>, sef@sef-pmax.slisp.cs.cmu.edu (Scott E. Fahlman) writes:

|> Based on my own obvservations, I would say that it is usually advantageous
|> to use shortcut connections.  In many problems, the inputs have a strong
|> direct effect on the outputs, and only a few hidden units are needed to
|> provide some sort of nonlinear correction.  If there are no direct
|> connections, you have to waste time training hidden units -- perhaps a lot
|> of them in multiple layers -- to transmit this information.

Am I wrong, or are you simply suggesting that it is often better to use 
fully connected feedforward nets instead of *layered* feedforward nets?

In fact, I've been wondering myself for a while why most NN designers
using backprop restrict themselves to the class of multi-layer nets,
despite the fact that the general form of the algo deals with *any*
feedforward net. Of course the existence proofs of Cybenko or Hornik et al.
deal with layered nets, but fully connected feedforward nets do contain
the subclass of multi-layer ones.
My guess was that introducing what you call the shortcut
connections was avoided because it increases the total number of weights,
but from what you say, it seems that you then need less neurons...

However I would expect that allowing many composed non-linearities
might make the cost hypersurface (in weight-space) even more complex
and increase the probability to fall in a local minimum much higher
than the global minimum. Does anyone has any comment or reference about that?


	Fabien


-- 

+===========================================+
|                                           |
| Fabien Moutarde                           |      +---------------+
| Alcatel Alsthom Recherche                 |      | A L C A T E L | 
| Route de Nozay                            |      +---------------+
| 91460 Marcoussis                          |      | A L S T H O M |
| FRANCE                                    |      +===============+
|                                           |          RECHERCHE  
| e-mail: moutarde@aar.alcatel-alsthom.fr   |      
|                                           |   
+===========================================+
Newsgroups: comp.ai.neural-nets
Subject: Re: (multi-layer + shortcut connections) vs
From: massimo@cui.unige.ch (DEFRANCESCO Massimo)
Date: 16 Apr 92 13:06:09 +0200
Reply-To: massimo@cui.unige.ch
Distribution: world
Organization: University of Geneva

In article 188461@cs.cmu.edu, sef@sef-pmax.slisp.cs.cmu.edu () writes:
>    From: massimo@cui.unige.ch (DEFRANCESCO Massimo)
>
>    There is an important reason why multi-layer networks are more interesting
>    than general feedforward networks. I would call it `composability' or
>    `modularity'.
>
> Well, I was talking about what is a good network architecture in terms of
> power and training speed.  It looks to me like you are talking about
> something completely unrelated: what can be described most elegantly when
> you implement it in a simulator.

You are right, I'm less interested in the insights and speed of neural network 
architectures, than in the higher level handling of neural networks to
build complex neural systems. What was important to me is to show that
there is no loss of generality WRT shortcut connections in a network calculus 
which uses as simple basic elements as a functional layer.


>    ...In fact, it is easy
>    to show that a general feedforward neural network can be transformed
>    into an `identical' multi-layer network, in the sense that the two 
>    networks trained with the same parameters, the `same' starting weights
>    and the same training set will result in two networks which will
>    give _exactly_ the same outputs for the same inputs.
>
>After training for the same length of time?  With only nonlinear units?  I
>guess I'm a little slow, because this doesn't look easy to me.  Please
>explain.  Or were you allowing yourself to make some subset of the units
>linear?  In that case I agree that it's easy, since some of the units can
>be turned into "shortcut wires", but then I don't see the point.
>    

Of course, I add what I call Identity and Multiplex functional networks, 
which can be implemented as layers with identity (hence "linear")
transfer functions. The point is that such a network is decomposable into 
indepent layers (that's what I'm looking for), and that there is no loss 
of generality in multi-layer networks as long as you are allowed to specify
any transfer function in your layers, even with a uniform learning procedure.

Massimo.
From: erwin@trwacs.fp.trw.com (Harry Erwin)
Newsgroups: comp.ai.neural-nets
Subject: Current Neural Networks Funding
Date: 8 Apr 92 00:06:53 GMT
Organization: TRW Systems Division, Fairfax VA

This question may be inappropriate, but with the evaporation of US
Government interest in neural network applications, who's left that's
sponsoring research? I have the distinct impression that there's no one
interested outside of academia.

-- 
Harry Erwin
Internet: erwin@trwacs.fp.trw.com

Newsgroups: comp.ai.neural-nets
From: yang@milton.cs.uiuc.edu (Der-Shung Yang)
Subject: transformation-invariant pattern recognition 
Organization: University of Illinois, Dept. of Comp. Sci., Urbana, IL
Date: Fri, 10 Apr 1992 15:11:47 GMT

Hi, I'm studying transformation-invariant pattern recognition (e.g. 
translations, rotations, and scalings). I have studied Fukushima's 
Neocognitron, High-order nets (Spirkovska, Reid, Giles, Maxwell, etc.),
and LeCun's constrained bp for handwritten zipcode digits recognition.
Does anybody have something more to add to the list? BTW, this list
is not restricted to 2D recognition only. 3D, 4D, 5D, etc.(if there are
such things:-) are also fine.

Thanks a lot for the info in advance.
 
Please email responses to yang@cs.uiuc.edu.

--
DerShung Yang    
yang@cs.uiuc.edu 
Beckman Institute
Univ. of Illinois at Urbana-Champaign
From: davisd@milton.u.washington.edu (Daniel Davis)
Newsgroups: comp.ai.neural-nets
Subject: Asipirin/Migraines
Date: 14 Apr 92 22:34:46 GMT
Organization: University of Washington, Seattle


Are there any mailing lists devoted to Aspirin/Migraines? How about any 
other NN simulation package?

Thanks - Dan Davis
Newsgroups: comp.ai.neural-nets
From: hannahp@topaz.ucq.edu.au
Newsgroups: comp.ai.neural-nets
Subject: Aspirin/Migraines
Date: 15 Apr 92 11:23:49 AET
Organization: University of Central Queensland, Australia



If there is anyone out there with information on FTP sites holding PC
versions of this, could they email me the address?

Thanks in advance.


Paul Hannah.
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
From: mwtilden@watmath.waterloo.edu (Mark W. Tilden)
Subject: Ok, how about 6 different neuron *classes*?
Organization: University of Waterloo
Date: Sun, 12 Apr 1992 03:06:47 GMT



Well, following so many replies telling me what a wally I am for assuming
only 6 neurons, I guess I'd better clarify.  Sorry my question came off 
so naive, but I was after details as to if there are some general 
neuron classes which would map simply to cybernetic control levels 
in biological vertebrates.

As the classic electronic Neural-Net Neuron is a corruption of the 
small brain synapse, are there some other *general* classes of neuron 
whose *main functions* are things like centralized wiring, electrical 
signal collision arbitration, motor interface, sensation, and 
locomotion preprocessing?  And if so, does anybody know if these classes
have specific titles?

I've been working at building robotic forms of such devices and have 
had some success.  The purpose behind my post was to find out 
if there might be some biologic names for the classes of modified neurons 
I use in my artificial nervous systems.

You know, have I discovered something in one science that has 
a matching label in another?  Lends some validity, usually.

Oh, by the way, after a bit of digging, it seems the picture flashed 
during the aforementioned Neural-Net lecture was just a breakdown of 
6 types of human sensory neuron.  No panic. 

Is all.


-- 
Mark Tilden: _-_-_-__--__--_      /(glitch!)  M.F.C.F Hardware Design Lab.
-_-___       |              \  /\/            U of Waterloo. Ont. Can, N2L-3G1
     |__-_-_-|               \/               (519)885-1211 ext. 2454 
"MY OPINIONS, YOU HEAR!? MINE! MINE! MINE! MINE! MINE! AH HAHAHAHAHAHAHAHAHA!!"
Newsgroups: comp.ai.neural-nets,bionet.neuroscience,bionet.info-theory
From: rbutera@pygmy.owlnet.rice.edu (Rob Butera)
Subject: Re: What are the six different types of human neurons?
Organization: Rice University
Distribution: na
Date: Fri, 10 Apr 1992 00:09:57 GMT

In article <1992Apr8.151118.4780@watmath.waterloo.edu>, mwtilden@watmath.waterloo.edu (Mark W. Tilden) writes:
> 
> During a recent neural-networks lecture, it was pointed out that
> comp-sci types only model one type of human neuron, where animal (human)
> bodys contain no less than six different types.
> 

This is bogus.  I'd love to see answer ...

And what kind of neuron do com-sci types think their modeling, anyway?
Maybe their modeling a neural network, but even that is up for questioning.

My own experience is that the comp-sci "neural networks" are mass-parallel
type architectures, combined with feeback and weighting schemes.  This
could be "inspired" by a biological neural network, but that's about it.

Of course, everyone has their own definition of a neural network.  Try
putting a neuroscientist and an comp-sci AI person in the same room ...

> 
> Just when I thought we were making progress in neural-nets, Ma Nature
> throws us a 6-sided multiplier.  Typical.
> 

I believe its many orders larger than just 6-sided ....

-- 
Rob Butera - ECE grad student   Rice University, Houston, TX   rbutera@rice.edu

"This attractive lady, whom I had only recently been introduced to, dropped
 into my lap.  I chose not to dump her off." - Gary Hart
Newsgroups: comp.ai.neural-nets
From: kavuri@lips2.ecn.purdue.edu (Surya N Kavuri )
Newsgroups: comp.ai.neural-nets
Subject: Hidden nodes ... How many ?
Date: 15 Apr 92 18:30:15 GMT
Organization: Purdue University Engineering Computer Network


  When considering the issue of hidden node number in a network
  is not the functional form of the hidden node an important
  factor ? If allowed arbitrary function at the hidden node,
  would not a single hidden node sufficient (assuming the single
  node has the freedom in form to represent the decision boundary).

  How can we talk of hidden node number without regard to functional
  form ? 


  Surya Kavuri
  
  
Organization: Carnegie Mellon, Pittsburgh, PA
Newsgroups: comp.ai.neural-nets
Date: Tue, 14 Apr 1992 23:37:26 -0400 
From: Guest-2-ITC <gi02+@andrew.cmu.edu>
Subject: neural net on image processing


Hi, 

      Could anyone recommend a good reference for neural net application
on image/signal
processing?  
      Thanks in advance!