HDTenEightyP/NetNews huggingface dataset:
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Rule extraction from NN
Organization: University of Alberta, Edmonton, Canada
Date: Mon, 27 Jul 1992 17:23:27 GMT

sef@sef-pmax.slisp.cs.cmu.edu writes:


>The general problem with rule extraction from neural nets is not solvable.
>A neural net provides a richer language than can be captured in a small,
>human-readable set of rules, whether symbolic or fuzzy.
  ...
>Looked at from the other side, this is a kind of advantage: neural nets
>allow us to explore the wide universe of mappings for which there is no
>simple rule-based equivalent.

>-- Scott

I agree with Scott.  An NN solution may or may not be easy to
understand; and a wide universe of mappings is offered.

However, a disadvantage of the NN approach is that the universe of
possible mappings may be *too wide* in the case of a particular
problem.  If a priori knowledge implies there are some restrictions on
the solution mapping, such as monotonic behavior or other a priori
constraints that have to be satisfied (e.g.  the value of output B is
always between output A and output C), it may be hard or impossible to
force the neural solution to respect those constraints.  Since the NN
solution would certainly be incorrect if the constraints don't hold,
it might be hard in such cases to have confidence in any explanation
derived from it.

Do people have any experience enforcing monotonicity constraints?  How
would this be done in a BP system?  If you forced all weights in the
net to be positive, then the function would be increasing in all
variables. If your variables are chosen as x1 (for monotonic
increasing), minus x2 (for decreasing),etc. then this would work. You
still have to worry about functions that change from increasing to
decreasing or vice-versa, and I have no idea how to do this within
the usual MLP paradigm.

Needless to say, in some control applications, monotonicity is very
important.  If the pressure in a boiler is above a certain point and
rising, you certainly don't want to add more fuel!

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
From: connolly@rabbit.cs.umass.edu (Christopher Ian Connolly)
Newsgroups: comp.ai.neural-nets
Subject: Re: Thanks Neural Nets, and Brains?
Date: 23 Jul 92 19:13:23 GMT
Organization: University of Massachusetts, Amherst

In article <arms.711907358@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
>First off, isn't it rather strange that the most widespread artificial
>model of neural operation. the multilayer perceptron, uses continuous
>quantities on its connections, while the dendrites and axons of
>neurons use "zero or one" type action potentials?

Don't forget that electrical synapses (gap junctions) are an analog
means of intercellular communication.  Whether they represent a
significant part of brain function is another question entirely.
-- 
	-	-	-	-	-	-	-
Christopher Ian Connolly			connolly@cs.umass.edu
Laboratory for Perceptual Robotics		wa2ifi
University of Massachusetts at Amherst		Amherst, MA 01003
From: hameed@hal.COM (Hameed Mohammed)
Newsgroups: comp.ai.neural-nets
Subject: Info Please..Companies working on Neural Networks
Date: 25 Jul 92 21:28:50 GMT
Distribution: usa
Organization: HaL Computer Systems, Inc.
NNTP-Posting-Host: sabre.hal.com

Hi 

A friend of mine is looking for jobs in the Neural Networks area.

He's done his MS in Computer Engineering with a lot of work in

NN.


I'd appreciate if you could mail me a few companies working in this
field.


Thanks in advance,

Hameed.

hameed@hal.com
From: weigl@sibelius.inria.fr (Konrad Weigl)
Newsgroups: comp.ai.neural-nets
Subject: Re: Rule extraction from NN
Date: 23 Jul 92 10:36:55 GMT
Organization: INRIA, Sophia-Antipolis (Fr)

There was a paper in the Washington conference IJCNN '90 about
rule extraction from a backprop network that had learn the optimum
opening bid for bridge;
it was a poster, and done by two frenchmen, of which one was by 
CEMAGREF; I am sorry, that is all I know.

Good luck,

Konrad Weigl               Tel. +33 93 65 78 63
Projet Pastis              Fax  +33 93 65 76 43
INRIA                      email weigl@sophia.inria.fr
B.P. 93
06902 Sophia Antipolis Cedex 
France
Newsgroups: comp.ai.neural-nets
From: davids@eng.auburn.edu (David Schreibman)
Subject: ANSPEC References/Information
Reply-To: davids@eng.auburn.edu
Organization: Auburn University Engineering
Date: Thu, 23 Jul 1992 23:30:34 GMT

Does anyone know of any references for descriptions of SAIC's
ANSPEC?  Any information about the language would be a great help.
Thanks in advance to anyone who can help.

-David
From: Daniel.K.R.McCormack@cm.cf.ac.uk (Daniel K R McCormack)
Newsgroups: comp.ai.neural-nets
Subject: Rule extraction from NN
Date: 22 Jul 92 11:05:37 GMT
Organization: University of Wales College of Cardiff, Cardiff, Wales, UK.

Hi,

	I was aware of some people asking questions on this
news group about extracting rules from NN after they have trained.
I would appreciate any refs or summaries on this topic.

Many thanks

	Daniel McCormack.
From: ast48365@uxa.cso.uiuc.edu (A Tull )
Newsgroups: comp.ai.neural-nets
Subject: beginner's question in neural net...
Summary: choice of appropriate data
Keywords: testing data vs training data
Date: 24 Jul 92 01:35:33 GMT
Organization: University of Illinois at Urbana

I was trying to implement a net for a simple mathematical function. My first
question is : should the TRAINING  data be deterministic or random ?  I was
trying to make a training set of about two hundred training pairs, but my
net failed to converge. Is it alright to split my training set into, say,
3 smaller training set, and re-train the net ? That is, I feed in the first
set of data, (assuming convergence), RESET the net, feed in the seconde
set of data, RESET, feed in the third set of data... and then, gather all
3 outputs , and put together to observe how the net learned, will the
final result make any sense ? (or do I have to analyse them each one on
their own ?

Another case is, assume now I have 3 training sets and 3 testing sets,
will the followsing 2 sequence make any difference (or sense) to the
final output ?

	train1, test1, train2, test2, train3, test3
		-OR-
	train1,train2,train3,test1,test2,test3


help DESPARATELY in need.
thanx in advance.

Alice Tull
tull@ibma0.cs.uiuc.edu

-- 
Alice Tull
ast48365@uxa.cso.uiuc.edu
Newsgroups: comp.ai.neural-nets
From: batra@boulder.Colorado.EDU (sajeev batra)
Subject: papers or books on Bayes Classifier
Organization: University of Colorado, Boulder
Date: 25 Jul 92 05:35:32 GMT


I'm interested in learning some more about this Bayesian classifier.
Could someone suggest to me some introductory level papers that either
use or explain the use of Bayesian classifiers in pattern recognition.
Or an introductory level book would also be fine.

Thank you.


sajeev batra
comp sci & mcd biology
univ of colorado @ boulder

Newsgroups: comp.ai.neural-nets
From: lever@voyager.corp.sgi.com (Karen Lever)
Subject: Free Classes in Tantric Meditation
Followup-To: poster
Originator: root@voyager.corp.sgi.com
Keywords: consciousness, meditation, Enlightenment
Organization: Silicon Graphics, Inc.
Distribution: ba
Date: Thu, 30 Jul 1992 08:40:20 GMT

Free intensive workshops will be offered in the Bay Area:

Stanford University                       U.C. Berkeley
    Thursday, July 30                         Monday, August 3
    Thursday, August 6                        


Topics include: Meditation, Buddhism, Power, Balance, Wisdom,
Enlightenment, Personal Happiness, Reincarnation, Career Success
and Miracles.  Classes consist of discussion and meditation, 
and are designed for individuals age 29 and under. 

Stanford classes will be held in the Escondido Village Center 
(next to the E.V. office), from 7:30 to 9:30 p.m.

U.C. Berkeley classes will be held in Wheeler Hall, Room 223,
from 7:30 to 9:30 p.m.

Additional classes will be offered in the Bay Area this summer
and fall.  For information, please call 415-995-4721.

Newsgroups: comp.ai.neural-nets
From: gibsoni@ee470.ee.mcgill.ca ( irwin roger  gibson)
Subject: Bayes Classifier & Cluster Analysis software required
Organization: EE Lab, McGill University EE Lab, Montreal
Date: Fri, 24 Jul 92 18:14:49 GMT

Bayesian Classifier & Cluster Analysis
=======================================

If anyone has or knows of the following types of software and there
availability: (ftp or email)

Bayesian Classifier & Cluster Analysis (DOS or UNIX)

This software is urgently required for use in Neural Network Research.


"I thank you for your cooperation and time."
********************************************
gibsoni@ee470.ee.mcgill.ca

Irwin Gibson
D'Ombrain Computer Facility
Dept. of Electrical Engineering
McGill University
Montreal, Quebec CANADA
H3A 2K6

Newsgroups: comp.ai.neural-nets
From: nnj@engr.ucf.edu (Namho Jung)
Subject: Info. Neural Nets and Expert System/Blackboard System
Organization: engineering, University of Central Florida, Orlando
Date: Mon, 27 Jul 1992 13:08:58 GMT

Hello netters,

I am looking for information regarding to the synergy of Neural Nets 
and Expert System/Blackboard System.  Papers, software (public or 
commericial) or full citation please.   

Thank in advance.

Namho Jung.

namho@ists.fir.engr.ucf.edu
nnj@pestilence.engr.ucf.edu


Newsgroups: comp.ai.neural-nets
From: ledwards@leland.Stanford.EDU (Laurence James Edwards)
Subject: Re: Neural Nets and Brains
Organization: DSG, Stanford University, CA 94305, USA
Date: Sat, 25 Jul 92 21:01:57 GMT

In article <1992Jul24.164544.11876@cs.ucf.edu>, clarke@acme.ucf.edu (Thomas Clarke) writes:
|> In article <arms.711986585@spedden> arms@cs.UAlberta.CA (Bill Armstrong)  
|> writes:
|> > ....  My point
|> > was that BP nets use continuous signals and the brain doesn't -- an
|> > obvious very significant difference.  I was asking why people would
|> > expect to understand the brain by studying a system (BP) that is
|> > *different* at the most basic level of signalling.
|> > 
|> > Don't you agree that if the brain works on 0-1 signals, then to study
|> > the brain one could beneficially look at logical systems? 
|> 
|> I believe the brain may well use continuous or analog signals.
|> 
|> While individual neuron action potentials are on or off, they 
|> often occcur in trains of varying frequency.  The time-density of 
|> action potentials then forms an analog or continuous variable 
|> that may be processed in much the same way as a BP network.  

Yes, I always assumed this was the standard interpretation ... i.e. the
activation level of the perceptron is analagous to the *rate* of firing of a
neuron, not to the peak voltage of individual spikes. This seems pretty
obvious to me.

Larry Edwards
edwards@sunrise.stanford.edu
From: william@kub.nl (William Verkooijen)
Subject: relation between trainingset and number of weigths?
Date: Tue, 21 Jul 92 15:12:44 GMT
Organization: KUB, The Netherlands

Hello, 

I have the following question : Is there some theory about the
relation between the size of the training set and the maximum number
of connections (or weights) that can be used in a network, given some
prespecified accuracy level?

(p.s. with accuracy level I mean a  possibility  P that the weights
represent the distribution they should have when infinite data would
be  available)

or, summarized

How many connections can be used, given a training set of size N,
without overfitting the network ?
-- 
William Verkooijen
E-mail: william@kub.nl



Newsgroups: comp.ai.neural-nets
From: floydb@sun01.admin.rpi.edu (Barry B Floyd)
Subject: Re: Dynamically adding nodes, layers ?
Date: Fri, 31 Jul 1992 17:02:32 GMT

atan@park.bu.edu (Ah-Hwee Tan) writes:


>Hi! The Adaptive Resonance Theory (ART) systems including ART1, ART2,
>ART3, Fuzzy ART, ARTMAP, and Fuzzy ARTMAP, all create new recognition
>code (new node) when a new pattern does not fit into those learned
>patterns. They however, only add nodes but not layers.

>The cascade-correlation algo of Scott Fahlman adds a new node which
>also forms a new layer. The new node is created to reduce the
>residue error to the minimum.

>Cheers,

>AhHwee Tan
>Cognitive and Neural Systems, Boston University

Do you or anyone else have specific citations, references, bibliographies
for this topic and/or these authors?
 
thanks
 
please post to the list
 
barry



-- 
+--------------------------------------------------------------------+ 
| Barry B. Floyd                   \\\       barry_floyd@mts.rpi.edu |
| Manager Information Systems - Admin.\\\             floydb@rpi.edu |
+-Rensselaer Polytechnic Institute--------------------troy, ny 12180-+
Newsgroups: comp.ai.neural-nets
From: tsioutsias-dimitris@CS.YALE.EDU (Dimitris Tsioutsias)
Subject: Re: ALN vs BP
Organization: Yale University Computer Science Dept., New Haven, CT 06520-2158
Date: Fri, 24 Jul 1992 23:44:00 GMT


<<<<<<<<
From: tedwards@src.umd.edu (Thomas Grant Edwards)

In article <1992Jul24.053623.22636@cs.yale.edu> tsioutsias-dimitris@CS.YALE.EDU (Dimitris Tsioutsias) writes:
>It seems that after the backprop fans, we have now the ALN ones. Why
>each group (or any other that shows strong support) is trying to pass
>its nets as the dominant ones?

Yeah, but there are alot of people who want to throw nets at real problems
today, and it would be a little silly for anyone to expect gradient 
descent MLP nets at any real problem and expect results.
>>>>>>>>>>


	It depends on the 'real problem'; such a generalization is
	a bit too inappropriate. (Although, PURE gradient descent is
	definetely not the best thing to do in most situations.)

->dimitris
 
Newsgroups: comp.ai.neural-nets
From: jondarr@macadam.mqcs.mq.oz.au (jondarr c g h 2 gibb)
Subject: Back-percolation
Organization: Macquarie University, School of Mathematics, Physics, Computing and Electronics
Date: Wed, 29 Jul 1992 04:31:38 GMT

Hello all,
     whatever happened to back-percolation ? Nobody ever seems to
refer to it these days, and I haven't got any references on it.

Could some kind soul please send me a list of references and/or 
tell me why the interest seems to have died out ?

					jondarr


+----------------------------+-------------------------------------+
|    jondarr c g h 2 gibb    |   jondarr@macadam.mpce.mq.edu.au    |
|    Computing Department    |   jondarr@macgraw.mpce.mq.edu.au    |
|    Macquarie University    |                                     |
|    NSW, Australia, 2109    |   tel.  +61(2) 805 8414             |
+----------------------------+-------------------------------------+
Newsgroups: comp.ai.neural-nets
From: bdbryan@eng.clemson.edu (Ben Bryant)
Subject: Re: Neural Net for Matlab
Reply-To: bdbryan@eng.clemson.edu
Organization: College of Engineering, Clemson Univ.
Date: Thu, 23 Jul 1992 01:49:30 GMT

In article 35984@sdcc12.ucsd.edu, demers@cs.ucsd.edu (David DeMers) writes:
>In article <1992Jul22.134759.25636@uceng.UC.EDU> chan@uceng.UC.EDU (Alistair Keating Chan) writes:
>
>>Has anyone seen or written a standard backpropgation algorithm
>>for Matlab that they would like to and be able to share?
>
>No, but MathWorks sells a Neural Networks Toolbox, just out.  I don't 
>know the price.  I saw it demonstrated a couple of months 
>ago and it seems pretty spiffy.  Sorry, I don't have a
>better or more detailed review than that :-)
>
>However, standard bp is pretty easy to code up, in my opinion.
>Forward activation is just matrix multiply, squash, matrix multiply,
>squash...
>and standard delta rule learning is 
>compute delta, update weights, back prop (matrix mult),
>compute delta, update weights, back prop...
>
>Dave
>
>-- 
>Dave DeMers			 ddemers@UCSD   demers@cs.ucsd.edu
>Computer Science & Engineering	C-014		demers%cs@ucsd.bitnet
>UC San Diego					...!ucsd!cs!demers
>La Jolla, CA 92093-0114	(619) 534-0688, or -8187, FAX: (619) 534-7029

Close.

compute delta, back prop (if more than one hidden layer),
compute delta, back prop (through all layers)...
then update weights

-B.
From: arms@cs.UAlberta.CA (Bill Armstrong)
Newsgroups: comp.ai.neural-nets
Subject: Re: need for unique test sets
Date: 31 Jul 92 02:21:15 GMT
Organization: University of Alberta, Edmonton, Canada

hcbarth@afterlife.ncsc.mil (Bart Bartholomew) writes:

>In article <arms.712338868@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:

>	Friend Armstrong, I meant no attack on you or even
>ALNs.  I merely predicted your propensity to champion your
>cause.  I suspect the rest of the group has already noticed
>it. :)  I have no complaint; if you have something of value
>that you believe in, who could do otherwise?  Note that I 
>am experimenting with V2.6 to learn what I can from it.

Thanks for having a look at it friend Bart. As you know, the first
time around for ALNs, in the 70s, they and all other neural systems
were simply ignored.  It could happen again, unless problems in the NN
approach are overcome.  I respect the BP researchers for their efforts
to speed up training, to find a great variety of interesting
applications, and so on.  But I don't believe the field has a lot of
time left without new blood, or in this case, very old blood that
still has lots of oxygen in it.

>	BTW, how the %$#@ do you get it to save the trained
>net?  Nothing (Including RTFM, & reading the code) has worked
>so far.  I get a parsing error from LFEDIT no matter what
>I try.

There is a bug in lfBatch regarding changing directories.  That will
be corrected in version 2.7 coming out probably this week.  Maybe you
have found another bug.  On the surface though, it sounds like a
simple syntax error.  Maybe the quotes?

  save tree to "mynet.tre"

I noticed some email from Rolf, that may already have solved your problem.

Thanks for the feedback on possible logic net applications in the military,
or rather lack thereof.

Best wishes,

Bill

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: waterlan@eplrx7.es.duPont.com (Robert Waterland)
Subject: Address for Barak Pearlmutter
Organization: DuPont Engineering Physics Laboratory
Distribution: usa
Date: Tue, 21 Jul 1992 15:36:43 GMT

I need an email address or phone number for Barak Pearlmutter.  He used to
be at Carnegie Mellon University.

Thanks,
Rob Waterland, DuPont
From: marcus@sun-of-smokey.NoSubdomain.NoDomain (Jeff Marcus)
Newsgroups: comp.ai.neural-nets
Subject: Re: need for unique test sets
Date: 21 Jul 92 14:10:26 GMT
Organization: MIT/LCS Spoken Language Systems

In article <arms.711688181@spedden>, arms@cs.UAlberta.CA (Bill
Armstrong) writes:
<Deleted lots of stuff with which I agree>

|> Anyway, maybe everyone will disagree with me that either brand of
|> testing, with or without overlap, is still inadequate because:
|> 
|> a. without a priori knowledge, no one can know what the "correct"
|> function is, based on a fixed finite sample.
|> 
|> 
|> b. in general, no set of tests unless they cover the whole space
|> can assure that the neural net output will do what is correct
|> even if you know what "correct" means.  There has to be a proof technique
|> somehow.
|> 
|> Bill
|> 
|> 
|> --
|> ***************************************************
|> Prof. William W. Armstrong, Computing Science Dept.
|> University of Alberta; Edmonton, Alberta, Canada T6G 2H1
|> arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071

I guess I disagree with your use of the word "inadequate." But maybe we
are coming
from two different cultures. Because I am working in speech recognition,
where nobody
expects zero error rate or perfectly separable classes (and I assume the
same should
apply to OCR), I don't expect to attain zero error rate and show that my
classifier
has learned some generating function perfectly. Nor is it reasonable to
think of
all possible inputs since the pool of possible speech is infinite.

However, the more test data I have, the more sure I am that the
result on the test data is a good predictor of the classifier's performance on
new data. This can be quantified with a confidence interval on my
classification
error. This scheme is only inadequate in the sense that I can never
achieve
a zero confidence bound. But I don't see why that is important.

Of course, all this assumes that I have a good way of
selecting a representative sample in my test set, a non-trivial problem
in its own right.

The reason I say two cultures is that I am under the impression that neural net
researchers are also interested in learning whether their networks can discover
functions like XOR and maybe a performance metric like classification error is
inappropriate. However, my guess would be that as long as there is some
random process
behind generating the training and test sets, you can frame performance
estimation as
a statistical estimation problem and the same arguments I just made
would apply: namely,
that as you get more data, you get more confident of your result, but
you can never
be sure that your result is exactly right.

  Jeff
From: henrik@mpci.llnl.gov (Henrik Klagges)
Newsgroups: comp.ai.neural-nets
Subject: Re: neural nets and generalization (was Why not trees?)
Date: 24 Jul 92 16:03:27 GMT

paulking@next.com (Paul King) writes:
>of events transform an input pattern into an output pattern.  The
>"goal" of the neural net is not only to memorize the input-to-output
                                         ^^^^^^^^
>mappings, 

If the black box 'memorizes' the patterns (literally), you are lost,
as lookup tables are pretty useless. I found that high information 
compression rates (rule of thumb: # of float invals/# of float weights)
lead to good generalization. 

--

Cheers, Henrik
MPCI at LLNL
IBM Research
From: grau@tnt.uni-hannover.de (Oliver Grau)
Newsgroups: comp.ai.neural-nets
Subject: Software: Associative Memories
Date: 29 Jul 92 15:18:20 GMT
Reply-To: grau@tnt.uni-hannover.de
Organization: Universitaet Hannover, Theoretische Nachrichtentechnik


I'm searching ftp-able software for associative memories. The modeles which primary 
interest me are:

	BAM  	- bidirectional associative memory
	ABAM	- adaptive bidirectional associative memory
	FAM	- fuzzy associative memory

and
	sparsely coded associative memories


Has anyone on the net some usefull hints ?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         _                Dipl.-Ing. Oliver Grau
    ____/o\               Inst.fuer Theoretische Nachrichtentechnik 
   / ==  _/               University of Hannover       
  / ___/\\                email: grau@tnt.uni-hannover.de
  \__E   \                Phone: +49-511-762-5328
                          Fax  : +49-511-762-5333
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From: atan@park.bu.edu (Ah-Hwee Tan)
Newsgroups: comp.ai.neural-nets
Subject: Re: Dynamically adding nodes, layers ?
Date: 31 Jul 92 16:23:41 GMT
Organization: Boston University Center for Adaptive Systems
In-reply-to: scottd@cs.hw.ac.uk's message of 31 Jul 92 13:27:15 GMT


Hi! The Adaptive Resonance Theory (ART) systems including ART1, ART2,
ART3, Fuzzy ART, ARTMAP, and Fuzzy ARTMAP, all create new recognition
code (new node) when a new pattern does not fit into those learned
patterns. They however, only add nodes but not layers.

The cascade-correlation algo of Scott Fahlman adds a new node which
also forms a new layer. The new node is created to reduce the
residue error to the minimum.

Cheers,

AhHwee Tan
Cognitive and Neural Systems, Boston University
Newsgroups: comp.ai.neural-nets