HDTenEightyP/NetNews huggingface dataset:
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Network Inversion
Organization: School of Computer Science, Carnegie Mellon
Date: Sat, 22 Aug 1992 03:30:45 GMT


    From: tjochem+@CS.CMU.EDU (Todd Jochem)

    I'm looking for references to network inversion papers. The basic idea
    I'm interested in is presenting the network's output with a signal and
    by back-propagating this signal though the layer(s), recreating an input
    which could have created the applied output signal.

Most backprop-type networks are information-losing systems: many distinct
inputs give rise to the same output.  You can't invert this kind of net
into another feed-forward net (of the usual kind) because you just end up
with a mushy average of the possible inputs for a given output.

-- Scott
===========================================================================
Scott E. Fahlman
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213

Internet: sef+@cs.cmu.edu
From: henrik@mpci.llnl.gov (Henrik Klagges)
Newsgroups: comp.ai.neural-nets
Subject: Re: Reducing Training time vs Generalisation
Keywords: back propagation, training, generalisation
Date: 17 Aug 92 16:26:51 GMT

arms@cs.UAlberta.CA (Bill Armstrong) writes:

>It would be interesting to hear Scott Fahlmann's ideas on how to get
>good generalization.  Then you might find out why you had problems
>using Cascade Correlation.

Could you elaborate a little bit on that point (why you had problems)
about CC ? 

Cheers, Henrik

IBM Research
Massively Parallel Group at Lawrence Livermore
Xref: sparky comp.ai.neural-nets:3177 comp.ai:3080 comp.theory:1754
From: peyote@umiacs.umd.edu (Gary W. Flake)
Newsgroups: comp.ai.neural-nets,comp.ai,comp.theory
Subject: Why AI people shouldn't talk about :(measuring complexity)
Date: 13 Aug 92 17:01:24 GMT
Followup-To: comp.ai.neural-nets
Organization: UMIACS, University of Maryland, College Park, MD 20742



I just thought that someone should point out that the discussion
taking place is full of holes.  People, please read-up on terms such
as "finite-state-machines", "infinite-state-machines", and "Turing
machine" before using them in a discussion such as this.  I know that
your intensions are good, as are your ideas, but every post made so
far has had a mistake, if only in the use of the terms.

Gary



-- 
Spoken: Gary W. Flake  Domain: peyote@umiacs.umd.edu  UUCP: uunet!mimsy!peyote
Phone: +1-301-405-6757 USPS: UMIACS, U. of Maryland,  College Park, MD 20742
Newsgroups: comp.ai.neural-nets
From: william@kub.nl (William Verkooijen)
Subject:   NN in real valued applications
Date: Wed, 19 Aug 92 15:29:55 GMT
Organization: KUB, The Netherlands

Hi,

Neural Nets were originally used for pattern recognition, often in a
0-1 coded problem setting. Does anyone know, who has applied Neural nets
to real valued prediction problems? 

ps:
The qualities  a neural net has to have, are quite different for both
problem areas. 
-- 
William Verkooijen
E-mail: william@kub.nl



Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Perceptrons
Organization: University of Alberta, Edmonton, Canada
Date: Thu, 13 Aug 1992 18:44:35 GMT

robert@cs.strath.ac.uk (Robert B Lambert) writes:

>To the NN community in general.
>To what extent to the objections raised in perceptrons carry over to ANNs
>other than BP? (In particular self-organizing NNs, recurrent NNs, ALNs).
> ... I believe ANNs only have a future if all the principal objections 
>raised in Perceptrons are addressed.

Since ALNs are special cases of multilayer perceptrons, of course they
can't have any greater theoretical computing power than the usual
multilayer perceptrons, whether used recurrently or not.  On the other
hand, with appropriate encodings of real numbers, ALNs can approximate
at least any continuous function (like those output by an ordinary
multilayer perceptron, say).  The two thus have equivalent power to
realize mappings in the finite digital domain.

In"Perceptrons", Minsky and Papert don't place emphasis on speed of
learning or execution, an attribute that may be considered an
advantage of one or the other type of net in a given practical
situation.  They were concerned about the limits of what can be
computed.  The backpropagation and ALN training procedures have made
it possible to train multilayer nets since they wrote the book, but
that doesn't give the resulting nets any more computing power.
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: ajr@eng.cam.ac.uk (Tony Robinson)
Newsgroups: comp.ai.neural-nets
Subject: Re: Temporal networks
Date: 13 Aug 92 14:19:32 GMT
Organization: Cambridge University Engineering Department, UK

In comp.ai.neural-nets bdbryan@eng.clemson.edu writes:
>I have recently been doing some work with speech recognition, and have used a
>few network configurations in my search for "the perfect network architecture".
>I wanted to find out from people in this group what some of the best networks
>for sequence recognition are?

Sequence recognition or speech recognition?  I'll design you a HMM-net for
sequence recognition that will be optimal (according to my definition :-).
For speech recognition, you (and everyone else) are welcome to look at the
reports on recurrent nets for phone/word recognition available by anonymous
ftp from svr-ftp.eng.cam.ac.uk in directory "reports".

Tony [Robinson]
Newsgroups: comp.ai.neural-nets
From: aw3@ukc.ac.uk (A.Wilson)
Newsgroups: comp.ai.neural-nets
Subject: Re: Smell in Neural Network
Date: 18 Aug 92 15:49:06 GMT
Reply-To: aw3@ukc.ac.uk (Alan Wilson)
Organization: Chemical Laboratory, University of Kent at Canterbury, CT2 7NH, UK

In article <1992Aug18.115759.21078@csd.brispoly.ac.uk> c_tsai@csd.brispoly.ac.uk (C Tsai) writes:
>Hello everybody!!!!!
>
>   Who knows 'smell' in Neural Network????  As I know that now only some people
>research 'Speech' or 'Image Processing' in Neural Network.  Why nobody research
>about 'Smell' in Neural Network!!!!
>
>   I think that If we use neural network to let computer thinking what is good
>smell or what is bad smell ( for example: A delicious food should be very good
>smell ), so How we can let computer to know and think this smell problem????
>
>   If we can go to out space,like moon or any stars, how we can let computer to
>learn "this star can let human to live or not", this must depend on how much 
>oxygon or gas is in this star????? 
>
>
>  I think that Smell in Neural Network is a very interesting project which can
>be research.  If someone knows about 'Smell in Neural Network' or where I can
>find some articles about this, can you tell me??????? Thinks a lot!!!!!!
>
>
>
>Chin-Yu Roman TSAI
>
>My Email address: c_tsai@csd.brispoly.ac.uk

Many researchers in the chemical sensors field are waking up to the idea
that their sensors are by no means perfect and are using the ever-increasing
computational power of computers to compensate for the physical defects
(lack of specificity in particular). Neural nets/pattern recognition is one
way ahead, the traditional approach is to use an array of non-selective
sensors and process the outputs (cluster analysis, principal components
etc.)

The work is being published in several journals, eg Sensors and Actuators,
Analytical Chemistry, Measurement Science and Technology. I have more
detailed references if anyone is interested.

Dr. Alan Wilson
Chemical Laboratory, UKC

--
Alan Wilson        (insert YOUR favourite witty remark..........)

"You would NOT believe what has been going on behind there" 
================================================================= 
Newsgroups: comp.ai.neural-nets
From: mpp@cs.brown.edu (Michael P. Perrone)
Subject: Re: Wild values (was Reducing Training time ...)
Organization: Center for Neural Science, Brown University
Date: Thu, 20 Aug 1992 21:29:05 GMT

 >A wild idea for people trying to avoid wild values (e.g. for safety
 >critical applications etc.): Once the network has been trained and the
 >weights are fixed, it should be possible to determine the maximum and
 >minimum output values for all inputs. This can be done with normal
 >gradient ascent and descent. Simply calculate partials of the output(s)
 >with respect to the input(s). Convergence should be much faster than
 >training networks in the first place due to the generally smaller number
 >of inputs than weights. Multiple runs from different starting positions
 >or the use of stochastic techniques likely to converge to global
 >maxima/minima can reduce the chance of not seeing a wild value that
 >actually exists. With a small number of inputs (definitely 1, maybe
 >a more) analytical techniques should be able to provably determine the
 >global maximum and minimum.

your idea is not wild at all.  it has been used by statisticians to 
evaluate the effect of various inputs on statistical models.
Newsgroups: comp.ai.neural-nets
Subject: Reducing Training time vs Generalisation
From: edwin@ccu1.aukuni.ac.nz (Edwin Ng)
Date: Sun, 16 Aug 1992 21:39:39 GMT
Organization: University of Auckland, New Zealand.
Keywords: back propagation, training, generalisation

koops@gaul.csd.uwo.ca (Luke Koops) writes:

>A while ago I posted a request asking how I could reduce the training
>time for a back-propagation neural network, and recieved these helpful
>respones.

... (summary deleted)

>   -Luke Koops (koops@csd.uwo.ca)


>-- 
>    /
>***/
>*\/* koops@csd.uwo.ca
>****

Thanks for the summary Luke.  I'd like to ask if anyone has 
anything to add about the quality of generalisation 
resulting from using different parameters to speed up
training??

I have only tried playing with different learning rates
with my data for image recognition work.  I found that
while I can use values of 0.1 or more to get quick
learning of the training set, I get high classification
errors in the test set.

I ended up using a learning rate of 0.001 which amounted
to very tedious training in order to good generalisation.

Does anyone have any advice on how I can speed up training
without losing generalisation?  Or is this a tradeoff
that can't be changed (some kind of conservation law) ?

I have tried using Scott Falman's Cascade Correlation but 
the generalisation was much worse than backprop although 
it learnt very quickly.

Any input from you fellow netters out there would be much
appreciated.  Thanks.

-----------------------------------------------------
* Edwin Ng                                          *
* Department of Electrical & Electronic Engineering *
* University of Auckland                            *
* Private Bag, Auckland                             *
* NEW ZEALAND                                       *
*                                                   *
* Fax: 64 9 366 0702                                *
* E-mail: edwin@ccu1.aukuni.ac.nz                   *
-----------------------------------------------------
From: henrik@mpci.llnl.gov (Henrik Klagges)
Newsgroups: comp.ai.neural-nets
Subject: ALN's FAIL ON 1-BIT ERRORS ! DISASTER IMMINENT ?
Date: 20 Aug 92 20:18:03 GMT

[Just to retaliate a little bit against Bill's nn-bashing 8-)]

The Neuronal Enquirer, 8/20/2002

ALN's FAIL TERMINALLY ON SINGLE BIT ERRORS ! SCIENTISTS PARALYZED !
According to informed sources, ALN's have a much hyped feature, 
lazy evaluation that allows the switchoff of subtrees of any size
by a single logic gate ! Pathological information or single bit 
failures could easily disable the major parts of the ALN, thus 
loosing all the recognition capabilities of these subtrees.

Numerous rumours indicate major impacts have already happened. The
Pentagon's B2.5 - super stealth bomber (it is so stealthy that even
at a price tag of $2e^9 it cannot be found in the federal budget !)
is grounded ! Many more marriage rings than usual vanished in kit-
chen sinks. U.S. trade negotiators were quoted 'We'll be real tough
with them Japanese this time to bring our domestic semiconductor 
marketshare back over 7.5%'. The U.S. budget deficit for the first
time exceeded the G.N.P., leading to the resignation of the complete
board of Harvard Business school ...

Cheers, Henrik
IBM Research Division physics group Munich
massively parallel group at Lawrence Livermore
Newsgroups: comp.ai.neural-nets
Subject: Re: Why AI people shouldn't talk about :(measuring co
From: swann@divsun.unige.ch (SWANN Philip)
Date: 14 Aug 92 14:34:05 +0200
Distribution: world
Organization: University of Geneva, Switzerland

In article <59656@mimsy.umd.edu>, peyote@umiacs.umd.edu (Gary W. Flake) writes:
> 
> 
> I just thought that someone should point out that the discussion
> taking place is full of holes.  People, please read-up on terms such
> as "finite-state-machines", "infinite-state-machines", and "Turing
> machine" before using them in a discussion such as this.  I know that
> your intensions are good, as are your ideas, but every post made so
> far has had a mistake, if only in the use of the terms.
> 
> Gary
> 
> 

I agree: "The Turing Machine. Let us be a little more precise. A Turing
machine M involves: (1) A finite alphabet of symbols; (2) a finite set
of [internal] states....." (Harel's "Algorithmics" p. 215 ... or any other
introductory text."

Philip Swann
University of Geneva

p.s. It's actually the "intensions" that are bad, while the "intentions"
may be good.  %-))
From: davidz@cae.wisc.edu (David Zhang)
Newsgroups: comp.ai.neural-nets
Subject: Re: Neural Networks as Continuous Filters
Date: 14 Aug 92 16:57:30 GMT
Organization: U of Wisconsin-Madison College of Engineering

In article <988@engcon.marshall.ltv.com> engcon!fenner@uunet.uu.net (Jon Fenner) writes:
>I am trying to find references to and information on using neural networks
>as continuous filters.  I know work has been done in this area, but I have
>not been able to locate much.  Any help would be greatly appreciate.
>Thanks in advance.

IEEE Trans. On Biomedical Engin. Vol 39, No. 4 Apr. 1992
<NN-based Adaptive Matched Filtering for QRS detection>

by Xue and Hu
From: kpfleger@NEURON.SIEMENS.COM (Karl Pfleger)
Newsgroups: comp.ai.neural-nets
Subject: Re: Wild values (was Reducing Training time ...)
Date: 20 Aug 92 15:51:31 GMT

Newsgroups: comp.ai.neural-nets
Subject: Re: Wild values (was Reducing Training time ...)
Summary: 
Expires: 
Followup-To: 
Distribution: 
Keywords: back propagation, training, generalisation

1 quick point and 1 wild idea:

First, if one desires to avoid wild output values for certain regions of
input space, one ought to have training pairs from that region of input
space in the training set. The point about desiring certain behavior on
0 to 1 and not including any training pairs from that region has already
been made.

Vaguely similar: since the inputs that will be thrown at the system in actual
use will have some probability distribution based on whatever the system is
doing, the training set should be generated by sampling the same
distribution, or something as close to it as possible, NOT by picking a
few values by hand or by using a lattice or points regularly spaced
(unless that represents the distribution well).

I have a much more difficult time picturing wild values coming from a
network trained on a significant number of random, real inputs than I do
coming from a network trained on a handful of regularly spaced
integers.


A wild idea for people trying to avoid wild values (e.g. for safety
critical applications etc.): Once the network has been trained and the
weights are fixed, it should be possible to determine the maximum and
minimum output values for all inputs. This can be done with normal
gradient ascent and descent. Simply calculate partials of the output(s)
with respect to the input(s). Convergence should be much faster than
training networks in the first place due to the generally smaller number
of inputs than weights. Multiple runs from different starting positions
or the use of stochastic techniques likely to converge to global
maxima/minima can reduce the chance of not seeing a wild value that
actually exists. With a small number of inputs (definitely 1, maybe
a more) analytical techniques should be able to provably determine the
global maximum and minimum.

This idea's usefulness is limited by a number of requirements such as 
fixed weights and wild values meaning large or small, but still seems
like it should have fairly wide applicability.

-Karl		kpfleger@learning.siemens.com
Newsgroups: comp.ai.neural-nets
From: pcm@cs.brown.edu (Peter C. McCluskey)
Subject: Re: Looking for a Recurrent Net algorithm
Organization: Brown Computer Science Dept.
Date: Sun, 16 Aug 1992 14:09:45 GMT

In article <Aug13.194044.23182@yuma.ACNS.ColoState.EDU>, ab509523@longs.LANCE.ColoState.Edu writes:
|> 
|>  I'me looking for a recurrent net algorithm as proposed by Pearlmutter.

See:
Barak A. Pearlmutter, "Learning State Space Trajectories in Recurrent
Neural Networks" in Neural Computation, vol 1, pp 263-269.

pearlmutter.dynets.ps.Z in the Neuroprose archives
-- 
         hardcore signature virus: "As a juror in a Trial by Jury, 
         you have the right, power and duty to acquit the defendant 
                 if you judge the law itself to be unjust."
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Reducing Training time vs Generalisation
Organization: University of Alberta, Edmonton, Canada
Date: Sat, 22 Aug 1992 20:58:58 GMT

henrik@mpci.llnl.gov (Henrik Klagges) writes:

>arms@cs.UAlberta.CA (Bill Armstrong) writes:

...

>>coming closer all the time to a logical net.  If you are able to
>>replace sigmoids with sharp thresholds, and not change the output of
>>the net significantly, then you are really using threshold *logic*
>>nets.  

>Well, if this replaceability is there ... would be great ! Don't 
>think so, though. I'd need a few more experiments on that. 
>An inital look suggests that the weights & sigmoids cannot easily
>(straightforwardly) replaced with 'gates'.

For training, maybe you have a problem.  ALNs don't.  So we can train
to get piecewise constant functions.  The final step is smoothing,
which is fairly straightforward, if not trivial, and is efficient.

Bill


--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: gat@forsight.jpl.nasa.gov (Erann Gat)
Subject: Re: measuring complexity (was Re: Kolmogorov Complexity)
Organization: Jet Propulsion Laboratory
Date: Thu, 13 Aug 1992 16:45:27 GMT

In article <1992Aug12.150857.219@access.digex.com> dzik@access.digex.com (Joseph Dzikiewicz) writes:
>Actually, unless you have an assembly language that allows addressing
>infinite memory, even an assembly language program has a finite
>number of states (a large number, admittedly, but still finite).

It has nothing to do with the language.  Any assembly language can
be used to control a tape drive (where have I heard that idea
before?)  Of course, tapes are finite - but wait!  Let's build
a tape drive with a tape-making machine attached to it.  Of course,
there is only a finite amount of raw material on the planet with
which to make tape...

My point is that the limiting factor on building a Turing Machine is
the size of the Universe, not the design of an assembly language.
Turing Machines are a convenient mathematical fiction which we use
to model certain physical phenomena.  The model is inaccurate at certain
extremes, but so are most mathematical models of the world.

Erann Gat
gat@robotics.jpl.nasa.gov

Newsgroups: comp.ai.neural-nets
From: yil@engr.ucf.edu (Yilmaz Cengeloglu)
Subject: HELP: Automated Air Traffic Controlling and Blackboard System
Organization: engineering, University of Central Florida, Orlando
Date: Fri, 14 Aug 1992 18:30:12 GMT



Hi :

I have asked help about Blackboard Architecture couple of months ago.
I have received some responses. Thanks to everyone who helped me. I have
the list of several references related with BBA. If any anyone needs please 
let me know. 

Currently I am still working on my project for my thesis and I neen some more
help. I am explaining what I need first. If you are interested what I am doing
you can continue to read after line. 

I will write simple knowledge bases to solve the conflict and other problems that
occurs at airspace. I do need some help to complete this project and graduate.

	If you know any publication related with
			- Automated Air Traffic Controlling,
			- books that I can get expertise to write knowledge bases       
			  to solve the problems at airspace.
			- Blackbard Systems AND Air Traffic Controlling. 

 	 please let me know. I will keep all information that I will receive
         and I will send anyone request later. 
											  

	  If you know any existing system, project, company related with
	  Automated Air traffic controlling, please let me know. It will
	  be very helpful for my project.

				                           Yilmaz Cengeloglu
				                           Univ. of Central Florida
							   Dept. of Computer Eng. 
							   Orlando, FL    

				                           yilmaz@ists.engr.ucf.edu
							          or
							   yil@engr.ucf.edu


---------------------------------------------------------------------------------

I am preparing prototype Automated Air Traffic Controlling (A-ATC) system to 
prepare and test knowledge bases for each plane in the airspace. I did need a Blackboard
shell for my project. Only commercial system that I found was GBB that marketed by 

Blackboard Technology Group, Inc.;
401 Main St., Amherst, MA  01002, 413/256-8990,
fax: 413/256-3179.

Their price was too high for me to buy as a graduate Student. I have implemented my 
own Blackboard System on SunOS operating system. ( I am not really sure if I should
call Blackboard System.) Different knowledge bases in CLIPS working as a different 
process on the background and uses Blackboard when it is necessary to share fact with 
other knowledge sources. Of course, any other process in C, C++, etc can access the
Blackboard. The type of architecture is easy to distribute more than one computer
using computers networks. If I would continue the project after I got my M.S.
I am still testing it.  

Also, I did implemented a Radar Simulator Using C/C++, X-Windows and IPC facilities
on the SunOS operating system.  Radar simulator is being used by Intelligent Simulation
and Training systems Project at UCF.

A-ATC project is integration of Radar Simulator and Blackboard System.


Newsgroups: comp.ai.neural-nets
From: mike@psych.ualberta.ca (Mike Dawson)
Subject: Re: Reducing Training time vs Generalisation
Keywords: back propagation, training, generalisation
Organization: Psychology, University of Alberta, Edmonton
Date: Mon, 17 Aug 1992 16:01:57 GMT

Bill Armstrong's point about defining generalization is an extremely
good one.  In my view, most ANN researchers view generalization from
an "output performance" perspective, as noted by Bill:  you train a
net on a subset of possible stims, and then measure the network's
responses to the remaining ones.

In a 1989 review of pattern classification by ANNs, Lippmann points
out an equally valid notion of generalization, in terms of "learning
performance".  Specifically, you train a network on some subset of
stims, and then train the network on the remaining patterns.  The
issue is whether learning on the new sets of patterns is helped
in any way from the previous training.  In the memory literature in
cognitive psychology, this effect is called "savings".

Given that it may be extremely difficult to interpolate judgements
about a function because of its complexity, or potential discontinuity,
it may be that generalization of neural networks is more appropriately
viewed from this savings perspective.
--
Michael R.W. Dawson                       email: mike@psych.ualberta.ca
Biological Computation Project, Department of Psychology
University of Alberta, Edmonton, AB CANADA T6G 2E9
Tel:  +1 403 492 5175   Fax: +1 403 492 1768