HDTenEightyP/NetNews huggingface dataset:
From: aboulang@bbn.com (Albert Boulanger)
Newsgroups: comp.ai.neural-nets
Subject: Re: clustering, genetic algorithm
Date: 22 Aug 92 20:43:45 GMT
Reply-To: aboulanger@bbn.com
Organization: BBN, Cambridge MA
NNTP-Posting-Host: wilma.bbn.com
In-reply-to: whsu@rhino.ecn.purdue.edu's message of Tue, 11 Aug 1992 18:56:53 GMT

In article <1992Aug11.185653.2652@noose.ecn.purdue.edu> whsu@rhino.ecn.purdue.edu
(William Hsu) writes:


   I am interested to know of any reference of using genetic type algorithm to
   do clustering. Thanks
   --
   William Hsu Department of Electrical Engineering, Purdue University 
   West Lafayette, IN 47907.    	Tel : (317) 494-3471.
   Internet:	whsu@panther.ecn.purdue.edu
   uucp: ...!{decvax,hplabs,ihnp4,linus,rutgers,seismo}!pur-ee!whsu


Jim Kelly at BBN and Dave Davis at TICA played with a "Weighted"
k-nearest neighbors idea. (Stricly speaking this was a supervised
method.) The weights were optimised by a GA. Here is a reference:



  "Using Machine Learning Techniques to Visualize and Refine Criteria
  for Biological Integrity" (with Kenneth Anderson, Lawrence Davis,
  Herbert Gish, James Kelly, & Jeffrey Morrill). In Proceedings of
  Biological Criteria: Research and Regulation Symposium, US EPA, Office
  of Water, Washington DC, July 1991, 123-128.
From: jpj@jet.uk (Jean-Paul V Jeral)
Newsgroups: comp.ai.neural-nets
Subject: Re: Smell in Neural Network
Date: 19 Aug 92 20:00:27 GMT
Organization: Joint European Torus

- - -
Smell and NN:

You might try :

J. Ambros-Ingerson, R. Granger, G. Lynch,
"Simulation of Paleocortex Performs Hierarchical Clustering",
Science, Vol 247, 1344-1348 (1990).

BTW if someone knows of a clustering algorithm source based
on that work, please let me know!

Bye,
Jean-Paul Jeral (jpj@jet.uk)
- - -

- Disclaimer: Please note that the above is a personal view and should not 
  be construed as an official comment from the JET project.
Newsgroups: comp.ai.neural-nets
From: remek@mamura.ee.mu.oz.au (Remek Lipinski)
Subject: Question: Xerion simulator TRACE outputs
Organization: Electrical and Electronic - University of Melbourne 
Date: Wed, 12 Aug 1992 11:53:30 GMT

	Hi everyone, I have been using the Xerion simulator for a few days
and have a problem with outputing training information into a file.
Namely, I would like to know how to save the network error during training
for every iteratin. I know that it can be plotted using the Graph funcion
but I would like to be able to save that as ascii data. I am using the
backpropegation module for this. Any help is very appreciated, thanks.
===============================================================================
** Remek Lipinski   :  remek@mullian.ee.mu.OZ.AU                              **
**                                              "I'll be back!"  - T800      **
===============================================================================
Newsgroups: comp.ai.neural-nets
From: tbbusser@uokmax.ecn.uoknor.edu (Tod B Bussert)
Subject: SUMMARY:  ART2-A
Summary: Location for ART2-A article
Organization: Engineering Computer Network, University of Oklahoma, Norman, OK, USA
Date: Tue, 18 Aug 1992 23:36:52 GMT


---------------------- Original Posting ----------------------
> I am looking for the article by Carpenter, G.A., Grossberg, S., and
> Rosen, D (1991).  ART2-A:  An adaptive resonance algorithm for
> rapid category learning and recognition.  The last time I had heard
> it was submitted for publication but I do not know what journal.
--------------------------------------------------------------

Thanks to Bill Gnadt, wgnadt@park.bu.edu, who said:

> Tod--
>
>       That article was published in Neural Networks, Volume 4,
> pages 493-504.  Have fun!
> 
> Bill Gnadt
> CNS Dept., Boston University
From: Monroe.Thomas@f36.n342.z1.UUCP (Monroe Thomas)
Newsgroups: comp.ai.neural-nets
Subject: Re: Wild values
Date: 19 Aug 92 19:51:02 GMT
Organization: FidoNet node 1:342/36 - Through the Looking, Edmonton Alta

I think that the whole point of Bill's argument is that you can't
guarantee that the output of a BP net on an input between two
neighbouring training points a and b will lie between the true
function values f(a) and f(b), unless you carefully craft your network
using a priori knowledge of the function.

So, eliminating "wild values" that can exist between training points
in a BP net becomes:

    1)  a design process... or,

    2)  you can exhaustively test your net on all possible points on
        the input domain to ensure no "wild values" exist on output...
        or,

    3)  you can examine the weights of your BP net after training to
        ensure that they will not cause "wild values".

Option 2 is out of the question for large input spaces... option 3
gets real hard, real fast as the number of elements and layers
increase (ie, NP complete).

Does anyone have a "safe" design process, so that they can be ensured,
a priori, that their BP net, once trained, will never produce "wild
values"?  One partially satisfactory method would be to bound the
magnitude of the weights... but this can lead to poor function
fitting, and may prevent convergence.

-Monroe

 * OLX 2.2 * "I'm going to kill Dracula," said Tom painstakingly.
Newsgroups: comp.ai.neural-nets,sci.image.processing,sci.astro
From: dbriggs@zia.aoc.nrao.edu (Daniel Briggs)
Subject: Neural nets in astronomy (pointers sought)
Organization: National Radio Astronomy Observatory, Socorro NM
Date: Sat, 22 Aug 92 01:45:18 GMT

Well, it's journal club time around here.  I've been tapped to go learn
about something interesting and recent, and present it to the assembled
masses.  I noticed an interesting title in a conference proceeding and
declared that I would talk about 'Neural Networks in Astronomy'.  Well,
a few hours later in the library convinced me that this isn't exactly a
hot topic.  I could find darned little on the topic in the astronomy
literature beyond my original (tiny) paper.  (BTW, it is "Faint Object
Classification using Neural Networks", by Serra-Ricart et. al., from
the first ADASS meeting.  ADASS == Astronomical Data Analysis Software
and Systems I.  It's the Astronomical Soc of the Pacific Conference
series, volume 25.  Edited by Worrall, Biemesderfer and Barnes.)

Now it turns out that the recent neural network papers referenced in this
thing are mostly 'in press', or too recent for our library to have.  I'll
have a crack at a larger university library this weekend.  Can anyone give
me some footholds on the topic to start the reference chain?  This object
classification chain is pretty bare so far, though it might get better.
I'm sure there there is at least one group which has used a neural network
as part of the control system for an adaptive optics telescope, but I have
no references here -- any help?  I think that perhaps I've heard about
people experimenting with neural nets and deconvolution?  Any other obvious
applications that I've missed?  Pointers to papers on same?  How about
any particularly good introductions to neural networks, period.  It doesn't
have to be deep, but I should have at least *some* basic idea of what I'm
talking about.

Many thanks!

-- 
| Daniel Briggs  (dbriggs@nrao.edu)                      |        USPA B-14993
| New Mexico Tech / National Radio Astronomy Observatory |            DoD #387
| P.O. Box O / Socorro, NM 87801   (505) 835-7391        |
Support the League for Programming Freedom  (info from league@prep.ai.mit.edu)
Newsgroups: comp.ai.neural-nets
From: arms@cs.UAlberta.CA (Bill Armstrong)
Subject: Re: Reducing Training time vs Generalisation
Organization: University of Alberta, Edmonton, Canada
Date: Sat, 22 Aug 1992 20:18:45 GMT

(Quite a bit of the original has been deleted)

henrik@mpci.llnl.gov (Henrik Klagges) writes:

>Hm, Bill - this reasonability applies to your example problem as well, which
>is pretty much constructed ad hoc. Just add a training point between your 
>center - or remove the symmetry otherwise, and your extremum is gone (!).

Add a training point and the peak is gone -- true.  Changing the
symmetry does nothing to remove the extremum though.

Pardon a strategic hint: you should avoid weakly pejorative words like
"ad hoc" in describing the example.  You need stronger adjectives.
Even "irresponsible" or "absurd", which have been tried, failed to
damage it.

>Concerning lazy evaluation: It is difficult to program on parallel machines.
>It is a kind of runtime loadunbalancing that constantly switches off tasks
>(=subtree evaluations) of varying size essentially at random. No way to do
>that in SIMD, and darn difficult to do it in MIMD at all, and especially not
>with a high efficiency.

I agree if you only have one task.  Then you would try to use eager
evaluations to get speed, wasting resources.  But if you can timeshare
tasks and only do the necessary ones, you can come out ahead.  On the
Myrias SPS-x machines (MIMD), we have trained several trees at once on
up to 64 processors.  A one-line change to the code was required which
took 20 minutes first time.  However, that machine is perhaps not
typical of the programming effort required, since it makes certain
parallel speedups trivial to program.  Execution of trained ALNs would
not have benefitted due to the exceedingly small granularity of the
task.

Efficiency on the Myrias was about 85%, which was felt to be very
good.

--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
arms@cs.ualberta.ca Tel(403)492 2374 FAX 492 1071
Newsgroups: comp.ai.neural-nets
From: ab509523@longs.LANCE.ColoState.Edu
Newsgroups: comp.ai.neural-nets
Subject: Looking for a Recurrent Net algorithm
Date: 13 Aug 92 19:40:44 GMT
Reply-To: ab509523@longs.LANCE.ColoState.Edu ()
Organization: Colorado State University, Fort Collins, CO  80523


 I'me looking for a recurrent net algorithm as proposed by Pearlmutter.

  The only reference I could come across was :
   Pearlmutter, B.A. - "learning state space trajectories in recurrent neural networks: a  preliminary           report"  (Tech. Rep. AIP - 54 ).  CMU , Dept. of Computer science.
 
  Unfortunately I have been unable to locate this report or any other reference to this 
  algorithm. 
    I would be grateful to anybody who could provide me with other references or the algorithm.

    thanks in advance. 
Newsgroups: comp.ai.neural-nets
From: tmb@arolla.idiap.ch (Thomas M. Breuel)
Newsgroups: comp.ai.neural-nets
Subject: Re: Characterizing learnable functions
Date: 12 Aug 92 21:13:33 GMT
	<1992Aug11.111206.25386@cs.tu-berlin.de> <arms.713550511@spedden>
	<1992Aug12.112845.1060@cs.tu-berlin.de>
Reply-To: tmb@idiap.ch
Organization: IDIAP (Institut Dalle Molle d'Intelligence Artificielle
	Perceptive)
In-reply-to: async@opal.cs.tu-berlin.de's message of 12 Aug 92 11:28:45 GMT

In article <1992Aug12.112845.1060@cs.tu-berlin.de> async@opal.cs.tu-berlin.de (Stefan M. Rueger) writes:

   >Maybe consumers of NNs should ask for and insist on Lipschitz conditions:
   >   |output(a) - output(b)| <=  const * | a - b |
   >
   >for all a, b in the domain of the function. 

   These Lipschitz-functions **are** continuous and thus covered  by the
   results of Cybernko, described in my original article.

Unfortunately, imposing a Lipschitz condition is not enough. While a
neural network might be able to approximate any Lipschitz function,
the number of training examples (the "sample complexity") you need for
actually finding a "good approximation" is provably too large in many
cases.

Proving that a learning method is capable of representing a result is
only a first step, and not even a decisive one. For example, you may
be able to do most natural language parsing with small, finite state
automata, even though they are theoretically not sufficiently to parse
natural language. Conversely, even if your learning method can
represent some result, you still need to consider questions of sample
complexity and algorithmic efficiency.

					Thomas.
From: demers@cs.ucsd.edu (David DeMers)
Newsgroups: comp.ai.neural-nets
Subject: Re: Reducing Training time vs Generalisation
Keywords: back propagation, training, generalisation
Date: 18 Aug 92 04:08:56 GMT
Organization: =CSE Dept., U.C. San Diego

In article <arms.714091659@spedden> arms@cs.UAlberta.CA (Bill Armstrong) writes:
...

>PS I hope you don't take the "violins" comment too much to heart, but
>the truth is that with a least squared error criterion on the training
>set, I can get the optimal learned function to create a disaster very
>easily.

No offense, certainly.  I guess I just don't understand what you
mean by "disaster" nor what you've meant in previous postings
about "wild" results...  

-- 
Dave DeMers			 ddemers@UCSD   demers@cs.ucsd.edu
Computer Science & Engineering	C-014		demers%cs@ucsd.bitnet
UC San Diego					...!ucsd!cs!demers
La Jolla, CA 92093-0114	(619) 534-0688, or -8187, FAX: (619) 534-7029
From: jason@lanai.cs.ucla.edu (Jason Rosenberg)
Newsgroups: comp.ai.neural-nets
Subject: Need electrotonics explained
Date: 17 Aug 92 22:47:06 GMT
Organization: UCLA Computer Science Department
Originator: jason@lanai.cs.ucla.edu

Well, I'm trying to educate myself about the neurophysiological branch of
connectionism, and I am coming to a few road-blocks in reading the literature.

The term "electronic" is puzzling me a bit.  It appears to refer to a voltage
gradient within a neural pathway (i.e. along a dendrite).  Is this correct?
I would appreciate a concise (if possible) explanation as to the use of
"electronics" in the literature, and it's significance towards neural modelling
and analysis.

Thanks....

Jason


-- 
Jason Rosenberg                           Computer Science Department
jason@cs.ucla.edu                         University of California
{uunet,rutgers,ucbvax}!ucla-cs!jason      Los Angeles, CA  90024
Newsgroups: comp.ai.neural-nets,comp.ai,comp.compression,comp.theory
From: sethb@fid.morgan.com (Seth Breidbart)
Subject: Re: measuring complexity (was Re: Kolmogorov Complexity)
Organization: Morgan Stanley & Co., New York, NY
Date: Sat, 15 Aug 1992 02:54:43 GMT

In article <1992Aug14.212005.3438@julian.uwo.ca>
koops@asterix.gaul.csd.uwo.ca (Luke Koops) writes:

>	Also, I beleive that K-complexity is not unbiased, but is
>biased towards Turing machine representations of concepts.  It just
>provides a really good estimate of complexity.  This is merely my
>opinion, I would appreciate it if someone could confirm it.

K-complexity is defined relative to any universal representation,
which can be Turing Machines, APL programs, 2-counter machines, ...
The difference between the complexity of all strings in any two models
is bounded by a constant.  Why do you feel that measuring complexity
of a string as the length of the shortest C program that produces that
string to be biased towards Turing Machines?

Seth		sethb@fid.morgan.com
Newsgroups: comp.ai.neural-nets,comp.ai,comp.theory
From: jamie@cs.sfu.ca (Jamie Andrews)
Subject: Re: measuring complexity (was Re: Kolmogorov Complexity)
Organization: CSS, Simon Fraser University, Burnaby, B.C., Canada
Date: Mon, 17 Aug 1992 17:19:34 GMT

     As far as I can tell, there is no mathematical reason why
K-complexity for learning applications should be based on any
one computational formalism (Turing machines, assembler code, C,
etc).  However, for the sake of consistency of notation, it
might be better to base it on some programming language with a
simple and standard semantics, such as pure LISP or pure Prolog.

     So, for instance, the K-complexity of a given list of
numbers might be defined as:  the length of the shortest program
in pure LISP such that the value of the s-expression "(result)"
is that list.  (Well, I guess it would have to be pure LISP plus
some arithmetic functions.)  The "length" of a program might be
defined as the total number of atoms plus the total number of
sets of parentheses, to give a more realistic idea of the
storage space needed by the program.

     A measure of K-complexity relative to some standard suite
of function definitions could be given, too.  The relative
K-complexity would be the length of the shortest pure LISP
program containing that suite of definitions, _minus_ the length
of those definitions.  The minimal program could then make use
of the "knowledge" in the standard functions without having to
derive it.

     By the way, I asked about K-complexity here a few years
ago, and was given convincing arguments that finding the
K-complexity of a given string is in general undecidable.
I don't know about randomized algorithms, though.

--Jamie.
  jamie@cs.sfu.ca
"Every \item command in item_list must have an optional argument." LaTeX pg.168
Newsgroups: comp.ai.neural-nets
From: smb3u@squeeg.psyc.virginia.edu (Steven M. Boker)
Subject: Re: Smell in Neural Network
Organization: University of Virginia, Department of Psychology
Date: Tue, 18 Aug 1992 18:03:46 GMT

In article <1476@eagle.ukc.ac.uk> aw3@ukc.ac.uk (Alan Wilson) writes:
>In article <1992Aug18.115759.21078@csd.brispoly.ac.uk> c_tsai@csd.brispoly.ac.uk (C Tsai) writes:
>>
>>  I think that Smell in Neural Network is a very interesting project which can
>>be research.  If someone knows about 'Smell in Neural Network' or where I can
>>find some articles about this, can you tell me??????? Thinks a lot!!!!!!
>>
>Many researchers in the chemical sensors field are waking up to the idea
>that their sensors are by no means perfect and are using the ever-increasing
>computational power of computers to compensate for the physical defects
>(lack of specificity in particular). Neural nets/pattern recognition is one
>way ahead, the traditional approach is to use an array of non-selective
>sensors and process the outputs (cluster analysis, principal components
>etc.)
>

You might look into the work of Walter Freeman who has done a lot of
work in modeling olfaction.  A good reference here is

@ARTICLE{Skarda87a,
	AUTHOR = {Skarda, Christine A. and Freeman, Walter J.},
	TITLE = {How brains make chaos in order to make sense of the world},
	JOURNAL = {Behavioral and Brain Sciences},
	VOLUME = {10},
	PAGES = {161--195},
	YEAR = {1987}
}

There are several articles in the NIPS proceedings.  I would suggest looking
at Neural Information Processing Systems 2 which has:
"Associative Memory in a Simple Model of Oscillating Cortex" by Bill Baird
and "Computer Simulation of Oscillatory Behavior in Cerebral Cortical
Networks" by Matthew Wilson and James Bower.

There is also a new book out that I haven't had a chance to plow through
called "Olfaction: A Model for Computational Neuroscience" by
Joel L. Davis and Howard Eichenbaum.

The distributed nature of olfactory perception has forced its students
to consider the behavior of large scale networks of neurons rather than
the retinotopic or tonotopic approaches that had been fruitful for some
time in the visual and auditory fields.

Good luck!

Steve


-- 
 #====#====#====#====#====#====#====#====#====#====#====#====#====#====#====#
 #  Steve Boker           #             "Two's bifurcation                  #
 #  smb3u@virginia.edu    #             but three's chaotic"                #
 #====#====#====#====#====#====#====#====#====#====#====#====#====#====#====#
Xref: sparky comp.ai.neural-nets:3169 comp.ai:3074 comp.compression:2993 comp.theory:1751
From: tomc@siesoft.co.uk (Tom Crossland)
Newsgroups: comp.ai.neural-nets,comp.ai,comp.compression,comp.theory
Subject: Re: measuring complexity (was Re: Kolmogorov Complexity)
Date: 13 Aug 92 10:45:11 GMT
Reply-To: tomc@siesoft.co.uk
Organization: Siemens Nixdorf Information Systems Ltd.

dzik@access.digex.com (Joseph Dzikiewicz) writes:

: Actually, unless you have an assembly language that allows addressing
: infinite memory, even an assembly language program has a finite
: number of states (a large number, admittedly, but still finite).

If  you  used indirect addressing it could be overcome. ie, if
your  instruction  pointed to a block of memory, which in turn
had  a  flag  and the offset of another block of memory, which
also  had  a  flag  and the offset to another block of memory,
etc,  etc,  etc you could in theory address an infinite amount
of  memory  (OK  so  it  might  take  an  infinite  number  of
instruction executions, but that's a small price to pay ;-)).

--
___ _ __
  ((_))))...
From: y_jin@csd.brispoly.ac.uk (Y Jin)
Newsgroups: comp.ai.neural-nets
Subject: Question: Real Time Local Learning and Global Learning
Date: 18 Aug 92 11:40:05 GMT
Organization: Bristol Polytechnic, England

Question: Real Time Local Learning and Global Learning

We are applying BP neural networks in nonlinear control problems. The work is
concentrated on real time learning. Relative references and suggestions will
be very appreciated. The main idea of our research is presented below:

Supposed that a BP neural network nearly approaches the desired nonlinear
function through off-line learning. The real time (on-line) learning is used
to improve the approximation accuracy. The neural network could increase the
real-time approximating speed locally by increasing the learning rate. However
by doing so the neural network global approximation features (gained in off-
line learning) will be damaged. We suggest that the neural network weights be
split into two parts: linear part for local approximation and nonlinear part
for global approximation. It is easy to develop a learning algorithm for linear
part weights, which results in stable control systems. The nonlinear learning
slowly shifts linear part weights to nonlinear part weights.

Please email ( y_jin@csd.brispoly.ac.uk), I will summarize.