From: muttiah@stable.ecn.purdue.edu (Ranjan S Muttiah)
Newsgroups: comp.ai.neural-nets
Subject: Feature selection of NNs (Take two)
Date: 8 Apr 92 05:30:29 GMT
Organization: Purdue University Engineering Computer Network

Let me restate again what I had originally intended.  Given any n-features
that have varying degrees of "importance" (and features that could also be 
context sensitive), how do I cluster them onto an output map that reflects their 
relative import ?  A priori, I don't know how important each feature is.  I
want the neural network to tell me that.  I thought that by clustering the data 
sets into some "onion ring" structure that the features representing the index
of lower importance could be embedded into the ones having higher importance.
In fact, in this case, I don't want any network to "faithfully preserve" the 
input features according to the frequency or order in which I present the input.
I want it to tear the features apart and form attractor(s) the strength of which
is adapted to the importance of different features that are presented to the
network. All the work that I have seen so far in the neural networks literature 
on feature maps gives no indication of how importance is attached (in the output
space) to different input features.  PCA won't quite do it for my application 
since I want a representation in a map (and not some ugly eigenvalues :-).

BTW, Patrick Simpson in his book on ANN has weight updates for LVQ 
("clustering ?").  Is this equivalent to updating the values of the code book 
vector for training samples (as found in Kohonen's papers) and then finding the 
distance metric for new input vectors ?

Here are the email replies I got on my previous request.  I hope the emailers
don't mind me posting the information.
-------------------------------------------------------------------------------
From: "A.J." Schofield <cod14@seq1.keele.ac.uk>
Subject: Re: Kohonen algorithm (help me control the clustering)

      Do you know what classes your data fall into? 
      or Are you guessing that the classification boundaried are onion rings
      or Do you have no idea about the nature of the data but wish to force
      the onion ring classification?

      that is if you plot the feature vectors of the data you have do you
      know that they fall into these concentric circles (if you can plot them
      at all). 

   The thing with kohonen nets is that theyn are unsuppervised and so you must
   let them cluster datum according to the inherent properties. I know of
   no way to pre-determine the outcome (that is not to say that it can't be
   done). If you allready know what classes your data fall into then you don't
   really want to use vanilla kohonen nets. In this case you might try 
   kohonen LVQ networks which are the suppervised version of Kohonen Self
   organising maps. Using LVQ you can force the correct classification onto the
   net but at the same time the net will make its own sub-clusters as required
   to perfrom the classification. I guess that LVQ will be able to solve some
   problems but I can't give any garantees in your case. 

   For details of LVQ see

   Kohonen T et al. `Statistical PAttern Recognition with Neural Networks: 
   Benckmark Studies'. Proc 2nd IEEE Int, Conference on Neural Networks, Vol 1.
   1988


   Kohonen T `Self-organisation and Associative Memory' 2nd Ed. 
   Springer-Verlag.
   Regards Andrew.


From: sxb@inel.gov (Steve Bryan)

I have tried to anchor the cluster points on the map with some success.  I
had features that represented speech from multiple individuals.  I wanted 
silence to appear at a known position on the map, so every time the 
features representing silence came along, I set the kohonen map coordinates
to a specific place (e.g. (0,0) on the map).  This seemed to work fine.

Hope this helps.
|Steven R. Bryan, Idaho National Engineering Lab.  INTERNET: sxb@INEL.GOV    |
-------------------------------------------------------------------------------
Let me also thank Surya Kavuri@purdue, Drago Indjic@Imperial College, and
James L. in Australia (university of Melbourne ?) for their suggestions.
--------------------------------------------------------------------------------

Additionally, here are some references that I have:

Koikkalainen P., and E. Oja. 1989. "Self-Organizing Hierarchical Feature Maps."
 
Kohonen T., G. Barna., and R. Chrisley. 1988. "Statistical Pattern Recognition
with Neural Networks: Benchmarking Studies." IEEE ICNN (San Diego) 1988.
 
Kohonen T. 1988. "The 'Neural' Phonetic Typewriter". IEEE Computer Magazine,
March 1988.

Ritter H. 1989. "Combining Self-Organizing Maps" IJCNN '89. pp 499-502.
[Has someone done the same type of work for decomposition of more complicated 
functions: non-montonic or non-linear ?]

_______ and K. Schulten. 1988.  "Kohonen's Self-Organizing Maps: Exploring
their Computational Capabilities" IEEE ICNN (San Diego) 1988.

_______ and T. Kohonen. 1989. "Self-Organizing Semantic Maps." Biol. Cybern.
61:241-254.