How to implement Kohonen’s Self Organizing Maps | by Kosala Sananthana | Towards Data Science

Artificial neural networks (ANN) are inspired by the early models of sensory processing by the brain. An artificial neural network can be created by simulating a network of model neurons in a computer. By applying algorithms that mimic the processes of real neurons, we can make the network ‘learn’ to solve many types of problems. — Anders Krogh1 (Nature Biotechnology)
In postmodern life, we engage with an enormous number of astonishing artificial neural network applications in each and every activity and we have no clue of their capacities and complexities. Artificial neural networks have been utilized to difficulties ranging from speech recognition to prediction of protein secondary structure, classification of cancers, and gene prediction1. Since the awareness of these advanced performances will be a necessity in the near future, we should have a better knowledge of these and we can start our journey of understanding ANNs from a simple level.
As a basic type of ANNs, let’s consider a self-organizing map (SOM) or self-organizing feature map (SOFM) that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map.
Self Organizing Map?
It converts the nonlinear statistical relationships between high-dimensional data into simple geometric relationships of their image points on a low-dimensional display, usually a regular two-dimensional grid of nodes. As the SOM thereby compresses information while preserving the most important topological and/or metric relationships of the primary data elements on the display — Teuvo Kohonen2
Basically, SOMs are characterized as a nonlinear, ordered, smooth mapping of high-dimensional input data manifolds onto the elements of a regular, low-dimensional array2. After training the SOM’s neurons we get a low dimensional representation of high dimensional input data without disturbing the shape of the data distribution and relationship between each input data element. Self-organizing maps differ from other ANNs as they apply unsupervised learning as compared to error-correction learning (backpropagation with gradient descent etc), and in the sense that they use a neighborhood function to preserve the topological properties of the input space. Since its simplicity, we can easily explain and demonstrate its capabilities. For a detailed explanation please refer to Self-Organizing Maps by Teuvo Kohonen2.
A simple illustration of the learning process is given in the above figure and we can understand the specialty of SOMs from this representation easily. Initially, input data(blue dots) occupy a special distribution in 2D space, and un-learned neuron(weights) values (red dots) are randomly distributed in a small area and after neurons get modified and learned by inputs, it gets the shape of the input data distribution step by step in the learning process. In addition, each neuron became a representation of one small cluster of input data space. Therefore in this demonstration, we were able to represent 1000 data points with 100 neurons, preserving the topology of the input data. That means we have built a relationship between high-dimension data to low-dimensional representation (map). For further calculations and predictions, we can utilize these few neuron values to represent the tremendous input data space which makes processes much faster.
As a basic model of SOM, we are mapping from the ‘n’-dimensional input data space to a two-dimensional array of neurons (‘N’ number of neurons). This SOM can be implemented using the following procedure:
“P” number of input vectors are available. ( i= 1, 2, ...,P)
ith input vector has n elements: Xi = (xi1, xi2, ..., xin)
“N” number of neurons (nodes or weights) are available. ( i= 1, 2, ..., N)
ith neuron vector has n elements: mi = (mi1, mi2, ..., miN)
These neuron vectors are arranged in a 2D matrix for representation.
Assume all vector elements are real numbers.
For a given input Xi, find the closest (smallest euclidean distance) neuron to the given input and signified the neuron by the c.
For a given input Xi, after finding mc neuron, only update the neighborhood neuron set of mc:
for t = 0, 1, 2, ... T. ( T is the number of iterations the models will get updated and mi (0) can be an initial arbitrary vector). The function hci(t) is the so-called neighborhood function, a smoothing kernel defined over the lattice points (matrix element).
Since we only need to update neighboring neurons around the mc neuron, first we need to find a neighborhood set of matrix points around neuron mc. A simple topological neighborhood finding method is given below and more advanced smooth neighborhood finding methods can be found in the literature2.
All the neurons inside the circle marked by Nc(t), we consider them as neighborhood set of ‘mc’ neurons. The neighborhood set of neurons’ radius Nc(t) is usually decreasing monotonically in iterations(t). We usually start with Nc(0) = √N/2 and we need to reduce the radius with each iteration.
After finding the set of neighborhood neurons that need to be updated we can use the following hci(t) function and update neurons around mc.
In this equation, || rc — ri|| define the distance between neuron’s 2D matrix positions (√N x √N matrix). The value of α(t) is identified as learning-rate factor (0 < α(t) < 1). Both α(t) and σ(t) are monotonically decreasing functions of time-varying as following:
Do this learning process for each and every input data vector ( ‘P’ number of input data vectors). Then again do this same process again and again for T time iterations over the same input data vectors. After the T number of iterations, you will get a fully learned neuron matrix that maps our input data values.
Using the above algorithm, a few interesting examples that have mentioned in Self-Organizing Maps Book by Teuvo Kohonen2 have been implemented using MATLAB and you can clone it to your local computer as follows:
git clone https://github.com/KosalaHerath/kohonen-som.git
Let’s define the repository’s home as <REPO_HOME>. Then, go to the following location and you can find three examples of MATLAB implementations and you can run them using any MATLAB version on your computer:
<REPO_HOME>/source/kohonen_examples
Otherwise, you can just click the following link and goto the implementation repository:
github.com
These examples are initiated with uniformly random distributed 2-dimensional (n=2) input data vectors. There were 1000 input data values (P=1000). In addition, we have defied the number of neurons as N = 10 x 10 = 100 and the number of iterations as T = 300. You can change these parameters and play with the model using the above implementations.
This example’s input data values are square shapely random distributed over the 2-dimensional space.
This example’s input data values are triangle shapely random distributed over the 2-dimensional space.
This example’s input data values are square shapely random distributed over the 2-dimensional space and specifically, we consider a 1D neuron array instead of a 2D matrix. Therefore, the line was made by neuron array would try to cover all of the input data distribution as follows.
Hence, now you can learn and play with these implementations with different inputs and modifications, and more you try will make you understand better. Please suggest any modifications that will improve these implementations from here.
Cheers! 🍺
[1] Krogh, A. (2008). What are artificial neural networks? Nature Biotechnology, 26(2), pp.195–197.
www.nature.com
[2] Teuvo Kohonen (2001). Self-organizing maps. New York Springer.