Growing Self Organizing Maps (GSOM)
By: Vivek Vinushanth Christopher
Tags: Machine Learning, Unsupervised Learning, Data Science, Clustering, Data Visualization

GSOM Algorithm

1. Initialization Phase

In this particular phase algorithm initializes the weight vectors of the starting nodes with random numbers. Generally GSOM algorithm initiates with 4 nodes which provides the autonomy to grow in a direction which they wanted to since all four of them are boundary nodes.

A numeric variable H_err is initialized to “0” at initialization. Spread factor (SF) has to be specified in this phase. And also the algorithm tends to calculate the growth threshold (GT) for the given data set according to the specified requirements. GT will act as a threshold value for initiating node generation.

Fig.2. Initial GSOM(A)

Initialization phase summary: - Initialize the weight vectors of the starting nodes (usually four) with random numbers between 0 and 1. - Determine Spread Factor (SF) - Calculate the growth threshold(GT) for the data set with (D) dimension and the spread factor (SF) which is already defined, using the formula GT = -D x ln(SF)

2. Growing Phase

We now feed the input to the network in this phase. Initially, similar to SOM (Competition Phase) algorithm, GSOM too determines the closest weight vector to the input vector as the winner (or BMU-Best Matching Unit), based on Euclidean distance. GSOM can be considered as the2D representation of the input space where input space is partitioned by weight vectors(W_i) as Voronoi regions(V_i) and each Voronoi region is represented by one neuron(i).

Fig.3. Voronoi Diagram — Photo by Charles Francis in Quora

“Voronoi diagram is a partition of a plane into regions close to each of a given set of objects. In the simplest case, these objects are just finitely many points in the plane. For each seed there is a corresponding region consisting of all points of the plane closer to that seed than to any other.” [3].

Cooperation and Adaptation phase of SOM follows where weight updation occurs only in BMU and the neighborhood of the winner. In GSOM the starting neighborhood is chosen smaller when compared to SOM.

When the total error (difference between the weight vector of a particular node and the input weight vector) of the node is lesser than the growth threshold(GT); i.e. if , H_err < GT then algorithm does not allow to grow new nodes. But when the total error of the node is greater than the growth threshold(GT); i.e. H_err >GT

Generate new nodes in a boundary node. And new nodes are generated on all free neighboring positions since computational cost of figuring out the best exact position for the new node is higher.

“A boundary node is one that has at least one of its immediate neighboring positions free of a node”[1]. In GSOM which defines that every node can have maximum of 4 immediate neighbors, refers to boundary node as a node where there is the availability of adding 3 more neighbours or can move in any of three remaining directions.

After generating the new nodes, it is essential to update the weights. There are four possible ways(See Fig 4. )

Fig.4. Weight updation in GSOM new nodes by Damminda et al.

When on one of its sides, New node has two old nodes in the consecutive manner (Fig. 4(a))

=> if W2 > W1 => W_new = W1-(W2 - W1)

=> if W1 > W2 => W_new = W1+(W1-W2)

2. When the new node is placed in between two older nodes (Fig. 4(b))

=> W_new = (W1+W2)/2

3. When the new node has only one direct neighbor which is an older node (Fig. 4(c))

=> if W2 > W1 => W_new = W1-(W2 -W1)

=> if W1 > W2 => W_new = W1+(W1-W2)

4. When the nodes are isolated due to nodes being removed after aging and generated new node has only one neighboring older node

=> W_new = (r_1 + r_2) /2 ; r_1, r_2 lower and upper values of the range of the weight vector of the only one neighbouring node.

Growing Phase Summary : - Winner node/neuron is chosen based on the closest proximity of input vectors to that of neuron. - The difference between the winner vector and the input vector of a particular input is accumulated as error value for that neuron(i). - If the error that is contributed by a particular weight vector is highly contributing to the accumulated error, it will be decided to generate a new neuron since the higher contribution of error refers that the winner neuron is not the best representer of input vector. - New nodes will be created to grow from a boundary node . Weight upation occurs for new nodes.

3. Smoothing Phase

The growing phase stops when there are less number of new nodes being created and smoothing phase begins there itself. In this Smoothing phase, algorithm reduces the learning rate and fix a small starting neighborhood.