<figure><img src="image_1.png" /></figure>

HAL
open science

# Exploring Self-Organizing Maps for Addressing Semantic Impairments

Jorge Graneri, Sebastián Basterrech, Eduardo Mizraji, Gerardo Rubino

► To cite this version:

Jorge Graneri, Sebastián Basterrech, Eduardo Mizraji, Gerardo Rubino. Exploring Self-Organizing Maps for Addressing Semantic Impairments. ESANN 2024 - 32nd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Oct 2024, Bruge, Belgium. pp.1-8. [hal-04756530](https://hal-04756530v1)

HAL Id: hal-04756530

https://inria.hal.science/hal-04756530v1

Submitted on 28 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
Exploring Self-Organizing Maps
for Addressing Semantic Impairments

in ESANN'24: 32nd European Symposium on Artificial Neural
Networks, Computational Intelligence and Machine Learning

Jorge Graneri¹, Sebastián Basterrech², Eduardo Mizraji³ and Gerardo Rubino⁴ *

1- Faculty of Engineering, UDELAR, Montevideo, Uruguay

2- DTU-Compute, Technical University of Denmark, Kongens Lyngby, Denmark

3- Faculty of Sciences, UDELAR, Montevideo, Uruguay

4- INRIA Rennes – Bretagne Atlantique, Rennes, France

**Abstract.** Since the 1990s, Self-Organizing Maps (SOMs) have been instrumental in reducing dimensionality and visualizing high-dimensional data. This study adapts SOMs to explore the neural representation of human concepts, their neural ‘word net’ mapping, and the deterioration of these mappings in certain neurological disorders. Our model draws inspiration from semantic dementia, a severe condition that degrades semantic knowledge in the brain. Although our exploration utilizes a low-dimensional model — a rough simplification with respect of our brains — it successfully replicates observed clinical patterns. These promising results inspire further research to enhance our understanding of language pathophysiology in neurological disorders.

<footnote>*This paper has been partially supported by ECOS Project U17E03 "MASC".</footnote>
# 1 Introduction

Self-Organizing Maps (SOMs) have demonstrated their effectiveness as a tool for clustering, reducing dimensionality, and visualizing high-dimensional data [1, 2, 3]. In addition, the method has the property of generating data in the latent space preserving the most important topological features of the reference data [1]. In this paper, we introduce a bio-inspired simulation tool that captures key aspects of a specific cognitive impairment. We chose to focus on semantic dementia [4, 5, 6], a particularly complex pathology for which little is known, and where semantic information in the brain degrades until almost disappearing [7]. The developed algorithm, a simple modification of SOMs, is straightforward, and the simulations show coherence with the real characteristics of the cognitive degradation in the semantic dementia. Preliminary results were presented in [8]. We first introduce SOMs, and then, after illustrating their interest for our goals, we show how to modify them with a model of degradation inspired by the cellular aging and other characteristics of the previously mentioned disease. Based on the observation that during the deterioration of a trained Kohonen map certain concepts show an unexpected robustness despite their low probability of presence, we also perform a systematic empirical study of this fact.

# 2 Basic concepts

## 2.1 Semantic dementia

The familiarity of specific concepts plays a crucial role in semantic cognition [9]. This understanding becomes particularly significant in certain brain diseases that impact the cognitive system. An example is semantic dementia, also known as the semantic variant of Primary Progressive Aphasia (svPPA) [10, 5, 6]. A notable consequence of this disease is the relatively rapid deterioration of the semantic system in the verbal domain with the loss of meaning [7]. This degradation is evident in the cognitive decline of the patients, as observed in several clinical tests. Patients may decrease the ability to match words or images with their correct meanings. The deterioration of knowledge in the animal category is a reference characteristic of this pathology. In the canonical study [10], it was shown the cognitive decline of a patient through naming animal images administered at different stages of the disease progression. The proportion of errors increases over time. Interestingly, towards the latter stages, patients often default to one of three primary concepts - “dog”, “cat”, or “horse”,
in response to nearly any prompt [10]. In most Western languages, these three animals are the most frequently mentioned and “seen”, not only in texts but also in everyday life, including movies, etc. Then, the system tends to associate any animal with the ones most frequently encountered during the patient’s lifetime. All of these characteristics led us to consider the concept of familiarity, which has been referred to as an important factor [9, 7].

## 2.2 The Self-organizing Map model

SOM consists of a non-linear parametric mapping employing Gaussian activation functions. The SOM's architecture is different from the classic feedforward one. The standard model has two layers, with the second layer usually called *feature map*. Each neuron in the feature map has, associated with, a weight vector with the same dimensions as the input patterns. The input to the network is a vector $v \in \mathbb{R}^M$ for some dimension $M$. Different instances of $v$ are sequentially broadcasted to a set of $N$ neurons organized in a regular way, say, to fix ideas, in a 2-dimensional grid (other structures are possible, keeping that regularity spirit). Each neuron $r$ is associated with a location in the grid and a weight vector $w_r \in \mathbb{R}^M$. The weights are adjusted using a biological background that considers lateral neural inhibition [2]. The optimization is iterative, composed of two main phases, a competitive learning scheme and an update regression step. In competitive learning, we determine the region of the feature map that will process the external stimulus (input pattern). The regression step, on the other hand, outlines how to update the network's state (weights) [11]. Given a pattern $v$, we say that the *maximal reaction neuron* or *winner neuron* in those circumstances is the neuron in the feature map with associated weight vector closest to the current input pattern, i.e. $r^* = \text{argmin}_r \|v - w_r\|$. Then, the update rule for each neuron $r$ is given by $w_r^{\text{new}} = w_r^{\text{old}} + \Delta w_r$, with $\Delta w_r = \eta h(\text{dist}(r, r^*)(v - w_r))$, where $\eta \in [0, 1]$ is the learning rate, $h(\cdot)$ is a neighborhood function [12] defined on the grid, and $\text{dist}(\cdot)$ is a distance in the grid. Typically, the vector norm and $\text{dist}(\cdot)$ are the Euclidean norm, and the neighborhood function is most often chosen from the exponential family. The role of function $h(\cdot)$ is to weight the region of the local neighborhood of the winner unit, and to control the sizes of the neighborhoods [12].
### 3 Modelling semantic impairments with a topographic map

The semantic cognitive problem was examined in [13], with additional insights in [14]. The authors presented a method of encoding semantic information into binary vectors. They further showcased the potential of using SOM as a tool for clustering. The authors examined a concise list of 16 animals, each characterized by a set of attributes represented in a 3-tuple $(\alpha, \beta, \gamma)$, where

* $\alpha \in \{\text{"small"}, \text{"medium"}, \text{"big"}\}$,

* $\beta \in \{\text{"hunt"}, \text{"run"}, \text{"fly"}, \text{"swim"}\}$

* and $\gamma \in \{\text{"2-legs"}, \text{"4-legs"}, \text{"hairs"}, \text{"hooves"}, \text{"mane"}, \text{"feathers"}\}$.

To introduce our modelling, we use the example of clustering those 16 animals using these set of attributes, as it was described in [14]. We modify the initial mapping to include the idea of conceptual familiarity and cognitive semantic degradation. We incorporate two key concepts: *conceptual familiarity* and *cellular aging*. We will progressively adjust the likelihood of presenting stimuli to the Kohonen algorithm. This modification emulates the semantic impairment. It will gradually enhance the likelihood of the vectors that are associated with “known” concepts. We follow the insights presented in [6], then we define the set of familiar concepts as {“dog”, “cat”, “horse”}. Let $F$ be the number of familiar concepts. An input pattern is presented at iteration $k$ with probability $(1 - \mu)/F$, and the rest of the patterns will be presented with probability $\mu/(K - F)$, where $K$ is the number of distinct objects (animals in our study case). In the initial phase of the algorithm, the parameter $\mu$ is set to $(K - F)/K$, which guarantees equal probability among the $K$ concepts. As the algorithm evolves, the value of $\mu$ is regularly decreasing using an update $\mu := \mu / \ln(\ln(k + 1))$. Furthermore, we incorporate the concept of cellular aging into the model. Starting from an arbitrary iteration, an *atrophy* is implemented with probability $p$. This consists of randomly and uniformly erasing neurons from the network, along with some of their neighbors. With probability $p$ an atrophy occurs, which means that a neuron is uniformly chosen on a set of close neighbors, and then removed. Let us call $f(\cdot)$ the discrete probability mass function parameterized by $\mu$ defined over the $K$ concepts. In our example, at the initial state of the procedure, $f(\text{"dog"}) = f(\text{"cat"}) = f(\text{"horse"}) = (1 - \mu)/F$, and for any other animal $a$, $f(a) = \mu/K$. The proposed iterative algorithm can be described in the following steps:
<figure><img src="image_2.png" /><figcaption>Figure 1: Three scenarios of the simulation of the semantic dementia.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Figure 2: Visualization of the mean entropy and border length.</figcaption></figure>

a) Initialization: select meta-parameters, initialize the network, and perform other necessary setup procedures.

b) Visual perception (sampling protocol): select an input vector $v$ using $f(\cdot)$.

c) Competitive learning: compute the maximal reaction neuron $r^*$ described in Section 2.2.

d) Regression step: apply the update rule described in Section 2.2.

e) Conceptual familiarity update: in case of reaching a temporal threshold, then update $\mu$ using $\mu := \mu / \ln(\ln(k+1))$.

f) Aging emulation: if time reaches a specific threshold, then randomly kill neurons using parameter $p$.
# 4 Analysis of results

We examine the progression of the simulations through a clustering analysis, and utilizing two metrics: entropy and border length.

**Evolution of the clustering.** Fig. 1 has three graphics that show the three types of simulations. One graph shows a simulation of the classical algorithm, while the other two respectively display the classical algorithm and the modified algorithm, emphasizing the three privileged concepts ("dog", "cat", and "horse"). Each color in the map corresponds to an animal; the color code is specified in Fig. 1.B. The map in Fig. 1.A presents the result of the standard Kohonen map, illustrating how the topology of the original space is maintained in the feature map, i.e. animals that share similar features are placed closely, while those with differing characteristics are placed further apart. The map in Fig. 1.C shows the clustering when the grouping is made between familiar concepts and all the other animals are grouped in a unique set. In this specific example, we see "dog", "cat" and "horse", the rest of the animals appearing as a single group (orange color). The map in Fig. 1.D depicts the output of the proposed algorithm, where the impact of the cellular aging is illustrated by the cellular atrophies (gray cells in the map). The orange color represents the animals that are not characterized as familiar (that is, "dog", "cat" and "horse"). The three figures were generated under identical experimental conditions, including the number of neurons in the map, the type of neighborhood function, the number of iterations, and so forth.

**Analysis of entropy.** Continuing with the main case study, let $p_i$ be the frequency, after $k$ steps, of concept $i$, for $i = 1, \dots, K$. The entropy of the obtained cortical representation is $E = \sum_{i=1}^K p_i \log(1/p_i)$. We can then get an idea of the richness of the representation in terms of frequencies, as we can observe in Fig. 2. Fig. 2 has two graphics. The left graphic shows the entropy measure and the right one shows the border length. For each point from $k = 1$ to 500, we show the mean of 50 instances of entropy obtained after $k$ training epochs. The blue curve has the score obtained with the original SOM algorithm, and the red curve reflects the score entropy of the proposed approach. We can observe, in both cases, that the entropy increases until it stabilizes. The main difference between the original SOM and the proposed algorithm is when the entropy is stabilized. It is naturally higher in the classic training model due to the equal probability in the sampling. The proposed approach tends to a more
balanced distribution at the end of the training, which is natural because
fewer non-familiar concepts emerge over time. Consequently, there is a
tendency to select only the familiar concepts with equal probability.

**Border length.** We also examine an additional metric, which we refer to as the map’s border length. If we calculate the length of the borders for each map, represented by a square matrix of size *m*: $w_{i,j}^{(k)}$, *i*, *j* = 1, ..., *m*, after *k* training iterations, the border length is given by $B_k = \sum_{i,j} 1_{\{w_{ij}^{(k)} \neq w_{i+1,j}^{(k)}\}} + \sum_{i,j} 1_{\{w_{ij}^{(k)} \neq w_{i,j+1}^{(k)}\}}$. Similarly to the entropy, there is a stabilization of the border lengths. The figure shows the mean of 50 realizations. The right graphic in Fig. 2 shows the impact of the parameter $\mu$ in the model, through the final proportion of the representation of the familiar concepts (combined) in the proposed map relative to the initial probability.

5 Conclusions

We introduced an algorithm, inspired from the classic SOM, for simulating
the cognitive degradation of the semantic dementia. We incorporate to
the classic algorithm the concepts of cellular degradation and familiarity,
both of which are features present in the disorder under study. Our results
do not resolve the many unknowns that exist about the pathophysiology
of dementias, and surely the work that must be done to get closer to
understanding these neurological disorders is extremely broad and complex.
However, we believe that this tool designed to simulate the deterioration
of the neural structures could prove beneficial and offer valuable insights
for a more comprehensive understanding of neurological disorders.

References

[1] Teuvo Kohonen. Essentials of the self-organizing map. *Neural Networks*, 37:52–65, 2013.

[2] Ayu Saraswati, Van Tuc Nguyen, Markus Hagenbuchner, and Ah Chung Tsoi. High-resolution Self-Organizing Maps for advanced visualization and dimension reduction. *Neural Networks*, 105:166–184, 2018.

[3] Sebastián Basterrech, Line Clemmensen, and Gerardo Rubino. A self-organizing clustering system for unsupervised distribution shift detection, 2024. URL: https://arxiv.org/abs/2404.16656.

[4] Marsel Mesulam. Slowly progressive aphasia without generalized dementia. *Ann. Neurol.*, 11:592–598, 1982.
[5] Julie S. Snowden, P.J. Peter J. Goulding, and David Neary. Semantic dementia: a form of circumscribed atrophy. *Behav. Neurol.*, 2:167–182, 1989.

[6] John R. Hodges and Karalyn Patterson. Semantic dementia: a unique clinico-pathological syndrome. *The Lancet Neurology*, 6:1004–1014, 2007.

[7] Jonathan A. Knibb and John R. Hodges. Semantic Dementia: Losing the Meaning of Everything. *Practical Neurology*, 5(4):236–239, 2005.

[8] Eduardo Mizraji Jorge Graneri, Sebastian Basterrech and Gerardo Rubino. A modified self organizing map for semantic impairment, 2023. MITA 2023-The 19th International Conference on Multimedia Information Technology and Applications, Ostrava, Czech Republic.

[9] Timothy Rogers, Karalyn Patterson, Elizabeth Jeffereis, and Matthew Lambon Ralph. Disorders of representation and control in semantic cognition: Effects of familiarity, typicality, and specificity. *Neuropsychologia*, 76:220–239, 2015.

[10] John R. Hodges, Naida Graham, and Karalyn Patterson. Charting the progression in semantic dementia: Implications for the organisation of semantic memory. *Memory*, 3:463–495, 1995.

[11] Sebastián Basterrech, Colin Fyfe, and Gerardo Rubino. Self-organizing Maps and Scale-invariant Maps in Echo State Networks. In *11th International Conference on Intelligent Systems Design and Applications, ISDA 2011*, Córdoba, Spain, November 22-24 2011.

[12] Teuvo Kohonen. *Self-Organizing Maps*. Springer Series in Information Sciences. Springer-Verlag, 3rd edition, 2001.

[13] Helge Ritter and Teuvo Kohonen. Self-organizing semantic maps. *Biological cybernetics*, 61(4):241–254, 1989.

[14] Helge Ritter, Thomas Martinetz, and Klaus Schulten. *Neural Computation and Self-Organizing Maps - An Introduction*. Addison-Wesley, New York, 1992.