# Enhancing Word Embeddings for Improved Semantic Alignment
## 3 Material and Methods
### Method of Semantic Word Embedding Creation
#### 3.2.1 Embedding Layer Training
It is built with two values:

* \(a\): the mean distance between a data sample and other data points in the same class (cluster). * \(b\): the mean distance between a data sample and other data points in the next nearest class. Considering a single sample, the Silhouette Coefficient is calculated based on Equation (7):

\[s=\frac{b-a}{max(a,b)} \tag{7}\]

It is important to note that the Silhouette Coefficient is computed for only one sample, whereas if more than one sample were considered, it would be sufficient to take the average of the individual values. In order to test the separability of all classes, we calculated the average Silhouette Coefficient for all keyword embeddings. ## 4 Results

In this section, we present a summary of the results for creating semantic embeddings using the methods discussed above. Each section refers to a specific method for testing the embeddings and provides the results in both tabular and graphical form. To maintain methodological clarity, we developed a naming scheme to denote specific embedding methods, namely, original embeddings (OE), the basic embeddings that served as the basis for the creation of others; neural embeddings (NE), embeddings created using the embedding layer in the DNN-based classification process; fine-tuned embeddings (FE), embeddings created with fine-tuning, that is, using the GloVe training method on new semantic data; and geometrical embeddings (GE), word vectors that were created by moving vectors in space. ### Text Classification

First, we tested the quality of text classification using the semantic embeddings mentioned above. It is worth recalling that a weighted variant of the vectors using the IDF measure was also used for testing. So additionally, in classification experiments, we added the results from the usage of the re-weighed embedded representations: original weighted embeddings (OWE), fine-tuned weighted embeddings (FWE), neural weighted embeddings (NWE), and geometrical weighted embeddings (GWE). First, text classification was checked by selecting classes for sentences by checking the shortest distance between the mean sentence embedding and the category embedding. Figure 1 shows the accuracy results for all the embedding methods, sorted in descending order for better readability. It turns out that geometric embeddings perform best (93.8%), with a huge lead of 6 percentage points over second place, which was taken by neural embeddings (87.78%). This is a logical consequence of how these embeddings were trained since the geometrical embeddings underwent the most drastic change and thus were moved the most in space compared to the other methods. Moving them exactly toward the category embeddings at the training level led to better results. The other two methods, namely neural embeddings and fine-tuned embeddings, performed comparably well, both around 87%. Each of the new methods outperformed the original embeddings (83.89%), so all of these methods improved the classification quality. All the weighted embeddings perform significantly worse than their unweighted counterparts, with a difference of 5 or 6 percentage points, which was expected given that semantic embeddings occur in every sentence in the dataset. It turns out that only the geometrically weighted embeddings outperform the original embeddings. Next, the classification quality was examined using the average sentence embeddings as features for the non-neural text classifiers. The results improved significantly over the previous classification method, and the ranking of the results also changed. This time (see Figure 2), fine-tuned embeddings and neural embeddings were the best, with the latter taking the advantage (97.95% and 96.58%, respectively). The geometric embedding method (92.39%) performed much worse, coming close to the original embeddings' result (91.97%). Thus, it seems that despite the stronger shift in the vectors in space in the case of geometric embeddings, they lose some features and the classification suffers.