Word embeddings are a popular way to improve downstream per-formances in contemporary language modeling. However, the un-derlying geometric structure of the embedding space is not wellunderstood . We present a series of explorations using bio-inspiredmethodology to traverse and visualize word embeddings, demon-strating evidence of discernible structure. Moreover, our modelalso produces word similarity rankings that are plausible yet verydifferent from common similarity metrics, mainly cosine similarityand Euclidean distance. We show that our bio-inspired model canbe used to investigate how different word embedding techniquesresult in different semantic outputs, which can emphasize or obscureparticular interpretations in textual data.
Word embeddings are a popular way to improve downstream performances in contemporary language modeling. However, the underlying geometric structure of the embedding space is not well understood . We present a series of explorations using bio-inspired methodology to traverse and visualize word embeddings, demonstrating evidence of discernible structure. Moreover, our model also produces word similarity rankings that are plausible yet very different from common similarity metrics, mainly cosine similarity and Euclidean distance. We show that our bio-inspired model can be used to investigate how different word embedding techniques result in different semantic outputs, which can emphasize or obscure particular interpretations in textual data.
R
per-formances
performances
R
un-derlying
underlying
R
wellunderstood
well understood
R
bio-inspiredmethodology
bio-inspired methodology
R
demon-strating
demonstrating
R
modelalso
model also
R
verydifferent
very different
R
similarityand
similarity and
R
canbe
can be
R
techniquesresult
techniques result
R
obscureparticular
obscure particular