How to download and use glove vectors?

n0obcoder: <p>First of all, I would like to know if Glove is the best pre-trained embedding for an NLP application ?<br>
Secondly, how can I get the glove embeddings in Pytorch?<br>
Thirdly, can i, for example, extract out the embedding for a specific word, like, ‘king’ and ‘queen’ ?</p>
<p>Thanks in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <ol>
<li>
<p>I’m not sure, but out NLP experts might know the answer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
</li>
<li>
<p><a href="https://medium.com/@martinpella/how-to-use-pre-trained-word-embeddings-in-pytorch-71ca59249f76" rel="nofollow noopener">This blog post</a> describes, how to load and use the embeddings. Note that now you can also use the classmethod <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Embedding.from_pretrained" rel="nofollow noopener"><code>from_pretrained</code></a> to load the weigths.</p>
</li>
<li>
<p>Yes, this is also shown in the blog post.</p>
</li>
</ol>

vdw: <p>If it helps, you can have a look at <a href="https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/utils/data/text/wordvectorloader.py" rel="nofollow noopener">my code</a> for that. You only need the <code>create_embedding_matrix</code> method – <code>load_glove</code> and <code>generate_embedding_matrix</code> were my initial solution, but there’s not need to load and store all word embeddings, since you need only those that match your vocabulary.</p>
<p>The <code>word_to_index</code> and <code>max_index</code> reflect the information from your vocabulary, with <code>word_to_index</code> mapping each word to a unique index from <code>0..max_index</code> (not that I’ve written it, you probably don’t need <code>max_index</code> as an extra parameter). I use my <a href="https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/utils/data/text/vectorizer.py" rel="nofollow noopener">own implementation of a vectorizer</a>, but <code>torchtext</code> should give you similar information.</p>
<p>A full example of how it works can been seen in <a href="https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/notebooks/text-rnn-ae.ipynb" rel="nofollow noopener">this notebook</a>.</p>
<p>To your other questions:</p>
<ul>
<li>There’s hardly ever one best solution out there, and new types embeddings are proposed on properly a weekly basis. My tip would be: Just the something running, see how it works, and then try different alternatives to compare.</li>
<li>Of course you can get the embedding for a specific word. That’s essentially the content for the GloVe files. Each line contains first the word and then the <code>n</code> values of the embedding vector (with <code>n</code> being the vector size, e.g., 50, 100, 300)</li>
</ul>

n0obcoder: <p>i get the idea, thanks for the clarification <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

n0obcoder: <p>ahhaha <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=9" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:"> thanks for the help <a class="mention" href="/u/ptrblck">@ptrblck</a></p>