How to save and load "torchtext.data.Field.build_vocab" result

Sirui_Li: <p>Hello,</p>
<p>How can I save and load the vocabulary of the “build_vocab”?</p>

J_Johnson: <p>I would like to bump this. Using torch.load gives an error.</p>

J_Johnson: <p>This seems to be a decent workaround:</p>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/bentrevett/pytorch-sentiment-analysis/issues/40" target="_blank" rel="noopener nofollow ugc">github.com/bentrevett/pytorch-sentiment-analysis</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/bentrevett/pytorch-sentiment-analysis/issues/40" target="_blank" rel="noopener nofollow ugc">Can we save the vocabulary generated by torchtext for later prediction?</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-06-20" data-time="02:39:02" data-timezone="UTC">02:39AM - 20 Jun 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-09-19" data-time="15:50:38" data-timezone="UTC">03:50PM - 19 Sep 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/littleflow3r" target="_blank" rel="noopener nofollow ugc">
          <img alt="littleflow3r" src="https://avatars.githubusercontent.com/u/36025078?v=4" class="onebox-avatar-inline" width="20" height="20">
          littleflow3r
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">Suppose I already saved the model and now want to use it only for inference, say to predict the sentiment of...</p>
</div>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<pre><code class="lang-auto">def save_vocab(vocab, path):
    with open(path, 'w+', encoding='utf-8') as f:     
        for token, index in vocab.stoi.items():
            f.write(f'{index}\t{token}\n')
</code></pre>
<p>Then :</p>
<pre><code class="lang-auto">def read_vocab(path):
    vocab = dict()
    with open(path, 'r', encoding='utf-8') as f:
        for line in f:
            index, token = line.split('\t')
            vocab[token] = int(index)
    return vocab
</code></pre>
<p>So you first define a vocab object, for example:<br>
<code>words=Field(**args**)</code></p>
<p>Then after using <code>words.build_vocab()</code>, call:<br>
<code>save_vocab(words.vocab, PATH)</code></p>
<p>And for loading:<br>
<code>quote.vocab=read_vocab(PATH)</code></p>
<p>So, finally, you would have:</p>
<pre><code class="lang-auto">def save_vocab(vocab, path):
    with open(path, 'w+', encoding='utf-8') as f:     
        for token, index in vocab.stoi.items():
            f.write(f'{index}\t{token}\n')

def read_vocab(path):
    vocab = dict()
    with open(path, 'r', encoding='utf-8') as f:
        for line in f:
            index, token = line.split('\t')
            vocab[token] = int(index)
    return vocab

words=Field(**args**)
words.build_vocab(dataset, dataset, dataset, ...)
save_vocab(words.vocab, PATH)

words_loaded=Field(**args**)
words_loaded.vocab=read_vocab(PATH)
</code></pre>