ValueError: Requested tokenizer basic_english, valid choices are a callable that takes a single string as input,

vainaijr: <p>basic_english tokenizer error in PyTorch attention tutorial.</p>
<pre><code class="lang-auto">import torchtext
from torchtext.data.utils import get_tokenizer
TEXT = torchtext.data.Field(tokenize=get_tokenizer("basic_english"),
                            init_token='&lt;sos&gt;',
                            eos_token='&lt;eos&gt;',
                            lower=True)
</code></pre>
<p>google colab notebook not found</p>
<p><a href="https://github.com/pytorch/tutorials/blob/gh-pages/_downloads/transformer_tutorial.ipynb" class="onebox" target="_blank" rel="nofollow noopener">https://github.com/pytorch/tutorials/blob/gh-pages/_downloads/transformer_tutorial.ipynb</a></p>

zhangguanheng66: <p>Feel free to open up an issue on (pytorch/tutorials](<a href="https://github.com/pytorch/tutorials/issues" rel="nofollow noopener">https://github.com/pytorch/tutorials/issues</a>). Could you also post the error messages?</p>

vainaijr: <p>I raised an issue, along with error messages on github.</p>

zhangguanheng66: <p>Can you send me the issue?</p>

vainaijr: <aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/tutorials/issues/684" target="_blank" rel="nofollow noopener">github.com/pytorch/tutorials</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/tutorials/issues/684" target="_blank" rel="nofollow noopener">notebook not found</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-10-10" data-time="00:22:29" data-timezone="UTC">12:22AM - 10 Oct 19 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/vainaijr" target="_blank" rel="nofollow noopener">
          <img alt="vainaijr" src="https://avatars0.githubusercontent.com/u/43755000?v=4" class="onebox-avatar-inline" width="20" height="20">
          vainaijr
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">SEQUENCE-TO-SEQUENCE MODELING WITH NN.TRANSFORMER AND TORCHTEXT
google colab notebook not found
https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/transformer_tutorial.ipynb
also error when running the code in colab</p>
</div>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


zhangguanheng66: <p>Thanks. I have pinged the maintainers for pytorch/tutorial repo. I run the script locally and it’s fine. So very likely, there is an issue with the setup.</p>

baris: <p>try this:</p>
<pre><code class="lang-auto">!pip3 install torchtext==0.4.0
</code></pre>

i-m-vivek: <aside class="quote no-group" data-username="baris" data-post="7" data-topic="57733">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/ea666f/48.png" class="avatar"> baris:</div>
<blockquote>
<p>!pip3 install torchtext==0.4.0</p>
</blockquote>
</aside>
<p>this works, in colab they have installed torchtext version 0.3.1</p>