Could not get the file at http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/training.tar.gz. [RequestException] None

tsap: <p>Hello,</p>
<pre><code class="lang-auto">Could not get the file at http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/training.tar.gz. [RequestException] None.
</code></pre>
<p>The error above showed up after calling the <code>build_vocab_from_iterator()</code> here:</p>
<pre><code class="lang-auto">vocab_src = build_vocab_from_iterator(
        yield_tokens(train + val + test, tokenize_de, index=0),
        min_freq=2,
        specials=["&lt;s&gt;", "&lt;/s&gt;", "&lt;blank&gt;", "&lt;unk&gt;"],
    )

</code></pre>
<p>I am using Google colab to run notebook on this <a href="https://github.com/harvardnlp/annotated-transformer" rel="noopener nofollow ugc">repo</a>. Link to notebook: <a href="https://colab.research.google.com/github/harvardnlp/annotated-transformer/blob/master/AnnotatedTransformer.ipynb" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a></p>
<p>Thank you in advance.</p>

tsap: <p>The problem is from the dataset <a href="https://pytorch.org/text/stable/_modules/torchtext/datasets/multi30k.html#Multi30k" rel="noopener nofollow ugc">multi30k</a> (source url: “<a href="http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/training.tar.gz" rel="noopener nofollow ugc">http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/training.tar.gz</a>”) that is not accessible right now.</p>
<hr>
<p>Note: the <code>torchtext.vocab.build_vocab_from_iterator()</code> on the Google Colab notebook above is calling this dataset. (Sorry for not being specific in describing the problem)</p>

Theo: <p>I’m experiencing the same problem for some time, and can’t seem to find a solution to it.</p>
<p>I think you could use other datasets instead.</p>

deng_wang: <p>I think I found the train, val , test file  url , you can change the multi30k URL to update the origal invalid url:</p>
<pre><code class="lang-auto">from torchtext.datasets import multi30k, Multi30k

multi30k.URL["train"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/training.tar.gz"
multi30k.URL["valid"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/validation.tar.gz"
multi30k.URL["test"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/mmt16_task1_test.tar.gz"
</code></pre>
<p>if you do not want to update the URL of multi30k,  you can just download the file from url above and put the tar.gz file to the  torch cache directory.  for my machine, the directory is :/root/.cache/torch/text/datasets/Multi30k ,  copy the tar.gz file into directory ,and run the code. pytorch will uncompress the file , get  <a href="http://train.de" rel="noopener nofollow ugc">train.de</a> train.en  files</p>

tsap: <p>On the google colab, when trying to use the archive from the URL above, there is this error on the block with <code>build_vocabulary()</code> function</p>
<pre><code class="lang-auto">RuntimeError: The computed hash 6d1ca1dba99e2c5dd54cae1226ff11c2551e6ce63527ebb072a1f70f72a5cd36 of /root/.torchtext/cache/Multi30k/mmt16_task1_test.tar.gz does not match the expectedhash 0681be16a532912288a91ddd573594fbdd57c0fbb81486eff7c55247e35326c2. Delete the file manually and retry.
</code></pre>
<p>Do you know how to fix this?</p>
<p>Here is my code I inserted before that:</p>
<pre><code class="lang-auto">!rm *.tar.gz
! rm -rf /root/.torchtext/cache/Multi30k/*

# datasets.multi30k.URL["train"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/training.tar.gz"
# datasets.multi30k.URL["valid"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/validation.tar.gz"
# datasets.multi30k.URL["test"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/mmt16_task1_test.tar.gz"


!wget https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/training.tar.gz
!wget https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/validation.tar.gz
!wget https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/mmt16_task1_test.tar.gz
! cp -f *.tar.gz /root/.torchtext/cache/Multi30k/
</code></pre>

swap_b: <p>Were u able to fix this?</p>

HY_Wu: <p>This problem happens because of the change in MD5 after changing the URL.</p>
<p>Refering to <a href="https://github.com/pytorch/text/blob/main/torchtext/datasets/multi30k.py#L18" class="inline-onebox" rel="noopener nofollow ugc">text/torchtext/datasets/multi30k.py at main · pytorch/text · GitHub</a><br>
, add the following lines of code before you use the dataset:</p>
<pre><code class="lang-auto">multi30k.URL["train"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/training.tar.gz"
multi30k.URL["valid"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/validation.tar.gz"
multi30k.URL["test"] = "https://raw.githubusercontent.com/neychev/small_DL_repo/master/datasets/Multi30k/mmt16_task1_test.tar.gz"

multi30k.MD5 = {
    "train": "20140d013d05dd9a72dfde46478663ba05737ce983f478f960c1123c6671be5e",
    "valid": "a7aa20e9ebd5ba5adce7909498b94410996040857154dab029851af3a866da8c",
    "test": "6d1ca1dba99e2c5dd54cae1226ff11c2551e6ce63527ebb072a1f70f72a5cd36",
}
</code></pre>