Loading pretrained resnet on Google Colab fails with Http 403: Forbidden

anujjamwal: <p>Hi All</p>
<p>I am very new to deep learning and pytorch. i am trying to lead the resnet34 model and it fails with 403 forbidden error. I am using Google Colab as the environment for notebooks.</p>
<p>resnet = torchvision.models.resnet34(pretrained=True)</p>
<pre><code class="lang-auto">Downloading: "https://download.pytorch.org/models/resnet34-333f7ec4.pth" to /root/.cache/torch/checkpoints/resnet34-333f7ec4.pth
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
&lt;ipython-input-16-9b84e690c25c&gt; in &lt;module&gt;()
      1 get_ipython().system('rm -rf /root/.cache/torch')
----&gt; 2 resnet = torchvision.models.resnet34(pretrained=True)

9 frames
/usr/local/lib/python3.6/dist-packages/torchvision/models/resnet.py in resnet34(pretrained, progress, **kwargs)
    247     """
    248     return _resnet('resnet34', BasicBlock, [3, 4, 6, 3], pretrained, progress,
--&gt; 249                    **kwargs)
    250 
    251 

/usr/local/lib/python3.6/dist-packages/torchvision/models/resnet.py in _resnet(arch, block, layers, pretrained, progress, **kwargs)
    221     if pretrained:
    222         state_dict = load_state_dict_from_url(model_urls[arch],
--&gt; 223                                               progress=progress)
    224         model.load_state_dict(state_dict)
    225     return model

/usr/local/lib/python3.6/dist-packages/torch/hub.py in load_state_dict_from_url(url, model_dir, map_location, progress, check_hash)
    490         sys.stderr.write('Downloading: "{}" to {}\n'.format(url, cached_file))
    491         hash_prefix = HASH_REGEX.search(filename).group(1) if check_hash else None
--&gt; 492         download_url_to_file(url, cached_file, hash_prefix, progress=progress)
    493 
    494     # Note: extractall() defaults to overwrite file if exists. No need to clean up beforehand.

/usr/local/lib/python3.6/dist-packages/torch/hub.py in download_url_to_file(url, dst, hash_prefix, progress)
    389     # We use a different API for python2 since urllib(2) doesn't recognize the CA
    390     # certificates in older Python
--&gt; 391     u = urlopen(url)
    392     meta = u.info()
    393     if hasattr(meta, 'getheaders'):

/usr/lib/python3.6/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    221     else:
    222         opener = _opener
--&gt; 223     return opener.open(url, data, timeout)
    224 
    225 def install_opener(opener):

/usr/lib/python3.6/urllib/request.py in open(self, fullurl, data, timeout)
    530         for processor in self.process_response.get(protocol, []):
    531             meth = getattr(processor, meth_name)
--&gt; 532             response = meth(req, response)
    533 
    534         return response

/usr/lib/python3.6/urllib/request.py in http_response(self, request, response)
    640         if not (200 &lt;= code &lt; 300):
    641             response = self.parent.error(
--&gt; 642                 'http', request, response, code, msg, hdrs)
    643 
    644         return response

/usr/lib/python3.6/urllib/request.py in error(self, proto, *args)
    568         if http_err:
    569             args = (dict, 'default', 'http_error_default') + orig_args
--&gt; 570             return self._call_chain(*args)
    571 
    572 # XXX probably also want an abstract factory that knows when it makes

/usr/lib/python3.6/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    502         for handler in handlers:
    503             func = getattr(handler, meth_name)
--&gt; 504             result = func(*args)
    505             if result is not None:
    506                 return result

/usr/lib/python3.6/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    648 class HTTPDefaultErrorHandler(BaseHandler):
    649     def http_error_default(self, req, fp, code, msg, hdrs):
--&gt; 650         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    651 
    652 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: Forbidden
</code></pre>

colesbury: <p>There were some DNS issues. It should be fixed now. Are you still having this problem?</p>

anujjamwal: <p>Yes. Just tried and it failed again with same error.</p>

alx: <p>Still having issues as well!</p>

alx: <p>When will it be fixed?</p>

NegatioN: <p>Just to chime in. Still seeing this issue from my own laptop in Japan, Colab and Google Cloud Instances</p>

smth: <p>This seems to be CloudFlare doing terrible things – for no reason whatsoever with it’s “protection and caching layer”.</p>
<p>I’ve switched it off for now, and in ~4 hours or so, all requests should be direct again.</p>
<p>I’ll keep tabs on this thread to hear back if things are fixed.</p>
<p>Sorry about this whole mess.</p>

Exlsunshine: <p>Oh I though my network is down… “happy” to see lots of people have the same issue… <img src="https://discuss.pytorch.org/images/emoji/apple/sneezing_face.png?v=9" title=":sneezing_face:" class="emoji" alt=":sneezing_face:">, hope we can download model again soon…</p>

varunvsharma: <p>Thanks <a class="mention" href="/u/smth">@smth</a>! Things seem to be back to normal for me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ChristianRasmussen: <p>Hey I had the same problem yesterday, however today did I get a new error while downloading a pretrained model from pytorch in google colab<br>
(URLError: &lt;urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)&gt;)</p>
<p>Do anybody know how to fix it or when there will be a fix available?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/b/7bc990e12036965c434f245f2cd7045875abef70.png" data-download-href="https://discuss.pytorch.org/uploads/default/7bc990e12036965c434f245f2cd7045875abef70" title="Screen Shot 2020-02-13 at 8.44.07 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7bc990e12036965c434f245f2cd7045875abef70_2_689x282.png" alt="Screen Shot 2020-02-13 at 8.44.07 AM" data-base62-sha1="hF4A4MAhpjQQHjEFhfqChz26XrG" width="689" height="282" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7bc990e12036965c434f245f2cd7045875abef70_2_689x282.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7bc990e12036965c434f245f2cd7045875abef70_2_1033x423.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7bc990e12036965c434f245f2cd7045875abef70_2_1378x564.png 2x" data-dominant-color="F5F4F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-02-13 at 8.44.07 AM</span><span class="informations">2362×966 302 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Thanks for letting us know.<br>
It’s most likely related to <a href="https://discuss.pytorch.org/t/tls-certificate-expired-for-download-pytorch-org/69619/2">this issue</a> and we are looking into it.</p>

smth: <p>The SSL Certificate issue is fixed now. The Cert was expired for ~4 hours, and we root-caused the auto-renewal problem, it wont happen again.</p>

Vikalp_Jain: <p>Still facing the same issue!<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/f/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d.png" data-download-href="https://discuss.pytorch.org/uploads/default/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d_2_690x345.png" alt="image" data-base62-sha1="dByu6LymvpnZIj95Bl9xk1bgXwN" width="690" height="345" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d_2_690x345.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d_2_1035x517.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/f/5f5b1309f09b76c1b2bb7b20263ba3df50c9c96d.png 2x" data-dominant-color="F3F5F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1228×614 35.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Can we have some permanent solution for this?</p>

ptrblck: <p>Could you recheck it please, as it’s working now on my system:</p>
<pre><code class="lang-python">&gt;&gt;&gt; model = models.resnet34(pretrained=True)
Downloading: "https://download.pytorch.org/models/resnet34-333f7ec4.pth" to /root/.cache/torch/checkpoints/resnet34-333f7ec4.pth
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 83.3M/83.3M [00:03&lt;00:00, 27.7MB/s]
&gt;&gt;&gt; 
</code></pre>

Vikalp_Jain: <p>Yes it worked. Still not sure but at that point of time i tried multiple times but it was not working and keep on failing. Thank you ptrblck.</p>

ptrblck: <p>Good to hear it’s working now. Please let us know, if you see such issues again. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

rasbt: <p>Unfortunately, this seems to be an issue again. For me, this happens when I try to download MNIST:</p>
<pre><code class="lang-python">train_dataset = datasets.MNIST(root='data', 
                               train=True, 
                               transform=None,
                               download=True)
</code></pre>
<p>Tried on both macOS and Ubuntu. I am wondering if this is due to some redirect because visiting the lecun’s mnist website and then downloading it manually works.</p>

ptrblck: <p>Thanks for notifying us. I can reproduce it and we’ll look into it.</p>
<p>PS: Do you see other datasets failing as well or just MNIST?<br>
So far I can just reproduce it for MNIST.</p>

fmassa: <p>Thanks for reporting <a class="mention" href="/u/rasbt">@rasbt</a>, we are tracking this issue in <a href="https://github.com/pytorch/vision/issues/1938">https://github.com/pytorch/vision/issues/1938</a></p>
<p>Still don’t know what the root cause is.</p>

smth: <p>this is a general issue with the download site of MNIST, i.e. <a href="http://yann.lecun.com">yann.lecun.com</a>. It has implemented CloudFlare protection.<br>
I’ll ask the hosts what is going on.</p>