Repository: ytdl-org/youtube-dl

Issue:
[8tracks] throws 403 error after first 4 songs. Nothing downloads.
8tracks recently updated their API to version 3. They probably updated their security on this (because, according to their license, you can only skip 3 songs within a certain time frame).



Discussion:
Please post the full output you get when running `youtube-dl SOME_8TRACK_URL -v`. [As stated in the bug reporting guide](https://github.com/rg3/youtube-dl#bugs), it's nearly impossible to debug the errors without that info.

Results:

cade@batman:~> youtube-dl --verbose "http://8tracks.com/cadejscroggins/blast-off"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://8tracks.com/cadejscroggins/blast-off']
[debug] Encodings: locale 'UTF-8', fs 'utf-8', out 'UTF-8', pref: 'UTF-8'
[debug] youtube-dl version 2014.02.04
[debug] Python version 2.7.6 - Darwin-13.0.0-x86_64-i386-64bit
[debug] Proxy map: {}
[8tracks] blast-off: Downloading webpage
[8tracks] blast-off: Downloading song information 1/13
[8tracks] blast-off: Downloading song information 2/13
[8tracks] blast-off: Downloading song information 3/13
[8tracks] blast-off: Downloading song information 4/13
[8tracks] blast-off: Downloading song information 5/13
ERROR: Failed to download song information: HTTP Error 403: Forbidden; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 191, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1157, in urlopen
    return self._opener.open(req)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(_args)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(_args)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Thanks, we will investigate. It seems that not all urls fail, at least our test url (http://8tracks.com/ytdl/youtube-dl-test-tracks-a) works fine (although it failed a few days ago).

So I found a fix. The download script (eighttracks.py) doesn't even use the 8tracks API. I modified it to use API version 3, and it seems to fix the issue.

Feel free to open a pull request with the fix.

Similar problem...

youtube-dl --verbose 'http://8tracks.com/noonpacific/noon-103'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://8tracks.com/noonpacific/noon-103']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.09.06
[debug] Python version 2.6.9 - Darwin-13.3.0-x86_64-i386-64bit
[debug] Proxy map: {}
[8tracks] noon-103: Downloading webpage
[8tracks] noon-103: Downloading song information 1/10
[8tracks] noon-103: Downloading song information 2/10
[8tracks] noon-103: Downloading song information 3/10
[8tracks] noon-103: Downloading song information 4/10
[8tracks] noon-103: Downloading song information 5/10
ERROR: Failed to download song information: HTTP Error 403: Forbidden; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 211, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1244, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 435, in error
    return self._call_chain(_args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(_args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 518, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)


Resolving patch:
diff --git a/youtube_dl/extractor/eighttracks.py b/youtube_dl/extractor/eighttracks.py
--- a/youtube_dl/extractor/eighttracks.py
+++ b/youtube_dl/extractor/eighttracks.py
@@ -4,10 +4,12 @@
 import json
 import random
 import re
+import time
 
 from .common import InfoExtractor
 from ..utils import (
     compat_str,
+    ExtractorError,
 )
 
 
@@ -112,14 +114,30 @@ def _real_extract(self, url):
         session = str(random.randint(0, 1000000000))
         mix_id = data['id']
         track_count = data['tracks_count']
+        duration = data['duration']
+        avg_song_duration = duration / track_count
         first_url = 'http://8tracks.com/sets/%s/play?player=sm&mix_id=%s&format=jsonh' % (session, mix_id)
         next_url = first_url
         entries = []
+
         for i in range(track_count):
-            api_json = self._download_webpage(
-                next_url, playlist_id,
-                note='Downloading song information %d/%d' % (i + 1, track_count),
-                errnote='Failed to download song information')
+            
+            api_json = None
+            download_tries = 0
+
+            while api_json is None:
+                try:
+                    api_json = self._download_webpage(
+                        next_url, playlist_id,
+                        note='Downloading song information %d/%d' % (i + 1, track_count),
+                        errnote='Failed to download song information')
+                except ExtractorError:
+                    if download_tries > 3:
+                        raise
+                    else:
+                        download_tries += 1
+                        time.sleep(avg_song_duration)
+
             api_data = json.loads(api_json)
             track_data = api_data['set']['track']
             info = {
@@ -131,6 +149,7 @@ def _real_extract(self, url):
                 'ext': 'm4a',
             }
             entries.append(info)
+
             next_url = 'http://8tracks.com/sets/%s/next?player=sm&mix_id=%s&format=jsonh&track_id=%s' % (
                 session, mix_id, track_data['id'])
         return {


Test patch: