Repository: ytdl-org/youtube-dl

Issue:
Support for downloads from rtve.es have stopped working sice a couple of days ago.
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like this: `[x]`)
- Use the *Preview* tab to see what your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.09.15*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.09.15**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [x] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl -v <your command line>`), copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):

```
edu@skinner:~/Downloads/Servir y proteger$ youtube-dl -v http://www.rtve.es/alacarta/videos/servir-y-proteger/servir-proteger-capitulo-104/4236788/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'http://www.rtve.es/alacarta/videos/servir-y-proteger/servir-proteger-capitulo-104/4236788/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.09.15
[debug] Python version 2.7.12 - Linux-4.4.0-96-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 2.8.11-0ubuntu0.16.04.1, avprobe 2.8.11-0ubuntu0.16.04.1, ffmpeg 2.8.11-0ubuntu0.16.04.1, ffprobe 2.8.11-0ubuntu0.16.04.1, rtmpdump 2.4
[debug] Proxy map: {}
[rtve.es:alacarta] Fetching manager info
[rtve.es:alacarta] 4236788: Downloading JSON metadata
[rtve.es:alacarta] 4236788: Downloading url information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'http://hlsvod.tcdn.rtve.es/auth/resources/TE_NGVA/mp4/6/3/1506011092536.mp4/playlist.m3u8'
[download] Destination: Servir y proteger - Capítulo 104 -4236788.m3u8
[debug] avconv command line: avconv -y -loglevel verbose -headers 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
' -i 'http://hlsvod.tcdn.rtve.es/auth/resources/TE_NGVA/mp4/6/3/1506011092536.mp4/playlist.m3u8' -c copy -f mp4 'file:Servir y proteger - Capítulo 104 -4236788.m3u8.part'
ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[http @ 0xc84c00] HTTP error 404 Not Found
http://hlsvod.tcdn.rtve.es/auth/resources/TE_NGVA/mp4/6/3/1506011092536.mp4/playlist.m3u8: Server returned 404 Not Found


ERROR: ffmpeg exited with code 1
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/edu/bin/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/home/edu/bin/youtube-dl/youtube_dl/__init__.py", line 465, in main
    _real_main(argv)
  File "/home/edu/bin/youtube-dl/youtube_dl/__init__.py", line 455, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1967, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 787, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 841, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1601, in process_video_result
    self.process_info(new_info)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1874, in process_info
    success = dl(filename, info_dict)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1813, in dl
    return fd.download(name, info)
  File "/home/edu/bin/youtube-dl/youtube_dl/downloader/common.py", line 361, in download
    return self.real_download(filename, info_dict)
  File "/home/edu/bin/youtube-dl/youtube_dl/downloader/external.py", line 57, in real_download
    self.get_basename(), retval))
  File "/home/edu/bin/youtube-dl/youtube_dl/downloader/common.py", line 163, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 604, in report_error
    self.trouble(error_message, tb)
  File "/home/edu/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 566, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())


```



### Description of your *issue*, suggested solution and other information

Downloads from rtve.es have stopped working since a couple of days ago. Seems the page structure has changed.


Discussion:
For some reason the extractor is requesting the hls playlist from;
`http://hlsvod.tcdn.rtve.es/auth ...`

When there is no "auth" in the actual path.

There is also http progressive mp4 source url;
`http://mvod.lvlt.rtve.es/resources/TE_NGVA/mp4/6/3/1506011092536.mp4`

Resolving patch:
diff --git a/youtube_dl/extractor/rtve.py b/youtube_dl/extractor/rtve.py
--- a/youtube_dl/extractor/rtve.py
+++ b/youtube_dl/extractor/rtve.py
@@ -10,6 +10,7 @@
     compat_struct_unpack,
 )
 from ..utils import (
+    determine_ext,
     ExtractorError,
     float_or_none,
     remove_end,
@@ -84,6 +85,18 @@ class RTVEALaCartaIE(InfoExtractor):
             'title': 'TODO',
         },
         'skip': 'The f4m manifest can\'t be used yet',
+    }, {
+        'url': 'http://www.rtve.es/alacarta/videos/servir-y-proteger/servir-proteger-capitulo-104/4236788/',
+        'md5': 'e55e162379ad587e9640eda4f7353c0f',
+        'info_dict': {
+            'id': '4236788',
+            'ext': 'mp4',
+            'title': 'Servir y proteger - Capítulo 104 ',
+            'duration': 3222.0,
+        },
+        'params': {
+            'skip_download': True,  # requires ffmpeg
+        },
     }, {
         'url': 'http://www.rtve.es/m/alacarta/videos/cuentame-como-paso/cuentame-como-paso-t16-ultimo-minuto-nuestra-vida-capitulo-276/2969138/?media=tve',
         'only_matching': True,
@@ -107,24 +120,41 @@ def _real_extract(self, url):
             video_id)['page']['items'][0]
         if info['state'] == 'DESPU':
             raise ExtractorError('The video is no longer available', expected=True)
+        title = info['title']
         png_url = 'http://www.rtve.es/ztnr/movil/thumbnail/%s/videos/%s.png' % (self._manager, video_id)
         png_request = sanitized_Request(png_url)
         png_request.add_header('Referer', url)
         png = self._download_webpage(png_request, video_id, 'Downloading url information')
         video_url = _decrypt_url(png)
-        if not video_url.endswith('.f4m'):
+        ext = determine_ext(video_url)
+
+        formats = []
+        if not video_url.endswith('.f4m') and ext != 'm3u8':
             if '?' not in video_url:
                 video_url = video_url.replace('resources/', 'auth/resources/')
             video_url = video_url.replace('.net.rtve', '.multimedia.cdn.rtve')
 
+        if ext == 'm3u8':
+            formats.extend(self._extract_m3u8_formats(
+                video_url, video_id, ext='mp4', entry_protocol='m3u8_native',
+                m3u8_id='hls', fatal=False))
+        elif ext == 'f4m':
+            formats.extend(self._extract_f4m_formats(
+                video_url, video_id, f4m_id='hds', fatal=False))
+        else:
+            formats.append({
+                'url': video_url,
+            })
+        self._sort_formats(formats)
+
         subtitles = None
         if info.get('sbtFile') is not None:
             subtitles = self.extract_subtitles(video_id, info['sbtFile'])
 
         return {
             'id': video_id,
-            'title': info['title'],
-            'url': video_url,
+            'title': title,
+            'formats': formats,
             'thumbnail': info.get('image'),
             'page_url': url,
             'subtitles': subtitles,


Test patch: