Repository: ytdl-org/youtube-dl

Issue:
[vlive] Unable to download live videos
### Make sure you are using the _latest_ version: run `youtube-dl --version` and ensure your version is _2016.04.24_. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2016.04.24**
### Before submitting an _issue_ make sure you have:
- [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
### What is the purpose of your _issue_?
- [x] Bug report (encountered problems with youtube-dl)
- [x] Site support request (request for adding support for a new site)
- [x] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---
### If the purpose of this _issue_ is a _bug report_, _site support request_ or you are not completely sure provide the full verbose output as follows:

(Note that this live might end when you check it.)

```
$ youtube-dl -v 'http://www.vlive.tv/video/7779/퇴근-저흰출근방송'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.vlive.tv/video/7779/퇴근-저흰출근방송']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.04.24
[debug] Python version 3.4.3 - Linux-4.5.0-gentoo-x86_64-Intel-R-_Core-TM-_i7-3820_CPU_@_3.60GHz-with-gentoo-2.2
[debug] exe versions: ffmpeg N-79272-gbd1dcaf, ffprobe N-79272-gbd1dcaf, rtmpdump 2.4
[debug] Proxy map: {}
[vlive] 7779: Downloading webpage
ERROR: Unable to extract long video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/kagami/.local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 673, in extract_info
    ie_result = ie.extract(url)
  File "/home/kagami/.local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
    return self._real_extract(url)
  File "/home/kagami/.local/bin/youtube-dl/youtube_dl/extractor/vlive.py", line 36, in _real_extract
    webpage, 'long video id')
  File "/home/kagami/.local/bin/youtube-dl/youtube_dl/extractor/common.py", line 644, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract long video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
```

---
### If the purpose of this _issue_ is a _site support request_ please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Live video: has the exact same type of URLs, just marked as live videos at the site; check fresh lives [here](http://www.vlive.tv/home)

---
### Description of your _issue_, suggested solution and other information

(Not sure which category is best suited for this issue, sorry.)

Lives are served with Flash through HLS streams, there is JSON with streams parameters at the code page:

```
vlive.tv.video.ajax.request.handler.init("7779", "{\"liveStatus\":\"LIVE\",\"resolutions\":[
{\"name\":\"audio\",\"width\":null,\"height\":null,\"cdnUrl\":\"http:\/\/vlive.hls.edgesuite.net\/kr002\/alow.stream\/playlist.m3u8?__gda__=1461682617_f30c9b77b4d56a116b0e436d36b7b1d1\",\"additionalProperties\":{}},
{\"name\":\"250\",\"width\":320,\"height\":184,\"cdnUrl\":\"http:\/\/vlive.hls.edgesuite.net\/kr002\/250.stream\/playlist.m3u8?__gda__=1461682617_5bd3a23663a5cd54dc204da558ff00e5\",\"additionalProperties\":{}},
{\"name\":\"400\",\"width\":640,\"height\":368,\"cdnUrl\":\"http:\/\/vlive.hls.edgesuite.net\/kr002\/400.stream\/playlist.m3u8?__gda__=1461682617_62f2ced6228651d617e7a7b9fff0da6a\",\"additionalProperties\":{}},
{\"name\":\"1200\",\"width\":640,\"height\":368,\"cdnUrl\":\"http:\/\/vlive.hls.edgesuite.net\/kr002\/1200.stream\/playlist.m3u8?__gda__=1461682617_88ce3b5362dc63e50439be087164e98b\",\"additionalProperties\":{}}
],\"additionalProperties\":{}}", "", "", "", "");
```



Discussion:

Resolving patch:
diff --git a/youtube_dl/extractor/common.py b/youtube_dl/extractor/common.py
--- a/youtube_dl/extractor/common.py
+++ b/youtube_dl/extractor/common.py
@@ -1061,7 +1061,7 @@ def _parse_f4m_formats(self, manifest, manifest_url, video_id, preference=None,
     def _extract_m3u8_formats(self, m3u8_url, video_id, ext=None,
                               entry_protocol='m3u8', preference=None,
                               m3u8_id=None, note=None, errnote=None,
-                              fatal=True):
+                              fatal=True, live=False):
 
         formats = [{
             'format_id': '-'.join(filter(None, [m3u8_id, 'meta'])),
@@ -1139,7 +1139,11 @@ def _extract_m3u8_formats(self, m3u8_url, video_id, ext=None,
                 if m3u8_id:
                     format_id.append(m3u8_id)
                 last_media_name = last_media.get('NAME') if last_media and last_media.get('TYPE') != 'SUBTITLES' else None
-                format_id.append(last_media_name if last_media_name else '%d' % (tbr if tbr else len(formats)))
+                # Bandwidth of live streams may differ over time thus making
+                # format_id unpredictable. So it's better to keep provided
+                # format_id intact.
+                if last_media_name and not live:
+                    format_id.append(last_media_name if last_media_name else '%d' % (tbr if tbr else len(formats)))
                 f = {
                     'format_id': '-'.join(format_id),
                     'url': format_url(line.strip()),
diff --git a/youtube_dl/extractor/vlive.py b/youtube_dl/extractor/vlive.py
--- a/youtube_dl/extractor/vlive.py
+++ b/youtube_dl/extractor/vlive.py
@@ -1,8 +1,11 @@
 # coding: utf-8
-from __future__ import unicode_literals
+from __future__ import division, unicode_literals
 
+import re
+import time
 from .common import InfoExtractor
 from ..utils import (
+    ExtractorError,
     dict_get,
     float_or_none,
     int_or_none,
@@ -31,16 +34,77 @@ def _real_extract(self, url):
         webpage = self._download_webpage(
             'http://www.vlive.tv/video/%s' % video_id, video_id)
 
-        long_video_id = self._search_regex(
-            r'vlive\.tv\.video\.ajax\.request\.handler\.init\(\s*"[0-9]+"\s*,\s*"[^"]*"\s*,\s*"([^"]+)"',
-            webpage, 'long video id')
+        # UTC+x - UTC+9 (KST)
+        tz = time.altzone if time.localtime().tm_isdst == 1 else time.timezone
+        tz_offset = -tz // 60 - 9 * 60
+        self._set_cookie('vlive.tv', 'timezoneOffset', '%d' % tz_offset)
 
-        key = self._search_regex(
-            r'vlive\.tv\.video\.ajax\.request\.handler\.init\(\s*"[0-9]+"\s*,\s*"[^"]*"\s*,\s*"[^"]+"\s*,\s*"([^"]+)"',
-            webpage, 'key')
+        status_params = self._download_json(
+            'http://www.vlive.tv/video/status?videoSeq=%s' % video_id,
+            video_id, 'Downloading JSON status',
+            headers={'Referer': url})
+        status = status_params.get('status')
+        air_start = status_params.get('onAirStartAt', '')
+        is_live = status_params.get('isLive')
 
+        video_params = self._search_regex(
+            r'vlive\.tv\.video\.ajax\.request\.handler\.init\((.+)\)',
+            webpage, 'video params')
+        live_params, long_video_id, key = re.split(
+            r'"\s*,\s*"', video_params)[1:4]
+
+        if status == 'LIVE_ON_AIR' or status == 'BIG_EVENT_ON_AIR':
+            live_params = self._parse_json('"%s"' % live_params, video_id)
+            live_params = self._parse_json(live_params, video_id)
+            return self._live(video_id, webpage, live_params)
+        elif status == 'VOD_ON_AIR' or status == 'BIG_EVENT_INTRO':
+            if long_video_id and key:
+                return self._replay(video_id, webpage, long_video_id, key)
+            elif is_live:
+                status = 'LIVE_END'
+            else:
+                status = 'COMING_SOON'
+
+        if status == 'LIVE_END':
+            raise ExtractorError('Uploading for replay. Please wait...',
+                                 expected=True)
+        elif status == 'COMING_SOON':
+            raise ExtractorError('Coming soon! %s' % air_start, expected=True)
+        elif status == 'CANCELED':
+            raise ExtractorError('We are sorry, '
+                                 'but the live broadcast has been canceled.',
+                                 expected=True)
+        else:
+            raise ExtractorError('Unknown status %s' % status)
+
+    def _get_common_fields(self, webpage):
         title = self._og_search_title(webpage)
+        creator = self._html_search_regex(
+            r'<div[^>]+class="info_area"[^>]*>\s*<a\s+[^>]*>([^<]+)',
+            webpage, 'creator', fatal=False)
+        thumbnail = self._og_search_thumbnail(webpage)
+        return {
+            'title': title,
+            'creator': creator,
+            'thumbnail': thumbnail,
+        }
 
+    def _live(self, video_id, webpage, live_params):
+        formats = []
+        for vid in live_params.get('resolutions', []):
+            formats.extend(self._extract_m3u8_formats(
+                vid['cdnUrl'], video_id, 'mp4',
+                m3u8_id=vid.get('name'),
+                fatal=False, live=True))
+        self._sort_formats(formats)
+
+        return dict(self._get_common_fields(webpage),
+            id=video_id,
+            formats=formats,
+            is_live=True,
+        )
+
+    def _replay(self, video_id, webpage, long_video_id, key):
         playinfo = self._download_json(
             'http://global.apis.naver.com/rmcnmv/rmcnmv/vod_play_videoInfo.json?%s'
             % compat_urllib_parse_urlencode({
@@ -62,11 +126,6 @@ def _real_extract(self, url):
         } for vid in playinfo.get('videos', {}).get('list', []) if vid.get('source')]
         self._sort_formats(formats)
 
-        thumbnail = self._og_search_thumbnail(webpage)
-        creator = self._html_search_regex(
-            r'<div[^>]+class="info_area"[^>]*>\s*<a\s+[^>]*>([^<]+)',
-            webpage, 'creator', fatal=False)
-
         view_count = int_or_none(playinfo.get('meta', {}).get('count'))
 
         subtitles = {}
@@ -77,12 +136,9 @@ def _real_extract(self, url):
                     'ext': 'vtt',
                     'url': caption['source']}]
 
-        return {
-            'id': video_id,
-            'title': title,
-            'creator': creator,
-            'thumbnail': thumbnail,
-            'view_count': view_count,
-            'formats': formats,
-            'subtitles': subtitles,
-        }
+        return dict(self._get_common_fields(webpage),
+            id=video_id,
+            formats=formats,
+            view_count=view_count,
+            subtitles=subtitles,
+        )


Test patch: