Repository: ytdl-org/youtube-dl

Issue:
Unable to download Replays' videos.
http://www.gameinformer.com/p/replay.aspx for all of its videos. Here is an example:

$ youtube-dl -v http://www.gameinformer.com/b/features/archive/2014/07/26/replay-raw-danger.aspx
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.gameinformer.com/b/features/archive/2014/07/26/replay-raw-danger.aspx']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.07.25.1
[debug] Python version 2.7.3 - Linux-3.2.0-4-amd64-x86_64-with-debian-7.6
[debug] Proxy map: {}
[generic] replay-raw-danger: Requesting header
WARNING: Falling back on generic information extractor.
[generic] replay-raw-danger: Downloading webpage
[generic] replay-raw-danger: Extracting information
ERROR: Unsupported URL: http://www.gameinformer.com/b/features/archive/2014/07/26/replay-raw-danger.aspx; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/extractor/generic.py", line 456, in _real_extract
    doc = parse_xml(webpage)
  File "/usr/bin/youtube-dl/youtube_dl/utils.py", line 1418, in parse_xml
    return xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1643, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1507, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 57, column 62
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 516, in extract_info
    ie_result = ie.extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 169, in extract
    return self._real_extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/generic.py", line 751, in _real_extract
    raise ExtractorError('Unsupported URL: %s' % url)
ExtractorError: Unsupported URL: http://www.gameinformer.com/b/features/archive/2014/07/26/replay-raw-danger.aspx; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.

Thank you in advance. :)



Discussion:
Note that some have age checks like http://www.gameinformer.com/b/features/archive/2014/08/02/replay-clock-tower.aspx ...

Still nothing? :(

FYI, very old Foxreal YouTube FLV Downloader Pro v1.0.2.1 had no problems.

It detailed logs show if it helps:
Download:   Thread created.
Download:   URL: rtmp://cp131630.edgefcs.net:1935/ondemand?videoId=4422714076001&lineUpId=&pubId=694940074001&playerId=1437116750001&affiliateId=/mp4:videos/694940074001/201508/1996/694940074001_4422995338001_Replay---EverQuest.mp4?videoId=4422714076001&lineUpId=&pubId=694940074001&playerId=1437116750001&affiliateId=.
Download:   File name changed to "E:\dl\Replay  EverQuest - Features - www.GameInformer.com.flv".
Download:   Create file "E:\dl\Replay  EverQuest - Features - www.GameInformer.com.flv" succeeded.

Provided URL does not work.

dstftw: Try today's post:  http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx

$ youtube-dl -v http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.08.09
[debug] Python version 2.7.3 - Linux-3.2.0-4-amd64-x86_64-with-debian-7.8
[debug] exe versions: avconv 1.2.12, avprobe 1.2.12, ffmpeg 1.2.12, ffprobe 1.2.12, rtmpdump 2.4
[debug] Proxy map: {}
[generic] replay-everquest: Requesting header
WARNING: Falling back on generic information extractor.
[generic] replay-everquest: Downloading webpage
[generic] replay-everquest: Extracting information
ERROR: Unsupported URL: http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx
Traceback (most recent call last):
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1211, in _real_extract
    doc = parse_xml(webpage)
  File "/home/ant/bin/youtube-dl/youtube_dl/utils.py", line 1594, in parse_xml
    tree = xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1643, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1507, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 25, column 843
Traceback (most recent call last):
  File "/home/ant/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 655, in extract_info
    ie_result = ie.extract(url)
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/common.py", line 286, in extract
    return self._real_extract(url)
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1804, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx

No:

```
Sorry for the inconvenience, but there’s an error with your location. Our diligent code monkeys have received this error, and if they are unable to fix it, they will be…dealt with.
```

Is it georestricted?

dstftw: I have no idea. I'm in USA for sure. Where are you?

Does not work with US proxy either:

```
<HTML><HEAD>
<TITLE>Internal Server Error</TITLE>
</HEAD><BODY>
<H1>Internal Server Error - Read</H1>
The server encountered an internal error or misconfiguration and was unable to
complete your request.<P>
Reference&#32;&#35;3&#46;c0070f17&#46;1439656629&#46;624b22c
</BODY></HTML>
```

Dsftw: Interesting.  Can you access any of the videos listed in http://www.gameinformer.com/p/replay.aspx ? They work for me. :/

Also, I updated my https://github.com/rg3/youtube-dl/issues/3376#issuecomment-131396478 comment.

@antdude no, any request to http://www.gameinformer.com/* leads to aforementioned results either direct or via anonymous US proxy.

Dstftw: That is so weird. I tried with two different Linux boxes on two different ISPs too. Here's the other box:

$ youtube-dl -v http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx
[debug] System config: [u'--prefer-free-formats']
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.08.09
[debug] Python version 2.7.10 - Linux-4.1.3-200.fc22.x86_64-x86_64-with-fedora-22-Twenty_Two
[debug] exe versions: ffmpeg 2.6.3, ffprobe 2.6.3, rtmpdump 2.4
[debug] Proxy map: {}
[generic] replay-everquest: Requesting header
WARNING: Falling back on generic information extractor.
[generic] replay-everquest: Downloading webpage
[generic] replay-everquest: Extracting information
ERROR: Unsupported URL: http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx
Traceback (most recent call last):
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1211, in _real_extract
    doc = parse_xml(webpage)
  File "/home/ant/bin/youtube-dl/youtube_dl/utils.py", line 1594, in parse_xml
    tree = xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1300, in XML
    parser.feed(text)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 25, column 843
Traceback (most recent call last):
  File "/home/ant/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 655, in extract_info
    ie_result = ie.extract(url)
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/common.py", line 286, in extract
    return self._real_extract(url)
  File "/home/ant/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1804, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.gameinformer.com/b/features/archive/2015/08/15/replay-everquest.aspx

Hi guys,

Referring to geo restrictions I think yes, it can be georestricted for ex. USSR countries(http://steamcommunity.com/app/225540/discussions/0/617319460973171504/)

I've checked it via UK proxy and it worked fine in this case. Also I checked it via US proxy and i got 'Internal error'. So might be it was some incorrect proxy - I use only one US proxy.

@ErgNoor I've checked with a bunch of another proxies and tor nodes all over the globe as well with same results.

@dstftw I mean this resource is available via UK proxy. I haven't used youtube-dl. I found it only today or yesterday. You can try to check with that - https://194.60.226.76:8080. but it's a little slow proxy.

@ErgNoor it's polish, not UK, and not even anonymous. And the most ridiculous thing is that it does work with this transparent proxy.

@dstftw it was filtered as UK on https://www.hidemyass.com/ :)

```
% This is the RIPE Database query service.
% The objects are in RPSL format.
...
inetnum:        194.60.226.0 - 194.60.227.255
netname:        SKYLINE-NET
descr:          Skyline Dawid Rachuta
country:        PL
...
```

GameInformer uses Brightcove AMF that is not supported currently. Same issues #6015, #5955.


Resolving patch:
diff --git a/youtube_dl/extractor/__init__.py b/youtube_dl/extractor/__init__.py
--- a/youtube_dl/extractor/__init__.py
+++ b/youtube_dl/extractor/__init__.py
@@ -200,6 +200,7 @@
 from .freespeech import FreespeechIE
 from .freevideo import FreeVideoIE
 from .funnyordie import FunnyOrDieIE
+from .gameinformer import GameInformerIE
 from .gamekings import GamekingsIE
 from .gameone import (
     GameOneIE,
diff --git a/youtube_dl/extractor/gameinformer.py b/youtube_dl/extractor/gameinformer.py
new file mode 100644
--- /dev/null
+++ b/youtube_dl/extractor/gameinformer.py
@@ -0,0 +1,43 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import int_or_none
+
+
+class GameInformerIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?gameinformer\.com/(?:[^/]+/)*(?P<id>.+)\.aspx'
+    _TEST = {
+        'url': 'http://www.gameinformer.com/b/features/archive/2015/09/26/replay-animal-crossing.aspx',
+        'info_dict': {
+            'id': '4515472681001',
+            'ext': 'm3u8',
+            'title': 'Replay - Animal Crossing',
+            'description': 'md5:2e211891b215c85d061adc7a4dd2d930',
+            'timestamp': 1443457610706,
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+    }
+
+    def _real_extract(self, url):
+        display_id = self._match_id(url)
+        webpage = self._download_webpage(url, display_id)
+
+        bc_api_url = self._search_regex(r"getVideo\('([^']+)'", webpage, 'brightcove api url')
+        json_data = self._download_json(
+            bc_api_url + '&video_fields=id,name,shortDescription,publishedDate,videoStillURL,length,IOSRenditions',
+            display_id)
+
+        return {
+            'id': compat_str(json_data['id']),
+            'display_id': display_id,
+            'url': json_data['IOSRenditions'][0]['url'],
+            'title': json_data['name'],
+            'description': json_data.get('shortDescription'),
+            'timestamp': int_or_none(json_data.get('publishedDate')),
+            'duration': int_or_none(json_data.get('length')),
+        }


Test patch: