Repository: ytdl-org/youtube-dl

Issue:
Twitter: Unsupported URL
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like this: `[x]`)
- Use the *Preview* tab to see what your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.09.15*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.09.15**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [x] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl -v <your command line>`), copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):

```
youtube-dl.py -v "https://twitter.com/i/web/status/910031516746514432"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://twitter.com/i/web/status/910031516746514432']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.09.15
[debug] Python version 3.5.1 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-71727-g46778ab, ffprobe 3.2, rtmpdump 2.4
[debug] Proxy map: {}
[generic] 910031516746514432: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 910031516746514432: Downloading webpage
[generic] 910031516746514432: Extracting information
[redirect] Following redirect to https://mobile.twitter.com/i/nojs_router?path=/i/web/status/910031516746514432
[generic] 910031516746514432: Requesting header
[redirect] Following redirect to https://mobile.twitter.com/i/web/status/910031516746514432
[generic] 910031516746514432: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 910031516746514432: Downloading webpage
[generic] 910031516746514432: Extracting information
ERROR: Unsupported URL: https://mobile.twitter.com/i/web/status/910031516746514432
Traceback (most recent call last):
  File "C:\Transmogrifier\youtube-dl.py\youtube_dl\YoutubeDL.py", line 776, in extract_info
    ie_result = ie.extract(url)
  File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\common.py", line 434, in extract
    ie_result = self._real_extract(url)
  File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\generic.py", line 2964, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://mobile.twitter.com/i/web/status/910031516746514432


---

### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Single video: https://twitter.com/i/web/status/910031516746514432
- Single video: https://mobile.twitter.com/i/web/status/910031516746514432

(same video...similar result)

---

### Description of your *issue*, suggested solution and other information

Direct link to video does work, using generic extractor/FFMPEG:

https://video.twimg.com/ext_tw_video/910030238373089285/pu/pl/1280x720/Ckxc5Q4SjNkjNVw5.m3u8

Thanks
Ringo



Discussion:
The redirect page: https://mobile.twitter.com/i/nojs_router?path=/i/web/status/910031516746514432

contains a link to the URL: https://twitter.com/Prefet971/status/910031516746514432/video/1


` <div class="dir-ltr" dir="ltr">  [Direct] <a href="/hashtag/Maria?src=hash"data-query-source="hashtag_click"class="twitter-hashtag dir-ltr"dir="ltr">#Maria</a> Le centre se trouve actuellement au sud de Basse-Terre. Restez confinÃ©s. RÃ©fugiez-vous dans la piÃ¨ce la + sÃ»re. <a href="https://t.co/mwx01Rs4lo"data-pre-embedded="true"rel="nofollow"data-entity-id="910030238373089285"dir="ltr"data-url="https://twitter.com/Prefet971/status/910031516746514432/video/1"data-tco-id="mwx01Rs4lo"class="twitter_external_link dir-ltr tco-link has-expanded-path"target="_top"data-expanded-path="/Prefet971/status/910031516746514432/video/1">pic.twitter.com/mwx01Rs4lo</a>
</div>
`


YouTube-DL downloades the video from this page correctly.


Resolving patch:
diff --git a/youtube_dl/extractor/twitter.py b/youtube_dl/extractor/twitter.py
--- a/youtube_dl/extractor/twitter.py
+++ b/youtube_dl/extractor/twitter.py
@@ -242,8 +242,9 @@ def _real_extract(self, url):
 
 class TwitterIE(InfoExtractor):
     IE_NAME = 'twitter'
-    _VALID_URL = r'https?://(?:www\.|m\.|mobile\.)?twitter\.com/(?P<user_id>[^/]+)/status/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.|m\.|mobile\.)?twitter\.com/(?:i/web|(?P<user_id>[^/]+))/status/(?P<id>\d+)'
     _TEMPLATE_URL = 'https://twitter.com/%s/status/%s'
+    _TEMPLATE_STATUSES_URL = 'https://twitter.com/statuses/%s'
 
     _TESTS = [{
         'url': 'https://twitter.com/freethenipple/status/643211948184596480',
@@ -322,9 +323,9 @@ class TwitterIE(InfoExtractor):
         'info_dict': {
             'id': 'MIOxnrUteUd',
             'ext': 'mp4',
-            'title': 'FilmDrunk - Vine of the day',
-            'description': 'FilmDrunk on Twitter: "Vine of the day https://t.co/xmTvRdqxWf"',
-            'uploader': 'FilmDrunk',
+            'title': 'Vince Mancini - Vine of the day',
+            'description': 'Vince Mancini on Twitter: "Vine of the day https://t.co/xmTvRdqxWf"',
+            'uploader': 'Vince Mancini',
             'uploader_id': 'Filmdrunk',
             'timestamp': 1402826626,
             'upload_date': '20140615',
@@ -372,6 +373,21 @@ class TwitterIE(InfoExtractor):
         'params': {
             'format': 'best[format_id^=http-]',
         },
+    }, {
+        'url': 'https://twitter.com/i/web/status/910031516746514432',
+        'info_dict': {
+            'id': '910031516746514432',
+            'ext': 'mp4',
+            'title': 'Préfet de Guadeloupe - [Direct] #Maria Le centre se trouve actuellement au sud de Basse-Terre. Restez confinés. Réfugiez-vous dans la pièce la + sûre.',
+            'thumbnail': r're:^https?://.*\.jpg',
+            'description': 'Préfet de Guadeloupe on Twitter: "[Direct] #Maria Le centre se trouve actuellement au sud de Basse-Terre. Restez confinés. Réfugiez-vous dans la pièce la + sûre. https://t.co/mwx01Rs4lo"',
+            'uploader': 'Préfet de Guadeloupe',
+            'uploader_id': 'Prefet971',
+            'duration': 47.48,
+        },
+        'params': {
+            'skip_download': True,  # requires ffmpeg
+        },
     }]
 
     def _real_extract(self, url):
@@ -380,11 +396,15 @@ def _real_extract(self, url):
         twid = mobj.group('id')
 
         webpage, urlh = self._download_webpage_handle(
-            self._TEMPLATE_URL % (user_id, twid), twid)
+            self._TEMPLATE_STATUSES_URL % twid, twid)
 
         if 'twitter.com/account/suspended' in urlh.geturl():
             raise ExtractorError('Account suspended by Twitter.', expected=True)
 
+        if user_id is None:
+            mobj = re.match(self._VALID_URL, urlh.geturl())
+            user_id = mobj.group('user_id')
+
         username = remove_end(self._og_search_title(webpage), ' on Twitter')
 
         title = description = self._og_search_description(webpage).strip('').replace('\n', ' ').strip('“”')


Test patch: