Repository: ytdl-org/youtube-dl

Issue:
improbable flash object is really youtube video
### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.04.03*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.04.03**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [x] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
```
youtube-dl -v 'http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.04.03
[debug] Python version 2.7.13 - Linux-4.9.0-1-rt-686-pae-i686-with-debian-9.0
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[generic] untrained-modern-youths-and-ancient-masters-in-selfie-portraits: Requesting header
WARNING: Falling back on generic information extractor.
[generic] untrained-modern-youths-and-ancient-masters-in-selfie-portraits: Downloading webpage
[generic] untrained-modern-youths-and-ancient-masters-in-selfie-portraits: Extracting information
ERROR: Unsupported URL: http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/
Traceback (most recent call last):
  File "/home/ben/.bin/youtube-dl/youtube_dl/extractor/generic.py", line 1805, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/home/ben/.bin/youtube-dl/youtube_dl/compat.py", line 2526, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "/home/ben/.bin/youtube-dl/youtube_dl/compat.py", line 2515, in _XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
    raise err
ParseError: undefined entity &raquo;: line 8, column 27
Traceback (most recent call last):
  File "/home/ben/.bin/youtube-dl/youtube_dl/YoutubeDL.py", line 761, in extract_info
    ie_result = ie.extract(url)
  File "/home/ben/.bin/youtube-dl/youtube_dl/extractor/common.py", line 429, in extract
    ie_result = self._real_extract(url)
  File "/home/ben/.bin/youtube-dl/youtube_dl/extractor/generic.py", line 2667, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/

<end of log>
```

---

---

### Description of your *issue*, suggested solution and other information

http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/ contains a flash video specifically http://www.youtube.com/v/msN87y-iEx0&rel=0 (which dowloads fine using that url.



Discussion:
Unless I'm mistaken, I really don't believe the aim of youtube-dl is to extract arbitrary webpages to look for embedded videos. If that's what you need, I'd suggest building your own scraper to get those urls and then feeding them into youtube-dl.

Resolving patch:
diff --git a/youtube_dl/extractor/generic.py b/youtube_dl/extractor/generic.py
--- a/youtube_dl/extractor/generic.py
+++ b/youtube_dl/extractor/generic.py
@@ -730,6 +730,21 @@ class GenericIE(InfoExtractor):
                 'skip_download': True,
             }
         },
+        # YouTube <object> embed
+        {
+            'url': 'http://www.improbable.com/2017/04/03/untrained-modern-youths-and-ancient-masters-in-selfie-portraits/',
+            'md5': '516718101ec834f74318df76259fb3cc',
+            'info_dict': {
+                'id': 'msN87y-iEx0',
+                'ext': 'webm',
+                'title': 'Feynman: Mirrors FUN TO IMAGINE 6',
+                'upload_date': '20080526',
+                'description': 'md5:0ffc78ea3f01b2e2c247d5f8d1d3c18d',
+                'uploader': 'Christopher Sykes',
+                'uploader_id': 'ChristopherJSykes',
+            },
+            'add_ie': ['Youtube'],
+        },
         # Camtasia studio
         {
             'url': 'http://www.ll.mit.edu/workshops/education/videocourses/antennas/lecture1/video/',
@@ -1938,6 +1953,7 @@ def _real_extract(self, url):
                 data-video-url=|
                 <embed[^>]+?src=|
                 embedSWF\(?:\s*|
+                <object[^>]+data=|
                 new\s+SWFObject\(
             )
             (["\'])


Test patch: