Repository: ytdl-org/youtube-dl

Issue:
Wrong title & id extracted from www.paragon-software.com page
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like this: `[x]`)
- Use the *Preview* tab to see what your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.08.27.1*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [X] I've **verified** and **I assure** that I'm running youtube-dl **2017.08.27.1**

### Before submitting an *issue* make sure you have:
- [X] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [X] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [X] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl -v <your command line>`), copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):

```
Download Log for url: https://www.paragon-software.com/home/rk-free/keyscenarios.html

Waiting
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-errors', '--newline', '-v', '-i', '--format', 'mp4', '--recode-video', 'mp4', '--write-description', '-o', 'C:\\Users\\Bob\\Downloads\\YouTube-Downloader\\%(extractor)s\\%(title)s - %(uploader)s (%(upload_date)s) %(extractor)s-%(id)s.%(ext)s', 'https://www.paragon-software.com/home/rk-free/keyscenarios.html']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2017.08.27.1
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 3.1.5, ffprobe 3.1.5
[debug] Proxy map: {}
[generic] keyscenarios: Requesting header
WARNING: Falling back on generic information extractor.
[generic] keyscenarios: Downloading webpage
[generic] keyscenarios: Extracting information
[download] Downloading playlist: None
[generic] playlist None: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxBios.mp4'
[download] Destination: C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4
[download] 1.00KiB at 500.04KiB/s (00:00)
[download] 3.00KiB at 999.99KiB/s (00:00)
[download] 7.00KiB at  2.28MiB/s (00:00)
[download] 15.00KiB at  3.66MiB/s (00:00)
[download] 31.00KiB at  3.36MiB/s (00:00)
[download] 63.00KiB at 286.35KiB/s (00:00)
[download] 127.00KiB at 291.27KiB/s (00:00)
[download] 255.00KiB at 385.17KiB/s (00:00)
[download] 455.25KiB at 162.17KiB/s (00:02)
[download] 100% of 455.25KiB in 00:02
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 2 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxEfi.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
WARNING: There's no description to write.
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 3 of 4
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 4 of 4
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Finished downloading playlist: None
Download Worker Finished.
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-errors', '--newline', '-v', '-i', '--format', 'mp4', '--recode-video', 'mp4', '--write-description', '-o', 'C:\\Users\\Bob\\Downloads\\YouTube-Downloader\\%(extractor)s\\%(title)s - %(uploader)s (%(upload_date)s) %(extractor)s-%(id)s.%(ext)s', 'https://www.paragon-software.com/home/rk-free/keyscenarios.html']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2017.08.27.1
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 3.1.5, ffprobe 3.1.5
[debug] Proxy map: {}
[generic] keyscenarios: Requesting header
WARNING: Falling back on generic information extractor.
[generic] keyscenarios: Downloading webpage
[generic] keyscenarios: Extracting information
[download] Downloading playlist: None
[generic] playlist None: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxBios.mp4'
[download] Destination: C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4
[download] 1.00KiB at 999.83KiB/s (00:00)
[download] 3.00KiB at 999.99KiB/s (00:00)
[download] 7.00KiB at  2.28MiB/s (00:00)
[download] 15.00KiB at  3.66MiB/s (00:00)
[download] 31.00KiB at  3.78MiB/s (00:00)
[download] 63.00KiB at 310.33KiB/s (00:00)
[download] 127.00KiB at 319.08KiB/s (00:00)
[download] 255.00KiB at 423.56KiB/s (00:00)
[download] 455.25KiB at 566.20KiB/s (00:00)
WARNING: There's no description to write.
[download] 100% of 455.25KiB in 00:00
WARNING: There's no description to write.
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 2 of 4
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxEfi.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 3 of 4
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 4 of 4
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Finished downloading playlist: None
Download Worker Finished.

...
<end of log>
```

---

### Description of your *issue*, suggested solution and other information

All four (4) downloaded videos assigned the name:
    Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4
Expected video names, based on source filenames (in this order):
    (1)  LinuxBios - NA (NA) generic-LinuxBios.mp4
    (2)  LinuxEfi - NA (NA) generic-LinuxEfi.mp4
    (3)  WinPE - NA (NA) generic-WinPE.mp4
    (4)  WinPE_drivers - NA (NA) generic-WinPE_drivers.mp4
 
Correct title & id assigned if playlist URLs are downloaded individually:

Download Log for url: https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4

Waiting
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-errors', '--newline', '-v', '-i', '--format', 'mp4', '--recode-video', 'mp4', '--write-description', '-o', 'C:\\Users\\Bob\\Downloads\\YouTube-Downloader\\%(extractor)s\\%(title)s - %(uploader)s (%(upload_date)s) %(extractor)s-%(id)s.%(ext)s', 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2017.08.27.1
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 3.1.5, ffprobe 3.1.5
[debug] Proxy map: {}
[generic] WinPE_drivers: Requesting header
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4'
[download] Destination: C:\Users\Bob\Downloads\YouTube-Downloader\generic\WinPE_drivers - NA (NA) generic-WinPE_drivers.mp4
[download] 1.00KiB at 999.83KiB/s (00:00)
[download] 3.00KiB at  2.93MiB/s (00:00)
[download] 7.00KiB at  6.83MiB/s (00:00)
[download] 15.00KiB at  7.32MiB/s (00:00)
[download] 31.00KiB at 15.13MiB/s (00:00)
[download] 63.00KiB at 308.81KiB/s (00:00)
[download] 127.00KiB at 312.79KiB/s (00:00)
[download] 255.00KiB at 415.96KiB/s (00:00)
[download] 511.00KiB at 494.17KiB/s (00:01)
[download] 661.92KiB at 538.99KiB/s (00:01)
[download] 100% of 661.92KiB in 00:01
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\WinPE_drivers - NA (NA) generic-WinPE_drivers.mp4 - already is in target format mp4
Download Worker Finished.




Discussion:
I tried debugging this and did not get the same result. Please provide the actual command line you issued, to make it more easy to reproduce. Especially remove the file location and name restrictions if not relevant for the problem.
I just reproduced the problem using these command line arguments:
'--ignore-errors', '--newline', '-v', '-i', '--format', 'mp4', '--recode-video', 'mp4', '--write-description', '-o', 'C:\\Users\\Bob\\Downloads\\YouTube-Downloader\\%(extractor)s\\%(title)s - %(uploader)s (%(upload_date)s) %(extractor)s-%(id)s.%(ext)s', 'https://www.paragon-software.com/home/rk-free/keyscenarios.html'
All four downloads were given the file name & path:  C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4
This resulted in the first download being saved and the subsequent downloads deleted due to the file name conflict.

I have included a copy of the verbose log entries from this latest attempt below.
I appreciate your looking into this problem.
Thanks!
Bob Schneider
Download Log for url: https://www.paragon-software.com/home/rk-free/keyscenarios.html

Waiting
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-errors', '--newline', '-v', '-i', '--format', 'mp4', '--recode-video', 'mp4', '--write-description', '-o', 'C:\\Users\\Bob\\Downloads\\YouTube-Downloader\\%(extractor)s\\%(title)s - %(uploader)s (%(upload_date)s) %(extractor)s-%(id)s.%(ext)s', 'https://www.paragon-software.com/home/rk-free/keyscenarios.html']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2017.08.27.1
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 3.1.5, ffprobe 3.1.5
[debug] Proxy map: {}
[generic] keyscenarios: Requesting header
WARNING: Falling back on generic information extractor.
[generic] keyscenarios: Downloading webpage
[generic] keyscenarios: Extracting information
[download] Downloading playlist: None
[generic] playlist None: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxBios.mp4'
[download] Destination: C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4
[download] 1.00KiB at Unknown speed (00:00)
[download] 3.00KiB at Unknown speed (00:00)
[download] 7.00KiB at  3.42MiB/s (00:00)
[download] 15.00KiB at  7.32MiB/s (00:00)
[download] 31.00KiB at  2.52MiB/s (00:00)
[download] 63.00KiB at 308.81KiB/s (00:00)
[download] 127.00KiB at 319.88KiB/s (00:00)
[download] 255.00KiB at 424.27KiB/s (00:00)
[download] 455.25KiB at 146.42KiB/s (00:03)
[download] 100% of 455.25KiB in 00:03
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 2 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/LinuxEfi.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 3 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Downloading video 4 of 4
WARNING: There's no description to write.
[debug] Invoking downloader on 'https://www.paragon-software.com/sites/paragonsoftware.com/images/rmb/WinPE_drivers.mp4'
[download] C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 has already been downloaded
[download] 100% of 455.25KiB
[ffmpeg] Not converting video file C:\Users\Bob\Downloads\YouTube-Downloader\generic\Rescue Kit 14 Free Edition - Getting started - NA (NA) generic-keyscenarios.mp4 - already is in target format mp4
[download] Finished downloading playlist: None
Download Worker Finished.



      From: "kayb94 - notifications@github.com" <notifications@github.com>
 To: rg3/youtube-dl <youtube-dl@noreply.github.com> 
Cc: ShadowLandWalker <GitHub.M.Nyssa@xoxy.net>; Author <author@noreply.github.com>
 Sent: Thursday, August 31, 2017 2:08 PM
 Subject: Re: [rg3/youtube-dl] Wrong title & id extracted from www.paragon-software.com page (#14080)
   
I tried debugging this and did not get the same result. Please provide the actual command line you issued, to make it more easy to reproduce. Especially remove the file location and name restrictions if not relevant for the problem.—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.  

   

|  | Virus-free. www.avg.com  |


What I meant with "please provide the actual command line" was something like:
`$ youtube-dl http://url`
So I could somehow copy and paste it. But never mind, I can see you put some effort in describing everything.
Glad to be able to tell you I even found the bug now and fixed it. I'll make a pull request tomorrow. Hope they'll accept it. Thanks for reporting!
Thank you for fixing (esp. so quickly!).
The youtube-dl team is the best for support and producing a very reliable tool.
Thanks!
Bob Schneider

      From: "kayb94 - notifications@github.com" <notifications@github.com>
 To: rg3/youtube-dl <youtube-dl@noreply.github.com> 
Cc: ShadowLandWalker <GitHub.M.Nyssa@xoxy.net>; Author <author@noreply.github.com>
 Sent: Friday, September 1, 2017 8:57 PM
 Subject: Re: [rg3/youtube-dl] Wrong title & id extracted from www.paragon-software.com page (#14080)
   
What I meant with "please provide the actual command line" was something like:
$ youtube-dl http://url
So I could somehow copy and paste it. But never mind, I can see you put some effort in describing everything.
Glad to be able to tell you I even found the bug now and fixed it. I'll make a pull request tomorrow. Hope they'll accept it. Thanks for reporting!—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.  

   

|  | Virus-free. www.avg.com  |



Resolving patch:
diff --git a/youtube_dl/extractor/generic.py b/youtube_dl/extractor/generic.py
--- a/youtube_dl/extractor/generic.py
+++ b/youtube_dl/extractor/generic.py
@@ -1879,6 +1879,15 @@ class GenericIE(InfoExtractor):
                 'title': 'Building A Business Online: Principal Chairs Q & A',
             },
         },
+        {
+            # multiple HTML5 videos on one page
+            'url': 'https://www.paragon-software.com/home/rk-free/keyscenarios.html',
+            'info_dict': {
+                'id': 'keyscenarios',
+                'title': 'Rescue Kit 14 Free Edition - Getting started',
+            },
+            'playlist_count': 4,
+        }
         # {
         #     # TODO: find another test
         #     # http://schema.org/VideoObject
@@ -2874,13 +2883,20 @@ def merge_dicts(dict1, dict2):
         # Look for HTML5 media
         entries = self._parse_html5_media_entries(url, webpage, video_id, m3u8_id='hls')
         if entries:
-            for entry in entries:
-                entry.update({
+            if len(entries) == 1:
+                entries[0].update({
                     'id': video_id,
                     'title': video_title,
                 })
+            else:
+                for num, entry in enumerate(entries, start=1):
+                    entry.update({
+                        'id': '%s-%s' % (video_id, num),
+                        'title': '%s (%d)' % (video_title, num),
+                    })
+            for entry in entries:
                 self._sort_formats(entry['formats'])
-            return self.playlist_result(entries)
+            return self.playlist_result(entries, video_id, video_title)
 
         jwplayer_data = self._find_jwplayer_data(
             webpage, video_id, transform_source=js_to_json)


Test patch: