Repository: ytdl-org/youtube-dl

Issue:
[fox.com] unable to get season_number and episode_number
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
- Use *Preview* tab to see how your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2017.02.21*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2017.02.21**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones

### What is the purpose of your *issue*?
- [ ] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [x] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other

---

### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*

---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
```
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(season)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.17
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
NA sNAeNA - Chris Has Got a Date, Date, Date, Date, Date.mp4
```

---

### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
- Single video: http://www.fox.com/watch/868757571821/7684301568
- Single video: http://www.fox.com/watch/853172291669/7684301568
- Single video: http://www.fox.com/watch/835908163744/7684301568
- Single video: http://www.fox.com/watch/796751939630/7684301568

Note that **youtube-dl does not support sites dedicated to [copyright infringement](https://github.com/rg3/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. In order for site support request to be accepted all provided example URLs should not violate any copyrights.



---

### Description of your *issue*, suggested solution and other information

Unable to get season, season_number, and episode_number.  Output results in NA for each field.  I am able to grab the title.  The links I provided clearly show the season number and episode number on the web page.  I think the info is contained in the following script example from the last link I provided above. 

```
<script type="application/ld+json">{"@context":"http:\/\/schema.org","@id":"801550915931","@type":"TVEpisode","name":"Chris Has Got a Date, Date, Date, Date, Date","description":"Taylor Swift agrees to go to a homecoming dance with Chris after he asks her via online video.","url":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568","image":"http:\/\/www.fox.com\/sites\/default\/files\/video-image-thumb\/eacx02_376_03a_0152_hires2_1280x720_801543747530.jpg","episodeNumber":"5","partOfSeason":{"@type":"TVSeason","seasonNumber":"15"},"partOfSeries":{"@type":"TVSeries","name":"Family Guy","sameAs":"http:\/\/www.imdb.com\/title\/tt0182576\/","url":"http:\/\/www.fox.com\/family-guy"},"releasedEvent":{"@type":"PublicationEvent","startDate":"2016-11-06","location":{"@type":"Country","name":"US"}},"potentialAction":[{"@type":"WatchAction","target":["http:\/\/www.fox.com\/watch\/796751939630\/7684301568?cmpid=xtra:-FOX-Goog_Watch","android-app:\/\/com.fox.now\/foxapp\/video_fullepisode\/freewheel\/796751939630?cmpid=xtra:-FOX-Goog_Watch","ios-app:\/\/571096102\/foxapp\/video_fullepisode\/freewheel\/796751939630?cmpid=xtra:-FOX-Goog_Watch"],"expectsAcceptanceOf":[{"@type":"Offer","availabilityStarts":"1969-12-31T19:00:00-05:00","availabilityEnds":"2016-11-14T03:01:00-05:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2016-11-14T03:01:00-05:00","availabilityEnds":"2017-01-09T02:59:00-05:00","category":"free","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2017-01-09T02:59:00-05:00","availabilityEnds":"2017-06-29T02:59:00-04:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]}]},{"@type":"WatchAction","target":[{"@type":"EntryPoint","urlTemplate":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568?cmpid=xtra:-FOX-Goog_Watch","actionPlatform":"http:\/\/schema.org\/DesktopWebPlatform","inLanguage":"en"},{"@type":"EntryPoint","urlTemplate":"android-app:\/\/com.fox.now\/foxapp\/video_fullepisode\/freewheel\/796751939630?extcmp=xtra_FOXNOW_Goog_Watch","actionPlatform":"http:\/\/schema.org\/AndroidPlatform","inLanguage":"en"},{"@type":"EntryPoint","urlTemplate":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568?extcmp=xtra_FOXNOW_Goog_Watch","actionPlatform":"http:\/\/schema.org\/IOSPlatform","inLanguage":"en"}],"expectsAcceptanceOf":[{"@type":"Offer","availabilityStarts":"1969-12-31T19:00:00-05:00","availabilityEnds":"2016-11-14T03:01:00-05:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2016-11-14T03:01:00-05:00","availabilityEnds":"2017-01-09T02:59:00-05:00","category":"free","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2017-01-09T02:59:00-05:00","availabilityEnds":"2017-06-29T02:59:00-04:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]}]}],"duration":"PT21M36S"}</script><meta name="description" content="FULL EPISODE - &quot;Chris Has Got a Date, Date, Date, Date, Date&quot; - Taylor Swift agrees to go to a homecoming dance with Chris after he asks her via online video."/>
```

This issue only occurs on Fox.  I do not have this issue with other major networks.  


Discussion:
Here's my fix:
```
# coding: utf-8
from __future__ import unicode_literals

from .adobepass import AdobePassIE
from ..utils import (
    int_or_none,
    smuggle_url,
    update_url_query,
)


class FOXIE(AdobePassIE):
    _VALID_URL = r'https?://(?:www\.)?fox\.com/watch/(?P<id>[0-9]+)'
    _TEST = {
        'url': 'http://www.fox.com/watch/255180355939/7684182528',
        'md5': 'ebd296fcc41dd4b19f8115d8461a3165',
        'info_dict': {
            'id': '255180355939',
            'ext': 'mp4',
            'title': 'Official Trailer: Gotham',
            'description': 'Tracing the rise of the great DC Comics Super-Villains and vigilantes, Gotham reveals an entirely new chapter that has never been told.',
            'duration': 129,
            'timestamp': 1400020798,
            'upload_date': '20140513',
            'uploader': 'NEWA-FNG-FOXCOM',
        },
        'add_ie': ['ThePlatform'],
    }

    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)

        video_metadata = self._search_json_ld(webpage, video_id, fatal=False)
        settings = self._parse_json(self._search_regex(
            r'jQuery\.extend\(Drupal\.settings\s*,\s*({.+?})\);',
            webpage, 'drupal settings'), video_id)
        fox_pdk_player = settings['fox_pdk_player']
        release_url = fox_pdk_player['release_url']
        query = {
            'mbr': 'true',
            'switch': 'http'
        }
        if fox_pdk_player.get('access') == 'locked':
            ap_p = settings['foxAdobePassProvider']
            rating = ap_p.get('videoRating')
            if rating == 'n/a':
                rating = None
            resource = self._get_mvpd_resource('fbc-fox', None, ap_p['videoGUID'], rating)
            query['auth'] = self._extract_mvpd_auth(url, video_id, 'fbc-fox', resource)

        return {
            '_type': 'url_transparent',
            'ie_key': 'ThePlatform',
            'url': smuggle_url(update_url_query(release_url, query), {'force_smil_url': True}),
            'id': video_id,
            'series': video_metadata.get('series'),
            'season_number': int_or_none(video_metadata.get('season_number')),
            'episode_number': int_or_none(video_metadata.get('episode_number')),
            'episode': video_metadata.get('episode'),
        }

```
Here is the latest verbose output with version 2017.02.22.  I have two computers and one was a few days out of date, although both had the same results.  
```
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(season)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.22
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
NA sNAeNA - Chris Has Got a Date, Date, Date, Date, Date.mp4
```
Am I able to test the above fix proposed by @gkoelln? 

I installed Python 3.4.4, downloaded the current Master branch code, updated fox.py with gkoelln's suggested code, tried running python -m YoutubDl.py and get a SystemError about the Parent module '' not loaded.  I am not sure where to go from there.  I am trying though. 
Ok, I figured out how to test the code and it was successful!!  Here is my output.

```
C:\Python34\Scripts>python -m youtube_dl --ap-mso DTV --ap-username 'PRIVATE' --ap-password 'PRIVATE" -v --get-filename -o "%(series)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s" http://www.fox.com/watch/796751939630/7684301568

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(series)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.22
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
Family Guy s15e5 - Chris Has Got a Date, Date, Date, Date, Date.mp4
```
How do we get this code submitted for review?  I am new to all of this.  
Well, either the developers can make my proposed changes above, or I can make those changes on my fork and do a pull request.  Though, I haven't had much success with any of my pull requests being merged so far.
@dstftw, what do you think about this? 

Resolving patch:
diff --git a/youtube_dl/extractor/fox.py b/youtube_dl/extractor/fox.py
--- a/youtube_dl/extractor/fox.py
+++ b/youtube_dl/extractor/fox.py
@@ -3,6 +3,7 @@
 
 from .adobepass import AdobePassIE
 from ..utils import (
+    int_or_none,
     smuggle_url,
     update_url_query,
 )
@@ -47,9 +48,12 @@ def _real_extract(self, url):
             resource = self._get_mvpd_resource('fbc-fox', None, ap_p['videoGUID'], rating)
             query['auth'] = self._extract_mvpd_auth(url, video_id, 'fbc-fox', resource)
 
-        return {
+        info = self._search_json_ld(webpage, video_id, fatal=False)
+        info.update({
             '_type': 'url_transparent',
             'ie_key': 'ThePlatform',
             'url': smuggle_url(update_url_query(release_url, query), {'force_smil_url': True}),
             'id': video_id,
-        }
+        })
+
+        return info


Test patch: