Repository: ytdl-org/youtube-dl

Issue:
[Pornhub] Cannot download private videos even with --cookies
## Please follow the guide below

- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like this: `[x]`)
- Use the *Preview* tab to see what your issue will actually look like

---

### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2018.02.22*. If it's not, read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [x] I've **verified** and **I assure** that I'm running youtube-dl **2018.02.22**

### Before submitting an *issue* make sure you have:
- [x] At least skimmed through the [README](https://github.com/rg3/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
- [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
- [x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

### What is the purpose of your *issue*?
- [x] Bug report (encountered problems with youtube-dl)
- [ ] Site support request (request for adding support for a new site)
- [ ] Feature request (request for a new functionality)
- [ ] Question
- [ ] Other


---

### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:

Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl -v <your command line>`), copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):

```
$ python -m youtube_dl -v -s https://www.pornhub.com/view_video.php?viewkey=ph5a91d5f3709a8 --cookies cookies.txt 
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-s', 'https://www.pornhub.com/view_video.php?viewkey=ph5a91d5f3709a8', '--cookies', 'cookies.txt']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2018.02.22
[debug] Git HEAD: 300148b
[debug] Python version 3.5.2 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-75976-gd59bfcd, ffprobe N-75976-gd59bfcd
[debug] Proxy map: {}
[PornHub] ph5a91d5f3709a8: Downloading webpage
ERROR: ph5a91d5f3709a8: PornHub said: Sorry, but this video is private. You must be friends with asjddsjfadsjk to view this video.
Traceback (most recent call last):
  File "C:\Users\Eitan\mine\programming\youtube-dl\youtube_dl\YoutubeDL.py", line 785, in extract_info
    ie_result = ie.extract(url)
  File "C:\Users\Eitan\mine\programming\youtube-dl\youtube_dl\extractor\common.py", line 440, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\Eitan\mine\programming\youtube-dl\youtube_dl\extractor\pornhub.py", line 238, in _real_extract
    expected=True, video_id=video_id)
youtube_dl.utils.ExtractorError: ph5a91d5f3709a8: PornHub said: Sorry, but this video is private. You must be friends with asjddsjfadsjk to view this video.
```

cookies.txt (just as an example):
```
# Netscape HTTP Cookie File
.pornhub.com	TRUE	/	FALSE	1574537958	_ga	GA1.2.1603300915.1511465945
.pornhub.com	TRUE	/	FALSE	3354375139	bs	mk81f1s9d231mcphc8z3f9bg7cnk1dby
.pornhub.com	TRUE	/	FALSE	1519593090	desired_username	asjddsjfadsjk%7Casjddsjfadsjk%40getnada.com
.pornhub.com	TRUE	/	FALSE	1522185176	expiredEnterModalShown	1
.pornhub.com	TRUE	/	FALSE	2895287351	platform	pc
.pornhub.com	TRUE	/	FALSE	1551043569	ss	579789539714731287
.pornhub.com	TRUE	/	FALSE	3223004990	userSession	tu7w3rvvdhpme351lwf7qf17pm99xidm
```

---

### Description of your *issue*, suggested solution and other information

I logged in from Firefox, verified that I can watch the video, exported my cookies and fed the resulting cookies.txt to youtube-dl, yet it still says "You must be friends with so-and-so". 

Here are the credentials for the video if anybody wants to use it to test:
```
https://www.pornhub.com/view_video.php?viewkey=ph5a91d5f3709a8
username asjddsjfadsjk
password asjddsjfadsjkasjddsjfadsjk
```

For a Youtube private video the --cookies argument **does** work. According to #8603, --cookies did work with Pornhub as of 2016-02-16. Also, curl **does** work with the same cookies.txt (`curl -b cookies.txt https://www.pornhub.com/view_video.php?viewkey=ph5a91d5f3709a8 | grep mediastring`) so maybe I'm just doing something wrong.



Discussion:
Please use this account:
```
username asjddsjfadsjk
password asjddsjfadsjkasjddsjfadsjk
```
Yeah I really don't get it. I can't manage to log in programmatically at all. I copied Firefox's requests exactly as curl commands, only substituting the random token, and yet still it rejects my login attempt, giving "Session timed out - reload and try again". Script below:
```
#!/usr/bin/bash
set -euo pipefail

cookie_jar=curl-cookies.txt
curl_cmd='curl -s -f --cookie-jar curl-cookies.txt -D /dev/stderr'

rm -f "$cookie_jar"

function sleep_random() {
echo >&2 'sleeping between requests...'
sleep $(( 2 + ( RANDOM % 4 ) ))
sleep $(printf '%.4f\n' ".$RANDOM")
}

login_page=$($curl_cmd "https://www.pornhub.com/login" \
-H "Host: www.pornhub.com" \
-H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0" \
-H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" \
-H "Accept-Language: en-US,en;q=0.5" --compressed \
-H "DNT: 1" \
-H "Connection: keep-alive" \
-H "Upgrade-Insecure-Requests: 1")

redirect=$(echo "$login_page"|sed -nr 's/.*redirect" value="(.+?)".*/\1/p'|uniq)
token=$(echo "$login_page"|sed -nr 's/.*token" value="(.+?)".*/\1/p'|uniq)

if [[ -z $redirect || -z $token ]]
    then
    echo >&2 'Could not extract hidden inputs'
    exit 1
fi

echo >&2 "redirect=$redirect"
echo >&2 "token=$token"
cat >&2 "$cookie_jar"
# exit

$curl_cmd "https://www.pornhub.com/front/menu_community?token=$token" -H "Host: www.pornhub.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0" -H "Accept: */*" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "Referer: https://www.pornhub.com/login" -H "DNT: 1" -H "Connection: keep-alive" >/dev/null
$curl_cmd "https://www.pornhub.com/front/menu_livesex?token=$token" -H "Host: www.pornhub.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0" -H "Accept: */*" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "Referer: https://www.pornhub.com/login" -H "DNT: 1" -H "Connection: keep-alive" >/dev/null
$curl_cmd "https://www.pornhub.com/front/menu_photos?token=$token" -H "Host: www.pornhub.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0" -H "Accept: */*" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "Referer: https://www.pornhub.com/login" -H "DNT: 1" -H "Connection: keep-alive" >/dev/null

sleep_random
$curl_cmd "https://www.pornhub.com/front/authenticate" \
-H "Host: www.pornhub.com" \
-H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0" \
-H "Accept: application/json, text/javascript, */*; q=0.01" \
-H "Accept-Language: en-US,en;q=0.5" --compressed \
-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" \
-H "X-Requested-With: XMLHttpRequest" \
-H "Referer: https://www.pornhub.com/login" \
-H "DNT: 1" \
-H "Connection: keep-alive" --data "loginPage=1&redirect=$redirect&token=$token&taste_profile=&username=asjddsjfadsjk&password=asjddsjfadsjkasjddsjfadsjk&remember_me=on" \
| head
```
Is there something other than headers, cookies, and POST data that needs to be faked?
This patch fixes it
```diff
diff --git a/youtube_dl/extractor/pornhub.py b/youtube_dl/extractor/pornhub.py
index 281a4f05e..5afa35dd2 100644
--- a/youtube_dl/extractor/pornhub.py
+++ b/youtube_dl/extractor/pornhub.py
@@ -116,11 +116,11 @@ class PornHubIE(InfoExtractor):
         video_id = self._match_id(url)
 
         def dl_webpage(platform):
+            self._set_cookie('pornhub.com', 'age_verified', '1')
+            self._set_cookie('pornhub.com', 'platform', platform)
             return self._download_webpage(
                 'http://www.pornhub.com/view_video.php?viewkey=%s' % video_id,
-                video_id, headers={
-                    'Cookie': 'age_verified=1; platform=%s' % platform,
-                })
+                video_id)
 
         webpage = dl_webpage('pc')
```

using a `cookies.txt` that contains
```
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

.pornhub.com	TRUE	/	FALSE	3355496222	bs	7j74bzosuxof58vpnzq2sa0yih5e00r4
.pornhub.com	TRUE	/	FALSE	1522185176	expiredEnterModalShown	1
.pornhub.com	TRUE	/	FALSE	3223004990	il	v1ijgTdGXTZHyST9GU2E1g8rRHDPCLtMXeq846YFHV5eYxNTIwMTUyOTQwaC1GSm5pLUhqZENDTG9RWVVKTnVVaHplTmlRbVA2VlVkZnMzZ1ZWMw..
.pornhub.com	TRUE	/	FALSE	1551604111	ss	257851818681477916
```
it works fine
Oh, yes that fixes this issue. Will you make a pull request?

Resolving patch:
diff --git a/youtube_dl/extractor/pornhub.py b/youtube_dl/extractor/pornhub.py
--- a/youtube_dl/extractor/pornhub.py
+++ b/youtube_dl/extractor/pornhub.py
@@ -114,13 +114,14 @@ def _extract_count(self, pattern, webpage, name):
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
+        
+        self._set_cookie('pornhub.com', 'age_verified', '1')
 
         def dl_webpage(platform):
+            self._set_cookie('pornhub.com', 'platform', platform)
             return self._download_webpage(
                 'http://www.pornhub.com/view_video.php?viewkey=%s' % video_id,
-                video_id, headers={
-                    'Cookie': 'age_verified=1; platform=%s' % platform,
-                })
+                video_id)
 
         webpage = dl_webpage('pc')
 


Test patch: