Repository: yt-dlp
Issue: Receiving 403 response even with required headers
Labels: bug,triage,core:downloader

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a bug unrelated to a specific site
- [X] I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

I am currently trying to download remaining segments of deleted Twitch streams. With the link of a specific segment, I downloaded it through the browser and copied the request as CURL and the download works as expected in the terminal. I was even able to determine with curl that the only headers actually needed to receive the 200 response are the user-agent and accept-encoding. But using the same headers and link in yt-dlp lead to code 403. In the command output you can see that yt-dlp replaced the accept-encoding header value with "identity", I don't know if this is the problem. Note that as the link in question is from a deleted stream, the file may disappear from the Twitch server soon, I will leave [here](http://sprunge.us/VmPGiS) the link to the complete playlist with more segments and I can also send other links as examples. It's also worth noting that I first thought of using ffmpeg in place of yt-dlp, but it also returns 403.

curl output:
```
curl 'https://d2nvs31859zcd8.cloudfront.net/619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' -H 'Accept-Language: pt-BR,en-US;q=0.7,en;q=0.3' -H 'Accept-Encoding: gzip, deflate, br' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: none' -H 'Sec-Fetch-User: ?1' -Iv
*   Trying 65.8.212.230:443...
* Connected to d2nvs31859zcd8.cloudfront.net (65.8.212.230) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.cloudfront.net
*  start date: Feb  1 00:00:00 2022 GMT
*  expire date: Jan 31 23:59:59 2023 GMT
*  subjectAltName: host "d2nvs31859zcd8.cloudfront.net" matched cert's "*.cloudfront.net"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x55c02aea6c60)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> HEAD /619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts HTTP/2
> Host: d2nvs31859zcd8.cloudfront.net
> user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0
> accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
> accept-language: pt-BR,en-US;q=0.7,en;q=0.3
> accept-encoding: gzip, deflate, br
> connection: keep-alive
> upgrade-insecure-requests: 1
> sec-fetch-dest: document
> sec-fetch-mode: navigate
> sec-fetch-site: none
> sec-fetch-user: ?1
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200 
HTTP/2 200 
< content-type: video/MP2T
content-type: video/MP2T
< content-length: 9859096
content-length: 9859096
< date: Sat, 03 Sep 2022 09:43:43 GMT
date: Sat, 03 Sep 2022 09:43:43 GMT
< last-modified: Sat, 03 Sep 2022 09:21:30 GMT
last-modified: Sat, 03 Sep 2022 09:21:30 GMT
< etag: "48d0c2375a2a52bebb1b4ab11eac8e93"
etag: "48d0c2375a2a52bebb1b4ab11eac8e93"
< x-amz-server-side-encryption: AES256
x-amz-server-side-encryption: AES256
< accept-ranges: bytes
accept-ranges: bytes
< server: AmazonS3
server: AmazonS3
< x-cache: Hit from cloudfront
x-cache: Hit from cloudfront
< via: 1.1 fcfe689f8e288cc056be935c1e8696a4.cloudfront.net (CloudFront)
via: 1.1 fcfe689f8e288cc056be935c1e8696a4.cloudfront.net (CloudFront)
< x-amz-cf-pop: GRU3-C1
x-amz-cf-pop: GRU3-C1
< x-amz-cf-id: H_kcXbbe97PjCsv0DvOYtpk_3u_XrM7m3YlC77FlytyEDZnWhGS_2g==
x-amz-cf-id: H_kcXbbe97PjCsv0DvOYtpk_3u_XrM7m3YlC77FlytyEDZnWhGS_2g==
< age: 66208
age: 66208

< 
* Connection #0 to host d2nvs31859zcd8.cloudfront.net left intact
```

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
yt-dlp --print-traffic --hls-use-mpegts --hls-prefer-native --add-header 'User-Agent:Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0' --add-header 'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' --add-header 'Accept-Language:pt-BR,en-US;q=0.7,en;q=0.3' --add-header 'Accept-Encoding:gzip, deflate, br' --add-header 'Connection:keep-alive' --add-header 'Upgrade-Insecure-Requests:1' --add-header 'Sec-Fetch-Dest:document' --add-header 'Sec-Fetch-Mode:navigate' --add-header 'Sec-Fetch-Site:none' --add-header 'Sec-Fetch-User:?1' https://d2nvs31859zcd8.cloudfront.net/619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts -Uv
[debug] Command-line config: ['--print-traffic', '--hls-use-mpegts', '--hls-prefer-native', '--add-header', 'User-Agent:Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0', '--add-header', 'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8', '--add-header', 'Accept-Language:pt-BR,en-US;q=0.7,en;q=0.3', '--add-header', 'Accept-Encoding:gzip, deflate, br', '--add-header', 'Connection:keep-alive', '--add-header', 'Upgrade-Insecure-Requests:1', '--add-header', 'Sec-Fetch-Dest:document', '--add-header', 'Sec-Fetch-Mode:navigate', '--add-header', 'Sec-Fetch-Site:none', '--add-header', 'Sec-Fetch-User:?1', 'https://d2nvs31859zcd8.cloudfront.net/619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts', '-Uv']
[debug] Home config "yt-dlp.conf": ['-o', '/home/matheus/gallery-dl/youtube-dl/%(extractor)s-%(uploader)s-%(title)s-%(format_id)s.%(ext)s', '--no-mtime', '--restrict-filenames']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.09.01 [5d7c7d656]
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.4 (CPython 64bit) - Linux-5.19.5-051905-generic-x86_64-with-glibc2.35 (glibc 2.35)
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg N-107964-g7de9c0e9d7-20220902 (setts), ffprobe N-107964-g7de9c0e9d7-20220902
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2021.10.08, mutagen-1.45.1, secretstorage-3.3.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1670 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
send: b'GET /repos/yt-dlp/yt-dlp/releases/latest HTTP/1.1\r\nHost: api.github.com\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\nAccept-Language: pt-BR,en-US;q=0.7,en;q=0.3\r\nSec-Fetch-Mode: navigate\r\nAccept-Encoding: gzip, deflate, br\r\nConnection: close\r\nUpgrade-Insecure-Requests: 1\r\nSec-Fetch-Dest: document\r\nSec-Fetch-Site: none\r\nSec-Fetch-User: ?1\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: GitHub.com
header: Date: Sun, 04 Sep 2022 04:00:55 GMT
header: Content-Type: application/json; charset=utf-8
header: Cache-Control: public, max-age=60, s-maxage=60
header: Vary: Accept, Accept-Encoding, Accept, X-Requested-With
header: ETag: W/"f0c95084847d04409826a60941eb8cbf66459d9be854dfcf583dd9eef08939f8"
header: Last-Modified: Thu, 01 Sep 2022 11:30:53 GMT
header: X-GitHub-Media-Type: unknown, github.v3
header: Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
header: Access-Control-Allow-Origin: *
header: Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
header: X-Frame-Options: deny
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 0
header: Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
header: Content-Security-Policy: default-src 'none'
header: Content-Encoding: gzip
header: X-RateLimit-Limit: 60
header: X-RateLimit-Remaining: 59
header: X-RateLimit-Reset: 1662267655
header: X-RateLimit-Resource: core
header: X-RateLimit-Used: 1
header: Accept-Ranges: bytes
header: Content-Length: 3382
header: X-GitHub-Request-Id: 846A:40DB:1F4155:21E961:631422F7
header: connection: close
Latest version: 2022.09.01, Current version: 2022.09.01
yt-dlp is up to date (2022.09.01)
[debug] [generic] Extracting URL: https://d2nvs31859zcd8.cloudfront.net/619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts
[generic] 1043: Downloading webpage
send: b'GET /619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts HTTP/1.1\r\nHost: d2nvs31859zcd8.cloudfront.net\r\nAccept-Encoding: *\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\nAccept-Language: pt-BR,en-US;q=0.7,en;q=0.3\r\nSec-Fetch-Mode: navigate\r\nConnection: close\r\nUpgrade-Insecure-Requests: 1\r\nSec-Fetch-Dest: document\r\nSec-Fetch-Site: none\r\nSec-Fetch-User: ?1\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: video/MP2T
header: Content-Length: 9859096
header: Connection: close
header: Date: Sat, 03 Sep 2022 09:43:43 GMT
header: Last-Modified: Sat, 03 Sep 2022 09:21:30 GMT
header: ETag: "48d0c2375a2a52bebb1b4ab11eac8e93"
header: x-amz-server-side-encryption: AES256
header: Accept-Ranges: bytes
header: Server: AmazonS3
header: X-Cache: Hit from cloudfront
header: Via: 1.1 0782aedbf68330ecd7314fa250b28bca.cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: GRU1-C2
header: X-Amz-Cf-Id: EF_rNqw6uZlGBmn7PsMG6nV4Ovfc_QeBHuJwJCKkSJrXneGhgvI6bw==
header: Age: 65834
[debug] Identified a direct video link
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 1043: Downloading 1 format(s): mp2t
[debug] Invoking http downloader on "https://d2nvs31859zcd8.cloudfront.net/619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts"
send: b'GET /619feb30372bec807b2f_joeykaotyk_41215225691_1662186447/chunked/1043.ts HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: d2nvs31859zcd8.cloudfront.net\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\nAccept-Language: pt-BR,en-US;q=0.7,en;q=0.3\r\nSec-Fetch-Mode: navigate\r\nConnection: close\r\nUpgrade-Insecure-Requests: 1\r\nSec-Fetch-Dest: document\r\nSec-Fetch-Site: none\r\nSec-Fetch-User: ?1\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Content-Type: application/xml
header: Transfer-Encoding: chunked
header: Connection: close
header: Date: Sun, 04 Sep 2022 04:00:56 GMT
header: Server: AmazonS3
header: X-Cache: Error from cloudfront
header: Via: 1.1 09df991859478a73886f19d57ec67574.cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: GRU1-C2
header: X-Amz-Cf-Id: l45ZC6q30cVF-UNLexlH_BKCo1Vp6io3bhNBNcRjHw1lkfZfL-1Xaw==
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3171, in process_info
    success, real_download = self.dl(temp_filename, info_dict)
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 2895, in dl
    return fd.download(name, new_info, subtitle)
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/downloader/common.py", line 443, in download
    ret = self.real_download(filename, info_dict)
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/downloader/http.py", line 371, in real_download
    establish_connection()
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/downloader/http.py", line 129, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/home/matheus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3656, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
```