Repository: yt-dlp/yt-dlp

Issue:
"Unknown content coding" error when trying to download from FC2 Live
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.03.08.1**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

_No response_

### Description

Tried downloading a few streams from https://live.fc2.com/ with the recently added FC2 Live extractor but yt-dlp throws an `Unknown content coding: br` error every time. Doesn't seem to be an issue with specific streams as I can't seem to find any live streams that downloads successfully.

### Verbose log

```shell
[debug] Command-line config: ['https://live.fc2.com/7964308/', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2022.03.08.1 [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg N-106355-ga24e6ca8bf-20220320 (setts), ffprobe N-106355-ga24e6ca8bf-20220320
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2022.03.08.1, Current version: 2022.03.08.1
yt-dlp is up to date (2022.03.08.1)
[debug] [fc2:live] Extracting URL: https://live.fc2.com/7964308/
[fc2:live] 7964308: Downloading webpage
[fc2:live] 7964308: Requesting member info
[fc2:live] 7964308: Downloading ControlServer data
[fc2:live] 7964308: Fetching HLS playlist info via WebSocket
[debug] [fc2:live] [debug] Sending HLS server request
[fc2:live] [debug] Server said: {"name":"connect_data","arguments":{"publish_start_time":1646662375737,"total_tip":0,"total_gift_cou...
[fc2:live] [debug] Server said: {"name":"user_count","arguments":{"pc_user_count":279,"pc_total_count":115609,"mobile_user_count":29...
[fc2:live] [debug] Server said: {"name":"video_information","arguments":{"type":"publish","width":1920,"height":1080}}
[fc2:live] [debug] Server said: {"name":"point_information","arguments":{"amount":0}}
[fc2:live] [debug] Server said: {"name":"ng_comment","arguments":{"admin_ng":1,"shared_ng_level":1,"ng_comments":[{"mode":"add","ng_...
[fc2:live] [debug] Server said: {"name":"comment","arguments":{"comments":[{"user_name":"[anonymous]","comment":"何でも 盗んで追跡","timestam...
[fc2:live] [debug] Server said: {"name":"user_count","arguments":{"pc_total_count":115610}}
[debug] [fc2:live] [debug] Goodbye.
[fc2:live] 7964308: Downloading m3u8 information
[fc2:live] 7964308: Downloading m3u8 information
[fc2:live] 7964308: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] 7964308: Downloading 1 format(s): playlists_middle_latency-4
[debug] Invoking downloader on "https://hls.live.fc2.com/hls/7964308/52/playlist?session_token=cfd1dcd2981f9a7f91321434454abe620f065aabc8933fa5a4116bbccd71f92c&playlist_token=c817bc697fb578b2ce8d055ded344529be08e299f4e484c08a73c4913d993853"
[download] Destination: ボブのスポンジ配信(˘•ᴥ•˘) 2022-03-21 16_19 [7964308].mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers "User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-encoding: gzip, deflate, br
Accept-language: en-us,en;q=0.5
Sec-fetch-mode: navigate
Cookie: PHPSESSID=oatf3girn9q2o6ls9h41arqejs; js-player_size=1; l_ortkn=c94a14e1075f1f2594da4fc996977b3fadc67d5a
" -i "https://hls.live.fc2.com/hls/7964308/52/playlist?session_token=cfd1dcd2981f9a7f91321434454abe620f065aabc8933fa5a4116bbccd71f92c&playlist_token=c817bc697fb578b2ce8d055ded344529be08e299f4e484c08a73c4913d993853" -c copy -f mp4 "file:ボブのスポンジ配信(˘•ᴥ•˘) 2022-03-21 16_19 [7964308].mp4.part"
ffmpeg version N-106355-ga24e6ca8bf-20220320 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220320
  libavutil      57. 24.101 / 57. 24.101
  libavcodec     59. 25.100 / 59. 25.100
  libavformat    59. 20.101 / 59. 20.101
  libavdevice    59.  6.100 / 59.  6.100
  libavfilter     8. 29.100 /  8. 29.100
  libswscale      6.  6.100 /  6.  6.100
  libswresample   4.  6.100 /  4.  6.100
  libpostproc    56.  5.100 / 56.  5.100
[tcp @ 0000016eaf2bc680] Starting connection attempt to 199.48.208.69 port 443
[tcp @ 0000016eaf2bc680] Successfully connected to 199.48.208.69 port 443
[https @ 0000016eb0a73380] Unknown content coding: br
[mp3 @ 0000016eb0a72dc0] Format mp3 detected only with low score of 1, misdetection possible!
[mp3 @ 0000016eb0a72dc0] Invalid frame size (1112): Could not seek to 1273.
[AVIOContext @ 0000016eb0a73f80] Statistics: 662 bytes read, 0 seeks
https://hls.live.fc2.com/hls/7964308/52/playlist?session_token=cfd1dcd2981f9a7f91321434454abe620f065aabc8933fa5a4116bbccd71f92c&playlist_token=c817bc697fb578b2ce8d055ded344529be08e299f4e484c08a73c4913d993853: Invalid argument


ERROR: ffmpeg exited with code 1
  File "yt_dlp\__main__.py", line 19, in <module>
  File "yt_dlp\__init__.py", line 864, in main
  File "yt_dlp\__init__.py", line 854, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3254, in download
  File "yt_dlp\YoutubeDL.py", line 3227, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1380, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1473, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1529, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2638, in process_video_result
  File "yt_dlp\YoutubeDL.py", line 3128, in process_info
  File "yt_dlp\YoutubeDL.py", line 2835, in dl
  File "yt_dlp\downloader\common.py", line 453, in download
  File "yt_dlp\downloader\fc2.py", line 41, in real_download
  File "yt_dlp\downloader\common.py", line 453, in download
  File "yt_dlp\downloader\external.py", line 68, in real_download
  File "yt_dlp\downloader\common.py", line 177, in report_error
  File "yt_dlp\YoutubeDL.py", line 939, in report_error
  File "yt_dlp\YoutubeDL.py", line 868, in trouble
```



Discussion:
 Temporary workaround:

`--add-header "Accept-Encoding:gzip, deflate"`


devs: Each downloader (except native) should have it's own Accept-Encoding header to what it supports, rather than assume it is the same as native.

Resolving patch:
diff --git a/yt_dlp/downloader/external.py b/yt_dlp/downloader/external.py
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -169,7 +169,7 @@ class CurlFD(ExternalFD):
     AVAILABLE_OPT = '-V'
 
     def _make_cmd(self, tmpfilename, info_dict):
-        cmd = [self.exe, '--location', '-o', tmpfilename]
+        cmd = [self.exe, '--location', '-o', tmpfilename, '--compressed']
         if info_dict.get('http_headers') is not None:
             for key, val in info_dict['http_headers'].items():
                 cmd += ['--header', '%s: %s' % (key, val)]
@@ -219,7 +219,7 @@ class WgetFD(ExternalFD):
     AVAILABLE_OPT = '--version'
 
     def _make_cmd(self, tmpfilename, info_dict):
-        cmd = [self.exe, '-O', tmpfilename, '-nv', '--no-cookies']
+        cmd = [self.exe, '-O', tmpfilename, '-nv', '--no-cookies', '--compression=auto']
         if info_dict.get('http_headers') is not None:
             for key, val in info_dict['http_headers'].items():
                 cmd += ['--header', '%s: %s' % (key, val)]
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -153,7 +153,6 @@ def random_user_agent():
 std_headers = {
     'User-Agent': random_user_agent(),
     'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
-    'Accept-Encoding': ', '.join(SUPPORTED_ENCODINGS),
     'Accept-Language': 'en-us,en;q=0.5',
     'Sec-Fetch-Mode': 'navigate',
 }
@@ -1392,6 +1391,9 @@ def http_request(self, req):
             if h.capitalize() not in req.headers:
                 req.add_header(h, v)
 
+        if 'Accept-encoding' not in req.headers:
+            req.add_header('Accept-encoding', ', '.join(SUPPORTED_ENCODINGS))
+
         req.headers = handle_youtubedl_headers(req.headers)
 
         if sys.version_info < (2, 7) and '#' in req.get_full_url():


Test patch: