Repository: yt-dlp/yt-dlp

Issue:
Instagram stories: cannot download, says need to log in when already logged in on browser
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I'm running yt-dlp version **2023.09.24** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

USA

### Provide a description that is worded well enough to be understood

I am still able to download Instagram 'reels' just fine, but suddenly cannot download any 'stories' as of early morning on October 5th. It says I need to log in when I am already logged in to the same account on Chrome browser. I have been doing this exact same thing daily for the last several months with no problem (downloading 'stories' from multiple users). 

I've seen the notes about how using yt-dlp with Instagram is risky, but somehow I've never had an Instagram account banned due to using yt-dlp.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', '-o', '%(uploader)s-%(id)s.%(ext)s', '--cookies', 'instagram.com_cookies.txt', 'https:/
/www.instagram.com/stories/kimkardashian/3206599444580335888/']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version stable@2023.09.24 [088add956] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 2022-05-26-git-0dcbe1c1aa-full_build-www.gyan.dev (setts), ffprobe 2022-05-26-git-0dcbe1c1aa-fu
ll_build-www.gyan.dev, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.35.5, websockets-
11.0.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.09.24, Current version: stable@2023.09.24
Current Build Hash: 125465ac4416bcd4bf1632b899630ceecc08b341e83fdf10bd34700c58ea5152
yt-dlp is up to date (stable@2023.09.24)
[instagram:story] Extracting URL: https://www.instagram.com/stories/kimkardashian/3206599444580335888/
[instagram:story] 3206599444580335888: Downloading webpage
[instagram:story] 3206599444580335888: Downloading JSON metadata
ERROR: [instagram:story] 3206599444580335888: You need to log in to access this content. Use --cookies, --cookies-from-brows
er, --username and --password, --netrc-cmd, or --netrc (instagram) to provide account credentials
  File "yt_dlp\extractor\common.py", line 715, in extract
  File "yt_dlp\extractor\instagram.py", line 713, in _real_extract
  File "yt_dlp\extractor\common.py", line 1175, in raise_login_required
```



Discussion:
 try with --cookies-from-browser  chrome
@october262 doesn't work for me, sorry 

```yt-dlp -F "https://www.instagram.com/p/CyBMsCIAXnp/" --cookies-from-browser chrome
Extracting cookies from chrome
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/bin/yt-dlp/__main__.py", line 17, in <module>
  File "/usr/local/bin/yt-dlp/yt_dlp/__init__.py", line 1008, in main
  File "/usr/local/bin/yt-dlp/yt_dlp/__init__.py", line 962, in _real_main
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 678, in __init__
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 4084, in build_request_director
  File "/usr/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 4018, in cookiejar
  File "/usr/local/bin/yt-dlp/yt_dlp/cookies.py", line 91, in load_cookies
  File "/usr/local/bin/yt-dlp/yt_dlp/cookies.py", line 112, in extract_cookies_from_browser
  File "/usr/local/bin/yt-dlp/yt_dlp/cookies.py", line 273, in _extract_chrome_cookies
FileNotFoundError: could not find chrome cookies database in "/home/neilthecellist/.config/google-chrome"```
had the exact same output
btw, i'm getting the same errors when downloading Reels, Posts, Stories... It's not just Stories that's affected AFAICT
just checked again and i can still download reels and posts (videos in photo albums)

currently using gallery-dl as replacement, everything works fine with that
> had the exact same output

Please post it anyway (preferably as a verbose log)
this is with the new yt-dlp update

[debug] Command-line config: ['-vU', '-o', '%(uploader)s-%(id)s.%(ext)s', '--cookies-from-browser', 'chrome', 'https://www.i
nstagram.com/stories/kimkardashian/3206599444580335888/']
Extracting cookies from chrome
[debug] Extracting cookies from: "C:\Users\Wiseman\AppData\Local\Google\Chrome\User Data\Default\Network\Cookies"
[debug] Found local state file at "C:\Users\Wiseman\AppData\Local\Google\Chrome\User Data\Local State"
Extracted 3161 cookies from chrome
[debug] cookie version breakdown: {'v10': 3161, 'other': 0, 'unencrypted': 0}
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version stable@2023.10.07 [377e85a17] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 2022-05-26-git-0dcbe1c1aa-full_build-www.gyan.dev (setts), ffprobe 2022-05-26-git-0dcbe1c1aa-fu
ll_build-www.gyan.dev, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.35.5, websockets-
11.0.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.07, Current version: stable@2023.10.07
Current Build Hash: 7c809be9ad981b2a057bd60d766a9dffb1e88ddea6a9530c416f4741bc297364
yt-dlp is up to date (stable@2023.10.07)
[instagram:story] Extracting URL: https://www.instagram.com/stories/kimkardashian/3206599444580335888/
[instagram:story] 3206599444580335888: Downloading webpage
[instagram:story] 3206599444580335888: Downloading JSON metadata
ERROR: [instagram:story] 3206599444580335888: You need to log in to access this content. Use --cookies, --cookies-from-brows
er, --username and --password, --netrc-cmd, or --netrc (instagram) to provide account credentials
  File "yt_dlp\extractor\common.py", line 715, in extract
  File "yt_dlp\extractor\instagram.py", line 713, in _real_extract
  File "yt_dlp\extractor\common.py", line 1175, in raise_login_required
Getting the same with --cookies and --cookies-from-browser, also giving user and pass throws this error:

[debug] Command-line config: ['https://www.instagram.com/stories/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx/', '--username', 'PRIVATE', '--password', 'PRIVATE', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out cp1252 (No VT), error cp1252 (No VT), screen cp1252 (No VT)
[debug] yt-dlp version stable@2023.10.07 [377e85a17] (pip)
[debug] Python 3.11.4 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1u  30 May 2023)
[debug] exe versions: ffmpeg 2022-08-03-git-d3f48e68b3-full_build-www.gyan.dev (setts), ffprobe 2022-08-03-git-d3f48e68b3-full_build-www.gyan.dev, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.07.22, mutagen-1.46.0, sqlite3-3.42.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.07, Current version: stable@2023.10.07
yt-dlp is up to date (stable@2023.10.07)
[instagram:story] Downloading login webpage
ERROR: 3208732104181607427: An extractor error has occurred. (caused by KeyError('config')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "C:\Python311\Lib\site-packages\yt_dlp\extractor\common.py", line 712, in extract
    self.initialize()
  File "C:\Python311\Lib\site-packages\yt_dlp\extractor\common.py", line 618, in initialize
    self._perform_login(username, password)
  File "C:\Python311\Lib\site-packages\yt_dlp\extractor\instagram.py", line 67, in _perform_login
    'X-CSRFToken': shared_data['config']['csrf_token'],
                   ~~~~~~~~~~~^^^^^^^^^^
KeyError: 'config'



Same issue here, have been using "--cookies-from-browser firefox" for several years (so much so I've aliased it in my .bashrc), but since the update a few days ago, it's acting as if I'm not logged in, and specifying username & password gives the 'An extractor error has occurred. (caused by KeyError('config'))' error.

Replicated on my end as well:

```
[debug] Command-line config: ['--no-warnings', '--ffmpeg-location', '/home/ryan/scripts/yt-dl-ffmpeg/bin/ffmpeg', '-vU', 'https://www.instagram.com/stories/quod.si.noluerit/3227962093125742161/', '--cookies-from-browser', 'firefox']
Extracting cookies from firefox
[debug] Extracting cookies from: "/home/ryan/.mozilla/firefox/4zr5lql7.default-nightly/cookies.sqlite"
Extracted 1740 cookies from firefox
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (zip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-76-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg N-112634-gf2687a3b69-20231101 (setts), ffprobe N-112634-gf2687a3b69-20231101
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2020.06.20, secretstorage-3.3.1, sqlite3-3.37.2
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.13, Current version: stable@2023.10.13
Current Build Hash: be5cfb6be8930e1a5f427533ec32f2a481276b3da7b249d0150ce2b740ccf1ce
yt-dlp is up to date (stable@2023.10.13)
[instagram:story] Extracting URL: https://www.instagram.com/stories/quod.si.noluerit/3227962093125742161/
[instagram:story] 3227962093125742161: Downloading webpage
[instagram:story] 3227962093125742161: Downloading JSON metadata
ERROR: [instagram:story] 3227962093125742161: You need to log in to access this content. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (instagram) to provide account credentials
  File "/home/ryan/scripts/yt-dlp-core/yt_dlp/extractor/common.py", line 715, in extract
    ie_result = self._real_extract(url)
  File "/home/ryan/scripts/yt-dlp-core/yt_dlp/extractor/instagram.py", line 713, in _real_extract
    self.raise_login_required('You need to log in to access this content')
  File "/home/ryan/scripts/yt-dlp-core/yt_dlp/extractor/common.py", line 1175, in raise_login_required
    raise ExtractorError(msg, expected=True)
```
I'm having the same issue using the --cookies option and passing a Netscape formatted cookie file
--cookies-from-browser chrome
same result
stories only
posts/reels r ok
same result
facing this issue in stories only
posts/reels are ok
While debugging this issue I found that the user `id` value that is extracted from the JSON is always null while the `pk` value is populated with a number
After replacing the `id` key with `pk` in the following line I successfully downloaded a story video from Instagram
https://github.com/yt-dlp/yt-dlp/blob/87264d4fdadcddd91289b968dd0e4bf58d449267/yt_dlp/extractor/instagram.py#L706 
Can confirm @ozelentok fix works, just tried with the current master commit 0b6f8
I'm on the latest master release (`2023.12.21.210644`) and I still have this issue.

```sh
$ yt-dlp --cookies-from-browser firefox <Story URL>
Extracting cookies from firefox
Extracted 753 cookies from firefox
[instagram:story] Extracting URL: https://www.instagram.com/stories/user/3262/
[instagram:story] 3262: Downloading webpage
[instagram:story] 3262: Downloading JSON metadata
ERROR: [instagram:story] 3262: You need to log in to access this content. Use --cookies, --cookies-from-browser, --username and --password, --netrc-cmd, or --netrc (instagram) to provide account credentials
```

If I use `--username --password` instead, I get:

```sh
[instagram:story] Downloading login webpage
ERROR: 3262: An extractor error has occurred. (caused by KeyError('config')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
```
same result on all 3 latest branches (-U)
albeit w/ chrome cookies
but otherwise getting the exact same output and then the same error
Makes sense; I hadn't realised @ozelentok's patch wasn't merged yet. Someone could PR the fix if they're sure of it; I'm not sure of the repercussions of this change. Comparing with [gallery-dl](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/instagram.py) didn't particularly help me figure it out, although gallery-dl does work as expected.

Resolving patch:
diff --git a/yt_dlp/extractor/instagram.py b/yt_dlp/extractor/instagram.py
--- a/yt_dlp/extractor/instagram.py
+++ b/yt_dlp/extractor/instagram.py
@@ -10,6 +10,7 @@
     ExtractorError,
     decode_base_n,
     encode_base_n,
+    filter_dict,
     float_or_none,
     format_field,
     get_element_by_attribute,
@@ -703,28 +704,31 @@ def _real_extract(self, url):
         user_info = self._search_json(r'"user":', story_info, 'user info', story_id, fatal=False)
         if not user_info:
             self.raise_login_required('This content is unreachable')
-        user_id = user_info.get('id')
 
+        user_id = traverse_obj(user_info, 'pk', 'id', expected_type=str)
         story_info_url = user_id if username != 'highlights' else f'highlight:{story_id}'
+        if not story_info_url:  # user id is only mandatory for non-highlights
+            raise ExtractorError('Unable to extract user id')
+
         videos = traverse_obj(self._download_json(
             f'{self._API_BASE_URL}/feed/reels_media/?reel_ids={story_info_url}',
             story_id, errnote=False, fatal=False, headers=self._API_HEADERS), 'reels')
         if not videos:
             self.raise_login_required('You need to log in to access this content')
 
-        full_name = traverse_obj(videos, (f'highlight:{story_id}', 'user', 'full_name'), (str(user_id), 'user', 'full_name'))
+        full_name = traverse_obj(videos, (f'highlight:{story_id}', 'user', 'full_name'), (user_id, 'user', 'full_name'))
         story_title = traverse_obj(videos, (f'highlight:{story_id}', 'title'))
         if not story_title:
             story_title = f'Story by {username}'
 
-        highlights = traverse_obj(videos, (f'highlight:{story_id}', 'items'), (str(user_id), 'items'))
+        highlights = traverse_obj(videos, (f'highlight:{story_id}', 'items'), (user_id, 'items'))
         info_data = []
         for highlight in highlights:
             highlight_data = self._extract_product(highlight)
             if highlight_data.get('formats'):
                 info_data.append({
-                    **highlight_data,
                     'uploader': full_name,
                     'uploader_id': user_id,
+                    **filter_dict(highlight_data),
                 })
         return self.playlist_result(info_data, playlist_id=story_id, playlist_title=story_title)


Test patch: