Repository: qutebrowser/qutebrowser

Issue:
PDF.js viewer tab corrupts session
<details>
<summary><strong>Version info</strong></summary>

```
qutebrowser v2.5.4
Git commit: bb302db71-dirty on makepkg (2023-05-08 17:21:20 +1200)
Backend: QtWebEngine 6.5, based on Chromium 108.0.5359.220 (from api)
Qt: 6.5.0

CPython: 3.11.3
PyQt: 6.5.0

sip: no
colorama: 0.4.6
jinja2: 3.1.2
pygments: 2.15.1
yaml: 6.0
adblock: 0.6.0
objc: no
PyQt6.QtWebEngineCore: 6.5.0
pdf.js: 3.6.172 (/usr/share/pdf.js/build/pdf.js)
sqlite: 3.41.2
QtNetwork SSL: OpenSSL 3.0.8 7 Feb 2023

Style: QFusionStyle
Platform plugin: wayland
OpenGL: Intel, 4.6 (Compatibility Profile) Mesa 23.0.3
Platform: Linux-6.3.1-zen1-1-zen-x86_64-with-glibc2.37, 64bit
Linux distribution: Arch Linux (arch)
Frozen: False
Imported from /usr/lib/python3.11/site-packages/qutebrowser
Using Python from /usr/bin/python3
Qt library executable path: /usr/lib/qt6, data path: /usr/share/qt6
```

</details>

**Does the bug happen if you start with `--temp-basedir`?**:

Yes, but only on the second try

**Description**

After quitting qutebrowser with `:wq` (or `:q` with `auto_save.session = True`) when a PDF.js tab is open, the next launch does not restore any tabs, and the following error message is logged:
```
ERROR: Failed to load session default: PyQt6.QtCore.QUrl('') is not valid
```
Manually inspecting `~/.local/share/qutebrowser/sessions/default.yml` (or `$basedir/data/sessions/default.yml`) reveals that the tab corresponding to the PDF.js viewer has the following contents:
```yaml
    - active: true
      last_visited: '1969-12-31T19:00:00'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: ''
      url: ''
      zoom: 1.0
```

**How to reproduce**

1. Run `qutebrowser --basedir /path/to/basedir`, where `/path/to/basedir` does not exist.
2. Navigate to a PDF file — e.g., <https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf> — and select "Open download via PDF.js".
3. Once the file has loaded in PDF.js, quit qutebrowser with `:wq`.
4. Run `qutebrowser --basedir /path/to/basedir` once again; the tab whose URL contains the source URL of the PDF should be restored, select "Open download via PDF.js" again in the dialog.
5. Once the file has loaded in PDF.js, quit qutebrowser with `:wq` again.
6. Run `qutebrowser --basedir /path/to/basedir` a third time. The session will not be successfully restored, and inspecting the session file will reveal that there is an entry with an empty title, empty URL, and last visited timestamp shortly before the Unix epoch.


Discussion:
I'm not sure if my issue was caused by pdf.js, but i had an issue today where qutebrowser started, failed to restore the session, console output revealed the `ERROR: Failed to load session default: PyQt6.QtCore.QUrl('') is not valid` error, and indeed, I had an entry in my autosave session like this:

```
  - active: true
    history:
    - active: true
      last_visited: '1970-01-01T02:00:00'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: ''
      url: ''
      zoom: 1.0
```

i took a copy of the session file, removed that entry, and was able to load it.
I've started seeing this recently as well, and have no idea where it was coming from. Thanks for the detailed reproduction steps!
Same here today, no pdf.js open.

```
   - active: true
     history:
     - active: true
       last_visited: '1969-12-31T21:00:00'
       pinned: false
       scroll-pos:
         x: 0
         y: 0
       title: ''
       url: ''
       zoom: 1.0
```
I'm having this issue as well. No pdf.js open. I have the following entry 13 times in my default.yml.
```
 - active: true
    history:
    - active: true
      last_visited: '1969-12-31T16:00:00'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: ''
      url: ''
      zoom: 1.5
```
This behavior started when I used paru to update qutebrowser-qt6-git from the AUR today. While it was updating I opened qutebrowser then thought maybe opening the program during an update could be bad so I immediately closed it. Not sure if that has anything to do with it.

edit: Deleting default.yml has fixed the issue for me. Tabs are now properly saved and loaded and there are no more strange entries in my default.yml from 1969.
can we at least handle this more gracefully, such that issues with 1 tab, don't affect other tabs?
I did a small amount of debugging (having updated to the latest commit on `main`), and it seems that the issue arises when restoring from a session that contains a PDF.js viewer tab (step 4 of my reproduction steps).

After closing qutebrowser for the first time (step 3), the contents of `$basedir/data/sessions/default.yml` are rather benign:

<details>
<summary>Session before corruption</summary>

```yaml
windows:
- active: true
  geometry: !!binary |
    AdnQywADAAAAAAACAAAAKAAABSoAAAStAAAAMgAAADIAAANRAAACiQAAAAACAAAAB4AAAAACAAAA
    KAAABSoAAASt
  tabs:
  - history:
    - active: true
      last_visited: '2023-08-04T22:32:01'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: DuckDuckGo — Privacy, simplified.
      url: https://start.duckduckgo.com/
      zoom: 1.0
  - active: true
    history:
    - active: true
      last_visited: '2023-08-04T22:32:07'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: tmpggpk3khv_sample.pdf
      url: qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf
      zoom: 1.0
```

</details>

However, just after qutebrowser is opened and has initialized (before interacting with the browser at all), `$basedir/data/sessions/_autosave.yml` looks like this:

<details>
<summary>Corrupted session</summary>

```yaml
windows:
- active: true
  geometry: !!binary |
    AdnQywADAAAAAAACAAAAKAAABSoAAAStAAAAMgAAADIAAANRAAACiQAAAAACAAAAB4AAAAACAAAA
    KAAABSoAAASt
  tabs:
  - history:
    - active: true
      last_visited: '1969-12-31T19:00:00'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: ''
      url: ''
      zoom: 1.0
  - active: true
    history:
    - active: true
      last_visited: '1969-12-31T19:00:00'
      pinned: false
      scroll-pos:
        x: 0
        y: 0
      title: ''
      url: ''
      zoom: 1.0
```

</details>

At this stage, selecting "Open download via PDF.js" causes the second of these to be replaced by the PDF.js viewer tab, while the first one keeps the empty `title`/`url` and the `last_visited` = Unix epoch in localtime.

Here is an excerpt from the debug log (`qutebrowser --debug --basedir /path/to/basedir`):

<details>
<summary>Debug log (excerpt)</summary>

```
22:32:36 DEBUG    init       mainwindow:__init__:245 Initializing modes...
22:32:36 DEBUG    init       mainwindow:_load_geometry:463 Loading mainwindow from b'\x01\xd9\xd0\xcb\x00\x03\x00\x00\x00\x00\x00\x02\x00\x00\x00(\x00\x00\x05*\x00\x00\x04\xad\x00\x00\x002\x00\x00\x002\x00\x00\x03Q\x00\x00\x02\x89\x00\x00\x00\x00\x02\x00\x00\x00\x07\x80\x00\x00\x00\x02\x00\x00\x00(\x00\x00\x05*\x00\x00\x04\xad'
22:32:36 DEBUG    init       mainwindow:_init_geometry:290 Initial main window geometry: PyQt6.QtCore.QRect(50, 50, 800, 600)
22:32:36 DEBUG    webview    tabbedbrowser:tabopen:630 Creating new tab with URL None, background False, related True, idx None
22:32:36 DEBUG    webview    tabbedbrowser:_get_new_tab_idx:717 tabs.new_position next -> opening new tab at -1, next left: 0 / right: 0
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:888 Current tab changed, focusing <qutebrowser.browser.webengine.webenginetab.WebEngineTab tab_id=0 url=''>
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:895 Mode before tab change: normal (mode_on_change = normal)
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.hint (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.caret (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.insert (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.passthrough (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:906 Mode after tab change: normal (mode_on_change = normal)
22:32:36 DEBUG    misc       eventfilter:eventFilter:38 <qutebrowser.browser.webengine.webview.WebEngineView object at 0x7f249bf1a440> got new child <PyQt6.QtWidgets.QWidget object at 0x7f249bf1aef0>, installing filter
22:32:36 DEBUG    statusbar  searchmatch:set_match:30 Clearing search match text.
22:32:36 DEBUG    webview    tabbedbrowser:_on_title_changed:793 Changing title for idx 0 to 'DuckDuckGo — Privacy, simplified.'
22:32:36 DEBUG    webview    browsertab:_on_before_load_started:1127 Going to start loading: https://start.duckduckgo.com/
22:32:36 DEBUG    webview    tabbedbrowser:_on_title_changed:793 Changing title for idx 0 to 'https://start.duckduckgo.com/'
22:32:36 DEBUG    webview    tabbedbrowser:tabopen:630 Creating new tab with URL None, background False, related True, idx None
22:32:36 DEBUG    webview    tabbedbrowser:_get_new_tab_idx:717 tabs.new_position next -> opening new tab at 1, next left: 0 / right: 2
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:888 Current tab changed, focusing <qutebrowser.browser.webengine.webenginetab.WebEngineTab tab_id=1 url=''>
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:895 Mode before tab change: normal (mode_on_change = normal)
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.hint (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.caret (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.insert (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.passthrough (reason tab changed) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      tabbedbrowser:_on_current_changed:906 Mode after tab change: normal (mode_on_change = normal)
22:32:36 DEBUG    misc       eventfilter:eventFilter:38 <qutebrowser.browser.webengine.webview.WebEngineView object at 0x7f249bf1b010> got new child <PyQt6.QtWidgets.QWidget object at 0x7f249bf1ba30>, installing filter
22:32:36 DEBUG    statusbar  searchmatch:set_match:30 Clearing search match text.
22:32:36 DEBUG    webview    tabbedbrowser:_on_title_changed:793 Changing title for idx 1 to 'tmpggpk3khv_sample.pdf'
22:32:36 DEBUG    webview    browsertab:_on_before_load_started:1127 Going to start loading: qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf
22:32:36 DEBUG    webview    tabbedbrowser:_on_title_changed:793 Changing title for idx 1 to 'qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf'
22:32:36 DEBUG    init       app:_process_args:211 Init finished after 0.808282s
22:32:36 DEBUG    init       app:init:161 Init done!
22:32:36 DEBUG    misc       mainwindow:_update_overlay_geometry:348 new geometry for <qutebrowser.mainwindow.messageview.MessageView object at 0x7f249bf18e50>: PyQt6.QtCore.QRect(0, 583, 801, 1)
22:32:36 DEBUG    misc       app:on_focus_object_changed:575 Focus object changed: <PyQt6.QtWidgets.QWidget object at 0x7f24a8f34c10>
22:32:36 DEBUG    misc       app:on_focus_object_changed:575 Focus object changed: <PyQt6.QtCore.QObject object at 0x7f24a8f34c10>
22:32:36 DEBUG    misc       app:on_focus_object_changed:575 Focus object changed: <PyQt6.QtWidgets.QWidget object at 0x7f24a8f34c10>
22:32:36 DEBUG    misc       mainwindow:_update_overlay_geometry:348 new geometry for <qutebrowser.mainwindow.messageview.MessageView object at 0x7f249bf18e50>: PyQt6.QtCore.QRect(0, 1141, 1322, 1)
22:32:36 DEBUG    webview    browsertab:_on_navigation_request:1151 navigation request: url https://start.duckduckgo.com/, type Type.typed, is_main_frame True
22:32:36 DEBUG    webview    webenginetab:_on_find_finished:212 Active search match: 0/0
22:32:36 DEBUG    signals    signalfilter:_filter_signals:78 ignoring: cur_search_match_changed(SearchMatch(current=0, total=0)) (tab 0)
22:32:36 DEBUG    webview    browsertab:_set_load_status:1093 load status for <qutebrowser.browser.webengine.webenginetab.WebEngineTab tab_id=0 url=''>: LoadStatus.loading
22:32:36 DEBUG    signals    signalfilter:_filter_signals:78 ignoring: cur_load_status_changed(<LoadStatus.loading: 6>) (tab 0)
22:32:36 DEBUG    signals    signalfilter:_filter_signals:78 ignoring: cur_load_started() (tab 0)
22:32:36 DEBUG    webview    browsertab:_on_navigation_request:1151 navigation request: url qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.ed…, type Type.typed, is_main_frame True
22:32:36 DEBUG    webview    webenginetab:_on_find_finished:212 Active search match: 0/0
22:32:36 DEBUG    signals    signalfilter:_filter_signals:72 emitting: cur_search_match_changed(SearchMatch(current=0, total=0)) (tab 1)
22:32:36 DEBUG    statusbar  searchmatch:set_match:30 Clearing search match text.
22:32:36 DEBUG    webview    browsertab:_set_load_status:1093 load status for <qutebrowser.browser.webengine.webenginetab.WebEngineTab tab_id=1 url=''>: LoadStatus.loading
22:32:36 DEBUG    signals    signalfilter:_filter_signals:72 emitting: cur_load_status_changed(<LoadStatus.loading: 6>) (tab 1)
22:32:36 DEBUG    signals    signalfilter:_filter_signals:72 emitting: cur_load_started() (tab 1)
22:32:36 DEBUG    modes      modeman:leave:421 Ignoring leave request for KeyMode.insert (reason load started) as we're in mode KeyMode.normal
22:32:36 DEBUG    modes      tabbedbrowser:_leave_modes_on_load:773 Ignoring leave_on_load request due to setting.
22:32:36 DEBUG    network    webenginequtescheme:requestStarted:82 Got request for qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf
22:32:36 DEBUG    misc       qutescheme:data_for_url:127 url: qute://pdfjs/web/viewer.html?filename=tmpggpk3khv_sample.pdf&file=&source=https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf, path: /web/viewer.html, host pdfjs
22:32:36 DEBUG    webview    browsertab:_on_navigation_request:1151 navigation request: url https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf, type Type.redirect, is_main_frame True
22:32:36 DEBUG    misc       webenginetab:_on_history_trigger:1432 Ignoring invalid URL being added to history
22:32:36 DEBUG    sessions   sessions:save:323 Saving session _autosave to /tmp/tmp.NSswUBnZv9/data/sessions/_autosave.yml...
```

</details>

I'm guessing that the problem is related to one or more of these lines:
```
22:32:36 DEBUG    webview    tabbedbrowser:tabopen:630 Creating new tab with URL None, background False, related True, idx None
22:32:36 DEBUG    webview    tabbedbrowser:tabopen:630 Creating new tab with URL None, background False, related True, idx None
22:32:36 DEBUG    webview    browsertab:_set_load_status:1093 load status for <qutebrowser.browser.webengine.webenginetab.WebEngineTab tab_id=1 url=''>: LoadStatus.loading
22:32:36 DEBUG    misc       webenginetab:_on_history_trigger:1432 Ignoring invalid URL being added to history
```
I don't think this is caused by the PDF.js viewer tab - it's caused by the "dead" tab left behind when a download triggers opening PDF.js. That also explains why it's also reproducible without PDF.js, this is just the most common case of such tabs.

With

```diff
diff --git i/qutebrowser/misc/sessions.py w/qutebrowser/misc/sessions.py
index da447c747..a3034bf41 100644
--- i/qutebrowser/misc/sessions.py
+++ w/qutebrowser/misc/sessions.py
@@ -239,6 +239,8 @@ def _save_tab(self, tab, active, with_history=True):
         for idx, item in enumerate(history):
             qtutils.ensure_valid(item)
             item_data = self._save_tab_item(tab, idx, item)
+            if not item.url().isValid():
+                breakpoint()
             if item.url().scheme() == 'qute' and item.url().host() == 'back':
                 # don't add qute://back to the session file
                 if item_data.get('active', False) and data['history']:
```

and doing (just the first PDF I happened to find):

```
qutebrowser --temp-basedir -s content.pdfjs true https://www.americanexpress.com/content/dam/amex/us/staticassets/pdf/GCO/Test_PDF.pdf   
```

indeed we see a "half-valid" history item:

```
(Pdb) p item.isValid()
True
(Pdb) p item.lastVisited()
PyQt6.QtCore.QDateTime(1970, 1, 1, 1, 0)
(Pdb) p item.url().isValid()
False
```

and this is only reproducible with Qt 6.5 (which [also explains](https://wiki.qt.io/Qt_6.5_Release) why this only showed up in May). I suppose the behavior of [`QWebEngineHistoryItem::isValid()`](https://github.com/qt/qtwebengine/blob/v6.5.2/src/core/api/qwebenginehistory.cpp#L69-L75) changed somehow between Qt 6.4 and 6.5 (or, looking at the timeline, maybe even 6.5.0 -> .1 or .1 -> .2?).

This smells like a Qt bug, but it should be easy to work around at least.

same here, can't restore my sessions since a recent update of qutebrowser (under Arch Linux), no PDF.js enabled 

```
qutebrowser v2.5.4
Git commit: 6aa53a3c9 on main (2023-07-30 11:18:05 +1200)
Backend: QtWebEngine 6.5.2, based on Chromium 108.0.5359.220 (from api)
Qt: 6.5.2

CPython: 3.11.3
PyQt: 6.5.2

Qt wrapper: PyQt6 (via QUTE_QT_WRAPPER)

colorama: 0.4.6
jinja2: 3.1.2
pygments: 2.15.1
yaml: 6.0.1
adblock: 0.6.0
objc: no
PyQt6.QtWebEngineCore: 6.5.0
PyQt6.sip: 6.7.10
pdf.js: 3.7.107 (/usr/share/pdf.js/build/pdf.js)
sqlite: 3.42.0
QtNetwork SSL: OpenSSL 3.1.2 1 Aug 2023
```

Resolving patch:
diff --git a/qutebrowser/misc/sessions.py b/qutebrowser/misc/sessions.py
--- a/qutebrowser/misc/sessions.py
+++ b/qutebrowser/misc/sessions.py
@@ -239,6 +239,13 @@ def _save_tab(self, tab, active, with_history=True):
         for idx, item in enumerate(history):
             qtutils.ensure_valid(item)
             item_data = self._save_tab_item(tab, idx, item)
+
+            if not item.url().isValid():
+                # WORKAROUND Qt 6.5 regression
+                # https://github.com/qutebrowser/qutebrowser/issues/7696
+                log.sessions.debug(f"Skipping invalid history item: {item}")
+                continue
+
             if item.url().scheme() == 'qute' and item.url().host() == 'back':
                 # don't add qute://back to the session file
                 if item_data.get('active', False) and data['history']:


Test patch:
diff --git a/tests/end2end/features/sessions.feature b/tests/end2end/features/sessions.feature
--- a/tests/end2end/features/sessions.feature
+++ b/tests/end2end/features/sessions.feature
@@ -328,6 +328,26 @@ Feature: Saving and loading sessions
                 - active: true
                   url: http://localhost:*/data/numbers/5.txt
 
+  # https://github.com/qutebrowser/qutebrowser/issues/7696
+  @qtwebkit_skip
+  Scenario: Saving session with an empty download tab
+    When I open data/downloads/downloads.html
+    And I run :click-element --force-event -t tab id download
+    And I wait for "Asking question <qutebrowser.utils.usertypes.Question default='*' mode=<PromptMode.download: 5> *" in the log
+    And I run :mode-leave
+    And I run :session-save current
+    And I run :session-load --clear current
+    And I wait until data/downloads/downloads.html is loaded
+    Then the session should look like:
+      windows:
+        - tabs:
+          - history:
+            - active: true
+              title: Simple downloads
+              url: http://localhost:*/data/downloads/downloads.html
+          - active: true
+            history: []
+
   # :session-delete
 
   Scenario: Deleting a directory