Repository: qutebrowser/qutebrowser

Issue:
Make sure we delete download items correctly
There's a [Qt change](https://codereview.qt-project.org/#/c/213707/3//ALL,unified) expanding the `QWebEngineDownloadItem` docs, and the following was added:

> If accept() \e is called by a signal handler, then the WebEngineProfile will
> take ownership of the item. However, it is safe for the application to
> delete the item at any time. The QWebEngineProfile being a long-lived
> object, it is in fact \b recommended that the application delete any items
> it is no longer interested in.

I'm not sure whether we do this correctly, or if the Python garbage collection takes care of deleting our `DownloadItem` and/or the underlying `_qt_item`.

@jgkamat since you know how to debug this kind of stuff, if you have the time, can you finish and remove a few downloads, and then make sure there's no garbage lying around? :wink:


Discussion:
At a first look, this seems to be happening properly, adding this patch:

```
diff --git a/qutebrowser/browser/webengine/webenginedownloads.py b/qutebrowser/browser/webengine/webenginedownloads.py
index d467724d5..007e50671 100644
--- a/qutebrowser/browser/webengine/webenginedownloads.py
+++ b/qutebrowser/browser/webengine/webenginedownloads.py
@@ -39,8 +39,12 @@ class DownloadItem(downloads.AbstractDownloadItem):
         _qt_item: The wrapped item.
     """
 
+    def __del__(self):
+        print("deleting!")
+
     def __init__(self, qt_item, parent=None):
         super().__init__(parent)
+        print("creating!")
         self._qt_item = qt_item
         qt_item.downloadProgress.connect(self.stats.on_download_progress)
         qt_item.stateChanged.connect(self._on_state_changed)
```


Says that the download item is getting properly removed, at least on webengine. I want to makes sure the qtobject is deleted too, so I'm going to take a deeper look at this later (probably by creating many duplicates intentionally).
I took a little more of a deeper look with `objgraph` and I verified that the `DownloadItem` objects are definitely being deleted properly. There seems to be a bit of weirdness with the underlying `qt_object` though.


```
diff --git a/qutebrowser/browser/webengine/webenginedownloads.py b/qutebrowser/browser/webengine/webenginedownloads.py
index d467724d5..5b711370a 100644
--- a/qutebrowser/browser/webengine/webenginedownloads.py
+++ b/qutebrowser/browser/webengine/webenginedownloads.py
@@ -44,6 +44,9 @@ class DownloadItem(downloads.AbstractDownloadItem):
         self._qt_item = qt_item
         qt_item.downloadProgress.connect(self.stats.on_download_progress)
         qt_item.stateChanged.connect(self._on_state_changed)
+        self.destroyed.connect(lambda: print("destroying python obj"))
+        # download.destroyed.connect(qt_item.deleteLater)
+        self._qt_item.destroyed.connect(lambda: print("destroying qt_item"))
 
     def _is_page_download(self):
         """Check if this item is a page (i.e. mhtml) download."""
```

Shows no trace of the qt item being destroyed via the qt signal. However, I think this might just be some weirdness with superclasses and slots, because the `QWebEngineDownload` objects get removed after a garbage collection.


```
>>> # Download 3 items
>>> objgraph.by_type('QWebEngineDownloadItem')
[<PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd2181628b8>, <PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd242dc1b88>, <PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd1cb295798>]
>>> # clear items
>>> objgraph.by_type('QWebEngineDownloadItem')
[<PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd1cb295798>, <PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd242dc1b88>, <PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7fd2181628b8>]
>>> # force a GC
>>> gc.collect()
28
>>> objgraph.by_type('QWebEngineDownloadItem')
[]
```

One thing to note is that using `deleteLater` to delete the object manually results in the signal being called, while it getting removed in GC results in it never being called.


Weirdly, `deleteLater` triggers the destroyed signal, but does not actually remove the object (but it eventually gets cleaned up via GC).

```
>>> objgraph.by_type('QWebEngineDownloadItem')[0].deleteLater()
>>> objgraph.by_type('QWebEngineDownloadItem')
[<PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7ff300036d38>]
>>> objgraph.by_type('QWebEngineDownloadItem')[0].deleteLater()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
RuntimeError: wrapped C/C++ object of type QWebEngineDownloadItem has been deleted
>>> objgraph.by_type('QWebEngineDownloadItem')
[<PyQt5.QtWebEngineWidgets.QWebEngineDownloadItem object at 0x7ff300036d38>]
>>> gc.collect()
100
>>> objgraph.by_type('QWebEngineDownloadItem')
[]
>>> 
```

~So, there is a possibility that the object isn't actually destroyed (because the signal isn't fired), but I think that if that's the case, I think that's a bug in PyQt5 (since we can't be expected to `deleteLater` every object).~ (as explained on IRC) This is actually an issue, as PyQt5 exposes C++ objects as 'python wrappers'. 


Resolving patch:
diff --git a/qutebrowser/browser/webengine/webenginedownloads.py b/qutebrowser/browser/webengine/webenginedownloads.py
--- a/qutebrowser/browser/webengine/webenginedownloads.py
+++ b/qutebrowser/browser/webengine/webenginedownloads.py
@@ -45,6 +45,10 @@ def __init__(self, qt_item, parent=None):
         qt_item.downloadProgress.connect(self.stats.on_download_progress)
         qt_item.stateChanged.connect(self._on_state_changed)
 
+        # Ensure wrapped qt_item is deleted manually when the wrapper object
+        # is deleted. See https://github.com/qutebrowser/qutebrowser/issues/3373
+        self.destroyed.connect(self._qt_item.deleteLater)
+
     def _is_page_download(self):
         """Check if this item is a page (i.e. mhtml) download."""
         return (self._qt_item.savePageFormat() !=


Test patch: