Repository: wagtail/wagtail

Issue:
Getting a "cannot pickle" error in Wagtail 5.1 when when using a custom image model
<!--
Found a bug? Please fill out the sections below. 👍
-->

### Issue Summary

I recently upgraded four small sites to Wagtail 5.1.  None of these sites have ever used caching, and each site uses a custom image model.  On all but one site I started getting the error: ` cannot pickle '_io.BufferedReader' object` every place an `{% image ... %}` tag is used.  All of these sites have been working fine for several years.  (As an aside, I see that in `Image.get_rendition()`, a `try...except` was removed in 5.1 that would have probably been masking any caching-related errors that might have occurred before 5.1.)  The pickling error occurs both when caching is absent and when caching (Redis) is used.  The `'_io.BufferedReader' object` error occurs in Django's `locmem.py` caching when no cache is specified in the settings, and a different pickling error message occurs when Redis is used, but in both cases pickling is the problem.

To test from the command line, I inserted a print statement in `locmem.py` to print out the pickled value.  I then looped through the images calling `get_rendition()`.  All pickled values printed correctly.  To test a specific image from the command line, I uploaded a new image to the `logo_admin` field described below and did:
```
r=CustomRendition.objects.get(pk=63)
r.image.get_rendition(r.image_spec)
```
The pickled value was printed correctly.  However, when this exact same image/rendition was called from an `{% image ... %}` tag in the admin as described below, the pickling error happens.  The `CustomImage(AbstractImage)/CustomRendition(AbstractRendition)` definitions are the same in each of these projects and are set up [as described in the docs](https://docs.wagtail.org/en/latest/advanced_topics/images/custom_image_model.html#custom-image-models).

The easiest way to test is from the admin as described above and in **Steps to Reproduce** below.  I also tried:
- deleting all existing renditions
- uploading a new image to the `logo_admin` field shown below.  Again, when I comment out the `{% image settings.main.SiteSettings.logo_admin width-150 %}` line shown below, the admin comes up fine.  When I uncomment it, the error happens again.  The error happens with both `png` and `jpg` images.

### Main Question
How could the exact same rendition be pickled fine when called from the admin but encounter a pickling error when called from an `{% image ... %}` tag??

### Steps to Reproduce

Use a `CustomImage` model and set up the following `SiteSettings` model:

```
@register_setting
class SiteSettings(BaseSiteSetting):
    logo_admin = models.OneToOneField(settings.WAGTAILIMAGES_IMAGE_MODEL, null=True, blank=True, on_delete=models.SET_NULL, related_name='+')
```

Render the following from the `base.html` template in `wagtailadmin`:

```
{% extends "wagtailadmin/base.html" %}
{% load wagtailimages_tags %}

{% block branding_logo %}
    {% image settings.main.SiteSettings.logo_admin width-150 %}
{% endblock %}
```

-   I have NOT confirmed that this issue can be reproduced as described on a fresh Wagtail project.

### Technical details

-   Python version:  3.8
-   Django version:  4.2.5
-   Wagtail version:  5.1



Discussion:
I can confirm this is an issue using Bakerydemo with

```python
# local.py

CACHES = {
    "default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache"},
}

# base/models.py
class SiteSettings(BaseSiteSetting):
    title_suffix = models.CharField(
        verbose_name="Title suffix",
        max_length=255,
        help_text="The suffix for the title meta tag e.g. ' | The Wagtail Bakery'",
        default="The Wagtail Bakery",
    )

    logo_admin = models.OneToOneField(get_image_model(), null=True, blank=True, on_delete=models.SET_NULL, related_name='+')

    panels = [
        FieldPanel("title_suffix"),
        FieldPanel("logo_admin"),
    ]
```

```jinja2
{# wagtail/admin/templates/wagtailadmin/base.html #}

{% load wagtailimages_tags %}

{% block branding_logo %}
    {% image settings.base.SiteSettings.logo_admin width-150 %}
{% endblock %}
```

<details>
<summary>Traceback</summary>

```
Environment:


Request Method: GET
Request URL: http://wagtail.test:8000/admin/settings/base/sitesettings/2/

Django Version: 4.1.4
Python Version: 3.10.13


Template error:
In template wagtail/wagtail/admin/templates/wagtailadmin/base.html, error at line 6
   cannot pickle '_io.BufferedReader' object
   1 : {% extends "wagtailadmin/admin_base.html" %}
   2 : {% load wagtailadmin_tags wagtailcore_tags wagtailimages_tags i18n %}
   3 : 
   4 : {% block furniture %}
   5 :     <template data-wagtail-sidebar-branding-logo>{% block branding_logo %}
   6 :          {% image settings.base.SiteSettings.logo_admin width-150 %} 
   7 :     {% endblock %}</template>
   8 :     {% sidebar_props %}
   9 :     <aside id="wagtail-sidebar" class="sidebar-loading" data-wagtail-sidebar aria-label="{% trans 'Sidebar' %}"></aside>
   10 :     <main class="content-wrapper w-overflow-x-hidden" id="main">
   11 :         <div class="content">
   12 :             {# Always show messages div so it can be appended to by JS #}
   13 :             <div class="messages" role="status" data-controller="w-messages" data-action="w-messages:add@document->w-messages#add" data-w-messages-added-class="new" data-w-messages-show-class="appear">
   14 :                 <ul data-w-messages-target="container">
   15 :                     {% if messages %}
   16 :                         {% for message in messages %}


Traceback (most recent call last):
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 220, in _get_response
    response = response.render()
  File "wagtail/wagtail/admin/auth.py", line 184, in overridden_render
    return render()
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/response.py", line 114, in render
    self.content = self.rendered_content
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/response.py", line 92, in rendered_content
    return template.render(context, self._request)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/backends/django.py", line 62, in render
    return self.template.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
  File "wagtail/wagtail/images/templatetags/wagtailimages_tags.py", line 129, in render
    rendition = get_rendition_or_not_found(
  File "wagtail/wagtail/images/shortcuts.py", line 13, in get_rendition_or_not_found
    return image.get_rendition(specs)
  File "wagtail/wagtail/images/models.py", line 455, in get_rendition
    Rendition.cache_backend.set(cache_key, rendition)
  File "bakerydemo/.venv/lib/python3.10/site-packages/django/core/cache/backends/locmem.py", line 53, in set
    pickled = pickle.dumps(value, self.pickle_protocol)

Exception Type: TypeError at /admin/settings/base/sitesettings/2/
Exception Value: cannot pickle '_io.BufferedReader' object

```
</details>

Investigating. It seems it's not necessary to use a custom image model, or override the admin base template - it can be replicated by adding the line `{% image settings.base.SiteSettings.logo_admin width-150 %}` to bakerydemo/templates/base/home_page.html and viewing the homepage.

Curiously, preceding that tag with `{% image page.image width-150 %}` - with the homepage's hero image set to the same image as logo_admin in SiteSettings - successfully creates the rendition and allows further calls to `{% image settings.base.SiteSettings.logo_admin width-150 %}` to succeed, until the server is restarted - so apparently there must be something peculiar to the image instance retrieved through SiteSettings that isn't the case for images related to the page.
If `logo_admin` is changed from a `OneToOneField` to a `ForeignKey`, the issue does not occur. On the other hand, I'm not able to trigger the issue by changing `HomePage.image` to a `OneToOneField`...
If I move the SiteSettings object lookup to Python code:

```
class HomePage(Page):
    # ...
    def get_context(self, request, *args, **kwargs):
        context = super().get_context(request, *args, **kwargs)
        context['logo_admin'] = SiteSettings.for_request(request).logo_admin
        return context
```

with `{% image logo_admin width-150 %}` in the template, the issue does still occur - but not if I change the lookup to `SiteSettings.objects.first().logo_admin`.
The problem goes away if this line is removed from `BaseSiteSetting`:

https://github.com/wagtail/wagtail/blob/e6e29b55181a635f6adf7771bd1a7eb808f352a0/wagtail/contrib/settings/models.py#L129-L130

I think I see what's going on now:

* When the `SiteSettings` instance is retrieved through `for_request`, the request object is attached to it as the attribute `_request`. Request objects can't be pickled, so this makes the SiteSettings instance un-pickleable
* That shouldn't be a problem, because we want to pickle a rendition object, not a SiteSettings instance...
* However, the Rendition object has a ForeignKey reference back to the Image, and because the Image was retrieved through a OneToOneField, it has a reverse relation pointing back to the SiteSettings instance. (This isn't the case for a ForeignKey, because the reverse relation is one-to-many, i.e. there isn't a single instance to point back to.)
* Apparently, pickling a Django model instance will cause both of those relations to be followed, and the related objects to be included in the pickled representation. This seems surprising, and might be a Django bug.

In summary:

* The easy workaround here is to change the OneToOneField to a ForeignKey
* We should fix BaseSiteSetting to leave out the `_request` attribute when being pickled, since people might have a legitimate use for pickling those (even if we're only doing it accidentally here)
* We should also investigate whether it's intended behaviour for relations to be followed when pickling a Django model, and report it as a Django bug if not.

Resolving patch:
diff --git a/wagtail/contrib/settings/models.py b/wagtail/contrib/settings/models.py
--- a/wagtail/contrib/settings/models.py
+++ b/wagtail/contrib/settings/models.py
@@ -131,6 +131,12 @@ def for_request(cls, request):
         setattr(request, attr_name, site_settings)
         return site_settings
 
+    def __getstate__(self):
+        # Leave out _request from the pickled state
+        state = super().__getstate__()
+        state.pop("_request", None)
+        return state
+
     @classmethod
     def for_site(cls, site):
         """


Test patch:
diff --git a/wagtail/contrib/settings/tests/site_specific/base.py b/wagtail/contrib/settings/tests/site_specific/base.py
--- a/wagtail/contrib/settings/tests/site_specific/base.py
+++ b/wagtail/contrib/settings/tests/site_specific/base.py
@@ -22,4 +22,11 @@ def setUp(self):
     def get_request(self, site=None):
         if site is None:
             site = self.default_site
-        return get_dummy_request(site=site)
+        request = get_dummy_request(site=site)
+
+        # Requests in general can't be pickled, but dummy requests can. Add an
+        # arbitrary lambda function to the request to make it fail loudly if
+        # someone tries to pickle it.
+        request._fn = lambda: None
+
+        return request
diff --git a/wagtail/contrib/settings/tests/site_specific/test_model.py b/wagtail/contrib/settings/tests/site_specific/test_model.py
--- a/wagtail/contrib/settings/tests/site_specific/test_model.py
+++ b/wagtail/contrib/settings/tests/site_specific/test_model.py
@@ -44,6 +44,13 @@ def test_for_request_result_caching(self):
                     for i in range(4):
                         TestSiteSetting.for_request(request)
 
+    def test_pickle_after_lookup_via_for_request(self):
+        request = self.get_request()
+        settings = TestSiteSetting.for_request(request)
+        pickled = pickle.dumps(settings)
+        unpickled = pickle.loads(pickled)
+        self.assertEqual(unpickled.title, "Site title")
+
     def _create_importantpagessitesetting_object(self):
         site = self.default_site
         return ImportantPagesSiteSetting.objects.create(