Repository: wagtail/wagtail

Issue:
Broken image tags when image is changed with rendition caching enabled
### Issue Summary

With rendition caching enabled, changing an image results in broken image tags (old file paths remain cached).

### Steps to Reproduce

1. Start a new project with `wagtail start myproject`
2. [Enable rendition caching](https://docs.wagtail.io/en/v2.9/advanced_topics/performance.html?highlight=RENDITIONS#caching-image-renditions)
3. Edit home/models.py as follows
```python
from django.db import models

from wagtail.core.models import Page
from wagtail.images.edit_handlers import ImageChooserPanel


class HomePage(Page):
    image = models.ForeignKey(
        'wagtailimages.Image', on_delete=models.SET_NULL, related_name='+', null=True, blank=True
    )

    content_panels = Page.content_panels + [
        ImageChooserPanel('image')
    ]
```
4. Edit home/templates/home/home_page.html as follows
```html
{% extends "base.html" %}
{% load static wagtailimages_tags %}

{% block content %}
{% image page.image width-200 %}
{% endblock content %}
```
5. Open the page in the frontend, ensure the rendition is cached
6. Change the image in the Wagtail admin (Change image file)
7. Reload the page from step 5, a broken image tag is now rendered because an old image path is still cached

I would expect the cache to be cleared when the image is changed.

* I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: no


### Technical details

* Python version: 3.8.2
* Django version: 3.0.5
* Wagtail version: 2.9.0



Discussion:
Looking at the [code](https://github.com/wagtail/wagtail/blob/52cb7ab77b8e9c64b2426b04165dc031395c279c/wagtail/images/models.py#L286-L290) this issue happens because the cache key only includes the `image id`, `filter.spec` and some other cache key (`Filter.get_cache_key`) that is based off the operations on the filter?

In any case, if the image is changed in any way (file change, focal point change) then the cache is not busted. One way to solve this would be to include the timestamp when the image was last changed in the cache key. Unfortunately the Wagtail image model currently doesn't track an updated timestamp.
I looked into this some more. It seems that when the image (file) is changed, the renditions are destroyed. So instead of making the cache key more robust, an alternative could be to destroy the cached renditions using signals when they are removed. Now, this could still leave some other scenarios where the cache might go stale?

@tomdyson maybe you have some more insight into this?
@pascalw thank you for outlining the issue (and testing cached renditions!)
IMHO, the rendition cache should be cleared when the image changes.

Would you be happy to submit a PR to speed things up? 
Thank you very much for the report and investigations, @pascalw. I'm sorry for introducing this bug.

I think the cache key can be made robust enough by including the file hash, i.e.:

```python
# images/models.py, line 286
rendition_cache_key = "image-{}-{}-{}-{}".format(
    self.id,
    self.file_hash,
    cache_key,
    filter.spec
)
```

I've tested this successfully with file and focal point changes. 
@tomdyson no need to apologize for bugs! (https://blog.danslimmon.com/2019/08/02/stop-apologizing-for-bugs/)

I'm not sure though that only adding the `file_hash` to the cache key is a full solution. It indeed will work most of the time, but for example it doesn't in this case:

1. Upload an image
2. Make sure renditions are cached
3. Change the image (upload a new file)
4. Change the image back to the previous file
→ Cache is poisoned

Now granted, this is a bit of an edge case but it's annoying to recover from.
Also if in the future new attributes or functionality are added to Images or Renditions, new scenario's might be introduced where the cache is poisoned in a similar way.

What do you think about using signals to purge the cache when renditions are destroyed or updated? This seems pretty robust and cover all current and future cases, and we don't have to add a column to the image table.

Something like this:

```python
def post_delete_purge_cache(instance, **kwargs):
    transaction.on_commit(lambda: purge_rendition_cache(instance))

def register():
    Image = get_image_model()
    Rendition = Image.get_rendition_model()

    post_delete.connect(post_delete_purge_cache, sender=Rendition)
```
> I'm not sure though that only adding the `file_hash` to the cache key is a full solution. It indeed will work most of the time, but for example it doesn't in this case:
> 
> 1. Upload an image
> 2. Make sure renditions are cached
> 3. Change the image (upload a new file)
> 4. Change the image back to the previous file
>    → Cache is poisoned

@pascalw I'm not sure what you mean here. If you change the image back to the previous file, then `file_hash` will change back to the original value, and the rendition retrieved from the cache will be the one that was originally cached before the change of file - which is the correct one, isn't it?
Ah, now I see the problem - the rendition files are deleted when the image file is changed, so the cache returns a path to an image file that no longer exists (but _would_ have been the right image, if it still existed :-) ).
@gasman exactly that's it: the files are deleted but the key will still be in the cache and thus point to an image that no longer exists.

Resolving patch:
diff --git a/wagtail/images/models.py b/wagtail/images/models.py
--- a/wagtail/images/models.py
+++ b/wagtail/images/models.py
@@ -278,12 +278,12 @@ def get_rendition(self, filter):
             filter = Filter(spec=filter)
 
         cache_key = filter.get_cache_key(self)
-
+        Rendition = self.get_rendition_model()
 
         try:
             rendition_caching = True
             cache = caches['renditions']
-            rendition_cache_key = "image-{}-{}-{}".format(
+            rendition_cache_key = Rendition.construct_cache_key(
                 self.id,
                 cache_key,
                 filter.spec
@@ -294,8 +294,6 @@ def get_rendition(self, filter):
         except InvalidCacheBackendError:
             rendition_caching = False
 
-        Rendition = self.get_rendition_model()
-
         try:
             rendition = self.renditions.get(
                 filter_spec=filter.spec,
@@ -568,6 +566,24 @@ def check(cls, **kwargs):
 
         return errors
 
+    @staticmethod
+    def construct_cache_key(image_id, filter_cache_key, filter_spec):
+        return "image-{}-{}-{}".format(
+            image_id,
+            filter_cache_key,
+            filter_spec
+        )
+
+    def purge_from_cache(self):
+        try:
+            cache = caches['renditions']
+            cache.delete(self.construct_cache_key(
+                self.image_id, self.focal_point_key, self.filter_spec
+            ))
+        except InvalidCacheBackendError:
+            pass
+
+
     class Meta:
         abstract = True
 
diff --git a/wagtail/images/signal_handlers.py b/wagtail/images/signal_handlers.py
--- a/wagtail/images/signal_handlers.py
+++ b/wagtail/images/signal_handlers.py
@@ -10,6 +10,10 @@ def post_delete_file_cleanup(instance, **kwargs):
     transaction.on_commit(lambda: instance.file.delete(False))
 
 
+def post_delete_purge_rendition_cache(instance, **kwargs):
+    instance.purge_from_cache()
+
+
 def pre_save_image_feature_detection(instance, **kwargs):
     if getattr(settings, 'WAGTAILIMAGES_FEATURE_DETECTION_ENABLED', False):
         # Make sure the image doesn't already have a focal point
@@ -25,3 +29,4 @@ def register_signal_handlers():
     pre_save.connect(pre_save_image_feature_detection, sender=Image)
     post_delete.connect(post_delete_file_cleanup, sender=Image)
     post_delete.connect(post_delete_file_cleanup, sender=Rendition)
+    post_delete.connect(post_delete_purge_rendition_cache, sender=Rendition)


Test patch:
diff --git a/wagtail/images/tests/test_models.py b/wagtail/images/tests/test_models.py
--- a/wagtail/images/tests/test_models.py
+++ b/wagtail/images/tests/test_models.py
@@ -23,7 +23,7 @@ def setUp(self):
         # Create an image for running tests on
         self.image = Image.objects.create(
             title="Test image",
-            file=get_test_image_file(),
+            file=get_test_image_file(colour='white'),
         )
 
     def test_is_portrait(self):
@@ -272,11 +272,30 @@ def test_renditions_cache_backend(self):
         self.assertEqual(cache.get(rendition_cache_key), rendition)
 
         # Mark a rendition to check it comes from cache
-        rendition._from_cache = True
+        rendition._from_cache = 'original'
         cache.set(rendition_cache_key, rendition)
 
         # Check if get_rendition returns the rendition from cache
-        self.assertEqual(self.image.get_rendition('width-500')._from_cache, True)
+        with self.assertNumQueries(0):
+            new_rendition = self.image.get_rendition('width-500')
+        self.assertEqual(new_rendition._from_cache, 'original')
+
+        # changing the image file should invalidate the cache
+        self.image.file = get_test_image_file(colour='green')
+        self.image.save()
+        # deleting renditions would normally happen within the 'edit' view on file change -
+        # we're bypassing that here, so have to do it manually
+        self.image.renditions.all().delete()
+        new_rendition = self.image.get_rendition('width-500')
+        self.assertFalse(hasattr(new_rendition, '_from_cache'))
+
+        # changing it back should also generate a new rendition and not re-use
+        # the original one (because that file has now been deleted in the change)
+        self.image.file = get_test_image_file(colour='white')
+        self.image.save()
+        self.image.renditions.all().delete()
+        new_rendition = self.image.get_rendition('width-500')
+        self.assertFalse(hasattr(new_rendition, '_from_cache'))
 
 
 class TestUsageCount(TestCase):