CVE-2023-28837 (published 2023-04-03T17:15Z)

Wagtail is an open source content management system built on Django. Prior to versions 4.1.4 and 4.2.2, a memory exhaustion bug exists in Wagtail's handling of uploaded images and documents. For both images and documents, files are loaded into memory during upload for additional processing. A user with access to upload images or documents through the Wagtail admin interface could upload a file so large that it results in a crash of denial of service.\n\nThe vulnerability is not exploitable by an ordinary site visitor without access to the Wagtail admin. It can only be exploited by admin users with permission to upload images or documents.\n\nImage uploads are restricted to 10MB by default, however this validation only happens on the frontend and on the backend after the vulnerable code. \n\nPatched versions have been released as Wagtail 4.1.4 and Wagtail 4.2.2). Site owners who are unable to upgrade to the new versions are encouraged to add extra protections outside of Wagtail to limit the size of uploaded files.

CVSS v3 base score: 4.9, MEDIUM (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:N/A:H)

Weakness information:
cwe_id: CWE-400
cwe_name: Uncontrolled Resource Consumption
description: The product does not properly control the allocation and maintenance of a limited resource, thereby enabling an actor to influence the amount of resources consumed, eventually leading to the exhaustion of available resources.
extended_description: 
            Limited resources include memory, file system storage, database connection pool entries, and CPU. If an attacker can trigger the allocation of these limited resources, but the number or size of the resources is not controlled, then the attacker could cause a denial of service that consumes all available resources. This would prevent valid users from accessing the product, and it could potentially have an impact on the surrounding environment. For example, a memory exhaustion attack against an application could slow down the application as well as its host operating system.
            There are at least three distinct scenarios which can commonly lead to resource exhaustion:
               
                  Lack of throttling for the number of allocated resources
                  Losing all references to a resource before reaching the shutdown stage
                  Not closing/returning a resource after processing
               
            Resource exhaustion problems are often result due to an incorrect implementation of the following situations:
               
                  Error conditions and other exceptional circumstances.
                  Confusion over which part of the program is responsible for releasing the resource.
               
         
         
cwe_id: CWE-770
cwe_name: Allocation of Resources Without Limits or Throttling
description: The product allocates a reusable resource or group of resources on behalf of an actor without imposing any restrictions on the size or number of resources that can be allocated, in violation of the intended security policy for that actor.
extended_description: Code frequently has to work with limited resources, so programmers must be careful to ensure that resources are not consumed too quickly, or too easily.  Without use of quotas, resource limits, or other protection mechanisms, it can be easy for an attacker to consume many resources by rapidly making many requests, or causing larger resources to be used than is needed. When too many resources are allocated, or if a single resource is too large, then it can prevent the code from working correctly, possibly leading to a denial of service.
         

Fix information:
repo_url: https://github.com/wagtail/wagtail
commit_details: repo_url: https://github.com/wagtail/wagtail
author: Jake Howard
author_date: 2023-03-10 12:00:46+00:00
msg: Don't load temporary uploaded files into memory

These can be quite large. Instead pass them straight to Willow to have it read just as much as it needs to
repository: repo_url: https://github.com/wagtail/wagtail
repo_name: wagtail/wagtail
description: A Django content management system focused on flexibility and user experience
repo_language: Python
owner: wagtail
file_changes: filename: fields.py
change_type: ModificationType.MODIFY
diff: @@ -146,11 +146,11 @@ class WagtailImageField(ImageField):
         if f is None:
             return None
 
-        # We need to get a file object for Pillow. When we get a path, we need to open
-        # the file first. And we have to read the data into memory to pass to Willow.
+        # Get the file content ready for Willow
         if hasattr(data, "temporary_file_path"):
-            with open(data.temporary_file_path(), "rb") as fh:
-                file = BytesIO(fh.read())
+            # Django's `TemporaryUploadedFile` is enough of a file to satisfy Willow
+            # Willow doesn't support opening images by path https://github.com/wagtail/Willow/issues/108
+            file = data
         else:
             if hasattr(data, "read"):
                 file = BytesIO(data.read())

programming_language: Python
filename: test_admin_views.py
change_type: ModificationType.MODIFY
diff: @@ -3,7 +3,7 @@ import json
 import urllib
 
 from django.contrib.auth.models import Group, Permission
-from django.core.files.uploadedfile import SimpleUploadedFile
+from django.core.files.uploadedfile import SimpleUploadedFile, TemporaryUploadedFile
 from django.template.defaultfilters import filesizeformat
 from django.template.loader import render_to_string
 from django.test import RequestFactory, TestCase, override_settings
@@ -567,6 +567,44 @@ class TestImageAddView(WagtailTestUtils, TestCase):
         images = Image.objects.filter(title="Test image")
         self.assertEqual(images.count(), 1)
 
+    def test_add_temporary_uploaded_file(self):
+        """
+        Test that uploading large files (spooled to the filesystem) work as expected
+        """
+        test_image_file = get_test_image_file()
+        uploaded_file = TemporaryUploadedFile(
+            "test.png", "image/png", test_image_file.size, "utf-8"
+        )
+        uploaded_file.write(test_image_file.file.getvalue())
+        uploaded_file.seek(0)
+
+        response = self.post(
+            {
+                "title": "Test image",
+                "file": uploaded_file,
+            }
+        )
+
+        # Should redirect back to index
+        self.assertRedirects(response, reverse("wagtailimages:index"))
+
+        # Check that the image was created
+        images = Image.objects.filter(title="Test image")
+        self.assertEqual(images.count(), 1)
+
+        # Test that size was populated correctly
+        image = images.first()
+        self.assertEqual(image.width, 640)
+        self.assertEqual(image.height, 480)
+
+        # Test that the file_size/hash fields were set
+        self.assertTrue(image.file_size)
+        self.assertTrue(image.file_hash)
+
+        # Test that it was placed in the root collection
+        root_collection = Collection.get_first_root_node()
+        self.assertEqual(image.collection, root_collection)
+
     @override_settings(
         DEFAULT_FILE_STORAGE="wagtail.test.dummy_external_storage.DummyExternalStorage"
     )

programming_language: Python
repo_url: https://github.com/wagtail/wagtail
commit_details: repo_url: https://github.com/wagtail/wagtail
author: Jake Howard
author_date: 2023-03-10 11:16:56+00:00
msg: Don't load images / documents into memory when calculating their hash

This can be done in chunks, which prevents crashes when uploading large files.
repository: repo_url: https://github.com/wagtail/wagtail
repo_name: wagtail/wagtail
description: A Django content management system focused on flexibility and user experience
repo_language: Python
owner: wagtail
file_changes: filename: models.py
change_type: ModificationType.MODIFY
diff: @@ -1,4 +1,3 @@
-import hashlib
 import os.path
 import urllib
 from contextlib import contextmanager
@@ -15,6 +14,7 @@ from taggit.managers import TaggableManager
 from wagtail.models import CollectionMember, ReferenceIndex
 from wagtail.search import index
 from wagtail.search.queryset import SearchableQuerySetMixin
+from wagtail.utils.file import hash_filelike
 
 
 class DocumentQuerySet(SearchableQuerySetMixin, models.QuerySet):
@@ -122,14 +122,13 @@ class AbstractDocument(CollectionMember, index.Indexed, models.Model):
 
         return self.file_size
 
-    def _set_file_hash(self, file_contents):
-        self.file_hash = hashlib.sha1(file_contents).hexdigest()
+    def _set_file_hash(self):
+        with self.open_file() as f:
+            self.file_hash = hash_filelike(f)
 
     def get_file_hash(self):
         if self.file_hash == "":
-            with self.open_file() as f:
-                self._set_file_hash(f.read())
-
+            self._set_file_hash()
             self.save(update_fields=["file_hash"])
 
         return self.file_hash
@@ -141,7 +140,7 @@ class AbstractDocument(CollectionMember, index.Indexed, models.Model):
         self.file_size = self.file.size
 
         # Set new document file hash
-        self._set_file_hash(self.file.read())
+        self._set_file_hash()
         self.file.seek(0)
 
     def __str__(self):

programming_language: Python
filename: test_models.py
change_type: ModificationType.MODIFY
diff: @@ -126,6 +126,19 @@ class TestDocumentFilenameProperties(TestCase):
             "application/octet-stream", self.extensionless_document.content_type
         )
 
+    def test_file_hash(self):
+        self.assertEqual(
+            self.document.get_file_hash(), "7d8c4778b182e4f3bd442408c64a6e22a4b0ed85"
+        )
+        self.assertEqual(
+            self.pdf_document.get_file_hash(),
+            "7d8c4778b182e4f3bd442408c64a6e22a4b0ed85",
+        )
+        self.assertEqual(
+            self.extensionless_document.get_file_hash(),
+            "7d8c4778b182e4f3bd442408c64a6e22a4b0ed85",
+        )
+
     def test_content_disposition(self):
         self.assertEqual(
             """attachment; filename=example.doc; filename*=UTF-8''example.doc""",

programming_language: Python
filename: models.py
change_type: ModificationType.MODIFY
diff: @@ -38,6 +38,7 @@ from wagtail.images.rect import Rect
 from wagtail.models import CollectionMember, ReferenceIndex
 from wagtail.search import index
 from wagtail.search.queryset import SearchableQuerySetMixin
+from wagtail.utils.file import hash_filelike
 
 logger = logging.getLogger("wagtail.images")
 
@@ -267,14 +268,13 @@ class AbstractImage(ImageFileMixin, CollectionMember, index.Indexed, models.Mode
 
     objects = ImageQuerySet.as_manager()
 
-    def _set_file_hash(self, file_contents):
-        self.file_hash = hashlib.sha1(file_contents).hexdigest()
+    def _set_file_hash(self):
+        with self.open_file() as f:
+            self.file_hash = hash_filelike(f)
 
     def get_file_hash(self):
         if self.file_hash == "":
-            with self.open_file() as f:
-                self._set_file_hash(f.read())
-
+            self._set_file_hash()
             self.save(update_fields=["file_hash"])
 
         return self.file_hash
@@ -286,7 +286,7 @@ class AbstractImage(ImageFileMixin, CollectionMember, index.Indexed, models.Mode
         self.file_size = self.file.size
 
         # Set new image file hash
-        self._set_file_hash(self.file.read())
+        self._set_file_hash()
         self.file.seek(0)
 
     def get_upload_to(self, filename):

programming_language: Python
filename: test_models.py
change_type: ModificationType.MODIFY
diff: @@ -112,6 +112,11 @@ class TestImage(TestCase):
         with self.assertRaises(SourceImageIOError):
             self.image.get_file_size()
 
+    def test_file_hash(self):
+        self.assertEqual(
+            self.image.get_file_hash(), "4dd0211870e130b7e1690d2ec53c499a54a48fef"
+        )
+
 
 class TestImageQuerySet(TestCase):
     def test_search_method(self):

programming_language: Python
filename: test_utils.py
change_type: ModificationType.MODIFY
diff: @@ -1,8 +1,11 @@
 # -*- coding: utf-8 -*
 import pickle
+from io import BytesIO, StringIO
+from pathlib import Path
 
 from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import ImproperlyConfigured, SuspiciousOperation
+from django.core.files.uploadedfile import SimpleUploadedFile
 from django.test import SimpleTestCase, TestCase, override_settings
 from django.utils.text import slugify
 from django.utils.translation import _trans
@@ -23,6 +26,7 @@ from wagtail.coreutils import (
     string_to_ascii,
 )
 from wagtail.models import Page, Site
+from wagtail.utils.file import hash_filelike
 from wagtail.utils.utils import deep_update
 
 
@@ -508,3 +512,58 @@ class TestDeepUpdate(TestCase):
                 "starship": "enterprise",
             },
         )
+
+
+class HashFileLikeTestCase(SimpleTestCase):
+    test_file = Path.cwd() / "LICENSE"
+
+    def test_hashes_io(self):
+        self.assertEqual(
+            hash_filelike(BytesIO(b"test")), "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3"
+        )
+        self.assertEqual(
+            hash_filelike(StringIO("test")), "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3"
+        )
+
+    def test_hashes_file(self):
+        with self.test_file.open(mode="r") as f:
+            self.assertEqual(
+                hash_filelike(f), "9e58400061ca660ef7b5c94338a5205627c77eda"
+            )
+
+    def test_hashes_file_bytes(self):
+        with self.test_file.open(mode="rb") as f:
+            self.assertEqual(
+                hash_filelike(f), "9e58400061ca660ef7b5c94338a5205627c77eda"
+            )
+
+    def test_hashes_django_uploaded_file(self):
+        """
+        Check Django's file shims can be hashed as-is.
+        `SimpleUploadedFile` inherits the base `UploadedFile`, but is easiest to test against
+        """
+        self.assertEqual(
+            hash_filelike(SimpleUploadedFile("example.txt", b"test")),
+            "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3",
+        )
+
+    def test_hashes_large_file(self):
+        class FakeLargeFile:
+            """
+            A class that pretends to be a huge file (~1.3GB)
+            """
+
+            def __init__(self):
+                self.iterations = 20000
+
+            def read(self, bytes):
+                self.iterations -= 1
+                if not self.iterations:
+                    return b""
+
+                return b"A" * bytes
+
+        self.assertEqual(
+            hash_filelike(FakeLargeFile()),
+            "187cc1db32624dccace20d042f6d631f1a483020",
+        )

programming_language: Python
filename: file.py
change_type: ModificationType.ADD
diff: @@ -0,0 +1,35 @@
+from hashlib import sha1
+from io import UnsupportedOperation
+
+from django.utils.encoding import force_bytes
+
+HASH_READ_SIZE = 65536  # 64k
+
+
+def hash_filelike(filelike):
+    """
+    Compute the hash of a file-like object, without loading it all into memory.
+    """
+    file_pos = 0
+    if hasattr(filelike, "tell"):
+        file_pos = filelike.tell()
+
+    try:
+        # Reset file handler to the start of the file so we hash it all
+        filelike.seek(0)
+    except (AttributeError, UnsupportedOperation):
+        pass
+
+    hasher = sha1()
+    while True:
+        data = filelike.read(HASH_READ_SIZE)
+        if not data:
+            break
+        # Use `force_bytes` to account for files opened as text
+        hasher.update(force_bytes(data))
+
+    if hasattr(filelike, "seek"):
+        # Reset the file handler to where it was before
+        filelike.seek(file_pos)
+
+    return hasher.hexdigest()

programming_language: Python
repo_url: https://github.com/wagtail/wagtail
commit_details: repo_url: https://github.com/wagtail/wagtail
author: Matt Westcott
author_date: 2023-04-03 12:00:49+02:00
msg: Release note for CVE-2023-28837 in 4.2.2
repository: repo_url: https://github.com/wagtail/wagtail
repo_name: wagtail/wagtail
description: A Django content management system focused on flexibility and user experience
repo_language: Python
owner: wagtail
file_changes: filename: CHANGELOG.txt
change_type: ModificationType.MODIFY
diff: @@ -106,6 +106,7 @@ Changelog
 ~~~~~~~~~~~~~~~~~~
 
  * Fix: CVE-2023-28836 - Stored XSS attack via ModelAdmin views (Thibaud Colas)
+ * Fix: CVE-2023-28837 - Denial-of-service via memory exhaustion when uploading large files (Jake Howard)
  * Fix: Fix radio and checkbox elements shrinking when using a long label (Sage Abdullah)
  * Fix: Fix select elements expanding beyond their container when using a long option label (Sage Abdullah)
  * Fix: Fix timezone handling of `TemplateResponse`s for users with a custom timezone (Stefan Hammer, Sage Abdullah)

programming_language: Markdown
filename: 4.2.2.md
change_type: ModificationType.MODIFY
diff: @@ -17,6 +17,16 @@ This release addresses a stored cross-site scripting (XSS) vulnerability on Mode
 
 Many thanks to Thibaud Colas for reporting this issue. For further details, please see [the CVE-2023-28836 security advisory](https://github.com/wagtail/wagtail/security/advisories/GHSA-5286-f2rf-35c2).
 
+
+### CVE-2023-28837: Denial-of-service via memory exhaustion when uploading large files
+
+This release addresses a memory exhaustion bug in Wagtail's handling of uploaded images and documents. For both images and documents, files are loaded into memory during upload for additional processing. A user with access to upload images or documents through the Wagtail admin interface could upload a file so large that it results in a crash or denial of service.
+
+The vulnerability is not exploitable by an ordinary site visitor without access to the Wagtail admin. It can only be exploited by admin users with permission to upload images or documents.
+
+Many thanks to Jake Howard for reporting this issue. For further details, please see [the CVE-2023-28837 security advisory](https://github.com/wagtail/wagtail/security/advisories/GHSA-33pv-vcgh-jfg9).
+
+
 ### Bug fixes
 
 * Fix radio and checkbox elements shrinking when using a long label (Sage Abdullah)

programming_language: Markdown
repo_url: https://github.com/wagtail/wagtail
commit_details: repo_url: https://github.com/wagtail/wagtail
author: Matt Westcott
author_date: 2023-04-03 12:01:43+02:00
msg: Release note for CVE-2023-28837 in 4.1.4
repository: repo_url: https://github.com/wagtail/wagtail
repo_name: wagtail/wagtail
description: A Django content management system focused on flexibility and user experience
repo_language: Python
owner: wagtail
file_changes: filename: CHANGELOG.txt
change_type: ModificationType.MODIFY
diff: @@ -295,6 +295,7 @@ Changelog
 ~~~~~~~~~~~~~~~~~~
 
  * Fix: CVE-2023-28836 - Stored XSS attack via ModelAdmin views (Thibaud Colas)
+ * Fix: CVE-2023-28837 - Denial-of-service via memory exhaustion when uploading large files (Jake Howard)
  * Fix: Fix radio and checkbox elements shrinking when using a long label (Sage Abdullah)
  * Fix: Fix select elements expanding beyond their container when using a long option label (Sage Abdullah)
  * Fix: Fix timezone handling of `TemplateResponse`s for users with a custom timezone (Stefan Hammer, Sage Abdullah)

programming_language: Markdown
filename: 4.1.4.md
change_type: ModificationType.MODIFY
diff: @@ -17,6 +17,16 @@ This release addresses a stored cross-site scripting (XSS) vulnerability on Mode
 
 Many thanks to Thibaud Colas for reporting this issue. For further details, please see [the CVE-2023-28836 security advisory](https://github.com/wagtail/wagtail/security/advisories/GHSA-5286-f2rf-35c2).
 
+
+### CVE-2023-28837: Denial-of-service via memory exhaustion when uploading large files
+
+This release addresses a memory exhaustion bug in Wagtail's handling of uploaded images and documents. For both images and documents, files are loaded into memory during upload for additional processing. A user with access to upload images or documents through the Wagtail admin interface could upload a file so large that it results in a crash or denial of service.
+
+The vulnerability is not exploitable by an ordinary site visitor without access to the Wagtail admin. It can only be exploited by admin users with permission to upload images or documents.
+
+Many thanks to Jake Howard for reporting this issue. For further details, please see [the CVE-2023-28837 security advisory](https://github.com/wagtail/wagtail/security/advisories/GHSA-33pv-vcgh-jfg9).
+
+
 ### Bug fixes
 
 * Fix radio and checkbox elements shrinking when using a long label (Sage Abdullah)

programming_language: Markdown