CVE-2024-1892: Inefficient Regular Expression Complexity (CWE-1333)
value: A Regular Expression Denial of Service (ReDoS) vulnerability exists in the XMLFeedSpider class of the scrapy/scrapy project, specifically in the parsing of XML content. By crafting malicious XML content that exploits inefficient regular expression complexity used in the parsing process, an attacker can cause a denial-of-service (DoS) condition. This vulnerability allows for the system to hang and consume significant resources, potentially rendering services that utilize Scrapy for XML processing unresponsive.
CWE description: The product uses a regular expression with an inefficient, possibly exponential worst-case computational complexity that consumes excessive CPU cycles.
Published: 2024-02-28T00:15Z
Repository: https://github.com/scrapy/scrapy
Language: Other
Files changed: docs/news.rst
Commit date: 2024-02-14 17:18:26 +0100
Commit message: Merge branch '2.11-redos' into 2.11

Diff: diff --cc docs/news.rst
index ad931d6db,f346d1239..16e7e79a7
--- a/docs/news.rst
+++ b/docs/news.rst
@@@ -6,84 -6,31 +6,103 @@@ Release note
  .. _release-2.11.1:
  
  Scrapy 2.11.1 (unreleased)
  --------------------------
  
 -**Security bug fix:**
 +Highlights:
 +
 +-   Security bug fixes.
 +
 +-   Support for Twisted >= 23.8.0.
 +
 +-   Documentation improvements.
 +
 +Security bug fixes
 +~~~~~~~~~~~~~~~~~~
  
+ -   Addressed `ReDoS vulnerabilities`_:
+ 
+     -   ``scrapy.utils.iterators.xmliter`` is now deprecated in favor of
+         :func:`~scrapy.utils.iterators.xmliter_lxml`, which
+         :class:`~scrapy.spiders.XMLFeedSpider` now uses.
+ 
+         To minimize the impact of this change on existing code,
+         :func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
+         the node namespace with a prefix in the node name, and big files with
+         highly nested trees when using libxml2 2.7+.
+ 
+     -   Fixed regular expressions in the implementation of the
+         :func:`~scrapy.utils.response.open_in_browser` function.
+ 
+     Please, see the `cc65-xxvf-f7r9 security advisory`_ for more information.
+ 
+     .. _ReDoS vulnerabilities: https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
+     .. _cc65-xxvf-f7r9 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cc65-xxvf-f7r9
+ 
 +-   :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
 +    to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
 +    advisory`_ for more information.
 +
 +    .. _7j7m-v7m3-jqm7 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7
 +
 +-   Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, the
 +    deprecated ``scrapy.downloadermiddlewares.decompression`` module has been
 +    removed.
 +
 +-   The ``Authorization`` header is now dropped on redirects to a different
 +    domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
 +    information.
 +
 +    .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv
 +
 +Modified requirements
 +~~~~~~~~~~~~~~~~~~~~~
 +
 +-   The Twisted dependency is no longer restricted to < 23.8.0. (:issue:`6024`,
 +    :issue:`6064`, :issue:`6142`)
 +
 +Bug fixes
 +~~~~~~~~~
 +
 +-   The OS signal handling code was refactored to no longer use private Twisted
 +    functions. (:issue:`6024`, :issue:`6064`, :issue:`6112`)
 +
 +Documentation
 +~~~~~~~~~~~~~
 +
 +-   Improved documentation for :class:`~scrapy.crawler.Crawler` initialization
 +    changes made in the 2.11.0 release. (:issue:`6057`, :issue:`6147`)
 +
 +-   Extended documentation for :attr:`Request.meta <scrapy.http.Request.meta>`.
 +    (:issue:`5565`)
 +
 +-   Fixed the :reqmeta:`dont_merge_cookies` documentation. (:issue:`5936`,
 +    :issue:`6077`)
 +
 +-   Added a link to Zyte's export guides to the :ref:`feed exports
 +    <topics-feed-exports>` documentation. (:issue:`6183`)
 +
 +-   Added a missing note about backward-incompatible changes in
 +    :class:`~scrapy.exporters.PythonItemExporter` to the 2.11.0 release notes.
 +    (:issue:`6060`, :issue:`6081`)
 +
 +-   Added a missing note about removing the deprecated
 +    ``scrapy.utils.boto.is_botocore()`` function to the 2.8.0 release notes.
 +    (:issue:`6056`, :issue:`6061`)
 +
 +-   Other documentation improvements. (:issue:`6128`, :issue:`6144`,
 +    :issue:`6163`, :issue:`6190`, :issue:`6192`)
 +
 +Quality assurance
 +~~~~~~~~~~~~~~~~~
 +
 +-   Added Python 3.12 to the CI configuration, re-enabled tests that were
 +    disabled when the pre-release support was added. (:issue:`5985`,
 +    :issue:`6083`, :issue:`6098`)
 +
 +-   Fixed a test issue on PyPy 7.3.14. (:issue:`6204`, :issue:`6205`)
 +
  
  .. _release-2.11.0:
  
  Scrapy 2.11.0 (2023-09-18)
  --------------------------
@@@ -2954,27 -2901,23 +2973,37 @@@ affect subclasses
  .. _release-1.8.4:
  
  Scrapy 1.8.4 (unreleased)
  -------------------------
  
 -**Security bug fix:**
 +**Security bug fixes:**
  
+ -   Due to its `ReDoS vulnerabilities`_, ``scrapy.utils.iterators.xmliter`` is
+     now deprecated in favor of :func:`~scrapy.utils.iterators.xmliter_lxml`,
+     which :class:`~scrapy.spiders.XMLFeedSpider` now uses.
+ 
+     To minimize the impact of this change on existing code,
+     :func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
+     the node namespace as a prefix in the node name, and big files with highly
+     nested trees when using libxml2 2.7+.
+ 
+     Please, see the `cc65-xxvf-f7r9 security advisory`_ for more information.
+ 
 +-   :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
 +    to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
 +    advisory`_ for more information.
 +
 +-   Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, use of the
 +    ``scrapy.downloadermiddlewares.decompression`` module is discouraged and
 +    will trigger a warning.
 +
 +-   The ``Authorization`` header is now dropped on redirects to a different
 +    domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
 +    information.
 +
 +    .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv
 +
  
  .. _release-1.8.3:
  
  Scrapy 1.8.3 (2022-07-25)
  -------------------------

Fixed code:  -   Addressed `ReDoS vulnerabilities`_:
 
     -   ``scrapy.utils.iterators.xmliter`` is now deprecated in favor of
         :func:`~scrapy.utils.iterators.xmliter_lxml`, which
         :class:`~scrapy.spiders.XMLFeedSpider` now uses.
 
         To minimize the impact of this change on existing code,
         :func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
         the node namespace with a prefix in the node name, and big files with
         highly nested trees when using libxml2 2.7+.
 
     -   Fixed regular expressions in the implementation of the
         :func:`~scrapy.utils.response.open_in_browser` function.
 
     Please, see the `cc65-xxvf-f7r9 security advisory`_ for more information.
 
     .. _ReDoS vulnerabilities: https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
     .. _cc65-xxvf-f7r9 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cc65-xxvf-f7r9
 
 -   Due to its `ReDoS vulnerabilities`_, ``scrapy.utils.iterators.xmliter`` is
     now deprecated in favor of :func:`~scrapy.utils.iterators.xmliter_lxml`,
     which :class:`~scrapy.spiders.XMLFeedSpider` now uses.
 
     To minimize the impact of this change on existing code,
     :func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
     the node namespace as a prefix in the node name, and big files with highly
     nested trees when using libxml2 2.7+.
 
     Please, see the `cc65-xxvf-f7r9 security advisory`_ for more information.