CVE-2023-2804: Out-of-bounds Write (CWE-787)
value: A heap-based buffer overflow issue was discovered in libjpeg-turbo in h2v2_merged_upsample_internal() function of jdmrgext.c file. The vulnerability can only be exploited with 12-bit data precision for which the range of the sample data type exceeds the valid sample range, hence, an attacker could craft a 12-bit lossless JPEG image that contains out-of-range 12-bit samples. An application attempting to decompress such image using merged upsampling would lead to segmentation fault or buffer overflows, causing an application to crash.
CWE description: The product writes data past the end, or before the beginning, of the intended buffer.
CVSS v3 base score: 6.5
Published: 2023-05-25T22:15Z
Repository: https://github.com/libjpeg-turbo/libjpeg-turbo
Language: C
Files changed: ChangeLog.md, jdlossls.c
Commit date: 2023-04-04 20:37:54 -0500
Commit message: Lossless decomp: Range-limit 12-bit samples

12-bit is the only data precision for which the range of the sample data
type exceeds the valid sample range, so it is possible to craft a 12-bit
lossless JPEG image that contains out-of-range 12-bit samples.
Attempting to decompress such an image using color quantization or merged
upsampling (NOTE: libjpeg-turbo cannot generate YCbCr or subsampled
lossless JPEG images, but it can decompress them) caused segfaults or
buffer overruns when those algorithms attempted to use the out-of-range
sample values as array indices.  This commit modifies the lossless
decompressor so that it range-limits the output of the scaler when using
12-bit samples.

Fixes #670
Fixes #672
Fixes #673
Fixes #674
Fixes #675
Fixes #676
Fixes #677
Fixes #678
Fixes #679
Fixes #681
Fixes #683

Diff: diff --git a/ChangeLog.md b/ChangeLog.md
index 93b08061e..de8e45dfb 100644
--- a/ChangeLog.md
+++ b/ChangeLog.md
@@ -6,10 +6,17 @@
 1. The TurboJPEG API now supports 4:4:1 (transposed 4:1:1) chrominance
 subsampling, which allows losslessly transposed or rotated 4:1:1 JPEG images to
 be losslessly cropped, partially decompressed, or decompressed to planar YUV
 images.
 
+2. Fixed various segfaults and buffer overruns that occurred when attempting to
+decompress various specially-crafted malformed 12-bit-per-component lossless
+JPEG images.  These issues were caused by out-of-range sample values that were
+not range-limited before being used as array indices.  The issues were specific
+to 12-bit data precision, since that is the only data precision for which the
+range of the sample data type exceeds the valid sample range.
+
 
 2.1.91 (3.0 beta2)
 ==================
 
 ### Significant changes relative to 2.1.5.1:
diff --git a/jdlossls.c b/jdlossls.c
index 4d15e6bba..cfdca7e3a 100644
--- a/jdlossls.c
+++ b/jdlossls.c
@@ -4,11 +4,11 @@
  * This file was part of the Independent JPEG Group's software:
  * Copyright (C) 1998, Thomas G. Lane.
  * Lossless JPEG Modifications:
  * Copyright (C) 1999, Ken Murchison.
  * libjpeg-turbo Modifications:
- * Copyright (C) 2022, D. R. Commander.
+ * Copyright (C) 2022-2023, D. R. Commander.
  * For conditions of distribution and use, see the accompanying README.ijg
  * file.
  *
  * This file contains prediction, sample undifferencing, point transform, and
  * sample scaling routines for the lossless JPEG decompressor.
@@ -215,20 +215,32 @@ jpeg_undifference_first_row(j_decompress_ptr cinfo, int comp_index,
 METHODDEF(void)
 simple_upscale(j_decompress_ptr cinfo,
                JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width)
 {
   do {
+#if BITS_IN_JSAMPLE == 12
+    /* 12-bit is the only data precision for which the range of the sample data
+     * type exceeds the valid sample range.  Thus, we need to range-limit the
+     * samples, because other algorithms may try to use them as array indices.
+     */
+    *output_buf++ = (_JSAMPLE)((*diff_buf++ << cinfo->Al) & 0xFFF);
+#else
     *output_buf++ = (_JSAMPLE)(*diff_buf++ << cinfo->Al);
+#endif
   } while (--width);
 }
 
 METHODDEF(void)
 noscale(j_decompress_ptr cinfo,
         JDIFFROW diff_buf, _JSAMPROW output_buf, JDIMENSION width)
 {
   do {
+#if BITS_IN_JSAMPLE == 12
+    *output_buf++ = (_JSAMPLE)((*diff_buf++) & 0xFFF);
+#else
     *output_buf++ = (_JSAMPLE)(*diff_buf++);
+#endif
   } while (--width);
 }
 
 
 /*

Vulnerable code:  * Copyright (C) 2022, D. R. Commander.
Fixed code: 2. Fixed various segfaults and buffer overruns that occurred when attempting to
decompress various specially-crafted malformed 12-bit-per-component lossless
JPEG images.  These issues were caused by out-of-range sample values that were
not range-limited before being used as array indices.  The issues were specific
to 12-bit data precision, since that is the only data precision for which the
range of the sample data type exceeds the valid sample range.

 * Copyright (C) 2022-2023, D. R. Commander.
#if BITS_IN_JSAMPLE == 12
    /* 12-bit is the only data precision for which the range of the sample data
     * type exceeds the valid sample range.  Thus, we need to range-limit the
     * samples, because other algorithms may try to use them as array indices.
     */
    *output_buf++ = (_JSAMPLE)((*diff_buf++ << cinfo->Al) & 0xFFF);
#else
#endif
#if BITS_IN_JSAMPLE == 12
    *output_buf++ = (_JSAMPLE)((*diff_buf++) & 0xFFF);
#else
#endif