CVE-2016-10050: Improper Restriction of Operations within the Bounds of a Memory Buffer (CWE-119)
value: Heap-based buffer overflow in the ReadRLEImage function in coders/rle.c in ImageMagick 6.9.4-8 allows remote attackers to cause a denial of service (application crash) or have other unspecified impact via a crafted RLE file.
CWE description: The product performs operations on a memory buffer, but it reads from or writes to a memory location outside the buffer's intended boundary. This may result in read or write operations on unexpected memory locations that could be linked to other variables, data structures, or internal program data.
CVSS v3 base score: 7.8
Published: 2017-03-23T17:59Z
Repository: https://github.com/ImageMagick/ImageMagick
Language: Other
Files changed: ChangeLog, coders/rle.c
Commit date: 2016-06-05 14:25:18 -0400
Commit message: RLE check for pixel offset less than 0 (heap overflow report from Craig Young)

Diff: diff --git a/ChangeLog b/ChangeLog
index 4f9c488d05..6164e26414 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -2,10 +2,12 @@
   * Release ImageMagick version 7.0.1-10, GIT revision 10859:d5ffaa3:20160605.
 
 2016-06-04  7.0.1-10 Cristy  <quetzlzacatenango@image...>
   * Deny indirect reads by policy, remove policy to permit, e.g.,
     convert caption:@mytext.txt ...
+  * RLE check for pixel offset less than 0 (heap overflow report from Craig
+    Young).
 
 2016-06-03  7.0.1-9 Cristy  <quetzlzacatenango@image...>
   * Release ImageMagick version 7.0.1-9, GIT revision 10847:339f803:20160602.
 
 2016-06-02  7.0.1-9 Cristy  <quetzlzacatenango@image...>
diff --git a/coders/rle.c b/coders/rle.c
index 59e2411942..e13e5c8b5c 100644
--- a/coders/rle.c
+++ b/coders/rle.c
@@ -173,15 +173,15 @@ static Image *ReadRLEImage(const ImageInfo *image_info,ExceptionInfo *exception)
     map_length,
     number_colormaps,
     number_planes,
     number_planes_filled,
     one,
-    offset,
     pixel_info_length;
 
   ssize_t
     count,
+    offset,
     y;
 
   unsigned char
     background_color[256],
     *colormap,
@@ -387,11 +387,12 @@ static Image *ReadRLEImage(const ImageInfo *image_info,ExceptionInfo *exception)
           if (opcode & 0x40)
             operand=ReadBlobLSBSignedShort(image);
           offset=((image->rows-y-1)*image->columns*number_planes)+x*
             number_planes+plane;
           operand++;
-          if (offset+((size_t) operand*number_planes) > pixel_info_length)
+          if ((offset < 0) ||
+              (offset+((size_t) operand*number_planes) > pixel_info_length))
             {
               if (number_colormaps != 0)
                 colormap=(unsigned char *) RelinquishMagickMemory(colormap);
               pixel_info=RelinquishVirtualMemory(pixel_info);
               ThrowReaderException(CorruptImageError,"UnableToReadImageData");
@@ -418,11 +419,12 @@ static Image *ReadRLEImage(const ImageInfo *image_info,ExceptionInfo *exception)
           pixel=(unsigned char) ReadBlobByte(image);
           (void) ReadBlobByte(image);
           offset=((image->rows-y-1)*image->columns*number_planes)+x*
             number_planes+plane;
           operand++;
-          if (offset+((size_t) operand*number_planes) > pixel_info_length)
+          if ((offset < 0) ||
+              (offset+((size_t) operand*number_planes) > pixel_info_length))
             {
               if (number_colormaps != 0)
                 colormap=(unsigned char *) RelinquishMagickMemory(colormap);
               pixel_info=RelinquishVirtualMemory(pixel_info);
               ThrowReaderException(CorruptImageError,"UnableToReadImageData");

Vulnerable code:     offset,
          if (offset+((size_t) operand*number_planes) > pixel_info_length)
          if (offset+((size_t) operand*number_planes) > pixel_info_length)
Fixed code:   * RLE check for pixel offset less than 0 (heap overflow report from Craig
    Young).
    offset,
          if ((offset < 0) ||
              (offset+((size_t) operand*number_planes) > pixel_info_length))
          if ((offset < 0) ||
              (offset+((size_t) operand*number_planes) > pixel_info_length))