Repository: python-pillow/Pillow

Issue:
Sun Raster files with RLE encoding and depth 1 fail to load
### What did you do?
Tried to open a 1-bit RLE encoded Sun Raster file
### What did you expect to happen?
Expected image to load
### What actually happened?
Got a 'buffer overrun when reading image file' error
### What versions of Pillow and Python are you using?
Python 2.7.8, Pillow 3.3.1

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, django, or buildout, try to replicate the issue just using Pillow. 

Use image in attached ZIP file or create a new image in GIMP, 640x400 with white background. Select black brush, draw a line, convert Image->Mode to Indexed, and select black and white (1-bit) palette. Save image as Sun Raster file with RLE encoding.

[sunraster.zip](https://github.com/python-pillow/Pillow/files/597607/sunraster.zip)

```
>>> from PIL import Image
>>> im = Image.open('sunraster.im1')
>>> print im.format,im.size, im.mode
SUN (640, 400) 1
>>> im.load()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/PIL/ImageFile.py", line 240, in load
    raise_ioerror(err_code)
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/PIL/ImageFile.py", line 59, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: buffer overrun when reading image file
>>>

```


Discussion:
This is an error that's triggered from within the decoder, where it's being requested to write more into a buffer than remaining space in that buffer. It's checking, failing, and returning an error message.

RLE is run length encoding, in this case there's a marker for RLE, then a byte of how many, then a byte for the value. e.g. `0x80 0xff 0x00` is the code for 255 zeros 

What's going on here, at least initially, is that the Sun RLE decoder is expecting that when you're unpacking 255 items into a buffer, that there's at least 255 bytes of space in that buffer. 

```
                if (state->x + n > state->bytes) {
                    /* FIXME: is this correct? */
                    state->errcode = IMAGING_CODEC_OVERRUN;
                    return -1;
                }
```

This is in general, a correct requirement, but the buffer that's getting passed in is width/8 since it's a 1 bit buffer. So, 80 bytes. Hence the overflow. 

Modifying that so that we're expecting a byte per pixel buffer, we get past the first error, then on the third RLE bit, it's requesting another 255 items, but the width is 640, so there's another overflow. 

So.  Time to go to the RLE docs. http://www.fileformat.info/format/sunraster/egff.htm

>   Note also that the Sun Raster bitmap is read as if it is a single stream of data. Therefore, the encoding of pixel runs does not stop at the end of each scan line.

Also, from the same page:

>  Bitmap files with a Depth of 1 contain 2-color image data. [...] Each bit in the bitmap represents a pixel, with a value of 0 representing black and a value of 1 representing white (the bits are stored most significant bit first within each byte). 

So, the above guess that we were dealing with a byte/bit confusion is wrong, but what we need to do is wrap the scan lines when writing. So, if we've got an 80 byte buffer, and we're requesting a 255 byte write, it's

```
xxxxxxxx  (80) 
xxxxxxxx  (80)
xxxxxxxx  (80)
x         (15)
```

As an aside, this code is 2 months shy of 20 years old, and the header at the top says:

```
/*                                                                                                  
 * THIS IS WORK IN PROGRESS                                                                         
 *                                                                                                  
```

Sigh.

FWIW, I'm getting this from imagemagick: 

```
display: unexpected end-of-file `sunraster.im1':  @ error/sun.c/ReadSUNImage/568.
```

I've got it parsing now, but it still looks like there's a bug somewhere. And we're definitely hitting some sort of parsing error towards the end, it's reporting a truncated file but we're not reading the last 178 bytes.  


Resolving patch:
diff --git a/PIL/ImageFile.py b/PIL/ImageFile.py
--- a/PIL/ImageFile.py
+++ b/PIL/ImageFile.py
@@ -150,15 +150,16 @@ def load(self):
 
         if use_mmap:
             # try memory mapping
-            d, e, o, a = self.tile[0]
-            if d == "raw" and a[0] == self.mode and a[0] in Image._MAPMODES:
+            decoder_name, extents, offset, args = self.tile[0]
+            if decoder_name == "raw" and len(args) >= 3 and args[0] == self.mode \
+                   and args[0] in Image._MAPMODES:
                 try:
                     if hasattr(Image.core, "map"):
                         # use built-in mapper  WIN32 only
                         self.map = Image.core.map(self.filename)
-                        self.map.seek(o)
+                        self.map.seek(offset)
                         self.im = self.map.readimage(
-                            self.mode, self.size, a[1], a[2]
+                            self.mode, self.size, args[1], args[2]
                             )
                     else:
                         # use mmap, if possible
@@ -167,7 +168,7 @@ def load(self):
                         size = os.path.getsize(self.filename)
                         self.map = mmap.mmap(fp.fileno(), size, access=mmap.ACCESS_READ)
                         self.im = Image.core.map_buffer(
-                            self.map, self.size, d, e, o, a
+                            self.map, self.size, decoder_name, extents, offset, args
                             )
                     readonly = 1
                     # After trashing self.im, we might need to reload the palette data.
diff --git a/PIL/SunImagePlugin.py b/PIL/SunImagePlugin.py
--- a/PIL/SunImagePlugin.py
+++ b/PIL/SunImagePlugin.py
@@ -38,6 +38,21 @@ class SunImageFile(ImageFile.ImageFile):
 
     def _open(self):
 
+        # The Sun Raster file header is 32 bytes in length and has the following format:
+
+        #     typedef struct _SunRaster
+        #     {
+        #         DWORD MagicNumber;      /* Magic (identification) number */
+        #         DWORD Width;            /* Width of image in pixels */
+        #         DWORD Height;           /* Height of image in pixels */
+        #         DWORD Depth;            /* Number of bits per pixel */
+        #         DWORD Length;           /* Size of image data in bytes */
+        #         DWORD Type;             /* Type of raster file */
+        #         DWORD ColorMapType;     /* Type of color map */
+        #         DWORD ColorMapLength;   /* Size of the color map in bytes */
+        #     } SUNRASTER;
+
+
         # HEAD
         s = self.fp.read(32)
         if i32(s) != 0x59a66a95:
@@ -48,31 +63,71 @@ def _open(self):
         self.size = i32(s[4:8]), i32(s[8:12])
 
         depth = i32(s[12:16])
+        data_length = i32(s[16:20])  # unreliable, ignore. 
+        file_type = i32(s[20:24])
+        palette_type = i32(s[24:28]) # 0: None, 1: RGB, 2: Raw/arbitrary
+        palette_length = i32(s[28:32])
+        
         if depth == 1:
             self.mode, rawmode = "1", "1;I"
+        elif depth == 4:
+            self.mode, rawmode = "L", "L;4"
         elif depth == 8:
             self.mode = rawmode = "L"
         elif depth == 24:
-            self.mode, rawmode = "RGB", "BGR"
+            if file_type == 3:
+                self.mode, rawmode = "RGB", "RGB"
+            else:
+                self.mode, rawmode = "RGB", "BGR"
+        elif depth == 32:
+            if file_type == 3:
+                self.mode, rawmode = 'RGB', 'RGBX'
+            else:
+                self.mode, rawmode = 'RGB', 'BGRX'
         else:
-            raise SyntaxError("unsupported mode")
-
-        compression = i32(s[20:24])
-
-        if i32(s[24:28]) != 0:
-            length = i32(s[28:32])
-            offset = offset + length
-            self.palette = ImagePalette.raw("RGB;L", self.fp.read(length))
+            raise SyntaxError("Unsupported Mode/Bit Depth")    
+        
+        if palette_length:
+            if palette_length > 1024:
+                raise SyntaxError("Unsupported Color Palette Length")
+
+            if palette_type != 1:
+                raise SyntaxError("Unsupported Palette Type")
+            
+            offset = offset + palette_length
+            self.palette = ImagePalette.raw("RGB;L", self.fp.read(palette_length))
             if self.mode == "L":
-                self.mode = rawmode = "P"
-
-        stride = (((self.size[0] * depth + 7) // 8) + 3) & (~3)
-
-        if compression == 1:
+                self.mode = "P"
+                rawmode = rawmode.replace('L', 'P')
+            
+        # 16 bit boundaries on stride
+        stride = ((self.size[0] * depth + 15) // 16) * 2  
+
+        # file type: Type is the version (or flavor) of the bitmap
+        # file. The following values are typically found in the Type
+        # field:
+        # 0000h	Old
+        # 0001h	Standard
+        # 0002h	Byte-encoded
+        # 0003h	RGB format
+        # 0004h	TIFF format
+        # 0005h	IFF format
+        # FFFFh	Experimental
+
+        # Old and standard are the same, except for the length tag.
+        # byte-encoded is run-length-encoded
+        # RGB looks similar to standard, but RGB byte order
+        # TIFF and IFF mean that they were converted from T/IFF
+        # Experimental means that it's something else.
+        # (http://www.fileformat.info/format/sunraster/egff.htm)
+
+        if file_type in (0, 1, 3, 4, 5):
             self.tile = [("raw", (0, 0)+self.size, offset, (rawmode, stride))]
-        elif compression == 2:
+        elif file_type == 2:
             self.tile = [("sun_rle", (0, 0)+self.size, offset, rawmode)]
-
+        else:
+            raise SyntaxError('Unsupported Sun Raster file type')
+        
 #
 # registry
 


Test patch:
diff --git a/Tests/images/sunraster.im1 b/Tests/images/sunraster.im1
new file mode 100644
Binary files /dev/null and b/Tests/images/sunraster.im1 differ
diff --git a/Tests/images/sunraster.im1.png b/Tests/images/sunraster.im1.png
new file mode 100644
Binary files /dev/null and b/Tests/images/sunraster.im1.png differ
diff --git a/Tests/test_file_sun.py b/Tests/test_file_sun.py
--- a/Tests/test_file_sun.py
+++ b/Tests/test_file_sun.py
@@ -1,7 +1,10 @@
-from helper import unittest, PillowTestCase
+from helper import unittest, PillowTestCase, hopper
 
 from PIL import Image, SunImagePlugin
 
+import os
+
+EXTRA_DIR = 'Tests/images/sunraster'
 
 class TestFileSun(PillowTestCase):
 
@@ -16,10 +19,32 @@ def test_sanity(self):
         # Assert
         self.assertEqual(im.size, (128, 128))
 
+        self.assert_image_similar(im, hopper(), 5) # visually verified
+        
         invalid_file = "Tests/images/flower.jpg"
         self.assertRaises(SyntaxError,
                           lambda: SunImagePlugin.SunImageFile(invalid_file))
 
-
+    def test_im1(self):
+        im = Image.open('Tests/images/sunraster.im1')
+        target = Image.open('Tests/images/sunraster.im1.png')
+        self.assert_image_equal(im, target)
+
+
+    @unittest.skipIf(not os.path.exists(EXTRA_DIR),
+                     "Extra image files not installed")
+    def test_others(self):
+        files = (os.path.join(EXTRA_DIR, f) for f in
+                   os.listdir(EXTRA_DIR) if os.path.splitext(f)[1]
+                   in ('.sun', '.SUN', '.ras'))
+        for path in files:
+            with Image.open(path) as im:
+                im.load()  
+                self.assertIsInstance(im, SunImagePlugin.SunImageFile)
+                target_path = "%s.png" % os.path.splitext(path)[0]
+                #im.save(target_file)
+                with Image.open(target_path) as target:
+                    self.assert_image_equal(im, target)
+        
 if __name__ == '__main__':
     unittest.main()
diff --git a/depends/install_extra_test_images.sh b/depends/install_extra_test_images.sh
new file mode 100755
--- /dev/null
+++ b/depends/install_extra_test_images.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# install extra test images
+
+if [ ! -f test_images.tar.gz ]; then
+    wget -O 'test_images.tar.gz' 'https://github.com/python-pillow/pillow-depends/blob/master/test_images.tar.gz?raw=true'
+fi
+
+rm -r test_images
+tar -xvzf test_images.tar.gz
+
+cp -r test_images/* ../Tests/images