Repository: python-pillow/Pillow

Issue:
Some pcx images are displayed incorrectly
Hi everyone!
I'm trying to use pcx images extracted from an old game (Might and Magic VI, the game has 2D UI over the 3D environment. This 2D UI is composed by various internal formats and pcx files.)

The problem is that one of these pcx is displayed incorrecly by pillow, while being displayed correctly by EyeOfGnome (gdk-pixbuf) image viewer.
Another pcx is displayed correctly by pillow!

What do I mean by incorrectly? The bad image has a kind of "red shift" of pixels colors. the left border is a bit red. It' like a "3D-ified" image, it's a bit blurry and the whole is a bit darker.

I opened the pcx files with an hex editor and the main difference I spotted in the pcx-header is that the wrong one has the "48 byte palette" field completely empty. while the other one has some data in it. But it shoudn't be a problem because other viewers display it correctly.

I simply use this code
img = Image.open(pcxfilename)
img.show()

this archive contains the pcx files https://db.tt/td291VTh

gdk-pixbuf decoding of pcx is a bit more complex than the pillow one:
gdk-pixbuf/io-pcx.c VS libImaging/PcxDecode.c

thanks for any help!

![border](https://f.cloud.github.com/assets/130094/2161053/c8ed7598-94c1-11e3-8607-1f2ad1f208c1.png)

Some pcx images are displayed incorrectly
Hi everyone!
I'm trying to use pcx images extracted from an old game (Might and Magic VI, the game has 2D UI over the 3D environment. This 2D UI is composed by various internal formats and pcx files.)

The problem is that one of these pcx is displayed incorrecly by pillow, while being displayed correctly by EyeOfGnome (gdk-pixbuf) image viewer.
Another pcx is displayed correctly by pillow!

What do I mean by incorrectly? The bad image has a kind of "red shift" of pixels colors. the left border is a bit red. It' like a "3D-ified" image, it's a bit blurry and the whole is a bit darker.

I opened the pcx files with an hex editor and the main difference I spotted in the pcx-header is that the wrong one has the "48 byte palette" field completely empty. while the other one has some data in it. But it shoudn't be a problem because other viewers display it correctly.

I simply use this code
img = Image.open(pcxfilename)
img.show()

this archive contains the pcx files https://db.tt/td291VTh

gdk-pixbuf decoding of pcx is a bit more complex than the pillow one:
gdk-pixbuf/io-pcx.c VS libImaging/PcxDecode.c

thanks for any help!

![border](https://f.cloud.github.com/assets/130094/2161053/c8ed7598-94c1-11e3-8607-1f2ad1f208c1.png)



Discussion:
Yeah, there's definitely something going on there.  I don't think that it's the palette, since they're both full color RGB, non-palettized images. It looks like the implementation in PcxDecode is pretty close to the reference version, so I'm not too worried about that. 

It looks like there's an off by one error in the G and B  bands. I'm guessing that it's because there's an odd number of bytes, so the even stride length is throwing something off. Comparing to an imagemagick converted pcx->png, the first 12 bytes of each band look like:

```
R band
.pcx b'\xb1\xa8\x9c\x8d}uZh\xb6\xa4NX'
.png b'\xb1\xa8\x9c\x8d}uZh\xb6\xa4NX'
G band
.pcx b'\x00\xa4\x9c\x95\x87vnVd\xb0\x9eK'
.png b'\xa4\x9c\x95\x87vnVd\xb0\x9eKS'
B band
.pcx b'H\x00\x9a\x95\x93\x84qkTc\xb0\x9e'
.png b'\x9a\x95\x93\x84qkTc\xb0\x9eJO'
```

That \x00 byte really looks like the last padding byte of the previous band's row.  I think that likely we need to make the image width match the stride, at least while decoding the band interleaved rows.

(for reference, http://en.wikipedia.org/wiki/PCX , and http://web.archive.org/web/20100206055706/http://www.qzx.com/pc-gpe/pcx.txt )

Thanks wiredfool! That's it!!
I'm going to study the format but I think I'm beginning to understand, the first row is red because b and g are 0. (but it is vertical? it't by column? the color planes are by row or col?)
I thought it was a problem of "offset". Is the image malformed or out of standard or there is a bug in the reference that pillow implemented? I see that the example code in the txt is really similar

from wiki:
PCX image data are compressed using run-length encoding (RLE), a simple lossless compression algorithm that collapses a series of three or more consecutive bytes with identical values into a two-byte pair...

It could be a RLE problem?

Anyway I'm going to check the implementation ;) thanks again!

If the RLE decompression was wrong, we'd have more wrong than just this, especially wince the other one works. I _think_ what's happening is that in the image file, one row looks like this for an odd pixel size once decompressed from RLE:
(P == padding, RGB = image bytes)

```
RRRRRRRRRP
GGGGGGGGGP
BBBBBBBBBP
```

(note, 10 bytes wide)

And what we're winding up with is:

```
RRRRRRRRR
PGGGGGGGG
PBBBBBBBB
```

(note 9 bytes wide)

which then gets decoded to:

```
RPPA RGBA RGBA ...
```

What's probably happening is that the PcxDecoder is returning all the bytes for in the stride, including the padding, for the raw decoder to take from line by line and convert to the standard RGBA int for a pixel. The raw decoder is taking that byte stream and not discarding the padding byte.  The only problem with that theory is that the B values are only one byte off, and not two. 

If I'm right, any .pcx with an odd pixel width is going to do this. (though, to be fair, I could be wrong and that could still be the case, but I think it would be good confirmation)

You are right. I've extracted other images from the game archive.
and every  odd [W]x[H] has this problem.

here is the archive with more pcx to carry tests : https://db.tt/hQw3eN0J

I'm not sure but I think that at PcxImagePlugin.py, line 128

```
# bytes per plane
stride = (im.size[0] * bits + 7) // 8
```

if width is odd, it should be + 1 to that calculation. anyway isn't the stride is in the header? after the number of planes, offset 0x42.

that's only the save function but it may be a mismatch with the original pcx

tomorrow I'm gonna do some tests ;)

Well, that particular line is in the _save routine, so while it's a bug, it's not your main problem right now. 

Yeah, there's definitely something going on there.  I don't think that it's the palette, since they're both full color RGB, non-palettized images. It looks like the implementation in PcxDecode is pretty close to the reference version, so I'm not too worried about that. 

It looks like there's an off by one error in the G and B  bands. I'm guessing that it's because there's an odd number of bytes, so the even stride length is throwing something off. Comparing to an imagemagick converted pcx->png, the first 12 bytes of each band look like:

```
R band
.pcx b'\xb1\xa8\x9c\x8d}uZh\xb6\xa4NX'
.png b'\xb1\xa8\x9c\x8d}uZh\xb6\xa4NX'
G band
.pcx b'\x00\xa4\x9c\x95\x87vnVd\xb0\x9eK'
.png b'\xa4\x9c\x95\x87vnVd\xb0\x9eKS'
B band
.pcx b'H\x00\x9a\x95\x93\x84qkTc\xb0\x9e'
.png b'\x9a\x95\x93\x84qkTc\xb0\x9eJO'
```

That \x00 byte really looks like the last padding byte of the previous band's row.  I think that likely we need to make the image width match the stride, at least while decoding the band interleaved rows.

(for reference, http://en.wikipedia.org/wiki/PCX , and http://web.archive.org/web/20100206055706/http://www.qzx.com/pc-gpe/pcx.txt )

Thanks wiredfool! That's it!!
I'm going to study the format but I think I'm beginning to understand, the first row is red because b and g are 0. (but it is vertical? it't by column? the color planes are by row or col?)
I thought it was a problem of "offset". Is the image malformed or out of standard or there is a bug in the reference that pillow implemented? I see that the example code in the txt is really similar

from wiki:
PCX image data are compressed using run-length encoding (RLE), a simple lossless compression algorithm that collapses a series of three or more consecutive bytes with identical values into a two-byte pair...

It could be a RLE problem?

Anyway I'm going to check the implementation ;) thanks again!

If the RLE decompression was wrong, we'd have more wrong than just this, especially wince the other one works. I _think_ what's happening is that in the image file, one row looks like this for an odd pixel size once decompressed from RLE:
(P == padding, RGB = image bytes)

```
RRRRRRRRRP
GGGGGGGGGP
BBBBBBBBBP
```

(note, 10 bytes wide)

And what we're winding up with is:

```
RRRRRRRRR
PGGGGGGGG
PBBBBBBBB
```

(note 9 bytes wide)

which then gets decoded to:

```
RPPA RGBA RGBA ...
```

What's probably happening is that the PcxDecoder is returning all the bytes for in the stride, including the padding, for the raw decoder to take from line by line and convert to the standard RGBA int for a pixel. The raw decoder is taking that byte stream and not discarding the padding byte.  The only problem with that theory is that the B values are only one byte off, and not two. 

If I'm right, any .pcx with an odd pixel width is going to do this. (though, to be fair, I could be wrong and that could still be the case, but I think it would be good confirmation)

You are right. I've extracted other images from the game archive.
and every  odd [W]x[H] has this problem.

here is the archive with more pcx to carry tests : https://db.tt/hQw3eN0J

I'm not sure but I think that at PcxImagePlugin.py, line 128

```
# bytes per plane
stride = (im.size[0] * bits + 7) // 8
```

if width is odd, it should be + 1 to that calculation. anyway isn't the stride is in the header? after the number of planes, offset 0x42.

that's only the save function but it may be a mismatch with the original pcx

tomorrow I'm gonna do some tests ;)

Well, that particular line is in the _save routine, so while it's a bug, it's not your main problem right now. 


Resolving patch:
diff --git a/PIL/PcxImagePlugin.py b/PIL/PcxImagePlugin.py
--- a/PIL/PcxImagePlugin.py
+++ b/PIL/PcxImagePlugin.py
@@ -55,12 +55,18 @@ def _open(self):
         bbox = i16(s,4), i16(s,6), i16(s,8)+1, i16(s,10)+1
         if bbox[2] <= bbox[0] or bbox[3] <= bbox[1]:
             raise SyntaxError("bad PCX image size")
+        if Image.DEBUG:
+            print ("BBox: %s %s %s %s" % bbox)
+
 
         # format
         version = i8(s[1])
         bits = i8(s[3])
         planes = i8(s[65])
         stride = i16(s,66)
+        if Image.DEBUG:
+            print ("PCX version %s, bits %s, planes %s, stride %s" %
+                   (version, bits, planes, stride))
 
         self.info["dpi"] = i16(s,12), i16(s,14)
 
@@ -98,7 +104,9 @@ def _open(self):
         self.size = bbox[2]-bbox[0], bbox[3]-bbox[1]
 
         bbox = (0, 0) + self.size
-
+        if Image.DEBUG:
+            print ("size: %sx%s" % self.size)
+            
         self.tile = [("pcx", bbox, self.fp.tell(), (rawmode, planes * stride))]
 
 # --------------------------------------------------------------------
@@ -126,6 +134,16 @@ def _save(im, fp, filename, check=0):
 
     # bytes per plane
     stride = (im.size[0] * bits + 7) // 8
+    # stride should be even
+    stride = stride + (stride % 2)
+    # Stride needs to be kept in sync with the PcxEncode.c version.
+    # Ideally it should be passed in in the state, but the bytes value
+    # gets overwritten. 
+
+
+    if Image.DEBUG:
+        print ("PcxImagePlugin._save: xwidth: %d, bits: %d, stride: %d" % (
+            im.size[0], bits, stride))
 
     # under windows, we could determine the current screen size with
     # "Image.core.display_mode()[1]", but I think that's overkill...


Test patch:
diff --git a/Tests/test_file_pcx.py b/Tests/test_file_pcx.py
--- a/Tests/test_file_pcx.py
+++ b/Tests/test_file_pcx.py
@@ -2,29 +2,30 @@
 
 from PIL import Image
 
-def test_sanity():
-
-    file = tempfile("temp.pcx")
-
-    lena("1").save(file)
-
-    im = Image.open(file)
-    im.load()
-    assert_equal(im.mode, "1")
-    assert_equal(im.size, (128, 128))
-    assert_equal(im.format, "PCX")
-
-    lena("1").save(file)
-    im = Image.open(file)
-
-    lena("L").save(file)
-    im = Image.open(file)
 
-    lena("P").save(file)
-    im = Image.open(file)
+def _roundtrip(im):
+    f = tempfile("temp.pcx")
+    im.save(f)
+    im2 = Image.open(f)
 
-    lena("RGB").save(file)
-    im = Image.open(file)
+    assert_equal(im2.mode, im.mode)
+    assert_equal(im2.size, im.size)
+    assert_equal(im2.format, "PCX")
+    assert_image_equal(im2, im)
+    
+def test_sanity():
+    for mode in ('1', 'L', 'P', 'RGB'):
+        _roundtrip(lena(mode))
+
+def test_odd():
+    # see issue #523, odd sized images should have a stride that's even.
+    # not that imagemagick or gimp write pcx that way. 
+    # we were not handling properly. 
+    for mode in ('1', 'L', 'P', 'RGB'):
+        # larger, odd sized images are better here to ensure that
+        # we handle interrupted scan lines properly.
+        _roundtrip(lena(mode).resize((511,511)))
+        
 
 def test_pil184():
     # Check reading of files where xmin/xmax is not zero.