Repository: python-pillow/Pillow

Issue:
16-bit PGM images
Is there support (or planned support) for [16-bit PGM](https://en.wikipedia.org/wiki/Portable_anymap#16-bit_extensions) images? I ask because I noticed that rather than throwing any error message, Pillow seems to read in 16-bit PGM images as if they were 8-bit. See below for a test case that demonstrates this.

```
$ curl -o 16_bit_binary.pgm https://raw.githubusercontent.com/myint/image-view/master/test/16_bit_binary.pgm
```

``` python
Python 3.4.0 (default, Mar 25 2014, 11:07:05)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
>>> import PIL
>>> PIL.VERSION
'1.1.7'
>>> PIL.PILLOW_VERSION
'2.4.0'
>>> image = PIL.Image.open('16_bit_binary.pgm')
>>> image
<PIL.PpmImagePlugin.PpmImageFile image mode=L size=32x16 at 0x105C76A58>
>>> len(image.getdata())
512
>>> image.save('foo.pgm')
```

```
$ identify foo.pgm
foo.pgm PGM 32x16 32x16+0+0 8-bit Grayscale Gray 525B 0.000u 0:00.000
```

``` diff
--- 16_bit_binary.pgm   2014-04-05 09:13:36.000000000 -0700
+++ foo.pgm 2014-04-05 09:17:49.000000000 -0700
@@ -1,5 +1,4 @@
 P5
-# Here is some 16-bit data.
 32 16
-65535
-L�L�I�J�M�H�J�H�G�L�E�F�H�C�C�D�E�G�M�PR;V�]p]pW�Y�U�G�=6:{$�uL�L�I�J�M�I�J�H�G�M�E�G�H�C�C�C�E�G�M�PR;V�]p]pW�Y�U�G�=6;b$�uM�J�I�I�K�F�G�F�F�H�C�C�F�D�C�F�F�D�K�PSQV�\H\HY�X�SQG�=64a"�I�H�I�F�I�I�C�@�D�E�B�@�E�C�@E�G�B�G�PTiV�Y�["["W�Q'F�<K-"�E�H�F�F�H�F�G�C�C�F�E�B�C�B�B�E�B�F�L�SQV�V�Y�["X�U�PB�9�0>#�uF�H�D�?G�F�E�D�E�E�F�E�@�C�D�C�D�H�M�R;V�V�["X�X�TiSQA�3�-�$�F�F�F�E�G�F�C�F�G�E�E�D�D�C�D�C�F�OR;V�X�W�V�V�V�SQPA�5:-�!��G�G�F�F�E�F�F�D�E�D�E�B�B�C�C�I�K�R;U�V�W�W�U�V�SQOF�A�6�+�"�E�F�E�A�F�E�F�E�C�F�E�D�A�@�E�M�M�U�V�SQV�SQR;R;R;K�H�@4a,K ��D�F�G�D�G�F�F�I�I�H�C�F�D�E�H�K�Q'U�SQTiU�R;SQM�Q'K�E�>#60>(��E�F�E�E�E�E�G�H�I�G�C�E�A�E�H�OR;U�OTiSQQ'Q'L�K�L�F�@6/p)C"3F�D�A�F�F�E�G�G�I�F�G�E�C�D�G�OV�W�SQOSQQ'PM�J�K�E�@�:{.�(�D�A�B�F�I�H�G�G�I�E�F�G�F�I�L�R;SQU�W�R;Q'OM�H�K�K�F�@�9�.�$>�D�D�F�D�F�I�H�F�G�I�F�E�G�L�Q'SQSQU�SQTiK�K�K�J�K�K�H�@�9�-� 8�F�I�H�F�G�G�H�G�H�I�I�I�K�R;U�X�W�U�TiPJ�J�G�H�M�M�J�C�7�-"3�F�H�G�H�H�H�J�I�I�I�J�OQ'TiTiW�W�TiSQM�J�H�E�H�K�OM�D�9�/p#�P5
+255
+L�L�I�J�M�H�J�H�G�L�E�F�H�C�C�D�E�G�M�PR;V�]p]pW�Y�U�G�=6:{$�uL�L�I�J�M�I�J�H�G�M�E�G�H�C�C�C�E�G�M�PR;V�]p]pW�Y�U�G�=6;b$�uM�J�I�I�K�F�G�F�F�H�C�C�F�D�C�F�F�D�K�PSQV�\H\HY�X�SQG�=64a"�I�H�I�F�I�I�C�@�D�E�B�@�E�C�@E�G�B�G�PTiV�Y�["["W�Q'F�<K-"�E�H�F�F�H�F�G�C�C�F�E�B�C�B�B�E�B�F�L�SQV�V�Y�["X�U�PB�9�0>#�uF�H�D�?G�F�E�D�E�E�F�E�@�C�D�C�D�H�M�R;V�V�["X�X�TiSQA�3�-�$�F�F�F�E�G�F�C�F�G�E�E�D�D�C�D�C�F�OR;V�X�W�V�V�V�SQPA�5:-�!��G�G�F�F�E�F�F�D�E�D�E�B�B�C�C�I�K�R;U�V�W�W�U�V�SQOF�A�6�+�"�
\ No newline at end of file
```

It looks like Pillow just read the first half of the PGM image.

16-bit PGM images
Is there support (or planned support) for [16-bit PGM](https://en.wikipedia.org/wiki/Portable_anymap#16-bit_extensions) images? I ask because I noticed that rather than throwing any error message, Pillow seems to read in 16-bit PGM images as if they were 8-bit. See below for a test case that demonstrates this.

```
$ curl -o 16_bit_binary.pgm https://raw.githubusercontent.com/myint/image-view/master/test/16_bit_binary.pgm
```

``` python
Python 3.4.0 (default, Mar 25 2014, 11:07:05)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
>>> import PIL
>>> PIL.VERSION
'1.1.7'
>>> PIL.PILLOW_VERSION
'2.4.0'
>>> image = PIL.Image.open('16_bit_binary.pgm')
>>> image
<PIL.PpmImagePlugin.PpmImageFile image mode=L size=32x16 at 0x105C76A58>
>>> len(image.getdata())
512
>>> image.save('foo.pgm')
```

```
$ identify foo.pgm
foo.pgm PGM 32x16 32x16+0+0 8-bit Grayscale Gray 525B 0.000u 0:00.000
```

``` diff
--- 16_bit_binary.pgm   2014-04-05 09:13:36.000000000 -0700
+++ foo.pgm 2014-04-05 09:17:49.000000000 -0700
@@ -1,5 +1,4 @@
 P5
-# Here is some 16-bit data.
 32 16
-65535
-L�L�I�J�M�H�J�H�G�L�E�F�H�C�C�D�E�G�M�PR;V�]p]pW�Y�U�G�=6:{$�uL�L�I�J�M�I�J�H�G�M�E�G�H�C�C�C�E�G�M�PR;V�]p]pW�Y�U�G�=6;b$�uM�J�I�I�K�F�G�F�F�H�C�C�F�D�C�F�F�D�K�PSQV�\H\HY�X�SQG�=64a"�I�H�I�F�I�I�C�@�D�E�B�@�E�C�@E�G�B�G�PTiV�Y�["["W�Q'F�<K-"�E�H�F�F�H�F�G�C�C�F�E�B�C�B�B�E�B�F�L�SQV�V�Y�["X�U�PB�9�0>#�uF�H�D�?G�F�E�D�E�E�F�E�@�C�D�C�D�H�M�R;V�V�["X�X�TiSQA�3�-�$�F�F�F�E�G�F�C�F�G�E�E�D�D�C�D�C�F�OR;V�X�W�V�V�V�SQPA�5:-�!��G�G�F�F�E�F�F�D�E�D�E�B�B�C�C�I�K�R;U�V�W�W�U�V�SQOF�A�6�+�"�E�F�E�A�F�E�F�E�C�F�E�D�A�@�E�M�M�U�V�SQV�SQR;R;R;K�H�@4a,K ��D�F�G�D�G�F�F�I�I�H�C�F�D�E�H�K�Q'U�SQTiU�R;SQM�Q'K�E�>#60>(��E�F�E�E�E�E�G�H�I�G�C�E�A�E�H�OR;U�OTiSQQ'Q'L�K�L�F�@6/p)C"3F�D�A�F�F�E�G�G�I�F�G�E�C�D�G�OV�W�SQOSQQ'PM�J�K�E�@�:{.�(�D�A�B�F�I�H�G�G�I�E�F�G�F�I�L�R;SQU�W�R;Q'OM�H�K�K�F�@�9�.�$>�D�D�F�D�F�I�H�F�G�I�F�E�G�L�Q'SQSQU�SQTiK�K�K�J�K�K�H�@�9�-� 8�F�I�H�F�G�G�H�G�H�I�I�I�K�R;U�X�W�U�TiPJ�J�G�H�M�M�J�C�7�-"3�F�H�G�H�H�H�J�I�I�I�J�OQ'TiTiW�W�TiSQM�J�H�E�H�K�OM�D�9�/p#�P5
+255
+L�L�I�J�M�H�J�H�G�L�E�F�H�C�C�D�E�G�M�PR;V�]p]pW�Y�U�G�=6:{$�uL�L�I�J�M�I�J�H�G�M�E�G�H�C�C�C�E�G�M�PR;V�]p]pW�Y�U�G�=6;b$�uM�J�I�I�K�F�G�F�F�H�C�C�F�D�C�F�F�D�K�PSQV�\H\HY�X�SQG�=64a"�I�H�I�F�I�I�C�@�D�E�B�@�E�C�@E�G�B�G�PTiV�Y�["["W�Q'F�<K-"�E�H�F�F�H�F�G�C�C�F�E�B�C�B�B�E�B�F�L�SQV�V�Y�["X�U�PB�9�0>#�uF�H�D�?G�F�E�D�E�E�F�E�@�C�D�C�D�H�M�R;V�V�["X�X�TiSQA�3�-�$�F�F�F�E�G�F�C�F�G�E�E�D�D�C�D�C�F�OR;V�X�W�V�V�V�SQPA�5:-�!��G�G�F�F�E�F�F�D�E�D�E�B�B�C�C�I�K�R;U�V�W�W�U�V�SQOF�A�6�+�"�
\ No newline at end of file
```

It looks like Pillow just read the first half of the PGM image.



Discussion:
FWIW, the `16_bit_binary.pgm` file is not quite [PGM](http://netpbm.sourceforge.net/doc/pgm.html) conform. The last three bytes indicate that there is a second P5 image in the file, but there is not.

Thanks for the information.

Just for completeness, I tried with a PGM that actually conforms. The same problem occurs.

```
$ curl -o 16.pgm https://gist.githubusercontent.com/paulrd/8042194/raw/256b4cee6885a3fe01582ed0983730343ec7205d/2013-11-05+11-49-49_MCA000_65.3.pgm
```

``` python
>>> import PIL
>>> image = PIL.Image.open('16.pgm')
>>> image
<PIL.PpmImagePlugin.PpmImageFile image mode=L size=1280x960 at 0x10915AA20>
>>> image.save('foo.pgm')
```

```
$ identify 16.pgm
16.pgm PGM 1280x960 1280x960+0+0 16-bit Grayscale Gray 2.458MB 0.010u 0:00.009
$ identify foo.pgm
foo.pgm PGM 1280x960 1280x960+0+0 8-bit Grayscale Gray 1.229MB 0.010u 0:00.009
```

Whoops, I accidentally pressed the close button. I've actually updated the original 16-bit PGM to use a valid one.

```
$ convert -size 32x16 pattern:checkerboard 16_bit_binary.pgm
```

```
$ curl -o 16_bit_binary.pgm https://raw.githubusercontent.com/myint/image-view/master/test/16_bit_binary.pgm
$ identify 16_bit_binary.pgm
16_bit_binary.pgm PGM 32x16 32x16+0+0 16-bit Grayscale Gray 1.04KB 0.000u 0:00.009
```

@myint Can you send a pull request?

Probably not. I'm not sure what the solution to this problem is. (And I haven't looked at the PIL/Pillow code base before.)

It looks like pillow is totally ignoring the maxgray entry, just reading the token and discarding it. This file format appears to be one of those that has multiple competing standards, and at least one of them (http://www.fileformat.info/format/pbm/egff.htm) says one byte/pixel. 

FWIW, the `16_bit_binary.pgm` file is not quite [PGM](http://netpbm.sourceforge.net/doc/pgm.html) conform. The last three bytes indicate that there is a second P5 image in the file, but there is not.

Thanks for the information.

Just for completeness, I tried with a PGM that actually conforms. The same problem occurs.

```
$ curl -o 16.pgm https://gist.githubusercontent.com/paulrd/8042194/raw/256b4cee6885a3fe01582ed0983730343ec7205d/2013-11-05+11-49-49_MCA000_65.3.pgm
```

``` python
>>> import PIL
>>> image = PIL.Image.open('16.pgm')
>>> image
<PIL.PpmImagePlugin.PpmImageFile image mode=L size=1280x960 at 0x10915AA20>
>>> image.save('foo.pgm')
```

```
$ identify 16.pgm
16.pgm PGM 1280x960 1280x960+0+0 16-bit Grayscale Gray 2.458MB 0.010u 0:00.009
$ identify foo.pgm
foo.pgm PGM 1280x960 1280x960+0+0 8-bit Grayscale Gray 1.229MB 0.010u 0:00.009
```

Whoops, I accidentally pressed the close button. I've actually updated the original 16-bit PGM to use a valid one.

```
$ convert -size 32x16 pattern:checkerboard 16_bit_binary.pgm
```

```
$ curl -o 16_bit_binary.pgm https://raw.githubusercontent.com/myint/image-view/master/test/16_bit_binary.pgm
$ identify 16_bit_binary.pgm
16_bit_binary.pgm PGM 32x16 32x16+0+0 16-bit Grayscale Gray 1.04KB 0.000u 0:00.009
```

@myint Can you send a pull request?

Probably not. I'm not sure what the solution to this problem is. (And I haven't looked at the PIL/Pillow code base before.)

It looks like pillow is totally ignoring the maxgray entry, just reading the token and discarding it. This file format appears to be one of those that has multiple competing standards, and at least one of them (http://www.fileformat.info/format/pbm/egff.htm) says one byte/pixel. 


Resolving patch:
diff --git a/PIL/PpmImagePlugin.py b/PIL/PpmImagePlugin.py
--- a/PIL/PpmImagePlugin.py
+++ b/PIL/PpmImagePlugin.py
@@ -63,7 +63,11 @@ def _token(self, s = b""):
             c = self.fp.read(1)
             if not c or c in b_whitespace:
                 break
+            if c > b'\x79':
+                raise ValueError("Expected ASCII value, found binary")
             s = s + c
+            if (len(s) > 9):
+                raise ValueError("Expected int, got > 9 digits")
         return s
 
     def _open(self):
@@ -96,7 +100,18 @@ def _open(self):
                 ysize = s
                 if mode == "1":
                     break
-
+            elif ix == 2:
+                # maxgrey
+                if s > 255:
+                    if not mode == 'L':
+                        raise ValueError("Too many colors for band: %s" %s)
+                    if s < 2**16:
+                        self.mode = 'I'
+                        rawmode = 'I;16B'
+                    else:
+                        self.mode = 'I';
+                        rawmode = 'I;32B'
+                        
         self.size = xsize, ysize
         self.tile = [("raw",
                      (0, 0, xsize, ysize),
@@ -116,6 +131,11 @@ def _save(im, fp, filename):
         rawmode, head = "1;I", b"P4"
     elif im.mode == "L":
         rawmode, head = "L", b"P5"
+    elif im.mode == "I":
+        if im.getextrema()[1] < 2**16:
+            rawmode, head = "I;16B", b"P5"
+        else:
+            rawmode, head = "I;32B", b"P5"
     elif im.mode == "RGB":
         rawmode, head = "RGB", b"P6"
     elif im.mode == "RGBA":
@@ -123,8 +143,15 @@ def _save(im, fp, filename):
     else:
         raise IOError("cannot write mode %s as PPM" % im.mode)
     fp.write(head + ("\n%d %d\n" % im.size).encode('ascii'))
-    if head != b"P4":
+    if head == b"P6":
         fp.write(b"255\n")
+    if head == b"P5":
+        if rawmode == "L":
+            fp.write(b"255\n")
+        elif rawmode == "I;16B":
+            fp.write(b"65535\n")
+        elif rawmode == "I;32B":
+            fp.write(b"2147483648\n")
     ImageFile._save(im, fp, [("raw", (0,0)+im.size, 0, (rawmode, 0, 1))])
 
     # ALTERNATIVE: save via builtin debug function


Test patch:
diff --git a/Tests/images/16_bit_binary.pgm b/Tests/images/16_bit_binary.pgm
new file mode 100644
Binary files /dev/null and b/Tests/images/16_bit_binary.pgm differ
diff --git a/Tests/images/16_bit_binary_pgm.png b/Tests/images/16_bit_binary_pgm.png
new file mode 100644
Binary files /dev/null and b/Tests/images/16_bit_binary_pgm.png differ
diff --git a/Tests/test_file_ppm.py b/Tests/test_file_ppm.py
--- a/Tests/test_file_ppm.py
+++ b/Tests/test_file_ppm.py
@@ -12,3 +12,25 @@ def test_sanity():
     assert_equal(im.mode, "RGB")
     assert_equal(im.size, (128, 128))
     assert_equal(im.format, "PPM")
+
+def test_16bit_pgm():
+    im = Image.open('Tests/images/16_bit_binary.pgm')
+    im.load()
+    assert_equal(im.mode, 'I')
+    assert_equal(im.size, (20,100))
+
+    tgt = Image.open('Tests/images/16_bit_binary_pgm.png')
+    assert_image_equal(im, tgt)
+
+
+def test_16bit_pgm_write():
+    im = Image.open('Tests/images/16_bit_binary.pgm')
+    im.load()
+
+    f = tempfile('temp.pgm')
+    assert_no_exception(lambda: im.save(f, 'PPM'))
+
+    reloaded = Image.open(f)
+    assert_image_equal(im, reloaded)
+
+