Repository: python-pillow/Pillow

Issue:
Add 16bit encode/decode to JPEG2K (WIP)
Add the ability to write 16bit JPEG2k Images (for DICOM medical images).

I'd trying to add a decoder too but I couldn't quite figure it out.  Here's my dumb test case:

``` python
import numpy as np
from PIL import Image

arr = np.zeros((64, 96), dtype=np.uint16)
arr[16:64, 0:32] = 300

img = Image.fromarray(arr, 'I;16')
img.save('grr.j2k')  # Works

img2 = Image.open('grr.j2k')
arr2 = np.array(img2)

np.testing.assert_almost_equal(arr, arr2)  # Fails
```

Thoughts?



Discussion:
Thanks for the patch.

I don't see anything obviously wrong.  I'm assuming that it runs and doesn't crash, since you got to the last line. A couple of things to check:

1) Is the image that's written correct/plausible when viewed?
2) is it the read that's failing?  If you compare from within PIL, does it look reasonable? Can we do roundtrip testing in Pillow to confirm that? (see Tests/test_file_jpeg2k.py)
3) Does the basic numpy->pillow->numpy work properly?  I wouldn't rule out something in the np.array(16_bit_image) part failing. e.g.:

```
img = Image.fromarray(arr, 'I;16')
arr2 = np.array(img)
assert_equal(arr, arr2)
```

Alright, I've just checked a bunch of things.

There's a bug in Preview that only shows JPEG2K at 8bits, but GIMP shows it correctly.  obj_dump shows a 16-bit image marked in the header.

Pillow->numpy->Pillow works.
numpy->Pillow->numpy works, but only because the buffer is being reused (I think?).

So I need to tweak the decoder somehow to add 16-bit support.

You could try img.copy or img.crop or some other image operation to force a new image for the numpy->Pillow->numpy test, but I'm suspecting that if the basic roundtrip there works, it's generally going to work.  The work of the Pillow->numpy stuff is in Image.py:**getattr**, where tobytes is called on the data. So, it's all a copy at that point anyway. 

Most likely explanation is the lack of decoder support.

Sorry, reading comprehension fail. I didn't see that you were having trouble with the decoder. (I'll blame jetlag).

First thing is to find the marker for 16bit bit depth in the header of the j2k file. (note that it's likely different between j2k and jp2 files, since they are a different format) If you see that marker and a csiz of 1, then instead of an mode 'L' image, it's mode 'I'. For jp2 files, it looks more involved, but something similar has to happen so that Pillow knows that it's getting a 16 bit image. 

Then in Jpeg2kDecode.c, there's an array of j2k_unpackers which has a mapping of mode, colorspace, and components to unpacker. An unpacker should be added here that has {"I",  OPJ_CLRSPC_GRAY, 1, j2ku_gray_i} where j2ku_gray_i is an unpacker to be written. 

That should do it for now. 

Yes, that's correct; you’ve only implemented the encode side; you need to add some code to deal with the _decode_ side. The JPEG2K code in Pillow can handle files with high bit depths, but it only ever decodes them to 8-bit PIL images at present.

FYI, the `j2ku_gray_i()` function should be easy enough to write; just copy the `j2ku_gray_l()` function, and change the line

```
int shift = 8 - in->comps[0].prec;
```

to read

```
int shift = 16 - in->comps[0].prec;
```

and then alter the `*row++` lines and/or change the `UINT8` types to `UINT16` in appropriate places.

You’ll also need to modify `_parse_codestream` and `_parse_jp2_header` in `Jpeg2KImagePlugin.py`. `j2k` files are the easy case, because almost everything is already done for you — you need to look at `ssiz[i]` to decide whether your data is 16bpp (or higher) or not.

As far as `jp2` files go, you need to add code to handle the `bpcc` box type — see the code around line 91:

```
if tbox == b'ihdr':
   ...
elif tbox == b'colr':
   ...
```

The format of this box, according to the spec, is _n_ bytes, where _n_ is the number of components, each of which contains `(bpp + 1) | s` where `s` is `0x80` if components are signed and `0x00` otherwise. You don’t need to worry about the sign, so just do `(x & 0x7f) + 1` to get the bit depth and do whatever checks you think are necessary.

Wow, sorry about the delay guys. The real world intervened.

I got my test code above to roundtrip. I can't seem to make the `j2k` codestream work though. `ssiz` is always `[0]` which seems maybe wrong? I'm pretty sure I'm still missing something, because GIMP stopped displaying the images I'm writing. Before it was just truncating at 8-bits...

I appreciate all the help.

[![Coverage Status](https://coveralls.io/builds/812318/badge)](https://coveralls.io/builds/812318)

Coverage increased (+0.25%) when pulling **e422ef4456e583848f4d6f557c547d0080b0157c on videan42:jpeg2k-16bit** into **b3dcf849b8b9b828510f0e2236dbeb04487d4d3f on python-imaging:master**.

Should I merge this?

I think we're waiting on decode for this one. 

Decode is working for me. I'm having trouble finding 16-bit images to test on in the wild. DICOMs are a little different than standard jpeg2000

That'll teach me to write before I've finished my coffee. 

@videan42 That’s the main reason I didn’t add any specific support for it in the original code :-)

A few comments:
1. I think `_parse_codestream` should apply the same test that `_parse_jp2_header` does to decide on the image mode. So, if you’re checking for a single-component 16-bit file (which you seem to be), it should check that `(ssiz[0] & 0x7f) == 15`. I know you say that your files seem to have 0 in `ssiz`, but that just means the files in question are broken (and in that case the effect will be to load them as 8 bit per pixel, which is no change from the present).
2. The checks for `bpc == 15` in `_parse_jp2_header` should mask `bpc` with `0x7f`, since just like `ssiz`, the `bpc` field also indicates the signedness of the component.
3. I’d previously suggested looking at the `bpcc` box, but I see that we’re really talking about single component 16-bit images here, so I think you’re right that checking the `ihdr` box is sufficient. There is a gotcha with the `bpc` field, in that it is allowed to be 255 if there is any variation in either bit depth or sign between different components (in which case you have to check `bpcc`), but I think in this restricted case we’re OK.
4. Should we decode to 16 bit if bits per component is greater than 8, rather than exactly 16? For instance, if we’re presented with a 12 bit image, should it be processed by pillow in 16 bits per pixel mode? I have no answer for this, I’m just putting the question out there.

Would love to see this in 2.5.0


Resolving patch:
diff --git a/PIL/Jpeg2KImagePlugin.py b/PIL/Jpeg2KImagePlugin.py
--- a/PIL/Jpeg2KImagePlugin.py
+++ b/PIL/Jpeg2KImagePlugin.py
@@ -40,7 +40,10 @@ def _parse_codestream(fp):
 
     size = (xsiz - xosiz, ysiz - yosiz)
     if csiz == 1:
-        mode = 'L'
+        if (yrsiz[0] & 0x7f) > 8:
+            mode = 'I;16'
+        else:
+            mode = 'L'
     elif csiz == 2:
         mode = 'LA'
     elif csiz == 3:
@@ -78,6 +81,7 @@ def _parse_jp2_header(fp):
 
     size = None
     mode = None
+    bpc = None
 
     hio = io.BytesIO(header)
     while True:
@@ -95,7 +99,9 @@ def _parse_jp2_header(fp):
                 = struct.unpack('>IIHBBBB', content)
             size = (width, height)
             if unkc:
-                if nc == 1:
+                if nc == 1 and (bpc & 0x7f) > 8:
+                    mode = 'I;16'
+                elif nc == 1:
                     mode = 'L'
                 elif nc == 2:
                     mode = 'LA'
@@ -109,13 +115,19 @@ def _parse_jp2_header(fp):
             if meth == 1:
                 cs = struct.unpack('>I', content[3:7])[0]
                 if cs == 16:   # sRGB
-                    if nc == 3:
+                    if nc == 1 and (bpc & 0x7f) > 8:
+                        mode = 'I;16'
+                    elif nc == 1:
+                        mode = 'L'
+                    elif nc == 3:
                         mode = 'RGB'
                     elif nc == 4:
                         mode = 'RGBA'
                     break
                 elif cs == 17:  # grayscale
-                    if nc == 1:
+                    if nc == 1 and (bpc & 0x7f) > 8:
+                        mode = 'I;16'
+                    elif nc == 1:
                         mode = 'L'
                     elif nc == 2:
                         mode = 'LA'
@@ -129,10 +141,10 @@ def _parse_jp2_header(fp):
 
     return (size, mode)
 
-
 ##
 # Image plugin for JPEG2000 images.
 
+
 class Jpeg2KImageFile(ImageFile.ImageFile):
     format = "JPEG2000"
     format_description = "JPEG 2000 (ISO 15444)"
@@ -174,7 +186,7 @@ def _open(self):
                 f.seek(pos, 0)
             except:
                 length = -1
-        
+
         self.tile = [('jpeg2k', (0, 0) + self.size, 0,
                       (self.codec, self.reduce, self.layers, fd, length))]
 


Test patch:
diff --git a/Tests/images/16bit.cropped.j2k b/Tests/images/16bit.cropped.j2k
new file mode 100644
Binary files /dev/null and b/Tests/images/16bit.cropped.j2k differ
diff --git a/Tests/images/16bit.cropped.jp2 b/Tests/images/16bit.cropped.jp2
new file mode 100644
Binary files /dev/null and b/Tests/images/16bit.cropped.jp2 differ
diff --git a/Tests/test_file_jpeg2k.py b/Tests/test_file_jpeg2k.py
--- a/Tests/test_file_jpeg2k.py
+++ b/Tests/test_file_jpeg2k.py
@@ -120,6 +120,42 @@ def test_rgba(self):
         self.assertEqual(j2k.mode, 'RGBA')
         self.assertEqual(jp2.mode, 'RGBA')
 
+    def test_16bit_monochrome_has_correct_mode(self):
+
+        j2k = Image.open('Tests/images/16bit.cropped.j2k')
+        jp2 = Image.open('Tests/images/16bit.cropped.jp2')
+
+        j2k.load()
+        jp2.load()
+
+        self.assertEqual(j2k.mode, 'I;16')
+        self.assertEqual(jp2.mode, 'I;16')
+
+    def test_16bit_monchrome_jp2_like_tiff(self):
+
+        tiff_16bit = Image.open('Tests/images/16bit.cropped.tif')
+        jp2 = Image.open('Tests/images/16bit.cropped.jp2')
+        self.assert_image_similar(jp2, tiff_16bit, 1e-3)
+
+    def test_16bit_monchrome_j2k_like_tiff(self):
+
+        tiff_16bit = Image.open('Tests/images/16bit.cropped.tif')
+        j2k = Image.open('Tests/images/16bit.cropped.j2k')
+        self.assert_image_similar(j2k, tiff_16bit, 1e-3)
+
+    def test_16bit_j2k_roundtrips(self):
+
+        j2k = Image.open('Tests/images/16bit.cropped.j2k')
+        im = self.roundtrip(j2k)
+        self.assert_image_equal(im, j2k)
+
+    def test_16bit_jp2_roundtrips(self):
+
+        jp2 = Image.open('Tests/images/16bit.cropped.jp2')
+        im = self.roundtrip(jp2)
+        self.assert_image_equal(im, jp2)
+
+
 if __name__ == '__main__':
     unittest.main()