Repository: python-pillow/Pillow

Issue:
test_file_blp.py fails with libjpeg 9e since 9.3.0
### What did you do?
I built Pillow 9.3.0 as downloaded from PyPI and ran the test suite with the following command:
`/usr/bin/python3.X -m pytest Tests/test_file_blp.py`

### What did you expect to happen?
Every test passes.

### What actually happened?
`test_load_blp1` failed with the following error:
```python
    def test_load_blp1():
        with Image.open("Tests/images/blp/blp1_jpeg.blp") as im:
>           assert_image_equal_tofile(im, "Tests/images/blp/blp1_jpeg.png")

Tests/test_file_blp.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/helper.py:105: in assert_image_equal_tofile
    assert_image_equal(a, img, msg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = <PIL.BlpImagePlugin.BlpImageFile image mode=RGB size=256x256 at 0x7FF952EF3AD10>
b = <PIL.PngImagePlugin.PngImageFile image mode=RGB size=256x256 at 0x7FF952EF3AF10>, msg = None

    def assert_image_equal(a, b, msg=None):
        assert a.mode == b.mode, msg or f"got mode {repr(a.mode)}, expected {repr(b.mode)}"
        assert a.size == b.size, msg or f"got size {repr(a.size)}, expected {repr(b.size)}"
        if a.tobytes() != b.tobytes():
            if HAS_UPLOADER:
                try:
                    url = test_image_results.upload(a, b)
                    logger.error(f"Url for test images: {url}")
                except Exception:
                    pass
    
>           assert False, msg or "got different content"
E           AssertionError: got different content
E           assert False
```

I don't see this failure in 9.2.0 - it seems that https://github.com/python-pillow/Pillow/commit/f7363c1091c70356d92e56abfca6b65bef9e7b26 caused this issue. When I revert it, the issue is gone (when I revert only the [BlpImagePlugin.py](https://github.com/python-pillow/Pillow/commit/f7363c1091c70356d92e56abfca6b65bef9e7b26#diff-30059d5b5f19c6de1f1b7fcb21e982958ef9432c2795ef68fb5470fb9eccb9dd) change, pillow segfaults on newly added [test_file_blp.py](https://github.com/python-pillow/Pillow/commit/f7363c1091c70356d92e56abfca6b65bef9e7b26#diff-ffd01a62e614987be01f44aeed0c978fc9576964b6e6ff9d9bbba177044f1a28) code, but it passes the `assert_image_equal_tofile` part this failure is about.

This happens on both SPARC and Intel meaning than endianness is most likely not the issue.

### What are your OS, Python and Pillow versions?

* OS: Oracle Solaris 11.4 (both on SPARC and x86)
* Python: 3.7.14, 3.9.14 and 3.11.0
* Pillow: 9.3.0



Discussion:
Hi. Just to help us reproduce this, what version of libjpeg are you using? `python3.X -m PIL | grep "JPEG support"`
`JPEG support ok, compiled for 9.0` - specifically it's 9e.

I will try running it with a different libjpeg and see if that helps things.

No need to do that.

JPEG images are [allowed to be loaded slightly differently](https://pillow.readthedocs.io/en/stable/releasenotes/9.0.0.html#switched-to-libjpeg-turbo-in-macos-and-linux-wheels). So presuming that the difference is not too great, the test just needs to be relaxed to allow for variation.

If you change test_file_blp.py, replacing
```python
assert_image_equal_tofile(im, "Tests/images/blp/blp1_jpeg.png")
```
with
```python
with Image.open("Tests/images/blp/blp1_jpeg.png") as expected:
    assert_image_similar(im, expected., 0.001)
```
the error should change to report an average pixel difference. What is that number?
I am getting the following:
```python
E           AssertionError:  average pixel value difference 537.3142 > epsilon 0.0010
E           assert 0.001 >= 537.3141937255859
```
Doesn't seem like a small difference :(.
I just tested it on older machine where we still have libjpeg v6b2 and I see no issue there.
Testing, I find that 9d is fine, but 9e is not. Interesting. The problem is not Solaris at least.

These are the images with 9d and 9e.

| 9d | 9e |
| --- | --- |
| ![d](https://user-images.githubusercontent.com/3112309/202418397-d73070c6-fa27-4ff9-a568-caafa54dd695.png) | ![e](https://user-images.githubusercontent.com/3112309/202418401-77778e5d-b22b-4d66-bedc-effafaee9d2e.png) |

Here is the extracted JPEG

![in](https://user-images.githubusercontent.com/3112309/202425717-21a20bfd-bee4-4428-9c1a-f8656817d66b.jpg)

And here is what it is resaved by Pillow. So this is essentially a JPEG problem.

| 9d | 9e |
| --- | --- |
| ![d](https://user-images.githubusercontent.com/3112309/202425780-f8220148-4f7a-430a-9de5-077459fe5c3a.jpg) | ![e](https://user-images.githubusercontent.com/3112309/202425746-81aff908-fbe3-4732-9b93-e79f4256928d.jpg) |
The reason that this wasn't detected sooner is that Pillow tests with libjpeg-turbo.
Thanks for the investigation! I renamed the issue to reflect that it's not Solaris but the latest libjpeg that causes this.

Interesting that when I revert https://github.com/python-pillow/Pillow/commit/f7363c1091c70356d92e56abfca6b65bef9e7b26, it works with 9e as well. But I guess that that might be just a coincidence.

So, should this issue be reported to libjpeg maintainers instead?
Do you know where to do that? https://sourceforge.net/p/libjpeg/bugs/ hardly seems active.
I've created a reproduction not using Pillow, but [jpeglib](https://pypi.org/project/jpeglib) - https://github.com/radarhere/Pillow/tree/6741. This is a library that allow easy switching between libjpeg versions.

```python
import jpeglib
jpeglib.version.set('9d')
im = jpeglib.read_spatial("extracted.jpg")
print("9d", im.spatial[0, 0])
jpeglib.version.set('9e')
im = jpeglib.read_spatial("extracted.jpg")
print("9e", im.spatial[0, 0])
```
gives
```
9d [  1  50   1 255]
9e [255 136 255 255]
```
I've submitted an enquiry using the contact form at https://jpegclub.org/reference/contact/
Thanks - I also don't know what the correct way to report issues is - https://sourceforge.net/p/libjpeg/bugs/ seems to have one bug filed not that long ago (and with a reaction), but the contact form might also be a good idea.

Resolving patch:
diff --git a/src/PIL/BlpImagePlugin.py b/src/PIL/BlpImagePlugin.py
--- a/src/PIL/BlpImagePlugin.py
+++ b/src/PIL/BlpImagePlugin.py
@@ -373,6 +373,9 @@ def _decode_jpeg_stream(self):
         data = BytesIO(data)
         image = JpegImageFile(data)
         Image._decompression_bomb_check(image.size)
+        if image.mode == "CMYK":
+            decoder_name, extents, offset, args = image.tile[0]
+            image.tile = [(decoder_name, extents, offset, (args[0], "CMYK"))]
         r, g, b = image.convert("RGB").split()
         image = Image.merge("RGB", (b, g, r))
         self.set_as_raw(image.tobytes())


Test patch: