Repository: python-pillow/Pillow

Issue:
RGBA image losing its transparency when saved as pdf
### What did you do?
convert an png to image to pdf or ~JPEG2000~
edit: 
   after checking with various  image viewer, I've observed that the issue could be linked with my original viewer(nomacs)
   In order to not bother any I will left on side JPEG2000 and carry on my investigations

### What did you expect to happen?

have the good alpha channel

### What actually happened?

alpha channel not correctly displayed in pdf  ~is not the good one~

### What are your OS, Python and Pillow versions?

* OS: windows 11
* Python:  3.10
* Pillow: 10.0
with this png
![tt](https://github.com/python-pillow/Pillow/assets/4083478/e054f4b7-dcbc-4173-ab28-e19e9b6d4fd9)


```python
from PIL import Image
img = Image.open("tt.png")
print("mode : ",img.mode)
img.save("tt.pdf")
img.save("tt.jp2")
img = Image.open("tt.jp2")
print("mode jp2 : ",img2.mode)
img.save("tt.jp2")
```
output
_mode : RGBA_
_mode jp2 : RGBA_

but the image in the PDF has lost its transparency
~same for the jpeg2000 image, although still reported as RGBA~



Discussion:
I can't confirm JPEG2000. Your image is transparent after conversion.
@homm 
this is the pdf I'm getting: 
[tt.pdf](https://github.com/python-pillow/Pillow/files/12233871/tt.pdf)
the  background is black whereas the page is normally white
can you confirm you have the same result ?

This is how your pdf is rendered to me. It's unclear to me if PDF really can be transparent. 

<img width="902" alt="Screenshot 2023-08-02 at 01 55 57" src="https://github.com/python-pillow/Pillow/assets/128982/3bedfe56-6446-4fac-9338-59ad1a0bd72d">

I'm also unable to replicate the problem with either format. Are you sure that you are using Pillow 10.0.0, and not a version before 9.5.0?
I reinstalled it before to confirm the bug was on the latest version.
Actually it looks like the problem is also linked with the viewer:
https://github.com/mozilla/pdf.js/issues/5649#issuecomment-1597489448

Personally I checked all under windows with acrobat reader (black) sumatra (transparent) pdf.js( black and damaged) chrome (black)
I am using Pillow 10.0.0 and getting the same incorrect results using the code above:

[Transparent_Image](https://github.com/python-pillow/Pillow/assets/131583283/643287e7-d6a9-4272-879e-732fb19a778e)
[Transparent_Image.pdf](https://github.com/python-pillow/Pillow/files/12236794/Transparent_Image.pdf)

[Transparent_Image2](https://github.com/python-pillow/Pillow/assets/131583283/06e4afe0-0f6b-4d8c-b733-b0378c3a0d0b)
[Transparent_Image2.pdf](https://github.com/python-pillow/Pillow/files/12236795/Transparent_Image2.pdf)


How did you install Pillow?

What version of OpenJPEG are you using?
```pycon
>>> from PIL import features
>>> features.version_codec("jpg_2000")
'2.5.0'
```
I have installed it using pip on Windows 11. The OpenJPEG version is 2.5.0.
Sorry, people, could you be a bit more verbose here? You are claiming that the initial problem is related to both PDF and JP2K. So could we isolate this? For example, if you are sure that the problem with PDF happens because of embedding bad JP2K, let's focus on the JP2K problem here. Otherwise, if there are separate problems, let's consider them separate.

I've tried to convert the example image to JP2K above and experienced no problems, it has transparency. If some of you have different results, could you please at least mention this and provide your result with broken transparency and expose why you think it s broken (software you are using to viewing, for example).
The problem is not (only) related to JP2K, because in my example you find 2 transparent png images, both are not correctly converted to PDF.

The PDF files have been opened with:

- Acrobat Reader 2023.003.20244
- Google Chrome 115.0.5790.110
- qpdf 11.5.0 (extracting the image)
- macOS 13.5 Preview App


"is not related to JP2K" and "is not only related to JP2K" are two completely different statements.

@pubpub-zz could you provide any example or proof that you are experiencing problems with JP2K?
> I am using Pillow 10.0.0 and getting the same incorrect results using the code above:

Why do you consider these results incorrect? Which result would be correct?

I've tried to convert the second image to PDF using macOS Preview App (with "Alpha" checkbox applied), the results are identical:

<img width="1420" alt="Screenshot 2023-08-02 at 16 20 54" src="https://github.com/python-pillow/Pillow/assets/128982/ea2e6483-1fd3-4781-a9d3-8b66d28590c7">
Your results are correct, but the PDFs generated using Pillow 10.0.0 look like this (on all PDF viewers mentioned above):

<img width="1007" alt="Screenshot 2023-08-02 152450" src="https://github.com/python-pillow/Pillow/assets/131583283/26d4b9c2-8c5c-4169-a83a-088199e60e8b">

> but the PDFs generated using Pillow 10.0.0 

The file opened in the right window on my screenshot is your PDF [from the comment](https://github.com/python-pillow/Pillow/issues/7312#issuecomment-1661580046) which claimed as generated using Pillow 10.0.
My test showed that the viewer allows to show or not the transparency. I have the same feeling about jpeg 2000 viewer (the image was not good with nomacs but displayed correctly in gimp and imagemagick)

digging further in the pdf specification I found this:
![image](https://github.com/python-pillow/Pillow/assets/4083478/47ebd03f-ecb3-40f6-8d5a-656b274e22d7)

I've removed ColorSpace to take into account
I've set the SMaskIn to 1 and the transparency "appeared" 😁 in Acrobat Reader.
I've checked in pdf.js and there the image is damaged: 
![image](https://github.com/python-pillow/Pillow/assets/4083478/a0741202-10d4-40e8-8d7c-8f79c40e9e19)
removing the "BitsPerComponent fixed this:
![image](https://github.com/python-pillow/Pillow/assets/4083478/d98dea0e-c62a-4613-a9f3-8ef140ebd39c)

There is still a color inversion but I think this is a bug in pdf.js(https://github.com/mozilla/pdf.js/issues/16782)


to sum-up, can you :
* add SMaskIn=1 (to enable alpha decoding) 
* remove ColorSpace and BitsPerComponents (to allow the extraction of data from the image)


with the fixed PDF:
[tt1.pdf](https://github.com/python-pillow/Pillow/files/12243293/tt1.pdf)

I've created PR #7316 to set SMaskInData to 1.

> I've removed ColorSpace to take into account

Could you explain this some more?

> remove ColorSpace and BitsPerComponents (to allow the extraction of data from the image)

What makes you think that the color inversion is a bug in pdf.js, but not the BitsPerComponent problem? Does "allow the extraction of data from the image" mean something special, or is that just another way of saying "appear correctly in a viewer"?
> I've created PR #7316 to set SMaskInData to 1.

👍 

> > I've removed ColorSpace to take into account
> Could you explain this some more?

This is an extract from Pdf 1.7 reference:
![image](https://github.com/python-pillow/Pillow/assets/4083478/b0bb9de1-7e00-4e2b-be58-154f8c90f09c)
(...)
![image](https://github.com/python-pillow/Pillow/assets/4083478/14aa7691-27e0-4e48-998b-a398eb4c0e63)

For the same reason we should ignore BitsPerComponents
![image](https://github.com/python-pillow/Pillow/assets/4083478/da3f7f7e-d197-4a50-9c6f-e36eaefd0dc0)


> > remove ColorSpace and BitsPerComponents (to allow the extraction of data from the image)
> 
> What makes you think that the color inversion is a bug in pdf.js, but not the BitsPerComponent problem? Does "allow the extraction of data from the image" mean something special, or is that just another way of saying "appear correctly in a viewer"?

I've tested a few set of PdfViewer( Acrobat / Sumatra / pdf.js / chrome) and the issue was observable in PDF.js.  previously this time of error was linked with Decode entry but setting the Decode entry was not improving that.
Thanks. I've updated the PR to remove those properties.

I presume that is sufficient to fix the original problem here. 'Transparent_Image2' is also an RGBA image, so I think it should fix that as well.

Resolving patch:
diff --git a/src/PIL/PdfImagePlugin.py b/src/PIL/PdfImagePlugin.py
--- a/src/PIL/PdfImagePlugin.py
+++ b/src/PIL/PdfImagePlugin.py
@@ -46,6 +46,132 @@ def _save_all(im, fp, filename):
 # (Internal) Image save plugin for the PDF format.
 
 
+def _write_image(im, filename, existing_pdf, image_refs):
+    # FIXME: Should replace ASCIIHexDecode with RunLengthDecode
+    # (packbits) or LZWDecode (tiff/lzw compression).  Note that
+    # PDF 1.2 also supports Flatedecode (zip compression).
+
+    params = None
+    decode = None
+
+    #
+    # Get image characteristics
+
+    width, height = im.size
+
+    dict_obj = {"BitsPerComponent": 8}
+    if im.mode == "1":
+        if features.check("libtiff"):
+            filter = "CCITTFaxDecode"
+            dict_obj["BitsPerComponent"] = 1
+            params = PdfParser.PdfArray(
+                [
+                    PdfParser.PdfDict(
+                        {
+                            "K": -1,
+                            "BlackIs1": True,
+                            "Columns": width,
+                            "Rows": height,
+                        }
+                    )
+                ]
+            )
+        else:
+            filter = "DCTDecode"
+        dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceGray")
+        procset = "ImageB"  # grayscale
+    elif im.mode == "L":
+        filter = "DCTDecode"
+        # params = f"<< /Predictor 15 /Columns {width-2} >>"
+        dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceGray")
+        procset = "ImageB"  # grayscale
+    elif im.mode == "LA":
+        filter = "JPXDecode"
+        # params = f"<< /Predictor 15 /Columns {width-2} >>"
+        procset = "ImageB"  # grayscale
+        dict_obj["SMaskInData"] = 1
+    elif im.mode == "P":
+        filter = "ASCIIHexDecode"
+        palette = im.getpalette()
+        dict_obj["ColorSpace"] = [
+            PdfParser.PdfName("Indexed"),
+            PdfParser.PdfName("DeviceRGB"),
+            255,
+            PdfParser.PdfBinary(palette),
+        ]
+        procset = "ImageI"  # indexed color
+
+        if "transparency" in im.info:
+            smask = im.convert("LA").getchannel("A")
+            smask.encoderinfo = {}
+
+            image_ref = _write_image(smask, filename, existing_pdf, image_refs)[0]
+            dict_obj["SMask"] = image_ref
+    elif im.mode == "RGB":
+        filter = "DCTDecode"
+        dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceRGB")
+        procset = "ImageC"  # color images
+    elif im.mode == "RGBA":
+        filter = "JPXDecode"
+        procset = "ImageC"  # color images
+        dict_obj["SMaskInData"] = 1
+    elif im.mode == "CMYK":
+        filter = "DCTDecode"
+        dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceCMYK")
+        procset = "ImageC"  # color images
+        decode = [1, 0, 1, 0, 1, 0, 1, 0]
+    else:
+        msg = f"cannot save mode {im.mode}"
+        raise ValueError(msg)
+
+    #
+    # image
+
+    op = io.BytesIO()
+
+    if filter == "ASCIIHexDecode":
+        ImageFile._save(im, op, [("hex", (0, 0) + im.size, 0, im.mode)])
+    elif filter == "CCITTFaxDecode":
+        im.save(
+            op,
+            "TIFF",
+            compression="group4",
+            # use a single strip
+            strip_size=math.ceil(width / 8) * height,
+        )
+    elif filter == "DCTDecode":
+        Image.SAVE["JPEG"](im, op, filename)
+    elif filter == "JPXDecode":
+        del dict_obj["BitsPerComponent"]
+        Image.SAVE["JPEG2000"](im, op, filename)
+    else:
+        msg = f"unsupported PDF filter ({filter})"
+        raise ValueError(msg)
+
+    stream = op.getvalue()
+    if filter == "CCITTFaxDecode":
+        stream = stream[8:]
+        filter = PdfParser.PdfArray([PdfParser.PdfName(filter)])
+    else:
+        filter = PdfParser.PdfName(filter)
+
+    image_ref = image_refs.pop(0)
+    existing_pdf.write_obj(
+        image_ref,
+        stream=stream,
+        Type=PdfParser.PdfName("XObject"),
+        Subtype=PdfParser.PdfName("Image"),
+        Width=width,  # * 72.0 / x_resolution,
+        Height=height,  # * 72.0 / y_resolution,
+        Filter=filter,
+        Decode=decode,
+        DecodeParms=params,
+        **dict_obj,
+    )
+
+    return image_ref, procset
+
+
 def _save(im, fp, filename, save_all=False):
     is_appending = im.encoderinfo.get("append", False)
     if is_appending:
@@ -109,6 +235,9 @@ def _save(im, fp, filename, save_all=False):
         number_of_pages += im_number_of_pages
         for i in range(im_number_of_pages):
             image_refs.append(existing_pdf.next_object_id(0))
+            if im.mode == "P" and "transparency" in im.info:
+                image_refs.append(existing_pdf.next_object_id(0))
+
             page_refs.append(existing_pdf.next_object_id(0))
             contents_refs.append(existing_pdf.next_object_id(0))
             existing_pdf.pages.append(page_refs[-1])
@@ -121,123 +250,7 @@ def _save(im, fp, filename, save_all=False):
     for im_sequence in ims:
         im_pages = ImageSequence.Iterator(im_sequence) if save_all else [im_sequence]
         for im in im_pages:
-            # FIXME: Should replace ASCIIHexDecode with RunLengthDecode
-            # (packbits) or LZWDecode (tiff/lzw compression).  Note that
-            # PDF 1.2 also supports Flatedecode (zip compression).
-
-            params = None
-            decode = None
-
-            #
-            # Get image characteristics
-
-            width, height = im.size
-
-            dict_obj = {"BitsPerComponent": 8}
-            if im.mode == "1":
-                if features.check("libtiff"):
-                    filter = "CCITTFaxDecode"
-                    dict_obj["BitsPerComponent"] = 1
-                    params = PdfParser.PdfArray(
-                        [
-                            PdfParser.PdfDict(
-                                {
-                                    "K": -1,
-                                    "BlackIs1": True,
-                                    "Columns": width,
-                                    "Rows": height,
-                                }
-                            )
-                        ]
-                    )
-                else:
-                    filter = "DCTDecode"
-                dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceGray")
-                procset = "ImageB"  # grayscale
-            elif im.mode == "L":
-                filter = "DCTDecode"
-                # params = f"<< /Predictor 15 /Columns {width-2} >>"
-                dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceGray")
-                procset = "ImageB"  # grayscale
-            elif im.mode == "LA":
-                filter = "JPXDecode"
-                # params = f"<< /Predictor 15 /Columns {width-2} >>"
-                procset = "ImageB"  # grayscale
-                dict_obj["SMaskInData"] = 1
-            elif im.mode == "P":
-                filter = "ASCIIHexDecode"
-                palette = im.getpalette()
-                dict_obj["ColorSpace"] = [
-                    PdfParser.PdfName("Indexed"),
-                    PdfParser.PdfName("DeviceRGB"),
-                    255,
-                    PdfParser.PdfBinary(palette),
-                ]
-                procset = "ImageI"  # indexed color
-            elif im.mode == "RGB":
-                filter = "DCTDecode"
-                dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceRGB")
-                procset = "ImageC"  # color images
-            elif im.mode == "RGBA":
-                filter = "JPXDecode"
-                procset = "ImageC"  # color images
-                dict_obj["SMaskInData"] = 1
-            elif im.mode == "CMYK":
-                filter = "DCTDecode"
-                dict_obj["ColorSpace"] = PdfParser.PdfName("DeviceCMYK")
-                procset = "ImageC"  # color images
-                decode = [1, 0, 1, 0, 1, 0, 1, 0]
-            else:
-                msg = f"cannot save mode {im.mode}"
-                raise ValueError(msg)
-
-            #
-            # image
-
-            op = io.BytesIO()
-
-            if filter == "ASCIIHexDecode":
-                ImageFile._save(im, op, [("hex", (0, 0) + im.size, 0, im.mode)])
-            elif filter == "CCITTFaxDecode":
-                im.save(
-                    op,
-                    "TIFF",
-                    compression="group4",
-                    # use a single strip
-                    strip_size=math.ceil(im.width / 8) * im.height,
-                )
-            elif filter == "DCTDecode":
-                Image.SAVE["JPEG"](im, op, filename)
-            elif filter == "JPXDecode":
-                del dict_obj["BitsPerComponent"]
-                Image.SAVE["JPEG2000"](im, op, filename)
-            elif filter == "FlateDecode":
-                ImageFile._save(im, op, [("zip", (0, 0) + im.size, 0, im.mode)])
-            elif filter == "RunLengthDecode":
-                ImageFile._save(im, op, [("packbits", (0, 0) + im.size, 0, im.mode)])
-            else:
-                msg = f"unsupported PDF filter ({filter})"
-                raise ValueError(msg)
-
-            stream = op.getvalue()
-            if filter == "CCITTFaxDecode":
-                stream = stream[8:]
-                filter = PdfParser.PdfArray([PdfParser.PdfName(filter)])
-            else:
-                filter = PdfParser.PdfName(filter)
-
-            existing_pdf.write_obj(
-                image_refs[page_number],
-                stream=stream,
-                Type=PdfParser.PdfName("XObject"),
-                Subtype=PdfParser.PdfName("Image"),
-                Width=width,  # * 72.0 / x_resolution,
-                Height=height,  # * 72.0 / y_resolution,
-                Filter=filter,
-                Decode=decode,
-                DecodeParms=params,
-                **dict_obj,
-            )
+            image_ref, procset = _write_image(im, filename, existing_pdf, image_refs)
 
             #
             # page
@@ -246,13 +259,13 @@ def _save(im, fp, filename, save_all=False):
                 page_refs[page_number],
                 Resources=PdfParser.PdfDict(
                     ProcSet=[PdfParser.PdfName("PDF"), PdfParser.PdfName(procset)],
-                    XObject=PdfParser.PdfDict(image=image_refs[page_number]),
+                    XObject=PdfParser.PdfDict(image=image_ref),
                 ),
                 MediaBox=[
                     0,
                     0,
-                    width * 72.0 / x_resolution,
-                    height * 72.0 / y_resolution,
+                    im.width * 72.0 / x_resolution,
+                    im.height * 72.0 / y_resolution,
                 ],
                 Contents=contents_refs[page_number],
             )
@@ -261,8 +274,8 @@ def _save(im, fp, filename, save_all=False):
             # page contents
 
             page_contents = b"q %f 0 0 %f 0 0 cm /image Do Q\n" % (
-                width * 72.0 / x_resolution,
-                height * 72.0 / y_resolution,
+                im.width * 72.0 / x_resolution,
+                im.height * 72.0 / y_resolution,
             )
 
             existing_pdf.write_obj(contents_refs[page_number], stream=page_contents)


Test patch:
diff --git a/Tests/test_file_pdf.py b/Tests/test_file_pdf.py
--- a/Tests/test_file_pdf.py
+++ b/Tests/test_file_pdf.py
@@ -48,6 +48,22 @@ def test_save_alpha(tmp_path, mode):
     helper_save_as_pdf(tmp_path, mode)
 
 
+def test_p_alpha(tmp_path):
+    # Arrange
+    outfile = str(tmp_path / "temp.pdf")
+    with Image.open("Tests/images/pil123p.png") as im:
+        assert im.mode == "P"
+        assert isinstance(im.info["transparency"], bytes)
+
+        # Act
+        im.save(outfile)
+
+    # Assert
+    with open(outfile, "rb") as fp:
+        contents = fp.read()
+    assert b"\n/SMask " in contents
+
+
 def test_monochrome(tmp_path):
     # Arrange
     mode = "1"