Repository: python-pillow/Pillow

Issue:
Memory leaks in Image.close() after saving as pdf or png
<!--
Thank you for reporting an issue.

Follow these guidelines to ensure your issue is handled properly.

If you have a ...

1.  General question: consider asking the question on Stack Overflow
    with the python-imaging-library tag:

    * https://stackoverflow.com/questions/tagged/python-imaging-library

    Do not ask a question in both places.

    If you think you have found a bug or have an unexplained exception
    then file a bug report here.

2.  Bug report: include a self-contained, copy-pastable example that
    generates the issue if possible. Be concise with code posted.
    Guidelines on how to provide a good bug report:

    * https://stackoverflow.com/help/mcve

    Bug reports which follow these guidelines are easier to diagnose,
    and are often handled much more quickly.

3.  Feature request: do a quick search of existing issues
    to make sure this has not been asked before.

We know asking good questions takes effort, and we appreciate your time.
Thank you.
-->

### What did you do?

I'm converting 16 PNG images (with resolution 3000x4000) to PDF file and since Pillow didn't close the images after converting, it can cause huge memory. So i modified it's code from https://github.com/python-pillow/Pillow/blob/9.2.0/src/PIL/PdfImagePlugin.py#L218-L227, it looks like this

```python
            existing_pdf.write_obj(contents_refs[page_number], stream=page_contents)

            page_number += 1

        # Close image after converting to save some memory
        im_sequence.close() # <--- This is the modified code

    #
    # trailer
    existing_pdf.write_xref_and_trailer()
    if hasattr(fp, "flush"):
        fp.flush()
    existing_pdf.close()
```

### What did you expect to happen?

The memory stay low during converting

### What actually happened?

Usually it worked for Pillow 9.0.1 and the memory stays between 20-60 MB. But after installing version 9.2.0, 9.1.0, and 9.1.1 it increased up to 700+ MB

Take a look at this footages

<details>
<summary>Pillow 9.0.1</summary>

https://user-images.githubusercontent.com/43638783/179394112-0aa70a49-b7e1-42e5-8dda-0306adc6d180.mp4

</details>

<details>
<summary>Pillow 9.1.0</summary>

https://user-images.githubusercontent.com/43638783/179394412-abc0d24f-f09a-414e-8b5c-f542c558ba75.mp4

</details>

<details>
<summary>Pillow 9.1.1</summary>

https://user-images.githubusercontent.com/43638783/179394649-84cdaaa4-85a8-41c5-ad15-c2c0198fe045.mp4

</details>

<details>
<summary>Pillow 9.2.0</summary>

https://user-images.githubusercontent.com/43638783/179395101-d9a77ae8-dc18-4e23-a7f0-b14064f85f2b.mp4

</details>



### What are your OS, Python and Pillow versions?

* OS: Windows 10 Home Single Language 64-bit (10.0, Build 19044)
* Python: 3.10.2 64-bit
* Pillow: 9.2.0

<!--
Please include **code** that reproduces the issue and whenever possible, an **image** that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as Plone, Django, or Buildout, try to replicate the issue just using Pillow.
-->

```python
import time
from pathlib import Path
from glob import glob
from PIL import Image

base_path = Path('./test_images')
pdf_file = Path('test.pdf')

if pdf_file.exists():
    pdf_file.unlink()

raw_files = glob('*', root_dir=base_path)

files = [Image.open(base_path / i) for i in raw_files]

im = files.pop(0)

print('Converting')
im.save(pdf_file, save_all=True, append_images=files)
print('Done converting')

time.sleep(9000)
```



Discussion:
Could we have copies of the images? It would be helpful to remove as much ambiguity as possible from your example.

You changed the title to "Memory leaks in PIL.PngImagePlugin.PngImageFile.close". So you believe that this problem can be replicated without saving as PDFs and without using your Pillow modification?
If you're concerned about memory when saving PDFs, you might also be interested in [this method that uses less memory.](https://github.com/python-pillow/Pillow/issues/4067#issuecomment-531517647) If that works, it would also remove the need for your Pillow modification.
> Could we have copies of the images? It would be helpful to remove as much ambiguity as possible from your example.

Unfortunately, i cannot send you the images in this issue publicly, because of copyright reason. Is there a way that i can send images privately ?

> You changed the title to "Memory leaks in PIL.PngImagePlugin.PngImageFile.close". So you believe that this problem can be replicated without saving as PDFs and without using your Pillow modification?

I assume that variable `im_sequence` is `PIL.PngImagePlugin.PngImageFile`. And no, the problem only happened in low-level function `_save()` in `PdfImagePlugin.py.
> If you're concerned about memory when saving PDFs, you might also be interested in https://github.com/python-pillow/Pillow/issues/4067#issuecomment-531517647 If that works, it would also remove the need for your Pillow modification.

I have tried this before. In my case, converting thousand of images to PDF file in single run can cause slow performance. I have looked at Pillow module and found that https://github.com/python-pillow/Pillow/blob/9.2.0/src/PIL/PdfImagePlugin.py#L112 is causing slow, because the PDF plugin is trying to load the whole contents of PDF file. So modifying Pillow module is best option so far.
I cannot send you the original images that i used for tests, but this should do.

<img src="https://upload.wikimedia.org/wikipedia/commons/4/49/Lattinaichnusa.png" width="300" height="300">

Source: https://commons.wikimedia.org/wiki/File:Lattinaichnusa.png

After testing further with every extensions available (`PIL.Image.EXTENSION`). This is the result i found.

<details> <summary>Image i used</summary>

![image](https://user-images.githubusercontent.com/43638783/180258829-86c65978-7f53-4946-bd98-f1daa746772a.png)

</details>

Not affected

- BMP
- DIB
- GIF
- TIFF
- JPEG
- PPM
- PCX
- DDS
- EPS
- JPEG2000
- ICNS
- ICO
- IM
- MPO
- SGI
- TGA
- WEBP

Failed

- BLP (ValueError: Unsupported BLP image mode)
- BUFR (OSError: BUFR save handler not installed)
- CUR (KeyError: 'CUR')
- DCX (KeyError: 'DCX')
- FITS (OSError: FITS save handler not installed)
- FLI (KeyError: 'FLI')
- FTEX (KeyError: 'FTEX')
- GBR (KeyError: 'GBR')
- GRIB (OSError: GRIB save handler not installed)
- HDF5 (OSError: HDF5 save handler not installed)
- IPTC (KeyError: 'IPTC')
- MPEG (KeyError: 'MPEG')
- MSP (OSError: cannot write mode RGB as MSP)
- PALM (OSError: cannot write mode RGB as Palm)
- PCD (KeyError: 'PCD')
- PIXAR (KeyError: 'PIXAR')
- PSD (KeyError: 'PSD')
- SUN (KeyError: 'SUN')
- WMF (OSError: WMF save handler not installed)
- XBM (OSError: cannot write mode RGB as XBM)
- XPM (KeyError: 'XPM')

Affected

- PDF
- PNG

Resolving patch:
diff --git a/src/PIL/ImageFile.py b/src/PIL/ImageFile.py
--- a/src/PIL/ImageFile.py
+++ b/src/PIL/ImageFile.py
@@ -499,9 +499,14 @@ def _save(im, fp, tile, bufsize=0):
     try:
         fh = fp.fileno()
         fp.flush()
-        exc = None
-    except (AttributeError, io.UnsupportedOperation) as e:
-        exc = e
+        _encode_tile(im, fp, tile, bufsize, fh)
+    except (AttributeError, io.UnsupportedOperation) as exc:
+        _encode_tile(im, fp, tile, bufsize, None, exc)
+    if hasattr(fp, "flush"):
+        fp.flush()
+
+
+def _encode_tile(im, fp, tile, bufsize, fh, exc=None):
     for e, b, o, a in tile:
         if o > 0:
             fp.seek(o)
@@ -526,8 +531,6 @@ def _save(im, fp, tile, bufsize=0):
                 raise OSError(f"encoder error {s} when writing image file") from exc
         finally:
             encoder.cleanup()
-    if hasattr(fp, "flush"):
-        fp.flush()
 
 
 def _safe_read(fp, size):


Test patch: