Repository: matrix-org/synapse

Issue:
Unable to build debian packages for debian buster with Pillow 7.1.0
<!--

**IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**:
You will likely get better support more quickly if you ask in ** #synapse:matrix.org ** ;)


This is a bug report template. By following the instructions below and
filling out the sections with your information, you will help the us to get all
the necessary data to fix your issue.

You can also preview your report before submitting it. You may remove sections
that aren't relevant to your particular case.

Text between <!-- and --​> marks will be invisible in the report.

-->

### Description

When trying to build debian packages for Synapse v1.12.1, we ran into a snag with the following error on `debian:buster`:

```
find debian/`dh_listpackages`/ -path '*/site-packages/psycopg2/*.so' | \
    xargs dpkg-shlibdeps -Tdebian/`dh_listpackages`.substvars \
        -pshlibs1 -dRecommends
dh_shlibdeps -X site-packages/PIL/.libs -X site-packages/psycopg2
dpkg-shlibdeps: error: cannot find library libz-a147dcb0.so.1.2.3 needed by debian/matrix-synapse-py3/opt/venvs/matrix-synapse/lib/python3.7/site-packages/Pillow.libs/libpng16-bedcb7ea.so.16.37.0 (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot continue due to the error above
```

This appears to be the same error as python-pillow/pillow-wheels#255, though it's unclear why this issue is only cropping up now.

Pillow did some PNG-related work in release 7.1.0, so that may be the cause. Pinning pillow to <v7.1.0 seems to work, so that's what we've done for now.


Discussion:
Aaaaand unfortunately this workaround has prevented Synapse from starting after upgrading to v1.12.2 with Pillow 7.1.0 installed.

The plan for now is to have a v1.12.3 release with a fixed workaround. This will likely arrive tomorrow.

Note that if you can't downgrade from v1.12.2 for whatever reason, you can downgrade Pillow to v7.0.0 and then Synapse will start:

```
pip install --upgrade --force pillow==7.0.0
```

If you would like to downgrade instead, `v1.12.1` is known working.
So the problem is that Pillow 7.1.0 has changed where it installs its libraries. For 7.0.0, they are installed to `site-packages/PIL/.libs`: 

```
$ ls env2/lib/python3.6/site-packages/PIL/.libs
libfreetype-69f25d5e.so.6.17.1  liblzma-6cd627ed.so.5.2.4     libtiff-bd1961ca.so.5.5.0       libwebpmux-40630b44.so.3.0.4
libjpeg-3b10b538.so.9.3.0       libopenjp2-b3d7668a.so.2.3.1  libwebp-3a2aeecf.so.7.0.5       libz-a147dcb0.so.1.2.3
liblcms2-a6801db4.so.2.0.8      libpng16-bedcb7ea.so.16.37.0  libwebpdemux-e9ec482e.so.2.0.6
```

But for 7.1.1:

```
$ ls env/lib/python3.6/site-packages/Pillow.libs/
libfreetype-69f25d5e.so.6.17.1  liblzma-99449165.so.5.2.5     libtiff-41910f6d.so.5.5.0       libwebpmux-ec1d5c76.so.3.0.5
libjpeg-ba7bf5af.so.9.4.0       libopenjp2-b3d7668a.so.2.3.1  libwebp-122bd20b.so.7.1.0       libz-a147dcb0.so.1.2.3
liblcms2-a6801db4.so.2.0.8      libpng16-bedcb7ea.so.16.37.0  libwebpdemux-2db559e5.so.2.0.6
```

Unfortunately we've hardcoded this path in the debian build scripts: https://github.com/matrix-org/synapse/blob/v1.12.1/debian/rules#L25-L28.

I'll make a PR to fix it.

Resolving patch:
diff --git a/synapse/python_dependencies.py b/synapse/python_dependencies.py
--- a/synapse/python_dependencies.py
+++ b/synapse/python_dependencies.py
@@ -61,9 +61,7 @@
     "pyasn1-modules>=0.0.7",
     "daemonize>=2.3.1",
     "bcrypt>=3.1.0",
-    # Pillow 7.1.0 causes the following issue on debian buster:
-    # https://github.com/python-pillow/Pillow/issues/2377
-    "pillow>=4.3.0,<7.1.0",
+    "pillow>=4.3.0",
     "sortedcontainers>=1.4.4",
     "pymacaroons>=0.13.0",
     "msgpack>=0.5.2",


Test patch: