Repository: benoitc/gunicorn

Issue:
Gunicorn 20.0.0: find_library('c') doesn't work in Alpine Linux
I was trying to run gunicorn 20.0.0 inside an Alpine Linux docker container (python:3.8-alpine).
Gunicorn 19.9.0 works just fine on the same docker image.

After installing `musl`, the error `libc not found` is still present.

Here's a block of code extracted from `gunicorn/socketfromfd.py` that doesn't work in Alpine:

test.py
```python
import ctypes
from ctypes.util import find_library

_libc_name = find_library('c')
if _libc_name is not None:
    libc = ctypes.CDLL(_libc_name, use_errno=True)
else:
    raise OSError('libc not found')
```

Dockerfile
```dockerfile
FROM python:3.8-alpine

WORKDIR /app

RUN apk add musl

COPY . /app

CMD ["python", "test.py"]
```


Discussion:
Right. We should probably test **if python=< 3.7** to use `socket.from_fd` in https://github.com/benoitc/gunicorn/blob/438371ee90b9676336a44c7abaeb30ee7fc57a5c/gunicorn/socketfromfd.py#L65
First, thanks for a great piece of software!
Second, I can confirm this issue; it has occurred in every installation of 20.0.0 on Alpine.
> Right. We should probably test **if python=< 3.7** to use `socket.from_fd` in
> 
> https://github.com/benoitc/gunicorn/blob/438371ee90b9676336a44c7abaeb30ee7fc57a5c/gunicorn/socketfromfd.py#L65

Can confirm that the issue isn't related to a specific Python version, have it happening in 3.6,3.7,3.8 so long as it's in Alpine.
 It's a side effect of using an operating system without all the basic libs in a container unfortunately. Maybe you could use an image using a debian? or an alpine image coming with a libc?

This api is there is pre-3.7 to replace a lack of API for it in Python. Detecting if the api is available (for python >= 3.7) would at least solve the issue for these versions. Before that we need to investigate what could be done.
> It's a side effect of using an operating system without all the basic libs in a container unfortunately. Maybe you could use an image using a debian? or an alpine image coming with a libc?
> 
> This api is there is pre-3.7 to replace a lack of API for it in Python. Detecting if the api is available (for python >= 3.7) would at least solve the issue for these versions. Before that we need to investigate what could be done.

Alpine uses musl libc in place of glibc as part of it being based on busybox, of course using other base images would be a workaround but it will limit gunicorn uses as both alpine & other minimal OS that are growing in popularity (which are often based on it\busybox\etc) all have musl instead of glibc so the affect is likely not limited to alpine (just the most popular one affected).

Sounds to me that something like detecting if musl is installed instead of glibc & if so use whatever solution was in place in 19.9.x is a potential fix that isn't python version specific as we know 19.9.x works well.
found a related issue in docker-library/python#111

maybe this code can be reused in a way: https://github.com/python/cpython/pull/10453/commits/e3f67780aab24401a50af64e688d38c24ee41ad0

There is one possible workaround for that, as twisted users encountered the same:
https://stackoverflow.com/questions/48234723/twisted-server-monkey-patch-file
That piece of code from `gunicorn/socketfromfd.py` could be rewritten as following:
```
_libc_name = find_library('c')
if _libc_name is None:
    _libc_name = 'libc.so.6'
libc = ctypes.CDLL(_libc_name, use_errno=True)
```
Just one of the possible solutions,but probably not the best one.
> maybe this code can be reused in a way: python/cpython@e3f6778

I think that's this issue: https://bugs.python.org/issue21622

Let's find a workaround and make a patch release.
@tilgovi the patch I linked above does the trick (and is linked in the bug ticket you added). I will work on a PR that use it.
@aljinovic can you try fda61b5479646f68faef30f3cb034ce16f3c61b6 . It should fix your issue. Let me know :)
I tried it and it doesn't work. I found a bug in your implementation, here's a proposal of the fix:

```python
for suffix in ['so', '*.so.*']:
    for f in glob('{0}.{1}'.format(prefix, suffix)):
```
hrm odd, can you paste me the error you had?
Your code: `find_library("c")` returns `None`.
After my change it returns: `libc.musl-x86_64.so.1`
Also, I had to manually set `LD_LIBRARY_PATH` first, so maybe this could help:

```python
paths = ['/lib', '/usr/local/lib', '/usr/lib']
if 'LD_LIBRARY_PATH' in os.environ:
    paths = os.environ['LD_LIBRARY_PATH'].split(':') + paths
```

Taken from:
https://github.com/alpinelinux/aports/blob/202f4bea916b0cf974b38ced96ab8fca0b192e3f/main/python2/musl-find_library.patch
ok fixing that, not sure about why the glob is nlot working the first way though. I need to test
@aljinovic should be OK now, let me know
Still doesn't work unless I change from:

```python
for suffix in ['so', 'so.*']:
```

to:

```python
for suffix in ['so', '*.so.*']:
```

The actual `libc` name is: `libc.musl-x86_64.so.1`
sorry I missed the additional star.
https://github.com/benoitc/gunicorn/commit/e150ca4ff87d52b1c2b3d5376fc71789b65a3fb4
should fix it now.

On Thu, Nov 21, 2019 at 11:47 AM Ante Aljinovic <notifications@github.com>
wrote:

> Still doesn't work unless I change from:
>
> for suffix in ['so', 'so.*']:
>
> to:
>
> for suffix in ['so', '*.so.*']:
>
> The actual libc name is: libc.musl-x86_64.so.1
>
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly, view it on GitHub
> <https://github.com/benoitc/gunicorn/issues/2160?email_source=notifications&email_token=AAADRISHZX5Q265IJQ5OCKLQUZRMJA5CNFSM4JLJUUP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEZZJPY#issuecomment-557028543>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAADRIXWUCJLU6ZCD2R2Z6DQUZRMJANCNFSM4JLJUUPQ>
> .
>

Great, that worked, thank you!
Nice work everyone!
can you all test #2208 as i changed the implementation to simply use the python standard library. Any feedback is welcome :)

Resolving patch:
diff --git a/gunicorn/util.py b/gunicorn/util.py
--- a/gunicorn/util.py
+++ b/gunicorn/util.py
@@ -3,7 +3,6 @@
 # This file is part of gunicorn released under the MIT license.
 # See the NOTICE for more information.
 import ast
-import ctypes.util
 import email.utils
 import errno
 import fcntl
@@ -638,34 +637,3 @@ def bytes_to_str(b):
 
 def unquote_to_wsgi_str(string):
     return urllib.parse.unquote_to_bytes(string).decode('latin-1')
-
-
-def _findWalk_ldpath(name):
-    def _is_elf(filepath):
-        try:
-            with open(filepath, 'rb') as fh:
-                return fh.read(4) == b'\x7fELF'
-        except:
-            return False
-    from glob import glob
-    if os.path.isabs(name):
-        return name
-
-    # search LD_LIBRARY_PATH list
-    paths = ['/lib', '/usr/local/lib', '/usr/lib']
-    if 'LD_LIBRARY_PATH' in os.environ:
-        paths = os.environ['LD_LIBRARY_PATH'].split(':') + paths
-
-    for d in paths:
-        f = os.path.join(d, name)
-        if _is_elf(f):
-            return os.path.basename(f)
-        prefix = os.path.join(d, 'lib'+name)
-        for suffix in ['so', 'so.*', '*.so.*', 'a']:
-            for f in glob('{0}.{1}'.format(prefix, suffix)):
-                if _is_elf(f) or suffix == 'a':
-                    return os.path.basename(f)
-
-
-def find_library(name):
-    return ctypes.util.find_library(name) or _findWalk_ldpath(name)


Test patch: