internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: benoitc/gunicorn

Issue:
Access log not emitted when using `logconfig_dict`
Using the unreleased version from `master`, HTTP requests do not create log records in the logger `gunicorn.access` when using the new `logconfig_dict`.

See relevant snippet from `glogging.py`:

```python
    def access(self, resp, req, environ, request_time):
        """ See http://httpd.apache.org/docs/2.0/logs.html#combined
        for format details
        """

        if not (self.cfg.accesslog or self.cfg.logconfig or
           (self.cfg.syslog and not self.cfg.disable_access_log_redirection)):
            return
```


Discussion:
Good catch, would you like to send a PR?

Resolving patch:
diff --git a/gunicorn/glogging.py b/gunicorn/glogging.py
--- a/gunicorn/glogging.py
+++ b/gunicorn/glogging.py
@@ -339,6 +339,7 @@ def access(self, resp, req, environ, request_time):
         """
 
         if not (self.cfg.accesslog or self.cfg.logconfig or
+           self.cfg.logconfig_dict or
            (self.cfg.syslog and not self.cfg.disable_access_log_redirection)):
             return
 


Test patch:
Repository: benoitc/gunicorn

Issue:
glogging._get_user() fails to catch malformed authorization header
I have a client that abuses the HTTP authorization header, passing "Basic (token)" where token is a random alphanumeric string. This causes the gunicorn logger to choke because, although it is a valid base64 string of data, it does not represent base64 encoded utf-8 string.

```
[2018-01-15 18:23:06 +0000] [15] [ERROR] Error handling request
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/glogging.py", line 269, in 
    'u': self._get_user(environ) or '-',
  File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/glogging.py", line 446, in 
    auth = auth.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 0: invalid start byte
```

You can reproduce it with the following line:
```base64.b64decode("ixsTtkKzIpVTncfQjbBcnoRNoDfbnaXG".encode('utf-8')).decode('utf-8')```

I don't expect _get_user() to work properly in this case but it would be good for it to catch the error. Currently the code is in a try/catch but it is only handling TypeError, and binascii.Error, it should also catch and handle UnicodeDecodeError. I'm working on a PR now, should be ready shortly.


Discussion:

Resolving patch:
diff --git a/gunicorn/glogging.py b/gunicorn/glogging.py
--- a/gunicorn/glogging.py
+++ b/gunicorn/glogging.py
@@ -471,10 +471,7 @@ def _get_user(self, environ):
                     if PY3:  # b64decode returns a byte string in Python 3
                         auth = auth.decode('utf-8')
                     auth = auth.split(":", 1)
-                except TypeError as exc:
-                    self.debug("Couldn't get username: %s", exc)
-                    return user
-                except binascii.Error as exc:
+                except (TypeError, binascii.Error, UnicodeDecodeError) as exc:
                     self.debug("Couldn't get username: %s", exc)
                     return user
                 if len(auth) == 2:


Test patch:
diff --git a/tests/test_logger.py b/tests/test_logger.py
--- a/tests/test_logger.py
+++ b/tests/test_logger.py
@@ -46,3 +46,22 @@ def test_get_username_from_basic_auth_header():
     logger = Logger(Config())
     atoms = logger.atoms(response, request, environ, datetime.timedelta(seconds=1))
     assert atoms['u'] == 'brk0v'
+
+
+def test_get_username_handles_malformed_basic_auth_header():
+    """Should catch a malformed auth header"""
+    request = SimpleNamespace(headers=())
+    response = SimpleNamespace(
+        status='200', response_length=1024, sent=1024,
+        headers=(('Content-Type', 'text/plain'),),
+    )
+    environ = {
+        'REQUEST_METHOD': 'GET', 'RAW_URI': '/my/path?foo=bar',
+        'PATH_INFO': '/my/path', 'QUERY_STRING': 'foo=bar',
+        'SERVER_PROTOCOL': 'HTTP/1.1',
+        'HTTP_AUTHORIZATION': 'Basic ixsTtkKzIpVTncfQjbBcnoRNoDfbnaXG',
+    }
+    logger = Logger(Config())
+
+    atoms = logger.atoms(response, request, environ, datetime.timedelta(seconds=1))
+    assert atoms['u'] == '-'
Repository: benoitc/gunicorn

Issue:
"Connection refused" when using keep-alive with gthread
Gunicorn version: 19.7.1 (also tried master branch)

This bug could be reproduced with attached files:
- test_http_gunicorn_raw_keep-alive-requests.py (test runner)
- myapp.py (gunicorn app that should be in the same directory)

Test case starts command:
```
gunicorn --worker-class gthread --workers 1 --threads 20 --keep-alive 9999 \
           --log-level DEBUG --bind localhost:7777 myapp:app
```
with logs redirected to `/tmp/_test_gunicorn.out`
and then create separate threads to open http session and send 1000 requests within it.

In my case gunicorn server received request `/request/1096` and then reset connection (see `tcp.stream eq 10` from attached `gunicorn_reset_keep_alived_connection.pcapng`

NOTE: this is race condition so it may happen that all requests finish with success (see myapp.py -> without `time.sleep` it almost never fails) but with that `time.sleep` it usually fails and then script should output line similar to:
`[1] Failed request id=1096 with ('Connection aborted.', error(104, 'Connection reset by peer'))`
it mean that thread sent request `GET /request/1096` and received `Connection reset by peer` (so the last one request from that session that succeed is `GET /request/1095`).


Discussion:

[gunicorn_bug.zip](https://github.com/benoitc/gunicorn/files/1699216/gunicorn_bug.zip)



Resolving patch:
diff --git a/gunicorn/workers/gthread.py b/gunicorn/workers/gthread.py
--- a/gunicorn/workers/gthread.py
+++ b/gunicorn/workers/gthread.py
@@ -74,11 +74,6 @@ def set_timeout(self):
     def close(self):
         util.close(self.sock)
 
-    def __lt__(self, other):
-        return self.timeout < other.timeout
-
-    __cmp__ = __lt__
-
 
 class ThreadWorker(base.Worker):
 


Test patch:
Repository: benoitc/gunicorn

Issue:
gunicorn crashed on start with --reload flag
Setup: Vagrant, virtualenv, gunicorn 19.3.0:

The following command produces this stack:

`gunicorn -c /data/shared/api/gunicorn_config.py -b unix:/tmp/api-dev-gunicorn.sock --log-level INFO --reload  wsgi:app`

```
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/vagrant/.pyenv/versions/2.7.6/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/data/virtualenv/default/lib/python2.7/site-packages/gunicorn/reloader.py", line 41, in run
    for filename in self.get_files():
  File "/data/virtualenv/default/lib/python2.7/site-packages/gunicorn/reloader.py", line 30, in get_files
    if hasattr(module, '__file__')
  File "/data/virtualenv/default/lib/python2.7/re.py", line 151, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer
```

If I remove --reload it boots up fine.



Discussion:
can you paste your config file (`gunicorn_config.py`) ?

```
import multiprocessing
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'sync'
max_requests = 0
workers = 1
```

@Schabernack can you increase the log level to debug? Maybe the trace will be more interesting there. It seems that `module.__file__` doesn't return a string there but I don't know why yet. Does one of your modules rewrite it?

@Schabernack bump.

Sorry Benoit, I fixed this by reinstalling gunicorn and forgot about the ticket. This can be closed.

@Schabernack np, thanks!

I got this issue as well. I'm using docker. Removing --reload was the fix
I'm seeing this problem.
```
Python 3.6.4+ (default, Feb 12 2018, 08:25:03) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> [module for module in sys.modules.values() if hasattr(module, "__file__") and module.__file__ is None]
[<module 'keyrings' (namespace)>, <module 'logilab' (namespace)>, <module 'ruamel' (namespace)>]
>>> 
```

Resolving patch:
diff --git a/gunicorn/reloader.py b/gunicorn/reloader.py
--- a/gunicorn/reloader.py
+++ b/gunicorn/reloader.py
@@ -28,7 +28,7 @@ def get_files(self):
         fnames = [
             re.sub('py[co]$', 'py', module.__file__)
             for module in list(sys.modules.values())
-            if hasattr(module, '__file__')
+            if getattr(module, '__file__', None)
         ]
 
         with self._extra_files_lock:


Test patch:
Repository: benoitc/gunicorn

Issue:
Maybe a bug of reopen_files when handle signal USR1 ？
I am sorry I don't know how to discuss on mailing list. 

#### phenomenon:
when gunicorn received signal USR1, my flask app's logger will write log to another file, which is unexpected. 

#### reason:
my logger in flask app use handler implemented by myself, and my handler does not open file of handler.baseFilename. when gunicorn worker handle USR1,  it will search all logger in that gunicorn worker process and reopen the handler's stream by `open(handler.baseFilename)`. 

#### related code:
```
# loggers method core logic
root = logging.root
existing = root.manager.loggerDict.keys()
return [logging.getLogger(name) for name in existing]

# reopen_files method core logic
for log in loggers():
    for handler in log.handlers:
        if isinstance(handler, logging.FileHandler):
            if handler.stream:
                handler.stream.close()
                handler.stream = open(handler.baseFilename,  handler.mode)
```

#### my idea:
To be honest, I do not understand why gunicorn need reopen **all logger** in a python process , shouldn’t it pick out only gunicorn's logger ？however, if it's really required,  the reopen_files logic must be safe or do not have effect to other logger at least. 
Many people use their own logger class or handler,  they may have different implementation, but, they must overwrite same name method in python logging module. If gunicorn want to reopen all logger, it can use FileHandler's method `close` and `_open`(see [here](https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py), line 898 )to support different custom logger or handler.

I am happy to submit a pr to optimize here  :)


Discussion:
> To be honest, I do not understand why gunicorn need reopen all logger in a python process , shouldn’t it pick out only gunicorn's logger?

I think I agree with that. That would be a small breaking change, so I think I like your other idea. Let's use `close` and `_open`. Using a `_` method doesn't feel any worse than using two undocumented properties, and at least it can be overridden.
I'd review a PR for this.
@tilgovi thanks for your comment. I will prepare a pr as quickly. 
and I want to know, in your opinion, what is purpose of gunicorn need to process all logger (except root logger, of course) ?

Resolving patch:
diff --git a/gunicorn/glogging.py b/gunicorn/glogging.py
--- a/gunicorn/glogging.py
+++ b/gunicorn/glogging.py
@@ -377,9 +377,8 @@ def reopen_files(self):
                     handler.acquire()
                     try:
                         if handler.stream:
-                            handler.stream.close()
-                            handler.stream = open(handler.baseFilename,
-                                    handler.mode)
+                            handler.close()
+                            handler.stream = handler._open()
                     finally:
                         handler.release()
 


Test patch:
Repository: benoitc/gunicorn

Issue:
I get error in this package AttributeError: 'NoneType' object has no attribute 'add_extra_file'
hi every one ..
when i try to deploy keras model into google cloud i get this error ...

```py
File "/home/falahgs07/keras/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/home/falahgs07/keras/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 148, in load_wsgi
    self.reloader.add_extra_file(exc_val.filename)
AttributeError: 'NoneType' object has no attribute 'add_extra_file'
```


Discussion:

Resolving patch:
diff --git a/gunicorn/workers/base.py b/gunicorn/workers/base.py
--- a/gunicorn/workers/base.py
+++ b/gunicorn/workers/base.py
@@ -137,7 +137,7 @@ def load_wsgi(self):
         try:
             self.wsgi = self.app.wsgi()
         except SyntaxError as e:
-            if self.cfg.reload == 'off':
+            if not self.cfg.reload:
                 raise
 
             self.log.exception(e)


Test patch:
Repository: benoitc/gunicorn

Issue:
Must explicitly define `setuptools` as a dependency
When running gunicorn in a hardened Python docker image (with most of the dependencies removed) `setuptools` might be missing.

For instance:

```
Traceback (most recent call last):
  File "/app/manage-docker.binary.runfiles/__main__/server.py", line 1, in <module>
    from gunicorn.app.base import BaseApplication
  File "/app/manage-docker.binary.runfiles/pypi__gunicorn_19_7_1/gunicorn/app/base.py", line 12, in <module>
    from gunicorn import util
  File "/app/manage-docker.binary.runfiles/pypi__gunicorn_19_7_1/gunicorn/util.py", line 12, in <module>
    import pkg_resources
ImportError: No module named pkg_resources
```

Can be fixed by defining `setuptools` as a direct dependency within the project' `requirements.txt` file, however, it could be fix at the gunicorn codebase level by using `install_requires = ['setuptools']` in setup.py. 


Discussion:
@benoitc if I remember correctly, you were against defining dependencies in `install_requires`. Do you still against the idea or can we now use `install_requires` since the Python packaging ecosystem is in a much better shape?

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -65,6 +65,14 @@ def run_tests(self):
         sys.exit(errno)
 
 
+install_requires = [
+    # We depend on functioning pkg_resources.working_set.add_entry() and
+    # pkg_resources.load_entry_point(). These both work as of 3.0 which
+    # is the first version to support Python 3.4 which we require as a
+    # floor.
+    'setuptools>=3.0',
+]
+
 extra_require = {
     'gevent':  ['gevent>=0.13'],
     'eventlet': ['eventlet>=0.9.7'],
@@ -84,6 +92,7 @@ def run_tests(self):
     url='http://gunicorn.org',
 
     python_requires='>=3.4',
+    install_requires=install_requires,
     classifiers=CLASSIFIERS,
     zip_safe=False,
     packages=find_packages(exclude=['examples', 'tests']),


Test patch:
Repository: benoitc/gunicorn

Issue:
Workers restart with --max-requests=0 when --max-requests-jitter is set to nonzero value
To reproduce, start gunicorn like this:
```
gunicorn -w 1 --max-requests=0 --max-requests-jitter=10 -b 0.0.0.0:8000 api:app
```

Then direct some traffic to it and observe the following log output:
```
[2019-02-05 20:27:23 +0000] [19] [INFO] Starting gunicorn 19.9.0
[2019-02-05 20:27:23 +0000] [19] [INFO] Listening at: http://0.0.0.0:8000 (19)
[2019-02-05 20:27:23 +0000] [19] [INFO] Using worker: sync
[2019-02-05 20:27:23 +0000] [22] [INFO] Booting worker with pid: 22
[2019-02-05 20:27:37 +0000] [22] [INFO] Autorestarting worker after current request.
[2019-02-05 20:27:37 +0000] [22] [INFO] Worker exiting (pid: 22)
[2019-02-05 20:27:37 +0000] [24] [INFO] Booting worker with pid: 24
```

I expect that `--max-requests-jitter` has no effect when `--max-requests` is set to 0.


Discussion:
What is the behavior you're expecting? Normally, `max-requests=0` should mean that workers never auto-restart but from your log it looks as though it does.

Resolving patch:
diff --git a/gunicorn/workers/base.py b/gunicorn/workers/base.py
--- a/gunicorn/workers/base.py
+++ b/gunicorn/workers/base.py
@@ -51,8 +51,13 @@ def __init__(self, age, ppid, sockets, app, timeout, cfg, log):
         self.reloader = None
 
         self.nr = 0
-        jitter = randint(0, cfg.max_requests_jitter)
-        self.max_requests = cfg.max_requests + jitter or sys.maxsize
+
+        if cfg.max_requests > 0:
+            jitter = randint(0, cfg.max_requests_jitter)
+            self.max_requests = cfg.max_requests + jitter
+        else:
+            self.max_requests = sys.maxsize
+
         self.alive = True
         self.log = log
         self.tmp = WorkerTmp(cfg)


Test patch:
Repository: benoitc/gunicorn

Issue:
Worker does not pick up proper files to watch for reloading when rebooting
When launching Gunicorn with the `--reload` option, workers will start the Reloader and then load the wsgi app. This is problematic because user files are loaded when the wsgi app is loaded, and the reloader depends on the user files to be loaded so that it can watch them.

So when the Reloader starts, it isn't watching any user files. This isn't an issue when using the default polling reloader as it updates what files it is watching every time it polls but this is an issue with the inotify reloader.




Discussion:

Resolving patch:
diff --git a/gunicorn/reloader.py b/gunicorn/reloader.py
--- a/gunicorn/reloader.py
+++ b/gunicorn/reloader.py
@@ -19,13 +19,11 @@ def __init__(self, extra_files=None, interval=1, callback=None):
         super().__init__()
         self.setDaemon(True)
         self._extra_files = set(extra_files or ())
-        self._extra_files_lock = threading.RLock()
         self._interval = interval
         self._callback = callback
 
     def add_extra_file(self, filename):
-        with self._extra_files_lock:
-            self._extra_files.add(filename)
+        self._extra_files.add(filename)
 
     def get_files(self):
         fnames = [
@@ -34,8 +32,7 @@ def get_files(self):
             if getattr(module, '__file__', None)
         ]
 
-        with self._extra_files_lock:
-            fnames.extend(self._extra_files)
+        fnames.extend(self._extra_files)
 
         return fnames
 
diff --git a/gunicorn/workers/base.py b/gunicorn/workers/base.py
--- a/gunicorn/workers/base.py
+++ b/gunicorn/workers/base.py
@@ -116,6 +116,8 @@ def init_process(self):
 
         self.init_signals()
 
+        self.load_wsgi()
+
         # start the reloader
         if self.cfg.reload:
             def changed(fname):
@@ -130,7 +132,6 @@ def changed(fname):
                                          callback=changed)
             self.reloader.start()
 
-        self.load_wsgi()
         self.cfg.post_worker_init(self)
 
         # Enter main run loop


Test patch:
Repository: benoitc/gunicorn

Issue:
gunicorn fails on IBM i with Python <3.7
This error, I believe, is introduced by this new feature: https://github.com/benoitc/gunicorn/commit/ee7af1247b8cc289af7b4bcfd2ff93ecfaf8c9df

Starting with gunicorn v20, it fails to start on the IBM i platform. Even a simple `gunicorn -h` fails with:
```
Traceback (most recent call last):
  File "/QOpenSys/pkgs/bin/gunicorn", line 5, in <module>
    from gunicorn.app.wsgiapp import run
  File "/QOpenSys/pkgs/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 9, in <module>
    from gunicorn.app.base import Application
  File "/QOpenSys/pkgs/lib/python3.6/site-packages/gunicorn/app/base.py", line 12, in <module>
    from gunicorn.arbiter import Arbiter
  File "/QOpenSys/pkgs/lib/python3.6/site-packages/gunicorn/arbiter.py", line 16, in <module>
    from gunicorn import sock, systemd, util
  File "/QOpenSys/pkgs/lib/python3.6/site-packages/gunicorn/sock.py", line 14, in <module>
    from gunicorn.socketfromfd import fromfd
  File "/QOpenSys/pkgs/lib/python3.6/site-packages/gunicorn/socketfromfd.py", line 26, in <module>
    raise OSError('libc not found')
OSError: libc not found
```
Several issues related to ctypes were fixed for IBM i and AIX in Python 3.7, but Python 3.6 is still widely used (and is the newest version distributed by IBM) on IBM i. 

Will send a PR with a proposed change soon. 


Discussion:

Resolving patch:
diff --git a/gunicorn/socketfromfd.py b/gunicorn/socketfromfd.py
--- a/gunicorn/socketfromfd.py
+++ b/gunicorn/socketfromfd.py
@@ -20,10 +20,18 @@
 SO_TYPE = getattr(socket, 'SO_TYPE', 3)
 SO_PROTOCOL = getattr(socket, 'SO_PROTOCOL', 38)
 
-
 _libc_name = find_library('c')
 if _libc_name is not None:
-    libc = ctypes.CDLL(_libc_name, use_errno=True)
+    if sys.platform.startswith("aix"):
+        member = (
+            '(shr_64.o)' if ctypes.sizeof(ctypes.c_voidp) == 8 else '(shr.o)')
+        # 0x00040000 correspondes to RTLD_MEMBER, undefined in Python <= 3.6
+        dlopen_mode = (ctypes.DEFAULT_MODE | 0x00040000 | os.RTLD_NOW)
+        libc = ctypes.CDLL(_libc_name+member,
+                           use_errno=True,
+                           mode=dlopen_mode)
+    else:
+        libc = ctypes.CDLL(_libc_name, use_errno=True)
 else:
     raise OSError('libc not found')
 
@@ -36,6 +44,7 @@ def _errcheck_errno(result, func, arguments):
         raise OSError(errno, os.strerror(errno))
     return arguments
 
+
 if platform.system() == 'SunOS':
     _libc_getsockopt = libc._so_getsockopt
 else:
diff --git a/gunicorn/util.py b/gunicorn/util.py
--- a/gunicorn/util.py
+++ b/gunicorn/util.py
@@ -659,9 +659,9 @@ def _is_elf(filepath):
         if _is_elf(f):
             return os.path.basename(f)
         prefix = os.path.join(d, 'lib'+name)
-        for suffix in ['so', 'so.*', '*.so.*']:
+        for suffix in ['so', 'so.*', '*.so.*', 'a']:
             for f in glob('{0}.{1}'.format(prefix, suffix)):
-                if _is_elf(f):
+                if _is_elf(f) or suffix == 'a':
                     return os.path.basename(f)
 
 


Test patch:
Repository: benoitc/gunicorn

Issue:
threading.Thread.setDaemon has been deprecated in favor of setting daemon attribute directly in Python 3.10
Ref : python/cpython#25174

https://github.com/benoitc/gunicorn/blob/cf55d2cec277f220ebd605989ce78ad1bb553c46/gunicorn/reloader.py#L20

https://github.com/benoitc/gunicorn/blob/cf55d2cec277f220ebd605989ce78ad1bb553c46/gunicorn/reloader.py#L77


Discussion:

Resolving patch:
diff --git a/gunicorn/reloader.py b/gunicorn/reloader.py
--- a/gunicorn/reloader.py
+++ b/gunicorn/reloader.py
@@ -17,7 +17,7 @@
 class Reloader(threading.Thread):
     def __init__(self, extra_files=None, interval=1, callback=None):
         super().__init__()
-        self.setDaemon(True)
+        self.daemon = True
         self._extra_files = set(extra_files or ())
         self._interval = interval
         self._callback = callback
@@ -74,7 +74,7 @@ class InotifyReloader(threading.Thread):
 
         def __init__(self, extra_files=None, callback=None):
             super().__init__()
-            self.setDaemon(True)
+            self.daemon = True
             self._callback = callback
             self._dirs = set()
             self._watcher = Inotify()


Test patch: