Repository: zulip/zulip

Issue:
AttributeError: 'LogRecord' object has no attribute 'status_code'
I have a development environment with the latest Git version. After performing many requests, I get blocked because of rate limiting. Then, the following error is logged in the console:

```
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 56444)
2020-04-16 11:35:49.159 INFO [zr] 127.0.0.1       POST    429  65ms (mem: 57ms/4) (+start: 24ms) /json/messages (10@zulip via website)
2020-04-16 11:35:49.160 INFO [zr] status=429, data=b'{"result":"error","msg":"API usage exceeded rate limit","retry-after":2.6131470203}\n', uid=10@zulip
2020-04-16 11:35:49.162 INFO [zr] 127.0.0.1       POST    429  11ms (mem: 7ms/2) /json/messages (10@zulip via website)
2020-04-16 11:35:49.162 WARN [django.server] "POST /json/messages HTTP/1.1" 429 84
2020-04-16 11:35:49.173 INFO [zr] status=429, data=b'{"result":"error","msg":"API usage exceeded rate limit","retry-after":2.6109778881}\n', uid=10@zulip
2020-04-16 11:35:49.179 INFO [zr] 127.0.0.1       POST    429  20ms (+start: 51ms) /json/messages (10@zulip via website)
2020-04-16 11:35:49.182 WARN [django.server] "POST /json/messages HTTP/1.1" 429 84
2020-04-16 11:35:49.195 INFO [zr] status=429, data=b'{"result":"error","msg":"API usage exceeded rate limit","retry-after":2.5940015316}\n', uid=10@zulip
Traceback (most recent call last):
  File "/usr/lib/python3.7/socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.7/socketserver.py", line 720, in __init__
    self.handle()
  File "/srv/zulip-py3-venv/lib/python3.7/site-packages/django/core/servers/basehttp.py", line 171, in handle
    self.handle_one_request()
  File "/srv/zulip-py3-venv/lib/python3.7/site-packages/django/core/servers/basehttp.py", line 187, in handle_one_request
    if not self.parse_request():  # An error code has been sent, just exit
  File "/usr/lib/python3.7/http/server.py", line 322, in parse_request
    "Bad request syntax (%r)" % requestline)
  File "/usr/lib/python3.7/http/server.py", line 456, in send_error
    self.log_error("code %d, message %s", code, message)
  File "/usr/lib/python3.7/http/server.py", line 558, in log_error
    self.log_message(format, *args)
  File "/srv/zulip-py3-venv/lib/python3.7/site-packages/django/core/servers/basehttp.py", line 154, in log_message
    level(format, *args, extra=extra)
  File "/usr/lib/python3.7/logging/__init__.py", line 1383, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.7/logging/__init__.py", line 1519, in _log
    self.handle(record)
  File "/usr/lib/python3.7/logging/__init__.py", line 1528, in handle
    if (not self.disabled) and self.filter(record):
  File "/usr/lib/python3.7/logging/__init__.py", line 762, in filter
    result = f.filter(record)
  File "/srv/zulip-py3-venv/lib/python3.7/site-packages/django/utils/log.py", line 147, in filter
    if self.callback(record):
  File "/home/sjoerd/zulip/zerver/lib/logging_util.py", line 122, in skip_200_and_304
    if getattr(record, 'status_code') in [200, 304]:
AttributeError: 'LogRecord' object has no attribute 'status_code'
----------------------------------------
```

Normally, [http.server logs request, status code, size](https://github.com/python/cpython/blob/master/Lib/http/server.py#L544-L545), and [Django extracts the status code from that](https://github.com/django/django/blob/master/django/core/servers/basehttp.py#L144-L157). However, [on errors http.server logs code and message](https://github.com/python/cpython/blob/master/Lib/http/server.py#L457) and Django doesn't extract the status code.

Parsing arguments to log messages seems pretty fragile to me, so maybe it's better to accept that there isn't always a status code on a log record. Making `getattr` default to `None` in [`skip_200_and_304`](https://github.com/zulip/zulip/blob/master/zerver/lib/logging_util.py#L122) is probably the best option.




Discussion:
This is likely a regression in the rate-limiter infrastructure rewrite we merged yesterday.

@mateuszmandera FYI.
Hello @zulip/server-production members, this issue was labeled with the "area: production" label, so you may want to check it out!

<!-- areaLabelAddition -->
@Sjord Could you provide steps to reproduce this reliably? I'm not able to reproduce so far - I'm hitting the rate limits and things are getting logged as expected
It turns out the rate limiting was a red herring. Sorry about that. The actual trigger is requests with invalid syntax. The following curl command triggers this exception in the developer console for me:

    curl  -X POST -H 'Transfer-Encoding : chunked' --data-binary 'a' 'http://zulipdev.com:9991/json/messages/57'

Resolving patch:
diff --git a/zerver/lib/logging_util.py b/zerver/lib/logging_util.py
--- a/zerver/lib/logging_util.py
+++ b/zerver/lib/logging_util.py
@@ -119,7 +119,7 @@ def skip_200_and_304(record: logging.LogRecord) -> bool:
     # Apparently, `status_code` is added by Django and is not an actual
     # attribute of LogRecord; as a result, mypy throws an error if we
     # access the `status_code` attribute directly.
-    if getattr(record, 'status_code') in [200, 304]:
+    if getattr(record, 'status_code', None) in [200, 304]:
         return False
 
     return True


Test patch: