Repository: ckan/ckan

Issue:
Not authorized and Not found generates errors in logs even though the errors are shown correctly in the UI
**CKAN version**
2.8, 2.9, master

**Describe the bug**
When some flask route produces NotFound or NotAuthorized, error is generated in the logs even though the UI handles it correctly.

For example 404:
![image](https://user-images.githubusercontent.com/830663/130610428-86b41119-e58e-4280-b7cc-e49f5eb05b0e.png)

```
2021-08-24 11:41:47,135 ERROR [ckan.config.middleware.flask_app] 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
Traceback (most recent call last):
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/flask_debugtoolbar/__init__.py", line 125, in dispatch_request    return view_func(**req.view_args)
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/flask_multistatic.py", line 96, in send_static_file
    raise NotFound()
werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
2021-08-24 11:41:47,153 INFO  [ckan.config.middleware.flask_app]  /foobar render time 0.024 seconds
```

403 (public user details are turned of to produce this):
![image](https://user-images.githubusercontent.com/830663/130610551-fa7adbb1-e543-4a2c-983b-1de46f62b03e.png)
```
2021-08-24 11:43:13,907 ERROR [ckan.config.middleware.flask_app] 403 Forbidden: Not authorized to see this page
Traceback (most recent call last):
  File "/vagrant/ckan/config/middleware/../../views/user.py", line 58, in _extra_template_variables
    user_dict = logic.get_action(u'user_show')(context, data_dict)
  File "/vagrant/ckan/logic/__init__.py", line 499, in wrapped
    result = _action(context, data_dict, **kw)
  File "/vagrant/ckan/logic/action/get.py", line 1399, in user_show
    _check_access('user_show', context, data_dict)
  File "/vagrant/ckan/logic/__init__.py", line 317, in check_access
    raise NotAuthorized(msg)
ckan.logic.NotAuthorized

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/flask_debugtoolbar/__init__.py", line 125, in dispatch_request
    return view_func(**req.view_args)
  File "/vagrant/ckan/config/middleware/../../views/user.py", line 148, in read
    extra_vars = _extra_template_variables(context, data_dict)
  File "/vagrant/ckan/config/middleware/../../views/user.py", line 62, in _extra_template_variables
    base.abort(403, _(u'Not authorized to see this page'))
  File "/vagrant/ckan/lib/base.py", line 47, in abort
    flask_abort(status_code, detail)
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/werkzeug/exceptions.py", line 822, in abort
    return _aborter(status, *args, **kwargs)
  File "/usr/lib/ckan/default/lib/python3.6/site-packages/werkzeug/exceptions.py", line 807, in __call__
    raise self.mapping[code](*args, **kwargs)
werkzeug.exceptions.Forbidden: 403 Forbidden: Not authorized to see this page
2021-08-24 11:43:13,930 INFO  [ckan.config.middleware.flask_app]  /user/foobar render time 0.039 seconds
```

This would not be a problem as such but when you use error logging middlewares like sentry or just sending errors by email, you will be swamped by errors which actually should work correctly.

**Steps to reproduce**
Open nonexisting page, check logs for stacktrace.

**Expected behavior**
Logs should not have have stacktrace in them





Discussion:
For both errors, capture and hide the exception, but add the status code to the `[ckan.config.middleware.flask_app]  /user/foobar render time 0.039 seconds` line (check if that's added on API requests as well)

Resolving patch:
diff --git a/ckan/config/middleware/flask_app.py b/ckan/config/middleware/flask_app.py
--- a/ckan/config/middleware/flask_app.py
+++ b/ckan/config/middleware/flask_app.py
@@ -395,8 +395,9 @@ def ckan_after_request(response):
 
     r_time = time.time() - g.__timer
     url = request.environ['PATH_INFO']
+    status_code = response.status_code
 
-    log.info(' %s render time %.3f seconds' % (url, r_time))
+    log.info(' %s %s render time %.3f seconds' % (status_code, url, r_time))
 
     return response
 
@@ -525,8 +526,9 @@ def _register_error_handler(app):
     u'''Register error handler'''
 
     def error_handler(e):
-        log.error(e, exc_info=sys.exc_info)
+        debug = asbool(config.get('debug', config.get('DEBUG', False)))
         if isinstance(e, HTTPException):
+            log.error(e, exc_info=sys.exc_info) if debug else log.error(e)
             extra_vars = {
                 u'code': e.code,
                 u'content': e.description,
@@ -535,6 +537,7 @@ def error_handler(e):
 
             return base.render(
                 u'error_document_template.html', extra_vars), e.code
+        log.error(e, exc_info=sys.exc_info)
         extra_vars = {u'code': [500], u'content': u'Internal server error'}
         return base.render(u'error_document_template.html', extra_vars), 500
 


Test patch: