Repository: bokeh/bokeh

Issue:
Don't log intentional exceptions during tests
In `bokeh/server/views/bbauth.py` in function `handle_auth_error()` on line 22:

```
    try:
        return func(*args, **kwargs)
    except AuthenticationException as e:
        logger.exception(e)
        return abort(401)
```

The call to `logger.exception()` makes any errors appear as exceptions. This is intended (I suppose) during normal running. But in the test `server/tests/cow_tests.py` there is:

```
def test_handle_auth_error_decorator(self):
   # ... removed a few lines for brevity
    @handle_auth_error
    def helper2():
        raise AuthenticationException('bad!')
    self.assertRaises(Unauthorized, helper2)
```

This test succeeds, but still an exception trace is written.

Possible solutions: 
- log a warning instead of an exception. 
- Make the error not being logged, maybe by running in a context manager? Not what is the common way to achieve this.



Discussion:
Here's a minimal piece of code to run the test:

```
from werkzeug.exceptions import Unauthorized
from bokeh.server.views.bbauth import handle_auth_error
from bokeh.exceptions import AuthenticationException

from nose.tools import assert_raises


@handle_auth_error
def helper2():
    raise AuthenticationException('bad!')
assert_raises(Unauthorized, helper2)
```

I could take this one, but need some advice on what you think is the best approach. Turning it into a warning is the simplest solution, and solves the main problem. Suppressing the error completely is more difficult, unless there is already some mechanism available to temporarily turn off logging during tests.

Not sure warning is a good idea because you _really_ need to test that Auth exception is handled.

I'd suggest monkeypatching logger in that `test_handle_auth_error_decorator` test in order to "turn off" logging just in that point of tests. It's not a nice solution but should do the trick...

Ah, it's a bit more subtle:

```
try:
    return func(*args, **kwargs)
except AuthenticationException as e:
    logger.exception(e)  # <--- prints red text to console that looks like an exception
    return abort(401)  # <-- raises an Unauthorized exception
```

The test (see also some more code in the first post) checks that the `Unauthorized` exception is raised. My proposal is to turn `logger.exception(e)` into `logger.warn(str(e))`. 

Now that I think of it, is there a good reason at all to call `logger.exception()` right before raising an exception? The `return` suggest that whoever wrote this was not aware that `abort()` raises an exception...

`logger.warn` seems to work too.

Why not use the `mock.patch` decorator to patch `logger.execption` in the one test? You can then verify `logger.exception` was called and even inspect the actual arguments if you wanted. That seems like the best solution to me and the intended use of `mock.patch`

pinging @hhuuggoo because he explicitly wants this log there... maybe we need some deep understanding of the use case to choose the best solution...

I think the log is good too, that's why I am proposing to use `mock.patch` to test it explicitly. 

Yeap agree. That's the reason why I proposed `monkeypatch` (sorry, should have been `mock.patch` but I prefer `pytest` too much :stuck_out_tongue_winking_eye: ) in first place.

I am actually interested in looking into `pytest` at some point, after seeing a presentation on it. Maybe you can open an issue to discuss, that shows what some of our tests might look like if we used `pytest` instead. 

> Why not use the mock.patch decorator to patch logger.execption in the one test?

I did not know that one. Thanks, this seems like the best way forward indeed.

I've experience with both nose and pytest and really prefer the latter. Also, pytest is compatible with nose, so you can aim for a gradual transition (e.g. only write new tests with pytest).

I'm inclined to give it a whirl soon then, we just need to make sure it works with our test automation. 


Resolving patch:
diff --git a/bokeh/server/configure.py b/bokeh/server/configure.py
--- a/bokeh/server/configure.py
+++ b/bokeh/server/configure.py
@@ -88,7 +88,7 @@ def configure_flask(config_argparse=None, config_file=None, config_dict=None):
 def register_blueprint():
     global registered
     if registered:
-        warnings.warn(
+        logging.warn(
             "register_blueprint has already been called, why is it being called again"
         )
         return


Test patch:
diff --git a/bokeh/server/tests/cow_tests.py b/bokeh/server/tests/cow_tests.py
--- a/bokeh/server/tests/cow_tests.py
+++ b/bokeh/server/tests/cow_tests.py
@@ -24,7 +24,8 @@ def helper1():
         def helper2():
             raise AuthenticationException('bad!')
         assert helper1() == 'foo'
-        self.assertRaises(Unauthorized, helper2)
+        with mock.patch('bokeh.utils.logging'):
+            self.assertRaises(Unauthorized, helper2)
 
     def test_write_auth_checks_rw_users_field(self):
         user = User('test1', 'sdfsdf', 'sdfsdf')
diff --git a/bokeh/server/tests/test_utils.py b/bokeh/server/tests/test_utils.py
--- a/bokeh/server/tests/test_utils.py
+++ b/bokeh/server/tests/test_utils.py
@@ -3,6 +3,7 @@
 import threading
 import time
 import unittest
+import mock
 
 #import redis
 import requests
@@ -104,7 +105,8 @@ def setUp(self):
             setattr(server_settings, k, v)
         server_settings.model_backend = {'type' : 'memory'}
         configure.configure_flask()
-        configure.register_blueprint()
+        with mock.patch('bokeh.utils.logging'):
+            configure.register_blueprint()
         #ugh..need better way to initialize this
         app.secret_key = server_settings.secret_key
         app.debug = True