Repository: bokeh/bokeh

Issue:
Using functools.partial in combination with add_next_tick_callback() throws exception in py2
There is example in the user guide which shows how we can update plots from blocking threads. It is suggested to run this example with `bokeh serve --show` and that works fine.  [full example](https://github.com/bokeh/bokeh/blob/008e63812da2655bb6331b9338a116a6ccd8ba47/sphinx/source/docs/user_guide/server.rst#updating-from-threads)

The problem is when we try to extend this example with `bokeh.client` and run it with py2. With py3 it works fine, but with py2 we get an exception.

Here is full code which can reproduce the issue. Run `bokeh serve` and run following python code with python2.

``` python
#!/usr/bin/python2
from functools import partial
from random import random
from threading import Thread
import time

from bokeh.client import push_session
from bokeh.models import ColumnDataSource
from bokeh.plotting import curdoc, figure, show

from tornado import gen

source = ColumnDataSource(data=dict(x=[0], y=[0]))

doc = curdoc()
session = push_session(doc)

@gen.coroutine
def update(x, y):
    source.stream(dict(x=[x], y=[y]))

def blocking_task():
    while True:
        time.sleep(0.1)
        x, y = random(), random()

        doc.add_next_tick_callback(partial(update, x=x, y=y))

p = figure(x_range=[0, 1], y_range=[0, 1])
l = p.circle(x='x', y='y', source=source)

doc.add_root(p)

thread = Thread(target=blocking_task)
thread.start()

session.show(p)
session.loop_until_closed()
```

This is the exception:

``` python
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "bokeh_test.py", line 32, in blocking_task
    doc.add_next_tick_callback(partial(update, x=x, y=y))
  File "/usr/local/lib/python2.7/dist-packages/bokeh/document.py", line 1040, in add_next_tick_callback
    return self._add_session_callback(cb, callback, one_shot=True)
  File "/usr/local/lib/python2.7/dist-packages/bokeh/document.py", line 965, in _add_session_callback
    @wraps(callback)
  File "/usr/lib/python2.7/functools.py", line 33, in update_wrapper
    setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'functools.partial' object has no attribute '__module__'
```

I'm not saying that there is a problem with documentation (like in previous issue I reported). I just didn't expect this to happen and I am not sure if I am doing something wrong or there is indeed a bigger problem. Thanks!



Discussion:
I am seeing the same. At the least we should document if this is py3 only code. I found this SO issue in case it helps: https://stackoverflow.com/questions/20594193/dynamic-create-method-and-decorator-got-error-functools-partial-object-has-no

I'm using code from that SO post as dirty fix. Redefining wraps decorator doesn't help in this case because it is imported from functools and used somewhere deep in bokeh modules, but redefining partials works.

I think that some warning or note should be put next to that example until we have py2 compatible one.

Well another option from a documentation POV is to change the example to not use `partial`, since it's certainly not a requirement. It's just what I happened to write to demo the usage. If `functools.partial` does not work with `@wraps` in py 2, that's unfortunate, but also largely beyond our control. Another thing might be to catch that exception and print out a more informative error stating that `partial` callbacks only work in python 3. 

My current solution is to redefine `functools.partial` in "client side" like suggested in that [SO discussion](https://stackoverflow.com/questions/20594193/dynamic-create-method-and-decorator-got-error-functools-partial-object-has-no):

``` python
def partial(func, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = keywords.copy()
        newkeywords.update(fkeywords)
        return func(*(args + fargs), **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc
```

_(code copied from SO discussion)_

Different approach would be to put complement redefinition of `@wraps` on `bokeh` side of the code. A good thing is that this redefinition will be triggered only in case of exception. That means that we don't change current behavior for py3. I am talking about redefinition like this:

``` python
from functools import wraps, partial, WRAPPER_ASSIGNMENTS

try:
    wraps(partial(wraps))(wraps)
except AttributeError:
    @wraps(wraps)
    def wraps(obj, attr_names=WRAPPER_ASSIGNMENTS, wraps=wraps):
        return wraps(obj, assigned=(name for name in attr_names if hasattr(obj, name)))
```

_(code copied from SO discussion)_

I can prepare PR for this, if you find this solution acceptable.

Slightly off the topic... What was the reason for choosing `functools.partial` for this example code in documentation? Was it about reducing number of arguments for `update()` handler from 2 to 0? In other words, is there other way than partials to pass some arguments to the callback when we register it with `add_next_tick_callback()`?

@Atlantic777 that seems reasonable to me at a glance. A PR would be useful and appreciated and would help focus any additional testing. 

As for the example, basically. Aside from `partial` you could write another function with no args, that call the original function with specific args passed in. Or instantiate a class that has a `__call__` method. But `partial` is the simplest thing I would reach for most times. 

Before I prepare PR I also want to check if there are any licensing issues when I am reusing code from stack overflow. Is mentioning user's stack overflow username and link to to stack overflow thread good approach? Or I shouldn't worry about this because licenses are compatible?

I'm not aware of any but if you want to add a link to the relevant SO answer in a comment that seems perfectly appropriate. 


Resolving patch:
diff --git a/bokeh/document.py b/bokeh/document.py
--- a/bokeh/document.py
+++ b/bokeh/document.py
@@ -8,7 +8,22 @@
 import logging
 logger = logging.getLogger(__file__)
 
-from functools import wraps
+# There is a problem with using @wraps decorator in combination with functools.partial.
+# This issue is not present in Python 3.
+# This redefinition will be triggered only if issue affects user,
+# otherwise regular definition of @wraps will be used.
+#
+# this code snippet was originally posted in followig stack overflow discussion:
+# http://stackoverflow.com/a/28752007
+
+from functools import wraps, partial, WRAPPER_ASSIGNMENTS
+try:
+    wraps(partial(wraps))(wraps)
+except AttributeError:
+    @wraps(wraps)
+    def wraps(obj, attr_names=WRAPPER_ASSIGNMENTS, wraps=wraps):
+        return wraps(obj, assigned=(name for name in attr_names if hasattr(obj, name)))
+
 from json import loads
 
 import jinja2


Test patch:
diff --git a/bokeh/tests/test_document.py b/bokeh/tests/test_document.py
--- a/bokeh/tests/test_document.py
+++ b/bokeh/tests/test_document.py
@@ -477,6 +477,27 @@ def cb(): pass
         assert isinstance(events[0], document.SessionCallbackAdded)
         assert isinstance(events[1], document.SessionCallbackRemoved)
 
+    def test_add_partial_callback(self):
+        from functools import partial
+        d = document.Document()
+
+        events = []
+        def listener(event):
+            events.append(event)
+        d.on_change(listener)
+
+        assert len(d.session_callbacks) == 0
+        assert not events
+
+        def _cb(): pass
+        cb = partial(_cb)
+
+        callback = d.add_timeout_callback(cb, 1)
+        assert len(d.session_callbacks) == len(events) == 1
+        assert isinstance(events[0], document.SessionCallbackAdded)
+        assert callback == d.session_callbacks[0] == events[0].callback
+        assert callback.timeout == 1
+
     def test_add_remove_next_tick_callback(self):
         d = document.Document()