Repository: bokeh/bokeh

Issue:
Handle slow callback functions (like querying a remote data source) when using x_range.on_change in Bokeh server
<sup>This started as [a question on Stackoverflow](http://stackoverflow.com/questions/34932010/handle-slow-callback-functions-when-using-x-range-on-change-in-bokeh-server)</sup>

I'm trying to use callback functions to generate new data in a Bokeh server plot when the axis range changes. This works fine in principle: I've attached the necessary function to the plot using p.x_range.on_change('end', my_callback_function), my_callback_function is executed and updates the plot.

The problem I'm facing is that the callback function can take a long time to complete, because I'm querying a server for new data (and then doing some downsampling). If the user pans the plot or zooms several steps while `my_callback_function` is still running, more `on_change` events are fired and new calls to `my_callback_function` pile up. This leads to the plot updating for a long time after the user has stopped panning or zooming.
- Is there a way to cancel the execution of a callback function in case a new call is executed?
- Or is it possible to specify that only one call to a callback function may be queued up at any one time?
- What other ways are there to get data from a remote server when the range is updated, without locking up the Bokeh server?

I tried deactivating the callback while the function is running by setting `p.x_range.remove_on_change('end', my_callback_function)` at the start of `my_callback_function`, but apparently that deactivation doesn't happen until the call to `my_callback_function` is complete: new calls are still piling up.

I also tried using the `threading` library to run the data generating function in a separate thread while the callback function merely checks whether the data generating function is currently running. However, it seems like the new thread is never started (I don't see any output, at least, and the plot doesn't update).

Here's a minimal example:

``` python
import bokeh.plotting
import numpy
import time

def generate_data(start, end):
    time.sleep(1)
    newx = numpy.linspace(start + 0.1 * (end - start), end - 0.1 * (end - start), 10)
    newy = numpy.random.rand(10)
    return newx, newy

def my_callback_function(param, old, new):
    # I tried deactivating the callback once it's running...
    # p.x_range.remove_on_change('end', my_callback_function)
    x, y = generate_data(p.x_range.start, p.x_range.end)
    ds.data = {
        'x': x,
        'y': y
    }
    # ... and activating it again when done, but that doesn't work, the function calls still pile up
    # p.x_range.on_change('end', my_callback_function)

x, y = generate_data(0, 10)

p = bokeh.plotting.figure(x_range=[0, 10])
l = p.line(x, y)
ds = l.data_source

p.x_range.on_change('end', my_callback_function)

bokeh.plotting.curdoc().add_root(p)
```



Discussion:
Hi @jdugge, I recall a conversation with @havocp about buffering events and not letting a buffer get too big....I'm not sure what happened with that - hopefully he can shed some light.

Oops, I had an email flagged to respond to this and never did it. Probably not relevant anymore, but for anyone wondering in the future - one way to handle this is to make your callback add a "next tick" handler, only if there isn't one already added (i.e. be sure you've added only one at a time - keep a variable for whether it's added/in-progress, and unset the variable at the end of the next tick handler). Then do the work in the next tick handler. A variation on this if you want to rate-limit updates is to add a timeout rather than a next tick handler, which will mean you only do the work every certain amount of time.

Thanks for the tip, @havocp. Unfortunately, I still can't get this to work: the `next_tick_callback` also seems to block the thread, so the `x_range.on_change` callbacks still pile up.

Maybe I'm misunderstanding something? Here's the adapted test code:

```
import bokeh.plotting
import numpy
import time

def generate_data(start, end):
    time.sleep(1)
    newx = numpy.linspace(start + 0.1 * (end - start), end - 0.1 * (end - start), 10)
    newy = numpy.random.rand(10)
    return newx, newy

def range_callback_function(param, old, new):
    global in_progress
    print in_progress
    if in_progress:
        print "In progress, doing nothing"
    else:
        in_progress = True
        print "Adding callback"
        doc.add_next_tick_callback(tick_callback_function)

def tick_callback_function():
    global in_progress
    print "Tick callback function"
    x, y = generate_data(p.x_range.start, p.x_range.end)
    ds.data = {
        'x': x,
        'y': y
    }
    in_progress = False

in_progress = True
x, y = generate_data(0, 10)
in_progress = False

p = bokeh.plotting.figure(x_range=[0, 10])
l = p.line(x, y)
ds = l.data_source

p.x_range.on_change('end', range_callback_function)

doc = bokeh.plotting.curdoc()

doc.add_root(p)
```

if your callback does a sleep() then yes; to avoid this you need to use a tornado nonblocking sleep or nonblocking IO or put your IO in a thread. 

well - with that sample code I'm not sure why you get pileup. but I would expect 1s latency

well, I would expect pileup but expect the piled up callbacks to all do nothing :-) sorry posting while distracted 

useful detail: if your next tick callback is a Tornado coroutine, it will just work (wont run the callback again until all the async work is done).

Thanks for the clarification, @havocp, the hint about the non-blocking function makes complete sense. Strangely, in my sample code there is a pile-up and the piled up function calls do regenerate the data. The `if in_progress: print "In progress, doing nothing"` branch is never executed.

My actual use case is accessing a MongoDB. I tried making that process non-blocking by using the Motor library (MongoDB for Tornado) with a `coroutine`, as you suggested, but the calls still pile up. Would you mind taking a look at that issue and pointing me in the right direction?

Sorry for being so dense, but I still can't manage to get this to work, even with my simple test case. I've adapted the example to now do the following:
- simulate an expensive non-blocking data regeneration process using `tornado.gen.sleep(1)` in a `@gen.coroutine`
- every 0.1 seconds, a periodic callback checks whether the expensive non-blocking data regeneration process is currently active:
  - if the data regeneration process is currently active, do nothing
  - if the axis range has changed, add the data regeneration process to the `next_tick_callback`

However, the periodic callback is not executed while the data regeneration process is active. There's nothing blocking in the data regeneration process, but even though I'm using a `@gen.coroutine`, Bokeh blocks the periodic callback while waiting for the `next_tick_callback` to finish.

Could you tell me where I'm going wrong, or point me to an example that demonstrates how to use asynchronous callbacks?

Apologies for taking up so much of your time.

``` python
import bokeh.plotting
from tornado import gen

@gen.coroutine
def update_data():
    global in_progress
    yield gen.sleep(1)
    in_progress = False


def my_periodic_callback():
    global in_progress
    if in_progress:
        # This line never runs, the periodic callback doesn't execute while the next_tick_callback is active
        print "in progress"
    else:
        print "regenerate data"
        in_progress = True
        doc.add_next_tick_callback(update_data)


p = bokeh.plotting.figure(x_range=[0, 10], y_range=[0,1000])
l = p.line(x=[0, 10], y=[0, 1000])

in_progress = False

doc = bokeh.plotting.curdoc()
doc.add_periodic_callback(my_periodic_callback, 100)
doc.add_root(p)
```

Sorry for the delayed response, there is alot going on... I can reproduce the behavior, and I believe it is a bug. Hopefully not a fundamental limitation of Tornado in some way (I don't think it should be)

OK! If I change the `next_tick_callback` to standard tornado, i.e.:

```
ioloop.IOLoop.instance().add_callback(update_data)
```

Then it works as expected. @havocp do you have any quick thoughts as to why this might work but `next_tick_callback` doesn't?

Maybe both callbacks lock the document so they can't run at the same time. We auto-lock the doc from handler callbacks (but not from lifecycle callbacks).

I'm not sure we should keep the lock held while a coroutine is yielding. yielding still allows other sessions to make progress, but right now you can't avoid tying up the session your callback belongs to.

The lock guarantees that the document doesn't change during the yield at the moment. Which could be convenient sometimes and inconvenient other times.

@havocp can you point me to the lock? I tried tracing from `next_tick_callback` and I am not seeing where anything gets locked. 

Edit: nm, found it in `Session`

yeah it's in a decorator 

So I guess the problem is that without the lock, any coroutine that modifies the document in a yield is in error, correct? Presumably with the potential for protocol corruption as was seen in the other issue? Is there any way we could detect and complain loudly if this happens? i.e., some kind of "soft-lock" state that just makes sure to notice if the document is modified and complains in a meaningful way (more meaningful than a protocol explosion)

Alternatively, an option to the callback adding functions that toggles whether the lock is held in a yield? I'm not actually sure how we'd toggle it on/off at the right point offhand, though. 

For the yields I don't think protocol corruption would happen since we'd still have the lock when actually running code. The problem is just that whenever you yield you have to be prepared for the client or another callback to change anything about the document. In general you'd want to avoid reading doc properties before a yield and then using them after a yield (currently that is safe to do).

my guess is that dropping the lock during yield is probably ok and perhaps less surprising than the current behavior. 

It might be nice to provide some way people can keep the lock across yields if necessary. not sure exactly how that would look. perhaps a decorator 

Ok, I can try to explore it, but I guess my question is how to know when you are entering a yield? There's no hook AFAIK. What about a context manager:

```
with release_doc():
    yield gen.sleep(1)
```

I thought there was a way to do this, but I guess I was wrong; I forgot that the generator gets entirely hidden and all we get from the callback is a Future that represents the entire callback completing. I made a little program to play around with this.

``` python
from tornado import gen
from tornado.ioloop import IOLoop

# Toggle this to see the two cases; where we wait for the
# callback's Future before we drop the lock, and where we don't.
with_wait_for_future = True

@gen.coroutine
def sample_callback():
  print("sample_callback entering")
  a = yield gen.sleep(0.5)
  print("sample_callback a=" + str(a))
  b = yield gen.sleep(0.5)
  print("sample_callback b=" + str(b))
  print("sample_callback exiting with result 42")
  raise gen.Return(42)

def stop_loop():
    print("exiting Tornado IOLoop")
    IOLoop.current().stop()

@gen.coroutine
def driver():
    print("driver entering")
    print("*  PRETEND WE TAKE A LOCK HERE")
    initial_future = sample_callback()
    print("driver initial_future=" + str(initial_future))
    if with_wait_for_future:
        print("driver yielding to wait for initial_future")
        result = yield initial_future
        print("driver result=" + str(result))
    else:
        print("driver just rolling ahead without waiting")
    print("*  PRETEND WE DROP THE LOCK HERE")
    IOLoop.current().call_later(3, stop_loop)
    print("driver exiting")

IOLoop.current().add_callback(driver)

print("starting Tornado IOLoop")
IOLoop.current().start()
```

Here is the output with_wait_for_future=True:

```
starting Tornado IOLoop
driver entering
*  PRETEND WE TAKE A LOCK HERE
sample_callback entering
driver initial_future=<tornado.concurrent.Future object at 0x7f1db35dc0f0>
driver yielding to wait for initial_future
sample_callback a=None
sample_callback b=None
sample_callback exiting with result 42
driver result=42
*  PRETEND WE DROP THE LOCK HERE
driver exiting
exiting Tornado IOLoop
```

And with_wait_for_future=False

```
starting Tornado IOLoop
driver entering
*  PRETEND WE TAKE A LOCK HERE
sample_callback entering
driver initial_future=<tornado.concurrent.Future object at 0x7f5cdc5bc0f0>
driver just rolling ahead without waiting
*  PRETEND WE DROP THE LOCK HERE
driver exiting
sample_callback a=None
sample_callback b=None
sample_callback exiting with result 42
exiting Tornado IOLoop
```

We do the equivalent of with_wait_for_future=True right now, here:
https://github.com/bokeh/bokeh/blob/master/bokeh/server/session.py#L45

yield_for_all_futures is our own utility that essentially deals with nested futures, I don't remember why we had nested futures, but anyway if a future contains a future we wait for the contained future too. In this simple case I think yield_for_all_futures can be thought of as equivalent to plain yield.

The relevant Tornado source is at http://www.tornadoweb.org/en/stable/_modules/tornado/gen.html#coroutine and the actual generator is driven in the Runner.run method (the send() calls are returning control to the callback). I don't see how we could hook into this, I guess.

Note that in the sample program, the part of the callback before the first yield always runs inside the lock. So this issue only arises when there's a yield in the callback.

We COULD drop the lock at the point of the _first_ yield - in fact I think it used to work that way by accident - but that's sort of a weird behavior I think, it doesn't make much sense that the first part of the callback has locking but none of the rest.

So maybe the best we could do is let people disable the lock for the entire callback, perhaps a `@without_document_lock` decorator? At that point they could use SessionContext.with_locked_document (https://github.com/bokeh/bokeh/blob/master/bokeh/server/application_context.py#L66 ) to modify the document, but of course they HAVE to use it; if they directly access curdoc() without taking the lock, stuff will explode. So a user who used our `@without_document_lock` decorator on a callback would be sort of bypassing the warranty and accepting some danger. (side note: not sure we have a way to get the SessionContext from inside the app handler script, but should be possible to add) (side note 2: perhaps we could arrange to unset curdoc() inside `without_document_lock` ?)


Resolving patch:
diff --git a/bokeh/core/property_containers.py b/bokeh/core/property_containers.py
--- a/bokeh/core/property_containers.py
+++ b/bokeh/core/property_containers.py
@@ -175,7 +175,8 @@ def _stream(self, doc, source, new_data, rollover=None):
             else:
                 L = self[k]
                 L.extend(new_data[k])
-                del L[:-rollover]
+                if rollover is not None:
+                    del L[:-rollover]
 
         from ..document import ColumnsStreamedEvent
 
diff --git a/bokeh/document.py b/bokeh/document.py
--- a/bokeh/document.py
+++ b/bokeh/document.py
@@ -8,6 +8,7 @@
 import logging
 logger = logging.getLogger(__file__)
 
+from functools import wraps
 from json import loads
 
 from six import string_types
@@ -25,6 +26,44 @@
 
 DEFAULT_TITLE = "Bokeh Application"
 
+class UnlockedDocumentProxy(object):
+    ''' Wrap a Document object so that only methods that can safely be used
+    from unlocked callbacks or threads are exposed. Attempts to otherwise
+    access or change the Document results in an exception.
+
+    '''
+
+    def __init__(self, doc):
+        self._doc = doc
+
+    def __getattr__(self, attr):
+        raise RuntimeError(
+            "Only add_next_tick_callback may be used safely without taking the document lock; "
+            "to make other changes to the document, add a next tick callback and make your changes "
+            "from that callback.")
+
+    def add_next_tick_callback(self, callback):
+        return self._doc.add_next_tick_callback(callback)
+
+    def remove_next_tick_callback(self, callback):
+        return self._doc.remove_next_tick_callback(callback)
+
+def without_document_lock(f):
+    ''' Mark a callback function to execute without first obtaining
+    the document lock.
+
+    .. warning::
+        The value of curdoc() inside the callback will be None. Any
+        attempt to modify the document inside the callback can result
+        in data or protocol corruption.
+
+    '''
+    @wraps(f)
+    def wrapper(*args, **kw):
+        return f(*args, **kw)
+    wrapper.nolock = True
+    return wrapper
+
 class DocumentChangedEvent(object):
     def __init__(self, document):
         self.document = document
@@ -487,14 +526,19 @@ def _with_self_as_curdoc(self, f):
         from bokeh.io import set_curdoc, curdoc
         old_doc = curdoc()
         try:
-            set_curdoc(self)
+            if getattr(f, "nolock", False):
+                set_curdoc(UnlockedDocumentProxy(self))
+            else:
+                set_curdoc(self)
             return f()
         finally:
             set_curdoc(old_doc)
 
     def _wrap_with_self_as_curdoc(self, f):
         doc = self
+        @wraps(f)
         def wrapper(*args, **kwargs):
+            @wraps(f)
             def invoke():
                 return f(*args, **kwargs)
             return doc._with_self_as_curdoc(invoke)
@@ -900,6 +944,7 @@ def _add_session_callback(self, callback_obj, callback, one_shot):
         if callback in self._session_callbacks:
             raise ValueError("callback has already been added")
         if one_shot:
+            @wraps(callback)
             def remove_then_invoke(*args, **kwargs):
                 if callback in self._session_callbacks:
                     obj = self._session_callbacks[callback]
@@ -996,4 +1041,3 @@ def _remove_session_callback(self, callback):
         cb = self._session_callbacks.pop(callback)
         # emit event so the session is notified and can remove the callback
         self._trigger_on_change(SessionCallbackRemoved(self, cb))
-
diff --git a/bokeh/server/session.py b/bokeh/server/session.py
--- a/bokeh/server/session.py
+++ b/bokeh/server/session.py
@@ -148,6 +148,8 @@ def with_document_locked(self, func, *args, **kwargs):
         return func(*args, **kwargs)
 
     def _wrap_document_callback(self, callback):
+        if getattr(callback, "nolock", False):
+            return callback
         def wrapped_callback(*args, **kwargs):
             return self.with_document_locked(callback, *args, **kwargs)
         return wrapped_callback


Test patch:
diff --git a/bokeh/tests/test_document.py b/bokeh/tests/test_document.py
--- a/bokeh/tests/test_document.py
+++ b/bokeh/tests/test_document.py
@@ -1,5 +1,6 @@
 from __future__ import absolute_import, print_function
 
+import pytest
 import unittest
 
 from copy import copy
@@ -891,3 +892,47 @@ def test_compute_add_root_patch(self):
         self.assertEqual(2, len(d2.roots))
         self.assertEqual(42, d2.roots[0].foo)
         self.assertEqual(57, d2.roots[1].foo)
+
+
+class TestUnlockedDocumentProxy(unittest.TestCase):
+
+    def test_next_tick_callback_works(self):
+        d = document.UnlockedDocumentProxy(document.Document())
+        assert curdoc() is not d
+        curdoc_from_cb = []
+        def cb():
+            curdoc_from_cb.append(curdoc())
+        callback = d.add_next_tick_callback(cb)
+        callback.callback()
+        assert len(curdoc_from_cb) == 1
+        assert curdoc_from_cb[0] is d._doc
+        def cb2(): pass
+        callback = d.add_next_tick_callback(cb2)
+        d.remove_next_tick_callback(cb2)
+
+    def test_other_attrs_raise(self):
+        d = document.UnlockedDocumentProxy(document.Document())
+        assert curdoc() is not d
+        with pytest.raises(RuntimeError) as e:
+            d.foo
+            assert str(e) == "Only add_next_tick_callback may be used safely without taking the document lock; "
+            "to make other changes to the document, add a next tick callback and make your changes "
+            "from that callback."
+        for attr in dir(d._doc):
+            if attr in ["add_next_tick_callback", "remove_next_tick_callback"]: continue
+            with pytest.raises(RuntimeError) as e:
+                getattr(d, "foo")
+
+    def test_without_document_lock(self):
+        d = document.Document()
+        assert curdoc() is not d
+        curdoc_from_cb = []
+        @document.without_document_lock
+        def cb():
+            curdoc_from_cb.append(curdoc())
+        callback = d.add_next_tick_callback(cb)
+        callback._callback()
+        assert callback.callback.nolock == True
+        assert len(curdoc_from_cb) == 1
+        assert curdoc_from_cb[0]._doc is d
+        assert isinstance(curdoc_from_cb[0], document.UnlockedDocumentProxy)