Repository: bokeh/bokeh

Issue:
Change default bokeh server backend to "memory"
I have an embedded bokeh visualisation (using `autoload_server`) that is crashing strangely.

The visualisation is composed of a dropdown widget, and 2 plots (Line & Patches). The dropdown controls the data displayed by the two plots.

After running it, it initially behaves flawlessly. Interaction is possible with the two plots, and the dropdown updates the data as expected.

However, after a few minutes, it will crash (see logs below). Even if there is no user interaction at all with the browser, a crash will occur in a few minutes.

I have an other embedded plot that works fine (Circle glyph), that is using the same general machinery to create the bokeh session / documents / create the autoload_server / regularly poll load_document. No crashes.

How can we track the source of that crash ? 

*\* edit**
my other visualization's bokeh server crash too, but it takes longer (about 20 minutes, and with a different error message)

```
HASH: Out of overflow pages.  Increase page size
ERROR:bokeh.server:Exception on /bokeh/bb/7f1440c9-8b76-4f5a-90a8-49fb2830bcb3/gc [POST]
....
error: cannot add item to database
ERROR:tornado.access:500 POST /bokeh/bb/7f1440c9-8b76-4f5a-90a8-49fb2830bcb3/gc (127.0.0.1) 99.50ms
```

Bokeh version :  '0.8.0'

javascript console :

![screen shot 2015-02-24 at 11 33 31 am](https://cloud.githubusercontent.com/assets/3415960/6342763/fd7c9f3a-bc18-11e4-816c-b7ea272367d7.png)

traceback from the python process

``` python
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
/Users/nicolas/Code/Dash/migdash/dashboards/dash_geog.py in <module>()
    184 
    185 if __name__ == "__main__":
--> 186     GeogDashboard(sDoc_name='migdash_geog').standalone_start()

/Users/nicolas/Code/Dash/migdash/dashboards/dashboard.pyc in standalone_start(self)
    144         view(link)
    145         print("\npress ctrl-C to exit")
--> 146         session.poll_document(doc)
    147 

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/session.pyc in poll_document(self, document, interval)
    650         try:
    651             while True:
--> 652                 self.load_document(document)
    653                 time.sleep(interval)
    654         except KeyboardInterrupt:

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/session.pyc in load_document(self, doc)
    541 
    542         """
--> 543         self.gc()
    544         json_objs = self.pull()
    545         doc.merge(json_objs)

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/session.pyc in gc(self)
    527     def gc(self):
    528         url = utils.urljoin(self.base_url, self.docid + "/", "gc")
--> 529         self.post_json(url)
    530 
    531     # convenience functions to use a session and store/fetch from server

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/session.pyc in post_json(self, url, headers, **kwargs)
    365 
    366         """
--> 367         return self.execute_json('post', url, headers=headers, **kwargs)
    368 
    369     @property

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/session.pyc in execute_json(self, method, url, headers, **kwargs)
    333         headers['content-type'] = 'application/json'
    334         resp = self.execute(method, url, headers=headers, **kwargs)
--> 335         return utils.get_json(resp)
    336 
    337     def get_json(self, url, headers=None, **kwargs):

/Users/nicolas/anaconda/lib/python2.7/site-packages/bokeh/utils.pyc in get_json(response)
     39         #requests
     40         if hasattr(response.json, '__call__'):
---> 41             return response.json()
     42         else:
     43             return response.json

/Users/nicolas/anaconda/lib/python2.7/site-packages/requests/models.pyc in json(self, **kwargs)
    797                     # used.
    798                     pass
--> 799         return json.loads(self.text, **kwargs)
    800 
    801     @property

/Users/nicolas/anaconda/lib/python2.7/site-packages/simplejson/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
    503             parse_constant is None and object_pairs_hook is None
    504             and not use_decimal and not kw):
--> 505         return _default_decoder.decode(s)
    506     if cls is None:
    507         cls = JSONDecoder

/Users/nicolas/anaconda/lib/python2.7/site-packages/simplejson/decoder.pyc in decode(self, s, _w, _PY3)
    368         if _PY3 and isinstance(s, binary_type):
    369             s = s.decode(self.encoding)
--> 370         obj, end = self.raw_decode(s)
    371         end = _w(s, end).end()
    372         if end != len(s):

/Users/nicolas/anaconda/lib/python2.7/site-packages/simplejson/decoder.pyc in raw_decode(self, s, idx, _w, _PY3)
    398             elif ord0 == 0xef and s[idx:idx + 3] == '\xef\xbb\xbf':
    399                 idx += 3
--> 400         return self.scan_once(s, idx=_w(s, idx).end())

JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

bokeh server crash :
bokeh server is launched with no arguments (`bokeh-server`)

``` shell
INFO:tornado.access:200 POST /bokeh/bb/bc56bddf-bca8-4b54-8627-d437036b4dfd/bulkupsert (127.0.0.1) 71.92ms
INFO:tornado.access:200 GET /bokeh/bb/523cfc6c-7374-4867-be7b-320781d0ca8c/ (127.0.0.1) 66.80ms
INFO:tornado.access:200 GET /bokeh/bb/523cfc6c-7374-4867-be7b-320781d0ca8c/ (127.0.0.1) 62.42ms
INFO:tornado.access:200 POST /bokeh/bb/dfe54b25-c157-434e-a885-d10434b22a4d/gc (127.0.0.1) 139.81ms
INFO:tornado.access:200 POST /bokeh/bb/bc56bddf-bca8-4b54-8627-d437036b4dfd/bulkupsert (127.0.0.1) 91.73ms
INFO:tornado.access:200 POST /bokeh/bb/bc56bddf-bca8-4b54-8627-d437036b4dfd/gc (127.0.0.1) 114.18ms
[1]    774 segmentation fault  bokeh-server
➜  ~
```



Discussion:
running bokeh-server with the memory backend seems to fix the problem.

This is really odd we have had the server gallery up and running for over a week now. Do you have a minimum code/deployment config to reproduce this that you can share? Are you trying to write and in-server app, or are you writing a secondary process that polls the server? 

@bryevdv I'm not sure you've seen my comment - as the logs are so long :-) - but running bokeh-server with the memory backend as solved the problem. Those crashes only happens with the shelve backend.

I'm working on a small flask app with 3 pages, each of which has a embedded bokeh plot. The plots are generated when the app starts. Every-time a page is rendered by flask, thread will start to poll the necessary bokeh document (with `session.load_document(document)`) (and the other threads will be stopped)

I'll try to prepare a minimum example

@knightofni OH, the shelve backend. That probably ought to be removed, actually. It's not intended as anything except a toy interface, mostly useful on window where redis is hard to install. 

Yep. Maybe the default backend for bokeh-server should be memory ?

Eminently reasonable. 

> Yep. Maybe the default backend for bokeh-server should be memory ?

I agree, it is a sensible default.

@Maggie-M can you look at this? It's probably just changing the default value in the `argparse` setup for the server. 


Resolving patch:
diff --git a/bokeh/server/__init__.py b/bokeh/server/__init__.py
--- a/bokeh/server/__init__.py
+++ b/bokeh/server/__init__.py
@@ -12,7 +12,7 @@
 from bokeh.server.app import bokeh_app
 from bokeh.settings import settings
 
-DEFAULT_BACKEND = os.environ.get('BOKEH_SERVER_DEFAULT_BACKEND', 'shelve')
+DEFAULT_BACKEND = os.environ.get('BOKEH_SERVER_DEFAULT_BACKEND', 'memory')
 if DEFAULT_BACKEND not in ['redis', 'shelve', 'memory']:
     print("Unrecognized default backend: '%s'. Accepted values are: 'redis', 'shelve', 'memory'" % DEFAULT_BACKEND)
     sys.exit(1)


Test patch: