Repository: bokeh/bokeh

Issue:
Connection errors when sending large amounts of data to browser via a bokeh server
Recently I came across a problem with sending lots of data to a bokeh server, between 20 - 50 MB. The command I used was 
```
bokeh serve data_visualisation --show --args -i path_to_large_file
```
What would happen is that my plot would render, and then almost instantly a timeout would occur so no communication between the browser and server was possible:

```
2018-01-05 10:19:58,280 200 GET /data_visualisation (::1) 9227.38ms
2018-01-05 10:19:58,455 101 GET /data_visualisation/ws?bokeh-protocol-version=1.0&bokeh-session-id=jbQmzoBZbYDBDUdi5Ix6QShbAJtAMGPhmKvC1SWjOuwA (::1) 0.87ms
2018-01-05 10:19:58,455 WebSocket connection opened
2018-01-05 10:19:58,456 ServerConnection created
2018-01-05 10:21:28,512 WebSocket connection closed: code=None, reason=None
```

In the browser console, the following errors where shown:
```
[bokeh] Lost websocket 0 connection, 1006 ()  connection.js:221
[bokeh] Websocket connection 0 disconnected, will not attempt to reconnect connection.js:105 
```

The problem was solved by editing tornado.py by changing the rule on line 221
```super(BokehTornado, self).__init__(all_patterns)```
 to 
``` super(BokehTornado, self).__init__(all_patterns, websocket_max_message_size=50 * 1024 * 1024) ```

By default, the maximum socket message size is 10MiB for the used version of Tornado, and would disconnect for this reason. A more meaningful error message or the possibility to change this value in bokeh would be appreciated.

#### Software version info
bokeh==0.12.13
Tornado==4.5.2
python==3.6.3
MacOS High Sierra (10.13.2)
Any browser (Chrome (63.0.3239.84) / Safari / Firefox)

  


Discussion:
@Belloco unfortunately it is not possible to provide a better error message. Those messages originate from the browser, and Tornado respectively, not Bokeh. Bokeh only sees that the connection is closed, it does not get any information about *why* it was closed. 

However, if it is possible to expose `websocket_max_message_size` as a user configurable option that that's certainly reasonable. However it is not completely trivial, as that option appears to be new to Tornado 4.5 and Bokeh support tornado >=4.4 so there will have to be some kind of version handling. Are you interested in helping work up a PR for this?
@bryevdv I'm not really interested in helping work up a PR. I mainly wanted to make sure that if other people encountered this problem, they wouldn't have to search as long as I did :)
@Belloco FWIW open source software depends crucially on spreading the workload to more people. Since you seem capable of doing this work, I do hope you might reconsider. Thanks for the detailed report in any case. 
THANK YOU so much for posting this solution! This was a huge problem for me, but by chance I stumbled across this post.
I'll second the thanks for this solution, which has allowed me a temporary workaround to this problem.  If anyone is able to work up a PR to help fix this as a configurable option in a future version that would be awesome.
After updating to Bokeh 0.12.15 I wanted to apply the "fix" to the file tornado.py and discovered that the relevant line is now 263 and has been changed to (in #7680):
```python
super(BokehTornado, self).__init__(all_patterns, **kwargs)
```
Can `**kwargs` somehow be used to set `websocket_max_message_size` in a "regular" Bokeh server application?
Or would that require an ["embedded" approach](https://bokeh.pydata.org/en/latest/docs/user_guide/server.html#embedding-bokeh-server-as-a-library) (something I have not tried yet)?
Has anyone found a solution to this issue yet? Or is switching to datashader the only solution for now?
Seems like switching to datashader is only one solution, and that the more direct solution is to change the tornado setting as outlined above.
I'm working on a PR for this right now. 

Resolving patch:
diff --git a/bokeh/command/subcommands/serve.py b/bokeh/command/subcommands/serve.py
--- a/bokeh/command/subcommands/serve.py
+++ b/bokeh/command/subcommands/serve.py
@@ -306,6 +306,7 @@
 from bokeh.resources import DEFAULT_SERVER_PORT
 from bokeh.util.logconfig import basicConfig
 from bokeh.util.string import nice_join, format_docstring
+from bokeh.server.tornado import DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES
 from bokeh.settings import settings
 
 from os import getpid
@@ -467,10 +468,20 @@ class Serve(Subcommand):
             metavar='N',
             action='store',
             help="Number of worker processes for an app. Default to one. Using "
-                 "0 will autodetect number of cores",
+                 "0 will autodetect number of cores (defaults to 1)",
             default=1,
             type=int,
         )),
+
+        ('--websocket-max-message-size', dict(
+            metavar='BYTES',
+            action='store',
+            help="Set the Tornado websocket_max_message_size value (defaults "
+                 "to 20MB) NOTE: This setting has effect ONLY for Tornado>=4.5",
+            default=DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES,
+            type=int,
+        )),
+
     )
 
 
@@ -520,6 +531,7 @@ def invoke(self, args):
                                                               'stats_log_frequency_milliseconds',
                                                               'mem_log_frequency_milliseconds',
                                                               'use_xheaders',
+                                                              'websocket_max_message_size',
                                                             ]
                           if getattr(args, key, None) is not None }
 
diff --git a/bokeh/server/server.py b/bokeh/server/server.py
--- a/bokeh/server/server.py
+++ b/bokeh/server/server.py
@@ -32,7 +32,7 @@
 from ..util.options import Options
 
 from .util import bind_sockets, create_hosts_whitelist
-from .tornado import BokehTornado
+from .tornado import BokehTornado, DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES
 
 # This class itself is intentionally undocumented (it is used to generate
 # documentation elsewhere)
@@ -78,6 +78,12 @@ class _ServerOpts(Options):
     ``X-Scheme``, ``X-Forwarded-Proto`` headers (if they are provided).
     """)
 
+    websocket_max_message_size = Int(default=DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES, help="""
+    Set the Tornado ``websocket_max_message_size`` value.
+
+    NOTE: This setting has effect ONLY for Tornado>=4.5
+    """)
+
 
 class BaseServer(object):
     ''' Explicitly coordinate the level Tornado components required to run a
@@ -391,7 +397,11 @@ def __init__(self, applications, io_loop=None, http_server_kwargs=None, **kwargs
 
         extra_websocket_origins = create_hosts_whitelist(opts.allow_websocket_origin, self.port)
         try:
-            tornado_app = BokehTornado(applications, extra_websocket_origins=extra_websocket_origins, prefix=self.prefix, **kwargs)
+            tornado_app = BokehTornado(applications,
+                                       extra_websocket_origins=extra_websocket_origins,
+                                       prefix=self.prefix,
+                                       websocket_max_message_size_bytes=opts.websocket_max_message_size,
+                                       **kwargs)
 
             http_server = HTTPServer(tornado_app, **http_server_kwargs)
 
diff --git a/bokeh/server/tornado.py b/bokeh/server/tornado.py
--- a/bokeh/server/tornado.py
+++ b/bokeh/server/tornado.py
@@ -28,11 +28,12 @@
 # Unfortuntely we can't yet use format_docstring to keep these automatically in sync in
 # the class docstring because Python 2 does not allow setting class.__doc__ (works with
 # Bokeh model classes because they are metaclasses)
-DEFAULT_CHECK_UNUSED_MS    = 17000
-DEFAULT_KEEP_ALIVE_MS      = 37000  # heroku, nginx default to 60s timeout, so use less than that
-DEFAULT_MEM_LOG_FREQ_MS    = 0
-DEFAULT_STATS_LOG_FREQ_MS  = 15000
-DEFAULT_UNUSED_LIFETIME_MS = 15000
+DEFAULT_CHECK_UNUSED_MS                  = 17000
+DEFAULT_KEEP_ALIVE_MS                    = 37000  # heroku, nginx default to 60s timeout, so use less than that
+DEFAULT_MEM_LOG_FREQ_MS                  = 0
+DEFAULT_STATS_LOG_FREQ_MS                = 15000
+DEFAULT_UNUSED_LIFETIME_MS               = 15000
+DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES = 20*1024*1024
 
 
 class BokehTornado(TornadoApplication):
@@ -124,6 +125,12 @@ class BokehTornado(TornadoApplication):
             If there are multiple Bokeh applications configured, this option
             has no effect.
 
+        websocket_max_message_size_bytes (int, optional):
+            Set the Tornado ``websocket_max_message_size`` value.
+            (default: 20*1024*1024)
+
+            NOTE: This setting has effect ONLY for Tornado>=4.5
+
     Any additional keyword arguments are passed to ``tornado.web.Application``.
     '''
 
@@ -142,6 +149,7 @@ def __init__(self,
                  mem_log_frequency_milliseconds=DEFAULT_MEM_LOG_FREQ_MS,
                  use_index=True,
                  redirect_root=True,
+                 websocket_max_message_size_bytes=DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES,
                  **kwargs):
 
         # This will be set when initialize is called
@@ -198,6 +206,13 @@ def __init__(self,
                 log.info("Log memory usage every %d milliseconds", mem_log_frequency_milliseconds)
         self._mem_log_frequency_milliseconds = mem_log_frequency_milliseconds
 
+        if websocket_max_message_size_bytes <= 0:
+            raise ValueError("websocket_max_message_size_bytes must be postitive")
+        elif websocket_max_message_size_bytes != DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES:
+            log.info("Torndado websocket_max_message_size set to %d bytes (%0.2f MB)",
+                     websocket_max_message_size_bytes,
+                     websocket_max_message_size_bytes/1024.0**2)
+
         if extra_websocket_origins is None:
             self._websocket_origins = set()
         else:
@@ -260,7 +275,9 @@ def __init__(self,
         for line in pformat(all_patterns, width=60).split("\n"):
             log.debug("  " + line)
 
-        super(BokehTornado, self).__init__(all_patterns, **kwargs)
+        super(BokehTornado, self).__init__(all_patterns,
+                                           websocket_max_message_size=websocket_max_message_size_bytes,
+                                           **kwargs)
 
     def initialize(self, io_loop):
         ''' Start a Bokeh Server Tornado Application on a given Tornado IOLoop.


Test patch:
diff --git a/bokeh/command/subcommands/tests/test_serve.py b/bokeh/command/subcommands/tests/test_serve.py
--- a/bokeh/command/subcommands/tests/test_serve.py
+++ b/bokeh/command/subcommands/tests/test_serve.py
@@ -6,6 +6,7 @@
 import socket
 import subprocess
 import sys
+from time import sleep
 
 import pytest
 import requests
@@ -167,10 +168,19 @@ def test_args():
              metavar='N',
              action='store',
              help="Number of worker processes for an app. Default to one. Using "
-                  "0 will autodetect number of cores",
+                  "0 will autodetect number of cores (defaults to 1)",
              default=1,
              type=int,
          )),
+
+         ('--websocket-max-message-size', dict(
+            metavar='BYTES',
+            action='store',
+            help="Set the Tornado websocket_max_message_size value (defaults "
+                 "to 20MB) NOTE: This setting has effect ONLY for Tornado>=4.5",
+            default=20*1024*1024,
+            type=int,
+        )),
     )
 
 
@@ -225,17 +235,29 @@ def test_port_not_available():
 
 @pytest.mark.skipif(six.PY2, reason="Travis bug causes bad file descriptor")
 def test_actual_port_printed_out():
+    from fcntl import fcntl, F_GETFL, F_SETFL
+    from os import O_NONBLOCK, read
+    pat = re.compile(r'Bokeh app running at: http://localhost:(\d+)')
+    m = None
     with run_bokeh_serve(["--port", "0"]) as p:
-        pat = re.compile(r'Bokeh app running at: http://localhost:(\d+)')
-        while True:
-            line = p.stdout.readline()
-            print("child stdout>", line)
-            m = pat.search(line.decode())
-            if m is not None:
-                break
-        else:
+        flags = fcntl(p.stdout, F_GETFL)
+        fcntl(p.stdout, F_SETFL, flags | O_NONBLOCK)
+        sleep(2)
+        o = read(p.stdout.fileno(), 100*1024)
+        m = pat.search(o.decode())
+        if m is None:
             pytest.fail("no matching log line in process output")
         port = int(m.group(1))
         assert port > 0
         r = requests.get("http://localhost:%d/" % (port,))
         assert r.status_code == 200
+
+@pytest.mark.skipif(six.PY2, reason="Travis bug causes bad file descriptor")
+def test_websocket_max_message_size_printed_out():
+    pat = re.compile(r'Torndado websocket_max_message_size set to 12345')
+    with run_bokeh_serve(["--websocket-max-message-size", "12345"]) as p:
+        sleep(2)
+    o, e = p.communicate()
+    m = pat.search(o.decode())
+    if m is None:
+        pytest.fail("no matching log line in process output")
diff --git a/bokeh/server/tests/test_tornado.py b/bokeh/server/tests/test_tornado.py
--- a/bokeh/server/tests/test_tornado.py
+++ b/bokeh/server/tests/test_tornado.py
@@ -60,6 +60,11 @@ def test_prefix():
         with ManagedServerLoop(application, prefix=prefix) as server:
             assert server._tornado.prefix == "/foo"
 
+def test_websocket_max_message_size_bytes():
+    app = Application()
+    t = tornado.BokehTornado({"/": app}, websocket_max_message_size_bytes=12345)
+    assert t.settings['websocket_max_message_size'] == 12345
+
 def test_websocket_origins():
     application = Application()
     with ManagedServerLoop(application) as server: