Repository: bokeh/bokeh

Issue:
notebook_embed.ipynb works locally but not remotely via ssh tunnel.
#### ALL software version info (bokeh, python, notebook, OS, browser, any other relevant packages):

bokeh: 0.12.10
python: 3.5
notebook: 5.0.0
jupyter: 1.0.0
jupyter_client 5.1.0
jupyter_console: 5.2.0
jupyter_core: 4.3.0
ipython: 5.4.1
ipython_genutils: 0.2.0
OS: ubuntu 14.04
broswer: Remote Safari Version 11.0.1 (13604.3.5);  Local Firefox

#### Description of expected behavior and the observed behavior

The example notebook *notebook_embed.ipynb* works when the browser is run on the same machine as the jupyter notebook server.  However, when the browser is on a separate machine and accesses the jupyter notebook server via an ssh tunnel (e.g. 8888:127.0.0.1:8888), the notebook does work.  When the last cell is executed, no figure appears.

#### Complete, minimal, self-contained example code that reproduces the issue

1. On server run jupyter notebook --port 8888 in directory where bokeh/examples/howto/server_embed/notebook_embed.ipynb is accessible.

2. On a remote computer set up ssh tunnel to the remote machine:
ssh -f user@server -L 8888:127.0.0.1:8888 -N

3. In a browser on the remote computer, connect to the jupyter notebook server by pasting in the url+token provided in step 1. Then navigate to and open notebook_embed.ipynb.  

4. Execute all the cells in the notebook. The last cell will not show a figure (even though it works locally).  

Important Note: The last cell of notebook_embed.ipynb contains just *show(modify_doc)*.  This cell crashes both locally and remotely.  To get the notebook to run, change the last cell to:
'''
from bokeh.application.handlers import FunctionHandler
from bokeh.application import Application
handler = FunctionHandler(modify_doc)
app = Application(handler)
show(app, notebook_url="127.0.0.1:8888")
'''


Discussion:
Unfortunately this is outside my knowledge of networking, and I am not sure whether it is possible at all. Some Googled info here does not seem promising:

https://www.unix.com/proxy-server/254514-websocket-over-ssh-tunnel-possible.html

In any case I think any solution would necessary be a user configuration of the tunnel, and not anything on bokeh's side
Thank you for the quick reply.  I've looked at the bokeh source code related to embedding an application, and I now have more clarity on the issue.  

The issue is important because ssh-tunneling is a common approach for accessing remote jupyter servers and other servers, including bokeh servers (see the bokeh server documentation at https://bokeh.pydata.org/en/latest/docs/user_guide/server.html). Ssh tunneling allows remote access without the added security risk of opening additional ports through the firewall. 

The issue is that the *show* call for embedding the app opens a server on a random port by setting the port to 0 (see the show_app method in io.notebooks.py).  Therefore for this functionality to work remotely, the firewall would have to open over the entire dynamic port range (e.g 49152 to 65535). Many IT departments would be extremely reluctant to do this (I know Stanford Univerity's IT department is).

I can partially hack around the issue by modifying show_app in io.notebooks.py to use a fixed port (5006). With this modification, I can set up an ssh-tunnel on port 5006, and then the server embedding works perfectly.  However, this is only a partial fix since I can't re-run the cells because the port is now already in use.  To rerun the cells, I have to restart the notebook kernel.

A fix this issue could look like:
1. Allow the show method to include a port parameter that specifies the port the server should use.
2. Create a system for shutting down the embedded server.

Together these enhancements would make the server embedding system usable in the situations where the browser is not local to the server - without opening up the entire dynamic port range on the firewall.

Alternatively, perhaps the issue could be worked around by not using *show* to embed the app. Could the server be embedded in the notebook manually by creating a Server instance programmatically?  However, I'm not sure how to do this.  It might look like this but somehow modified to run inside the notebook's output cell.

Thanks again.



@nerduno Thanks for the info. shortly after posting it occurred to me that it must work somehow, or else the notebook would not function either. Thanks, for digging further. I would support a small PR that did the following:

1. add an `app_port` property to [`State`](https://github.com/bokeh/bokeh/blob/master/bokeh/io/state.py) with a default value of 0

* change [`show_app`](https://github.com/bokeh/bokeh/blob/master/bokeh/io/notebook.py#L455) to use `curstate.app_port` instead of always setting 0

This should immediately offer a supported method to accomplish what you need by setting `curstate().app_port = 1234`. 

Would you be interested in working in this PR? It would need some fairly minimal unit tests but I am happy to help advise.
@bryevdv Thanks again for the quick reply.

I like your proprosed solution. Using State makes more sense than adding parameters to show().

One additional thought.  Subsequent calls to show() will crash, because they will try to make a Server on a port that is already in use by the Server created on the first call.  It might make sense to add an `app_server` property to `State`.  The show_app method would check the `app_server' property.  If an app_server has already been created it could:
1) Stop the current server and start a new one (this would be easy).
2) Or, it could reuse the existing server by overwriting the app at '/' or by adding new app to the server (not sure if overwriting apps is possible).

Does that make sense?

I'm happy to make a PR, but I can't work on it for a few weeks.

PS how do you insert those links to lines of code in your responses?

@nerduno Thank for the additional analysis. So far, I have resisted trying to manage the bookkeeping of multiple apps on a server in a notebook. A server is just a lightweight collection of tornado callbacks, that is easy to install and remove, so it already seems like the readily available unit of bookkeeping for apps. Adding a bunch of code to manage different apps on one server just seems complicated to maintain and duplicative, when servers can just be added and removed per-app already. However, this use case of SSH tunneling was not in mind before. 

I take it that it would it suffice to be able to specify a different (user-specified) port per-app? If so, I think I would revert my suggestion and instead propose adding a new `port=0` parameter to `show_app`. Then you could do:
```
# cell
show_app(app1, curstate(), port=1234)

# cell
show_app(app2, curstate(), port=2345)
```
This is a little more work for folks that need to SSH tunnel, but we could add a section in the user's guide about it. Going this route would allow to maintain the current one app per server system which is definitely my preference. 
It's also possible we could add the `port=0` arg all the way back to the top level `show` but I'd start with the above first. 
@bryevdv I'm okay with one server for each app.  This only adds a minor inconvenience of opening one ssh-tunnel for each app.  That said, it is important to consider the following scenario:

Let's say I have a notebook cell that constructs an app and shows it. As I develop/debug the app, I will need to rerun the cell many times. I don't want to have to specify a new port and open an ssh-tunnel every time I run the cell. Therefore, I need a way to close the server/session/app so that I can free up the port before a new server is created. (Note this not a problem if a new random port is chosen everytime I run the cell, but this impractical in a remote access setting).

Ideally, the port could be freed up behind the scenes. For example, show_app could loop through the dictionary of servers in curstate(), and stop any servers that are using the specified port before starting a new server.  Perhaps `State` could keep a separate dict of embedded servers so that no manually created servers are accidently stopped.  Does that sound reasonable?

Alternatively, `show` could return the `Server` it starts, such that the user could explicitly stop it.

On a separate topic, I'm looking at your color mapper classes, and there appears to be no difference between the LinearColorMapper and a LogColorMapper. Is that bug, or am I missing something?


 
@nerduno severs should already be getting shut down any time an output cell is cleared (i.e. when it is deleted, or its input cell re-executed):

https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/autoload_nb_js.js#L46

Are you saying this is not happening?

Regarding the color map classes, do you mean on the Python side? If so, that's because the python side largely exists to be a configuration system. All the actual work and implementation is in BokehJS:

https://github.com/bokeh/bokeh/blob/master/bokehjs/src/coffee/models/mappers/linear_color_mapper.ts
https://github.com/bokeh/bokeh/blob/master/bokehjs/src/coffee/models/mappers/log_color_mapper.ts
@bryevdv Yes, that is not happening.  I've implemented the curstate().port solution you described above.  Everything works the first time I run a cell containing`show` (including using the specified port, and working over ssh-tunnel).  However, if I rerun the cell I get an exception that the server address is already in use.  I can prevent the exception by calling server.stop() before I rerun the cell.

Thanks for explaining color maps.
Thanks again for all your help on this.  Seems like the solution of adding `port` to `show_app` will work once the servers are being destroyed automatically.  I'm happy to make a PR once we have this sorted.

This evening I posted a question to the google group about whether the default TapTool behavior can be modified.  I'd love your insight if you have time.  Thanks.
@canavandl any ideas on the best way to debug this? `destroy_server` calls `server.stop()` so it's either not getting called or not getting called in time
@bryevdv @canavandl One update: if I explicitly clear the cell output before rerunning the cell, everything works fine.  However, if I rerun the cell without first clearing the cell output, then I get "Address already in use".  Rerunning the cell multiple times does not solve the issue, so perhaps it not a timing issue?
I'm not sure, we do this:
```
    events.on('clear_output.CodeCell', handleClearOutput);
    events.on('delete.Cell', handleClearOutput);
```

@canavandl is there a separate event we should add a handler for when cells are re-executed? I guess I would have thouhgt that should trigger the clear_output event but maybe not.
I'm not familiar with jupyter's javascript, but perhaps an `execute.CodeCell` event needs to call handleClearOutput before executing?

Resolving patch:
diff --git a/bokeh/io/notebook.py b/bokeh/io/notebook.py
--- a/bokeh/io/notebook.py
+++ b/bokeh/io/notebook.py
@@ -430,7 +430,7 @@ def publish_display_data(*args, **kw):
     return publish_display_data(*args, **kw)
 
 @dev((1,0,0))
-def show_app(app, state, notebook_url):
+def show_app(app, state, notebook_url, port=0):
     ''' Embed a Bokeh serer application in a Jupyter Notebook output cell.
 
     Args:
@@ -440,6 +440,9 @@ def show_app(app, state, notebook_url):
 
         notebook_url (str) :
 
+        port (int) : the port the embedded server will listen on. By default
+        the port is 0, which results in the server listening on a random dynamic port.
+
     Returns:
         None
 
@@ -452,7 +455,7 @@ def show_app(app, state, notebook_url):
     loop = IOLoop.current()
 
     origin = _origin_url(notebook_url)
-    server = Server({"/": app}, io_loop=loop, port=0,  allow_websocket_origin=[origin])
+    server = Server({"/": app}, io_loop=loop, port=port,  allow_websocket_origin=[origin])
 
     server_id = uuid4().hex
     curstate().uuid_to_server[server_id] = server


Test patch: