Repository: bokeh/bokeh

Issue:
Permission Denied error when invoking show() from inside Jupyter Notebook
Hi,

I'm using bokeh 0.12.3 and jupyter 1.0.0. I'm writing some code that produces a simple bar chart and calls show() on this bar chart inside the notebook.

Here's the backtrace that I get:
```
PermissionError                           Traceback (most recent call last)
<ipython-input-3-e94210215dfe> in <module>()
      1 results = r._get_results()
----> 2 v.visualize(results, None, None)

/home/lukas/src/tcpsp/TCPSPsuite/scripts/visualizer.py in visualize(self, results, x_axis, y_axis)
     11         chart = Bar(df, 'job_count', values='elapsed', agg='mean')
     12 
---> 13         show(chart)

/home/lukas/.virtualenvs/tcpsp/lib/python3.5/site-packages/bokeh/io.py in show(obj, browser, new, notebook_handle)
    312     if obj not in _state.document.roots:
    313         _state.document.add_root(obj)
--> 314     return _show_with_state(obj, _state, browser, new, notebook_handle=notebook_handle)
    315 
    316 

/home/lukas/.virtualenvs/tcpsp/lib/python3.5/site-packages/bokeh/io.py in _show_with_state(obj, state, browser, new, notebook_handle)
    330 
    331     if state.file or not shown:
--> 332         _show_file_with_state(obj, state, new, controller)
    333 
    334     return comms_handle

/home/lukas/.virtualenvs/tcpsp/lib/python3.5/site-packages/bokeh/io.py in _show_file_with_state(obj, state, new, controller)
    335 
    336 def _show_file_with_state(obj, state, new, controller):
--> 337     filename = save(obj, state=state)
    338     controller.open("file://" + filename, new=_new_param[new])
    339 

/home/lukas/.virtualenvs/tcpsp/lib/python3.5/site-packages/bokeh/io.py in save(obj, filename, resources, title, state, validate)
    394 
    395     filename, resources, title = _get_save_args(state, filename, resources, title)
--> 396     _save_helper(obj, filename, resources, title, validate)
    397     return os.path.abspath(filename)
    398 

/home/lukas/.virtualenvs/tcpsp/lib/python3.5/site-packages/bokeh/io.py in _save_helper(obj, filename, resources, title, validate)
    462         html = standalone_html_page_for_models(obj, resources, title)
    463 
--> 464         with io.open(filename, "w", encoding="utf-8") as f:
    465             f.write(decode_utf8(html))
    466 

PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.5/runpy.html'
```

It seems like I'm not the only one experiencing the problem: see for example https://github.com/yhat/rodeo/issues/211

I'm pretty sure the problem is in _detect_filename() in io.py: If no filename to save a chart to is supplied, it tries to save the file (with an autodetected filename) into the directory of the currently executing file (more exactly: the file executing in the topmost stackframe, which is assumed to be the main file being run.) This fails spectacularly if that file is located in a read-only location as is the case when you run an jupyter notebook, which on my system seems to use "/usr/lib/python3.5/runpy.py" to run the notebook.

I think the cleanest solution would be to use a temporary directory if no save-to directory is specified. If no directory is specified, the user is probably not really interested in the file, but only needs it temporarily: This happens in the show() method, for example.

Is there a reason for saving temporary files in the directory of the script? In that case, should the save() method be amended by a "temporary" argument (with default False), which in turn tells _detect_filename() to use a temporary directory? In that case, show() could set that flag and avoid the permission problem…?

I'd be willing to cook up a pull request if I get any feedback on how you think this problem should be solved.


Discussion:
Hi, can you provide a complete sample code? It's hard to say anything concrete when speculation is required. For instance, the code path taken suggests you have executed `ouptut_file` at some point (which would explain things). Is this the case? Providing a complete minimal code sample is the most efficient way to help us help you. 
D'Oh, the problem does actually not arise when output_notebook() is properly called, since then save() seems not to be called at all… Sorry, that was kinda stupid on my side.

However, I'm still not sure whether to consider this a bug or not: With output_file(), it is probably safe to assume that the caller of show() is interested in the file. Still, just assuming that it should be put into the directory of the topmost executing file…? I do not have a better idea, though. I'll close this bug (since it works for notebooks when you set them up properly).
@tinloaf no worries, I will re-open this for discussion because I do think you have a possible valid issue about assumed file location. 

Ping @philippjfr (I think you added this feature?) for thoughts?
@bryevdv I'm afraid I have no real idea what's going on here. I don't think I ever touched anything to do with file saving or file locations. Maybe I just don't understand the actual issue.
@philippjfr no worries, someone added a feature to save to an implicit filename and I misremembered it as you. I think it may have actually been @almarklein 
We may need to add additional clause in `_detetect_filename()`, something along `os.access(parent_dir, os.W_OK | os.X_OK)`.
Also file name detection magic could be disabled in IPython. This can be detected with `if IPython is None or IPython.get_ipython() is None:`. Alternatively we could use `join(os.getcwd(), "unnamed.%s" % ext)`. @bryevdv, any preference?
I'm honestly not very fond of the feature at all so I am probably not the best person to ask. @almarklein do you have an opinion about @mattpap question?
A bit late to the party, sorry; I still get Bokeh emails, but they get in a seperate folder which I stopped checking.

@bryevdv yes it was @mattpap and me who introduced this feature. I'm surprised that we don't have some sort of test for writability of the file name. Maybe we should test that the filename is relative to `os.path.expanduser('~')`, and use the temp dir otherwise.

As for @mattpap's questions, I don't really follow, can you provide some context?


@tinloaf A PR would be appreciated, are you still interested in working on one? 

My personal preference would just be to always use a writable temp file. I think the observation that if someone is calling `show` without `output_file`, they they probably just want to see the result in a browser, and don't really care about a file, is correct. I think that behavior would actually fill in a capability gap that currently exists. This approach would also be simple to implement, explain, and avoid magic. 

@mattpap @almarklein does that sound ok? I think wanting `show` to "just work" i.e. open a browser with the output without a filename is reasonable. But I also think if you *do* want the file, it is not too much to require that the filename be specified to mark that interest (i.e. with `output_file`). 
I completely agree.
@tinloaf I think we have agreement (always use tmp file) Are you still interested in working up a PR?
Hi. Sorry, didn't check on this issue for a while. Il'll cook comething up! This shouldn't be to complicated.
@bryevdv I can't recreate this anymore. Possibly it got fixed as collateral on another PR. 
@jsignell are you sure the destination for the output file was to a read-only location? I'd be surprised if any recent work resolved this. 
@bryevdv I see now. 

Resolving patch:
diff --git a/bokeh/io.py b/bokeh/io.py
--- a/bokeh/io.py
+++ b/bokeh/io.py
@@ -22,6 +22,7 @@
 import json
 import os
 import warnings
+import tempfile
 
 # Third-party imports
 
@@ -390,19 +391,23 @@ def save(obj, filename=None, resources=None, title=None, state=None, validate=Tr
 
 def _detect_filename(ext):
     """ Detect filename from the name of the script being run. Returns
-    None if the script could not be found (e.g. interactive mode).
+    temporary file if the script could not be found or the location of the
+    script does not have write permission (e.g. interactive mode).
     """
     import inspect
-    from os.path import isfile, dirname, basename, splitext, join
+    from os.path import dirname, basename, splitext, join, curdir
 
     frame = inspect.currentframe()
     while frame.f_back and frame.f_globals.get('name') != '__main__':
         frame = frame.f_back
 
     filename = frame.f_globals.get('__file__')
-    if filename and isfile(filename):
-        name, _ = splitext(basename(filename))
-        return join(dirname(filename), name + "." + ext)
+
+    if filename is None or not os.access(dirname(filename) or curdir, os.W_OK | os.X_OK):
+        return tempfile.NamedTemporaryFile(suffix="." + ext).name
+
+    name, _ = splitext(basename(filename))
+    return join(dirname(filename), name + "." + ext)
 
 def _get_save_args(state, filename, resources, title):
     warn = True
@@ -414,9 +419,6 @@ def _get_save_args(state, filename, resources, title):
         warn = False
         filename = _detect_filename("html")
 
-    if filename is None:
-        raise RuntimeError("save() called but no filename was supplied or detected, and output_file(...) was never called, nothing saved")
-
     if resources is None and state.file:
         resources = state.file['resources']
 


Test patch: