Repository: bokeh/bokeh

Issue:
Headless static (svg, png) image generation 
Enable generation of PNG images directly on the server, similar to the Vega.js headless mode (https://github.com/trifacta/vega/wiki/Headless-Mode).



Discussion:
@mattpap do you have any thoughts about what it would take to expose the phantomjs PNG generation we do with the test script to users? 

But using the phantom approach we have an image with the buttons and uncentered... so probably we need to pre-process the plot to render properly... just thoughts....

Right it won't work as-is. I can think of a few possible ways to use phantoms:
- allow automation of the existing preview/save tool to save the image buffer directly to disk without having the user have to navigate a file save dialog.
- putting the plot in a different template/div layout for headless image generation, that does not have the toolbar, etc., and using the current "screenshot" approach that the test script uses to save images.

There may be others too.

**Update**: it does look as though [node-canvas](https://github.com/Automattic/node-canvas) may work for this, I am going to start some preliminary experimentation. 

Initial experiments on OS X are encouraging that `node-canvas` should be suitable, even for plots with image data. However, installing `node-canvas` and its dependencies is not trivial and will take some work to make it as simple and easy as possible for users. 

Getting BokehJS working in node.js is a first prerequisite, I have made a separate issue for that task, 

we should rename this issue - there is no reason to involve the server at all here right?

@hhuuggoo, the idea is to be able to produce plots on the server rather than having them rendered in the browser. This can be useful when you need to generate plots to be used in some context other than display in the browser (e.g., generation of a PDF report).

@abastardi yes but in general you will be able to generate images from python, whether that's in the server, or in an interactive session or IPython notebook makes no difference.

Got it. It sounded like @hhuuggoo meant this shouldn't happen on the server.

I see that .8 has shipped but this issue is still open. Is there an update as to the disposition of this capability? 

@scherrey Unfortunately not. Here is a message I sent recently the Bokeh mailing list on 2015/03/04:

---

Programmatic generation of static images is definitely something we would very much like to support. Unfortunately, it is also a surprisingly difficult proposition. The first observation is that browsers canvas implementation only generate raster images, and don't help with SVG or PDF. Worse, browser security models mean that getting the data out is also very problematic. You can follow this history of this topic in a few issues:

https://github.com/bokeh/bokeh/issues/538
https://github.com/bokeh/bokeh/issues/1589

MPL support is currently provided by a third party library, `mplexporter`. It is very useful in a large number of cases, but it is by no means complete, which explains errors you can run into with some MPL figures. There is an MPL enhancement proposal to make a "serious" complete JSON serialization layer, much better than `mplexorter`. In principle this would work fantastically, allowing us to leverage all the MPL work for static image generation. But realistically, there are no indications that this work will materialize or even really get started any time in the near future (best guess: years, if at all). The most recent plan was to try and use the BokehJS patched with `node-canvas` to generate images inside Node. But BokehJS is a large and complicated library and it is unclear that Node support is currently up to the task. Experiments and tests are ongoing, but not encouraging. In the past we have also tried PhantomJS, but the lack of support for typed arrays in PhantomJS means that it too is not up to the task (besides the fact that it also would only generate raster images).

If the `node-canvas` plan proves unworkable, I am afraid that really only leaves the very undesirable proposition of creating and maintaining our own "duplicate" rendering code path in python that mirrors the BokehJS rendering code, and uses something like Cairo or Agg directly to render static images. This is obviously bad from an effort/maintenance viewpoint, but it's also disappointing because it means static image generation won't "just work" for other language bindings without including some python code or a python tool. However, given that we are probably going to make a move to WebGL for browser rendering, this may truly be the "best worst" possibility. But obviously it is a nontrivial undertaking that will take a nontrivial amount of time and effort. As of yet, nothing concrete in this direction is scheduled.

I wish I had some better news to give you. It is my earnest hope that we will have a solution in 2015 but I can't provide any better estimates at this time. 

I'm probably out of my depth here, but would it be easier and nearly as useful to generate SVGs via a javascript library in the browser? My main concern with adopting bokeh for analysis and plotting would be that I eventually need to produce publication-ready or presentation-ready graphics. To minimize file sizes and maximize quality, I prefer to work with vector formats (eps, pdf or possibly svg). But as I understand it, bokeh cannot currently produce vector graphics. This would be a barrier for me, but I don't mind much whether the impasse is solved on the server side or browser side. e.g., I would be satisfied if I could develop plots in an IPython notebook and then export them as vector graphics from there.

In a quick search on this subject, I came across several javascript libraries that can generate SVG output using similar calls to drawing on an HTML5 canvas (which I assume is what bokeh does). I haven't investigated these very far, but I think they work either by accepting standard canvas drawing commands and then translating them into SVG elements, or they provide an alternative API which can be used both for drawing on the canvas and generating SVGs. Here are some of the key libraries I found:

[Fabric.js](http://fabricjs.com/) was recommended in a couple of stackoverflow posts on this subject ([1](http://stackoverflow.com/questions/8571294/method-to-convert-html5-canvas-to-svg), [2](http://stackoverflow.com/questions/10273516/get-svg-from-canvas-element-and-save-it)). It seems to be one of the most mature options.

[canvas-svg](http://code.google.com/p/canvas-svg/) and [canvas2svg](https://github.com/gliffy/canvas2svg) were also mentioned in those stackoverflow posts but seem to be less mature.

[two.js](http://jonobr1.github.io/two.js/) offers what may be a slightly different path forward -- a single API for drawing either to the canvas or to SVGs. I imagine this would block the two-way communication needed by Bokeh.

Another option, hinted at in yet another [stackoverflow post](http://stackoverflow.com/questions/9547497/can-my-html5-canvas-code-be-converted-to-svg) might be to write graphs in the browser as SVG instead of canvas drawings. This would provide natural support for generating SVG files. This would also use a higher level interface than the canvas, which might simplify the interaction code (e.g., vertices could be native, clickable objects in the browser). Of course this would require a huge rewrite of the front end.

Hi @mfripp thanks for the researched and informative post. The landscape has definitely changed and we are in fact  already planning to look into some of those options mentioned. However, by themselves those libraries don't solve the problem. People want to be able to do `save(plot)` in a python script on a headless server, and get and SVG file, without ever interacting with a browser. Any solution that requires running a browser and having a user actually push a button solves a different problem than this issue. 

For headless output with JS libraries to work, Bokeh would need to be able to be run in Node.js or similar. But last time we tried, BokehJS was too big and unwieldy and we could not get it to run. So a possibility is to split up BokehJS so that (hopefully) a minimal part can be to run in Node.js (just enough to generate . And in fact, this absolutely must happen for a variety of reason, and so work is under way. But that is a very difficult architectural refactor to make that happen in a way that is as transparent and as least disruptive as possible. It will probably not be complete for a few months, 

Completely unmentioned yet is that whatever solution we adopt of this has implications for being able to display math text in the saved output. Depending on the route we take, perhaps we have to give that up completely. To summarize, the core of the problem is that there are actually a few big pieces that interact in non-trivial ways. Some of the pieces have several possible options, none of which are very good or pleasant in some way. Or they may be blocked by major work in another area (splitting up BokehJS). 

Hi @bryevdv, thanks for your comments on this. Sorry for the slow response. I've been thinking a little more about this, but didn't get a chance to post until now.

From the user's perspective I can see three new features here:
1. Produce vector graphics files from BokehJS.
2. Produce graphics files from BokehJS under program control.
3. Run BokehJS entirely on the "server" side (i.e., without a full web browser).

These are mostly independent, because any code written to provide features 1 and 2 in a web browser should also run in a server-side javascript environment (since you'll need a pretty robust javascript environment already).

For the needs of scientific users, I would rank 1 as very important, 2 as moderately important and 3 as interesting but not particularly important. I would also say that feature 3 is not useful without feature 2, and feature 2 is not useful without feature 1 (other users may want to produce raster images of plots programmatically, but for scientific applications, users will immediately want vector graphics).

Given that any code you write to implement features 1 and 2 in a full web browser would also be usable in a server-side environment, why not start with 1 and 2 in a web browser and then move them to the server environment along with the rest of BokehJS later? 

This would be an especially attractive approach since it will take an unknown length of time to get BokehJS running on the server side. i.e., the high-priority features must work in a browser, can be implemented more easily in a browser, and can be migrated eventually from the browser to the server, so why wait for the server implementation before developing them?

@mfripp They are somewhat independent. However, BokehJS is already _very_ large. Adding in yet another dependency and chunk of code to the current monolithic BokehJS, for a feature that is not actually the one that is most often requested [1] is not something I can justify. Once BokehJS is split up (we are about to have a kickoff on that front) it becomes more reasonable to think about adding an in-browser capability as an _optional_ component, that users only have to pay for if they use. This is even more true if a new contributor wanted to help push on that front. (One of the biggest motivations for splitting up BokehJS is to enable outside contributions that can be developed independently and still integrated easily.)

[1]  **by far** most users ask us for option 2: programmatic, headless generation. This is by any metric I can think of, the higher priority feature. 

Hi bokeh!

i've been [working](https://github.com/Anaconda-Server/nbpresent/pull/16) [on](https://github.com/Anaconda-Server/anaconda-notebook/issues/43) [nbpresent](https://github.com/Anaconda-Server/nbpresent), and have made some decent progress on headless bokeh rendering in notebooks.

Here's a [sample output](https://github.com/Anaconda-Server/nbpresent/files/37500/output.pdf). Actually, it's the first time it worked, fully.

I have given up on phantomjs in favor of [ghost.py](http://jeanphix.me/Ghost.py/), which gives much finer-grained access to the underlying qt api.

Glad for any feedback... and maybe some thoughts on how to crank up the resolution on bokeh renders, as I am really gunning for print-quality output (>600dpi). I am totally fine with adding bokeh-specific fix-all lines, or giving users ways to apply specific patches!

We actually have an open PR already: https://github.com/bokeh/bokeh/pull/3047

This approach is a start to using node.js to create static raster images but it is hoped that we can leverage canvas-cairo to produce SVGs. If not, I think we will resort to implementing an alternative partial pycairo-based backed. Long term we would like to use MPL as a backend as well. 

Being completely honest, I am personally fairly negative on the long term utility of "screenshot" approaches like the one above, I think they will be fragile by nature and of course limited to raster images in any case. 

Great, glad that's working out! Will there be a way to access the vector format in the frontend, a la `output_svg`? That would certainly nail my resolution issue!

The eventual goal is programmatic generation of SVG. That may only be available as a saved file, it's too early to say. 

Any news regarding this to share?

I made a prototype (#3867) but other things are shuffling into place before we finish it.

Brainstorming here but what about supporting png & jpeg creation by compiling those open source C libraries to asm.js and running in phantomjs? We've done a little bit of C++ coding that ran under asm.js with good result. Any ideas why this wouldn't work? 

@scherrey I think we want a solution that would also work without phantomjs - there's kind of a lot going on in this issue though, sowe may be talking about different things.

@scherrey you may be right - I'm doing some experiments with phantomjs at the moment.

@bryevdv pointed out that phantomjs won't be a solution until it supports TypedArrays. I'm poking around the currently in-development headless chrome.

Hi Team,

I really love bokeh and am currently using it to host a grid of plots 6x11. I can load the first 5 plots just fine, clicking 1 plot at a time but then my browser (chrome) crashes if I try to open any other plot. It would be really great to be able to export the plots to png without the need of the 'save' tool. At this point I can try to create a hack by trying to use some third party tool (phantomjs) or just switch to another package (matplotlib, etc), which I don't really want to do because I really really like Bokeh :(

I see that this thread is over 2 years old... so it is safe to assume your team has no real interest in providing this feature?

Thanks for your time.

-- Grateful Bokeh User

@tux2014, I'm confused, the last comment on this thread was 18 days ago.

Also "your team" seems like a bit of a misconception. This is open source. You are part of the team too. We welcome your help in getting this feature done.

> I see that this thread is over 2 years old... so it is safe to assume your team has no real interest in providing this feature?

No that is not a safe assumption at all. What is actually the case, is that it is extremely difficult from a technical perspective, with lots of potentially unpleasant but also unavoidable trade-offs, and  that it competes with many other very high priorities for extremely limited human resources, from a project management perspective. 

+1 this would be so useful

+1 The lack of static (SVG) image support is the main reason I'm not using bokeh right now.
+1 Needed here as well
Any updates to share? I would appreciate a sort of roadmap of what needs to be done before this is put into place. 
Please please please =)

SVG and PNG output would be amazing
I believe chrome headless is going to be the right way to do this and that project is moving very quickly with decent linux support already. There is a lack of resources on this at the moment, but hopefully that will be changing in the coming months.
I don't know the current state of using headless chrome, but [wkhtmltopdf](http://wkhtmltopdf.org/) [(github)](https://github.com/wkhtmltopdf/wkhtmltopdf) could be another option. It also may be useful in the short term for people to use as a workaround; I recently used it to programmatically generate a few hundred bokeh pdfs. I realize this issue is titled `svg, png` but I *think* it can also do pngs and maybe [this](http://stackoverflow.com/a/10290006/2561747) could be useful for svgs.

Current Limitations:
* For Bokeh plots to render the 0.13 [alpha version](http://wkhtmltopdf.org/downloads.html) is [required](https://github.com/wkhtmltopdf/wkhtmltopdf/issues/3211).
* This version so far only supports windows and linux (using `xvfb`, not fully headless yet).

After downloading the alpha build, I was able to use the following to generate a pdf:

`wkhtmltopdf --no-stop-slow-scripts --javascript-delay 3000 http://bokeh.pydata.org/en/latest/docs/gallery/image_rgba.html image_rgba_out.pdf`

It looks like there's also a handy python wrapper called [pdfkit](https://pypi.python.org/pypi/pdfkit) [(github)](https://github.com/JazzCore/python-pdfkit).

Resolving patch:
diff --git a/bokeh/io.py b/bokeh/io.py
--- a/bokeh/io.py
+++ b/bokeh/io.py
@@ -25,14 +25,14 @@
 import tempfile
 import uuid
 
-# Third-party imports
-
 # Bokeh imports
 from .core.state import State
 from .document import Document
 from .embed import autoload_server, notebook_div, file_html
 from .layouts import gridplot, GridSpec ; gridplot, GridSpec
+from .resources import INLINE
 import bokeh.util.browser as browserlib  # full import needed for test mocking to work
+from .util.dependencies import import_required, detect_phantomjs
 from .util.deprecation import deprecated
 from .util.notebook import get_comms, load_notebook, publish_display_data, watch_server_cells
 from .util.string import decode_utf8
@@ -564,3 +564,94 @@ def _destroy_server(div_id):
 
     except Exception as e:
         logger.debug("Could not destroy server for id %r: %s" % (div_id, e))
+
+def _crop_image(image, left=0, top=0, right=0, bottom=0, **kwargs):
+    '''Crop the border from the layout'''
+    cropped_image = image.crop((left, top, right, bottom))
+
+    return cropped_image
+
+def _get_screenshot_as_png(obj):
+    webdriver = import_required('selenium.webdriver',
+                                'To use bokeh.io.export you need selenium ' +
+                                '("conda install -c bokeh selenium" or "pip install selenium")')
+    from selenium.webdriver.support.ui import WebDriverWait
+    from selenium.common.exceptions import TimeoutException
+
+    Image = import_required('PIL.Image',
+                            'To use bokeh.io.export you need pillow ' +
+                            '("conda install pillow" or "pip install pillow")')
+    # assert that phantomjs is in path for webdriver
+    detect_phantomjs()
+
+    html_path = tempfile.NamedTemporaryFile(suffix=".html").name
+    save(obj, filename=html_path, resources=INLINE, title="")
+
+    driver = webdriver.PhantomJS()
+    driver.get("file:///" + html_path)
+    script = """
+        // override body width CSS for PhantomJS compat
+        document.body.style.width = '100%';
+        // add private window prop to check that render is complete
+        window._bokeh_render_complete = false;
+        window.addEventListener("bokeh:rendered", function() {
+            window._bokeh_render_complete = true;
+        });
+        """
+    driver.execute_script(script)
+
+    def is_bokeh_render_complete(driver):
+        return driver.execute_script('return window._bokeh_render_complete;')
+
+    try:
+        WebDriverWait(driver, 5, poll_frequency=0.1).until(is_bokeh_render_complete)
+    except TimeoutException:
+        logger.warn("The webdriver raised a TimeoutException while waiting for \
+                     a 'bokeh:rendered' event to signify that the layout has rendered. \
+                     Something may have gone wrong.")
+    finally:
+        browser_logs = driver.get_log('browser')
+        severe_errors = [l for l in browser_logs if l.get('level') == 'SEVERE']
+        if len(severe_errors) > 0:
+            logger.warn("There were severe browser errors that may have affected your export: {}".format(severe_errors))
+
+    png = driver.get_screenshot_as_png()
+
+    bounding_rect_script = "return document.getElementsByClassName('bk-root')[0].children[0].getBoundingClientRect()"
+    b_rect = driver.execute_script(bounding_rect_script)
+
+    driver.quit()
+
+    image = Image.open(io.BytesIO(png))
+    cropped_image = _crop_image(image, **b_rect)
+
+    return cropped_image
+
+def export(obj, filename=None):
+    ''' Export the LayoutDOM object as a PNG.
+
+    If the filename is not given, it is derived from the script name
+    (e.g. ``/foo/myplot.py`` will create ``/foo/myplot.png``)
+
+    Args:
+        obj (LayoutDOM object) : a Layout (Row/Column), Plot or Widget object to display
+
+        filename (str, optional) : filename to save document under (default: None)
+            If None, infer from the filename.
+
+    Returns:
+        filename (str) : the filename where the static file is saved.
+
+    .. warning::
+        Responsive sizing_modes may generate layouts with unexpected size and
+        aspect ratios. It is recommended to use the default ``fixed`` sizing mode.
+
+    '''
+    image = _get_screenshot_as_png(obj)
+
+    if filename is None:
+        filename = _detect_filename("png")
+
+    image.save(filename)
+
+    return os.path.abspath(filename)
diff --git a/bokeh/settings.py b/bokeh/settings.py
--- a/bokeh/settings.py
+++ b/bokeh/settings.py
@@ -229,6 +229,9 @@ def css_files(self):
     def nodejs_path(self, default=None):
         return self._get_str("NODEJS_PATH", default)
 
+    def phantomjs_path(self, default=None):
+        return self._get_str("PHANTOMJS_PATH", default)
+
 #: A global settings object that other parts of Bokeh can refer to.
 #:
 #:
diff --git a/bokeh/util/dependencies.py b/bokeh/util/dependencies.py
--- a/bokeh/util/dependencies.py
+++ b/bokeh/util/dependencies.py
@@ -3,7 +3,9 @@
 '''
 from importlib import import_module
 import logging
+from subprocess import Popen, PIPE
 
+from ..settings import settings
 
 logger = logging.getLogger(__name__)
 
@@ -48,3 +50,19 @@ def import_required(mod_name, error_msg):
         return import_module(mod_name)
     except ImportError:
         raise RuntimeError(error_msg)
+
+def detect_phantomjs():
+    '''Detect if PhantomJS is avaiable in PATH.'''
+    if settings.phantomjs_path() is not None:
+        phantomjs_path = settings.phantomjs_path()
+    else:
+        phantomjs_path = "phantomjs"
+
+    try:
+        proc = Popen([phantomjs_path, "--version"], stdout=PIPE, stderr=PIPE)
+        proc.wait()
+    except OSError:
+        raise RuntimeError('PhantomJS is not present in PATH. Try "conda install phantomjs" or \
+                           "npm install -g phantomjs-prebuilt"')
+    else:
+        return phantomjs_path


Test patch:
diff --git a/bokeh/tests/test_io.py b/bokeh/tests/test_io.py
--- a/bokeh/tests/test_io.py
+++ b/bokeh/tests/test_io.py
@@ -8,12 +8,14 @@
 
 from __future__ import absolute_import
 from mock import patch, Mock, PropertyMock
+from PIL import Image
 import pytest
 import unittest
 
 import bokeh.io as io
 from bokeh.resources import Resources
 from bokeh.models.plots import Plot
+from bokeh.models import Range1d
 
 class TestDefaultState(unittest.TestCase):
 
@@ -293,3 +295,20 @@ def _test_children_removed_from_root(layout_generator, children=None):
     layout_generator(component if children is None else children)
     assert component not in io.curdoc().roots
     io.curdoc().clear()
+
+def test__crop_image():
+    image = Image.new(mode="RGBA", size=(10,10))
+    rect = dict(left=2, right=8, top=3, bottom=7)
+    cropped = io._crop_image(image, **rect)
+    assert cropped.size == (6,4)
+
+def test__get_screenshot_as_png():
+    layout = Plot(x_range=Range1d(), y_range=Range1d(),
+                  plot_height=2, plot_width=2, toolbar_location=None,
+                  outline_line_color=None, background_fill_color=None,
+                  border_fill_color=None)
+
+    png = io._get_screenshot_as_png(layout)
+    assert png.size == (2, 2)
+    # a 2x2px image of transparent pixels
+    assert png.tobytes() == b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
diff --git a/bokeh/util/tests/test_dependencies.py b/bokeh/util/tests/test_dependencies.py
--- a/bokeh/util/tests/test_dependencies.py
+++ b/bokeh/util/tests/test_dependencies.py
@@ -14,3 +14,11 @@ def test_required_fail():
     with pytest.raises(RuntimeError) as excinfo:
         dep.import_required('bleepbloop', 'nope')
     assert 'nope' in str(excinfo.value)
+
+def test_detect_phantomjs_success():
+    assert dep.detect_phantomjs() is not None
+
+def test_detect_phantomjs_fail(monkeypatch):
+    monkeypatch.setenv("BOKEH_PHANTOMJS_PATH", "bad_path")
+    with pytest.raises(RuntimeError):
+        dep.detect_phantomjs()