Repository: bokeh/bokeh

Issue:
export_png not exporting figures correctly
This is a continuation of #7621

export_png (and export_svgs for that matter) do not work as expected when attempting to export figures after they were added to ~~a layout~~ `curdoc()`.

Here is the example code:

```python
# -*- coding: utf-8 -*-
'''
After the layout has been added to the root, exporting the png of a fig is
not possible anymore
With "bokeh serve export_test.py", the console output for me is the following:
doc_layout works
fig works
Layout is added to root
doc_layout works
fig throws error
'''
from bokeh.io import curdoc
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.layouts import layout, widgetbox
from bokeh.io import export_png
from bokeh.models.widgets import Button


def download():
    try:
        export_png(doc_layout, filename="export doc_layout.png")
        print('doc_layout works')
    except Exception as ex:
        print('doc_layout throws error:', ex)
    try:
        export_png(fig, filename="export fig.png")
        print('fig works')
    except Exception as ex:
        print('fig throws error:', ex)


source = ColumnDataSource({'x': [2, 3, 4], 'y': [0.5, 1.5, 2.5]})
fig = figure(plot_width=1000, plot_height=300)
fig.hbar(y='y', right='x', height=0.4, source=source)

button = Button(label="Download")
button.on_click(download)
doc_layout = layout([[widgetbox(button)], [fig]], sizing_mode='fixed')

#download()  # Test 1

print('Layout is added to root')
curdoc().add_root(doc_layout)
curdoc().title = "Export PNG test"

#download()  # Test 2
```

I start the server with `bokeh serve export_test.py`. After I click the Download button, the following output is generated:

```
You are generating standalone HTML/JS output, but trying to use real Python
callbacks (i.e. with on_change or on_event). This combination cannot work.

Only JavaScript callbacks may be used with standalone output. For more
information on JavaScript callbacks with Bokeh, see:

    http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/callbacks.html

Alternatively, to use real Python callbacks, a Bokeh server application may
be used. For more information on building and running Bokeh applications, see:

    http://bokeh.pydata.org/en/latest/docs/user_guide/server.html

doc_layout works
fig throws error: Message: {"errorMessage":"undefined is not an object (evaluating 'document.getElementsByClassName('bk-root')[0].children[0].getBoundingClientRect')","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"171","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:62171","User-Agent":"selenium/3.12.0 (python linux)"},"httpVersion":"1.1","method":"POST","post":"{\"script\": \"\\nreturn document.getElementsByClassName('bk-root')[0].children[0].getBoundingClientRect()\\n\", \"args\": [], \"sessionId\": \"5f2d8590-7620-11e8-94ca-bf888776a2b5\"}","url":"/execute","urlParsed":{"anchor":"","query":"","file":"execute","directory":"/","path":"/execute","relative":"/execute","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/execute","queryKey":{},"chunks":["execute"]},"urlOriginal":"/session/5f2d8590-7620-11e8-94ca-bf888776a2b5/execute"}}
Screenshot: available via screen
```
If I comment out the line `download()  # Test 1`, the correct png is generated. This is only possible before `curdoc().add_root(doc_layout)`.
I still cannot save the png of a figure, once it has been included in a layout and added to root. 
Am I doing someting wrong?
Does it have something to do with the new standalone HTML warning?

Software:
Bokeh = 0.13.0
Tornado = 5.0.2
Selenium = 3.12.0
Pillow = 5.1.0



Discussion:
@AzraelDD to be clear, this is not really a continuation of #7621, and the problem has nothing to to with things being added to any layout. 

The issue is that Bokeh `0.13.0` added an error condition and detailed exception when a user tries to use `on_change` in a standalone document (i.e. *not* in a Bokeh server app). The support burden of answering users who tried to do this and then asked "why doesn't on_change work?" had become entirely too heavy. This was an attempt to provide immediate, actionable feedback to users trying to use `on_change` outside of a Bokeh server app. 

Unfortunately, it was not appreciated at the time that the mechanism for `export_png` is essentially to create a standalone document for a headless browser to render and take a screenshot of! This is always the way export works (even in a Bokeh server app), and so that process unintentionally triggers the new error message (even in a bokeh server app).

There are two options:

* remove the new error check entirely, and continue to deal with the tide of "why doesn't this work" questions with ongoing human labor, or

* find a way to special case things with `export_png` so that the error check is short-circuited only during exports

Since "human labor" is basically just me and my time, I'd obviously prefer to find a way to make the latter work. 
Dear @bryevdv, I am sorry if there is something I do not understand. But while I think that I get what you are trying to say, it does not match what I am currently experiencing. Consider this reduced piece of code from the example above:

```python
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.layouts import layout, widgetbox
from bokeh.io import export_png
from bokeh.models.widgets import Button


def download():
    try:
        export_png(doc_layout, filename="export doc_layout.png")
        print('doc_layout works')
    except Exception as ex:
        print('doc_layout throws error:', ex)
    try:
        export_png(fig, filename="export fig.png")
        print('fig works')
    except Exception as ex:
        print('fig throws error:', ex)


source = ColumnDataSource({'x': [2, 3, 4], 'y': [0.5, 1.5, 2.5]})
fig = figure(plot_width=1000, plot_height=300)
fig.hbar(y='y', right='x', height=0.4, source=source)

button = Button(label="Download")
button.on_click(download)
doc_layout = layout([[widgetbox(button)], [fig]], sizing_mode='fixed')

download()  # Test 1
```

Then do `bokeh serve export_test.py`. Then connect to the server with your browser. It creates the output:
```
You are generating standalone HTML/JS output, but trying to use real Python
callbacks (i.e. with on_change or on_event). This combination cannot work.

Only JavaScript callbacks may be used with standalone output. For more
information on JavaScript callbacks with Bokeh, see:

    http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/callbacks.html

Alternatively, to use real Python callbacks, a Bokeh server application may
be used. For more information on building and running Bokeh applications, see:

    http://bokeh.pydata.org/en/latest/docs/user_guide/server.html

doc_layout works
fig works
```

Despite your "standalone HTML" warning, exporting both the layout and the figure works. The expected files are generated. Since `curdoc()` is not even referred to, nothing is visible in the broswer, but that is expected.
My point is:  The "standalone HTML" warning does not prevent `export_png` from working correctly. It is irritating in this case, but nothing more.


While you said that it has nothing to do with the problem, the only line of code that causes `export_png` to fail is `curdoc().add_root(doc_layout)` - at least from my point of view.

Following that, from my point of view there are two seperate issues:

-  A useful "standalone HTML" warning, that is irritating in this special case but does no harm
- `export_png` not working under some conditions

Since I trust your explanation, I expect to have a misunderstanding in the above train of thought. But reiterating my experience seems to be the only helpful thing I could do.


A related question that might help clear things up: Is the "standalone HTML" message intended as a) an error (preventing all following code from working) or as b) a warning (not interfering with the rest of the code)?
To me it seems like b) is the case. But from your explanation I feel like a) was intended.

Partly I think you are using a definition I don't recognize. Adding things to the document (using `curdoc().add_root`) is not what I would think of when "adding to a layout", which I would only consider to be adding to something in `bokeh.layouts` e.g. a `row` or `column`. Also this is the relevant error message, that was not included:

> fig throws error: Message: {"errorMessage":"undefined is not an object (evaluating 'document.getElementsByClassName('bk-root')[0].children[0].getBoundingClientRect')","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"171","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:49953","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"script\": \"\\nreturn document.getElementsByClassName('bk-root')[0].children[0].getBoundingClientRect()\\n\", \"args\": [], \"sessionId\": \"060e11a0-788c-11e8-83b8-2779613502fa\"}","url":"/execute","urlParsed":{"anchor":"","query":"","file":"execute","directory":"/","path":"/execute","relative":"/execute","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/execute","queryKey":{},"chunks":["execute"]},"urlOriginal":"/session/060e11a0-788c-11e8-83b8-2779613502fa/execute"}}
Screenshot: available via screen

@mattpap when you are back next week, my offhand guess is this has to do with the DOM structure changes for the custom HTML template work. Perhaps the `export_png` code for the headless browser needs some kind of update. 

In any case this is not the same issue as before, even if it presents a similar outcome. 
Lets simplify the code a bit:
```py
export_png(doc_layout, filename="export doc_layout.png") # works
export_png(fig, filename="export fig.png")               # works

curdoc().add_root(doc_layout)
# navigate

export_png(doc_layout, filename="export doc_layout.png") # works
export_png(fig, filename="export fig.png")               # fails
```
The difference between the former and the later set of `export_png()` calls, is that the former doesn't have a document assigned to models, whereas the later has. This results in the later `export_png(fig)` being understood as (~) `export_png(fig.document.roots)`, which results in a mismatch between roots and their assigned element ids and `fig`. I think we should simply disallow usage of sub-models as roots after a document is assigned. One can always clone a model to start from scratch and avoid confusion. An alternative we could ignore the document in certain scenarios, but I think that would be too messy. Perhaps we may need to introduce a concept of (lets call it) bound and unbound models to further clarify what happens when a document is assigned and what when it's not.
Dear @mattpap,
you hinted at
> One can always clone a model to start from scratch and avoid confusion.

as a workaround, if I understand correctly. I tried some (stupid, I guess) stuff like
```python
fig_copy = figure()
fig_copy._property_values = fig._property_values
```
which always results in errors like:
`fig throws error: Sub-model (id='f8b61051-2e18-4a08-97a4-8267f28e4386', ...) of the root model Figure(id='af8f75c4-f9e2-49d3-9f07-8b9242bea45b', ...) is already owned by another document (Models must be owned by only a single document). This may indicate a usage error.`

Or did you mean to actually recreate the whole figure from scratch?

-------------
Anyways, I found another workaround: Temporarily removing the figure from the document. When you click "Download", the figure will vanish until the export is completed. This is not a good solution, though. In my main project it fails, with the error message shown above. I guess there are more dependencies between documents and models that I don't understand. Also locating your figure in this way `doc_layout.children[1].children` seems cumbersome.
```python
from bokeh.io import curdoc
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.layouts import layout, widgetbox
from bokeh.io import export_png
from bokeh.models.widgets import Button


def download():
    try:
        export_png(doc_layout, filename="export doc_layout.png")
        print('doc_layout works')
    except Exception as ex:
        print('doc_layout throws error:', ex)
    try:
        doc_layout.children[1].children.remove(fig)  # remove to allow export
        export_png(fig, filename="export fig.png")
        doc_layout.children[1].children.append(fig)  # add fig back in again
        print('fig works')
    except Exception as ex:
        print('fig throws error:', ex)


source = ColumnDataSource({'x': [2, 3, 4], 'y': [0.5, 1.5, 2.5]})
fig = figure(plot_width=1000, plot_height=300)
fig.hbar(y='y', right='x', height=0.4, source=source)

button = Button(label="Download")
button.on_click(download)
doc_layout = layout([[widgetbox(button)], [fig]], sizing_mode='fixed')

#download()  # Test 1

print('Layout is added to root')
curdoc().add_root(doc_layout)
curdoc().title = "Export PNG test"

#download()  # Test 2

```



This took me quite a while to figure out, but here is another workaround with a more general approach. It works in my main use case, so hopefully it should always work. I hope this helps those with the same issue who come here via search.

If the model we are trying to export must not be assigned to a document, a general solution is to remove the complete root from `curdoc()` during the export. Instead we show a temporary root with a status message:
```python
from bokeh.io import curdoc
from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.layouts import layout, widgetbox
from bokeh.io import export_png
from bokeh.models.widgets import Button, Div


def download():
    div_temp = Div(text='''<div>Please wait for export</div>''', width=1000)
    root_temp = layout(div_temp)
    curdoc().add_root(root_temp)  # create temporary view
    root_main = curdoc().roots[0]  # store the main root
    curdoc().remove_root(root_main)  # required for export_png to work

    try:
        export_png(doc_layout, filename="export doc_layout.png")
        print('doc_layout works')
    except Exception as ex:
        print('doc_layout throws error:', ex)
    try:
        export_png(fig, filename="export fig.png")
        print('fig works')
    except Exception as ex:
        print('fig throws error:', ex)

    # Restore original view
    curdoc().add_root(root_main)
    curdoc().remove_root(root_temp)


source = ColumnDataSource({'x': [2, 3, 4], 'y': [0.5, 1.5, 2.5]})
fig = figure(plot_width=1000, plot_height=300)
fig.hbar(y='y', right='x', height=0.4, source=source)

button = Button(label="Download")
button.on_click(download)
doc_layout = layout([[widgetbox(button)], [fig]], sizing_mode='fixed')

curdoc().add_root(doc_layout)
curdoc().title = "Export PNG test"
```

The "standalone HTML" warning may still be shown, but can be ignored.
The new `OutputDocumentFor` context manager might handle this, I will try that out soon
@AzraelDD I believe the work to add `OutputDocumentFor` actually just solved the issue. Your original code is working for me. Can you verify this with a recent dev build, or local dev install? 

If so, I will task this issue with suppressing the warning about callbacks when exporting. 

Resolving patch:
diff --git a/bokeh/embed/standalone.py b/bokeh/embed/standalone.py
--- a/bokeh/embed/standalone.py
+++ b/bokeh/embed/standalone.py
@@ -245,7 +245,8 @@ def file_html(models,
               title=None,
               template=FILE,
               template_variables={},
-              theme=FromCurdoc):
+              theme=FromCurdoc,
+              suppress_callback_warning=False):
     ''' Return an HTML document that embeds Bokeh Model or Document objects.
 
     The data for the plot is stored directly in the returned HTML, with
@@ -256,11 +257,14 @@ def file_html(models,
         models (Model or Document or seq[Model]) : Bokeh object or objects to render
             typically a Model or Document
 
-        resources (Resources or tuple(JSResources or None, CSSResources or None)) : i
+        resources (Resources or tuple(JSResources or None, CSSResources or None)) :
             A resource configuration for Bokeh JS & CSS assets.
 
-        title (str, optional) : a title for the HTML document ``<title>`` tags or None. (default: None)
-            If None, attempt to automatically find the Document title from the given plot objects.
+        title (str, optional) :
+            A title for the HTML document ``<title>`` tags or None. (default: None)
+
+            If None, attempt to automatically find the Document title from the given
+            plot objects.
 
         template (Template, optional) : HTML document template (default: FILE)
             A Jinja2 Template, see bokeh.core.templates.FILE for the required
@@ -276,6 +280,12 @@ def file_html(models,
             already specified in the document. Any other value must be an
             instance of the ``Theme`` class.
 
+        suppress_callback_warning (bool, optional) :
+            Normally generating standalone HTML from a Bokeh Document that has
+            Python callbacks will result in a warning stating that the callbacks
+            cannot function. However, this warning can be suppressed by setting
+            this value to True (default: False)
+
     Returns:
         UTF-8 encoded HTML
 
@@ -287,7 +297,7 @@ def file_html(models,
         models = models.roots
 
     with OutputDocumentFor(models, apply_theme=theme) as doc:
-        (docs_json, render_items) = standalone_docs_json_and_render_items(models)
+        (docs_json, render_items) = standalone_docs_json_and_render_items(models, suppress_callback_warning=suppress_callback_warning)
         title = _title_from_models(models, title)
         bundle = bundle_for_objs_and_resources([doc], resources)
         return html_page_for_render_items(bundle, docs_json, render_items, title=title,
diff --git a/bokeh/embed/util.py b/bokeh/embed/util.py
--- a/bokeh/embed/util.py
+++ b/bokeh/embed/util.py
@@ -93,7 +93,7 @@ def OutputDocumentFor(objs, apply_theme=None, always_new=False):
 
             If None, use whatever theme is on the document that is found or created
 
-            If FromCurdox, use curdoc().theme, restoring any previous theme afterwards
+            If FromCurdoc, use curdoc().theme, restoring any previous theme afterwards
 
             If a Theme instance, use that theme, restoring any previous theme afterwards
 
@@ -233,7 +233,7 @@ def __getattr__(self, key):
     def to_json(self):
         return OrderedDict([ (root._id, elementid) for root, elementid in self._roots.items() ])
 
-def standalone_docs_json_and_render_items(models):
+def standalone_docs_json_and_render_items(models, suppress_callback_warning=False):
     '''
 
     '''
@@ -243,7 +243,7 @@ def standalone_docs_json_and_render_items(models):
     if not (isinstance(models, Sequence) and all(isinstance(x, (Model, Document)) for x in models)):
         raise ValueError("Expected a Model, Document, or Sequence of Models or Documents")
 
-    if submodel_has_python_callbacks(models):
+    if submodel_has_python_callbacks(models) and not suppress_callback_warning:
         log.warn(_CALLBACKS_WARNING)
 
     docs = {}
diff --git a/bokeh/io/export.py b/bokeh/io/export.py
--- a/bokeh/io/export.py
+++ b/bokeh/io/export.py
@@ -239,7 +239,7 @@ def get_layout_html(obj, resources=INLINE, **kwargs):
             obj.plot_width = kwargs.get('width', old_width)
 
     try:
-        html = file_html(obj, resources, title="")
+        html = file_html(obj, resources, title="", suppress_callback_warning=True)
     finally:
         if resize:
             obj.plot_height = old_height


Test patch:
diff --git a/bokeh/embed/tests/test_util.py b/bokeh/embed/tests/test_util.py
--- a/bokeh/embed/tests/test_util.py
+++ b/bokeh/embed/tests/test_util.py
@@ -528,6 +528,25 @@ def test_log_warning_if_python_event_callback(self, caplog):
             assert len(caplog.records) == 1
             assert caplog.text != ''
 
+    def test_suppress_warnings(self, caplog):
+        d = Document()
+        m1 = EmbedTestUtilModel()
+        c1 = _GoodPropertyCallback()
+        c2 = _GoodEventCallback()
+        d.add_root(m1)
+
+        m1.on_change('name', c1)
+        assert len(m1._callbacks) != 0
+
+        m1.on_event(Tap, c2)
+        assert len(m1._event_callbacks) != 0
+
+        with caplog.at_level(logging.WARN):
+            beu.standalone_docs_json_and_render_items(m1, suppress_callback_warning=True)
+            assert len(caplog.records) == 0
+            assert caplog.text == ''
+
+
 #-----------------------------------------------------------------------------
 # Private API
 #-----------------------------------------------------------------------------