hvplot_docs/Plots_and_Renderers.md

```python
import holoviews as hv
import numpy as np
```

HoloViews ordinarily hides the plotting machinery from the user. This allows for very quick iteration over different visualizations to explore a dataset, however it is often important to customize the precise details of a plot. HoloViews makes it very easy to customize existing plots, or even create completely novel plots. This manual will provide a general overview of the plotting system.

The separation of the data from the precise details of the visualization is one of the core principles of the HoloViews. [``Elements``](http://holoviews.org/reference/index.html#elements) provide thin wrappers around chunks of actual data, while [containers](http://holoviews.org/reference/index.html#containers) allow composing these Elements into overlays, layouts, grids and animations/widgets. Each Element or container type has a corresponding plotting class, which renders a visual representation of the data for a particular backend. While the precise details of the implementation differ between backends to accommodate the vastly different APIs  plotting backends provide, many of the high-level details are shared across backends.

# The Store object

The association between an Element or container and the backend specific plotting class is held on the global ``Store`` object. The ``Store`` object holds a ``registry`` of plot objects for each backend. We can view the registry for each backend by accessing ``Store.registry`` directly:


```python
import holoviews.plotting.mpl
list(hv.Store.registry['matplotlib'].items())[0:5]
```

The Store object provides a global registry not only for the plots themselves but also creates an entry in the OptionsTree for that particular backend. This allows options for that backend to be validated and enables setting plot, style and normalization options via the [Options system](03-Applying_Customizations.ipynb) system. We can view the ``OptionsTree`` object by requesting it from the store. We'll make a copy with just the first few entries so we can view the structure of the tree:


```python
opts = hv.Store.options(backend='matplotlib')
hv.core.options.OptionTree(opts.items()[0:10], groups=['plot', 'style', 'norm'])
```

# Saving and rendering

The easiest entry points to rendering a HoloViews object either to the backend specific representation (e.g. a matplotlib figure) or directly to file are the ``hv.render`` and ``hv.save`` functions. Both are shortcuts for using an actual ``Renderer`` object, which will be introduced in the next section. To start with we will create a simple object, a ``Scatter`` element:


```python
curve = hv.Curve(range(10))
curve
```

This abstract and declarative representation of a ``Curve`` can be turned into an actual plot object, defaulting to the currently selected (or default) backend:


```python
fig = hv.render(curve)
print(type(fig))
```

By providing an explicit ``backend`` keyword the plot can be rendered using a different backend.


```python
p = hv.render(curve, backend='bokeh')
print(type(p))
```

This can often be useful to customize a plot in more detail by tweaking styling in ways that are not directly supported by HoloViews. An alternative to ``hv.render`` in case you do not want to fully switch to the underlying plotting API are ``hooks``, which are a plot option on all elements. These allow defining hooks which modify a plot after it has been rendered in the backend but before it is displayed.

A ``hook`` is given the HoloViews plot instance and the currently rendered element and thereby provides access to the rendered plot object to apply any customizations and changes which are not exposed by HoloViews:


```python
def hook(plot, element):
    # Allows accessing the backends figure object
    plot.state
    
    # The handles contain common plot objects
    plot.handles

curve = curve.opts(hooks=[hook])
```

Hooks allow for extensive tweaking of objects before they are finished rendering, without having to entirely abandon HoloViews' API and render the backend's plot object manually.

In much the same way the ``hv.save`` function allows exporting plots straight to a file, by default inferring the format from the file extension:


```python
hv.save(curve, 'curve.png')
```

Just like ``hv.render`` the ``hv.save`` function also allows specifying an explicit backend.


```python
hv.save(curve, 'curve.html', backend='bokeh')
```

Additionally for ambiguous file extensions such as HTML it may be necessary to specify an explicit fmt to override the default, e.g. in the case of 'html' output the widgets will default to ``fmt='widgets'``, which may be changed to scrubber widgets using ``fmt='scrubber'``.

# Renderers

HoloViews provides a general ``Renderer`` baseclass, which defines a general interface to render the output from different backends to a number of standard output formats such as ``png``, ``html`` or ``svg``. The ``__call__`` method on the Renderer automatically looks up and instantiates the registered plotting classes for an object it is passed and then returns the output in the requested format. To make this a bit clearer we'll break this down step by step. First we'll get a handle on the ``MPLRenderer`` and create an object to render.

Renderers aren't registered with the Store until the corresponding backends have been imported. Loading the notebook extension with ``hv.notebook_extension('matplotlib')`` is one way of loading a backend and registering a renderer. Another is to simply import the corresponding plotting module as we did above.


```python
hv.Store.renderers
```

This is one way to access a Renderer, another is to instantiate a Renderer instance directly, allowing you to override some of the default plot options.


```python
renderer = hv.plotting.mpl.MPLRenderer.instance(dpi=120)
```

The recommended way to get a handle on a renderer is to use the ``hv.renderer`` function which will also handle imports for you:


```python
hv.renderer('matplotlib')
```

# Working with a Renderer

A ``Renderer`` in HoloViews is responsible for instantiating a HoloViews plotting class. It does this by looking up the plotting class in the ``Store.registry``:


```python
hv.Store.registry['matplotlib'][hv.Curve]
```

If we create a ``Curve`` we can instantiate a plotting class from it using the ``Renderer.get_plot`` method:


```python
curve = hv.Curve(range(10))
curve_plot = renderer.get_plot(curve)
curve_plot
```

We will revisit how to work with ``Plot`` instances later. For now all we need to know is that they are responsible for translating the HoloViews object (like the ``Curve``) into a backend specific plotting object, accessible on ``Plot.state``:


```python
print(type(curve_plot.state), curve_plot.state)
curve_plot.state
```

In case of the matplotlib backend this is a ``Figure`` object. However the ``Renderer`` ignores the specific representation of the plot, instead providing a unified interface to translating it into a representation that can displayed, i.e. either an image format or an HTML representation.

In this way we can convert the curve directly to its ``png`` representation by calling the ``Renderer`` with the object and the format:


```python
from IPython.display import display_png
png, info = renderer(curve, fmt='png')
print(info)
display_png(png, raw=True)
```

<div class="alert alert-success">
Tip: To find more information about any HoloViews object use <code>hv.help</code> to print a detailed docstring.
</div>

The valid figure display formats can be seen in the docstring of the Renderer or directly on the parameter:


```python
renderer.param['fig'].objects
```

Note that to export PNG files using the [bokeh renderer](./Plotting_with_Bokeh.ipynb) you may need to install some additional dependencies [detailed here](https://bokeh.pydata.org/en/latest/docs/user_guide/export.html). If you are using conda, it is currently sufficient to run ``conda install selenium phantomjs pillow``.

In this way we can easily render the plot in different formats:


```python
from IPython.display import display_svg
svg, info = renderer(curve, fmt='svg')
print(info)
display_svg(svg, raw=True)
```

We could save these byte string representations ourselves but the ``Renderer`` provides a convenient ``save`` method to do so. Simply supply the object the filename and the format, which doubles as the file extension:


```python
renderer.save(curve, '/tmp/test', fmt='png')
```

Another convenient way to render the object is to wrap it in HTML, which we can do with the ``html`` method:


```python
from IPython.display import display_html
html = renderer.html(curve)
display_html(html, raw=True)
```

Rendering plots containing ``HoloMap`` and ``DynamicMap`` objects will automatically generate a Panel HoloViews pane which can be rendered in the notebook, saved or rendered as a server app:


```python
holomap = hv.HoloMap({i: hv.Image(np.random.rand(10, 10)) for i in range(3)})
widget = renderer.get_widget(holomap, 'widgets')
widget
```

However most of the time it is more convenient to let the Renderer export the widget HTML, again via a convenient method, which will export a HTML document with all the required JS and CSS dependencies:


```python
html = renderer.static_html(holomap)
```

This covers the basics of working with HoloViews renderers. This API is consistent across plotting backends, whether matplotlib, bokeh or plotly.

# Plots

Above we saw how the Renderer looks up the appropriate plotting class but so far we haven't seen how the plotting actually works. Since HoloViews already nests the data into semantically meaningful components, which define the rough layout of the plots on the page, the plotting classes follow roughly the same hierarchy. To review this hierarchy have a look at the [nesting diagram](./06-Building_Composite_Objects.ipynb#Nesting-hierarchy-) in the Building Composite objects guide.

The Layout and GridSpace plotting classes set up the figure and axes appropriately and then instantiate the subplots for all the objects that are contained within. For this purpose we will create a relatively complex object, a ``Layout`` of ``HoloMap``s containing ``Overlay``s containing ``Elements``. We'll instantiate the matching plotting hierarchy and then inspect it.


```python
hmap1 = hv.HoloMap({i: hv.Image(np.random.rand(10, 10)) * hv.Ellipse(0, 0, 0.2*i) for i in range(5)})
element = hv.Curve((range(10), np.random.rand(10)))
layout = hmap1 + element
```

We can see the hierarchy in the object's repr:


```python
print( repr(layout) )
```

Now that we've created the object we can again use the ``MPLRenderer`` to instantiate the plot:


```python
layout_plot = renderer.get_plot(layout)
layout_plot
```

During instantiation the LayoutPlot expanded each object and created subplots. We can access them via a row, column based index and thereby view the first plot.


```python
adjoint_plot = layout_plot.subplots[0, 0]
adjoint_plot
```

This plotting layer handles plots adjoined to the plot. They are indexed by their position in the AdjointLayout which may include 'top', 'right' and 'main':


```python
overlay_plot = adjoint_plot.subplots['main']
overlay_plot
```

Now we've drilled all the way down to the OverlayPlot level, we see as expected that this contains two further subplots, one for the ``Image`` and one for ``Text`` Element. 


```python
overlay_plot.subplots.keys()
```

Now you might have noticed that the HoloMap seems to have disappeared from the hierarchy. This is because updating a particular plot is handled by the ``ElementPlots`` itself. With that knowledge we can now have a look at the actual plotting API.

### Traversing plots

When working with such deeply nested plots accessing leafs can be a lot of effort, therefore the plots also provide a ``traverse`` method letting you specify the types of plots you want to access:


```python
layout_plot.traverse(specs=[hv.plotting.mpl.CurvePlot])
```

# Plotting API

There a few methods shared by all plotting classes, which allow the renderer to easily create, update and render a plot. The three most important methods and attributes are:

* ``Plot.__init__`` - The constructor already handles a lot of the processing in a plot, it sets up all the subplots if there are any, computes ranges across the object to normalize the display, sets the options that were specified and instantiates the figure, axes, model graphs, or canvas objects depending on the backend.
* ``Plot.initialize_plot`` - This method draws the initial frame to the appropriate figure, axis or canvas, setting up the various artists (matplotlib) or glyphs (bokeh).
* ``Plot.update`` - This method updates an already instantiated plot with the data corresponding to the supplied key. This key should match the key in the HoloMap.

### Initializing

The Renderer and the widgets use these three methods to instantiate and update a plot to render both static frames and animations or widgets as defined by the ``HoloMap`` or ``DynamicMap``. Above we already instantiated a plot, now we initialize it, thereby drawing the first (or rather last frame).


```python
fig = layout_plot.initialize_plot()
fig
```

### Updating

We can see ``initialize_plot`` has rendered the last frame with the key ``4``. We can update the figure with another key simply by calling the ``Plot.update`` method with the corresponding key.


```python
layout_plot.update(0)
```


```python
plot = hv.plotting.mpl.RasterPlot(holomap)
plot
```

Internally each level of the plotting hierarchy updates all the objects below it, all the way down to the ElementPlots, which handle updating the plotting data.

### Dynamic plot updates

Since DynamicMaps may be updated based on stream events they don't work via quite the same API. Each stream automatically captures the plots it is attached to and whenever it receives an event it will update the plot.


```python
dmap = hv.DynamicMap(lambda x: hv.Points(np.arange(x)), kdims=[], streams=[hv.streams.PointerX(x=10)])
plot = renderer.get_plot(dmap)
plot.initialize_plot()
```

Internally the update to the stream value will automatically trigger the plot to update, here we will disable this by setting ``trigger=False`` and explicitly calling ``refresh`` on the plot:


```python
dmap.event(x=20,)
plot.refresh()
plot.state
```

### Plotting handles

In addition to accessing the overall state of the plot each plotting class usually keeps direct handles for important plotting elements on the ``handles`` attribute:


```python
plot.handles
```

Here we can see how the ``PointPlot`` keeps track of the artist, which is a matplotlib ``PathCollection``, the axis, the figure and the plot title. In addition all matplotlib plots also keep track of a list of any additional plotting elements which should be considered in the bounding box calculation.

This is only the top-level API, which is used by the Renderer to render the plot, animation or widget. Each backend has internal APIs to create and update the various plot components.

### Using the matplotlib renderer outside of a Jupyter notebook

The matplotlib renderer can be used outside of a notebook by using the `show` method. If running in a python script, this will cause the usual matplotlib window to appear. This is done as follows:

    mr = hv.renderer('matplotlib')
    mr.show(curve)