```python
import holoviews as hv
import numpy as np
import panel as pn

hv.extension('bokeh')
```

HoloViews-generated plots generally convey information from Python _to_ a viewer of the plot, but there are also circumstances where information needs to be collected _from_ the viewer and made available for processing in Python:

* annotating data with contextual information to aid later interpretation
* labeling or tagging data for automated machine-learning or other processing pipelines
* indicating regions of interest, outliers, or other semantic information
* specifying inputs to a query, command, or simulation
* testing sensitivity of analyses to adding, changing, or deleting user-selected data points

In such cases, it is important to be able to augment, edit, and annotate datasets and to access those values from Python. To perform these actions, HoloViews provides an ``annotate`` helper using [Bokeh's drawing tools](https://docs.bokeh.org/en/latest/docs/reference/models/tools.html#bokeh.models.tools.PointDrawTool) to make it easy to edit HoloViews Elements and add additional information using an associated table. The `annotate` helper:

* Adds plot tools that allow editing and adding new elements to a plot  
* Adds table(s) to allow editing the element in a tabular format
* Returns a layout of these two components
* Makes the edits, annotations, and selections available on a property of the annotate object so that they can be utilized in Python

## Basics

Let us start by annotating a small set of Points. To do this, we need two things:

1. A Points Element to annotate or edit
2. An annotator object to collect and store annotations

The annotator is a callable object with its own state that can be called to return a Layout consisting of the object to be annotated and an Overlay of editable table(s):


```python
points = hv.Points([(0.0, 0.0), (1.0, 1.0), (200000.0, 2000000.0)]).opts(size=10, min_height=500)

annotator = hv.annotate.instance()
layout = annotator(hv.element.tiles.OSM() * points, annotations=['Label'], name="Point Annotations")

print(layout)
```

This layout of a DynamicMap (the user-editable Element data) and an Overlay (the user-editable table) lets a user input the required information:


```python
layout
```

Here we have pre-populated the Element with three points. Each of the points has three bits of information that can be edited using the table: the x location, y location, and a "Label",  which was initialized to dummy values when we called `annotate` and asked that there be a `Label` column.  Try clicking on one of the rows and editing the location or the label to anything you like. As long as Python is running and the new location is in the viewport, you should see the dot move when you edit the location, and any labels you entered should be visible in the table.

You can also edit the locations graphically using the [PointDraw tool](../reference/streams/bokeh/PointDraw.ipynb) in the toolbar:<img src="https://bokeh.pydata.org/en/latest/_images/PointDraw.png">

Once you select that tool, you should be able to click and drag any of the existing points and see the location update in the table. Whether you click on the table or the points, the same object should be selected in each, so that you can see how the graphical and tabular representations relate.

The PointDraw tool also allows us to add completely new points; once the tool is selected, just click on the plot above in locations not already containing a point and you can see a new point and a new table row appear ready for editing. You can also delete points by selecting them in the plot then pressing Backspace or Delete (depending on operating system).

All the above editing and interaction could have been done if we had simply called `hv.annotate(points, annotations=['Label'])` directly, but instead we first saved an "instance" of the annotator object so that we'd also be able to access the resulting data. So, once we are done collecting data from the user, let's use the saved `annotator` object handle to read out the values (by re-evaluating the following line):


```python
annotator.annotated.dframe()
```

You should see that you can access the current set of user-provided or user-modified points and their user-provided labels from within Python, ready for any subsequent processing you need to do. 

We can also access the currently `selected` points, in case we care only about a subset of the points (which will be empty if no points/rows are selected):


```python
annotator.selected.dframe()
```

## Configuring the Annotator

In addition to managing the list of `annotations`, the `annotate` helper exposes a few additional parameters. Remember like most Param-based objects you can get help about `annotate` parameters using the `hv.help` function:


```python
hv.help(hv.annotate)
```

### Annotation types

The default annotation type is a string, to allow you to put in arbitrary information that you later process in Python. If you want to enforce a more specific type, you can specify the annotation-value types explicitly using a dictionary mapping from column name to the type:


```python
hv.annotate(points, annotations={'int': int, 'float': float, 'str': str})
```

This example also shows how to collect multiple columns of information for the same data point.

## Types of Annotators

Currently only a limited set of Elements may be annotated, which include:

* ``Points``/``Scatter``
* ``Curve``
* ``Path``
* ``Polygons``
* ``Rectangles``

Adding support for new elements, in most cases, requires adding corresponding drawing/edit tools to Bokeh itself. But if you have data of other types, you may still be able to annotate it by casting it to one of the indicated types, collecting the data, then casting it back.

## Annotating Curves

To allow dragging the vertices of the Curve, the ``Curve`` annotator uses the PointDraw tool in the toolbar: <img src="https://bokeh.pydata.org/en/latest/_images/PointDraw.png">
The vertices will appear when the tool is selected or a vertex is selected in the table. Unlike most other annotators the Curve annotator only allows editing the vertices and does not allow adding new ones.


```python
curve = hv.Curve(np.random.randn(50).cumsum())

curve_annotator = hv.annotate.instance()

curve_annotator(curve.opts(width=800, height=400, responsive=False), annotations={'Label': str})
```

To access the data you can make use of the ``annotated`` property on the annotator:


```python
curve_annotator.annotated.dframe().head(5)
```

## Annotating Rectangles

The `Rectangles` annotator behaves very similarly to the Points annotator. It allows adding any number of annotation columns, using Bokeh's `BoxEdit` tool that allows both drawing and moving boxes. To see how to use the BoxEdit tool, refer to the HoloViews [BoxEdit stream reference](../reference/streams/bokeh/BoxEdit.ipynb), but briefly:

* Select the `BoxEdit` tool in the toolbar: <img src="https://bokeh.pydata.org/en/latest/_images/BoxEdit.png">
* Click and drag on an existing Rectangle to move it
* Double click to start drawing a new Rectangle at one corner, and double click to complete the rectangle at the opposite corner
* Select a rectangle and press the Backspace or Delete key (depending on OS) to delete it
* Edit the box coordinates in the table to resize it


```python
boxes = hv.Rectangles([(0, 0, 1, 1), (1.5, 1.5, 2.5, 2.5)])

box_annotator = hv.annotate.instance()

box_annotator(boxes.opts(width=800, height=400, responsive=False), annotations=['Label'])
```

To access the data we can make use of the ``annotated`` property on the annotator instance:


```python
box_annotator.annotated.dframe()
```

### Annotating paths/polygons

Unlike the Points and Boxes annotators, the Path and Polygon annotators allow annotating not just each individual entity but also the vertices that make up the paths and polygons. For more information about using the editing tools associated with this annotator refer to the HoloViews [PolyDraw](../reference/streams/PolyDraw.ipynb) and [PolyEdit](../reference/streams/PolyEdit.ipynb) stream reference guides, but briefly:

##### Drawing/Selecting Deleting Paths/Polygons

- Select the PolyDraw tool in the toolbar: <img src="https://bokeh.pydata.org/en/latest/_images/PolyDraw.png">
- Double click to start a new object, single click to add each vertex, and double-click to complete it.
- Delete paths/polygons by selecting and pressing Delete key (OSX) or Backspace key (PC)

##### Editing Paths/Polygons

- Select the PolyEdit tool in the toolbar: <img src="https://bokeh.pydata.org/en/latest/_images/PolyEdit.png">
- Double click a Path/Polygon to start editing
- Drag vertices to edit them, delete vertices by selecting them

To edit and annotate the vertices, use the draw tool or the first table to select a particular path/polygon and then navigate to the Vertices tab.


```python
path = hv.Path([hv.Box(0, 0, 1), hv.Ellipse(1, 1, 1)])

path_annotator = hv.annotate.instance()

path_annotator(path.opts(width=800, height=400, responsive=False), annotations=['Label'], vertex_annotations=['Value'])
```

To access the data we can make use of the iloc method on `Path` objects to access a particular path, and then access the `.data` or convert it to a dataframe:


```python
path_annotator.annotated.iloc[0].dframe()
```

## Composing Annotators

Often we will want to add some annotations on top of one or more other elements which provide context, e.g. when annotating an image with a set of `Points`. As long as only one annotation layer is required you can pass an overlay of multiple elements to the `annotate` operation and it will automatically associate the annotator with the layer that supports annotation. Note however that this will error if multiple elements that support annotation are provided. Below we will annotate a two-photon microscopy image with a set of Points, e.g. to mark the location of each cell:


```python
img = hv.Image(np.load('../assets/twophoton.npz')['Calcium'][..., 0])
cells = hv.Points([]).opts(width=500, height=500, responsive=False, padding=0)

hv.annotate(img * cells, annotations=['Label'], name="Cell Annotator")
```

### Multiple Annotators

If you want to work with multiple annotators in the same plot, you can recompose and rearrange the components returned by each `annotate` helper manually, but doing so can get confusing. To simplify working with multiple annotators at the same time, the `annotate` helper provides a special classmethod that allows composing multiple annotators and other elements, e.g. making a set of tiles into a combined layout consisting of all the components:


```python
point_annotate = hv.annotate.instance()
points = hv.Points([(500000, 500000), (1000000, 1000000)]).opts(size=10, color='red', line_color='black')
point_layout = point_annotate(points, annotations=['Label'])

poly_annotate = hv.annotate.instance()
poly_layout = poly_annotate(hv.Polygons([]), annotations=['Label'])

hv.annotate.compose(hv.element.tiles.OSM(), point_layout, poly_layout)
```

## Internals

The `annotate` function builds on [Param](https://param.holoviz.org) and [Panel](https://panel.holoviz.org), creating and wrapping Panel `Annotator` panes internally. These objects make it easy to include the annotator in Param-based workflows and trigger actions when parameters change and/or update the annotator in response to external events. The Annotator of a `annotate` instance can be accessed using the `annotator` attribute:


```python
print(point_annotate.annotator)
```

This object can be included directly in a Panel layout, be used to watch for parameter changes, or updated directly. To see the effect of updating directly, uncomment the line below, execute that cell, and then look at the previous plot of Africa, which should get updated with 10 randomly located blue dots.


```python
#point_annotate.annotator.object = hv.Points(np.random.randn(10, 2)*1000000).opts(color='blue')
```