Repository: plotly/plotly.py
Pull request #4177 (2023-04-27 00:50:52)

Issue #3755 Bug Fix for: Shapes and Annotations With yref Parameter Not Drawing on Correct Axis
This PR is related to issue #3755 which, to put more generally, is that shapes and annotations when supplied with a `yref` parameter are automatically drawn on the primary y-axis despite the supplied `yref` value. The only way that a shape or annotation can currently be drawn on an axis that differs from the primary axes is when the figure object is created with `subplots.make_subplots()` with the `specs` list parameter set to `'secondary_y':True` for that specific subplot. In the case of a non-subplotted figure object, it is impossible as the following line: https://github.com/plotly/plotly.py/blob/956ab2cd158f0d20b322aa49e2f717a56aa6b9e2/packages/python/plotly/plotly/basedatatypes.py#L4051 explicitly sets `yref = 'y'` when there is not more than one subplot. It is also possible to make a figure object with subplot grid of (1,1) when using plotly express. Using that method there is no way to allow the `secondary_y:True` to the figure object therefore specifying `yref` will result in the shape or annotation plotting on the primary y axis regarless of the specified `yref` and if the `secondary_y=True` parameter is supplied in the method to add the shape or annotation, that will result in the following error:
https://github.com/plotly/plotly.py/blob/956ab2cd158f0d20b322aa49e2f717a56aa6b9e2/packages/python/plotly/plotly/basedatatypes.py#L1562-L1567
This error is also missing it's `string.format()` method and parameters which are fixed in this as well.

For subplots that use the `subplots.make_subplots()` with the `specs` list parameter set to `'secondary_y':True` this change does not cause the loss of any functionality it just extends the ability to draw shapes and annotations on additional y-axes when generating figure objects without the `subplots.make_subplots()` method. Specifying `yref` and if the `secondary_y` parameters will not result in an error when used with plotly express (a figure object with a (1,1) subplot grid) or a non subplotted figure object (such as when using `go.Figure()`), it will just proceed with applying it to the specified `yref`.

## Code PR

- [x] I have read through the [contributing notes](https://github.com/plotly/plotly.py/blob/master/contributing.md) and understand the structure of the package. In particular, if my PR modifies code of `plotly.graph_objects`, my modifications concern the `codegen` files and not generated files.
- [x] I have added tests (if submitting a new feature or correcting a bug) or
  modified existing tests.
- [ ] For a new feature, I have added documentation examples in an existing or
  new tutorial notebook (please see the doc checklist as well).
- [ ] I have added a CHANGELOG entry if fixing/changing/adding anything substantial.
- [ ] For a new feature or a change in behaviour, I have updated the relevant docstrings in the code to describe the feature or behaviour (please see the doc checklist as well).




Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cbbaf3cb8..454d2210a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
   - Fixed another compatibility issue with Pandas 2.0, just affecting `px.*(line_close=True)` [[#4190](https://github.com/plotly/plotly.py/pull/4190)]
   - Added some rounding to the `make_subplots` function to handle situations where the user-input specs cause the domain to exceed 1 by small amounts [[#4153](https://github.com/plotly/plotly.py/pull/4153)]
   - Sanitize JSON output to prevent an XSS vector when graphs are inserted directly into HTML [[#4196](https://github.com/plotly/plotly.py/pull/4196)]
+  - Fixed issue with shapes and annotations plotting on the wrong y axis when supplied with a specific axis in the `yref` parameter [[#4177](https://github.com/plotly/plotly.py/pull/4177)]
   - Remove `use_2to3` setuptools arg, which is invalid in the latest Python and setuptools versions [[#4206](https://github.com/plotly/plotly.py/pull/4206)]
   - Fix [#4066](https://github.com/plotly/plotly.py/issues/4066) JupyterLab v4 giving tiny default graph height [[#4227](https://github.com/plotly/plotly.py/pull/4227)]
 
diff --git a/packages/python/plotly/plotly/basedatatypes.py b/packages/python/plotly/plotly/basedatatypes.py
index bf5eac3df..8fa70e0ad 100644
--- a/packages/python/plotly/plotly/basedatatypes.py
+++ b/packages/python/plotly/plotly/basedatatypes.py
@@ -1559,17 +1559,31 @@ is of type {subplot_type}.""".format(
                         subplot_type=refs[0].subplot_type,
                     )
                 )
-            if len(refs) == 1 and secondary_y:
-                raise ValueError(
-                    """
-Cannot add {prop_singular} to secondary y-axis of subplot at position ({r}, {c})
-because subplot does not have a secondary y-axis"""
-                )
-            if secondary_y:
-                xaxis, yaxis = refs[1].layout_keys
+
+            # If the new_object was created with a yref specified that did not include paper or domain, the specified yref should be used otherwise assign the xref and yref from the layout_keys
+            if (
+                new_obj.yref is None
+                or new_obj.yref == "y"
+                or "paper" in new_obj.yref
+                or "domain" in new_obj.yref
+            ):
+                if len(refs) == 1 and secondary_y:
+                    raise ValueError(
+                        """
+    Cannot add {prop_singular} to secondary y-axis of subplot at position ({r}, {c})
+    because subplot does not have a secondary y-axis""".format(
+                            prop_singular=prop_singular, r=row, c=col
+                        )
+                    )
+                if secondary_y:
+                    xaxis, yaxis = refs[1].layout_keys
+                else:
+                    xaxis, yaxis = refs[0].layout_keys
+                xref, yref = xaxis.replace("axis", ""), yaxis.replace("axis", "")
             else:
-                xaxis, yaxis = refs[0].layout_keys
-            xref, yref = xaxis.replace("axis", ""), yaxis.replace("axis", "")
+                yref = new_obj.yref
+                xaxis = refs[0].layout_keys[0]
+                xref = xaxis.replace("axis", "")
             # if exclude_empty_subplots is True, check to see if subplot is
             # empty and return if it is
             if exclude_empty_subplots and (
@@ -1591,6 +1605,11 @@ because subplot does not have a secondary y-axis"""
             new_obj.update(xref=xref, yref=yref)
 
         self.layout[prop_plural] += (new_obj,)
+        # The 'new_obj.xref' and 'new_obj.yref' parameters need to be reset otherwise it
+        # will appear as if user supplied yref params when looping through subplots and
+        # will force annotation to be on the axis of the last drawn annotation
+        # i.e. they all end up on the same axis.
+        new_obj.update(xref=None, yref=None)
 
         return self
 
@@ -4034,6 +4053,7 @@ Invalid property path '{key_path_str}' for layout
                 row=row,
                 col=col,
                 exclude_empty_subplots=exclude_empty_subplots,
+                yref=shape_kwargs.get("yref", "y"),
             )
         # update xref and yref for the new shapes and annotations
         for layout_obj, n_layout_objs_before in zip(
@@ -4045,10 +4065,13 @@ Invalid property path '{key_path_str}' for layout
             ):
                 # this was called intending to add to a single plot (and
                 # self.add_{layout_obj} succeeded)
-                # however, in the case of a single plot, xref and yref are not
-                # specified, so we specify them here so the following routines can work
-                # (they need to append " domain" to xref or yref)
-                self.layout[layout_obj][-1].update(xref="x", yref="y")
+                # however, in the case of a single plot, xref and yref MAY not be
+                # specified, IF they are not specified we specify them here so the following routines can work
+                # (they need to append " domain" to xref or yref). If they are specified, we leave them alone.
+                if self.layout[layout_obj][-1].xref is None:
+                    self.layout[layout_obj][-1].update(xref="x")
+                if self.layout[layout_obj][-1].yref is None:
+                    self.layout[layout_obj][-1].update(yref="y")
             new_layout_objs = tuple(
                 filter(
                     lambda x: x is not None,


Test patch:
diff --git a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_annotations.py b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_annotations.py
index 23c3af279..d1c81a311 100644
--- a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_annotations.py
+++ b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_annotations.py
@@ -5,6 +5,7 @@ from unittest import TestCase
 
 import plotly.graph_objs as go
 from plotly.subplots import make_subplots
+
 import pytest
 
 
@@ -351,6 +352,64 @@ def test_no_exclude_empty_subplots():
         assert fig.layout[k][3]["xref"] == "x4" and fig.layout[k][3]["yref"] == "y4"
 
 
+def test_supplied_yref_on_single_plot_subplot():
+    ### test a (1,1) subplot figure object
+    fig = make_subplots(1, 1)
+    fig.add_trace(go.Scatter(x=[1, 2, 3, 4], y=[1, 2, 2, 1]))
+    fig.add_trace(go.Scatter(x=[1, 2, 3, 4], y=[4, 3, 2, 1], yaxis="y2"))
+    fig.update_layout(
+        yaxis=dict(title="yaxis1 title"),
+        yaxis2=dict(title="yaxis2 title", overlaying="y", side="right"),
+    )
+    # add horizontal line on y2. Secondary_y can be True or False when yref is supplied
+    fig.add_hline(y=3, yref="y2", secondary_y=True)
+    assert fig.layout["shapes"][0]["yref"] == "y2"
+
+
+def test_supplied_yref_on_non_subplot_figure_object():
+    ### test a non-subplot figure object from go.Figure
+    trace1 = go.Scatter(x=[1, 2, 3, 4], y=[1, 2, 2, 1])
+    trace2 = go.Scatter(x=[1, 2, 3, 4], y=[4, 3, 2, 1], yaxis="y2")
+    data = [trace1, trace2]
+    layout = go.Layout(
+        yaxis=dict(title="yaxis1 title"),
+        yaxis2=dict(title="yaxis2 title", overlaying="y", side="right"),
+    )
+    fig = go.Figure(data=data, layout=layout)
+    # add horizontal line on y2. Secondary_y can be True or False when yref is supplied
+    fig.add_hline(y=3, yref="y2", secondary_y=False)
+    assert fig.layout["shapes"][0]["yref"] == "y2"
+
+
+def test_supplied_yref_on_multi_plot_subplot():
+    ### test multiple subploted figure object with subplots.make_subplots
+    fig = make_subplots(
+        rows=1,
+        cols=2,
+        shared_yaxes=False,
+        specs=[[{"secondary_y": True}, {"secondary_y": True}]],
+    )
+    ### Add traces to the first subplot
+    fig.add_trace(go.Scatter(x=[1, 2, 3], y=[1, 2, 3]), row=1, col=1)
+    fig.add_trace(
+        go.Scatter(x=[1, 2, 3], y=[3, 2, 1], yaxis="y2"), row=1, col=1, secondary_y=True
+    )
+    ### Add traces to the second subplot
+    fig.add_trace(go.Scatter(x=[1, 2, 3], y=[1, 2, 3], yaxis="y"), row=1, col=2)
+    fig.add_trace(
+        go.Scatter(x=[1, 2, 3], y=[1, 1, 2], yaxis="y2"), row=1, col=2, secondary_y=True
+    )
+    # add a horizontal line on both subplots on their respective secondary y.
+    # When using the subplots.make_subplots() method yref parameter should NOT be supplied per docstring instructions.
+    # Instead secondary_y specs and secondary_y parameter MUST be True to plot on secondary y
+    fig.add_hline(y=2, row=1, col=1, secondary_y=True)
+    fig.add_hline(y=1, row=1, col=2, secondary_y=True)
+    assert fig.layout["shapes"][0]["yref"] == "y2"
+    assert fig.layout["shapes"][0]["xref"] == "x domain"
+    assert fig.layout["shapes"][1]["yref"] == "y4"
+    assert fig.layout["shapes"][1]["xref"] == "x2 domain"
+
+
 @pytest.fixture
 def select_annotations_integer():
     fig = make_subplots(2, 3)