Repository: plotly/plotly.py
Pull request: Figure select function selector
Closes #2629 and more.
Any function calling `BaseFigure._selector_matches` can now pass a function for the `selector` argument. This function is called on each object and if it returns True, then that object is included in the selection.

Problem statement:
select traces doesn't do partial matches on `meta`
Today, the output of 

```python
import plotly.graph_objects as go

fig = go.Figure(go.Scatter(meta=dict(a=1,b=2)))

print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1)))]))
```

is `1, 0`, meaning that we can't use `selector` to do partial matches on `meta`.

It would be really convenient if PX could tag all of its traces with some structured data in `meta` so that users could target these traces with updates, but for that to be useful we would need to do partial matches.

@jonmmease any major reason not to do this?

Hints from issue discussion:
Does this do what you want?
```python
print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))
```

I seem to remember making the design decision that selectors keys needed to be matched in full, and magic underscore notation could be used to target subsets.  

Not sure off hand if anything goes wrong if we support partial matches, but it would be a breaking change. We'd also need to decide whether the partial match logic is applied recursively.


that doesn't work, no, possibly because `meta` is an open-ended dict with no schema below it?

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-20-9a7a6c9a1903> in <module>
      4 
      5 print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
----> 6 print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))

<ipython-input-20-9a7a6c9a1903> in <listcomp>(.0)
      4 
      5 print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
----> 6 print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector)
    805 
    806             # Filter by selector
--> 807             if not self._selector_matches(trace, selector):
    808                 continue
    809 

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in _selector_matches(obj, selector)
    816 
    817         for k in selector:
--> 818             if k not in obj:
    819                 return False
    820 

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in __contains__(self, prop)
   3907                     return False
   3908             else:
-> 3909                 if obj is not None and p in obj._valid_props:
   3910                     obj = obj[p]
   3911                 else:

AttributeError: 'dict' object has no attribute '_valid_props'
```
(to be clear: I would be fine with having this work, it would meet my needs just fine :)
Ok, yeah. this is special because meta doesn't have a sub-schema.

I think having special case magic underscore logic to handle dicts is a good approach.  This would also be useful in the `update_*` operations for things like:

```python
fig.update_layout(meta_a=8)
```

It would mean we're asserting that dict keys are strings without underscores.  Not sure at the moment if this should be a special case for `meta`, or if we would want this to work for any property that has a dict value :thinking:  But in either case, I think this would be a backward compatible change to make.
Or we could have it do partial matches off a dict, as a special case... either way :)
On further thought: I want to store keys in `meta` which are user-provided and likely to have underscores in them...
From reading the dialogue above, I gather that `select_traces` should allow partial matches using `meta` no matter if it was made using a dict or the underscore notation. Furthermore, `meta_a_b=value` should produce `meta=dict(a=dict(b=value))` but `meta=dict(a_b=another_value)` should be allowed, too.
Another thing: do you want another keyword in `select_traces`, say `partial_match` which allows turning on or off the functionality of matching subsets of trace parameters?
I would be OK if partial matches on `meta` didn't work with magic underscores at all actually, if that's easier to implement.

I don't think I would favour a new `partial_match` kwarg, no, I think we can just make a special case for `meta`
Ok sounds good.
I don't want to go too far afield here, but what if selector could be a function that returned `True` if that trace should be selected? The function would be passed the trace. The old dict functionality could be left in, too.
that's not a bad idea... I could implement my PX stuff on top of that, yes. Is that easier to build?

Commit message: BaseFigure._selector_matches accepts function selector

Any function that calls BaseFigure._selector_matches can use a function
as the selector. This includes the "Figure.for_each_*" functions, the
"Figure.update_*" functions, and the "Figure.select_*" functions.

See test_selector_matches.py for the behaviour of the selector argument.

Patch to review:
diff --git a/packages/python/plotly/plotly/basedatatypes.py b/packages/python/plotly/plotly/basedatatypes.py
index 7bc0e66f56d..c35da7f5c5a 100644
--- a/packages/python/plotly/plotly/basedatatypes.py
+++ b/packages/python/plotly/plotly/basedatatypes.py
@@ -813,24 +813,33 @@ def _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector)
     def _selector_matches(obj, selector):
         if selector is None:
             return True
+        # If selector is a dict, compare the fields
+        if (type(selector) == type(dict())) or isinstance(selector, BasePlotlyType):
+            # This returns True if selector is an empty dict
+            for k in selector:
+                if k not in obj:
+                    return False
 
-        for k in selector:
-            if k not in obj:
-                return False
-
-            obj_val = obj[k]
-            selector_val = selector[k]
-
-            if isinstance(obj_val, BasePlotlyType):
-                obj_val = obj_val.to_plotly_json()
+                obj_val = obj[k]
+                selector_val = selector[k]
 
-            if isinstance(selector_val, BasePlotlyType):
-                selector_val = selector_val.to_plotly_json()
+                if isinstance(obj_val, BasePlotlyType):
+                    obj_val = obj_val.to_plotly_json()
 
-            if obj_val != selector_val:
-                return False
+                if isinstance(selector_val, BasePlotlyType):
+                    selector_val = selector_val.to_plotly_json()
 
-        return True
+                if obj_val != selector_val:
+                    return False
+            return True
+        # If selector is a function, call it with the obj as the argument
+        elif type(selector) == type(lambda x: True):
+            return selector(obj)
+        else:
+            raise TypeError(
+                "selector must be dict or a function "
+                "accepting a trace returning a boolean."
+            )
 
     def for_each_trace(self, fn, selector=None, row=None, col=None, secondary_y=None):
         """
diff --git a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py
new file mode 100644
index 00000000000..138d0b48921
--- /dev/null
+++ b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py
@@ -0,0 +1,91 @@
+import pytest
+
+import plotly.graph_objects as go
+from plotly.basedatatypes import BaseFigure
+
+
+def test_selector_none():
+    # should return True
+    assert BaseFigure._selector_matches({}, None) == True  # arbitrary,
+
+
+def test_selector_empty_dict():
+    # should return True
+    assert (
+        BaseFigure._selector_matches(dict(hello="everybody"), {}) == True  # arbitrary,
+    )
+
+
+def test_selector_matches_subset_of_obj():
+    # should return True
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, today="cloudy"),
+        )
+        == True
+    )
+
+
+def test_selector_has_nonmatching_key():
+    # should return False
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, cronenberg="scanners"),
+        )
+        == False
+    )
+
+
+def test_selector_has_nonmatching_value():
+    # should return False
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, today="sunny"),
+        )
+        == False
+    )
+
+
+def test_baseplotlytypes_could_match():
+    # should return True
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    assert BaseFigure._selector_matches(obj, sel) == True
+
+
+def test_baseplotlytypes_could_not_match():
+    # should return False
+    obj = go.layout.Annotation(x=1, y=3, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    assert BaseFigure._selector_matches(obj, sel) == False
+
+
+def test_baseplotlytypes_cannot_match_subset():
+    # should return False because "undefined" keys in sel return None, and are
+    # compared (because "key in sel" returned True, it's value was None)
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2,)
+    assert BaseFigure._selector_matches(obj, sel) == False
+
+
+def test_function_selector_could_match():
+    # should return True
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+
+    def _sel(d):
+        return d["x"] == 1 and d["y"] == 2 and d["text"] == "pat metheny"
+
+    assert BaseFigure._selector_matches(obj, _sel) == True
+
+
+def test_function_selector_could_not_match():
+    # should return False
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+
+    def _sel(d):
+        return d["x"] == 1 and d["y"] == 3 and d["text"] == "pat metheny"
+
+    assert BaseFigure._selector_matches(obj, _sel) == False
diff --git a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
index b1c8487ceae..ccfe50382e0 100644
--- a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
+++ b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
@@ -225,6 +225,43 @@ def test_select_property_and_grid(self):
         # Valid row/col and valid selector but the intersection is empty
         self.assert_select_traces([], selector={"type": "markers"}, row=3, col=1)
 
+    def test_select_with_function(self):
+        def _check_trace_key(k, v):
+            def f(t):
+                try:
+                    return t[k] == v
+                except LookupError:
+                    return False
+
+            return f
+
+        # (1, 1)
+        self.assert_select_traces(
+            [0], selector=_check_trace_key("mode", "markers"), row=1, col=1
+        )
+        self.assert_select_traces(
+            [1], selector=_check_trace_key("type", "bar"), row=1, col=1
+        )
+
+        # (2, 1)
+        self.assert_select_traces(
+            [2, 9], selector=_check_trace_key("mode", "lines"), row=2, col=1
+        )
+
+        # (1, 2)
+        self.assert_select_traces(
+            [4], selector=_check_trace_key("marker.color", "green"), row=1, col=2
+        )
+
+        # Valid row/col and valid selector but the intersection is empty
+        self.assert_select_traces(
+            [], selector=_check_trace_key("type", "markers"), row=3, col=1
+        )
+
+    def test_select_traces_type_error(self):
+        with self.assertRaises(TypeError):
+            self.assert_select_traces([0], selector=123, row=1, col=1)
+
     def test_for_each_trace_lowercase_names(self):
         # Names are all uppercase to start
         original_names = [t.name for t in self.fig.data]


Code review comments:
diff_hunk: @@ -813,24 +813,33 @@ def _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector)
     def _selector_matches(obj, selector):
         if selector is None:
             return True
+        # If selector is a dict, compare the fields
+        if (type(selector) == type(dict())) or isinstance(selector, BasePlotlyType):
+            # This returns True if selector is an empty dict
+            for k in selector:
+                if k not in obj:
+                    return False
 
-        for k in selector:
-            if k not in obj:
-                return False
-
-            obj_val = obj[k]
-            selector_val = selector[k]
-
-            if isinstance(obj_val, BasePlotlyType):
-                obj_val = obj_val.to_plotly_json()
+                obj_val = obj[k]
+                selector_val = selector[k]
 
-            if isinstance(selector_val, BasePlotlyType):
-                selector_val = selector_val.to_plotly_json()
+                if isinstance(obj_val, BasePlotlyType):
+                    obj_val = obj_val.to_plotly_json()
 
-            if obj_val != selector_val:
-                return False
+                if isinstance(selector_val, BasePlotlyType):
+                    selector_val = selector_val.to_plotly_json()
 
-        return True
+                if obj_val != selector_val:
+                    return False
+            return True
+        # If selector is a function, call it with the obj as the argument
+        elif type(selector) == type(lambda x: True):
+            return selector(obj)
+        else:
+            raise TypeError(
+                "selector must be dict or a function "
+                "accepting a trace returning a boolean."
path: packages/python/plotly/plotly/basedatatypes.py
text: @user1:
this is now more than just "a trace" right? maybe "a graph object" ?

@author:
ah yes, copy paste error! good catch :+1: