Repository: plotly/plotly.py

Issue:
select traces doesn't do partial matches on `meta`
Today, the output of 

```python
import plotly.graph_objects as go

fig = go.Figure(go.Scatter(meta=dict(a=1,b=2)))

print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1)))]))
```

is `1, 0`, meaning that we can't use `selector` to do partial matches on `meta`.

It would be really convenient if PX could tag all of its traces with some structured data in `meta` so that users could target these traces with updates, but for that to be useful we would need to do partial matches.

@jonmmease any major reason not to do this?


Discussion:
Does this do what you want?
```python
print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))
```

I seem to remember making the design decision that selectors keys needed to be matched in full, and magic underscore notation could be used to target subsets.  

Not sure off hand if anything goes wrong if we support partial matches, but it would be a breaking change. We'd also need to decide whether the partial match logic is applied recursively.


that doesn't work, no, possibly because `meta` is an open-ended dict with no schema below it?

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-20-9a7a6c9a1903> in <module>
      4 
      5 print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
----> 6 print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))

<ipython-input-20-9a7a6c9a1903> in <listcomp>(.0)
      4 
      5 print(len([f for f in fig.select_traces(selector=dict(meta=dict(a=1,b=2)))]))
----> 6 print(len([f for f in fig.select_traces(selector=dict(meta_a=1))]))

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector)
    805 
    806             # Filter by selector
--> 807             if not self._selector_matches(trace, selector):
    808                 continue
    809 

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in _selector_matches(obj, selector)
    816 
    817         for k in selector:
--> 818             if k not in obj:
    819                 return False
    820 

~/plotly/plotly.py/packages/python/plotly/plotly/basedatatypes.py in __contains__(self, prop)
   3907                     return False
   3908             else:
-> 3909                 if obj is not None and p in obj._valid_props:
   3910                     obj = obj[p]
   3911                 else:

AttributeError: 'dict' object has no attribute '_valid_props'
```
(to be clear: I would be fine with having this work, it would meet my needs just fine :)
Ok, yeah. this is special because meta doesn't have a sub-schema.

I think having special case magic underscore logic to handle dicts is a good approach.  This would also be useful in the `update_*` operations for things like:

```python
fig.update_layout(meta_a=8)
```

It would mean we're asserting that dict keys are strings without underscores.  Not sure at the moment if this should be a special case for `meta`, or if we would want this to work for any property that has a dict value :thinking:  But in either case, I think this would be a backward compatible change to make.
Or we could have it do partial matches off a dict, as a special case... either way :)
On further thought: I want to store keys in `meta` which are user-provided and likely to have underscores in them...
From reading the dialogue above, I gather that `select_traces` should allow partial matches using `meta` no matter if it was made using a dict or the underscore notation. Furthermore, `meta_a_b=value` should produce `meta=dict(a=dict(b=value))` but `meta=dict(a_b=another_value)` should be allowed, too.
Another thing: do you want another keyword in `select_traces`, say `partial_match` which allows turning on or off the functionality of matching subsets of trace parameters?
I would be OK if partial matches on `meta` didn't work with magic underscores at all actually, if that's easier to implement.

I don't think I would favour a new `partial_match` kwarg, no, I think we can just make a special case for `meta`
Ok sounds good.
I don't want to go too far afield here, but what if selector could be a function that returned `True` if that trace should be selected? The function would be passed the trace. The old dict functionality could be left in, too.
that's not a bad idea... I could implement my PX stuff on top of that, yes. Is that easier to build?
Maybe, I'm looking into it right now :+1: 
Yes I think it would be very easy, basically in `packages/python/plotly/plotly/basedatatypes.py` you have this function:
```python
def _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector): 
    from plotly.subplots import _get_subplot_ref_for_trace                        
                                                                                  
    for trace in self.data:                                                       
        # Filter by subplot                                                       
        if filter_by_subplot:                                                     
            trace_subplot_ref = _get_subplot_ref_for_trace(trace)                 
            if trace_subplot_ref not in grid_subplot_refs:                        
                continue                                                          
                                                                                  
        # Filter by selector                                                      
        if not self._selector_matches(trace, selector):                           
            continue                                                              
                                                                                  
        yield trace                                                               
```
So I would just add a check to see if selector is a `dict`, if it is, it calls `self._selector_matches`, if it's a function it calls that function instead.
And if we just leave the `dict` functionality as it is maybe it avoids breaking code that assumed a full match had to be made?
OK, let's do that then!
Would it be crazy to implement this for all the `fig.select_*` methods?

Resolving patch:
diff --git a/packages/python/plotly/plotly/basedatatypes.py b/packages/python/plotly/plotly/basedatatypes.py
--- a/packages/python/plotly/plotly/basedatatypes.py
+++ b/packages/python/plotly/plotly/basedatatypes.py
@@ -813,24 +813,33 @@ def _perform_select_traces(self, filter_by_subplot, grid_subplot_refs, selector)
     def _selector_matches(obj, selector):
         if selector is None:
             return True
+        # If selector is a dict, compare the fields
+        if (type(selector) == type(dict())) or isinstance(selector, BasePlotlyType):
+            # This returns True if selector is an empty dict
+            for k in selector:
+                if k not in obj:
+                    return False
 
-        for k in selector:
-            if k not in obj:
-                return False
-
-            obj_val = obj[k]
-            selector_val = selector[k]
-
-            if isinstance(obj_val, BasePlotlyType):
-                obj_val = obj_val.to_plotly_json()
+                obj_val = obj[k]
+                selector_val = selector[k]
 
-            if isinstance(selector_val, BasePlotlyType):
-                selector_val = selector_val.to_plotly_json()
+                if isinstance(obj_val, BasePlotlyType):
+                    obj_val = obj_val.to_plotly_json()
 
-            if obj_val != selector_val:
-                return False
+                if isinstance(selector_val, BasePlotlyType):
+                    selector_val = selector_val.to_plotly_json()
 
-        return True
+                if obj_val != selector_val:
+                    return False
+            return True
+        # If selector is a function, call it with the obj as the argument
+        elif type(selector) == type(lambda x: True):
+            return selector(obj)
+        else:
+            raise TypeError(
+                "selector must be dict or a function "
+                "accepting a graph object returning a boolean."
+            )
 
     def for_each_trace(self, fn, selector=None, row=None, col=None, secondary_y=None):
         """


Test patch:
diff --git a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py
new file mode 100644
--- /dev/null
+++ b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_selector_matches.py
@@ -0,0 +1,91 @@
+import pytest
+
+import plotly.graph_objects as go
+from plotly.basedatatypes import BaseFigure
+
+
+def test_selector_none():
+    # should return True
+    assert BaseFigure._selector_matches({}, None) == True  # arbitrary,
+
+
+def test_selector_empty_dict():
+    # should return True
+    assert (
+        BaseFigure._selector_matches(dict(hello="everybody"), {}) == True  # arbitrary,
+    )
+
+
+def test_selector_matches_subset_of_obj():
+    # should return True
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, today="cloudy"),
+        )
+        == True
+    )
+
+
+def test_selector_has_nonmatching_key():
+    # should return False
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, cronenberg="scanners"),
+        )
+        == False
+    )
+
+
+def test_selector_has_nonmatching_value():
+    # should return False
+    assert (
+        BaseFigure._selector_matches(
+            dict(hello="everybody", today="cloudy", myiq=55),
+            dict(myiq=55, today="sunny"),
+        )
+        == False
+    )
+
+
+def test_baseplotlytypes_could_match():
+    # should return True
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    assert BaseFigure._selector_matches(obj, sel) == True
+
+
+def test_baseplotlytypes_could_not_match():
+    # should return False
+    obj = go.layout.Annotation(x=1, y=3, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    assert BaseFigure._selector_matches(obj, sel) == False
+
+
+def test_baseplotlytypes_cannot_match_subset():
+    # should return False because "undefined" keys in sel return None, and are
+    # compared (because "key in sel" returned True, it's value was None)
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+    sel = go.layout.Annotation(x=1, y=2,)
+    assert BaseFigure._selector_matches(obj, sel) == False
+
+
+def test_function_selector_could_match():
+    # should return True
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+
+    def _sel(d):
+        return d["x"] == 1 and d["y"] == 2 and d["text"] == "pat metheny"
+
+    assert BaseFigure._selector_matches(obj, _sel) == True
+
+
+def test_function_selector_could_not_match():
+    # should return False
+    obj = go.layout.Annotation(x=1, y=2, text="pat metheny")
+
+    def _sel(d):
+        return d["x"] == 1 and d["y"] == 3 and d["text"] == "pat metheny"
+
+    assert BaseFigure._selector_matches(obj, _sel) == False
diff --git a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
--- a/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
+++ b/packages/python/plotly/plotly/tests/test_core/test_update_objects/test_update_traces.py
@@ -225,6 +225,43 @@ def test_select_property_and_grid(self):
         # Valid row/col and valid selector but the intersection is empty
         self.assert_select_traces([], selector={"type": "markers"}, row=3, col=1)
 
+    def test_select_with_function(self):
+        def _check_trace_key(k, v):
+            def f(t):
+                try:
+                    return t[k] == v
+                except LookupError:
+                    return False
+
+            return f
+
+        # (1, 1)
+        self.assert_select_traces(
+            [0], selector=_check_trace_key("mode", "markers"), row=1, col=1
+        )
+        self.assert_select_traces(
+            [1], selector=_check_trace_key("type", "bar"), row=1, col=1
+        )
+
+        # (2, 1)
+        self.assert_select_traces(
+            [2, 9], selector=_check_trace_key("mode", "lines"), row=2, col=1
+        )
+
+        # (1, 2)
+        self.assert_select_traces(
+            [4], selector=_check_trace_key("marker.color", "green"), row=1, col=2
+        )
+
+        # Valid row/col and valid selector but the intersection is empty
+        self.assert_select_traces(
+            [], selector=_check_trace_key("type", "markers"), row=3, col=1
+        )
+
+    def test_select_traces_type_error(self):
+        with self.assertRaises(TypeError):
+            self.assert_select_traces([0], selector=123, row=1, col=1)
+
     def test_for_each_trace_lowercase_names(self):
         # Names are all uppercase to start
         original_names = [t.name for t in self.fig.data]