Repository: nolar/kopf

Issue:
Error when starting up kopf
## Long story short

Kopf starts up with error when using a handler with a callback & a core resource handler.

## Description

Not sure if this is a bug or just a poorly written callback but it doesn't seem to make sense why the callback is being called when handling core resources.
When I start kopf with the below handlers errors are logged by kopf on startup. However if I remove the `@kopf.on.create('apps', 'v1', 'deployments')` handler or change the callback to use `dict.get()` method then everything starts up correctly.

```python
import kopf

def is_good_enough(spec, **_):
    return spec['field'] in spec['items']

@kopf.on.create('company.com', 'v1', 'testresources', when=is_good_enough)
def my_create(spec, **_):
    pass

@kopf.on.create('apps', 'v1', 'deployments')
def create(spec, **kwargs):
    pass

```
</details>

<details><summary>The exact command to reproduce the issue</summary>

```bash
kopf run handler.py --standalone
```
</details>

<details><summary>The full output of the command that failed</summary>

```
2021-01-19 16:43:29,972] kopf.reactor.activit [INFO    ] Initial authentication has been initiated.
[2021-01-19 16:43:30,129] kopf.activities.auth [INFO    ] Activity 'login_via_client' succeeded.
[2021-01-19 16:43:30,129] kopf.reactor.activit [INFO    ] Initial authentication has finished.
[2021-01-19 16:43:30,359] kopf.objects         [ERROR   ] [kube-system/coredns] Throttling for 1 seconds due to an unexpected error:
Traceback (most recent call last):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/effects.py", line 200, in throttled
    yield should_run
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/processing.py", line 79, in process_resource_event
    delays, matched = await process_resource_causes(
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/processing.py", line 167, in process_resource_causes
    not registry._resource_changing.prematch(cause=resource_changing_cause)):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 211, in prematch
    if prematch(handler=handler, cause=cause):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 348, in prematch
    _matches_filter_callback(handler, cause, kwargs),
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 501, in _matches_filter_callback
    return handler.when(**kwargs)
  File "/Users/etilley/Documents/kubernetes/kopf/handler-testing/handler.py", line 11, in is_good_enough
    return spec['field'] in spec['items']
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/structs/dicts.py", line 249, in __getitem__
    return resolve(self._src, self._path + (item,))
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/structs/dicts.py", line 83, in resolve
    result = result[key]
KeyError: 'field'
[2021-01-19 16:43:30,368] kopf.objects         [ERROR   ] [local-path-storage/local-path-provisioner] Throttling for 1 seconds due to an unexpected error:
Traceback (most recent call last):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/effects.py", line 200, in throttled
    yield should_run
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/processing.py", line 79, in process_resource_event
    delays, matched = await process_resource_causes(
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/processing.py", line 167, in process_resource_causes
    not registry._resource_changing.prematch(cause=resource_changing_cause)):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 211, in prematch
    if prematch(handler=handler, cause=cause):
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 348, in prematch
    _matches_filter_callback(handler, cause, kwargs),
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/reactor/registries.py", line 501, in _matches_filter_callback
    return handler.when(**kwargs)
  File "/Users/etilley/Documents/kubernetes/kopf/handler-testing/handler.py", line 11, in is_good_enough
    return spec['field'] in spec['items']
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/structs/dicts.py", line 249, in __getitem__
    return resolve(self._src, self._path + (item,))
  File "/Users/etilley/.pyenv/versions/3.8.0/lib/python3.8/site-packages/kopf/structs/dicts.py", line 83, in resolve
    result = result[key]
KeyError: 'field'
[2021-01-19 16:43:31,361] kopf.objects         [INFO    ] [kube-system/coredns] Throttling is over. Switching back to normal operations.
[2021-01-19 16:43:31,369] kopf.objects         [INFO    ] [local-path-storage/local-path-provisioner] Throttling is over. Switching back to normal operations.
```
</details>


## Environment

<!-- The following commands can help:
     `kopf --version` or `pip show kopf`
     `kubectl version`
     `python --version`
-->

* Kopf version:  1.29.0rc2
* Kubernetes version:  v1.19.1
* Python version: 3.8.0
* OS/platform: mac OS

```
Package            Version
------------------ ---------
aiohttp            3.6.2
aiojobs            0.2.2
astroid            2.3.3
async-timeout      3.0.1
attrs              19.3.0
boto3              1.16.30
botocore           1.19.30
cachetools         3.1.1
certifi            2019.9.11
cffi               1.14.3
chardet            3.0.4
Click              7.0
cryptography       3.1.1
deepdiff           4.2.0
dictdiffer         0.8.1
elasticsearch      7.5.1
elasticsearch-dsl  7.1.0
google-auth        1.7.1
idna               2.8
iniconfig          1.1.1
iso8601            0.1.12
isort              4.3.21
javaobj-py3        0.4.1
Jinja2             2.10.3
jmespath           0.10.0
jsondiff           1.2.0
kopf               1.29.0rc2
kubernetes         10.0.1
kubetest           0.8.1
lazy-object-proxy  1.4.3
MarkupSafe         1.1.1
mccabe             0.6.1
multidict          4.6.1
oauthlib           3.1.0
ordered-set        3.1.1
packaging          20.4
pip                19.2.3
pluggy             0.13.1
prometheus-client  0.7.1
py                 1.9.0
pyasn1             0.4.8
pyasn1-modules     0.2.7
pycparser          2.20
pycryptodomex      3.9.8
pyjks              20.0.0
pykube-ng          19.10.0
pylint             2.4.4
pymongo            3.9.0
pyOpenSSL          19.1.0
pyparsing          2.4.7
pytest             6.1.2
python-dateutil    2.8.1
python-json-logger 2.0.0
PyYAML             5.1.2
rabbitmq-admin     0.2
requests           2.22.0
requests-oauthlib  1.3.0
rope               0.18.0
rsa                4.0
s3transfer         0.3.3
setuptools         41.2.0
six                1.13.0
toml               0.10.2
twofish            0.3.0
typing-extensions  3.7.4.1
urllib3            1.25.7
websocket-client   0.56.0
wrapt              1.11.2
yarl               1.3.0
```


Discussion:

Patch:
diff --git a/kopf/reactor/registries.py b/kopf/reactor/registries.py
index 87dcff3..d42f4a7 100644
--- a/kopf/reactor/registries.py
+++ b/kopf/reactor/registries.py
@@ -340,13 +340,13 @@ def prematch(
 ) -> bool:
     # Kwargs are lazily evaluated on the first _actual_ use, and shared for all filters since then.
     kwargs: MutableMapping[str, Any] = {}
-    return all([
-        _matches_resource(handler, cause.resource),
-        _matches_labels(handler, cause, kwargs),
-        _matches_annotations(handler, cause, kwargs),
-        _matches_field_values(handler, cause, kwargs),
-        _matches_filter_callback(handler, cause, kwargs),
-    ])
+    return (
+        _matches_resource(handler, cause.resource) and
+        _matches_labels(handler, cause, kwargs) and
+        _matches_annotations(handler, cause, kwargs) and
+        _matches_field_values(handler, cause, kwargs) and
+        _matches_filter_callback(handler, cause, kwargs)  # the callback comes in the end!
+    )
 
 
 def match(
@@ -355,14 +355,14 @@ def match(
 ) -> bool:
     # Kwargs are lazily evaluated on the first _actual_ use, and shared for all filters since then.
     kwargs: MutableMapping[str, Any] = {}
-    return all([
-        _matches_resource(handler, cause.resource),
-        _matches_labels(handler, cause, kwargs),
-        _matches_annotations(handler, cause, kwargs),
-        _matches_field_values(handler, cause, kwargs),
-        _matches_field_changes(handler, cause, kwargs),
-        _matches_filter_callback(handler, cause, kwargs),
-    ])
+    return (
+        _matches_resource(handler, cause.resource) and
+        _matches_labels(handler, cause, kwargs) and
+        _matches_annotations(handler, cause, kwargs) and
+        _matches_field_values(handler, cause, kwargs) and
+        _matches_field_changes(handler, cause, kwargs) and
+        _matches_filter_callback(handler, cause, kwargs)  # the callback comes in the end!
+    )
 
 
 def _matches_resource(


Test patch:
diff --git a/tests/registries/test_matching_of_callbacks.py b/tests/registries/test_matching_of_callbacks.py
new file mode 100644
index 0000000..5d1e02d
--- /dev/null
+++ b/tests/registries/test_matching_of_callbacks.py
@@ -0,0 +1,75 @@
+import dataclasses
+from unittest.mock import Mock
+
+import pytest
+
+from kopf.reactor.causation import ResourceWatchingCause
+from kopf.reactor.registries import match, prematch
+from kopf.structs.bodies import Body
+from kopf.structs.dicts import parse_field
+from kopf.structs.handlers import ResourceWatchingHandler
+from kopf.structs.references import Resource
+
+
+# Used in the tests. Must be global-scoped, or its qualname will be affected.
+def some_fn(x=None):
+    pass
+
+
+@pytest.fixture()
+def callback():
+    mock = Mock()
+    mock.return_value = True
+    return mock
+
+
+@pytest.fixture(params=['annotations', 'labels', 'value', 'when'])
+def handler(request, callback, selector):
+    handler = ResourceWatchingHandler(
+        selector=selector,
+        annotations={'known': 'value'},
+        labels={'known': 'value'},
+        field=parse_field('spec.field'),
+        value='value',
+        when=None,
+        fn=some_fn, id='a', errors=None, timeout=None, retries=None, backoff=None,  # irrelevant
+    )
+    if request.param in ['annotations', 'labels']:
+        handler = dataclasses.replace(handler, **{request.param: {'known': callback}})
+    else:
+        handler = dataclasses.replace(handler, **{request.param: callback})
+    return handler
+
+
+@pytest.fixture()
+def cause(cause_factory, callback):
+    return cause_factory(
+        cls=ResourceWatchingCause,
+        body=Body(dict(
+            metadata=dict(
+                labels={'known': 'value'},
+                annotations={'known': 'value'},
+            ),
+            spec=dict(
+                field='value',
+            ),
+        )))
+
+
+@pytest.mark.parametrize('match_fn', [match, prematch])
+def test_callback_is_called_with_matching_resource(
+        match_fn, callback, handler, cause,
+):
+    result = match_fn(handler=handler, cause=cause)
+    assert result
+    assert callback.called
+
+
+@pytest.mark.parametrize('match_fn', [match, prematch])
+def test_callback_is_not_called_with_mismatching_resource(
+        match_fn, callback, handler, cause,
+):
+    cause = dataclasses.replace(cause, resource=Resource(group='x', version='y', plural='z'))
+    result = match_fn(handler=handler, cause=cause)
+    assert not result
+    assert not callback.called