Repository: XKNX/xknx
Pull request #933 (2022-04-28 10:11:32)

Bump cryptography from 37.0.0 to 37.0.1
Bumps [cryptography](https://github.com/pyca/cryptography) from 37.0.0 to 37.0.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's changelog</a>.</em></p>
<blockquote>
<p>37.0.1 - 2022-04-27</p>
<pre><code>
* Fixed an issue where parsing an encrypted private key with the public
  loader functions would hang waiting for console input on OpenSSL 3.0.x rather
  than raising an error.
* Restored some legacy symbols for older ``pyOpenSSL`` users. These will be
  removed again in the future, so ``pyOpenSSL`` users should still upgrade
  to the latest version of that package when they upgrade ``cryptography``.
<p>.. _v37-0-0:
</code></pre></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pyca/cryptography/commit/3fb93cfde75d073a91bc4a73a51f62962092501e"><code>3fb93cf</code></a> 37.0.1 changelog and version bump (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7139">#7139</a>)</li>
<li><a href="https://github.com/pyca/cryptography/commit/8ec3192af06c566691d8ce24797a1c0f49393469"><code>8ec3192</code></a> restore some bindings for older pyopenssl temporarily (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7137">#7137</a>)</li>
<li><a href="https://github.com/pyca/cryptography/commit/2d4e0b29646cbc40b701f258a330ed8217b688b8"><code>2d4e0b2</code></a> Fix parsing of priv keys via pub key APIs to error properly in ossl3 (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7135">#7135</a>)</li>
<li><a href="https://github.com/pyca/cryptography/commit/82d9339b3b13df07ae3514b5945376a2d8ee7607"><code>82d9339</code></a> Add typings to default_backend() (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7133">#7133</a>)</li>
<li><a href="https://github.com/pyca/cryptography/commit/cfdfb1a382c8c3201971d35aed91b37288c350d0"><code>cfdfb1a</code></a> update wheel builder for lib64 path (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7122">#7122</a>)</li>
<li><a href="https://github.com/pyca/cryptography/commit/c45033108ec06150cb50173ed1761ae8a4475468"><code>c450331</code></a> pep 527 actually prohibits xztar (<a href="https://github-redirect.dependabot.com/pyca/cryptography/issues/7121">#7121</a>)</li>
<li>See full diff in <a href="https://github.com/pyca/cryptography/compare/37.0.0...37.0.1">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=37.0.0&new-version=37.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

codecov[bot]: # [Codecov](https://codecov.io/gh/XKNX/xknx/pull/933?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XKNX) Report
> Merging [#933](https://codecov.io/gh/XKNX/xknx/pull/933?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XKNX) (907ca7f) into [main](https://codecov.io/gh/XKNX/xknx/commit/8c6d0e61de0fe848ac16f716c1802319447546b2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XKNX) (8c6d0e6) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/XKNX/xknx/pull/933/graphs/tree.svg?width=650&height=150&src=pr&token=irWbIygS84&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XKNX)](https://codecov.io/gh/XKNX/xknx/pull/933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XKNX)

```diff
@@           Coverage Diff           @@
##             main     #933   +/-   ##
=======================================
  Coverage   95.66%   95.66%           
=======================================
  Files         130      130           
  Lines        7767     7767           
=======================================
  Hits         7430     7430           
  Misses        337      337           
```




Patch:
diff --git a/changelog.md b/changelog.md
index 9e4d6fa1..9e2ac1c4 100644
--- a/changelog.md
+++ b/changelog.md
@@ -2,6 +2,8 @@
 
 ## Unreleased changes
 
+## 0.21.0 Search and connect 2022-04-30
+
 ### Discovery
 
 - Use unicast discovery endpoint to receive SearchRespones frames
@@ -9,13 +11,14 @@
 - Skip SearchResponse results for Core-V2 devices - wait for SearchResponseExtended
 - Identify interfaces having KNX IP Secure Tunneling required and skip if using Automatic connection mode
 - Only send SearchRequests from one interface for each `scan()` call
+- Connect to next found interface in case of unsuccessful initial connection when using "automatic" mode
 
 ### Internals
 
 - Use `ifaddr` instead of `netifaces`
 - make HPAI hashable and add `addr_tuple` convenice property
 
-## 0.20.4 Fix exposure of time and date
+## 0.20.4 Fix exposure of time and date 2022-04-20
 
 ### Bugfixes
 
diff --git a/requirements/production.txt b/requirements/production.txt
index c9a013b8..d202a09c 100644
--- a/requirements/production.txt
+++ b/requirements/production.txt
@@ -1,2 +1,2 @@
-cryptography==37.0.0
+cryptography==37.0.1
 ifaddr==0.1.7
diff --git a/xknx/__version__.py b/xknx/__version__.py
index ad78310e..35ca99b7 100644
--- a/xknx/__version__.py
+++ b/xknx/__version__.py
@@ -1,3 +1,3 @@
 """XKNX version."""
 
-__version__ = "0.20.4"
+__version__ = "0.21.0"
diff --git a/xknx/core/task_registry.py b/xknx/core/task_registry.py
index 330891d9..b4f848b2 100644
--- a/xknx/core/task_registry.py
+++ b/xknx/core/task_registry.py
@@ -3,11 +3,11 @@ from __future__ import annotations
 
 import asyncio
 import logging
-from typing import TYPE_CHECKING, Any, Awaitable, Generator, Union
+from typing import TYPE_CHECKING, Any, Coroutine, Generator, Union
 
 from xknx.core import XknxConnectionState
 
-AsyncCallbackType = Union[Generator[Any, None, Any], Awaitable]
+AsyncCallbackType = Union[Generator[Any, None, Any], Coroutine[Any, Any, None]]
 
 if TYPE_CHECKING:
     from xknx import XKNX
diff --git a/xknx/io/gateway_scanner.py b/xknx/io/gateway_scanner.py
index 61afc268..843a4da7 100644
--- a/xknx/io/gateway_scanner.py
+++ b/xknx/io/gateway_scanner.py
@@ -9,7 +9,7 @@ from __future__ import annotations
 import asyncio
 from functools import partial
 import logging
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, AsyncGenerator
 
 from xknx.exceptions import XKNXException
 from xknx.io import util
@@ -182,27 +182,51 @@ class GatewayScanner:
     def __init__(
         self,
         xknx: XKNX,
-        timeout_in_seconds: float = 4.0,
-        stop_on_found: int | None = 1,
+        local_ip: str | None = None,
+        timeout_in_seconds: float = 3.0,
+        stop_on_found: int | None = None,
         scan_filter: GatewayScanFilter = GatewayScanFilter(),
     ):
         """Initialize GatewayScanner class."""
         self.xknx = xknx
+        self.local_ip = local_ip
         self.timeout_in_seconds = timeout_in_seconds
-        self.stop_on_found = stop_on_found or 0
+        self.stop_on_found = stop_on_found
         self.scan_filter = scan_filter
         self.found_gateways: dict[HPAI, GatewayDescriptor] = {}
         self._response_received_event = asyncio.Event()
 
-    async def scan(
-        self,
-        local_ip: str | None = None,
-    ) -> list[GatewayDescriptor]:
+    async def scan(self) -> list[GatewayDescriptor]:
         """Scan and return a list of GatewayDescriptors on success."""
-        local_ip = local_ip or await util.get_default_local_ip(
+        await self._scan()
+        return list(self.found_gateways.values())
+
+    async def async_scan(self) -> AsyncGenerator[GatewayDescriptor, None]:
+        """Search and yield found gateways."""
+        queue: asyncio.Queue[GatewayDescriptor | None] = asyncio.Queue()
+        scan_task = asyncio.create_task(self._scan(queue=queue))
+        try:
+            while True:
+                gateway = await queue.get()
+                if gateway is None:
+                    return
+                yield gateway
+        finally:
+            # cleanup after GeneratorExit or XKNXExceptions
+            if not scan_task.done():
+                scan_task.cancel()
+            await scan_task  # to bubble up exceptions
+
+    async def _scan(
+        self, queue: asyncio.Queue[GatewayDescriptor | None] | None = None
+    ) -> None:
+        """Scan for gateways."""
+        local_ip = self.local_ip or await util.get_default_local_ip(
             remote_ip=self.xknx.multicast_group
         )
         if local_ip is None:
+            if queue is not None:
+                queue.put_nowait(None)
             raise XKNXException("No usable network interface found.")
         interface_name = util.get_local_interface_name(local_ip=local_ip)
         logger.debug("Searching on %s / %s", interface_name, local_ip)
@@ -212,7 +236,7 @@ class GatewayScanner:
             remote_addr=(self.xknx.multicast_group, self.xknx.multicast_port),
         )
         udp_transport.register_callback(
-            partial(self._response_rec_callback, interface=interface_name),
+            partial(self._response_rec_callback, interface=interface_name, queue=queue),
             [
                 KNXIPServiceType.SEARCH_RESPONSE,
                 KNXIPServiceType.SEARCH_RESPONSE_EXTENDED,
@@ -226,10 +250,12 @@ class GatewayScanner:
             )
         except asyncio.TimeoutError:
             pass
+        except asyncio.CancelledError:
+            pass
         finally:
             udp_transport.stop()
-
-        return list(self.found_gateways.values())
+            if queue is not None:
+                queue.put_nowait(None)
 
     @staticmethod
     async def _send_search_requests(udp_transport: UDPTransport) -> None:
@@ -261,6 +287,7 @@ class GatewayScanner:
         source: HPAI,
         udp_transport: UDPTransport,
         interface: str = "",
+        queue: asyncio.Queue[GatewayDescriptor | None] | None = None,
     ) -> None:
         """Verify and handle knxipframe. Callback from internal udp_transport."""
         if not isinstance(knx_ip_frame.body, (SearchResponse, SearchResponseExtended)):
@@ -292,6 +319,7 @@ class GatewayScanner:
         logger.debug("Found KNX/IP device at %s: %s", source, repr(gateway))
         if self.scan_filter.match(gateway):
             self.found_gateways[knx_ip_frame.body.control_endpoint] = gateway
-
-            if len(self.found_gateways) >= self.stop_on_found:
+            if queue is not None:
+                queue.put_nowait(gateway)
+            if self.stop_on_found and len(self.found_gateways) >= self.stop_on_found:
                 self._response_received_event.set()
diff --git a/xknx/io/knxip_interface.py b/xknx/io/knxip_interface.py
index 67138ede..9888338b 100644
--- a/xknx/io/knxip_interface.py
+++ b/xknx/io/knxip_interface.py
@@ -23,7 +23,7 @@ from xknx.io import util
 from xknx.secure import Keyring, load_key_ring
 
 from .connection import ConnectionConfig, ConnectionType
-from .gateway_scanner import GatewayDescriptor, GatewayScanFilter, GatewayScanner
+from .gateway_scanner import GatewayDescriptor, GatewayScanner
 from .routing import Routing
 from .tunnel import SecureTunnel, TCPTunnel, UDPTunnel, _Tunnel
 
@@ -139,24 +139,34 @@ class KNXIPInterface:
 
     async def _start_automatic(self) -> None:
         """Start GatewayScanner and connect to the found device."""
-        gateway = await self.find_gateway(
-            scan_filter=self.connection_config.scan_filter,  # secure disabled by default
+        async for gateway in GatewayScanner(
+            self.xknx,
             local_ip=self.connection_config.local_ip,
-        )
-        self._gateway_info = gateway
-
-        if gateway.supports_tunnelling_tcp:
-            await self._start_tunnelling_tcp(
-                gateway_ip=gateway.ip_addr,
-                gateway_port=gateway.port,
-            )
-        elif gateway.supports_tunnelling:
-            await self._start_tunnelling_udp(
-                gateway_ip=gateway.ip_addr,
-                gateway_port=gateway.port,
-            )
-        elif gateway.supports_routing:
-            await self._start_routing(local_ip=self.connection_config.local_ip)
+            scan_filter=self.connection_config.scan_filter,  # secure disabled by default
+        ).async_scan():
+            try:
+                if gateway.supports_tunnelling_tcp:
+                    await self._start_tunnelling_tcp(
+                        gateway_ip=gateway.ip_addr,
+                        gateway_port=gateway.port,
+                    )
+                elif gateway.supports_tunnelling:
+                    await self._start_tunnelling_udp(
+                        gateway_ip=gateway.ip_addr,
+                        gateway_port=gateway.port,
+                    )
+                elif gateway.supports_routing:
+                    await self._start_routing(
+                        local_ip=self.connection_config.local_ip,
+                    )
+            except CommunicationError as ex:
+                logger.debug("Could not connect to %s: %s", gateway, ex)
+                continue
+            else:
+                self._gateway_info = gateway
+                break
+        else:
+            raise CommunicationError("No usable KNX/IP device found.")
 
     async def _start_tunnelling_tcp(
         self,
@@ -283,18 +293,6 @@ class KNXIPInterface:
             return await self._interface.request_description()
         return None
 
-    async def find_gateway(
-        self,
-        scan_filter: GatewayScanFilter,
-        local_ip: str | None = None,
-    ) -> GatewayDescriptor:
-        """Find KNX/IP Gateway."""
-        gatewayscanner = GatewayScanner(self.xknx, scan_filter=scan_filter)
-        gateways = await gatewayscanner.scan(local_ip=local_ip)
-        if not gateways:
-            raise XKNXException("No Gateways found")
-        return gateways[0]
-
 
 class KNXIPInterfaceThreaded(KNXIPInterface):
     """Class for managing KNX/IP Tunneling or Routing connections."""


Test patch:
diff --git a/requirements/testing.txt b/requirements/testing.txt
index e74fac75..b165acbe 100644
--- a/requirements/testing.txt
+++ b/requirements/testing.txt
@@ -10,4 +10,4 @@ pytest-asyncio==0.18.3
 pytest-cov==3.0.0
 setuptools==62.1.0
 tox==3.25.0
-mypy==0.942
+mypy==0.950
diff --git a/test/io_tests/gateway_scanner_test.py b/test/io_tests/gateway_scanner_test.py
index dc51b5df..e98f3ec8 100644
--- a/test/io_tests/gateway_scanner_test.py
+++ b/test/io_tests/gateway_scanner_test.py
@@ -1,8 +1,11 @@
 """Unit test for KNX/IP gateway scanner."""
 import asyncio
-from unittest.mock import create_autospec, patch
+from unittest.mock import Mock, create_autospec, patch
+
+import pytest
 
 from xknx import XKNX
+from xknx.exceptions import XKNXException
 from xknx.io import GatewayScanFilter, GatewayScanner
 from xknx.io.gateway_scanner import GatewayDescriptor
 from xknx.io.transport import UDPTransport
@@ -170,7 +173,19 @@ class TestGatewayScanner:
             gateway_scanner.found_gateways[test_search_response.body.control_endpoint]
         ) == str(self.gateway_desc_both)
 
-    async def test_scan_timeout(self, time_travel):
+    @patch("xknx.io.gateway_scanner.UDPTransport.connect")
+    @patch("xknx.io.gateway_scanner.UDPTransport.send")
+    @patch(
+        "xknx.io.gateway_scanner.UDPTransport.getsockname",
+        return_value=("10.1.1.2", 56789),
+    )
+    async def test_scan_timeout(
+        self,
+        getsockname_mock,
+        udp_transport_send_mock,
+        udp_transport_connect_mock,
+        time_travel,
+    ):
         """Test gateway scanner timeout."""
         xknx = XKNX()
         gateway_scanner = GatewayScanner(xknx)
@@ -179,6 +194,83 @@ class TestGatewayScanner:
         # Unsuccessfull scan() returns empty list
         assert await timed_out_scan_task == []
 
+    @patch("xknx.io.gateway_scanner.UDPTransport.connect")
+    @patch("xknx.io.gateway_scanner.UDPTransport.send")
+    async def test_async_scan_timeout(
+        self,
+        udp_transport_send_mock,
+        udp_transport_connect_mock,
+        time_travel,
+    ):
+        """Test gateway scanner timeout for async generator."""
+
+        async def test():
+            xknx = XKNX()
+            async for _ in GatewayScanner(xknx).async_scan():
+                break
+            else:
+                return True
+
+        # timeout
+        with patch(
+            "xknx.io.util.get_default_local_ip",
+            return_value="10.1.1.2",
+        ), patch(
+            "xknx.io.gateway_scanner.UDPTransport.getsockname",
+            return_value=("10.1.1.2", 56789),
+        ):
+            timed_out_scan_task = asyncio.create_task(test())
+            await time_travel(3)
+            assert await timed_out_scan_task
+        # no matching interface found
+        with patch(
+            "xknx.io.util.get_default_local_ip",
+            return_value=None,
+        ):
+            timed_out_scan_task = asyncio.create_task(test())
+            await time_travel(3)
+            with pytest.raises(XKNXException):
+                await timed_out_scan_task
+
+    @patch("xknx.io.gateway_scanner.UDPTransport.connect")
+    @patch("xknx.io.gateway_scanner.UDPTransport.send")
+    async def test_async_scan_exit(
+        self,
+        udp_transport_send_mock,
+        udp_transport_connect_mock,
+        time_travel,
+    ):
+        """Test gateway scanner timeout for async generator."""
+        xknx = XKNX()
+        test_search_response = fake_router_search_response()
+        udp_transport_mock = Mock()
+        udp_transport_mock.local_addr = ("10.1.1.2", 56789)
+
+        gateway_scanner = GatewayScanner(xknx, local_ip="10.1.1.2")
+
+        async def test():
+            async for gateway in gateway_scanner.async_scan():
+                assert isinstance(gateway, GatewayDescriptor)
+                return True
+            return False
+
+        with patch(
+            "xknx.io.gateway_scanner.UDPTransport.getsockname",
+            return_value=("10.1.1.2", 56789),
+        ), patch(
+            "xknx.io.gateway_scanner.UDPTransport.register_callback"
+        ) as register_callback_mock:
+            scan_task = asyncio.create_task(test())
+            await time_travel(0)
+            _fished_response_rec_callback = register_callback_mock.call_args.args[0]
+            _fished_response_rec_callback(
+                test_search_response,
+                HPAI("192.168.42.50", 0),
+                udp_transport_mock,
+            )
+            assert await scan_task
+            await time_travel(0)  # for task cleanup
+
     @patch("xknx.io.gateway_scanner.UDPTransport.connect")
     @patch("xknx.io.gateway_scanner.UDPTransport.send")
     async def test_send_search_requests(
diff --git a/test/io_tests/knxip_interface_test.py b/test/io_tests/knxip_interface_test.py
index a50a074e..0af39050 100644
--- a/test/io_tests/knxip_interface_test.py
+++ b/test/io_tests/knxip_interface_test.py
@@ -7,12 +7,14 @@ import pytest
 
 from xknx import XKNX
 from xknx.exceptions.exception import (
+    CommunicationError,
     InterfaceWithUserIdNotFound,
     InvalidSecureConfiguration,
 )
 from xknx.io import (
     ConnectionConfig,
     ConnectionType,
+    GatewayDescriptor,
     SecureConfig,
     knx_interface_factory,
 )
@@ -32,12 +34,42 @@ class TestKNXIPInterface:
         """Test starting automatic connection."""
         connection_config = ConnectionConfig()
         assert connection_config.connection_type == ConnectionType.AUTOMATIC
+        interface = knx_interface_factory(self.xknx, connection_config)
         with patch("xknx.io.KNXIPInterface._start_automatic") as start_automatic_mock:
-            interface = knx_interface_factory(self.xknx, connection_config)
             await interface.start()
             start_automatic_mock.assert_called_once_with()
             assert threading.active_count() == 1
 
+        async def gateway_generator_mock(_):
+            yield GatewayDescriptor(
+                ip_addr="10.1.2.3", port=3671, supports_tunnelling_tcp=True
+            )
+            yield GatewayDescriptor(
+                ip_addr="10.1.2.3", port=3671, supports_tunnelling=True
+            )
+            yield GatewayDescriptor(
+                ip_addr="10.1.2.3", port=3671, supports_routing=True
+            )
+
+        with patch(
+            "xknx.io.knxip_interface.GatewayScanner.async_scan",
+            new=gateway_generator_mock,
+        ), patch(
+            "xknx.io.KNXIPInterface._start_tunnelling_tcp",
+            side_effect=CommunicationError("Error"),
+        ) as start_tunnelling_tcp_mock, patch(
+            "xknx.io.KNXIPInterface._start_tunnelling_udp",
+            side_effect=CommunicationError("Error"),
+        ) as start_tunnelling_udp_mock, patch(
+            "xknx.io.KNXIPInterface._start_routing",
+            side_effect=CommunicationError("Error"),
+        ) as start_routing_mock:
+            with pytest.raises(CommunicationError):
+                await interface.start()
+            start_tunnelling_tcp_mock.assert_called_once()
+            start_tunnelling_udp_mock.assert_called_once()
+            start_routing_mock.assert_called_once()
+
     async def test_start_udp_tunnel_connection(self):
         """Test starting UDP tunnel connection."""
         # without gateway_ip automatic is called