Repository: zigpy/zigpy-znp (language: python)

Issue:
Connect multiple times on startup and speed up initial connect
Instead of just sending the bootloader skip bytes and connecting a single time, zigpy-znp now connects multiple times (similar to the way Z2M currently does it):

1. First just connect to the serial port.
2. If that doesn't work, try sending the bootloader skip bytes and try again.
3. If that doesn't work, try toggling the RTS pin and try once more.

This increases the chances of a connection attempt succeeding significantly while still taking the same amount of time as before.

A side effect of this change (and the original motivation) is that the CC2530 with an external USB-serial adapter is now fully supported.

To support custom coordinator boards (e.g. where RTS or DTR is mapped to RST), the pin toggling behavior is now configurable as well.
Slae.sh USB stick not working on v0.6.3
After updating HA OS to 2021.12.0b2, ZHA fails to start up. Downgrading to 2021.11.5 allows ZHA to start up again. Enabling debug logging shows the logs below for 2021.12.0b2:

```
2021-12-07 10:08:07 DEBUG (MainThread) [zigpy_znp.uart] Connecting to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 at 115200 baud
2021-12-07 10:08:07 DEBUG (MainThread) [zigpy_znp.uart] Opened /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 serial port
2021-12-07 10:08:07 DEBUG (MainThread) [zigpy_znp.uart] Connected to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 at 115200 baud
2021-12-07 10:08:07 DEBUG (MainThread) [zigpy_znp.api] Sending request: SYS.Ping.Req()
2021-12-07 10:08:10 DEBUG (MainThread) [zigpy_znp.api] Sending request: SYS.Ping.Req()
2021-12-07 10:08:10 DEBUG (MainThread) [zigpy_znp.api] Toggling RTS/DTR pins to skip bootloader or reset chip
2021-12-07 10:08:10 DEBUG (MainThread) [zigpy_znp.uart] Setting serial pin states: DTR=False, RTS=False
2021-12-07 10:08:10 DEBUG (MainThread) [zigpy_znp.uart] Setting serial pin states: DTR=False, RTS=True
2021-12-07 10:08:11 DEBUG (MainThread) [zigpy_znp.uart] Setting serial pin states: DTR=False, RTS=False
2021-12-07 10:08:11 DEBUG (MainThread) [zigpy_znp.api] Sending request: SYS.Ping.Req()
2021-12-07 10:08:12 DEBUG (MainThread) [zigpy_znp.api] Connection to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 failed, cleaning up
2021-12-07 10:08:12 DEBUG (MainThread) [zigpy_znp.uart] Closing serial port
2021-12-07 10:08:12 ERROR (MainThread) [zigpy.application] Couldn't start application
```

And these are the equivalent logs for 2021.11.5:

```
2021-12-07 10:18:35 DEBUG (MainThread) [zigpy_znp.uart] Connecting to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 at 115200 baud
2021-12-07 10:18:35 DEBUG (MainThread) [zigpy_znp.uart] Opened /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 serial port
2021-12-07 10:18:35 DEBUG (MainThread) [zigpy_znp.uart] Toggling RTS/CTS to skip CC2652R bootloader
2021-12-07 10:18:36 DEBUG (MainThread) [zigpy_znp.uart] Connected to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0 at 115200 baud
2021-12-07 10:18:36 DEBUG (MainThread) [zigpy_znp.api] Waiting 1s before sending anything
2021-12-07 10:18:42 DEBUG (MainThread) [zigpy_znp.api] Received command: SYS.ResetInd.Callback(Reason=<ResetReason.PowerUp: 0>, TransportRev=2, ProductId=1, MajorRel=2, MinorRel=7, MaintRel=1)
2021-12-07 10:18:42 WARNING (MainThread) [zigpy_znp.api] Received an unhandled command: SYS.ResetInd.Callback(Reason=<ResetReason.PowerUp: 0>, TransportRev=2, ProductId=1, MajorRel=2, MinorRel=7, MaintRel=1)
2021-12-07 10:18:43 DEBUG (MainThread) [zigpy_znp.api] Sending bootloader skip byte
2021-12-07 10:18:43 DEBUG (MainThread) [zigpy_znp.api] Waiting 1s or until a reset indication is received
2021-12-07 10:18:44 DEBUG (MainThread) [zigpy_znp.api] Testing connection to /dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_21_CC_45_F5-if00-port0
2021-12-07 10:18:44 DEBUG (MainThread) [zigpy_znp.api] Sending request: SYS.Ping.Req()
2021-12-07 10:18:44 DEBUG (MainThread) [zigpy_znp.api] Received command: SYS.Ping.Rsp(Capabilities=<MTCapabilities.APP_CNF|GP|UTIL|ZDO|AF|SYS: 1625>)
2021-12-07 10:18:44 DEBUG (MainThread) [zigpy_znp.api] Sending request: SYS.OSALNVLength.Req(Id=<OsalNvIds.NWKKEY: 130>)
2021-12-07 10:18:44 DEBUG (MainThread) [zigpy_znp.api] Received command: SYS.OSALNVLength.Rsp(ItemLen=24)
```

It appears connection to the slaesh USB stick is failing. This seems related to the new connection logic introduced in PR #104. Happy to provide more info if necessary. 


Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index b383dc94..a828f854 100644
--- a/README.md
+++ b/README.md
@@ -37,10 +37,11 @@ Launch Home Assistant with the `--skip-pip` command line option to prevent zigpy
  - Add https://github.com/home-assistant/hassio-addons-development as an addon repository.
  - Install the "Custom deps deployment" addon.
  - Add the following to your `configuration.yaml` file:
-	```yaml
-	pypi:
-	  - git+https://github.com/zigpy/zigpy-znp/
-	```
+   ```yaml
+   apk: []
+   pypi:
+     - git+https://github.com/zigpy/zigpy-znp/
+	 ```
 
 # Configuration
 Below are the defaults with the top-level Home Assistant `zha:` key.
diff --git a/zigpy_znp/api.py b/zigpy_znp/api.py
index 19599027..8e1c9ed4 100644
--- a/zigpy_znp/api.py
+++ b/zigpy_znp/api.py
@@ -38,7 +38,7 @@
 NETWORK_COMMISSIONING_TIMEOUT = 30
 BOOTLOADER_PIN_TOGGLE_DELAY = 0.15
 CONNECT_PING_TIMEOUT = 0.50
-CONNECT_PROBE_TIMEOUT = 5.0
+CONNECT_PROBE_TIMEOUT = 10
 
 
 class ZNP:
@@ -468,6 +468,16 @@ async def _skip_bootloader(self) -> c.SYS.Ping.Rsp:
         """
 
         async def ping_task():
+            LOGGER.debug("Toggling RTS/DTR pins to skip bootloader or reset chip")
+
+            # The default sequence is DTR=false and RTS toggling false/true/false
+            for dtr, rts in zip(
+                self._znp_config[conf.CONF_CONNECT_DTR_STATES],
+                self._znp_config[conf.CONF_CONNECT_RTS_STATES],
+            ):
+                self._uart.set_dtr_rts(dtr=dtr, rts=rts)
+                await asyncio.sleep(BOOTLOADER_PIN_TOGGLE_DELAY)
+
             # First, just try pinging
             try:
                 async with async_timeout.timeout(CONNECT_PING_TIMEOUT):
@@ -478,30 +488,18 @@ async def ping_task():
             # If that doesn't work, send the bootloader skip bytes and try again.
             # Sending a bunch at a time fixes UART issues when the radio was previously
             # probed with another library at a different baudrate.
+            LOGGER.debug("Sending CC253x bootloader skip bytes")
             self._uart.write(256 * bytes([c.ubl.BootloaderRunMode.FORCE_RUN]))
 
             await asyncio.sleep(AFTER_BOOTLOADER_SKIP_BYTE_DELAY)
 
-            try:
-                async with async_timeout.timeout(CONNECT_PING_TIMEOUT):
-                    return await self.request(c.SYS.Ping.Req())
-            except asyncio.TimeoutError:
-                pass
-
-            # This is normally done just for Slaesh's CC2652RB stick
-            LOGGER.debug("Toggling RTS/DTR pins to skip bootloader or reset chip")
-
-            # The default sequence is DTR=false and RTS toggling false/true/false
-            for dtr, rts in zip(
-                self._znp_config[conf.CONF_CONNECT_DTR_STATES],
-                self._znp_config[conf.CONF_CONNECT_RTS_STATES],
-            ):
-                self._uart.set_dtr_rts(dtr=dtr, rts=rts)
-                await asyncio.sleep(BOOTLOADER_PIN_TOGGLE_DELAY)
-
-            # At this point we have nothing else to try, don't catch the timeout
-            async with async_timeout.timeout(CONNECT_PING_TIMEOUT):
-                return await self.request(c.SYS.Ping.Req())
+            # At this point we have nothing left to try
+            while True:
+                try:
+                    async with async_timeout.timeout(2 * CONNECT_PING_TIMEOUT):
+                        return await self.request(c.SYS.Ping.Req())
+                except asyncio.TimeoutError:
+                    pass
 
         async with self.capture_responses([CatchAllResponse()]) as responses:
             ping_task = asyncio.create_task(ping_task())
diff --git a/zigpy_znp/zigbee/application.py b/zigpy_znp/zigbee/application.py
index a145583a..07000a53 100644
--- a/zigpy_znp/zigbee/application.py
+++ b/zigpy_znp/zigbee/application.py
@@ -135,15 +135,15 @@ async def probe(cls, device_config: conf.ConfigType) -> bool:
         LOGGER.debug("Probing %s", znp._port_path)
 
         try:
-            async with async_timeout.timeout(PROBE_TIMEOUT):
-                await znp.connect()
-
-            return True
+            # `ZNP.connect` times out on its own
+            await znp.connect()
         except Exception as e:
             LOGGER.debug(
                 "Failed to probe ZNP radio with config %s", device_config, exc_info=e
             )
             return False
+        else:
+            return True
         finally:
             znp.close()
 


Test patch:
diff --git a/tests/api/test_connect.py b/tests/api/test_connect.py
index c7168aab..3630437e 100644
--- a/tests/api/test_connect.py
+++ b/tests/api/test_connect.py
@@ -102,17 +102,9 @@ async def test_connect_skip_bootloader_rts_dtr_pins(make_znp_server, mocker):
     mocker.patch.object(znp.nvram, "determine_alignment", new=CoroutineMock())
     mocker.patch.object(znp, "detect_zstack_version", new=CoroutineMock())
 
-    num_pings = 0
-
-    def ping_rsp(req):
-        nonlocal num_pings
-        num_pings += 1
-
-        if num_pings >= 3:
-            return c.SYS.Ping.Rsp(Capabilities=t.MTCapabilities.SYS)
-
-    # Ignore the first few pings so we trigger the pin toggling
-    znp_server.reply_to(c.SYS.Ping.Req(), responses=ping_rsp)
+    znp_server.reply_to(
+        c.SYS.Ping.Req(), responses=[c.SYS.Ping.Rsp(Capabilities=t.MTCapabilities.SYS)]
+    )
 
     await znp.connect(test_port=True)