Repository: AppDaemon/appdaemon

Issue:
turn_on service not working with async apps
## What did I do
I bumped into an [issue](https://community.home-assistant.io/t/need-help-with-appdaemon-4-and-async/163937) from the community forum that resulted to be the same problem I had recently. Consider the following simple application:

```python
import appdaemon.plugins.hass.hassapi as hass

class Simple(hass.Hass):
    async def toggle(self, light):
        super().toggle(light)

    def initialize(self):
        self.create_task(self.toggle("light.0xccccccfffe995875_light"))
```

## What happened
Next error appears after 10 seconds that the app was initialized
```
2020-01-16 19:27:59.061352 WARNING test5: ------------------------------------------------------------
2020-01-16 19:27:59.062493 WARNING test5: Unexpected error running initialize() for test5
2020-01-16 19:27:59.063452 WARNING test5: ------------------------------------------------------------
2020-01-16 19:27:59.064921 WARNING test5: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/appdaemon/app_management.py", line 145, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/lib/python3.8/site-packages/appdaemon/utils.py", line 276, in run_in_executor
    response = future.result()
asyncio.exceptions.TimeoutError
2020-01-16 19:27:59.065899 WARNING test5: ------------------------------------------------------------
2020-01-16 19:27:59.072580 WARNING AppDaemon: Excessive time spent in utility loop: 10089.0ms, 10088.0ms in check_app_updates(), 1.0ms in other
```

However, if I remove the `async` keyword, it works. Also, if I change `super().toggle(light)` for `self.call_service("homeassistant/toggle", entity_id=light)` and leaving the `async`, it works.

So the problem is that `self.toggle` does not work with async features. It also does not work with `self.turn_on` and `self.turn_off`.

## What do I expect
I'd expect self.toggle to work with async functions.



Discussion:
Same here, tried to use this in my AutoMoLi app but it shows exactly the same behavior described here 
all i can say is:

AD now has the option to create async apps, but that option is created to use where you cant get around it.
i wouldnt advise to start writing all apps in async, just because it can be done.

the threaded sync core from AD is way more stable, and apps build without async in general wont effect each other, which is good because you dont want all apps to fail, when 1 app fails.

that said ill ask @Odianosen25 or @dlashua to look at why those functions dont work with async, or what you need to get them to work.
@xaviml,

It is true that the `turn_on` in async isn't working as I have tested it, and we will try fixing it before the next release. 
argh shit, spent hours to debug this because I thought it must be my buggy code 😂🤦‍♀️

thanks in advance for fixing :D
No worries 😂. But honestly still haven't figured out why it happens. Its a bit strange
@benleb you can use self.call_service("homeassistant/turn_on") as a workaround.
We will be looking at this for after the 4.0.1 bugfix release
I had the same issue and as a workaround just replaced turn_on() with call_service("your_domain/turn_on").
I think that the problem here is that the [`hass_check`](https://github.com/home-assistant/appdaemon/blob/dev/appdaemon/plugins/hass/hassapi.py) decorator calls `run_coroutine_threadsafe`, which submits a coroutine to the event loop and waits for its completion. However, since this method is called from the event loop itself, that waiting blocks the event loop and as such the coroutine we're waiting for never gets to run.

The cleanest solution is probably to invert the order of the `sync_wrapper` and `hass_check` decorators, and write the hass_check method in async style.

Resolving patch:
diff --git a/appdaemon/__main__.py b/appdaemon/__main__.py
--- a/appdaemon/__main__.py
+++ b/appdaemon/__main__.py
@@ -7,22 +7,30 @@
 
 """
 
-import sys
 import argparse
+import asyncio
 import os
 import os.path
-import signal
 import platform
-import yaml
-import asyncio
-import pytz
-import pid
-import uvloop
+import signal
+import sys
 
-import appdaemon.utils as utils
 import appdaemon.appdaemon as ad
 import appdaemon.http as adhttp
 import appdaemon.logging as logging
+import appdaemon.utils as utils
+import pytz
+import yaml
+
+try:
+    import pid
+except ImportError:
+    pid = None
+
+try:
+    import uvloop
+except ImportError:
+    uvloop = None
 
 
 class ADMain:
@@ -113,7 +121,7 @@ def run(self, appdaemon, hadashboard, admin, api, http):
         try:
 
             # if to use uvloop
-            if appdaemon.get("uvloop") is True:
+            if appdaemon.get("uvloop") is True and uvloop:
                 self.logger.info("Running AD using uvloop")
                 uvloop.install()
 
diff --git a/appdaemon/plugins/hass/hassapi.py b/appdaemon/plugins/hass/hassapi.py
--- a/appdaemon/plugins/hass/hassapi.py
+++ b/appdaemon/plugins/hass/hassapi.py
@@ -13,22 +13,22 @@
 requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
 
 
-def hass_check(func):
-    @wraps(func)
-    def func_wrapper(*args, **kwargs):
+def hass_check(coro):
+    @wraps(coro)
+    async def coro_wrapper(*args, **kwargs):
         self = args[0]
         ns = self._get_namespace(**kwargs)
-        plugin = utils.run_coroutine_threadsafe(self, self.AD.plugins.get_plugin_object(ns))
+        plugin = await self.AD.plugins.get_plugin_object(ns)
         if plugin is None:
-            self.logger.warning("non_existent namespace (%s) specified in call to %s", ns, func.__name__)
-            return lambda *args: None
-        if not utils.run_coroutine_threadsafe(self, plugin.am_reading_messages()):
-            self.logger.warning("Attempt to call Home Assistant while disconnected: %s", func.__name__)
-            return lambda *args: None
+            self.logger.warning("non_existent namespace (%s) specified in call to %s", ns, coro.__name__)
+            return None
+        if not await plugin.am_reading_messages():
+            self.logger.warning("Attempt to call Home Assistant while disconnected: %s", coro.__name__)
+            return None
         else:
-            return func(*args, **kwargs)
+            return await coro(*args, **kwargs)
 
-    return func_wrapper
+    return coro_wrapper
 
 
 #
@@ -336,8 +336,8 @@ def constrain_input_select(self, value):
     # Helper functions for services
     #
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def turn_on(self, entity_id, **kwargs):
         """Turns `on` a Home Assistant entity.
 
@@ -387,8 +387,8 @@ async def turn_on(self, entity_id, **kwargs):
         rargs["namespace"] = namespace
         await self.call_service("homeassistant/turn_on", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def turn_off(self, entity_id, **kwargs):
         """Turns `off` a Home Assistant entity.
 
@@ -436,8 +436,8 @@ async def turn_off(self, entity_id, **kwargs):
         else:
             await self.call_service("homeassistant/turn_off", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def toggle(self, entity_id, **kwargs):
         """Toggles between ``on`` and ``off`` for the selected entity.
 
@@ -477,8 +477,8 @@ async def toggle(self, entity_id, **kwargs):
         rargs["namespace"] = namespace
         await self.call_service("homeassistant/toggle", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def set_value(self, entity_id, value, **kwargs):
         """Sets the value of an `input_number`.
 
@@ -517,8 +517,8 @@ async def set_value(self, entity_id, value, **kwargs):
         rargs["namespace"] = namespace
         await self.call_service("input_number/set_value", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def set_textvalue(self, entity_id, value, **kwargs):
         """Sets the value of an `input_text`.
 
@@ -558,8 +558,8 @@ async def set_textvalue(self, entity_id, value, **kwargs):
         rargs["namespace"] = namespace
         await self.call_service("input_text/set_value", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def select_option(self, entity_id, option, **kwargs):
         """Sets the value of an `input_option`.
 
@@ -602,8 +602,8 @@ async def select_option(self, entity_id, option, **kwargs):
         rargs["namespace"] = namespace
         await self.call_service("input_select/select_option", **rargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def notify(self, message, **kwargs):
         """Sends a notification.
 
@@ -639,8 +639,8 @@ async def notify(self, message, **kwargs):
 
         await self.call_service(service, **kwargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def persistent_notification(self, message, title=None, id=None):
         """
 
@@ -662,8 +662,8 @@ async def persistent_notification(self, message, title=None, id=None):
             kwargs["notification_id"] = id
         await self.call_service("persistent_notification/create", **kwargs)
 
-    @hass_check
     @utils.sync_wrapper
+    @hass_check
     async def get_history(self, **kwargs):
         """Gets access to the HA Database.
         This is a convenience function that allows accessing the HA Database, so the
@@ -747,8 +747,9 @@ async def get_history(self, **kwargs):
             )
             return None
 
+    @utils.sync_wrapper
     @hass_check
-    def render_template(self, template, **kwargs):
+    async def render_template(self, template, **kwargs):
         """Renders a Home Assistant Template
 
         Args:
@@ -780,7 +781,7 @@ def render_template(self, template, **kwargs):
         rargs["namespace"] = namespace
         rargs["template"] = template
 
-        result = self.call_service("template/render", **rargs)
+        result = await self.call_service("template/render", **rargs)
         try:
             return literal_eval(result)
         except (SyntaxError, ValueError):


Test patch: