Repository: tornadoweb/tornado

Issue:
Asyncio equivalent of call_soon
Currently calling `add_callback` with Tornado's asyncio backend eventually calls `call_soon_threadsafe`, which incurs a bit of a cost (see [this stack overflow question](https://stackoverflow.com/questions/51582394/which-functions-are-free-when-profiling-tornado-asyncio))

Would it be cheaper to call asyncio's `call_soon` instead when scheduling a callback from code operating on the same thread as the event loop?

If yes, then is there any Tornado API that allows this.  By grepping through the codebase I think that there isn't. Should there be?

Also, I modified tornado's `add_callback` to include a `threadsafe` keyword to try this out and got a reduction of overhead, but also a surprising increase in latency that I can't explain.  I suspect that this is something to do with my system, but thought I'd mention it in case someone else could explain it easily.

```diff
+++ b/tornado/platform/asyncio.py
@@ -146,8 +146,13 @@ class BaseAsyncIOLoop(IOLoop):
         timeout.cancel()
 
     def add_callback(self, callback, *args, **kwargs):
+        threadsafe = kwargs.pop('threadsafe', False)
+        if threadsafe:
+            call_soon = self.asyncio_loop.call_soon
+        else:
+            call_soon = self.asyncio_loop.call_soon_threadsafe
         try:
-            self.asyncio_loop.call_soon_threadsafe(
+            call_soon(
                 self._run_callback,
                 functools.partial(callback, *args, **kwargs))
         except RuntimeError:
```


Discussion:
Sorry, to be clear the increase in latency was when I used this in `ioloop.py::IOLoop.add_future`'s use of `add_callback`
Yeah, that seems like a worthwhile optimization. Tornado's `add_callback` was thread-safe with an optimization for the same-thread case; I naively assumed that asyncio's `call_soon_threadsafe` would do the same. Since it doesn't, we should probably reintroduce a special case like that to `AsyncIOLoop.add_callback`. 

The only trick is how to identify the event loop's thread, since we don't necessarily start the loop ourselves anymore. Maybe we schedule a callback at startup to record the thread ID when it first runs. 
> Maybe we schedule a callback at startup to record the thread ID when it first runs

I do something similar in Dask today for some of our internal use cases.  I've been pretty happy with it.  
I'm happy to implement this if desired.  My guess is that the required changes would be small.  I would propose the following:

1.  In `IOLoop.initialize` add a callback to set `self.thread_identity` to `threading.get_ident()`.  Alternativley should this be private as `self._thread_identity` or only on the `BaseAsyncIOLoop`?  We should probably set `thread_identity = None` either in `__init__` or as a class variable (any preference)?
2.  In `BaseAsyncIOLoop.add_callback` check if `threading.get_ident() == self.thread_identity`, and if so use `call_soon`.  It looks like calling `threading.get_ident()` is around 100ns.

I don't have a good sense on how to test this.
Sounds good. 

The difference between IOLoop and BaseAsyncIOLoop is not very meaningful any more, but I think I'd put this on BaseAsyncIOLoop. I tend to prefer setting things in `__init__` to putting defaults in class variables (although now that I've been playing with mypy the class variable approach might become preferable, at least for python3.6+ (which is a long way off for us)).

I'm fine without a test (besides some informal benchmarking). The old optimization didn't have a test either.

Resolving patch:
diff --git a/tornado/platform/asyncio.py b/tornado/platform/asyncio.py
--- a/tornado/platform/asyncio.py
+++ b/tornado/platform/asyncio.py
@@ -22,6 +22,7 @@
 import concurrent.futures
 import functools
 
+from threading import get_ident
 from tornado.gen import convert_yielded
 from tornado.ioloop import IOLoop, _Selectable
 
@@ -60,8 +61,16 @@ def initialize(self, asyncio_loop: asyncio.AbstractEventLoop,  # type: ignore
             if loop.is_closed():
                 del IOLoop._ioloop_for_asyncio[loop]
         IOLoop._ioloop_for_asyncio[asyncio_loop] = self
+
+        self._thread_identity = 0
+
         super(BaseAsyncIOLoop, self).initialize(**kwargs)
 
+        def assign_thread_identity() -> None:
+            self._thread_identity = get_ident()
+
+        self.add_callback(assign_thread_identity)
+
     def close(self, all_fds: bool=False) -> None:
         self.closing = True
         for fd in list(self.handlers):
@@ -157,8 +166,12 @@ def remove_timeout(self, timeout: object) -> None:
         timeout.cancel()  # type: ignore
 
     def add_callback(self, callback: Callable, *args: Any, **kwargs: Any) -> None:
+        if get_ident() == self._thread_identity:
+            call_soon = self.asyncio_loop.call_soon
+        else:
+            call_soon = self.asyncio_loop.call_soon_threadsafe
         try:
-            self.asyncio_loop.call_soon_threadsafe(
+            call_soon(
                 self._run_callback,
                 functools.partial(callback, *args, **kwargs))
         except RuntimeError:
@@ -169,7 +182,13 @@ def add_callback(self, callback: Callable, *args: Any, **kwargs: Any) -> None:
             # eventually execute).
             pass
 
-    add_callback_from_signal = add_callback
+    def add_callback_from_signal(self, callback: Callable, *args: Any, **kwargs: Any) -> None:
+        try:
+            self.asyncio_loop.call_soon_threadsafe(
+                self._run_callback,
+                functools.partial(callback, *args, **kwargs))
+        except RuntimeError:
+            pass
 
     def run_in_executor(self, executor: Optional[concurrent.futures.Executor],
                         func: Callable[..., _T], *args: Any) -> Awaitable[_T]:


Test patch:
diff --git a/tornado/test/httpclient_test.py b/tornado/test/httpclient_test.py
--- a/tornado/test/httpclient_test.py
+++ b/tornado/test/httpclient_test.py
@@ -547,6 +547,7 @@ def test_str(self):
 class SyncHTTPClientTest(unittest.TestCase):
     def setUp(self):
         self.server_ioloop = IOLoop()
+        event = threading.Event()
 
         @gen.coroutine
         def init_server():
@@ -554,10 +555,15 @@ def init_server():
             app = Application([('/', HelloWorldHandler)])
             self.server = HTTPServer(app)
             self.server.add_socket(sock)
-        self.server_ioloop.run_sync(init_server)
+            event.set()
 
-        self.server_thread = threading.Thread(target=self.server_ioloop.start)
+        def start():
+            self.server_ioloop.run_sync(init_server)
+            self.server_ioloop.start()
+
+        self.server_thread = threading.Thread(target=start)
         self.server_thread.start()
+        event.wait()
 
         self.http_client = HTTPClient()