Repository: tornadoweb/tornado

Issue:
wsgi adapter throwing runtime error
The 5.0.2 wsgi adapter seems to be throwing a runtime error:
[Wed May 02 19:43:09.367974 2018] [:error] [pid 31085] [remote 172.31.12.172:164] mod_wsgi (pid=31085): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Wed May 02 19:43:09.368100 2018] [:error] [pid 31085] [remote 172.31.12.172:164] Traceback (most recent call last):
[Wed May 02 19:43:09.368138 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/opt/python/run/venv/local/lib64/python3.6/site-packages/tornado/wsgi.py", line 222, in __call__
[Wed May 02 19:43:09.368147 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     self.application(request)
[Wed May 02 19:43:09.368154 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/opt/python/run/venv/local/lib64/python3.6/site-packages/tornado/web.py", line 2041, in __call__
[Wed May 02 19:43:09.368157 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     return dispatcher.execute()
[Wed May 02 19:43:09.368162 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/opt/python/run/venv/local/lib64/python3.6/site-packages/tornado/web.py", line 2172, in execute
[Wed May 02 19:43:09.368165 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     **self.path_kwargs)
[Wed May 02 19:43:09.368180 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/opt/python/run/venv/local/lib64/python3.6/site-packages/tornado/gen.py", line 288, in wrapper
[Wed May 02 19:43:09.368183 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     future = _create_future()
[Wed May 02 19:43:09.368188 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/opt/python/run/venv/local/lib64/python3.6/site-packages/tornado/gen.py", line 186, in _create_future
[Wed May 02 19:43:09.368190 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     future = Future()
[Wed May 02 19:43:09.368195 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/usr/lib64/python3.6/asyncio/events.py", line 678, in get_event_loop
[Wed May 02 19:43:09.368198 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     return get_event_loop_policy().get_event_loop()
[Wed May 02 19:43:09.368202 2018] [:error] [pid 31085] [remote 172.31.12.172:164]   File "/usr/lib64/python3.6/asyncio/events.py", line 584, in get_event_loop
[Wed May 02 19:43:09.368205 2018] [:error] [pid 31085] [remote 172.31.12.172:164]     % threading.current_thread().name)
[Wed May 02 19:43:09.368219 2018] [:error] [pid 31085] [remote 172.31.12.172:164] RuntimeError: There is no current event loop in thread 'Dummy-1'.

I switched to 4.5.3 and don't get this error. Looks like something in 5.0.2 is causing this issue.


Discussion:
threads ... https://github.com/tornadoweb/tornado/issues/2183
This is a little different from the usual threading problem as in #2183. In `WSGIAdapter`, asynchronous features aren't supported, and we shouldn't be trying to access the IOLoop at all. This is a side effect of the fact that Future creation accesses the IOLoop in 5.0. `RequestHandler._execute` is decorated with `@gen.coroutine` so it creates a `Future`, even though with `WSGIContainer` it will always be synchronous in practice. So it looks like `WSGIAdapter` is just broken in 5.0 for any multi-threaded WSGI container (maybe only on Python 3).

I don't immediately see a clean solution to this. We need some way to bypass the coroutine decorator when the application is being called from WSGIAdapter. You might be able to work around this by configuring mod_wsgi in a one-thread-per-process mode and running multiple processes instead of multiple threads.

This is only going to get worse in Tornado 6.0. I wonder if it's time to deprecate WSGIAdapter and WSGIApplication and only support async usage. May I ask why you're using mod_wsgi and WSGIAdapter instead of Tornado's native HTTP server (perhaps behind mod_proxy if you're using apache for other things)?
The main reason I'm using this method is because this is the easiest way to get it to work with Amazon Elastic Beanstalk. 
OK. I think I have a workaround. You should be able to do this at startup (probably import time):

    import asyncio
    from tornado.platform.asyncio import AnyThreadEventLoopPolicy
    asyncio.set_event_loop_policy(AnyThreadEventLoopPolicy())

I think it would be possible but tricky to avoid this workaround in 5.0, and it may become impossible with the move to use native coroutines in 6.0. So I think we're probably going to need to deprecate WSGIApplication/WSGIContainer and remove them in 6.0 to focus on asynchronous operation. 

Resolving patch:
diff --git a/tornado/wsgi.py b/tornado/wsgi.py
--- a/tornado/wsgi.py
+++ b/tornado/wsgi.py
@@ -33,6 +33,7 @@
 import sys
 from io import BytesIO
 import tornado
+import warnings
 
 from tornado.concurrent import Future
 from tornado import escape
@@ -76,6 +77,7 @@ class WSGIApplication(web.Application):
     .. deprecated:: 4.0
 
        Use a regular `.Application` and wrap it in `WSGIAdapter` instead.
+       This class will be removed in Tornado 6.0.
     """
     def __call__(self, environ, start_response):
         return WSGIAdapter(self)(environ, start_response)
@@ -179,9 +181,25 @@ def get(self):
     that it is not possible to use `.AsyncHTTPClient`, or the
     `tornado.auth` or `tornado.websocket` modules.
 
+    In multithreaded WSGI servers on Python 3, it may be necessary to
+    permit `asyncio` to create event loops on any thread. Run the
+    following at startup (typically import time for WSGI
+    applications)::
+
+        import asyncio
+        from tornado.platform.asyncio import AnyThreadEventLoopPolicy
+        asyncio.set_event_loop_policy(AnyThreadEventLoopPolicy())
+
     .. versionadded:: 4.0
+
+    .. deprecated:: 5.1
+
+       This class is deprecated and will be removed in Tornado 6.0.
+       Use Tornado's `.HTTPServer` instead of a WSGI container.
     """
     def __init__(self, application):
+        warnings.warn("WSGIAdapter is deprecated, use Tornado's HTTPServer instead",
+                      DeprecationWarning)
         if isinstance(application, WSGIApplication):
             self.application = lambda request: web.Application.__call__(
                 application, request)


Test patch:
diff --git a/tornado/test/wsgi_test.py b/tornado/test/wsgi_test.py
--- a/tornado/test/wsgi_test.py
+++ b/tornado/test/wsgi_test.py
@@ -3,6 +3,7 @@
 
 from tornado.escape import json_decode
 from tornado.test.httpserver_test import TypeCheckHandler
+from tornado.test.util import ignore_deprecation
 from tornado.testing import AsyncHTTPTestCase
 from tornado.web import RequestHandler, Application
 from tornado.wsgi import WSGIApplication, WSGIContainer, WSGIAdapter
@@ -26,7 +27,7 @@ def test_simple(self):
         self.assertEqual(response.body, b"Hello world!")
 
 
-class WSGIApplicationTest(AsyncHTTPTestCase):
+class WSGIAdapterTest(AsyncHTTPTestCase):
     def get_app(self):
         class HelloHandler(RequestHandler):
             def get(self):
@@ -40,11 +41,13 @@ def get(self, path):
         # another thread instead of using our own WSGIContainer, but this
         # fits better in our async testing framework and the wsgiref
         # validator should keep us honest
-        return WSGIContainer(validator(WSGIApplication([
-            ("/", HelloHandler),
-            ("/path/(.*)", PathQuotingHandler),
-            ("/typecheck", TypeCheckHandler),
-        ])))
+        with ignore_deprecation():
+            return WSGIContainer(validator(WSGIAdapter(
+                Application([
+                    ("/", HelloHandler),
+                    ("/path/(.*)", PathQuotingHandler),
+                    ("/typecheck", TypeCheckHandler),
+                ]))))
 
     def test_simple(self):
         response = self.fetch("/")
@@ -70,18 +73,29 @@ def test_types(self):
 # survives repeated disassembly and reassembly.
 class WSGIConnectionTest(httpserver_test.HTTPConnectionTest):
     def get_app(self):
-        return WSGIContainer(validator(WSGIApplication(self.get_handlers())))
+        with ignore_deprecation():
+            return WSGIContainer(validator(WSGIAdapter(Application(self.get_handlers()))))
 
 
 def wrap_web_tests_application():
     result = {}
     for cls in web_test.wsgi_safe_tests:
-        class WSGIApplicationWrappedTest(cls):  # type: ignore
-            def get_app(self):
-                self.app = WSGIApplication(self.get_handlers(),
-                                           **self.get_app_kwargs())
-                return WSGIContainer(validator(self.app))
-        result["WSGIApplication_" + cls.__name__] = WSGIApplicationWrappedTest
+        def class_factory():
+            class WSGIApplicationWrappedTest(cls):  # type: ignore
+                def setUp(self):
+                    self.warning_catcher = ignore_deprecation()
+                    self.warning_catcher.__enter__()
+                    super(WSGIApplicationWrappedTest, self).setUp()
+
+                def tearDown(self):
+                    super(WSGIApplicationWrappedTest, self).tearDown()
+                    self.warning_catcher.__exit__(None, None, None)
+
+                def get_app(self):
+                    self.app = WSGIApplication(self.get_handlers(),
+                                               **self.get_app_kwargs())
+                    return WSGIContainer(validator(self.app))
+        result["WSGIApplication_" + cls.__name__] = class_factory()
     return result
 
 
@@ -95,7 +109,8 @@ class WSGIAdapterWrappedTest(cls):  # type: ignore
             def get_app(self):
                 self.app = Application(self.get_handlers(),
                                        **self.get_app_kwargs())
-                return WSGIContainer(validator(WSGIAdapter(self.app)))
+                with ignore_deprecation():
+                    return WSGIContainer(validator(WSGIAdapter(self.app)))
         result["WSGIAdapter_" + cls.__name__] = WSGIAdapterWrappedTest
     return result