Repository: jupyterhub/jupyterhub

Issue:
Spawner poll return value during spawn
This is a question about how `start_timeout` and `slow_spawn_timeout` work together.

In `BaseHandler.spawn_single_user()` `user.spawn()` is called which should complete with in `start_timeout` I think. On line 296 the `slow_spawn_timeout` is used, which is much shorter. If we reach the timeout then `user.spawner.poll()` is called and if it is not `None` an exception is raised.

My expectation would be that `user.spawner.poll()` returns something that isn't `None` while still spawning, as it isn't yet running. However this means you effectively only have `slow_spawn_timeout` seconds to be done spawning.

Should `poll()` already return `None` while still spawning?



Discussion:
Since `Spawner.poll()` took its inspiration from `Popen.poll()`, the interpretation of its return value is:
- None: has not exited
- integer: exit code

So `poll` really means "Has it exited?" and None means "No, it has not exited." even if the reason is that it has not actually finished starting, since it doesn't make a difference _why_ it hasn't exited.

As it is interpreted now, poll should return None from the moment `Spawner.start` has been called to the time that the server has stopped.

The slow spawn timeout is used to show an HTML page to the user before HTTP timeouts / human patience would give up, but the spawner is still allowed to continue. The reason poll is called after the slow timeout is a check for "is the reason it's taking so long to start really that it died already?" The 'true' spawn timeout is used to actually give up and destroy the spawner so you can try again later.


Resolving patch:
diff --git a/jupyterhub/handlers/base.py b/jupyterhub/handlers/base.py
--- a/jupyterhub/handlers/base.py
+++ b/jupyterhub/handlers/base.py
@@ -296,16 +296,23 @@ def finish_user_spawn(f=None):
             yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
         except gen.TimeoutError:
             if user.spawn_pending:
+                # still in Spawner.start, which is taking a long time
+                # we shouldn't poll while spawn_pending is True
+                self.log.warn("User %s server is slow to start", user.name)
+                # schedule finish for when the user finishes spawning
+                IOLoop.current().add_future(f, finish_user_spawn)
+            else:
+                # start has finished, but the server hasn't come up
+                # check if the server died while we were waiting
                 status = yield user.spawner.poll()
                 if status is None:
-                    # hit timeout, but spawn is still pending
-                    self.log.warn("User %s server is slow to start", user.name)
+                    # hit timeout, but server's running. Hope that it'll show up soon enough,
+                    # though it's possible that it started at the wrong URL
+                    self.log.warn("User %s server is slow to become responsive", user.name)
                     # schedule finish for when the user finishes spawning
                     IOLoop.current().add_future(f, finish_user_spawn)
                 else:
                     raise web.HTTPError(500, "Spawner failed to start [status=%s]" % status)
-            else:
-                raise
         else:
             yield finish_user_spawn()
     
diff --git a/jupyterhub/user.py b/jupyterhub/user.py
--- a/jupyterhub/user.py
+++ b/jupyterhub/user.py
@@ -206,6 +206,7 @@ def spawn(self, options=None):
         self.state = spawner.get_state()
         self.last_activity = datetime.utcnow()
         db.commit()
+        self.spawn_pending = False
         try:
             yield self.server.wait_up(http=True, timeout=spawner.http_timeout)
         except Exception as e:
@@ -232,7 +233,6 @@ def spawn(self, options=None):
                 ), exc_info=True)
             # raise original TimeoutError
             raise e
-        self.spawn_pending = False
         return self
 
     @gen.coroutine


Test patch: