internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: jupyterhub/jupyterhub

Issue:
Running `pip install -e .` in the project root installs (no longer supported) IPython 2.3
Not sure if there's a straightforward fix for this until there's a version of IPython 3 on PyPI.  I spent a while playing with the setuptools `dependency_links` option to get this to install from github, to no avail.



Discussion:
This also causes `pip install -e git+https://github.com/jupyter/jupyterhub.git#egg=jupyterhub-dev` to install the incorrect IPython version.


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -190,8 +190,8 @@ def run(self):
     with open('requirements.txt') as f:
         for line in f.readlines():
             req = line.strip()
-            if req.startswith('-e'):
-                req = line.split('#egg=', 1)[1]
+            if not req or req.startswith(('-e', '#')):
+                continue
             install_requires.append(req)
 
 #---------------------------------------------------------------------------


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Missing zmq and jsonschema
It seems like two additional dependencies are missing.

``` bash
Traceback (most recent call last):
  File "/home/stanleygu/.virtualenvs/localpy/src/ipython/IPython/utils/zmqrelated.py", line 35, in check_for_zmq
    import zmq
ImportError: No module named 'zmq'
```

``` bash
Traceback (most recent call last):
  File "/home/stanleygu/.virtualenvs/localpy/src/ipython/IPython/nbformat/validator.py", line 10, in <module>
    from jsonschema import ValidationError
ImportError: No module named 'jsonschema'

```



Discussion:
Those are dependencies of ipython[notebook], which is a dependency of jupyterhub.


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -190,7 +190,7 @@ def run(self):
     with open('requirements.txt') as f:
         for line in f.readlines():
             req = line.strip()
-            if not req or req.startswith(('-e', '#')):
+            if not req or req.startswith('#') or '://' in req:
                 continue
             install_requires.append(req)
 


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
`cleanup` doesn't fire when process is killed by SIGTERM
This is documented behavior for the `atexit` module, but since SIGTERM is meant to be used for graceful shutdown, I'd expect the hub to exit gracefully on receipt of a TERM.  I assume fixing this is just a matter of registering a SIGTERM handler, but I'm not sure if there are complications with the tornado eventloop.



Discussion:

Resolving patch:
diff --git a/jupyterhub/app.py b/jupyterhub/app.py
--- a/jupyterhub/app.py
+++ b/jupyterhub/app.py
@@ -8,6 +8,7 @@
 import binascii
 import logging
 import os
+import signal
 import socket
 import sys
 from datetime import datetime
@@ -864,12 +865,13 @@ def cleanup(self):
         
         # clean up proxy while SUS are shutting down
         if self.cleanup_proxy:
-            if self.proxy_process and self.proxy_process.poll() is None:
+            if self.proxy_process:
                 self.log.info("Cleaning up proxy[%i]...", self.proxy_process.pid)
-                try:
-                    self.proxy_process.terminate()
-                except Exception as e:
-                    self.log.error("Failed to terminate proxy process: %s", e)
+                if self.proxy_process.poll() is None:
+                    try:
+                        self.proxy_process.terminate()
+                    except Exception as e:
+                        self.log.error("Failed to terminate proxy process: %s", e)
             else:
                 self.log.info("I didn't start the proxy, I can't clean it up")
         else:
@@ -977,10 +979,22 @@ def start(self):
         # start the webserver
         self.http_server = tornado.httpserver.HTTPServer(self.tornado_application, xheaders=True)
         self.http_server.listen(self.hub_port)
+        
+        # register cleanup on both TERM and INT
         atexit.register(self.atexit)
+        signal.signal(signal.SIGTERM, self.sigterm)
     
+    def sigterm(self, signum, frame):
+        self.log.critical("Received SIGTERM, shutting down")
+        self.io_loop.stop()
+        self.atexit()
+    
+    _atexit_ran = False
     def atexit(self):
         """atexit callback"""
+        if self._atexit_ran:
+            return
+        self._atexit_ran = True
         # run the cleanup step (in a new loop, because the interrupted one is unclean)
         IOLoop.clear_current()
         loop = IOLoop()


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
No such file or directory: 'configurable-http-proxy'
I tried out jupyterhub 0.1 today and got the following exception:

```
[I 2015-03-17 13:10:37.737 JupyterHub app:509] Loading cookie_secret from /Volumes/Raptor/jupyterhub_cookie_secret
[W 2015-03-17 13:10:37.765 JupyterHub app:245] 
    Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require restarting the proxy.
    Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config to avoid this message.

[I 2015-03-17 13:10:37.771 JupyterHub app:602] Not using whitelist. Any authenticated user will be allowed.
[I 2015-03-17 13:10:37.786 JupyterHub app:733] Starting proxy @ http://localhost:8000/
[C 2015-03-17 13:10:37.789 JupyterHub app:962] Failed to start proxy
    Traceback (most recent call last):
      File "/Volumes/Raptor/miniconda3/envs/production/lib/python3.4/site-packages/jupyterhub/app.py", line 960, in start
        yield self.start_proxy()
      File "/Volumes/Raptor/miniconda3/envs/production/lib/python3.4/site-packages/jupyterhub/app.py", line 735, in start_proxy
        self.proxy_process = Popen(cmd, env=env)
      File "/Volumes/Raptor/miniconda3/envs/production/lib/python3.4/subprocess.py", line 859, in __init__
        restore_signals, start_new_session)
      File "/Volumes/Raptor/miniconda3/envs/production/lib/python3.4/subprocess.py", line 1457, in _execute_child
        raise child_exception_type(errno_num, err_msg)
    FileNotFoundError: [Errno 2] No such file or directory: 'configurable-http-proxy'
```

Is this a known issue?



Discussion:
Oops, just saw it's a non-Python dependency hence why pip didn't install it :)


Resolving patch:
diff --git a/jupyterhub/app.py b/jupyterhub/app.py
--- a/jupyterhub/app.py
+++ b/jupyterhub/app.py
@@ -732,7 +732,15 @@ def start_proxy(self):
             cmd.extend(['--ssl-cert', self.ssl_cert])
         self.log.info("Starting proxy @ %s", self.proxy.public_server.url)
         self.log.debug("Proxy cmd: %s", cmd)
-        self.proxy_process = Popen(cmd, env=env)
+        try:
+            self.proxy_process = Popen(cmd, env=env)
+        except FileNotFoundError as e:
+            self.log.error(
+                "Failed to find proxy %r\n"
+                "The proxy can be installed with `npm install -g configurable-http-proxy`"
+                 % self.proxy_cmd
+            )
+            self.exit(1)
         def _check():
             status = self.proxy_process.poll()
             if status is not None:


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
The 'user' trait of a SingleUserNotebookApp instance must be a unicode string, but a value of 1384482<class 'int'> was specified.
I'm using the `SudoSpawner` in combination with Active Directory+PAM based authentication. This results in users being created on the Linux server with numeric IDs that map to AD users. This works fine with users whos AD logins are name-based, but users with numeric IDs cannot log in.

Looking at the logs I've found that the error occurs when trying to launch the instance of `SingleUserNotebookApp` due to the user ID being interpreted as an `int` rather than `str`. 

```
The 'user' trait of a SingleUserNotebookApp instance must be a unicode string, 
     but a value of 1384482 <class 'int'> was specified.
mfitzp@bms-jupyter:~$ /usr/bin/python3 -m jupyterhub.singleuser --user="1384482" 
     --port=36359 --cookie-name=jupyter-hub-token-1384482 --base-url=/user/1384482
     --hub-prefix=/hub/ --hub-api-url=http://localhost:8081/hub/api --ip=localhost 
```

Looking at the source the `user` property is defined as a `Unicode` traitlet so I'm unsure what is going on here? Is this perhaps a IPython issue? In `argparse` it's possible to define the type for command-line arguments to be mapped to, but I could not find the equivalent in the traitlet system.



Discussion:

Resolving patch:
diff --git a/jupyterhub/singleuser.py b/jupyterhub/singleuser.py
--- a/jupyterhub/singleuser.py
+++ b/jupyterhub/singleuser.py
@@ -15,6 +15,7 @@
 from IPython.utils.traitlets import (
     Integer,
     Unicode,
+    CUnicode,
 )
 
 from IPython.html.notebookapp import NotebookApp
@@ -114,7 +115,7 @@ def get(self):
 
 class SingleUserNotebookApp(NotebookApp):
     """A Subclass of the regular NotebookApp that is aware of the parent multiuser context."""
-    user = Unicode(config=True)
+    user = CUnicode(config=True)
     def _user_changed(self, name, old, new):
         self.log.name = new
     cookie_name = Unicode(config=True)


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Admin Panel - Last seen column includes shutdown activity
The "Last Seen" column includes shutdown activity. I'm assuming this is an artifact of the proxy itself, but is a bit misleading. After shutting down several servers from users that had not been back in days, I noticed they all came up as visiting recently.



Discussion:
What do you mean by shutdown activity? How are you shutting them down?

I clicked "stop server" for each of the users.


Resolving patch:
diff --git a/jupyterhub/orm.py b/jupyterhub/orm.py
--- a/jupyterhub/orm.py
+++ b/jupyterhub/orm.py
@@ -429,7 +429,6 @@ def stop(self):
                 yield self.spawner.stop()
             self.spawner.clear_state()
             self.state = self.spawner.get_state()
-            self.last_activity = datetime.utcnow()
             self.server = None
             inspect(self).session.commit()
         finally:


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
500 : Internal Server Error KeyError: 'user'
Hi,

I'm trying to install jupyterhub locally.
but running and connecting to the server I get a 500, I can get to the administrator pages (when sudo jupyterhub 'ing) but not add users etc.

however when I go to the root page (or users dashboard) I get a 500 with :
Uncaught exception GET /user/user/tree (127.0.0.1)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.4/dist-packages/tornado/web.py", line 1346, in _execute
        result = method(_self.path_args, *_self.path_kwargs)
      File "/usr/local/lib/python3.4/dist-packages/tornado/web.py", line 2613, in wrapper
        if not self.current_user:
      File "/usr/local/lib/python3.4/dist-packages/tornado/web.py", line 1028, in current_user
        self._current_user = self.get_current_user()
      File "/usr/local/lib/python3.4/dist-packages/IPython/html/base/handlers.py", line 72, in get_current_user
        return self.login_handler.get_user(self)
      File "/src/jupyterhub/jupyterhub/singleuser.py", line 95, in get_user
        user = auth_data['user']
    KeyError: 'user'

any ideas ?



Discussion:

Resolving patch:
diff --git a/jupyterhub/singleuser.py b/jupyterhub/singleuser.py
--- a/jupyterhub/singleuser.py
+++ b/jupyterhub/singleuser.py
@@ -92,7 +92,7 @@ def get_user(self):
             if not auth_data:
                 # treat invalid token the same as no token
                 return None
-            user = auth_data['user']
+            user = auth_data['name']
             if user == my_user:
                 self._cached_user = user
                 return user


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
base_url is used twice in redirects
For  jupyterhub running at _http://address/prefix/_ almost every redirect has `base_url` twice. It was caused by PR #276. Reverting changes to jupyterhub/handlers/pages.py:27 [1] fixes the issue.

[1] https://github.com/Crunch-io/jupyterhub/commit/05e4ab41fe6691dd697eecc27d9abee2ac4e23b6



Discussion:

Resolving patch:
diff --git a/jupyterhub/handlers/pages.py b/jupyterhub/handlers/pages.py
--- a/jupyterhub/handlers/pages.py
+++ b/jupyterhub/handlers/pages.py
@@ -24,7 +24,7 @@ def get(self):
         user = self.get_current_user()
         if user:
             if user.running:
-                url = url_path_join(self.hub.server.base_url, user.server.base_url)
+                url = user.server.base_url
                 self.log.debug("User is running: %s", url)
             else:
                 url = url_path_join(self.hub.server.base_url, 'home')


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Custom authenticator cannot set custom login handler to "/login"
In my custom authenticator, I return "/login" as the resource path for my custom login handler.

```
def get_handlers(self, app):
    return [
        (r'/login', RemoteUserLoginHandler),
    ]
```

However, it gets installed _after_ the default login handler that seems to originate in `jupyterhub/handlers/login.py`:

```
# Only logout is a default handler.
default_handlers = [
    (r"/login", LoginHandler),
    (r"/logout", LogoutHandler),
]
```

(Notice the comment doesn't quite match what is returned).
This means my custom handler is never matched.



Discussion:

Resolving patch:
diff --git a/jupyterhub/app.py b/jupyterhub/app.py
--- a/jupyterhub/app.py
+++ b/jupyterhub/app.py
@@ -464,13 +464,12 @@ def add_url_prefix(prefix, handlers):
     
     def init_handlers(self):
         h = []
-        h.extend(handlers.default_handlers)
-        h.extend(apihandlers.default_handlers)
         # load handlers from the authenticator
         h.extend(self.authenticator.get_handlers(self))
-
+        # set default handlers
+        h.extend(handlers.default_handlers)
+        h.extend(apihandlers.default_handlers)
         self.handlers = self.add_url_prefix(self.hub_prefix, h)
-
         # some extra handlers, outside hub_prefix
         self.handlers.extend([
             (r"%s" % self.hub_prefix.rstrip('/'), web.RedirectHandler,


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Allow location of home directories to be configured
When JupyterHub is run with `create_system_users` it would be nice to be able to configure the locations of the home directories (something other than `/home`). I think this can be done with the `-d /otherhome` flag to the `useradd` command.



Discussion:

Resolving patch:
diff --git a/jupyterhub/auth.py b/jupyterhub/auth.py
--- a/jupyterhub/auth.py
+++ b/jupyterhub/auth.py
@@ -123,6 +123,17 @@ class LocalAuthenticator(Authenticator):
         should I try to create the system user?
         """
     )
+    system_user_home = Unicode('', config=True,
+        help="""Specify root home directory for users if different from system default.
+        
+        Passed to `useradd -b`.
+        If specified, when system users are created their home directories will be created in
+        
+            system_user_home/username
+        
+        Only has an effect when users are created with `create_system_users=True`.
+        """
+    )
 
     group_whitelist = Set(
         config=True,
@@ -182,8 +193,7 @@ def system_user_exists(user):
         else:
             return True
     
-    @staticmethod
-    def add_system_user(user):
+    def add_system_user(self, user):
         """Create a new *ix user on the system. Works on FreeBSD and Linux, at least."""
         name = user.name
         for useradd in (
@@ -198,8 +208,11 @@ def add_system_user(user):
                 break
         else:
             raise RuntimeError("I don't know how to add users on this system.")
-    
-        check_call(useradd + [name])
+        
+        cmd = useradd
+        if self.system_user_home:
+            cmd = cmd + ['-b', self.system_user_home]
+        check_call(cmd + [name])
 
 
 class PAMAuthenticator(LocalAuthenticator):


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Spawner custom form validation
Are there ideas for allowing form validation for spawners that have a custom form?

I was thinking of raising an exception in `options_from_form()` and moving the `try` up by one line in [SpawnHandler](https://github.com/jupyter/jupyterhub/blob/master/jupyterhub/handlers/pages.py#L97).



Discussion:

Resolving patch:
diff --git a/jupyterhub/handlers/pages.py b/jupyterhub/handlers/pages.py
--- a/jupyterhub/handlers/pages.py
+++ b/jupyterhub/handlers/pages.py
@@ -93,8 +93,8 @@ def post(self):
             form_options[key] = [ bs.decode('utf8') for bs in byte_list ]
         for key, byte_list in self.request.files.items():
             form_options["%s_file"%key] = byte_list
-        options = user.spawner.options_from_form(form_options)
         try:
+            options = user.spawner.options_from_form(form_options)
             yield self.spawn_single_user(user, options=options)
         except Exception as e:
             self.log.error("Failed to spawn single-user server with form", exc_info=True)


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Proxy error on user spawn but not admin spawn.
In some cases, when our hub users click on My Server to start their server, the hub responds with an HTTP 503. When this happens, admins can click on "start server" for the user from the Admin page and "stop server" to stop. The user can then hit My Server without error.

When the user does see the 503, the routing table has an entry for them pointing at a spawned container that no longer exists. I don't know how relevant this is because it plays no part in the admin being able to start the server. The user's server spawned by the admin ends up with an updated entry in the routing table (new node/ip).

We're using [swarmspawner](https://github.com/data-8/jupyterhub-deploy/blob/master/roles/jupyterhub/files/swarmspawner.py).



Discussion:
This went away with a restart of the hub container. We haven't seen recurring issues with the 503 since.

Interesting! Hopefully it's fixed. Do you have logs saved that we might dig through for insight?

No logs from that time period unfortunately. I'll leave this open for maybe a few more weeks in case it pops up, but we haven't seen this particular issue for a while.


Resolving patch:
diff --git a/jupyterhub/orm.py b/jupyterhub/orm.py
--- a/jupyterhub/orm.py
+++ b/jupyterhub/orm.py
@@ -178,6 +178,9 @@ def add_user(self, user, client=None):
         self.log.info("Adding user %s to proxy %s => %s",
             user.name, user.proxy_path, user.server.host,
         )
+        if user.spawn_pending:
+            raise RuntimeError(
+                "User %s's spawn is pending, shouldn't be added to the proxy yet!", user.name)
         
         yield self.api_request(user.proxy_path,
             method='POST',
@@ -213,7 +216,7 @@ def add_all_users(self, user_dict):
         futures = []
         for orm_user in db.query(User):
             user = user_dict[orm_user]
-            if (user.server):
+            if user.running:
                 futures.append(self.add_user(user))
         # wait after submitting them all
         for f in futures:
@@ -230,6 +233,9 @@ def check_routes(self, user_dict, routes=None):
         db = inspect(self).session
         for orm_user in db.query(User).filter(User.server != None):
             user = user_dict[orm_user]
+            if not user.running:
+                # Don't add users to the proxy that haven't finished starting
+                continue
             if user.server is None:
                 # This should never be True, but seems to be on rare occasion.
                 # catch filter bug, either in sqlalchemy or my understanding of its behavior
diff --git a/jupyterhub/user.py b/jupyterhub/user.py
--- a/jupyterhub/user.py
+++ b/jupyterhub/user.py
@@ -145,6 +145,8 @@ def __repr__(self):
     @property
     def running(self):
         """property for whether a user has a running server"""
+        if self.spawn_pending:
+            return False # server is not running if spawn is still pending
         if self.server is None:
             return False
         return True


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
Can not add additional items to environment of Spawner
```
c.Spawner.env.update({
    'wat': 'waaaaaaaaaaaat'
})
```

Seems to have no effect. Is this because env is just a `Dict` without `config=True` set in the Spawner base class?



Discussion:
That's right, env is not configurable at the moment. We can rearrange a few things to make that work.


Resolving patch:
diff --git a/jupyterhub/spawner.py b/jupyterhub/spawner.py
--- a/jupyterhub/spawner.py
+++ b/jupyterhub/spawner.py
@@ -116,14 +116,15 @@ def options_from_form(self, form_data):
     ], config=True,
         help="Whitelist of environment variables for the subprocess to inherit"
     )
-    env = Dict()
-    def _env_default(self):
-        env = {}
-        for key in self.env_keep:
-            if key in os.environ:
-                env[key] = os.environ[key]
-        env['JPY_API_TOKEN'] = self.api_token
-        return env
+    env = Dict(help="""Deprecated: use Spawner.get_env or Spawner.environment
+    
+    - extend Spawner.get_env for adding required env in Spawner subclasses
+    - Spawner.environment for config-specified env
+    """)
+    
+    environment = Dict(config=True,
+        help="Environment variables to load for the Spawner."
+    )
     
     cmd = Command(['jupyterhub-singleuser'], config=True,
         help="""The command used for starting notebooks."""
@@ -204,12 +205,27 @@ def clear_state(self):
         self.api_token = ''
     
     def get_env(self):
-        """Return the environment we should use
-        
-        Default returns a copy of self.env.
+        """Return the environment dict to use for the Spawner.
+
+        This applies things like `env_keep`, anything defined in `Spawner.environment`,
+        and adds the API token to the env.
+
         Use this to access the env in Spawner.start to allow extension in subclasses.
         """
-        return self.env.copy()
+        env = {}
+        if self.env:
+            warnings.warn("Spawner.env is deprecated, found %s" % self.env, DeprecationWarning)
+            env.update(self.env)
+        
+        for key in self.env_keep:
+            if key in os.environ:
+                env[key] = os.environ[key]
+        
+        # config overrides
+        env.update(self.environment)
+        
+        env['JPY_API_TOKEN'] = self.api_token
+        return env
     
     def get_args(self):
         """Return the arguments to be passed after self.cmd"""


Test patch:
Repository: jupyterhub/jupyterhub

Issue:
`hub_prefix` option does not work
Set `JupyterHub.hub_prefix` option, and error occurred.

```
TypeError: _hub_prefix_changed() missing 2 required positional arguments: 'old' and 'new'
```

Update `_hub_prefix_changed()`.

``` diff
--- old/jupyterhub/app.py▸     2016-05-31 18:01:49.633124540 +0900
+++ new/jupyterhub/app.py▸     2016-05-31 18:01:49.661125012 +0900
@@ -332,7 +332,8 @@
         return url_path_join(self.base_url, '/hub/')

     @observe('hub_prefix')
-    def _hub_prefix_changed(self, name, old, new):
+    def _hub_prefix_changed(self, change):
+        new = change['new']
         if new == '/':
             raise TraitError("'/' is not a valid hub prefix")
         if not new.startswith(self.base_url):
```



Discussion:

Resolving patch:
diff --git a/jupyterhub/app.py b/jupyterhub/app.py
--- a/jupyterhub/app.py
+++ b/jupyterhub/app.py
@@ -354,7 +354,8 @@ def _hub_prefix_default(self):
         return url_path_join(self.base_url, '/hub/')
 
     @observe('hub_prefix')
-    def _hub_prefix_changed(self, name, old, new):
+    def _hub_prefix_changed(self, change):
+        new = change['new']
         if new == '/':
             raise TraitError("'/' is not a valid hub prefix")
         if not new.startswith(self.base_url):


Test patch: