Repository: jupyterhub/jupyterhub

Issue:
[rbac] separate role creation from assignment
Right now, `init_roles` does two things:

1. define the roles, and
2. assign users, services, groups, tokens to those roles

The second part has to run after users and services exist, while the first part has to run before e.g. services with `admin: true` are created.

I just encountered this because jupyterhub with an admin service will fail to start because the admin role doesn't exist yet because `init_roles()` is called after `init_services()`.

I think we should rearrange this a bit to:

1. create roles before init_users or other entities
2. have a separate `init_role_assignments` stage that assigns entities to roles

And also probably *remove* support for assigning roles directly to tokens. Looking at it, I think that's maybe more complicated and niche than we need. Let's keep role assignment from config only to the other encapsulating entities.

Question: **Do we still have a single `load_roles` config with both role definitions and assignments, or two separate `load_roles` and `assign_roles` config inputs for the two stages?**

We should also make it *very* clear what happens between two jupyterhub starts with:

1. launch with load_roles with `{name: "myrole", users: ['alice', 'bob']}`
2. launch with load_roles with `{name: "myrole", users: ['alice'] }`

Does bob still have role `myrole` or not? If he does, how should I update bob's role assignment to remove `myrole`?


Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

> I think we should rearrange this a bit to:
> 
> 1. create roles before init_users or other entities
> 2. have a separate `init_role_assignments` stage that assigns entities to roles
> 
> And also probably _remove_ support for assigning roles directly to tokens. Looking at it, I think that's maybe more complicated and niche than we need. Let's keep role assignment from config only to the other encapsulating entities.

Agree, I'll look into this.

> Question: **Do we still have a single `load_roles` config with both role definitions and assignments, or two separate `load_roles` and `assign_roles` config inputs for the two stages?**

I am inclined to the latter, having separate definitions and assignments will be clearer.

> We should also make it _very_ clear what happens between two jupyterhub starts with:
> 
> 1. launch with load_roles with `{name: "myrole", users: ['alice', 'bob']}`
> 2. launch with load_roles with `{name: "myrole", users: ['alice'] }`
> 
> Does bob still have role `myrole` or not? If he does, how should I update bob's role assignment to remove `myrole`?

Right now bob will still have `myrole` (docs do mention this as well). We thought that removing roles will be left to APIs once we implement them. But I can see that we should indeed provide a way to do this now as well. If we split the definitions and assignments, it will be easier to track who has the role so removing a user/service/group from assignment part should result in them losing the role. @0mar what do you think?
I agree, having the hub initialize roles before initializing the users and services leads to some awkward constructions in the code base. Splitting this would resolve it. 

As far as the config assignments are concerned, I don't really know what is the best solution. 
I would propose to make sure that each time the hub is restarted, it reinitializes all roles for the users/services mentioned in the configuration file (perhaps with a warning when overwriting roles).
This way, there is no need to define any delete/remove methods in the configuration, since I think that could lead to inconsistent situations (what if a role is assigned and stripped in the same configuration?)
A downside is that any assignments and removals are reset when the hub is reset, which might not be ideal. So perhaps, a compromise would be to reinitialize all roles only for the users/services that do not exist in the database yet.

What do you think?

Resolving patch:
diff --git a/jupyterhub/app.py b/jupyterhub/app.py
--- a/jupyterhub/app.py
+++ b/jupyterhub/app.py
@@ -21,6 +21,7 @@
 from functools import partial
 from getpass import getuser
 from glob import glob
+from itertools import chain
 from operator import itemgetter
 from textwrap import dedent
 from urllib.parse import unquote
@@ -329,11 +330,10 @@ def _load_classes(self):
             load_roles = [
                             {
                                 'name': 'teacher',
-                                'description': 'Access users information, servers and groups without create/delete privileges',
+                                'description': 'Access to users' information and group membership',
                                 'scopes': ['users', 'groups'],
                                 'users': ['cyclops', 'gandalf'],
                                 'services': [],
-                                'tokens': [],
                                 'groups': []
                             }
                         ]
@@ -1975,25 +1975,70 @@ async def init_groups(self):
                 group.users.append(user)
         db.commit()
 
-    async def init_roles(self):
+    async def init_role_creation(self):
         """Load default and predefined roles into the database"""
-        db = self.db
-        # tokens are added separately
-        role_bearers = ['users', 'services', 'groups']
-
-        # load default roles
         self.log.debug('Loading default roles to database')
         default_roles = roles.get_default_roles()
-        for role in default_roles:
-            roles.create_role(db, role)
+        config_role_names = [r['name'] for r in self.load_roles]
 
+        init_roles = default_roles
+        for role_name in config_role_names:
+            if config_role_names.count(role_name) > 1:
+                raise ValueError(
+                    f"Role {role_name} multiply defined. Please check the `load_roles` configuration"
+                )
+        for role_spec in self.load_roles:
+            init_roles.append(role_spec)
+        init_role_names = [r['name'] for r in init_roles]
+        if not orm.Role.find(self.db, name='admin'):
+            self._rbac_upgrade = True
+        else:
+            self._rbac_upgrade = False
+        for role in self.db.query(orm.Role).filter(
+            orm.Role.name.notin_(init_role_names)
+        ):
+            app_log.info(f"Deleting role {role.name}")
+            self.db.delete(role)
+        self.db.commit()
+        for role in init_roles:
+            roles.create_role(self.db, role)
+
+    async def init_role_assignment(self):
+        # tokens are added separately
+        role_bearers = ['users', 'services', 'groups']
+        admin_role_objects = ['users', 'services']
+        config_admin_users = set(self.authenticator.admin_users)
+        db = self.db
         # load predefined roles from config file
+        if config_admin_users:
+            for role_spec in self.load_roles:
+                if role_spec['name'] == 'admin':
+                    app_log.warning(
+                        "Configuration specifies both admin_users and users in the admin role specification. "
+                        "If admin role is present in config, c.authenticator.admin_users should not be used."
+                    )
+                    app_log.info(
+                        "Merging admin_users set with users list in admin role"
+                    )
+                    role_spec['users'] = set(role_spec.get('users', []))
+                    role_spec['users'] |= config_admin_users
         self.log.debug('Loading predefined roles from config file to database')
+        has_admin_role_spec = {role_bearer: False for role_bearer in admin_role_objects}
         for predef_role in self.load_roles:
-            roles.create_role(db, predef_role)
+            predef_role_obj = orm.Role.find(db, name=predef_role['name'])
+            if predef_role['name'] == 'admin':
+                for bearer in admin_role_objects:
+                    has_admin_role_spec[bearer] = bearer in predef_role
+                    if has_admin_role_spec[bearer]:
+                        app_log.info(f"Admin role specifies static {bearer} list")
+                    else:
+                        app_log.info(
+                            f"Admin role does not specify {bearer}, preserving admin membership in database"
+                        )
             # add users, services, and/or groups,
             # tokens need to be checked for permissions
             for bearer in role_bearers:
+                orm_role_bearers = []
                 if bearer in predef_role.keys():
                     for bname in predef_role[bearer]:
                         if bearer == 'users':
@@ -2009,22 +2054,40 @@ async def init_roles(self):
                                 )
                         Class = orm.get_class(bearer)
                         orm_obj = Class.find(db, bname)
-                        roles.grant_role(
-                            db, entity=orm_obj, rolename=predef_role['name']
-                        )
-        # make sure that on no admin situation, all roles are reset
-        admin_role = orm.Role.find(db, name='admin')
-        if not admin_role.users:
+                        if orm_obj:
+                            orm_role_bearers.append(orm_obj)
+                        else:
+                            app_log.warning(
+                                f"User {bname} not added, only found in role definition"
+                            )
+                            # Todo: Add users to allowed_users
+                        # Ensure all with admin role have admin flag
+                        if predef_role['name'] == 'admin':
+                            orm_obj.admin = True
+                setattr(predef_role_obj, bearer, orm_role_bearers)
+        db.commit()
+        allowed_users = db.query(orm.User).filter(
+            orm.User.name.in_(self.authenticator.allowed_users)
+        )
+        for user in allowed_users:
+            roles.grant_role(db, user, 'user')
+        admin_role = orm.Role.find(db, 'admin')
+        for bearer in admin_role_objects:
+            Class = orm.get_class(bearer)
+            for admin_obj in db.query(Class).filter_by(admin=True):
+                if has_admin_role_spec[bearer]:
+                    admin_obj.admin = admin_role in admin_obj.roles
+                else:
+                    roles.grant_role(db, admin_obj, 'admin')
+        db.commit()
+        # make sure that on hub upgrade, all users, services and tokens have at least one role (update with default)
+        if getattr(self, '_rbac_upgrade', False):
             app_log.warning(
-                "No admin users found; assuming hub upgrade. Initializing default roles for all entities"
+                "No admin role found; assuming hub upgrade. Initializing default roles for all entities"
             )
-            # make sure all users, services and tokens have at least one role (update with default)
             for bearer in role_bearers:
                 roles.check_for_default_roles(db, bearer)
 
-            # now add roles to tokens if their owner's permissions allow
-            roles.add_predef_roles_tokens(db, self.load_roles)
-
             # check tokens for default roles
             roles.check_for_default_roles(db, bearer='tokens')
 
@@ -2067,13 +2130,6 @@ async def _add_tokens(self, token_dict, kind):
                     db.add(obj)
                     db.commit()
                 self.log.info("Adding API token for %s: %s", kind, name)
-                # If we have roles in the configuration file, they will be added later
-                # Todo: works but ugly
-                config_roles = None
-                for config_role in self.load_roles:
-                    if 'tokens' in config_role and token in config_role['tokens']:
-                        config_roles = []
-                        break
                 try:
                     # set generated=False to ensure that user-provided tokens
                     # get extra hashing (don't trust entropy of user-provided tokens)
@@ -2081,7 +2137,6 @@ async def _add_tokens(self, token_dict, kind):
                         token,
                         note="from config",
                         generated=self.trust_user_provided_tokens,
-                        roles=config_roles,
                     )
                 except Exception:
                     if created:
@@ -2140,6 +2195,12 @@ def init_services(self):
                 # not found, create a new one
                 orm_service = orm.Service(name=name)
                 if spec.get('admin', False):
+                    self.log.warning(
+                        f"Service {name} sets `admin: True`, which is deprecated in JupyterHub 2.0."
+                        " You can assign now assign roles via `JupyterHub.load_roles` configuration."
+                        " If you specify services in the admin role configuration, "
+                        "the Service admin flag will be ignored."
+                    )
                     roles.update_roles(self.db, entity=orm_service, roles=['admin'])
                 self.db.add(orm_service)
             orm_service.admin = spec.get('admin', False)
@@ -2623,11 +2684,12 @@ def _log_cls(name, cls):
         self.init_hub()
         self.init_proxy()
         self.init_oauth()
+        await self.init_role_creation()
         await self.init_users()
         await self.init_groups()
         self.init_services()
         await self.init_api_tokens()
-        await self.init_roles()
+        await self.init_role_assignment()
         self.init_tornado_settings()
         self.init_handlers()
         self.init_tornado_application()
diff --git a/jupyterhub/auth.py b/jupyterhub/auth.py
--- a/jupyterhub/auth.py
+++ b/jupyterhub/auth.py
@@ -112,7 +112,7 @@ class Authenticator(LoggingConfigurable):
 
         Use this with supported authenticators to restrict which users can log in. This is an
         additional list that further restricts users, beyond whatever restrictions the
-        authenticator has in place.
+        authenticator has in place. Any user in this list is granted the 'user' role on hub startup.
 
         If empty, does not perform any additional restriction.
 
diff --git a/jupyterhub/roles.py b/jupyterhub/roles.py
--- a/jupyterhub/roles.py
+++ b/jupyterhub/roles.py
@@ -193,10 +193,14 @@ def _overwrite_role(role, role_dict):
 
     for attr in role_dict.keys():
         if attr == 'description' or attr == 'scopes':
-            if role.name == 'admin' and role_dict[attr] != getattr(role, attr):
-                raise ValueError(
-                    'admin role description or scopes cannot be overwritten'
-                )
+            if role.name == 'admin':
+                admin_role_spec = [
+                    r for r in get_default_roles() if r['name'] == 'admin'
+                ][0]
+                if role_dict[attr] != admin_role_spec[attr]:
+                    raise ValueError(
+                        'admin role description or scopes cannot be overwritten'
+                    )
             else:
                 if role_dict[attr] != getattr(role, attr):
                     setattr(role, attr, role_dict[attr])
@@ -399,7 +403,6 @@ def assign_default_roles(db, entity):
         db.commit()
     # users and services can have 'user' or 'admin' roles as default
     else:
-        # todo: when we deprecate admin flag: replace with role check
         app_log.debug('Assigning default roles to %s', type(entity).__name__)
         _switch_default_role(db, entity, entity.admin)
 
@@ -428,25 +431,6 @@ def update_roles(db, entity, roles):
             grant_role(db, entity=entity, rolename=rolename)
 
 
-def add_predef_roles_tokens(db, predef_roles):
-
-    """Adds tokens to predefined roles in config file
-    if their permissions allow"""
-
-    for predef_role in predef_roles:
-        if 'tokens' in predef_role.keys():
-            token_role = orm.Role.find(db, name=predef_role['name'])
-            for token_name in predef_role['tokens']:
-                token = orm.APIToken.find(db, token_name)
-                if token is None:
-                    raise ValueError(
-                        "Token %r does not exist and cannot assign it to role %r"
-                        % (token_name, token_role.name)
-                    )
-                else:
-                    update_roles(db, token, roles=[token_role.name])
-
-
 def check_for_default_roles(db, bearer):
 
     """Checks that role bearers have at least one role (default if none).


Test patch:
diff --git a/jupyterhub/tests/test_roles.py b/jupyterhub/tests/test_roles.py
--- a/jupyterhub/tests/test_roles.py
+++ b/jupyterhub/tests/test_roles.py
@@ -2,11 +2,13 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
 import json
+import os
 from itertools import chain
 
 import pytest
 from pytest import mark
 from tornado.log import app_log
+from traitlets.config import Config
 
 from .. import orm
 from .. import roles
@@ -247,7 +249,7 @@ async def test_load_default_roles(tmpdir, request):
     hub = MockHub(**kwargs)
     hub.init_db()
     db = hub.db
-    await hub.init_roles()
+    await hub.init_role_creation()
     # test default roles loaded to database
     default_roles = roles.get_default_roles()
     for role in default_roles:
@@ -444,9 +446,9 @@ async def test_load_roles_users(tmpdir, request):
     db = hub.db
     hub.authenticator.admin_users = ['admin']
     hub.authenticator.allowed_users = ['cyclops', 'gandalf', 'bilbo', 'gargamel']
+    await hub.init_role_creation()
     await hub.init_users()
-    await hub.init_roles()
-
+    await hub.init_role_assignment()
     admin_role = orm.Role.find(db, 'admin')
     user_role = orm.Role.find(db, 'user')
     # test if every user has a role (and no duplicates)
@@ -456,7 +458,7 @@ async def test_load_roles_users(tmpdir, request):
         assert len(user.roles) == len(set(user.roles))
         if user.admin:
             assert admin_role in user.roles
-            assert user_role not in user.roles
+            assert user_role in user.roles
 
     # test if predefined roles loaded and assigned
     teacher_role = orm.Role.find(db, name='teacher')
@@ -507,13 +509,13 @@ async def test_load_roles_services(tmpdir, request):
     hub = MockHub(**kwargs)
     hub.init_db()
     db = hub.db
+    await hub.init_role_creation()
     await hub.init_api_tokens()
     # make 'admin_service' admin
     admin_service = orm.Service.find(db, 'admin_service')
     admin_service.admin = True
     db.commit()
-    await hub.init_roles()
-
+    await hub.init_role_assignment()
     # test if every service has a role (and no duplicates)
     admin_role = orm.Role.find(db, name='admin')
     user_role = orm.Role.find(db, name='user')
@@ -580,8 +582,9 @@ async def test_load_roles_groups(tmpdir, request):
     hub = MockHub(**kwargs)
     hub.init_db()
     db = hub.db
+    await hub.init_role_creation()
     await hub.init_groups()
-    await hub.init_roles()
+    await hub.init_role_assignment()
 
     assist_role = orm.Role.find(db, name='assistant')
     head_role = orm.Role.find(db, name='head')
@@ -612,7 +615,6 @@ async def test_load_roles_user_tokens(tmpdir, request):
             'name': 'reader',
             'description': 'Read all users models',
             'scopes': ['read:users'],
-            'tokens': ['super-secret-token'],
         },
     ]
     kwargs = {
@@ -627,15 +629,10 @@ async def test_load_roles_user_tokens(tmpdir, request):
     db = hub.db
     hub.authenticator.admin_users = ['admin']
     hub.authenticator.allowed_users = ['cyclops', 'gandalf']
+    await hub.init_role_creation()
     await hub.init_users()
     await hub.init_api_tokens()
-    await hub.init_roles()
-
-    # test if gandalf's token has the 'reader' role
-    reader_role = orm.Role.find(db, 'reader')
-    token = orm.APIToken.find(db, 'super-secret-token')
-    assert reader_role in token.roles
-
+    await hub.init_role_assignment()
     # test if all other tokens have default 'user' role
     token_role = orm.Role.find(db, 'token')
     secret_token = orm.APIToken.find(db, 'secret-token')
@@ -653,163 +650,6 @@ async def test_load_roles_user_tokens(tmpdir, request):
         roles.delete_role(db, role['name'])
 
 
-@mark.role
-async def test_load_roles_user_tokens_not_allowed(tmpdir, request):
-    user_tokens = {
-        'secret-token': 'bilbo',
-    }
-    roles_to_load = [
-        {
-            'name': 'user-creator',
-            'description': 'Creates/deletes any user',
-            'scopes': ['admin:users'],
-            'tokens': ['secret-token'],
-        },
-    ]
-    kwargs = {
-        'load_roles': roles_to_load,
-        'api_tokens': user_tokens,
-    }
-    ssl_enabled = getattr(request.module, "ssl_enabled", False)
-    if ssl_enabled:
-        kwargs['internal_certs_location'] = str(tmpdir)
-    hub = MockHub(**kwargs)
-    hub.init_db()
-    db = hub.db
-    hub.authenticator.allowed_users = ['bilbo']
-    await hub.init_users()
-    await hub.init_api_tokens()
-
-    response = 'allowed'
-    # bilbo has only default 'user' role
-    # while bilbo's token is requesting role with higher permissions
-    with pytest.raises(ValueError):
-        await hub.init_roles()
-
-    # delete the test tokens
-    for token in db.query(orm.APIToken):
-        db.delete(token)
-    db.commit()
-
-    # delete the test roles
-    for role in roles_to_load:
-        roles.delete_role(db, role['name'])
-
-
-@mark.role
-async def test_load_roles_service_tokens(tmpdir, request):
-    services = [
-        {'name': 'idle-culler', 'api_token': 'another-secret-token'},
-    ]
-    service_tokens = {
-        'another-secret-token': 'idle-culler',
-    }
-    roles_to_load = [
-        {
-            'name': 'idle-culler',
-            'description': 'Cull idle servers',
-            'scopes': [
-                'read:users:name',
-                'read:users:activity',
-                'read:users:servers',
-                'users:servers',
-            ],
-            'services': ['idle-culler'],
-            'tokens': ['another-secret-token'],
-        },
-    ]
-    kwargs = {
-        'load_roles': roles_to_load,
-        'services': services,
-        'service_tokens': service_tokens,
-    }
-    ssl_enabled = getattr(request.module, "ssl_enabled", False)
-    if ssl_enabled:
-        kwargs['internal_certs_location'] = str(tmpdir)
-    hub = MockHub(**kwargs)
-    hub.init_db()
-    db = hub.db
-    await hub.init_api_tokens()
-    await hub.init_roles()
-
-    # test if another-secret-token has idle-culler role
-    service = orm.Service.find(db, 'idle-culler')
-    culler_role = orm.Role.find(db, 'idle-culler')
-    token = orm.APIToken.find(db, 'another-secret-token')
-    assert len(token.roles) == 1
-    assert culler_role in token.roles
-
-    # delete the test services
-    for service in db.query(orm.Service):
-        db.delete(service)
-    db.commit()
-
-    # delete the test tokens
-    for token in db.query(orm.APIToken):
-        db.delete(token)
-    db.commit()
-
-    # delete the test roles
-    for role in roles_to_load:
-        roles.delete_role(db, role['name'])
-
-
-@mark.role
-async def test_load_roles_service_tokens_not_allowed(tmpdir, request):
-    services = [{'name': 'some-service', 'api_token': 'secret-token'}]
-    service_tokens = {
-        'secret-token': 'some-service',
-    }
-    roles_to_load = [
-        {
-            'name': 'user-reader',
-            'description': 'Read-only user models',
-            'scopes': ['read:users'],
-            'services': ['some-service'],
-        },
-        # 'idle-culler' role has higher permissions that the token's owner 'some-service'
-        {
-            'name': 'idle-culler',
-            'description': 'Cull idle servers',
-            'scopes': [
-                'read:users:name',
-                'read:users:activity',
-                'read:users:servers',
-                'users:servers',
-            ],
-            'tokens': ['secret-token'],
-        },
-    ]
-    kwargs = {
-        'load_roles': roles_to_load,
-        'services': services,
-        'service_tokens': service_tokens,
-    }
-    ssl_enabled = getattr(request.module, "ssl_enabled", False)
-    if ssl_enabled:
-        kwargs['internal_certs_location'] = str(tmpdir)
-    hub = MockHub(**kwargs)
-    hub.init_db()
-    db = hub.db
-    await hub.init_api_tokens()
-    with pytest.raises(ValueError):
-        await hub.init_roles()
-
-    # delete the test services
-    for service in db.query(orm.Service):
-        db.delete(service)
-    db.commit()
-
-    # delete the test tokens
-    for token in db.query(orm.APIToken):
-        db.delete(token)
-    db.commit()
-
-    # delete the test roles
-    for role in roles_to_load:
-        roles.delete_role(db, role['name'])
-
-
 @mark.role
 @mark.parametrize(
     "headers, rolename, scopes, status",
@@ -1124,3 +964,315 @@ async def test_user_group_roles(app, create_temp_role):
     assert len(reply[0]['roles']) == 1
     assert reply[0]['name'] == 'jack'
     assert group_role.name not in reply[0]['roles']
+
+
+async def test_config_role_list():
+    roles_to_load = [
+        {
+            'name': 'elephant',
+            'description': 'pacing about',
+            'scopes': ['read:hub'],
+        },
+        {
+            'name': 'tiger',
+            'description': 'pouncing stuff',
+            'scopes': ['shutdown'],
+        },
+    ]
+    hub = MockHub(load_roles=roles_to_load)
+    hub.init_db()
+    hub.authenticator.admin_users = ['admin']
+    await hub.init_role_creation()
+    for role_conf in roles_to_load:
+        assert orm.Role.find(hub.db, name=role_conf['name'])
+    # Now remove elephant from config and see if it is removed from database
+    roles_to_load.pop(0)
+    hub.load_roles = roles_to_load
+    await hub.init_role_creation()
+    assert orm.Role.find(hub.db, name='tiger')
+    assert not orm.Role.find(hub.db, name='elephant')
+
+
+async def test_config_role_users():
+    role_name = 'painter'
+    user_name = 'benny'
+    user_names = ['agnetha', 'bjorn', 'anni-frid', user_name]
+    roles_to_load = [
+        {
+            'name': role_name,
+            'description': 'painting with colors',
+            'scopes': ['users', 'groups'],
+            'users': user_names,
+        },
+    ]
+    hub = MockHub(load_roles=roles_to_load)
+    hub.init_db()
+    hub.authenticator.admin_users = ['admin']
+    hub.authenticator.allowed_users = user_names
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user = orm.User.find(hub.db, name=user_name)
+    role = orm.Role.find(hub.db, name=role_name)
+    assert role in user.roles
+    # Now reload and see if user is removed from role list
+    roles_to_load[0]['users'].remove(user_name)
+    hub.load_roles = roles_to_load
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user = orm.User.find(hub.db, name=user_name)
+    role = orm.Role.find(hub.db, name=role_name)
+    assert role not in user.roles
+
+
+async def test_duplicate_role_users():
+    role_name = 'painter'
+    user_name = 'benny'
+    user_names = ['agnetha', 'bjorn', 'anni-frid', user_name]
+    roles_to_load = [
+        {
+            'name': role_name,
+            'description': 'painting with colors',
+            'scopes': ['users', 'groups'],
+            'users': user_names,
+        },
+        {
+            'name': role_name,
+            'description': 'painting with colors',
+            'scopes': ['users', 'groups'],
+            'users': user_names,
+        },
+    ]
+    hub = MockHub(load_roles=roles_to_load)
+    hub.init_db()
+    with pytest.raises(ValueError):
+        await hub.init_role_creation()
+
+
+async def test_admin_role_and_flag():
+    admin_role_spec = [
+        {
+            'name': 'admin',
+            'users': ['eddy'],
+        }
+    ]
+    hub = MockHub(load_roles=admin_role_spec)
+    hub.init_db()
+    hub.authenticator.admin_users = ['admin']
+    hub.authenticator.allowed_users = ['eddy']
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    admin_role = orm.Role.find(hub.db, name='admin')
+    for user_name in ['eddy', 'admin']:
+        user = orm.User.find(hub.db, name=user_name)
+        assert user.admin
+        assert admin_role in user.roles
+    admin_role_spec[0]['users'].remove('eddy')
+    hub.load_roles = admin_role_spec
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user = orm.User.find(hub.db, name='eddy')
+    assert not user.admin
+    assert admin_role not in user.roles
+
+
+async def test_custom_role_reset():
+    user_role_spec = [
+        {
+            'name': 'user',
+            'scopes': ['self', 'shutdown'],
+            'users': ['eddy'],
+        }
+    ]
+    hub = MockHub(load_roles=user_role_spec)
+    hub.init_db()
+    hub.authenticator.allowed_users = ['eddy']
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user_role = orm.Role.find(hub.db, name='user')
+    user = orm.User.find(hub.db, name='eddy')
+    assert user_role in user.roles
+    assert 'shutdown' in user_role.scopes
+    hub.load_roles = []
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user_role = orm.Role.find(hub.db, name='user')
+    user = orm.User.find(hub.db, name='eddy')
+    assert user_role in user.roles
+    assert 'shutdown' not in user_role.scopes
+
+
+async def test_removal_config_to_db():
+    role_spec = [
+        {
+            'name': 'user',
+            'scopes': ['self', 'shutdown'],
+        },
+        {
+            'name': 'wizard',
+            'scopes': ['self', 'read:groups'],
+        },
+    ]
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    await hub.init_role_creation()
+    assert orm.Role.find(hub.db, 'user')
+    assert orm.Role.find(hub.db, 'wizard')
+    hub.load_roles = []
+    await hub.init_role_creation()
+    assert orm.Role.find(hub.db, 'user')
+    assert not orm.Role.find(hub.db, 'wizard')
+
+
+async def test_no_admin_role_change():
+    role_spec = [{'name': 'admin', 'scopes': ['shutdown']}]
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    with pytest.raises(ValueError):
+        await hub.init_role_creation()
+
+
+async def test_user_config_respects_memberships():
+    role_spec = [
+        {
+            'name': 'user',
+            'scopes': ['self', 'shutdown'],
+        }
+    ]
+    user_names = ['eddy', 'carol']
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    hub.authenticator.allowed_users = user_names
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user_role = orm.Role.find(hub.db, 'user')
+    for user_name in user_names:
+        user = orm.User.find(hub.db, user_name)
+        assert user in user_role.users
+
+
+async def test_admin_role_respects_config():
+    role_spec = [
+        {
+            'name': 'admin',
+        }
+    ]
+    admin_users = ['eddy', 'carol']
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    hub.authenticator.admin_users = admin_users
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    admin_role = orm.Role.find(hub.db, 'admin')
+    for user_name in admin_users:
+        user = orm.User.find(hub.db, user_name)
+        assert user in admin_role.users
+
+
+async def test_empty_admin_spec():
+    role_spec = [{'name': 'admin', 'users': []}]
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    hub.authenticator.admin_users = []
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    admin_role = orm.Role.find(hub.db, 'admin')
+    assert not admin_role.users
+
+
+# Todo: Test that services don't get default roles on any startup
+
+
+async def test_hub_upgrade_detection(tmpdir):
+    db_url = f"sqlite:///{tmpdir.join('jupyterhub.sqlite')}"
+    os.environ['JUPYTERHUB_TEST_DB_URL'] = db_url
+    # Create hub with users and tokens
+    hub = MockHub(db_url=db_url)
+    await hub.initialize()
+    user_names = ['patricia', 'quentin']
+    user_role = orm.Role.find(hub.db, 'user')
+    for name in user_names:
+        user = add_user(hub.db, name=name)
+        user.new_api_token()
+        assert user_role in user.roles
+    for role in hub.db.query(orm.Role):
+        hub.db.delete(role)
+    hub.db.commit()
+    # Restart hub in emulated upgrade mode: default roles for all entities
+    hub.test_clean_db = False
+    await hub.initialize()
+    assert getattr(hub, '_rbac_upgrade', False)
+    user_role = orm.Role.find(hub.db, 'user')
+    token_role = orm.Role.find(hub.db, 'token')
+    for name in user_names:
+        user = orm.User.find(hub.db, name)
+        assert user_role in user.roles
+        assert token_role in user.api_tokens[0].roles
+    # Strip all roles and see if it sticks
+    user_role.users = []
+    token_role.tokens = []
+    hub.db.commit()
+
+    hub.init_db()
+    hub.init_hub()
+    await hub.init_role_creation()
+    await hub.init_users()
+    hub.authenticator.allowed_users = ['patricia']
+    await hub.init_api_tokens()
+    await hub.init_role_assignment()
+    assert not getattr(hub, '_rbac_upgrade', False)
+    user_role = orm.Role.find(hub.db, 'user')
+    token_role = orm.Role.find(hub.db, 'token')
+    allowed_user = orm.User.find(hub.db, 'patricia')
+    rem_user = orm.User.find(hub.db, 'quentin')
+    assert user_role in allowed_user.roles
+    assert token_role not in allowed_user.api_tokens[0].roles
+    assert user_role not in rem_user.roles
+    assert token_role not in rem_user.roles
+
+
+async def test_token_keep_roles_on_restart():
+    role_spec = [
+        {
+            'name': 'bloop',
+            'scopes': ['read:users'],
+        }
+    ]
+
+    hub = MockHub(load_roles=role_spec)
+    hub.init_db()
+    hub.authenticator.admin_users = ['ben']
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_role_assignment()
+    user = orm.User.find(hub.db, name='ben')
+    for _ in range(3):
+        user.new_api_token()
+    happy_token, content_token, sad_token = user.api_tokens
+    roles.grant_role(hub.db, happy_token, 'bloop')
+    roles.strip_role(hub.db, sad_token, 'token')
+    assert len(happy_token.roles) == 2
+    assert len(content_token.roles) == 1
+    assert len(sad_token.roles) == 0
+    # Restart hub and see if roles are as expected
+    hub.load_roles = []
+    await hub.init_role_creation()
+    await hub.init_users()
+    await hub.init_api_tokens()
+    await hub.init_role_assignment()
+    user = orm.User.find(hub.db, name='ben')
+    happy_token, content_token, sad_token = user.api_tokens
+    assert len(happy_token.roles) == 1
+    assert len(content_token.roles) == 1
+    print(sad_token.roles)
+    assert len(sad_token.roles) == 0
+    for token in user.api_tokens:
+        hub.db.delete(token)
+    hub.db.commit()
diff --git a/jupyterhub/tests/test_scopes.py b/jupyterhub/tests/test_scopes.py
--- a/jupyterhub/tests/test_scopes.py
+++ b/jupyterhub/tests/test_scopes.py
@@ -311,7 +311,7 @@ def temp_user_creator(*scopes, name=None):
 def create_service_with_scopes(app, create_temp_role):
     """Generate a temporary service with specific scopes.
     Convenience function that provides setup, database handling and teardown"""
-    temp_service = []
+    temp_services = []
     counter = 0
     role_function = create_temp_role
 
@@ -325,11 +325,12 @@ def temp_service_creator(*scopes, name=None):
         app.init_services()
         orm_service = orm.Service.find(app.db, name)
         app.db.commit()
+        temp_services.append(orm_service)
         roles.update_roles(app.db, orm_service, roles=[role.name])
         return orm_service
 
     yield temp_service_creator
-    for service in temp_service:
+    for service in temp_services:
         app.db.delete(service)
     app.db.commit()
 
@@ -619,7 +620,6 @@ async def test_server_state_access(
             )
         service = create_service_with_scopes(*scopes)
         api_token = service.new_api_token()
-        await app.init_roles()
         headers = {'Authorization': 'token %s' % api_token}
         r = await api_request(app, 'users', user.name, headers=headers)
         r.raise_for_status()
diff --git a/jupyterhub/tests/test_services.py b/jupyterhub/tests/test_services.py
--- a/jupyterhub/tests/test_services.py
+++ b/jupyterhub/tests/test_services.py
@@ -89,10 +89,10 @@ async def test_external_service(app):
             }
         ]
         await maybe_future(app.init_services())
-        await maybe_future(app.init_roles())
+        await maybe_future(app.init_role_creation())
         await app.init_api_tokens()
         await app.proxy.add_all_services(app._service_map)
-        await app.init_roles()
+        await app.init_role_assignment()
 
         service = app._service_map[name]
         api_token = service.orm.api_tokens[0]