codalab/codalab — pull request #367 by cpoulain (2014-02-23 07:57:29+00:00)
Title: Add OAuth2 authorization server and token validation API for bundle server

This PR enables the web site to act as an OAuth2 authorization server. This is a step to enable an instance of the CodaLab command line tool to connect securely with the bundle service.

**Workflow**

A picture of the expected authentication flow is given below. Labels in the picture mirror the OAuth 2 [terminology](http://tools.ietf.org/html/rfc6749#section-1.1). The key parties are:
- **Resource owner**: a CodaLab end-user with access-rights to a set of bundles stored in the CodaLab bundle server. Bundles are the protected resources.
- **Resource server**: CodaLab bundle server which stores and serves the protected bundles.
- **Client**: CLI tool.
- **Authorization server** (labelled **Authentication Server** on the picture): it is the service which authenticates the user.

![flow](https://f.cloud.github.com/assets/3453398/2239844/8f4c16d8-9c5e-11e3-96e4-b198bd1c24d4.jpg)

**Implementation details**

The OAuth authentication service is a component hosted in the Django web site. The implementation relies on a stack of two open-source libraries:
- [OAuthLib](https://oauthlib.readthedocs.org/en/latest/index.html) implements the core logic of the various OAuth flows but leaves parts of implementing an OAuth provider unspecified in order to not tie into a specific framework;
- [Django OAuth toolkit](https://django-oauth-toolkit.readthedocs.org/) extends OAuthLib by providing a pluggable Django app.

The two packages were added to the list of requirements `codalab/requirements/common.txt`:
```
oauthlib==0.6.1
django-oauth-toolkit==0.6.1
```

CodaLab specific implementations is in app `apps.authenz`. Some things to note:

- `~/clients/token/` is the endpoint used by a client to authenticate and obtain a token. This is provided by Django OAuth toolkit. Additional endpoints provided by the toolkit are excluded since they are not needed today.
- `~/clients/validation/` is the endpoint used by a trusted bundle service to validate a token and translate it into an actual user ID (and eventually associated group IDs). The implementation of the endpoint is class `ValidationApi` in `apps\authenz\views.py`. Access to the endpoint is protected using `ScopedProtectedResourceView`, thus access is only granted when a valid token with scope `'token-validation` is presented.
- Scope validation happens in class `Validator` of `apps/authenz/oauth.py`. The validation should eventually rely on groups/roles that the user belongs too. For now, the implementation rely on the `authorization_grant_type` of the client app. In the current setup, a trusted bundle service will use "Client Credentials Grant" while other clients will use "Resource Owner Password Credentials Grant".
- For any CodaLab user, an OAuth client is automatically created to allow that user to access CodaLab with the CLI tool. The OAuth client is created after sign-up using the hook installed in `apps\authenz\views.py`. For backwards compatibility, we also check for the client's existence after a user logs in. I'll plan to remove the 'on login' hook after a script to update existing databases is written.

That's it for the implementation, now to some usage examples.

**Sample client code**

The code below shows how a client app can call the authentication service to obtain a token and to refresh a token. The code is straight Python and does not require additional libraries. The code assumes that the authentication service is running at the URL given by `get_token_url` (i.e. the CodaLab web site is running locally in this example). In addition, it is assumed that user `test1` is registered with the local CodaLab web site and that the password of `test1` is `abcdef`.

```
import json
import time
import urllib
import urllib2
from base64 import encodestring

username = 'test1'
password = 'abcdef'
appname = 'cli_client_{0}'.format(username)
get_token_url = 'http://localhost:1367/clients/token/'

# Get OAuth2 token using Resource Owner Password Credentials Grant

headers = {
'Authorization': 'Basic {0}'.format(encodestring('%s:' % appname).replace('\n',''))
}
data = [
('grant_type', 'password'),
('username', username),
('password', password)
]
request = urllib2.Request(get_token_url, urllib.urlencode(data, True), headers)
try:
response = urllib2.urlopen(request)
token_info = json.load(response)
print 'Token type: %s' % token_info['token_type']
print 'Access token %s' % token_info['access_token']
print 'Refresh token %s' % token_info['refresh_token']
print 'Expires at %s' % (time.time() + int(token_info['expires_in']))
except urllib2.URLError as e:
print e
except Exception as e:
print e

# If token was acquired successfully, this shows how to refresh it.

if token_info:

data = [
('grant_type', 'refresh_token'),
('refresh_token', token_info['refresh_token'])
]
request = urllib2.Request(get_token_url, urllib.urlencode(data, True), headers)
try:
response = urllib2.urlopen(request)
token_info = json.load(response)
print 'Token type: %s' % token_info['token_type']
print 'Access token %s' % token_info['access_token']
print 'Refresh token %s' % token_info['refresh_token']
print 'Expires at %s' % (time.time() + int(token_info['expires_in']))
except urllib2.URLError as e:
print e
except Exception as e:
print e
```

**Sample token validation code**

In this example, the trusted bundle server is the client calling the authentication server. First the trusted client authenticates itself and obtains a valid token. Notice that the request for the token is specifying the desired scope of `token-validation`.

The bundle service will receive a call from a CLI client. The CLI client will provide an OAuth token. In the second part of the example, the code shows how the trusted bundle service calls the validation endpoint `http://localhost:1367/clients/validation/` to translate the token received from the CLI client into a CodaLab user ID.

```
import json
import time
import urllib
import urllib2
from base64 import encodestring

# Trusted bundle service client starts by obtaining a valid token.

get_token_url = 'http://localhost:1367/clients/token/'
app_id = 'D9TqDG=nSNL;D!u4if=Tb?T3A1ppQ0Gx=fZT@A5p'
app_secret = 'BS7B;@IFM3yO0DXX??FlC1-q13Y@QxEQwjQOW9hrgxVKhBA'

app_sig = '%s:%s' % (app_id, app_secret)
headers = {
'Authorization': 'Basic {0}'.format(encodestring(app_sig).replace('\n',''))
}
data = [
('grant_type', 'client_credentials'),
('scope', 'token-validation')
]
request = urllib2.Request(get_token_url, urllib.urlencode(data, True), headers)
try:
response = urllib2.urlopen(request)
token_info = json.load(response)
print 'Token type: %s' % token_info['token_type']
print 'Access token %s' % token_info['access_token']
print 'Expires at %s' % (time.time() + int(token_info['expires_in']))
except urllib2.URLError as e:
print e
except Exception as e:
print e

# Bundle service uses its access token to validate the access token that it received
# from the CLI client.

service_auth_token = token_info['access_token']
user_auth_token = '54KVfvHgk960QY5UEJ7uPZLJcQaBSM'
validation_url = 'http://localhost:1367/clients/validation/'

headers = {
'Authorization': 'Bearer {0}'.format(service_auth_token)
}
data = [
('token', user_auth_token)
]
request = urllib2.Request(validation_url, urllib.urlencode(data, True), headers)
try:
response = urllib2.urlopen(request)
result = json.load(response)
status_code = result['code'] if 'code' in result else 500
if status_code == 200:
print 'User ID: %s' % result['user']['id']
print 'Username: %s' % result['user']['name']
elif status_code == 403 or status_code == 404:
print 'User credentials are not valid'
else:
print 'The token translation failed.'
except urllib2.URLError as e:
print e
except Exception as e:
print e
```

** Created a client for a trusted bundle service **

This needs to be done by running a script during deployment. The script will resemble this:
```
import sys, os.path, os

root_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "codalab")
sys.path.append(root_dir)

# Set things for django configurations
os.environ.setdefault("DJANGO_CONFIGURATION", "Dev")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "codalab.settings")

# Import the configuration
from configurations import importer
importer.install()

from django.contrib.auth import get_user_model
User = get_user_model()

user = User.objects.get(username='codalab')

from oauth2_provider.models import Application

client, created = Application.objects.get_or_create(
user=user,
client_type=Application.CLIENT_CONFIDENTIAL,
authorization_grant_type=Application.GRANT_CLIENT_CREDENTIALS,
name='Bundle service client')

print 'Client id: %s' % client.client_id
print 'Client secret: %s' % client.client_secret
```