derks/drest — Issue #7: Cache http object for improved performance, esp. for Auth
Author: aalcock
Created: 2012-03-08 07:20:46+00:00
State: open

When drest is used against a service requiring HTTP AUTH (Basic or Digest), each request accesses the resource receives an HTTP 401 (Unauthorized) from the server, causing a second request with the authentication tokens to be issued from httplib2.

For REST services without AUTH, the situation is much better, but the http connection obviously cannot take advantage of keep-alive.

This works from a functional perspective but suffers obvious performance issues (could easily introduce 100's of ms of latency per request, throttling maximum throughput between 10x and 100x).

Instead, drest could cache the http object and allow httplib2 to optimize throughput. This could either be done at the
* Module level (disadvantage being that the developer cannot connect to multiple REST services on different servers), or
* The API object level (just the 'right' number of http objects, but problematic if Resources do not share the same credentials), or
* The Resource level (more http objects, but less complexity).

The code snippet below is a naive implementation at the Resource level (simplest):

```python
class RequestHandler(MetaMixin):
# snip

def set_auth_credentials(self, user, password):
# snip
self._auth_credentials = (user, password)
self._clear_http()

# snip

def _get_http(self):
if self._http == None:
if self._meta.ignore_ssl_validation:
self._http = Http(disable_ssl_certificate_validation=True)
else:
self._http = Http()

if self._auth_credentials:
self._http.add_credentials(self._auth_credentials[0],
self._auth_credentials[1])
return self._http

def _clear_http(self):
self._http = None

def _make_request(self, url, method, payload={}, headers={}):
# snip
try:
return self._get_http().request(url, method, payload, headers=headers)
except socket.error as e:
raise dRestAPIError(e.args[1])
```