odoo/odoo — pull request #3989 by xmo-odoo (2014-12-02 11:30:23+00:00)
Title: [ADD] new XMLRPC endpoint POC

The current XMLRPC layers (`/xmlrpc/` and `/xmlrpc/2/`) use the old calling conventions (CC). They can't be converted to the new CC as there is not enough information at their point to do so (knowledge of whether/where ids and contexts are). This means new-CC methods *must* currently be decorated with `@model` or `@multi` to be exposed over RPC, or calling them raises a TypeError (number of parameters does not match). The old CC can't be deprecated/removed until RPC has been converted to the new CC.

At the same time, while working on the webservice doc, I found out Odoo's xmlrpc layer is pretty idiosyncratic and doesn't really take advantage of xmlrpc and the facilities provided by xmlrpc libraries. The endpoint switch caused by the new CC was the occasion to explore that part as well.

# Calling Conventions

To match the new calling conventions, the new endpoint has to split up the ids (if any) and context (if any) from the positional and keyword arguments. This was done by adding a mandatory `subject` positional argument which can be:
* falsy, in which case the method is called on an empty recordset with a context-less environment
* a list, which is assumed to be a list of record ids used for the subject recordset
* a mapping which may provide the keys `records` and `context`. The `records` key is browsed (or ignored) and the `context` key is used as base for the environment
this scheme seems to provide the best flexibility and terseness, as well as extensibility by using more keys from the `subject` mapping.

Like `execute_kw`, calls then take a `args` list and a `kwargs` dict, both are optional rather than just the kwargs: a method can be called with only args, only kwargs, or neither.

# Authentication
All xmlrpc libraries surveyed provide built-in support for HTTP Basic Authentication. The new endpoint thus replaces the custom authentication by HTTP Basic
* no less (or more) secure than the existing scheme: relies on underlying transport being HTTPS for security
* avoids an extraneous auth step to resolve a uid
* no need to keep uid/password around, they're embedded in the xmlrpc proxy object
* easy to integrate (Basic sends "cleartext" username and password)

# Endpoint
Uses `/RPC2`:
* does not conflict with existing endpoints
* used for spec examples
* a number of documents call it the conventional XML-RPC path
* at least [one stdlib defaults to /RPC2](http://www.ruby-doc.org/stdlib-2.1.5/libdoc/xmlrpc/rdoc/XMLRPC/Client.html#method-c-new) when no `path` is provided

# Leverage XML-RPC conventions
XML-RPC `methodName` is conventionally a dotted path (e.g. `system.listMethods`), which some libraries (`ripcord`, `xmlrpclib`) can map onto the language's own dereferencing/attribute access. This maps nicely to Odoo's models: the last segment of the `methodName` is the method itself, the rest is the model name. This means `read` for `account.account` can be called as:
```python
a_db.account.account.read(…)
```

Originally the database name was part of the path, but because authentication depends on the db (or lack thereof), it was moved to the endpoint (URL query parameter)
* Global methods (e.g. create/drop database) are done on a DB-less endpoint
* DB-specific methods (none currently but maybe printing reports? Sending workflow signals?) are function calls on DB'd endpoint (no dot in path)
* rest is method call on a model

# Code comparison
## setup
```python
common = xmlrpclib.ServerProxy('https://{}/xmlrpc/2/common'.format(domain))
uid = common.authenticate(db, username, password, {})
models = xmlrpclib.ServerProxy('https://{}/xmlrpc/2/object'.format(domain))
```
becomes
```python
db = xmlrpclib.ServerProxy('https://{}:{}@{}/RPC2?db={}'.format(username, password, domain, db))
partners = db.res.partner
```
## simple method call
```python
models.execute_kw(db, uid, password, 'res.partner', 'check_access_rights', ['read'], {'raise_exception': False})
```
becomes
```python
partners.check_access_rights((), ['read'], {'raise_exception': False})
```
## search
```python
models.execute_kw(db, uid, password, 'res.partner', 'search', [[['is_company', '=', True], ['customer', '=', True]]])
```
becomes
```python
partners.search((), [[['is_company', '=', True], ['customer', '=', True]]])
```
## read
```python
models.execute_kw(db, uid, password, 'res.partner', 'read', [ids])
```
becomes
```python
partners.read(ids)
```
# Custom XMLRPC serializer
New CC methods can return recordsets. Customizing `xmlrpclib.Marshaller` didn't work because it dispatches by strict type equality, so it wasn't possible to hook into it to serialise arbitrary recordsets (they get types created on-the-fly). A new marshaller was reimplemented on top of lxml instead. It provides the following features not provided by `xmlrpclib.Marshaller`:
* serialises recordsets to lists of ids, uses `BaseModel.ids` (ignores NewId)
* converts all mapping keys to strings (same as `json.dumps`) instead of raising an error
* can serialise arbitrary mappings (e.g. Counter, defaultdict) not just `dict`
* can serialise arbitrary iterables, not just `list` and `tuple`

# Possible additions/reflexions
* enable allow_none

/cc @odony @rco-odoo