Use AuthRef for some client fields

This tackles some TODO items left over.

Change-Id: Ib062744acbf56f05d09857d244b78b35c0ef4d39
Signed-off-by: Julien Danjou <julien@danjou.info>
Merge "Use AuthRef for some client fields"