Async princ rekey

As suggested by Benjamin Kaduk, I send this pull request concerning the work performed to add client side principal rekeying using new commands ktutil/ukt and kadmin/ktsync.

The logic/reasons of that addition are discussed in the krb5dev thread called "Question related to keytab entries upgrade" started on Jan 14th 2013.


The following notes explain the logic :

# Enhancement for async hot-rekeying of service principals

## Introduction

In order to improve the security of kerberized systems, it is necessary to regularly
rekey the principals involved in the environment.

Kerberos V enables to manage versioning of principal keys for principal using a
particular field associated to each key entry, the **kvno** field (Key Version Number).

While **kvno** helps to manage the transition in different version of keys, some 
limitations have been identified in the current possible workflow using the MIT krb5
implementation and command-line clients (MIT krb5-1.10 at the time of writing).

The **kvno** of key entries are incremented by one every time the KDC handles a modification
of the keys associated to a principal. For users, it corresponds to the modification of their 
password, and for the hosts, it corresponds to a new *ktadd* operation using *kadmin* or
*kadmin.local* for that hosts principal(s).

As soon as a new version of a key (incremented kvno) is available on the KDC for a particular
principal, all the new AS_REQ or TGS_REQ replies  will be made based on the new key material
offered by this highest kvno entries. As a result, as soon as the key has been upgraded on
the KDC, every new attempt to contact an upgraded kerberized service will use the latest
keys. It is thus mandatory to push the new versions of the keys to the server as soon as
possible.

This race window is short in common scenario, where each host of a realm automatically
rekey its own keytab entries, contacting the *kadmind* daemon of the realm and performing
a *ktadd*operation. However, the race window is still present and can result in connection
failure during the *hot-add* of the new entries. Furthermore, having the hosts handling
the rekey operation requires the kadmind service to be available from those hosts, so it
requires to open the kadmind port to every host in the realm.

In scenario where one wants to centralized the management of the keys, the KDC or a 
dedicated and authorized host can be used to do the rekey operations and then push the new key
materials to targeted remote hosts. This solution has the benefit of not requiring to open
the kadmind port for all the hosts but still suffers from the race window, and probably a little
bit more than in the previous case as the window no longer correspond to a *kadmin/ktadd* operation
bu to a *kadmin/ktadd + scp* operation.

The target of this document is to present a alternative to that logic in order to enhance the
*hot-rekeying* of kerberos principals.

## Proposal

This proposal is the result of email discussions on the MIT Kerberos V *krb5dev* mailing-list.
Thanks to Greg Hudson, Russ Albery, Nico Williams and Roland Dowdeswell for their inputs on that
problem that help to define this alternate rekeying solution.

The main idea of the proposal is to remove the race window by ensuring that the upgraded key
materials are present on the host keytabs before being used by the KDC.

For that purpose, the *set-key* kadmin RPC is used. This RPC enables to push new versions of
keys to the KDC database for a particular. Using that RPC, it is possible to randomly generate
new keys (incremented kvno) on a node for a particular principal, ensure that the service 
associated to the keys has access to both the old and the new versions and when necessary,
inform the KDC that a new version of the key has to be used when delivering TGS.

The proposal is based on the modification of two MIT krb5 command-line clients, *ktutil* and
*kadmin[.local]*, adding the following capabilities :

* *ktutil/upgrade_kt* function : automatically extract the highest version entries from a keytab
and upgrade (rekey+kvno increase) that entries.
* *kadmin/ktsync* : push entries corresponding to the next kvno value of a principal from a keytab
to the KDC database.

*ktutil/upgrade_kt* can be used to prepare in advance the *hot-add* of new keys to kerberized
services keytabs. *kadmin/ktsync* can be used after the upgraded keytab deployment to inform
the KDC that new versions of the keys can now be used when accessing the associated principal.

## Example

This chapter illustrates the implemented logic and is splitted in two sections, one corresponding
to the upgrade of the keytab, the other one to the synchronization of the KDC entries with the
upgraded one.

Different workflows can be derived from this example.

## Keytab entries update

As soon as the keytab entries will be updated, the kadmin function will no longer work as
the KDC keys will not match the highest kvno keys of the keytab. This will have to solved in
the future, for now, just start by making a copy of the keytab for later usage in kadmin.

    [root@foo ~]# cp /etc/krb5/krb5.keytab /etc/krb5/krb5.keytab.backup
    [root@foo ~]# 

The current entries are in kvno 3 in the keytab :

    [root@foo ~]# klist -ekt /etc/krb5.keytab
    Keytab name: WRFILE:/etc/krb5.keytab
    KVNO Timestamp         Principal
    ---- ----------------- --------------------------------------------------------
       3 01/23/13 10:14:38 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
       3 01/23/13 10:14:38 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
       3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
       3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
       3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
       3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
       3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
       3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
       3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
       3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
       3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
       3 01/23/13 10:14:40 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
    [root@foo ~]#

First upgrade the entries using ktutil/ukt :

	[root@foo ~]# ktutil
	ktutil:  ukt /etc/krb5/krb5.keytab
	12 entry(s) upgraded
	ktutil:  wkt /etc/krb5/krb5.keytab
	ktutil:  quit
	[root@foo ~]#
	
Confirmation of the upgrade using klist, the keytab can now serve AP_REQ for kvno 3 and 4 :


	[root@foo ~]# klist -ekt /etc/krb5.keytab
	Keytab name: WRFILE:/etc/krb5.keytab
	       KVNO Timestamp         Principal
	---- ----------------- --------------------------------------------------------
	   3 01/23/13 10:14:38 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
	   3 01/23/13 10:14:38 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
	   3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
	   3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
	   3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
	   3 01/23/13 10:14:39 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
	   3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
	   3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
	   3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
	   3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
	   3 01/23/13 10:14:39 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
	   3 01/23/13 10:14:40 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
	   4 01/23/13 10:17:16 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
	   4 01/23/13 10:17:16 host/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
	   4 01/23/13 10:17:16 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
	   4 01/23/13 10:17:16 host/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
	   4 01/23/13 10:17:17 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
	   4 01/23/13 10:17:17 host/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes256-cts-hmac-sha1-96)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (aes128-cts-hmac-sha1-96)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des3-cbc-sha1)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (arcfour-hmac)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-hmac-sha1)
	   4 01/23/13 10:17:17 nfs/foo.athena.mit.edu@ATHENA.MIT.EDU (des-cbc-md5)
	[root@foo ~]#
	

## Keytab entries synchronization 

First, we need to be sure that kadmin ACL are correct. In our case we need to have the following lines in kadm5.acl :

	[root@foo ~]# tail -n 1 /var/kerberos/krb5kdc/kadm5.acl
	host/foo.athena.mit.edu@ATHENA.MIT.EDU si */foo.athena.mit.edu@ATHENA.MIT.EDU
	[root@foo ~]#
	
Then we can proceed with the synchronization :

	[root@foo ~]# kadmin -p host/foo.athena.mit.edu@ATHENA.MIT.EDU -kt /etc/krb5/krb5.keytab.backup
	Authenticating as principal host/foo.athena.mit.edu@ATHENA.MIT.EDU with keytab /etc/krb5/krb5.keytab.backup.
	kadmin:  ktsync host/foo.athena.mit.edu@ATHENA.MIT.EDU
	6 KDC entry(s) for principal host/foo.athena.mit.edu@ATHENA.MIT.EDU synced from keytab WRFILE:/etc/krb5.keytab.
	kadmin:  ktsync nfs/foo.athena.mit.edu@ATHENA.MIT.EDU
	6 KDC entry(s) for principal nfs/foo.athena.mit.edu@ATHENA.MIT.EDU synced from keytab WRFILE:/etc/krb5.keytab.
	kadmin:  quit
	[root@foo ~]#
	

# Known limitations of the solution

One known limitation of that solution, presented by Roland Dowesdell and mentioned in the 
example section, is the fact that once a new version of a key is available in a keytab, the krb5
library will automatically use it when trying to grant TGT for the associated principal.
There is no mechanism to try the oldest keys in case of failure of the latest.
This should be modified to ensure a complete and transparent *hot-rekeying* of principles using 
MIT krb5 implementation.