KDC Audit infrastructure and plugin implementation

Per project http://k5wiki.kerberos.org/wiki/Projects/Audit

The purpose of this project is to create an Audit infrastructure to monitor
security related events on the KDC.

The following events are targeted in the initial version:
- startup and shutdown of the KDC;
- AS_REQ and TGS_REQ exchanges.  This includes client address and port, KDC
  request and request ID, KDC reply, primary and derived ticket and their
  ticket IDs, second ticket ID, cross-realm referral, was ticket renewed and
  validated, local policy violation and protocol constraints, and KDC status
  message.

Ticket ID is introduced to allow to link tickets to their initial TGT at any
stage of the Kerberos exchange. For the purpose of this project it is a private
to KDC ticket ID: each successfully created ticket is hashed and recorded
into audit log. The administrators can correlate the primary and derived
ticket IDs after the fact.

Request ID is a randomly generated alpha-numeric string. Using this ID an
administrator can easily correlate multiple audit events related to a single
request. It should be informative both in cases when the request is sent to
multiple KDCs, or to the same KDC multiple times.

For the purpose of testing and demo of the Audit, the JSON based modules are
implemented: "test" and "simple" audit modules respectively.
The file plugins/audit/j_dict.h is a dictionary used in this implememtations.

The new Audit system is build-time enabled and run-time pluggable.

[kaduk@mit.edu: remove potential KDC crashes, minor reordering]

ticket: 7712
target_version: 1.12
Fix audit test module initialization

Check if the fopen result is NULL, not whether it's less than zero.

ticket: 7713 (new)
target_version: 1.12
tags: pullups
Update for krb5-1.13-prerelease
Perform complete referrals in t_referral.py

t_referral.py was written to exercise KDC host referral logic, and did
not actually create the target realm, instead just looking at the
error message from gcred to determine whether the KDC returned a
referral or not.  It's only a small amount of additional work to
actually set up the target realm and check that the client code
successfully retrieves the referral, so do that instead.

Since the referral and non-referral outcomes aren't all that similar
any more, split test() into testref() and testfail().  Get rid of the
message argument, since it wouldn't be output in most cases where we
get an unexpected result.
Fix GSSAPI krb5 cred ccache import

json_to_ccache was incorrectly indexing the JSON array when restoring
a memory ccache.  Fix it.

Add test coverage for a multi-cred ccache by exporting/importing the
synthesized S4U2Proxy delegated cred in t_s4u2proxy_krb5.c; move
export_import_cred from t_export_cred.c to common.c to facilitate
this.  Make a note in t_export_cred.py that this case is covered in
t_s4u.py.

ticket: 7706
target_version: 1.11.4
Change KRB5KDC_ERR_NO_ACCEPTABLE_KDF to 100

draft-ietf-krb-wg-pkinit-alg-agility-07 specifies
KDC_ERR_NO_ACCEPTABLE_KDF as 82, but this value conflicts with
KRB_AP_ERR_PRINCIPAL_UNKNOWN from RFC 6111.  The former value has been
reassigned to 100 to fix the conflict.  Use the correct value.

We believe that this error won't crop up in practice for a long time
(when SHA-2 has been superceded by other hash algorithms and people
are desupporting it), by which time implementations will mostly have
been upgraded to use the new value.

ticket: 7715 (new)
target_version: 1.12
Add missing entries to tests/gssapi Makefile.in

Some test sources files, objects, or programs were missing from SRCS,
OBJS, all, check-pytests, or clean.  t_oid was also out of order in a
couple of places.
Fix gss_accept_sec_context error tokens

A GSS krb5 error response contains a KRB-ERROR message, which is
required to have a server principal name, although few recipients
actually use it.  Starting in 1.3, accept_sec_context would fail to
encode the error in the GSS_C_NO_NAME/GSS_C_NO_CREDENTIAL case
(introduced by #1370) because cred->princ (which became
cred->name->princ in 1.8) is unset.

This problem got worse in 1.10 because we stopped setting the server
field in all cases due to the changes for #6855.  In 1.11 the problem
got worse again when a misguided change to the mechglue started
discarding output tokens when the mechanism returns an error; the
mechglue should only do so when it itself causes the error.

Fix krb5 gss_accept_sec_context by unconditionally decoding the AP-REQ
and using krb5_rd_req_decoded, and then using the requested ticket
server in the KRB-ERROR message.  Fix the mechglue
gss_accept_sec_context by reverting that part of commit
56feee187579905c9101b0cdbdd8c6a850adcfc9.  Add a test program which
artificially induces a replay cache failure (the easiest failure we
can produce which has an associated RFC 4120 error code) and checks
that this can be communicated back to the initiator via an error
token.

ticket: 1445
target_version: 1.12
tags: pullup
Use protocol error for PKINIT cert expiry

If we fail to create a cert chain in cms_signeddata_create(), return
KRB5KDC_ERR_PREAUTH_FAILED, which corresponds to a protocol code,
rather than KRB5_PREAUTH_FAILED, which doesn't.  This is also more
consistent with other error clauses in the same function.

ticket: 7718 (new)
target_version: 1.12
tags: pullup
Discuss cert expiry, no-key princs in PKINIT docs

In pkinit.rst, add "-days" options to the example commands for
creating certificate and briefly discuss the issue of expiration dates
so that the administrator thinks about it.  In troubleshoot.rst, add
an entry for the "certificate has expired" error which results from
PKINIT (when linked with OpenSSL) when a certificate has expired.

ticket: 7719 (new)
target_version: 1.12
tags: pullup
Fix race in util/profile/Makefile.in

$(BUILDTOP)/include/profile.h was being updated by two different
rules, which could collide with make -j.  Use a dependency from
includes instead of a redundant rule.
Fix typos in kdb5_util master key command outputs

kdb5_util list_mkeys was beginning lines with "KNVO" instead of
"KVNO".  kdb5_util purge_mkeys was displaying "follwing" instead of
"following" for both dry-run and normal cases.

ticket: 7730 (new)
target_version: 1.12
tags: pullup
Use active master key in update_princ_encryption

kdb5_util update_princ_encryption should update to the active master
key version, not the most recent.

ticket: 6507
target_version: 1.12
tags: pullup
Reload master keys to find active mkey if needed

Refactor krb5_dbe_find_act_mkey and make it reload the master key list
if it doesn't find a master key matching the active mkvno.

ticket: 7685
Don't cache active master key list in kadmind

"kdb5_util use_mkey" should not require a kadmind restart to take
effect.  At the cost of fetching the K/M principal once for each key
change operation, make kadmind use the current active master key list
for each operation.

ticket: 7685
target_version: 1.12
tags: pullup
Correctly activate master keys in pre-1.7 KDBs

Starting with 1.7, databases are created with actkvno tl-data in the
K/M entry which gives the initial master key version an activation
time of 0.  A database created before 1.7 will not have this tl-data,
but we should behave in the same way as we do for a more recent
database.

Move the actkvno list synthesis code from krb5_dbe_fetch_act_key_list
to krb5_dbe_lookup_actkvno so it applies to kdb5_util commands as well
as libkadm5.  Synthesize the same list as we would have initialized
the KDB with, with an activation time of 0 for the earliest master
key.

ticket: 7686
target_version: 1.12
tags: pullup
Add master key rollover tests in k5test framework

Add a new script t_mkey.py using the k5test framework.  Test the fixes
for #6507, #7685, and #7686 as well as basic functionality and
old-stashfile compatibility.

dump.16 was created by running "kdb5_util create -s -P footes" and
"kdb5_util dump dumpfile" with krb5 1.6.  The key from the resulting
stash file was extracted and placed in the struct.pack() call in the
new test script.
Remove old master key tests

Remove tests/mkeystash_compat and tests/mk_migr.  These are superseded
by t_mkey.py, with two exceptions:

tests/mk_migr included tests for password history across master key
rollovers.  Historical keys are encrypted in the kadmin/history key
(which is accessed like any other key), so there isn't a specific need
to test this unless we implement #1221.

tests/mk_migr had provisions for testing master key rollover with the
LDAP KDB module.  All master key logic used in the LDAP KDB module is
shared with the DB2 module in lib/kdb, so there is no specific need to
test this combination.
Simplify kdb5_list_mkeys actkvno list retrieval

After recent changes, krb5_dbe_lookup_actkvno cannot yield an empty
list and cannot return KRB5_KDB_NOACTMASTERKEY.
Reset key-generation parameters for each enctype

In add_key_pwd, initialize s2k_params to NULL inside the loop over
enctypes instead of outside the loop, so that if the afs3 salt type
is used it does not contaminate later enctype/salt pairs in the list.

ticket: 7733
tags: pullup
target_version: 1.12