Refactor KDC option/flag processing

A lot of KDC code was spent copying options to flags, and copying
header_ticket flags to the output ticket.

Behavior change: previous code didn't copy PROXY from the
header_ticket, but this seems to have been a minor bug rather than
intentional.  This also seems to have been an omission from RFC 4120.
Remove compile_et.c

We have used awk scripts in preference to the C version of compile_et
since 1994 (r4426).  Get rid of the sources for the C version and all
of the Makefile.in references to them.
Add helper to change extended error message code

k5_change_error_message_code allows an extended error message to be
used when an error code is remapped.
Use k5_change_error_message_code in krb5 GSS mech

Simplify acquire_accept_cred using the new helper function.
Improve krb5_rd_req decryption failure errors

When krb5_rd_req cannot decrypt a ticket, try to produce the most
helpful diagnostic we can, and return an error code which corresponds
to the most applicable Kerberos protocol error.  Add a trace log
containing the error message for ticket decryption failures, in case
the application server does not log it.

Add new tests to cover krb5_rd_req error messages and adjust existing
tests to match the new messages.  Also adjust svc_auth_gssapi.c to
look for KRB5KRB_AP_ERR_NOT_US instead of KRB5KRB_AP_WRONG_PRINC.

ticket: 7232
Don't remove ccache creds before storing them

Revert #6291, which attempts to suppress duplicate credentials by
calling krb5_cc_remove in krb5_cc_store_cred.  Most of our ccache
types don't implement remove_cred, and avoiding duplicate credentials
is a responsibility better suited to the ccache implementation for
atomicity reasons.

Removing this call gets rid of the misleading "Removing ... from ..."
messages in trace logs.

ticket: 7906 (new)
Fix unlikely memory leak exporting lucid context

If the GSSAPI context has an unrecognized proto field (which should
never happen), free the context we allocated before returning EINVAL.

[ghudson@mit.edu: clarify commit message]
Fix unlikely memory error in krb5_rd_cred

If an error occurs in the for loop in krb5_rd_cred_basic (which should
only happen on an ENOMEM), do not leave the caller with a dangling
reference to the freed credential array.

ticket: 7908 (new)
target_version: 1.12.2
Allow GSS mechs to force mechlistMIC in SPNEGO

During a SPNEGO negotiation, if the NTLMSSP mechanism is used and a
MIC is produced within the final initiator mechanism token, Microsoft
servers require a mechlistMIC even if NTLMSSP was the most preferred
mechanism.

In spnego_mech.c, add a helper function mech_requires_mechlistMIC
which queries the mechanism to determine whether we might need to
produce a mechlistMIC for interoperability.  Call it after each call
to the mechanism's gss_init_sec_context and set sc->mic_reqd if it
returns true.  Although only the second call to NTLMSSP will actually
ever return true, the first call makes the mechanism aware that the
SPNEGO implementation supports this feature.

[ghudson@mit.edu: clarified commit message and code]

ticket: 7907 (new)
Modernize rd_cred.c

Adjust the internal abstractions so that decrypt_encpart is
responsible for the fallback from receiving subkey to session key, and
krb5_rd_cred is responsible for decoding and calling decrypt_encpart.
Rename krb5_rd_cred_basic to make_cred_list since it is now only
responsible for constructing the list.
Add k5_buf_get_space

Add a new k5_buf method to make room in the buffer for the caller to
fill in.
Add k5-input.h

Add a header containing a declaration for struct k5input and static
inline functions to use it.  A k5input structure can be used to safely
extract bytes and integers from a fixed-sized input buffer without
overrunning the buffer.  It supports deferred error checking (similar
to k5buf) by returning dummy values and storing a status value in the
input structure.
Add cred marshalling functions

Add a new file ccmarshal.c containing functions to marshal and
unmarshal credentials in file formats version 1-4.  These will replace
the functions in cc_file.c and cc_keyring.c, and can be used for KCM
in the future.
Add ccache marshalling tests

Add a new ccache test program t_marshal.c which verifies the new
marshalling functions and also the FILE ccache type against the
expected representations of the ccache header, default principal, and
credentials.
Modernize cc_file.c and cc_keyring.c

Rename functions not to use the krb5_ prefix, and to use the fcc_ or
krcc_ prefixes only for ccache method implementations.  Use shorter
function comments and add missing comments in some cases.  Remove
forward declarations where they aren't needed.  Use native types
(uint16_t, void *, etc.) in preference to Kerberos type wrappers.  Use
"ret" as the variable name for krb5_error_code return values.  Use 0
instead of KRB5_OK.  Make whitespace conform to current practice.
Remove old #if 0 blocks.  Don't cast to and from void * or between
integer types when C guarantees an implicit cast.  Use literal 2 and 4
for the size of fixed-width 16-bit and 32-bit types.

In cc_file.c, rewrite the header comment to specify the header format
as updated by version 4 and refer to ccmarshal.c for the principal and
credential format.  Also add a helper function to return the cache
version as an integer from 1 to 4, allowing more concise version
checks.
Use cred marshalling functions in cc_file.c

Replace the cc_file.c credential and principal marshalling and
unmarshalling functions with calls to the ccmarshal.c functions.  For
unmarshalling, we still need code to read the appropriate amount of
data into a memory buffer.  Because there is no outer length header
for principals and credentials, this code needs to understand the
credential and principal representations, but is much simpler than the
old code as it only needs to store the data it reads, not assemble a
data structure.
Use cred marshalling functions in cc_keyring.c

Replace the cc_keyring.c credential and principal marshalling
functions with calls to the ccmarshal.c functions.  Simplify the
remaining parsing functions (for the index and time offsets) and fold
the resulting code into the calling functions where appropriate.