Simplify iprop update locking and avoid deadlock

Since we are no longer treating the update log like a journal (#7552),
we don't need two-stage update logging.  In kdb5.c, add an update log
entry after each DB change in one step, without getting an explicit
lock.  In kdb_log.c, combine ulog_add_update with ulog_finish_update,
and make ulog_add_update lock the ulog internally.

This change avoids deadlock by removing the only cases where the ulog
is locked before the DB.

ticket: 7861
Factor out ulog serial number status check

Add a new function ulog_get_sno_status, which checks a serial number
and timestamp against the ulog for currency.  Use it in kdb5_util dump
and in ulog_get_entries.  Adjust parse_iprop_header's contract in
dump.c to better match the ulog_get_sno_status contract.

This change causes some minor behavior differences.  kadmind will
check for an empty ulog unless the last serial number matches exactly,
and will never set lastentry when returning UPDATE_FULL_RESYNC_NEEDED
(which was pointless).  kdb5_util dump will recognize a dump file as
current if it exactly matches the last serial number, even if the ulog
is empty; it will be more robust in the presence of non-monotonic
clocks; and it will properly lock around the ulog access.
Lock around more ulog operations

Always lock the ulog when accessing it.  We can currently get away
with some laxness on iprop slaves because they are mostly synchronous,
but hierarchical iprop will allow master and slave operations to take
place concurrently, requiring more strict locking.

Add new functions ulog_get_last and ulog_set_last, which access the
ulog header with locking, and use them in kdb5_util and kpropd.  Add
locking to ulog_replay and ulog_init_header.

ulog_lock and ulog_sync_header are no longer used outside of kdb_log.c
after these changes, so make them static functions and remove the
ulog_ prefix.  Add an unlock_ulog function for clarity.
Simplify ulog_map

Get rid of the caller parameter.  The kproplog semantics (without -R)
for mapping the ulog are simple and almost completely different from
other users of the ulog, so implement them as a static helper in
kproplog.  With hierarchical iprop, kpropd will need the same
semantics as FKCOMMAND and FKADMIND, which were already identical.

Get rid of the db_args parameter, since ulog_map no longer opens the
database after #7552.

Remove an inoperative lseek() call when creating a new ulog file.
Rename ulog_filesize to filesize and compute it from scratch each time
we use it, for easier analysis.  If kdb_hmagic is zero, init the ulog
header but don't skip the rest of the function; it's possible that we
need to expand the ulog file.  Remove an unneeded conditional before
calling extend_file_to for an existing ulog.

ticket: 7855
Maintain complete ulog on iprop slaves

Factor out most of ulog_add_update into a helper function named
store_update, and make ulog_add_update just responsible for assigning
a serial number and timestamp to the update before storing it.  In
ulog_replay, use store_update and ulog_finish_update to add each
update to the ulog in addition to replaying it to the database.  Don't
use incr_ret->lastentry to set kdb_last_sno/kdb_last_time, since it
will have been set properly by adding the individual updates; instead,
just reinitialize the ulog on error.

Slave ulogs use serial numbers provided from upstream, and thus do not
always begin at serial number 1 after a header reset.  As a result, we
must: (A) in store_update, detect the first update (for which we must
assign kdb_first_sno/kdb_first_time) by comparing kdb_num to 0,
instead of by comparing the serial number to 1; (B) in store_update,
detect that we are overwriting the first update by comparing kdb_num
to ulogentries, instead of comparing the serial number to ulogentries;
and (C) in ulog_map, detect that ulogentries changed by verifying the
first and last serial number and timestamp against the actual ulog
entries, rather than simply comparing kdb_last_sno to kdb_num.

Based on code submitted by Richard Basch.

ticket: 7855
Implement kadmind -proponly

The -proponly option causes kadmind to only service the iprop service,
not the kpasswd or kadmin services.  An intermediate slave in a
hierarchical iprop setup runs kadmind -proponly in order to provide
incremental updates to downstream slaves.

Based on code submitted by Richard Basch.

ticket: 7855
Implement kpropd -A

The -A option causes kpropd to contact a specified admin server
(normally an intermediate slave running kadmind -proponly) instead of
the master admin server.

Based on code submitted by Richard Basch.

ticket: 7855
Test iprop slave ulog management

Check the ulog more thoroughly after each operation, including the
principal names we expect in each update entry.  Verify that the slave
ulog contains actual update entries received from master.  Add a
second slave which receives updates from the first.  Test a wider
variety of principal operations.  Add two additional operations after
the full resync to test that incremental updates resume after a full
resync (albeit with some lag).

ticket: 7855
Document hierarchical iprop

Also remove an outdated caveat, but add a new one about policy changes
causing full resyncs.

ticket: 7855
Update kadmin set_string document

Added 'otp' as an acceptable attribute name.  Added examples.
Also, x-ref kadmin document from otp.rst.
Avoid duplicate "/etc/krb5.conf" in profile path

If configure gets run with --sysconfdir=/etc, "/etc/krb5.conf" shows
up twice in the profile path, which causes its contents to be read
twice.  This can cause some confusing and possibly problematic
behavior.

Add some logic to configure.in to avoid adding the duplicate entry for
"/etc/krb5.conf".

Reported independently by Denis Vlasenko and Fredrik Tolf.

ticket: 3277
tags: pullup
target_version: 1.12.2
Minimize race conditions in t_iprop.py

Augment kpropd's debugging output so we can see the starting and
ending serial numbers of a sync, and check them in t_iprop.py's
wait_for_prop to make sure we're observing the correct sync each time.
Try to make sure we only send SIGUSR1 when kpropd is actually
sleeping, by waiting for it to print its pre-sleep message to stderr
after a full dump and after each sync.  This isn't perfect, as there
is a short window between printing to stderr and actually sleeping,
but we can't do any better until kpropd has a more reliable mechanism
for triggering immediate updates.
Assume <stdint.h> and fixed-width types

Make unconditional use of <stdint.h> and fixed-width types such as
uint32_t.  k5-plugin.h doesn't use any special integer types, so
remove the conditional include block there.  Nothing uses
INT64_FMT/UINT64_FMT, so leave those out of k5-platform.h for now.
Get rid of builtin AES uitypes.h

Remove uitypes.h and just include stdint.h; all we need from it is
uint{8,16,32}_t.
Stop generating gssapi_krb5.h

We started generating gssapi_krb5.h from gssapi_krb5.hin when we
needed to use a 64-bit type for lucid contexts.  Since we can now
assume a standard name for 64-bit types, we can stop generating the
header.
Eliminate internal fixed-width type wrappers

Directly use stdint.h names for integer types in preference to the
various internal names we have made up for them.
In kdb5_util dump, only lock DB for iprop dumps

Revert #7384, as there are no longer policy refcounts.  For iprop
dumps we want to make sure that the reported serial number matches the
DB state (although we could perhaps relax that requirement with enough
analysis), but for non-iprop dumps we don't need any transactional
guarantees.

Also use the correct constant name for the locking mode (the numeric
value is the same, fortunately), and only unlock the database if we
successfully locked it.

ticket: 7869 (new)
Fix two omissions in stdint.h changes

Since we no longer prefix an "#include <stdint.h>" in the gssapi.h
preamble at build time, include it in gssapi.hin.

Update util/gss/kernel-lib/Makefile.in to account for gssapi_krb5.h
being in the source tree.
Missing type refs in docs due to stdint.h changes

References to krb5_int16.rst and krb5_ui_2.rst were missing, but
recently began causing sphinx-build errors because Doxygen started
generating them after the stdint.h changes.
Correct docs for "kdb5_util load" switches

Remove obsolete parenthetical statement in the description of the
"-update" switch to "kdb5_util load".  Using the "-update" flag to
load a second dump made using the "-ov" switch is no longer necessary
now that the "-r13" format is the default.  Add statement that the
"-b7" dump format was the default for versions prior to 1.2.2 (to
match the statement from "dump" section).

[tlyu@mit.edu: commit message]

ticket: 7865
tags: pullup
target_version: 1.12.2