Redmine #5621: Fix failing safe_open in create race.

On single core systems with certain kernel versions (CentOS), the
ordering of open calls is deterministic, and if two threads try to
open the same file with O_CREAT, one will fail because the other
thread creates it at the same moment.

This is actually the very thing that safe_open was designed to protect
against, but since it can happen even in daily operation when there is
no foul play, we need to be a bit more tolerant. So if we find that we
have hit a race condition, instead of failing immediately, retry the
operation to see if we hit the race condition again. Only after a few
attempts do we give up.

This changes the expected result of one of the unit tests from
safe_open returning -1, to returning a file descriptor.
Add an additional test for opening a dangling symlink, not just creating
Fix possible file open race condition on platforms without openat().

When opening a relative path, the first path component opened is the
current directory. However, this is opened using normal open(). This
is fine on most platforms, but on those where we emulate openat(), we
do not protect the open() call using the lock inside generic_at.c,
which means we can potentially be in a different directory at the time
of the call.

The fix is to switch to openat() with the AT_FDCWD flag, so that we
run the emulation code with the lock also for the first file open.
Rename internal test hook to make it clear that it itself isn't a test
Merge pull request #1670 from kacfengine/safe_open_fixes

Redmine #5621: Safe open fixes
Attempt expansion of naked nested scalars in ExpandListEntry

Close #5480
libcfnet/tls_client.c: mention 3.5.x protocol incompatibility
Merge pull request #1671 from tzz/tls-mention-3.5.x

libcfnet/tls_client.c: mention 3.5.x protocol incompatibility
Redmine#5486: disable db_test and db_concurrent_test temporarily
Test db->mc only once and store result on stack.

This reduces the possibility of race conditions in multi threaded
client code, where one thread opens a cursor while another thread
operates on the DB without cursors. In that case, db->mc changes
in the middle of the operation.

This is a quick workaround, the correct fix is to either remove this
optimization or protect the db structure with a mutex.
Merge pull request #1673 from vohi/mitigate_racecondition

Test db->mc only once and store result on stack.
new autoconf option to make state directory configurable: --with-statedir=<directory>
make use of preprocessor macros for static known_dirs.c functions
rename MAX_WORKDIR_LENGTH macro to generic MAX_DIR_LENGTH
add GetStateDir() and supporting functions
use GetStateDir() and replace hardcoded state/ paths
libpromises/generic_agent.c: chmod and chown internal directories only when needed.

Avoids gratuitous chmod and chown calls on internal directories by examining
the populated stat struct after stat(). Important for devices backed by flash
in order to avoid wanton wearing of the storage medium.
new $(sys.statedir) special variable
move embedded databases exclusive to state/ directory
LogTotalCompliance(): prevent cf-agent from bailing with custom logdir