util: improve user lookup helper

https://bugzilla.redhat.com/show_bug.cgi?id=964358

A future patch needs to look up pw_gid; but it is wasteful
to crawl through getpwuid_r twice for two separate pieces
of information, and annoying to copy that much boilerplate
code for doing the crawl.  The current internal-only
virGetUserEnt is also a rather awkward interface; it's easier
to just design it to let callers request multiple pieces of
data as needed from one traversal.

And while at it, I noticed that virGetXDGDirectory could deref
NULL if the getpwuid_r lookup fails.

* src/util/virutil.c (virGetUserEnt): Alter signature.
(virGetUserDirectory, virGetXDGDirectory, virGetUserName): Adjust
callers.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit c1983ba4e3902308054e961fcae75cece73ef4ba)

Conflicts:
	src/util/virutil.c - oom reporting/strdup changes not backported
util: add virGetGroupList

https://bugzilla.redhat.com/show_bug.cgi?id=964358

Since neither getpwuid_r() nor initgroups() are safe to call in
between fork and exec (they obtain a mutex, but if some other
thread in the parent also held the mutex at the time of the fork,
the child will deadlock), we have to split out the functionality
that is unsafe.  At least glibc's initgroups() uses getgrouplist
under the hood, so the ideal split is to expose getgrouplist for
use before a fork.  Gnulib already gives us a nice wrapper via
mgetgroups; we wrap it once more to look up by uid instead of name.

* bootstrap.conf (gnulib_modules): Add mgetgroups.
* src/util/virutil.h (virGetGroupList): New declaration.
* src/util/virutil.c (virGetGroupList): New function.
* src/libvirt_private.syms (virutil.h): Export it.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 75c125641ac73473ba4b0542524d67a184769c8e)

Conflicts:
	bootstrap.conf - not updating gnulib submodule...
	configure.ac - ...so checking for getgrouplist by hand...
	src/util/virutil.c - ...and copying only the getgrouplist implementation rather than calling the gnulib function
util: make virSetUIDGID async-signal-safe

https://bugzilla.redhat.com/show_bug.cgi?id=964358

POSIX states that multi-threaded apps should not use functions
that are not async-signal-safe between fork and exec, yet we
were using getpwuid_r and initgroups.  Although rare, it is
possible to hit deadlock in the child, when it tries to grab
a mutex that was already held by another thread in the parent.
I actually hit this deadlock when testing multiple domains
being started in parallel with a command hook, with the following
backtrace in the child:

 Thread 1 (Thread 0x7fd56bbf2700 (LWP 3212)):
 #0  __lll_lock_wait ()
     at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
 #1  0x00007fd5761e7388 in _L_lock_854 () from /lib64/libpthread.so.0
 #2  0x00007fd5761e7257 in __pthread_mutex_lock (mutex=0x7fd56be00360)
     at pthread_mutex_lock.c:61
 #3  0x00007fd56bbf9fc5 in _nss_files_getpwuid_r (uid=0, result=0x7fd56bbf0c70,
     buffer=0x7fd55c2a65f0 "", buflen=1024, errnop=0x7fd56bbf25b8)
     at nss_files/files-pwd.c:40
 #4  0x00007fd575aeff1d in __getpwuid_r (uid=0, resbuf=0x7fd56bbf0c70,
     buffer=0x7fd55c2a65f0 "", buflen=1024, result=0x7fd56bbf0cb0)
     at ../nss/getXXbyYY_r.c:253
 #5  0x00007fd578aebafc in virSetUIDGID (uid=0, gid=0) at util/virutil.c:1031
 #6  0x00007fd578aebf43 in virSetUIDGIDWithCaps (uid=0, gid=0, capBits=0,
     clearExistingCaps=true) at util/virutil.c:1388
 #7  0x00007fd578a9a20b in virExec (cmd=0x7fd55c231f10) at util/vircommand.c:654
 #8  0x00007fd578a9dfa2 in virCommandRunAsync (cmd=0x7fd55c231f10, pid=0x0)
     at util/vircommand.c:2247
 #9  0x00007fd578a9d74e in virCommandRun (cmd=0x7fd55c231f10, exitstatus=0x0)
     at util/vircommand.c:2100
 #10 0x00007fd56326fde5 in qemuProcessStart (conn=0x7fd53c000df0,
     driver=0x7fd55c0dc4f0, vm=0x7fd54800b100, migrateFrom=0x0, stdin_fd=-1,
     stdin_path=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE,
     flags=1) at qemu/qemu_process.c:3694
 ...

The solution is to split the work of getpwuid_r/initgroups into the
unsafe portions (getgrouplist, called pre-fork) and safe portions
(setgroups, called post-fork).

* src/util/virutil.h (virSetUIDGID, virSetUIDGIDWithCaps): Adjust
signature.
* src/util/virutil.c (virSetUIDGID): Add parameters.
(virSetUIDGIDWithCaps): Adjust clients.
* src/util/vircommand.c (virExec): Likewise.
* src/util/virfile.c (virFileAccessibleAs, virFileOpenForked)
(virDirCreate): Likewise.
* src/security/security_dac.c (virSecurityDACSetProcessLabel):
Likewise.
* src/lxc/lxc_container.c (lxcContainerSetID): Likewise.
* configure.ac (AC_CHECK_FUNCS_ONCE): Check for setgroups, not
initgroups.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit ee777e994927ed5f2d427fbc5a53cbe8b5969bda)

Conflicts:
	src/lxc/lxc_container.c - did not use setUIDGID before 1.1.0
	src/util/virutil.c - oom handling changes not backported
	src/util/virfile.c - functions still lived in virutil.c this far back
	configure.ac - context with previous commit
security: framework for driver PreFork handler

https://bugzilla.redhat.com/show_bug.cgi?id=964358

A future patch wants the DAC security manager to be able to safely
get the supplemental group list for a given uid, but at the time
of a fork rather than during initialization so as to pick up on
live changes to the system's group database.  This patch adds the
framework, including the possibility of a pre-fork callback
failing.

For now, any driver that implements a prefork callback must be
robust against the possibility of being part of a security stack
where a later element in the chain fails prefork.  This means
that drivers cannot do any action that requires a call to postfork
for proper cleanup (no grabbing a mutex, for example).  If this
is too prohibitive in the future, we would have to switch to a
transactioning sequence, where each driver has (up to) 3 callbacks:
PreForkPrepare, PreForkCommit, and PreForkAbort, to either clean
up or commit changes made during prepare.

* src/security/security_driver.h (virSecurityDriverPreFork): New
callback.
* src/security/security_manager.h (virSecurityManagerPreFork):
Change signature.
* src/security/security_manager.c (virSecurityManagerPreFork):
Optionally call into driver, and allow returning failure.
* src/security/security_stack.c (virSecurityDriverStack):
Wrap the handler for the stack driver.
* src/qemu/qemu_process.c (qemuProcessStart): Adjust caller.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit fdb3bde31ccf8ff172abf00ef5aa974b87af2794)
security_dac: compute supplemental groups before fork

https://bugzilla.redhat.com/show_bug.cgi?id=964358

Commit 75c1256 states that virGetGroupList must not be called
between fork and exec, then commit ee777e99 promptly violated
that for lxc's use of virSecurityManagerSetProcessLabel.  Hoist
the supplemental group detection to the time that the security
manager needs to fork.  Qemu is safe, as it uses
virSecurityManagerSetChildProcessLabel which in turn uses
virCommand to determine supplemental groups.

This does not fix the fact that virSecurityManagerSetProcessLabel
calls virSecurityDACParseIds calls parseIds which eventually
calls getpwnam_r, which also violates fork/exec async-signal-safe
safety rules, but so far no one has complained of hitting
deadlock in that case.

* src/security/security_dac.c (_virSecurityDACData): Track groups
in private data.
(virSecurityDACPreFork): New function, to set them.
(virSecurityDACClose): Clean up new fields.
(virSecurityDACGetIds): Alter signature.
(virSecurityDACSetSecurityHostdevLabelHelper)
(virSecurityDACSetChardevLabel, virSecurityDACSetProcessLabel)
(virSecurityDACSetChildProcessLabel): Update callers.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 29fe5d745fbe207ec2415441d4807ae76be05974)

Conflicts:
	src/security/security_dac.c - virSecurityDACSetSecurityUSBLabel needed similar treatment
security: fix deadlock with prefork

https://bugzilla.redhat.com/show_bug.cgi?id=964358

Attempts to start a domain with both SELinux and DAC security
modules loaded will deadlock; latent problem introduced in commit
fdb3bde and exposed in commit 29fe5d7.  Basically, when recursing
into the security manager for other driver's prefork, we have to
undo the asymmetric lock taken at the manager level.

Reported by Jiri Denemark, with diagnosis help from Dan Berrange.

* src/security/security_stack.c (virSecurityStackPreFork): Undo
extra lock grabbed during recursion.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit bfc183c1e377b24cebf5cede4c00f3dc0d1b3486)
lxc: correctly backport /dev/tty fix

https://bugzilla.redhat.com/show_bug.cgi?id=982317

maint-only patch; reported by Geert Jansen

Commit 17cdc298 tried to backport upstream 90a0c6d, but in
resolving conflicts, failed to account that upstream commit
e1d32bb refactored code to leave off a leading /dev.

* src/lxc/lxc_container.c (lxcContainerPopulateDevices): Use
correct device name.

Signed-off-by: Eric Blake <eblake@redhat.com>
maint: update to latest gnulib

Among others, this fixes a cosmetic bug where bootstrap stated:

./bootstrap: Bootstrapping from checked-out http://libvirt.org sources...

instead of the intended:

./bootstrap: Bootstrapping from checked-out libvirt sources...

* .gnulib: Update to latest, for bootstrap improvement.
* bootstrap: Resync from gnulib.

(cherry picked from commit 3dfc2b71aa070dad6e2a460705426031e77889e2)
build: update to latest gnulib, for syntax-check

This picks up a fix for a syntax-check weakness mentioned here:
https://www.redhat.com/archives/libvir-list/2013-May/msg00811.html

* .gnulib: Update to latest, for maint.mk improvement.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 12bd22c70d34dda461d090620037e109d5d48b21)
FreeBSD: disable buggy -fstack-protector-all

FreeBSD ships an old gcc 4.2.1 which generates
bogus code, e.g. getsockopt() call returns
struct xucred with bogus values, which doesn't even
allow to connect to libvirtd:

error: Failed to find group record for gid '1284660778': No error: 0

So roll back to just -fstack-protector on FreeBSD.

(cherry picked from commit cc7cd6232ebfbf1a0e4edd51d9f5ca43a73eda10)
maint: update to latest gnulib

The latest mingw headers on Fedora 19 fail to build with gnulib
without an update.

Meanwhile, now that upstream gnulib has better handling of -W
probing for clang, we can drop some of our own solutions in
favor of upstream; thus this reverts commit c1634100, "Correctly
detect warning flags with clang".

* .gnulib: Update to latest, for mingw and clang.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit cdd703f4758facf37bda891deacd561816926e75)
build: honor autogen.sh --no-git

Based on a report by Chandrashekar Shastri, at
https://bugzilla.redhat.com/show_bug.cgi?id=979360

On systems where git cannot access the outside world, a developer
can instead arrange to get a copy of gnulib at the right commit
via side channels (such as NFS share drives), set GNULIB_SRCDIR,
then use ./autogen.sh --no-git.  In this setup, we will now
avoid direct use of git.  Of course, this means no automatic
gnulib updates when libvirt.git updates its submodule, but it
is expected that any developer in such a situation is already
prepared to deal with the fallout.

* .gnulib: Update to latest, for bootstrap.
* bootstrap: Synchronize from gnulib.
* autogen.sh (no_git): Avoid git when requested.
* cfg.mk (_update_required): Skip automatic rerun of bootstrap if
we can't use git.
* docs/compiling.html.in: Document this setup.
* docs/hacking.html.in: Mention this.
* HACKING: Regenerate.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 1e503ee534af166d8bbcdd9857fa5946449634b6)
maint: update to latest gnulib

Future patches need LGPLv2+ versions of some modules that had
recent license changes; but separating the gnulib update from
the actual use of the modules makes it easier to backport to
an older version while avoiding a submodule update (assuming,
of course, that the backport is to a system where glibc provides
adequate functionaliy without needing the gnulib module).

* .gnulib: Update to latest, for modules needed in later patches.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 7961ad21077684d758fe32baa09bac5316840ea7)
maint: update to latest gnulib

Upstream gnulib recently patched a bug in bootstrap, for projects
that use a different name than build-aux for a subdirectory.  We
don't, but it doesn't hurt to update.

* .gnulib: Update, for bootstrap fix.
* bootstrap: Sync to upstream.
* bootstrap.conf: Match upstream bug fix.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit ac0852c72a872be0abca855f29c4c9fa98747de9)
Fix build with clang

Partially revert cdd703f's revert of c163410, as linking with clang
with --param=ssp-buffer-size=4 still fails with:
"argument unused during compilation".

(cherry picked from commit 4b91dc24d1ed6a5ec6d5d9ab0d8522375dd77f3a)
build: avoid build failure without gnutls

Found while trying to cross-compile to mingw:

  CC       libvirt_driver_remote_la-remote_driver.lo
../../src/remote/remote_driver.c: In function 'doRemoteOpen':
../../src/remote/remote_driver.c:487:23: error: variable 'verify' set but not used [-Werror=unused-but-set-variable]

* src/remote/remote_driver.c (doRemoteOpen): Also ignore 'verify'.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 4e6a78e712fc1d649f4e730815f906bf8e14f8a2)
build: configure must not affect tarball contents

On mingw, configure sets the name of the lxc symfile to
libvirt_lxc.defs rather than libvirt_lxc.syms.  But tarballs
must be arch-independent, regardless of the configure options
used for the tree where we ran 'make dist'.  This led to the
following failure in autobuild.sh:

  CCLD     libvirt-lxc.la
  CCLD     libvirt-qemu.la
/usr/lib64/gcc/i686-w64-mingw32/4.7.2/../../../../i686-w64-mingw32/bin/ld: cannot find libvirt_lxc.def: No such file or directory
collect2: error: ld returned 1 exit status
make[3]: *** [libvirt-lxc.la] Error 1
make[3]: *** Waiting for unfinished jobs....

We were already doing the right thing with libvirt_qemu.syms.

* src/Makefile.am (EXTRA_DIST): Don't ship a built file which
depends on configure for its final name.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit d79c9273b0724dfd39eaf13bef2953fcea5b8fa4)
build: work around mingw header pollution

On Fedora 18, when cross-compiling to mingw with the mingw*-dbus
packages installed, compilation fails with:

  CC       libvirt_net_rpc_server_la-virnetserver.lo
In file included from /usr/i686-w64-mingw32/sys-root/mingw/include/dbus-1.0/dbus/dbus-connection.h:32:0,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/dbus-1.0/dbus/dbus-bus.h:30,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/dbus-1.0/dbus/dbus.h:31,
                 from ../../src/util/virdbus.h:26,
                 from ../../src/rpc/virnetserver.c:39:
/usr/i686-w64-mingw32/sys-root/mingw/include/dbus-1.0/dbus/dbus-message.h:74:58: error: expected ';', ',' or ')' before 'struct'

I have reported this as a bug against two packages:
- mingw-headers, for polluting the namespace
https://bugzilla.redhat.com/show_bug.cgi?id=980270
- dbus, for not dealing with the pollution
https://bugzilla.redhat.com/show_bug.cgi?id=980278

At least dbus has agreed that a future version of dbus headers will
do s/interface/iface/, regardless of what happens in mingw. But it
is also easy to workaround in libvirt in the meantime, without having
to wait for either mingw or dbus to upgrade.

* src/util/virdbus.h (includes): Undo mingw's pollution so that
dbus doesn't fail.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 1528e8b23a0c0014074b06772368e00a17020a79)
build: fix virutil build on mingw

https://bugzilla.redhat.com/show_bug.cgi?id=964358

On platforms without decent group support, the build failed:

Cannot export virGetGroupList: symbol not defined
./.libs/libvirt_security_manager.a(libvirt_security_manager_la-security_dac.o): In function `virSecurityDACPreFork':
/home/eblake/libvirt-tmp/build/src/../../src/security/security_dac.c:248: undefined reference to `virGetGroupList'
collect2: error: ld returned 1 exit status

* src/util/virutil.c (virGetGroupList): Provide dummy implementation.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit cd725c7a9350c8331a6f915127cf7007706f9926)
examples: fix mingw build vs. printf

Mingw *printf is a moving target; newer mingw now provides a version
of asprintf() that fails to understand %lld:

  CC       event_test-event-test.o
../../../../examples/domain-events/events-c/event-test.c: In function 'myDomainEventRTCChangeCallback':
../../../../examples/domain-events/events-c/event-test.c:270:18: error: unknown conversion type character 'l' in format [-Werror=format=]
                  virDomainGetID(dom), offset) < 0)
                  ^

But since our examples already admitted that they were hacking around
a mingw deficiency, it is easier to just use printf() directly, coupled
with <inttypes.h> macros, for a more portable work-around.

* examples/domain-events/events-c/event-test.c
(myDomainEventRTCChangeCallback): Use PRIdMAX instead of asprintf.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 6f4458a017789535ee4a62fc6b5b846dec0499c9)