Increase size of buffer in network timeout tests

Linux 3.8 has increased the amount of network traffic it's willing
to buffer, so increase the size that we try to send to force a
timeout from 512KB to 4MB.

Change-Id: Id7287cca9d0713991717f1fe49035f05c2b0d39a
Reviewed-on: https://gerrit.stanford.edu/1199
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Make Ruby extension builds verbose

We want the verbose logging for automated builds, such as Debian
package builds (which do compiler and linker flag analysis).  We
could do something more complex based on the setting of verbosity
for the main package, but that's not trivial and it's not very
important to suppress those two lines.  PHP and Perl are much more
verbose anyway.

Change-Id: I15d4590ea03a93f017579a47769492c092e0a290
Reviewed-on: https://gerrit.stanford.edu/1200
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Copy Test::RRA::* modules to perl directory earlier

The Test::RRA::* modules were only copied as part of make check,
which resulted in warnings from Module::Build about the modules
missing from the manifest.  Copy them before the build to avoid
that and be sure to clean them on make clean.  Copy from srcdir
explicitly.

Change-Id: I65620451c48592dadb4676015486bc927b3066ce
Reviewed-on: https://gerrit.stanford.edu/1201
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Redo the Perl build system integration

Pass in all the appropriate flags from the main build to the Perl
build so that we don't need Autoconf substitutions in Build.PL.
Redo Build.PL so that it doesn't assume in-tree builds by default
and can therefore be used in a standalone module distribution with
a system libremctl library.

Change-Id: I00780c225242fcba018ab2af76ebe7c292131b0a
Reviewed-on: https://gerrit.stanford.edu/1202
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Fix compilation on systems without gss_krb5_import_cred

The code that actually called gss_krb5_import_cred wasn't conditional
and the compiler wasn't smart enough to discard it.  Provide a stub
of the relevant open routine for systems that don't have the function.

Change-Id: I93af1fbafe6fee192e44fb29cf65d06b738e5897
Reviewed-on: https://gerrit.stanford.edu/1206
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Fix race condition in remctld with output before process end

The backend being run is a shell script that does the following:

        cat /root/tmp/databases/tw.db
        rm -rf /root/tmp/databases
        exit 0

remctld doesn't wait for the backend program to close its standard output
and standard error since, in some cases (such as when the backend program
spawns a daemon via an init script), standard output and standard error
are never closed.  Instead, it waits until the child process exits, and
then considers the command finished.

The problem is that the operating system buffers pipes.  In the above, cat
exits as soon as it's successfully written all of the file to its output
pipe.  However, that success from the operating system just indicates that
all of the output has been buffered.  With newer versions of Linux, that
buffer can be quite large.  It then goes on to run rm and exit.

In the remctld process, on each trip through the select loop, it polls to
see if the child process has exited.  If so, it attempts one more time to
read output data from the process with a timeout of 0, and then considers
the command complete.

The problem is that the remctld server process reads data in chunks of
64000 bytes.  If the file is larger, say 100KB, a modern version of Linux
will buffer the entire file contents in the pipe buffer and cat will
essentially exit as soon as the file contents have been copied into kernel
space (which is extremely fast since it's just a move of pages from the
disk cache to the pipe cache).  It's then possible, if remctld loses the
race, for the rm and exit to finish and for remctld to be signaled before
remctld reads any of the data.  remctld will then observe the child
process has exited, read one more block of 64000 bytes, and then leave the
rest of the data unread in the pipe buffer, resulting in truncated
output.  Usually only a small amount of data in excess of a multiple of
64000 at the end of the file is truncated.

The fix is to reduce the timeout to 0 but continue looping on select with
a timeout of 0 (indicating a poll of the file descriptors) until no
further output is immediately available for reading, and only then
consider the command complete.

Change-Id: I38f77a8a9b9d9938ef0521a3df73ea75399cd0bf
Reviewed-on: https://gerrit.stanford.edu/1207
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Add a regression test for truncation of large output

Add a new backend program that prints out large amounts of output
and add to server/streaming a regression test to ensure that we
get the proper amount of output even when the backend exits
immediately after sending its last output.

Change-Id: I00262b9b31345b5d0f886d1682c50825958a5a67
Reviewed-on: https://gerrit.stanford.edu/1208
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Add another stopword for Test::RRA::Config

Change-Id: I6bc5b12f3fc2351f8c08dce07014bd8f1ff3fa2e
Reviewed-on: https://gerrit.stanford.edu/1209
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>
Add NEWS entry for the remctld race condition fix

Change-Id: If557dcb6346ec7c6ee3c30fcad7b0b59731cc558
Reviewed-on: https://gerrit.stanford.edu/1210
Reviewed-by: Russ Allbery <rra@stanford.edu>
Tested-by: Russ Allbery <rra@stanford.edu>