base64: improve encoding I/O efficiency

Since the I/O overhead is significant to the relatively
simple processing done by this utility, use fputs() rather
than fputc() to output '\n'.
Time to process a 100MiB file was measured to
decrease from 0.417s to 0.383s, i.e. an 8% improvement.

Related to these changes, is a processing improvement in
gnulib, which increases throughput by 60% when processing
full buffers, which improves processing of a 100MiB file
with standard wrapped output, down to 0.256s.
http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commit;h=43fd1e7b

Also increase the encoding buffer size from 3 to 30KiB.
This was seen to give a further 8% improvement, taking
processing time down to 0.235s in the wrapped output case.
The decoding size buffer is not adjusted,
due to the noted caveat with --ignore-garbage.

* src/base64.c (BLOCKSIZE): Split into ENC_ and DEC_ variants,
with the former increased from 3KiB to 30KiB.
(wrap_write): Use the simpler fputc() rather than fputs()
to output the '\n' character.  Also check against EOF
rather than < 0 for errors.
(do_encode): Likewise.
* NEWS: Mention the large increase in performance, which
with the I/O improvements in coreutils and the processing
improvement in gnulib, amount to about a 60% throughput increase.
stty: add support for mark/space parity

This adds support for using a constant or "stick" parity bit.

* src/stty.c (usage): Mention the new flag.
* tests/misc/stty.sh: Adjust for the new flag.
* NEWS: Mention the improvement.
* docs/coreutils.texi (stty invocation): Mention the new flag.
tests: fix false rm -I test failure when run as root

Fix a recent regression introduced in commit v8.21-127-g5ee7d8f

Also related to this is the recent query about root run `rm -I`
ignoring the mode bits of a file: https://bugzilla.redhat.com/1013171

* tests/rm/interactive-once.sh: Avoid the messages and
corresponding file presence checks with write protected files
when running as root.
doc: enhance diagnostic when rm skips "." or ".." arguments

The error diagnostic
  "rm: cannot remove directory: '.'"
does not give the user a hint for the reason.
Issue a clearer error message.

* src/remove.c (rm_fts): Enhance the error diagnostic in the above
case to emphasize that skipping is done deliberately.
In the corresponding comment, mention that POSIX mandates this
behavior.  Likewise in the subsequent comment for skipping "/".
* doc/coreutils.texi (rm invocation): In the paragraph describing
the above behavior, mention that POSIX mandates it.
maint: avoid perl warning in sc_check-AUTHORS

With newer perl, "make syntax-check" issues many warnings like:
  -i used with no filenames on the command line, reading from STDIN.

* cfg.mk (sc_check-AUTHORS): Remove the -i flag in the perl
invocation as it is reading from a pipe.
tests: enhance rm test regarding "." and ".."

Recent commit 2da7009d changed the error diagnostic of rm(1) trying
to remove "." or "..".  Enhance the corresponding test.

* tests/rm/r-4.sh: Ensure rm(1) outputs the expected error diagnostic.
dircolors: add a new entry to colorize 'lz4' files

* src/dircolors.hin: Add entry for the speed optimized 'lz4' compressor.
doc: clarify the operation of ls -k

* src/ls.c (usage): Mention -k only changes the display
for disk usage (directory total, and ls -s), and imply
that it can be overridden (by --block-size, and -h).
* doc/coreutils.texi (block size): Mention that ls -k
handling is different to other utilities.
Addresses http://bugs.gnu.org/14525
sort: avoid issues when issuing diagnostics from child processes

* src/sort.c: (async_safe_die): A new limited version of error(),
that outputs fixed strings and unconverted errnos to stderr.
This is safe to call in the limited context of a signal handler,
or in this particular case, between the fork() and exec() of
a multithreaded process.
(move_fd_or_die): Use the async_safe_die() rather than error().
(maybe_create_temp): Likewise.
(open_temp): Likewise.
Fixes http://bugs.gnu.org/15970
tests: ensure factor tests can be regenerated

* tests/local.mk (factor-tests:) Add -f to the mv command that
replaces any existing generated tests.  This is required to avoid
prompts when root initially generates the tests, and they subsequently
need to be regenerated by a non root user.
timeout: avoid unlikely issues with --kill-after

* src/timeout.c (cleanup): When calling settimeout() from
this signal handler, ensure we don't call out to error()
or gettext(), which are not async-signal-safe.
Also reset the errno which may be cleared by settimeout().
df: add --output=file to directly output specified arguments

* src/df.c (usage): Document the new 'file' --output field.
(get_dev): Add a new parameter to pass the specified
argument from the command line through.  Use '-' if a
command line parameter is not being used.
* doc/coreutils.texi (df invocation): Describe the new 'file' field.
* tests/df/df-output.sh: Adjust all fields test, and
add a specific test for --output=file.
* NEWS: Mention the new feature.
tail: improve inotify handling of symlinks

Previous behavior failed to read contents of a (re)appearing file,
when symlinked by tail's watched file.  Also we now diagnose other
edge cases when running in inotify mode, where an initially
missing or regular file changes to a symlink.

* src/tail.c (main): If any arg is a symlink, use polling mode.
(recheck): Diagnose the edge case where a symlink appears during
inotify processing.
* tests/tail-2/symlink.sh: Test the fix. Mention the edge cases.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Reported by: Ondrej Oprala
shred: provide --remove methods to avoid excessive syncing

A sync operation is very often expensive.  For illustration
I timed the following python script which indicated that
each ext4 dir sync was taking about 2ms and 12ms, on an
SSD and traditional disk respectively.

  import os
  d=os.open(".", os.O_DIRECTORY|os.O_RDONLY)
  for i in range(1000):
     os.fdatasync(d)

So syncing for each character for each file can result
in significant delays.  Often this overhead is redundant,
as only the data is sensitive and not the file name.
Even if the names are sensitive, your file system may
employ synchronous metadata updates, which also makes
explicit syncing redundant.

* tests/misc/shred-remove.sh: Ensure all the new parameters
actually unlink the file.
* doc/coreutils.texi (shred invocation): Describe the new
parameters to the --remove option.
* src/shred.c (Usage): Likewise.
(main): Parse the new options.
(wipename): Inspect the new enum to see which of
the now optional tasks to perform.
* NEWS: Mention the new feature.
* THANKS.in: Add reporter Joseph D. Wagner
selinux: a new module implementing "restorecon" functionality

* src/selinux.c: A new module implementing "restorecon" functionality.
There are two main functions to adjust the type of the referenced
file system item.  defaultcon() will setup the process context so
that new items will have the required context without races.  This is
the preferred method.  For existing files, the equivalent restorecon()
is available which has two modes.  With the "local" parameter set to
false, restorecon() will adjust the type according to the system
configuration for that file, and set to true will update the context
as per the context for the current process (disregarding type).
* src/selinux.h: Likewise.
* po/POTFILES.in: Reference the new module.
selinux: adjust utils to run restorecon with -Z

cp, mv, install, mkdir, mkfifo, mknod are adjusted so that:
 -Z no longer accepts an argument.
 -Z or --context without an argument do not warn without SELinux.
 --context with an argument will warn without SELinux.

* src/local.mk: Reference the new selinux module where required.
* src/system.h: Make the argument to --context optional.
* src/mkdir.c: Likewise.  Also handle the SMACK case for --context.
Note we currently silently ignore -Z with SMACK.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/install.c: Likewise.  Note install(1) by default already
set the context for target files to their system default,
albeit with an older method.  Use the -Z option to select between
the old and new context restoration behavior, and document
the differences and details for how context restoration
is done in new and old methods, with a view disabling the
old method entirely in future.
* src/cp.c: Make the argument to --context optional.
Note -Z implies --no-preserve=context.  I.E. -Z overrides
that aspect of -a no matter what order specified.
(struct cp_options): Document the context handling options.
(main): Check/adjust option combinations after all
options are processed, to both simplify processing
and to make handling independent of order of options
on the command line.  Also improve the diagnostics
from a failed call to setfscreatecon().
(set_process_security_ctx): A new function,
refactored to set the default context from the source file,
or with the type adjusted as per the system default for
the destination path.
(set_file_security_ctx): A new function refactored to
set the security context of an existing file, either based on
the process context or the default system context for a path.
(copy_internal): Use the refactored functions to simplify
error handling and consistently fail or warn as needed.
(copy_reg): Likewise.
(copy_internal): With --preserve=context, also copy
context from non regular files.  Note for directories this may
impact the copying of subsequent files to that directory?
(copy_attr): If we're handling SELinux explicitly,
then exclude to avoid the redudant copy with --preserve=context,
and the problematic copy with -Z.  Note SELinux attribute exclusion
also now honors cp -a --no-preserve=context.  Note there was a
very small window over 10 years ago, where attr_copy_file was
available, while attr_copy_check_permissions was not, so we
don't bother adding an explicit m4 check for the latter function.
* src/mv.c: Support --context, but don't allow specifying an argument.
* src/chcon.c: Adjust a comment to be specific to SELinux.
* src/runcon.c: Likewise.
* src/copy.c: Honor the context settings to "restorecon" as appropriate.
* src/copy.h: Add a new setting to select "restorecon" functionality.
* tests/mkdir/selinux.sh: s/-Z/--context=/
* tests/cp/cp-a-selinux.sh: Augment this test with cases
testing basic -Z functionality, and also test the various
invalid option combinations and option precedence.
* tests/mkdir/restorecon.sh: Add a new test for the
more involved mkdir -Z handling, since the directory changing
and non existent directories need to be specially handled.
Also check the similar but simpler handling of -Z by mk{nod,fifo}.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (cp invocation): Update as per interface changes.
(mv invocation): Likewise.
(install invocation): Likewise.
(mkfifo invocation): Likewise.
(mknod invocation): Likewise.
(mkdir invocation): Likewise.
* NEWS: Mention the new feature and change in behavior.
build: avoid build failure on non selinux systems

* src/selinux.c: This module introduced in commit v8.21-159-gd8e27ab
doesn't need to include <selinux/flask.h>.  That header file
isn't catered for by gnulib, but is not needed as we're not
explicitly referencing any class IDs.
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/6920020
build: avoid more selinux build failures

Handle both newer selinux libraries with mode_to_security_class(),
and systems without selinux at all.  We could easily adjust
gnulib to provide the necessary stubs for use by this module,
but it's more efficient to just stub out the module completely,
when not using selinux.

* src/selinux.h: Define stubs for the two module functions,
when SELinux is not available.
* src/selinux.c: Exclude all logic in preference for the stubs
when selinux isn't used.  Also when newer selinux libs are used,
don't use our conflicting static version of mode_to_security_class().
m4/jm-macros.m4: Check for the system mode_to_security_class().
build: fix selinux.c build failure on 32 bit

* src/selinux.c: Don't include the system "fts.h" as
that disallows _FILE_OFFSET_BITS=64 which gnulib auto enables
to support large files on 32 bit systems.  Instead include
our "xfts.h" which includes the less limited gnulib replacement,
and also a checked version of xfts_open().
(restorecon): Use the checked xfts_open() rather than the standard
fts_open().
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/6934169