dd: use more robust SIGUSR1 handling

* src/dd.c (ifd_reopen): A new wrapper to ensure we
don't exit upon receiving a SIGUSR1 in a blocking open()
on a fifo for example.
(iftruncate): Likewise for ftruncate().
(iread): Process signals also after a short read.
(install_signal_handlers): Install SIGINFO/SIGUSR1 handler
even if set to SIG_IGN, as this is what the parent can easily
set from a shell script that can send SIGUSR1 without the
possiblity of inadvertently killing the dd process.
* doc/coreutils.texi (dd invocation): Improve the example to
show robust usage wrt signal races and short reads.
* tests/dd/stats.sh: A new test for various signal races.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
dd: new status=progress level to print stats periodically

* src/dd.c: Report the transfer progress every second when the
new status=progress level is used.  Adjust the handling and
description of the status= option so that they're treated as
mutually exclusive levels, rather than flags with implicit precedence.
* doc/coreutils.texi (dd invocation): Document the new progress
status level.  Reference the new level in the description of SIGUSR1.
* tests/dd/stats.sh: Add new test for status=progress.
* tests/dd/misc.sh: Change so status=none only takes precedence
if it's the last level specified.
* NEWS: Mention the feature.
maint: avoid double semicolon syntax check failure

A syntax-check recently added to gnulib would trigger a failure
(once gnulib gets updated here) for a statement introduced with
commit v8.23-43-gaf2a4ed:

  src/dd.c:806:  char const *time_fmt = _(", %g s, %s/s\n");;
  maint.mk: Double semicolon detected
  make: *** [sc_prohibit_double_semicolon] Error 1

* src/dd.c (print_xfer_stats): s/;;/;/
tests: fix false failure for test referencing libdl

* init.cfg (gcc_shared_): -ldl has to be positioned after the object
files that may rely upon it.  This fixes tests/cp/nfs-removal-race.sh
which references dlsym() from libdl.
doc: document stat's output with the --terse option

* doc/coreutils.texi (stat invocation): Add a paragraph documenting
stat's output format when the --terse option is specified, both in
normal and in --file-system mode.

Reported by Dan Jacobson <jidanni@jidanni.org>
in http://bugs.gnu.org/18624
wc: don't miscount /sys and similar file systems

Fix similar problems in head, od, split, tac, and tail.
Reported by George Shuklin in: http://bugs.gnu.org/18621
* NEWS: Document this.
* src/head.c (elseek): Move up.
(elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg
CURRENT_POS.  All uses changed.
(elide_tail_bytes_file, elide_tail_lines_file):
New arg ST and remove arg SIZE.  All uses changed.
* src/head.c (elide_tail_bytes_file):
* src/od.c (skip): Avoid optimization for /sys files, where
st_size is bogus and st_size == st_blksize.
Don't report error at EOF when not optimizing.
* src/head.c, src/od.c, src/tail.c: Include "stat-size.h".
* src/split.c (input_file_size): New function.
(bytes_split, lines_chunk_split, bytes_chunk_extract): New arg
INITIAL_READ.  All uses changed. Use it to double-check st_size.
* src/tac.c (tac_seekable): New arg FILE_POS.  All uses changed.
(copy_to_temp): Return size of temp file.  All uses changed.
* src/tac.c (tac_seekable):
* src/tail.c (tail_bytes):
* src/wc.c (wc):
Don't trust st_size; double-check by reading.
* src/wc.c (wc): New arg CURRENT_POS.  All uses changed.
* tests/local.mk (all_tests): Add tests/misc/wc-proc.sh,
tests/misc/od-j.sh, tests/tail-2/tail-c.sh.
* tests/misc/head-c.sh:
* tests/misc/tac-2-nonseekable.sh:
* tests/split/b-chunk.sh:
Add tests for problems with /proc and /sys files.
* tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh:
New files.
maint: avoid syntax-check failures in previous patch

* tests/misc/od-j.sh: Non standard comparison order.
* tests/split/b-chunk.sh: Confusing input file name.
* tests/tail-2/tail-c.sh: Redundant require ulimit.
tests: split/b-chunk.sh: avoid spurious fail on non-Linux

* tests/split/b-chunk.sh: Skip each file that does not exist.
Some systems lack /proc/version or /sys/kernel/profiling
maint: avoid new signed overflow warning on 32 bit

Prompted by http://hydra.nixos.org/build/15682577
with GCC 4.8.3 on i686

  src/tac.c:557:6: error: assuming signed overflow does not occur
  when simplifying conditional to constant [-Werror=strict-overflow]
     if (bytes_copied < 0)

This happens because copy_to_temp() is inlined in tac_nonseekable(),
thus reducing the comparison to the bytes_copied variable in
copy_to_temp.  Now this can't overflow on either 32 or 64 bit
due to the protection of the preceding fwrite().  We could use a
guard like "if (bytes_copied <= OFF_T_MAX - bytes_read)" to avoid
the warning, but rather than a runtime branch, just use an unsigned
type to avoid apparent signed overflow on systems where the accumulation
is not promoted to unsigned (32 bit size_t, 64 bit off_t).

* src/tac.c (copy_to_temp): Increment an unsigned type to
avoid the subsequent signed overflow warning.
doc: clarify that timeout limits are not 2038 seconds

* man/timeout.x: The 2038 that the sentence is referring to is the year
2038, not 2038 seconds (the default unit used for timeouts).
copy: detect smaller holes than the copy buffer size

Previously cp would not detect runs of NULs that were
smaller than the buffer size used for I/O (currently 128KiB).

* src/copy.c (copy_reg): Use an independent hole_size, set to
st_blksize, to increase the chances of detecting a representable hole,
in a run of NULs read from the input.
(create_hole): A new function refactored from sparse_copy() and
extent_copy() so we have a single place to handle holes.
(sparse_copy): Adjust to loop over the larger input buffer
in chunks of the passed hole size.  Also adjust to only call
lseek once per hole, rather than at least once per input buffer.
* tests/cp/sparse.sh: Add test cases for various sparse chunk sizes.
* NEWS: Mention the improvement.
cp: avoid speculative preallocation with --sparse=always

With --sparse=always use fallocate(...PUNCH_HOLE...) to
avoid any permanent allocation due to speculative
preallocation employed by file systems such as XFS.

* m4/jm-macros.m4: Check for <linux/falloc.h> and fallocate().
* src/copy.c (punch_hole): A new function to try and punch
a hole at the specified offset if supported.
(create_hole): Call punch_hole() after requesting a hole.
(extent_copy): Likewise.
* NEWS: Mention the improvement.
cp: read sparse files more efficiently with non regular destination

* src.copy.c (copy_reg): Use fiemap to read sparse files, even
if the output is not to a regular file.
* NEWS: Mention the improvement.
copy: avoid an extraneous error when reporting errors

* src/copy.c (copy_reg): If sparse_copy() failed, then an
erroneous error about failing to extend the file would be reported.
chroot: call chroot() unconditionally to handle bind mounted "/"

* src/chroot.c (is_root): Adjust to compare canonicalized paths
rather than inodes, to handle (return false in) the case where
we have a tree that is constructed by first bind mounting "/"
(thus having the same inode).
(main): Unconditionally call chroot() because it's safer
and of minimal performance benefit to avoid in this case.
This will cause inconsistency with some platforms
not allowing `chroot / true` for non root users.
* tests/misc/chroot-fail.sh: Adjust appropriately.
* NEWS: Mention the bug fixes.
Fixes http://bugs.gnu.org/18736
tests: fix test hang with unstable inodes in /proc

* cp/proc-zero-len.sh: Search the 'err' file
for the error to ignore, not stdin.
tests: avoid false failure when comparing /proc files

At least the MHz number in /proc/cpuinfo may change, thus leading to
a false positive failure when comparing the expected against the
actual output file.  Use an invariant file instead: /proc/version.

* tests/misc/head-c.sh: s/cpuinfo/version/
tests: d_type-check: don't hardcode the C library name

* tests/d_type-check: The hardcoded name doesn't hold true for all
Linux/glibc platforms, let alone Linux/non-glibc.
Use ctypes.util.find_library() instead to search for the library.
maint: improve sc_long_lines syntax-check speed

sc_long_lines was the slowest syntax check

before$ time make sc_long_lines
        long_lines
        real    0m2.740s

after $ time make sc_long_lines
        long_lines
        real    0m0.677s

* src/cfg.mk (sc_dd_max_sym_length): s/--max-line-length/-L/
for compat with BSDs.
(sc_long_lines): Prefilter with wc -L to only identify lines
in files that have lines longer than 80 characters.
doc: mention that df -a includes duplicate file systems

* src/df.c (usage): Mention that duplicate file systems are shown
with this option, not just dummy file systems.
* doc/coreutils.texi (df invocation): For the --all option, expand
on the class of normally suppressed mount entries that it includes.

Reported in http://bugs.debian.org/737399