lisp: better support of VPATH builds

Fixes automake bug#11806.

* lib/am/lisp.am: Pass the value of '$(abs_srcdir)' to the
elisp-compile script in the environment.
* lib/elisp-comp: Add the vale of '$abs_srcdir' to the emacs
load-path.
* t/lisp-pr11806.sh: New test.
* t/list-of-tests.mk: Add it.

Copyright-paperwork-exempt: yes
Co-authored-by: Stefano Lattarini <stefano.lattarini@gnu.org>
Reported-by: Makoto Fujiwara <makoto@ki.nu>
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
coverage: test that AM_PROG_MKDIR_P and $(mkdir_p) still works

They are deprecated, but should continue to work in the 1.12.x
release series.  Report from Benoit Sigoure and Diego Elio Pattenò:
<http://lists.gnu.org/archive/html/automake/2012-07/msg00001.html>

* t/mkdirp-deprecation.sh: Enhance.
* t/mkdir_p.sh: New test, check that AM_INIT_AUTOMAKE still defines
the $(mkdir_p) make variable.  Currently xfailing.
* t/list-of-tests.mk (handwritten_TESTS, XFAIL_TESTS): Add the new
test.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
compat: automake should define $(mkdir_p), for backward compatibility

That has been unwittingly broken by commit 'v1.12-19-g7a1eb9f'
of 2012-04-28, "AM_PROG_MKDIR_P: deprecate, to be removed in
Automake 1.13".

Report from Benoit Sigoure and Diego Elio Pattenò:
<http://lists.gnu.org/archive/html/automake/2012-07/msg00001.html>

* lib/am/header-vars.am (mkdir_p): Define as an alias for $(MKDIR_P).
* t/list-of-tests.mk (XFAIL_TESTS): Remove 't/mkdir_p.sh'.
* NEWS: Update.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
tests: avoid spurious failures when @MKDIR_P@ points to "install-sh -d"

* t/self-check-cleanup.tap: Several checks in this test were failing on
NetBSD 5.1.  That happened because on that system, '@MKDIR_P@' expands to
an "install-sh -d" invocation that references the $(builddir), and the
code trying to duplicate some of the Automake testsuite infrastructure
in the test subdirectory of this self test wasn't smart enough to cater
to that situation.  Granted, we could tweak the test case once more to
fix this Yet Another Spurious Failure, but at this point it has become
clear that the extra coverage offered by this test is not worth all the
hassle.  Just remove the test.  Since the testsuite is regularly run on
several systems and with different setups, most issues with the testsuite
framework will reveal themselves anyway; no actual need to unit-test them
in our testsuite, if that's too tricky.
* t/list-of-tests.mk: Adjust.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
tests: don't clutter the top-level dir with temporary test directories

* t/self-check-me.tap: Be sure to initialize '$am_create_testdir' to "no"
in all the shell invocations sourcing './defs'.  Otherwise, when running
the testsuite with 'keep_testdirs=yes', the following temporary director
are left cluttering the top-level directory:

  ./012.dir
  ./abc..dir
  ./a.b.c.dir
  ./foo.dir
  ./foo.bar.dir
  ./foo-bar-.dir
  ./_foo__bar.dir

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
tests: ignore minor 'recheck' regression for BSD make

It turns out that, with NetBSD 5.1 make and FreeBSD 9 make, running
"make recheck" two times in a row quickly fails to correctly re-run
the failed tests in the second run.

That issue has been introduced likely introduced in commit
'v1.12.1-95-gd5443e4' of 20102-07-01, "parallel-tests: reimplement
fix for bug#11791".

Anyway, the use case that has been broken is not realistic (who is
going to run "make recheck" two times in one second, without modifying
any of the tests or the tested programs in the meantime?), so we believe
the best fix is to simply work around the issue in the affected test
cases, rather than risking to slow down or uglify the 'recheck' rule.

* t/parallel-tests9.sh: Enhance a little.
* t/tap-recheck.sh: Adjust adding proper '$sleep' calls were required.
* t/parallel-tests-log-override-recheck.sh: Likewise.
* t/test-driver-custom-multitest-recheck.sh: Likewise.
* t/test-driver-custom-multitest-recheck2.sh: Likewise.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
yacc tests: fix spurious failure with parallel make

* t/yacc-deleted-headers.sh: Here, by adding a missing dependency
in the Makefile.am.  Revealed by a failure with Sun Distributed make
run on Solaris 10 in parallel mode.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
tests init: don't automatically re-execute tests with a POSIX shell

I've unwittingly broken support for that feature *again* in some of
my recent testsuite tweaking.  In this case, the re-execution code
works correctly when the tests are executed with a POSIX shells, but
breaks when they are invoked by an old-style Bourne shells (e.g.,
/bin/sh on Solaris).

It's time to face it: that feature is too much brittle, and too seldom
used (because the Makefile takes care of running the tests with the
correct shell anyway, so that a breakage is only experienced when
running the tests by hand).  It just don't remain working for long, not
when we often touch the testsuite setup (which we are going to do again
when we'll try to move part of our testsuite framework to Gnulib, or a
similar project).

So, instead of trying to be extra-smart and automatically re-execute the
tests with the correct shell, we now offer a simple wrapper script that
the user can employ to run the test scripts with the proper shell.  And
while we are at it, we write this wrapper to also deal with TAP tests in
a better way, running them through the prove(1) utility, so that their
results are correctly recognized and reported.

* t/ax/test-runner.in: New file; the wrapper script we were talking about.
* configure.ac (AC_CONFIG_FILES): Process it into 't/ax/test-runner'.
* .gitignore: Add 't/ax/test-runner'.
* defs: Remove code for automatic re-execution of the scripts with the
correct shell.  This file now just a very thin layer around 'defs-static'
and 't/ax/test-init.sh'.
* t/README: Adjust, and remove or fix some imprecise or outdated text in
the process (like "... test scripts are written with portability in mind,
so that they should run with any decent Bourne-compatible shell ..." ).
* Makefile.am (AM_TESTS_ENVIRONMENT): No need to export 'AM_TESTS_REEXEC'
to "no" anymore.
* t/self-check-explicit-skips.sh: Likewise.
* t/self-check-exit.tap: Likewise.
* t/self-check-me.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-reexec.tap: Remove as obsolete.
* t/list-of-tests.mk: Adjust.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Merge branch 'fix-pr11806' into maint

* fix-pr11806:
  lisp: better support of VPATH builds
test runner: work correctly in VPATH setups

Due to a "feature" of AC_CONFIG_FILES, because 't/ax/test-runner.in'
is in a subdirectory, the '@srcdir@' value that is AC_SUBST'd in it
gets tweaked to contain as much '..' components as are the directory
components of 't/ax/test-runner'.  Because our build system operates
in a non-recursive setup, this substitution is wrong; for example,
the final 't/ax/test-runner' build in a VPATH builds where the source
directory is ".." contains the line:

    : ${srcdir='../../../t/ax'}

instead of the expected (and correct):

    : ${srcdir='../t/ax'}

We solve the issue by building 't/ax/test-runner' with a Makefile
recipe instead of config.status substitutions; this is already done
for other testsuite-related files, like 'defs-static'.

* configure.ac (AC_CONFIG_FILES): Don't build 't/ax/test-runner'
anymore.
* Makefile.am (t/ax/test-runner): New rule.
(EXTRA_DIST): Add 't/ax/test-runner.in'.
(CLEANFILES, noinst_SCRIPTS): Add 't/ax/test-runner'.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
fixup: another "make recheck" failure with BSD make

* t/parallel-tests-log-override-recheck.sh: Here, add a proper $sleep
before calling "make recheck".  This should ideally have been done
in the earlier commit 'v1.12.1-100-g19d84bc', but it somehow slipped
through the cracks.
* t/test-metadata-recheck.sh: Likewise.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
cosmetics: rename t/ax/test-runner => t/ax/runtest

The latter is shorter and clearer.  Better to do the rename early,
before other developers or contributors begin to get used to the
'test-runner' name.

* t/ax/test-runner.in: Rename ...
* t/ax/runtest.in: ... like this.
* Makefile.am, t/README, .gitignore: Adjust.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Merge branch 'maint'

* maint:
  cosmetics: rename t/ax/test-runner => t/ax/runtest
  fixup: another "make recheck" failure with BSD make
  test runner: work correctly in VPATH setups
  compat: automake should define $(mkdir_p), for backward compatibility
  coverage: test that AM_PROG_MKDIR_P and $(mkdir_p) still works
  tests init: don't automatically re-execute tests with a POSIX shell
  yacc tests: fix spurious failure with parallel make
  tests: ignore minor 'recheck' regression for BSD make
  tests: don't clutter the top-level dir with temporary test directories
  tests: avoid spurious failures when @MKDIR_P@ points to "install-sh -d"
  lisp: better support of VPATH builds
  news: fixlets and updates

+ Extra non-trivial edits:

* NEWS: State that $(mkdir_p) is still provided as an alias to
$(MKDIR_P), for (partial) backward-compatibility.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
tests: move 'runtest' into the top-level directory

Move our wrapper script 'runtest' (meant to allow the execution of
Automake test cases from the command line) from the 't/ax/' directory
to the top-level one.  This makes the script easier to find and to
invoke.  Much more importantly, our DejaGNU-checking test cases won't
try anymore to use that script instead of the 'runtest' program provided
by DejaGNU (that happened because '$(srcdir)/t/ax/' is automatically
added early to the $PATH variable in our test cases), which was causing
spurious SKIPs.

* t/ax/runtest.in: Move ...
* runtest.in: ... here.
* Makefile.am, t/README, .gitignore: Adjust.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Merge branch 'maint'

* maint:
  tests: move 'runtest' into the top-level directory
fixup: t/README: it's ./runtest, not ./t/ax/runtest

* t/README (Supported shells): Here. And remove an extra empty line.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
compat: automake should substitute @mkdir_p@, for backward compatibility

That has been unwittingly broken by commit v1.12-19-g7a1eb9f of 2012-04-28,
"AM_PROG_MKDIR_P: deprecate, to be removed in Automake 1.13". We thought it
wasn't a big deal, but Jim Meyering reported that @mkdir_p@ is used in
gettext's Makefile.in.in template:
<http://lists.gnu.org/archive/html/automake/2012-07/msg00014.html>

* lib/am/header-vars.am (mkdir_p): Don't define.
* m4/init.m4 (AM_INIT_AUTOMAKE): AC_SUBST 'mkdir_p' with $(MKDIR_P).
* t/mkdir_p.sh, t/mkdirp-deprecation.sh: Enhance.
* NEWS: Update.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
distcheck: never make part of $(distdir) world-writable

This fixes a locally-exploitable security vulnerability (CVE-2012-3386).

In the 'distcheck' rule, we used to make the just-extracted (from
the distribution tarball) $(distdir) directory and all its files and
subdirectories read-only; then, in order to create the '_inst' and
'_build' subdirectories in there (used by the rest of the recipe) we
made the top-level $(distdir) *world-writable* for an instant (the
time to create those two directories) before making it read-only
again.

Making that directory world-writable (albeit only briefly) introduced a
locally exploitable race condition for those who run "make distcheck" with
a non-restrictive umask (e.g., 022) in a directory that is accessible by
others.  A successful exploit would result in arbitrary code execution
with the privileges of the user running "make distcheck" -- game over.
Jim Meyering wrote a proof-of-concept script showing that such exploit is
easily implemented.

This issue is similar to the CVE-2009-4029 vulnerability:
<http://lists.gnu.org/archive/html/automake/2009-12/msg00012.html>

* lib/am/distdir.am (distcheck): Don't make $(distdir) world-writable,
not even for an instant; make it user-writable instead, which is enough.

Helped-By: Jim Meyering <jim@meyering.net>
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Merge branch 'distcheck-vulnerability-CVE-2012-3386' into maint

* distcheck-vulnerability-CVE-2012-3386:
  distcheck: never make part of $(distdir) world-writable
release: stable release 1.12.2

* configure.ac (AC_INIT): Bump version number to 1.12.2.
* m4/amversion.m4: Likewise (auto-updated by "./bootstrap").

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>