chapel-lang/chapel — pull request #736 by vasslitvinov (2014-10-31 22:59:20+00:00)
Title: Fix replicated-dist testing

Fixed failures in dist-replicated nightly testing.

As of today, we have 60 failures in dist-replicated nightly testing.
With these changes, there should be none.
Most of the changes are:

* Created separate .good files for testing with replicated.
Checked their correctness.

Given that the replicated distribution is different by nature
from the others, we chose this instead of trying to make
the tests behave the same as for other distributions.

* Provided PREDIFFs to dispatch among .good.
Augmented CLEANFILES and .gitignore.

* Skipif-ed several tests for replicated.

* Modified a test to account for replication.

* Reduced the problem size for a test to avoid timeout.

Details below.

Note: testing, e.g.

start_test -compopts -sdistType=DistType.replicated ...

will fail if run under comm=none. That's because
.replicated.good are specific to the number of locales.

Aside: a common pattern in a lot of tests results in this:

var Dom1D = {1..100} dmapped Replicated();
var AA1d: [Dom1D] real;
for i in Dom1D do
...AA1d[i]...;

// same for 2d, 3d, etc.

When run e.g. on 4 locales, it is equivalent to this loop:

for i in 1..100 do
...AA1d[i]...;

performed *4 times*. Each time it accesses the same local
replicand of AA1d.

This is not very intuitive and can be used as an argument to change
the behavior of Replicated.

DETAILS

* Added

distributions/robust/arithmetic/PREDIFF

derived from distributions/robust/arithmetic/performance/multilocale/PREDIFF.

For a given test, it fires only when .default.good exists.
If so, it checks which .good is pertinent - .replicated, .block, ... -
and copies that one - or .default.good if that one does not exist -
into .good.

* a PREDIFF in a subdirectory of distributions/robust/arithmetic
invokes the above prediff. I chose that over a symlink to make
the failure more obvious.

* For each test that exhibits non-standard behavior of Replicated,
I moved .good into .default.good and created .replicated.good.
NB .replicated.good is sensitive to the number of locales.

I also added .good to CLEANFILES and .gitignore.

The behavioral differences are discussed in the thread re
"Cron Dist replicated (linux64.gnu.gasnet.fifo.flat) 60/0/60"
on the internal mailing list in mid-february 2014.
In summary:

- domains and arrays print out differently
- parallel and serial iterations go over each index multiple times,
once for each locale
- therefore things like reductions and 'for a in A do a=next();'
behave differently too

* For each of the above tests, .replicated.good came from the current
test behavior. For each, I checked, sometimes thoroughly sometimes not,
whether that behavior is correct. For a couple of tests I wrote a test
that mimicks the behavior using DefaultRectangular.

The following tests are exceptions - no checking for correctness,
except the first part of Random:

modules/test_module_Random
modules/test_module_Norm
performance/multilocale/assignSlice.chpl

* These tests already had .replicated.good. They were failing
because they had a significant reduction in fork counts.
I did not investigate, just updated .replicated.good:

performance/multilocale/assignSlice
performance/multilocale/reduceAlias
performance/multilocale/reduceSlice
performance/multilocale/assignAlias
performance/multilocale/alloc

* Added a skipif for replicated for the following tests:

- output is nondeterministic under Replicated:

basics/test_zipper_default
basics/test_zipper_range

- the expected optimization does not fire under Replicated;
a todo is to check whether that is OK:

optimizations/optimizeOnClauses_basic_record
optimizations/optimizeOnClauses_basic_tuple
optimizations/optimizeOnClauses_basic

- MergeSort gets into "zippered iterations have non-equal lengths"
which seems correct based on how MergeSort si written:

modules/test_module_Sort

- This test crashes under Replicated. I skipif-ed it and created
a separate trimmed-down future (#706) distributions/dm/hplx.chpl:

kernels/hpl

* Modified this test to account for replication when present:

stress/test_many_domains

* Reduced the domain sizes to avoid timeout:

slicing/test_array_slicing3

This test creates lots of array slices, which is slow for privatized
distributions such as Cyclic and Replicated. The new (rather small)
sizes make the test run, under Replicated and on the machine that
tests it nightly, in ~70 seconds.

* While there, clarified instructions on running the robustness suite
in distributions/robust/arithmetic/README.