Stop performance testing if OPTIMIZED=1 build failed

* If the compiler OPTIMIZED=1 build fails, turn off compiler perf testing.
* If the runtime OPTIMIZED=1 build fails and it is a performance test run, it
  is a fatal error, so send the failure message and exit.
* Added the ability to exit on a fatal error back to mysystem after it was
  accidentally removed in a previous commit.

Tested by doing: nightly -debug -performance -notest
with and without things to warn about added to the compiler and runtime.
Quiet TAGS complaints when building Qthreads

We've had some noise in the runtime's build of the TAGS file due
to the fact that the generic glob patterns that were designed to
cover the header files for the task/ and thread/ layers didn't
match any files.

The fix I've taken here is to move the specification of the .h
files out of the tag command itself, in Makefile.help, and into
the Makefile.include files in question.  So far I've only done
it for the default (Qthreads) configuration, which seems to be
working.

Next step is to do the same for other task/thread options, and
arguably for other aspects of the runtime that utilize header
files.
Updated fifo and massivethreads Makefile.includes to name .h files
Made the threads Makefile.include files include their header files
Turn off guard pages for stress/deitz/test_10k_coforall

I see this test timeout sometimes when I do a paratest when other people are
also testing. It creates a lot of small tasks so it makes sense to turn off
guard pages for it.
Unify stress and stress.numthr execopts and turn off guard pages

There are two versions of this test because we don't have a way to use a
different execenv for a single test. I believe they are intended to be
identical so I made the execopts the same.

This also turns off guard pages for both as it is a stress test that creates a
large number of tasks and takes a long amount of time to complete with guard
pages. stress.numthr already had guard pages off but as a compopt instead of
as part of the execenv.

The compopts is nice because it's general and the way we want users to do it,
but the execenv is nice because you don't have to copy it for each compopt. For
now I decided to stick with using QT_GUARD_PAGES=false over the compopt, and
once CHPL_STACK_CHECKS also backs whether guard pages are used at execution time
I will replace all uses of QT_GUARD_PAGES
Copied the execenv, but didn't clear the unwanted env vars
Merge pull request #266 from ronawho/qthreads-timeouts

Turn off guard pages for two stress tests

[reviewed by @sungeunchoi]

Turn off guard pages for stress/deitz/test_10k_coforall:

I see this test timeout sometimes when I do a paratest when other people are
also testing. It creates a lot of small tasks so it makes sense to turn off
guard pages for it.

Unify stress and stress.numthr execopts and turn off guard pages:

There are two versions of this test because we don't have a way to use a
different execenv for a single test. I believe they are intended to be
identical so I made the execopts the same.

This also turns off guard pages for both as it is a stress test that creates a
large number of tasks and takes a long amount of time to complete with guard
pages. stress.numthr already had guard pages off but as a compopt instead of
as part of the execenv.

The compopts is nice because it's general and the way we want users to do it,
but the execenv is nice because you don't have to copy it for each compopt. For
now I decided to stick with using QT_GUARD_PAGES=false over the compopt, and
once CHPL_STACK_CHECKS also backs whether guard pages are used at execution time
I will replace all uses of QT_GUARD_PAGES
Turn off guard pages for distributions/robust/associative/stress/many_domain

Guard pages are already off for
distributions/robust/associative/stress/many_array and this is a similar test,
that takes way too much time with guard pages on.
Merge pull request #268 from ronawho/qthreads-timeouts

Turn off guard pages for distributions/robust/associative/stress/many_domain

[trivial, not reviewed]

Guard pages are already off for
distributions/robust/associative/stress/many_array and this is a similar test,
that takes way too much time with guard pages on.
Add an option to sub_test to warn if a test took over X seconds to execute

This functionality is motivated by trying to find stress tests for which guard
pages should be turned off, and applications that may need a higher timeout as
a result of increased execution time from guard pages.

I think it is useful beyond that too, and can easily be extended to report long
compilations times as well.
Merge pull request #269 from ronawho/qthreads-timeouts

Add option to sub_test to warn if a test took over X seconds to execute

[reviewed by @noakesmichael]

This functionality is motivated by trying to find stress tests for which guard
pages should be turned off, and applications that may need a higher timeout as
a result of increased execution time from guard pages.

I think it is useful beyond that too, and can easily be extended to report long
compilations times as well.

Note that this currently must be manually set by a user, and nothing in start_test 
sets it. It's also unaffected by the timeout value, it's just a simple "did this test
take over X seconds to execute?"
Turn on warnings for nightly for tests that took > 60 seconds to execute

I'm using this to try to find stress tests that need guard pages turned off and
codes that may need timeouts added because they now take longer with guard
pages. This is going to make testing noisy for the next two days.

It will not report results for valgrind
Merge pull request #271 from ronawho/qthreads-timeouts

Turn on warnings for nightly for tests that took > 60 seconds to execute

[ok'd by sungeunchoi and thomasvandoren]

I'm using this to try to find stress tests that need guard pages turned off and
codes that may need timeouts added because they now take longer with guard
pages. This is going to make testing noisy for the next two days.

It will not report results for valgrind

Note this is temporary and will only run for tonight.
Forgot to lower stack size for stress like we so for stress.numthr
Merge pull request #272 from ronawho/qthreads-timeouts

Forgot to lower stack size for stress like we so for stress.numthr

[trivial, not reviewed]
Merge pull request #264 from daviditen/nightlyPerf

Stop performance testing if OPTIMIZED=1 build failed

* If the compiler OPTIMIZED=1 build fails, turn off compiler perf testing.
* If the runtime OPTIMIZED=1 build fails and it is a performance test run, it is a fatal error, so send the failure      
  message and exit.
* Added the ability to exit on a fatal error back to mysystem after it was accidentally removed in a
  previous commit.

Tested by doing: nightly -debug -performance -notest
with and without things to warn about added to the compiler and runtime.
[reviewed by gbtitus]
Fixed some regressions from last nightly testing
Merge pull request #273 from vasslitvinov/fixup-tests-for-coercion-wrapper-elimination

Fixed some regressions from last nightly testing.

.good changes in the robustness suite are due to improved fork counts, possibly due to coercion wrapper elimination. I did not investigate.

I added a missing .good file.

For the rest:


functions/vass/setter-passed-to-another-fun.bad

=> I verified that the new contents of .bad are legitimate


types/enum/sungeun/cast_enum_weird_4.bad

=> This might not be related to my coercion-wrapper commit.
At any rate, the updated line number makes more sense
than the one that was there before.


types/records/const-checking/scenario-1-array-of-record-with-const-fld.bad
types/records/const-checking/scenario-2-init-array-of-rec-with-const-fld.bad
types/records/const-checking/scenario-3-assoc-dom-of-record-with-const-fld.bad
types/records/const-checking/todo-linenumber-in-array-assignment-error.future
types/records/const-checking/todo-linenumber-in-array-assignment-error.bad

=> The new line number reported by the compiler is not better than the old one.
Neither is perfect; passing --print-callstack-on-error gives the desired info.
The following future requests fixing this:
  todo-linenumber-in-array-assignment-error.bad
Merge pull request #267 from bradcray/fixTags

Fix runtime tasks/threads TAGS

This change replaces some catch-all glob patterns that we were using to
get source files from the tasking and threading libraries into the
emacs TAGS (or vim ctags) files.  For tasks=qthreads, threads=none,
the globs didn't match anything resulting in warnings at TAGS creation
time.

The fix taken here is to

(a) push the glob patterns for header files from the top-level
    Makefile.help file down into the task- and thread-specific
    Makefile.include files so that they can be more precise
    about pointing to the files in question (if any).  The only
    real downside of doing this is that TAGS files should generally
    process source (.c) files before header (.h) files in order
    to take a TAGS surfer to the definition of a symbol rather than
    its prototype.  The current approach has the downside of
    mixing .c files and .h files in the ALL_SRCS variable used to
    create the TAGS.  However, since each tasking/threading layer
    gives its .c files first and then its .h files, I expect things
    will still work out OK.  If they didn't we could create an
    ALL_HEADERS variable that each layer contributes to and then
    have the tags command name ALL_SRCS before ALL_HEADERS.

(b) Had Qthreads point to the sources in third-party

(c) Had threads=none not list any files (left the pattern in the
    file, commented out)


The only testing I did on this was to make sure that the various
tasking configurations built and created TAGS without warnings.


Further TODOs:

* looking at this, it's apparent that we don't include any .h
  files in ALL_SRCS beyond those in tasks/ and threads/ (such
  as comm/, mem/, atomics/, etc.  These should be added into
  ALL_SRCS as well.  I didn't do so here primarily because it's
  5pm on a Friday holiday weekend.

* Sung had a request to rename ALL_SRCS something like ALL_RUNTIME_SRCS
  or possibly ALL_RUNTIME_TAG_SRCS to make its role clearer.  I
  forgot about that until I was typing this up (see previous note
  about 5pm on a holiday weekend)