Introduced the "capture for tasks" feature - implemented in the compiler; adjusted some tests and/or .good to work correctly.

[reviewed by David and Sung]

This feature, aka "task intents semantics", was originally proposed
by Michael Ferguson and further developed jointly by Michael and
the Chapel group.

Warning: under this semantics, certain 'const' variables are created
implicitly. Since currently the compiler does not perform a complete
const-ness check, additional overlooked modifications of consts may
result. Most notably, updates to const record fields and const tuple
components are overlooked.

Also, this feature is not implemented for --serial compilation.
This is because
 (a) we intend to remove the --serial feature, and
 (b) there is not much lost without the capture for --serial
     (see a discussion under Implementation Notes).


LANGUAGE DEFINITION
-------------------

"capture for tasks" is a new language feature, defined as follows.
Consider a variable that:

- is referenced in the lexical scope of a Chapel task construct, i.e.,
  begin, cobegin, or coforall,

- is declared outside of that lexical scope, and

- has the type for which, in the case of argument passing, the blank
  intent means passing by "const in", e.g. primitive types, records
  and classes.

All references to this variable within the lexical scope of this
construct now refer to an implicit "shadow" copy of this original
variable that:

- is made at the task creation time, implicitly, and

- is 'const'.

There are no syntactic changes in the above.

The programmer can create exceptions to this rule as follows. A shadow
copy is not created, and all references within this construct refer to
the original variable, if the construct contains a 'ref' clause that
lists the variable.

The ref clause is optional and has the keyword 'ref' followed by a
parenthesized list of identifiers, for example:

  begin ref(myVar) { ... }
  cobegin ref(var1, var2) { ... }
  coforall i in myIter() ref(anotherVar) { ... }

If there is a nested task construct, the ref clause does not extend
into that construct. For example:

  var x, y, z: int;

  begin ref(x,y) { // construct (A)
    ... x ...   // the original
    ... y ...   // the original
    ... z ...   // a shadow copy

    begin ref(x,z) { // construct(B)
      ... x ...   // the original
      ... y ...   // the shadow copy created for (B)
      ... z ...   // the shadow copy created for (A)

  } }

TODO: we would like to allow a "blanket" ref clause that exempts *all*
variables from shadowing. At present we think it will be "ref(*)".


DESIGN DISCUSSION
-----------------

The primary motivation for this feature is to eliminate some races on
scalar/record variables, which are possible when one task modifies a
variable and another task reads it. Prior to "capture for tasks" it was
easy to introduce and overlook a bug illustrated by this condensed example:

    var i = 0;
    while i < 10 {
      begin f(i);
      i += 1;
    }

Another motivation is that referring to a copy of a scalar in a
'begin'-introduced task is often more efficient than referring to the
original scalar. That's because the copy is a local constant, e.g. it
could be placed in a register, while the original scalar would need to
be accessed via pointer dereference. In the former case, the original
scalar does not need to be put on the heap. The latter itself could
not be optimized and furthermore can prevent other optimizations such
as loop-invariant code motion. Also, with "capture for tasks", the
original scalar never needs to be put on the heap.

While developing this feature, we made the following decisions, among others:

- Treat cobegins and coforalls the same way as begins. This is for consistency
  and to make the race-avoidance benefit available to more code.

- Capture global variables the same way as local variables. Again, this is
  for consistency. One could view global variables differently than
  local variables (e.g. a global variable is "always available"),
  bug we favored consistency over such an approach.

- Do not capture the "closure" variables, i.e., the variables in the
  dynamic scope of a task construct. This is to keep capture manageable,
  so that all variables subject to capture can be obtained by examining
  just the lexical scope. The set of closure variables can be hard to
  determine, unweildy to implement and reason about, it is unclear
  what to do with extern functions, etc.


IMPLEMENTATION NOTES
--------------------

* The key piece is the added code in createTaskFunctions. Since,
during compilation, each Chapel task construct is converted to a
"task function", capture is implemented by adding certain variables
to that function. Specifically, the variables referenced within the
construct but defined externally to it are added. If the construct
has a ref clause, the variables listed there are excluded.

  The remaining variables are passed by blank intent.
That way, if the variable's type does not require its capture,
the variable is passed by [const] ref and its copy is not created.

  This added code is copied from the flattenFunctions pass, with
(mostly) simplifications. One difference is that we do not differentiate
between by value vs. by reference passing. Another difference is that
we include, when appropriate, global variables in addition to local
variables. BTW recall that createTaskFunctions occurs before resolution;
flattenFunctions occurs soon after resolution.

>> passes/createTaskFunctions.cpp
 addVarsToActuals
 addVarsToFormals
 createTaskFunctions
 findOuterVars
 isOuterVar
 pruneOuterVars
 pruneThisArg
 replaceVarUsesWithFormals

* In resolveNormalCall() - during resolution - we fix up the types
of each task function's formals and clear its "generic" flag
(if present), using handleCaptureArgs().

  One would expect these things to happen naturally during the
resolution. For some reason though, if we do not do the above,
gatherCandidates() fails to instantiate the task function in some cases.

  Currently during the resolution pass I expect each task function
to be called by exactly one CallExpr. That CallExpr should be
right after the function's defPoint.

  TODO: I would like to exclude the following variables from getting
added to a task function's argument list:

  - The variables not to be captured due to their type. The result
    will be that the global variables that are not to be captured will
    be left alone - there will be fewer arguments to pass. Cf. the
    local variables would still be converted to arguments during
    flattenFunctions.

    This differentiation cannot be done prior to resolution because
    the types are not available yet. So either we calculate the
    to-capture sets during resolution (downside: more stuff in
    resolution), or weed out the ones we do not want afterwards
    (downside: extra work to replace their uses within the task
    functions back to where we started).

  - The variables to be captured for cobegins and coforalls.
    As Tom observed, they do not *have* to be captured to implement
    "capture for tasks", perhaps unless they have non-trivial
    constructors/destructors.
      For now I am still capturing those. Given that we may overlook
    some modifications of constants, I prefer those modifications to
    happen locally to one task - because this is closer to our final goal
    than allowing the modifications still to be visible across tasks.
      Also not capturing those variables eliminates the benefit of
    by-value vs. by-reference access in the generated C code.

  - More constants, e.g. index variables (FLAG_INDEX_VAR,
    FLAG_COFORALL_INDEX_VAR). I have not thought about when it is
    legal to exclude these; also need to ensure that other parts
    of compiler, e.g. flattenFunctions, recognize them as consts.
    Example: 'chunk' in DefaultRectangularDom leader.

>> functionResolution.cpp
 handleCaptureArgs 
 resolveNormalCall

* Added needsCapture(FnSymbol*) to denote those functions that are
task functions falling in the scope of "capture for tasks".

>> expr.h
 needsCapture

  Note that 'on' statements are also converted to task functions in
the compiler. Such a task function is subject to capture of its
variables when it is the result of a "begin+on" optimization.
In such a case it has the flag FLAG_NON_BLOCKING. Recall that the
"begin+on" optimization happens in build.cpp in
buildCoforallLoopStmt(), buildBeginStmt(), and buildCobeginStmt().

>> build.cpp
 addByrefVars
 buildBeginStmt
 buildCobeginStmt
 buildCoforallLoopStmt

* The extensions to the parser are to support the ref clause. They are
straightforward. The variables listed in the ref clauses are stashed in
BlockStmt::byrefVars - see the next two bullets.

>> parser/chapel.ypp

* I created BlockStmt::byrefVars to represent the variables in the ref
clause, if any. It is a CallExpr*, primarily because that's what
comes out of the parser. Each argument of the CallExpr is initially
an UnresolvedSymExpr. It gets resolved to a SymExpr in the resolve
pass. That happens without any mods to the resolve pass because it
already goes over all UnresolvedSymExprs in the AST.

>> stmt.h
 BlockStmt::byrefVars
>> stmt.cpp
 BlockStmt::verify
 et al.

* Under --serial, i.e. when fSerial is true, "capture for tasks" is abandoned.
That's because at present in this case the compiler treats task constructs
as straightline code. Task functions are created only for 'on' statements,
and those are not subject for capturing.

  We are not modifying that because we intend to remove the --serial feature.

  However, if a program compiles correctly without --serial, then
under --serial, we believe, it will behave the same way whether
capturing is implemented or not. The only difference in execution is
that constructors/destructors for shadow copies are not invoked
because there are no shadow copies, to begin with. The only other
thing that's missing under --serial is const-ness checking for
shadow copies - for the same reason.


TEST CHANGES
------------

* Mostly I simply added ref clauses. Here are some patterns where that
was needed:

- Lots of small tests:

  var x = 1;
  ...
  begin/cobegin/coforall ref(x) {
    ...
    x = 2; // or x += 1; etc.
    ...
  }

- test/multilocale/bradc/needMultiLocales/remoteReal.chpl et al.

  var r: real = 1;
  var flag: sync bool;

  begin ref(r) {
    flag;
    writeln(r);
  }

  r = 2;
  flag = true;

- test/parallel/taskPar/sungeun/barrier/basic.chpl et al.
  test/multilocale/diten/needMultiLocales/coforall_on.chpl

  record Barrier {
    var count: atomic int;
    var done: atomic bool;
    ...
  }

  For it to work, Barrier-typed variables need to avoid capture.
  This is the same situation as with atomics.
  Since Barrier does not have built-in compiler support, need
  to mark it explicitly to avoid capture.

- The modules ChapelLocale and AdvancedIters required modifications
  for the same reasons as above.

* test/classes/figueroa/RecordDestructor1b.chpl
  test/classes/figueroa/RecordDestructor1b.prediff

The test contains a record destructor with an observable side-effect -
a writeln(). The test creates a shadow copy of a record variable,
which is then destructed, so an extra writeln() is observed.

Since there is non-determinism in ordering some of the output,
I am adding a .prediff to sort the output.

The compiler currently does not insert any constructors for this code.
This is independent of this commit. Therefore even if this code
contained constructors with side effects, today they would not be observed.

* test/parallel/begin/deitz/test_global_for_begin.chpl
  test/parallel/begin/deitz/test_global_for_begin.good

I replicated the existing code to execute it once with capturing and
once without capturing, i.e. with a ref clause.

* test/parallel/taskPar/sungeun/barrier/basic.chpl
  test/parallel/taskPar/sungeun/barrier/commDiags.chpl
  test/parallel/taskPar/sungeun/barrier/reuse.chpl
  test/parallel/taskPar/sungeun/barrier/split-phase.chpl

See 'record Barrier' above. Also, while there, I changed the functions
localTest() and removeTest() to accept instances of Barrier by
reference (approved by Sung). This seems more appropriate, but does
not change the test behavior because those instances were not relied
upon across those function calls.

* test/performance/sungeun/multilocale/syncsingle.good
  test/studies/ssca2/performance/SSCA2_commDiags.good

Minor improvements to comm counts. I did not investigate the causes.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@21940 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
SubLoc -> Subloc as per r21937



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@21941 3a8e244f-b0f2-452b-bcba-4c88e055c3ca