Make BlockDom iterator ignore running tasks on the initiating task

[reviewed by Brad and David]

This patch addresses a performance issue that arose when the default
for dataParIgnoreRunningTasks was changed to false (r22641).  On the
initiating locale, the local task created by the coforall statement
can start before the initiating task is finished creating all the
tasks.  When this local task computes the number of tasks to use
locally using ignoreRunning=false, it may count the initiating task.

This commit gets around this issue in cases where the initiating task
is thought to be the only task running on the locale, by using
ignoreRunning=true in these cases.

This is a some what dissatisfactory solution, but it's the best we can
do with the current tasking interface.  Here are a couple of thoughts
on other options.

- We have talked about moving making the task counting a Chapel-level
  concept, and in that world, we could have a task request that it
  *not* be counted or provide a count that excludes such tasks.

- We have also talked about providing a vector fork operation for
  coforalls.  In that situation, the initiating task could pull off
  one local task for itself.  This has the downside of delaying
  starting of the local task, but it has the advantage of not having
  to go into the tasking layer.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22892 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Optimize candidate pair removal in copyPropagation. [reviewed by eronagha]

Added back the reverse map of available pairs to optimize the invalidation of
candidate pairs.  This has the desired optimization effect (making the
algorithm faster without changing its behavior).

On the studies/lulesh/sungeun/lulesh test case where copy propagation was
taking a minute or more, the time dropped to 3-4s.  The number of emitted lines
of code remained the same.

The reverse map uses the RHS symbols as keys.  Since several LHS symbols may
have the same RHS, the mapped value is a list of LHSs associated with that
RHS.  When a new candidate pair is created, the LHS is pushed onto the list
associated with that RHS.  

When a symbol is invalidated (because its associated value is update), there
will be at most one candidate pair that contains it on the LHS.  This pair is
removed from the forward map directly.  Using the reverse map, all candidate
pairs having that symbol on the RHS are also removed.  This leaves that entry
in the reverse map empty, so it can be removed.  

Note that when the pair (lhs1, rhs1) is removed because lhs1 == sym, the entry
(rhs1, [..., lhs1, ...]) will remain in the reverse map.  That is OK, because
if rhs1 is invalidated at a later time, the attempt to remove lhs1 from the
forward map will fail silently.  This is probably cheaper than trying to keep
the reverse map precisely up-to-date.


[tested on / (std)]


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22893 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Better const args for denested functions and remote value forwarding

[reviewed by hilde, vass, with one slight post-review modification
that should be reviewed post-commit]

This commit improves the insertion of formals to represent non-local
variables a bit better when de-nesting functions by strengthening the
check for the const-ness of the original variable and not making the
argument type have 'ref' intent if it is constant.  This reduces the
number of unnecessary references we use for nested functions (user or
compiler-inserted).

The one bit of this commit that was not reviewed is that in the 11th
hour, I switched from using the isConstant() method to adding a new,
stricter, method called isConstValWillNotChange().  The former says
that something is constant in the sense that you cannot assign to it
(e.g., it includes 'const ref' arguments) while the latter says that
something is constant in the sense that you can rely on it not
changing out from under you (so 'const ref' is excluded).  As a
cleanup step, I think we should change isConstant() to
isConstCannotAssign() and then switch existing calls to isConstant()
to one or the other version.

The other major part of this patch is an improvement to remote value
forwarding for arguments with 'const in' intent in the presence of
sync variable accesses.  Previously some of these cases were not
remote value forwarded, but they can/should be.  This change will have
more benefit once more formal_temps are removed.  For the 'reduce*'
tests in the distribution robustness performance suite, this resulted
in one fewer get per locale.

I added an editorial note in the comments of flattenFunctions.cpp
pointing out that making a formal have a 'ref' type but 'blank'
intent, as we currently do, is arguably inconsistent, but left that as
future work (perhaps near-future work, as we've talked about pushing
the insertion of 'ref' types much closer to codegen in recent
discussions).  See the comment itself for more detail.

I also added a test that presents a simplified version of the case
that caused me to note the lack of remote value forwarding in the
instance of 'const in' arguments (co-written by Vass).  This test
works at present with or without r.v.f. but once fewer formal temps
are inserted (coming soon to a trunk near you), lack of r.v.f.  breaks
this test (and SSCA2, from which it was derived).



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22894 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Add quotes to compiler performance description 

[trivial, not reviewed] 

When nightly passed the compiler performance description to start_test the
parenthesis caused a syntax error. This just puts quotes around it so something
like -compperformance-description (--fast) will now be
-compperformance-description "(--fast)


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22895 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Retiring a passing future

This future seems to be passing now and from the description seems
plausibly due to the improved use of consts and remote value
forwarding.  The test is large and complex enough that I didn't look
through it by hand.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22896 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Triage week wrap-up:

Removed numLocales2Space, numLocales2 and printNumLocalesTwo errors as they are
resolved.

Marked the IS performance failures as no longer silent.

Added an entry for release/.../hpl's validation failure in case it happens again
(although we hope it doesn't)

Marked an additional date where we saw studies/jacobi/waynew/example2 fail

Added an entry for numChunks non-determinism in forwardDefaultDom



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22897 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Add test to check for changes to the compiler passes

[reviewed by tom]

This is a simple check to test if the compiler passes have changed for the
purpose of notifying us when we have to update the compiler performance
testing. 

Unfortunately it's hard to automate adding/reordering/removing passes. At least
for anything even a little complex. How do you automatically tell the
difference between if a pass was removed and a new one was added vs simple
renaming.

When a pass is changed the compilerPerformance.perfkeys has to be updated. The
.dat files have to be updated and the .graph files have to be updated.

For now I think it makes sense just give me a heads up when a pass changes and
I'll take care of it. Passes don't change often so this shouldn't be a big
deal.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22898 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Add bulkCopyRecords pass to compiler performance graphs 

[not reviewed, trivial] 

This just adds a recently added passes to the compiler performance graphs. It
updates the perfkeys and the .graph files. I'll update the nightly .dat files
now. 


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22899 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Reduce the number of (unnecessary) formal temps inserted by the compiler

[reviewed by hilde]

At some point about six months ago, I found myself trying to figure
out why we insert so many formal temps (temps for formal arguments to
functions) in the generated code; in looking into it, I was
overwhelmed by the logic in function resolution that determines
whether or not we do.  My conjecture was that such temps should be
required in only a very small number of cases -- those requiring
copies on the way into the procedure that were not handled by the
back-end function calls themselves.

To that end, I tried ripping out all the logic and replacing it with
something far simpler: Insert a formal temp if the argument is 'inout'
or 'out' (because we pass by reference in these cases, so we need an
explicit copy); or for 'in or 'const in' intents if the type is one
for which we can't rely on C to do the copy for us.

This new logic worked surprisingly well, except that it unearthed some
pre-existing faults in the implementation that inserting formal temps
had previously been papering over.  The main one of those was fixed by
my recent commit relating to remote value forwarding.  This commit
addresses a few other smaller ones.

The benefits of this commit are: Simpler function resolution logic
w.r.t. formal temps; fewer formal temps inserted, reducing the size of
the AST (some of these were already eliminated before codegen by later
passes like copy propagation; others made it through to codegen and
now don't); fewer copies for routines that take record and tuple
arguments where formal temps had been inserted previously.

Overall, this commit also reduces the number of memory leaks and total
amount of memory allocated/freed in the examples directory.  However,
it does add one new (to me) leak case -- for some cases of zippered
forall iteration, the lack of a formal temp for the 'followThis'
argument results in an iterator class being leaked.  In talking with
Tom, he indicated that he had not done much to plug leaks for
iterators and was not surprised that there would be fragility here.
The overall win seemed worth the loss.  I do have a hack (currently
commented out) in this commit in functionResolution.cpp which
special-cases followThis arguments and can be used to plug cases of
this leak that I've seen, if we choose to do this.

With respect to "unearthing pre-existing faults", the main one worth
calling out is one related to function resolution of dynamically
dispatched methods.  The way in which I saw this show up is as
follows:

An array's copy code makes calls to here.id/locale.chpl_id() while it
is reasoning about bulk copy optimizations.  If:

(a) a code contains no calls to here.id and 

(b) these copy routines are not introduced into the user's code prior
to function resolution and 

(c) calls to these copies are introduced
after function resolution

then the compiler incorrectly decides that the chpl_id() call on the
locale model should be statically dispatched to the abstract base
class type's "pure virtual" function which throws an error rather than
properly dynamically dispatching the call.  Needless to say, this is
problematic.  The workaround I used to get around this issue was to
introduce a call to here.id in user code -- specifically in a new
internal module ChapelDynDispHack.chpl.  If this call is removed,
tests like the following will fail:

        test/classes/bradc/arrayInClass/arrayDomInClassRecord-simpler.chpl

I'm 95% certain that this is a pre-existing problem with function
resolution, yet it's one that I was unable to reproduce in a simpler
program.  Again, the reason it's showing up now is that I've reduced
the number of temps inserted, resulting in fewer copies in user code,
resulting in fewer calls to here.id/chpl_id() in user code (in cases
like the above, none), and so the copy function only gets resolved
in the special resolveAutoCopies() pass of functionResolution.cpp;
at which point it is incorrectly resolved statically rather than
dynamically.


The main other problematic case that I'm aware of is

  test/multilocale/diten/needMultiLocales/remoteStringTuple.chpl

which now fails on multiple locales due to passing a non-local string
to an extern routine.  My hypothesis was that it would be better to
wrestle with this case after strings are converted into records and
Sung agreed, so we'll let this one fail for now.


Here are some additional file-by-file notes:

functionResolution.cpp: Contains the changes to formal_temp insertion
logic as described above.

symbol.cpp: Fixes one of the formerly hidden sins, by flagging that
sync variables passed by blank intent are not constant.

ChapelBase.chpl: chpldev_refToString() used to be a hack that relied
on 'inout' intents; now that we have ref intents, it can (and, without
formal temps, needs to be) written more properly using a 'ref' intent.

ChapelStandard.chpl and ChapelDynDispHack.chpl: These are the module
changes that contain the hack relating to dynamic dispatch of here.id
noted above.

printModuleInitOrder.good: changes to reflect the new
ChapelDynDispatchHack.chpl module described just above.

refcnt.good: now that we don't put in formal temps for records, the
number of constructor/destructor calls in this program are now
significantly reduced (presumably a good thing), so the .good is
updated to reflect that.

test/*/*.chpl: removing formal temps revealed that some temps were
changing records (passed by 'const' intent by default) illegally.  Our
const checking isn't good enough to flag these cases; and the formal
temps were preventing the modifications from showing up back at the
callsite; once they were removed, the problems became evident.

arrayDomInClassRecord-illegal.chpl, recordpass6-illegal.chpl: These
futures capture the case that assignment to records passed by blank
intent should flag an illegal assignment error, but don't currently.
Previously, because of the formal temps that were inserted, such local
assignments were permitted and didn't reflect back to the callsite;
now that we're squashing formal temps, the illegal modifications were
reflected back at the callsite.

arrayDomInClassRecord-simpler.chpl: As noted above, this is the
simplified version of the code that causes the pure virtual function
error when there aren't user-level calls to here.id.






git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22900 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Oops -- This was meant to go in with the previous commit

Tom and I decided to futurize this memory test for the time being
rather than enable the commented hack in functionResolution.cpp that
would keep it working.  See my previous commit and the .future file
itself for more information.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22901 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Remove extra space from compiler performance script 

[trivial, not removed]

There was an extra space in the combine script for compiler performance work
that led to extra spaces in the .dat files. Not really a big deal but it's nice
ot get rid of them. I removed the spaces in the .dat files already. 


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22902 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Update atomic intrinsics to support more compilers 

[reviewed by sung] 

Previously we had support for using atomic intrinsics for gcc, clang, and
intel. This updates the intrinsic implementation to support cray as well. 

This also updates the atomic chplenv script to set intrinsics to the default
for cray (and makes sure that for intel, gnu, and cray both the normal and the
PrgEnv versions are supported. 

Removed use of __sync_lock_test_and_set(obj, value) intrinsic. 
 
 Many implementations of this only support using 1 as the value so I don't think
 we should have been using it anyways. Plus it's not supported by cray so it's
 nice to get rid of it for that reason. I believe that instead of being an
 actual intrinsic this just turned into a regular function call since most
 targets don't support it. 

 This appeared in:
  - atomic_flag_test_and_set_explicit 
  - atomic_exchange_explicit 
  - atomic_exchange_explicit (real)

 The atomic_flag_test_and_set is explained below. For the other two this was
 replaced with a call to __sync_val_compare_and_swap(). We pass the dereferenced
 object as the expected to this function so that it is really being used as a
 swap function. We do:
 
 oldVal = *obj;
 __synv_val_compare_and_swap(obj, oldVal, newVal) 
 
 so we have to put this in a loop because a thread could interleave the
 dereference and the compare and swap. For some compilers this just got turned
 into an atomic exchange but it wasn't consistent across compilers so the loops
 is needed. 

 Changed how the flag test and set and clear are implemented 
  - They now basically atomically and 0 to clear and atomically or 1 to set.  
    This was to get rid of __sync_lock_test_and_set and because it's cleaner. 

Added is_lock_free for the real atomics. It seemed to be missing even though it
was externed in the atomic module. 

Changed __sync_synchronize to a macro full_memory_barrier. This is because cray
doesn't support __sync_synchronize. Instead we use __builtlin_ia32_mfence which
does the same thing. 

__sync_bool_compare_and_swap is not supported on cray. However
__sync_val_compare_and_swap is (though it's undocumented.) To get around this
a macro was added that fakes __sync_bool_compare_and_swap using
__sync_val_compare_and_swap for cray. Note that adding
__sync_bool_compare_and_swap is an RFE for cray so we may be able to remove
this macro in the near future. 

There are a couple of cleanup items as well: 
 - Reordered real atomics to mirror integer atomics
 - Deleted prototype code that was lying around in comments 
 - Added headers to each section because the macros can make things hard to
   find

Some misc notes: 
 - PGI does not support atomic intrinsics so we do not have any implementation
   for them. It may be possible to use the openmp atomic statement to
   implement atomics (but it's not clear if that will map down to a lock free
   implementation or if we can implement our entire interface.) 

 - We should revisit some of the functions because they might not fully mirror
   the c11 spec. For instance I'm not sure that all the intrinsics map down to
   lock free operations so our is_lock_free might be incorrect. Also our
   implementation of atomic_compare_exchange_* is wrong. It is supposed to be
   atomic_compare_exchange_*(A* obj, C* expected, C desired) but we have atomic_
   compare_exchange_*(A* obj, C expected, C desired) Additionally on failure
   expected is supposed to be set to obj (which clearly we can't do right now
   since expected is not a pointer.) This change should be easy to make for
   locks, and shouldn't be too bad for intrinsics. I don't think it's that
   important for now, but it will be more important once we have a c11
   implementation.

 - Since our interface mimics the c11 interface it makes sense that once c11 is
   implemented by more compilers we should add an implementation that uses
   them. It looks like gcc, and clang will be releasing new versions that will
   support these in the next few months so we should be able to follow and add
   something for our next release. I don't have a roadmap for cray, intel, or
   pgi. 

 - For the leadz functions they don't appear to be used anywhere, and they
   don't really belong here in the first place. I'll take a look at this
   post-commit and see if they can be removed.  


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22903 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Make atomic_flag_test_and_set_explicit return a dummy value

[reviewed by brad] 

gcc complains about a value computed is note used if we don't do this...


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22904 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Updating .bad to reflect lack of formal temps

With the reduction in formal temps yesterday, the error message for
this test changed from referring to a formal temp to the formal
itself.  Updating the .bad to reflect this.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22905 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Retiring a pair of futures that now pass due to reduced formal temps



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22906 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Updating .good for this test as well

I'm realizing belatedly that the .good for this test needed to be updated
to reflect the lack of formal temps as well.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22907 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Change performance test verification to display the test output on failure

Performance test verification was quitting computePerfStats after a check
failed. This commit changes it to continue and print the test output as we
do when a perf key is not found.


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22908 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Clean up of stray ifs in normalization and functionResolution [reviewed and discussed with Tom]

Some sequences of if statements within the compiler were accomplishing the same
purpose under mutually exclusive conditions, so I condensed those if statements
into if/else branches and logically divided a couple of others as a compromise
between Tom's and my school of thought.



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22909 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Remove confusing use of backslash

[trivial and not reviewed]



git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22910 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Bulk read/write operations on DefaultRectangular

[Reviewed by Michael]

Add the capability to read and write some DefaultRectangular arrays with a
single IO operation. This works by adding a new method to the Reader/Writer
and ChannelReader/Writer classes, [read|write]Byes that take a pointer
(_ddata) and length which is then used via qio_channel_[read|write]_amt.
DefaultRectangularArr.dsiSerial* have been modified to use this faster method
if possible.

To facilitate these changes two new QIO_STYLE_ELEMENT_* variables have been
added: BYTE_ORDER and IS_NATIVE_BYTE_ORDER.

Checking QIO_STYLE_ELEMENT_BYTE_ORDER returns the byte order of a Reader or
Writer (ionative, iobig, iolittle).

Checking QIO_STYLE_ELEMENT_IS_NATIVE_BYTE_ORDER returns 1 if the Reader or
Writer is set to use a native equivalent byte order, 0 otherwise.

At the moment these operations only work when the DefaultRectangular has an
element type of bool, integral, real, imag, complex, or enum. In the future
this could possibly be extended to other types.


git-svn-id: svn+ssh://svn.code.sf.net/p/chapel/code/trunk@22911 3a8e244f-b0f2-452b-bcba-4c88e055c3ca