Use sublocales in DefaultRectangular and range leader iterators

[Reviewed by Sung]

If there are sublocales, do a coforall+on over them and a coforall over their
cores to distribute the iterations for DefaultRectangular and for ranges.

For numa, this breaks the iterations up across the domains and also runs in
parallel on multicore domains.

If the locale model doesn't have sublocales the behavior is unchanged.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@21944 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
[Co-authored with Greg.  Reviewed by David and Greg]

This commit replaces calls to chpl_mem_alloc() and chpl_mem_free()
with calls to the locale model specific chpl_here_alloc() and
chpl_here_free().

In the compiler, such calls can be inserted both before and after
resolution.  Since type information is incomplete before resolution,
we must fix up the type information for some of the primitives
inserted for chpl_here_alloc() and chpl_here_free().  In particular,
for chpl_here_alloc(), we fix up the type for sizeof and insert a cast
to the type.  For chpl_here_free() we dereference any ref types that
were used.

Currently both locale models directly call the corresponding
chpl_mem_* functions.

This change exposes an existing bug in iterator inlining.  After
discussions with the group, it was decided that we would live with the
additional memory leaks.  Also, test_nested_var_iterator2.chpl fails
now, but it is already known to expose this bug when scalar
replacement is disabled (that's what test_nested_var_iterator3.chpl
is).  It has been turned into a future, that points to
test_nested_var_iterator3.future.

=======
modules/internal/localeModels/flat/LocaleModel.chpl:
modules/internal/localeModels/numa/LocaleModel.chpl:
- Have chp_here_* allocator and free routines call the chpl_mem_*
  varieties

modules/internal/ChapelArray.chpl:
- Replace call to chpl_mem_free with chpl_here_free

modules/internal/ChapelSyncvar.chpl:
- Don't need special chpl_here* functions any more

compiler/include/symbol.h:
compiler/AST/symbol.cpp:
- Added gChplHereAlloc and gChplHereFree, used for inserting calls
  post-resolution

compiler/include/flags_list.h:
- Added FLAG_LOCALE_MODEL_ALLOC and FLAG_LOCALE_MODEL_FREE

compiler/include/primitive.h:
compiler/AST/primitive.cpp:
compiler/AST/checkAST.cpp:
- Remove PRIM_CHPL_ALLOC and PRIM_CHPL_FREE
- Fixed up return type of PRIM_SIZEOF to be SIZE_TYPE

compiler/include/expr.h:
- remove here_alloc()
- Add insertChplHereAlloc() for post-resolution calls

compiler/AST/expr.cpp:
- Remove PRIM_CHPL_ALLOC and PRIM_CHPL_FREE
- Rewrite callChplHereAlloc, callChplHereFree
- Added insertChplHereAlloc()

compiler/passes/normalize.cpp:
- Call callChplHereFree() rather than creating the call expr
- Set gChplHereAlloc and gChplHereFree

compiler/resolution/wrappers.cpp:
compiler/passes/scopeResolve.cpp:
- Change here_alloc() to callChplHereFree()

compiler/resolution/lowerIterators.cpp:
compiler/passes/parallel.cpp:
- Change chplHereAlloc() to insertChplHereAlloc()

compiler/resolution/functionResolution.cpp:
- Fix up pre-resolution calls inserted for chpl_here_alloc() and
  chpl_free_alloc()

compiler/resolution/callDestructors.cpp:
- Remove no longer relevant comment

compiler/optimizations/narrowWideReferences.cpp:
- Replace PRIM_CHPL_ALLOC/FREE with similar uses of
  FLAG_LOCALE_MODEL_FREE/FLAG_ALLOCATOR

compiler/optimizations/scalarReplace.cpp:
- Replace PRIM_CHPL_ALLOC/FREE with similar uses of
  FLAG_LOCALE_MODEL_FREE/FLAG_ALLOCATOR
- Also removed long commented out check for PRIM_GET_LOCALEID

compiler/optimizations/optimizeOnClauses.cpp:
- Remove PRIM_CHPL_ALLOC/FREE (will be handled elsewhere)




git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@21945 3a8e244f-b0f2-452b-bcba-4c88e055c3ca