Fixed 2 race conditions in kernel 4.

Fixed 2 race conditions in my kernel 4 implementation that were due to
replicating the level set work queues.  Both required a barrier to
insure that globally visible writes were completed before proceeding
to the next iteration.

This never showed up during nightly testing of the RMAT graph, because
the average degree is largish and the paths are shortish and so we got
lucky.  Torus graph testing hit one or both of the race conditions
every time, since the average degree is small and the paths can be
very long.

After a few nights of successful testing, I will promote this to the
top level version.



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19032 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Fix for Sung's ambiguous module paths future

Prior to this change, directories that were included in the module
search path due to both being part of a filename (e.g.,
subdir/foo.chpl) and a module path setting (e.g., -M subdir) would be
entered into the module path multiple times which would result in a
message saying that the same module was found along multiple paths,
yet the paths would be the same.  This fix addresses that issue by
removing duplicates between these two lists, in particular by adding
directories specified by module path mechanisms (like -M) into the
user's module path only if they're unique.

This change is relatively minor, but has a bigger apparent impact in
the diffs because I took the liberty of renaming things for clarity
and adding some comments as I went.

TODO: document module search path in man page



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19033 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Added in missing barrier per changes Sung just made to SSCA#2 Kernel4


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19034 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Improved documentation of compiler's module search path.

Added a description of how modules are searched for in the
CHPL_MODULE_PATH section of the man page and put some forward
references to it in the -M and --print-search-path descriptions.

For reference, one of the questions that came up in these past few
changes I did is why compiling a file name subdir/foo.chpl should
result in subdir/ getting added to the search path, so I wanted to
capture that rationale here.

In C, if you have the following files:

        subdir/testit.c   // which #includes "testit.h"
        subdir/testit.h

and you compile 'cc subdir/testit.c', testit.h will be found in spite
of the fact that you didn't add -I to the command-line.  We wanted to
support a similar capability in Chapel.  A motivating example is found
in test/release/examples/hpcc/ra.chpl which relies on other helper
modules in that same directory.  It's convenient to be able to do:

        cd test/release/examples/hpcc && chpl ra.chpl
or:
        chpl test/release/examples/hpcc/ra.chpl

without having to add an explicit module search path to the compiler's
command-line.  

Chapel differs from C, though, in that a 'use' is not the same as
'#include' in that the file isn't processed immediately at that point.
Rather it indicates a need to find a module with the given name at
some point, and moreover, for one module globally with that name
across multiple files.  Contrast this with C where each .c file could
have the same #include line and yet get different header files:

        subdir/testit.c   // which #includes "testit.h"
        subdir/testit.h
        subdir2/testit.c  // which #includes "testit.h"
        subdir2/testit.h

When compiling 'cc subdir/testit.c subdir2/testit.c', each .c file
#includes its respective .h file without any conflict.  But for the
equivalent Chapel case:

        subdir/foo.chpl  // contains 'use bar'
        subdir/bar.chpl  // defines 'module bar'
        subdir2/baz.chpl // contains 'use bar'
        subdir2/bar.chpl // defines 'module bar'

the program will only define a single 'bar module and therefore we
have to have a single, consistent, global module search path.  To that
end, we add all subdirectories named by source files to the module
search path, to get the convenience.  We add -I and $CHPL_MODULE_PATH
directories afterwards in a manner that's intended to be consistent
with the C compiler.




git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19035 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Start t4sting using multiple locales


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19036 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Made normalize_returns() idempotent w.r.t. iterators.

git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19037 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Preliminary checkin of addInitGuards.cpp.

git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19038 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Add 2 feature request futures for a module/sub-module warning message.

The following situation is subtle and it would be nice to have a
warning:

     blah.chpl
     ---------
     module blah {
       ...
     };

The semicolon makes the compiler think that it needs to create a new
top-level module, and thus any symbols in the module will not be
visible unless you do a "use blah.blah".

See r18043 for a similar warning for top-level use statements.



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19040 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Added divceil(m,n) (formerly ceil(m,n)), divfloor(m,n), mod(m,n), mod(x,y).

Also:

* added a test to lock the behavior of the integer operations
(the test ensures it matches the corresponding operations on reals)

* modified exercises/histogram* to use divceil()

* renamed a future test

* modified/added entries to the Standard Modules chapter of the spec

* added (undocumented) isNonnegative(i:integral) to ChapelBase.chpl -
it produces a param result whenever possible,
i.e. when the argument is an unsigned int or a param

Explanations: 

divceil/divfloor compute ceiling/floor of the fraction between two integers.
The naming distinguishes them from ceil/floor - single-argument functions.

mod() is a two-argument operation, so 'div' was not prepended.
The definition was taken from wikipedia's entry on "ceiling function".


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19041 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Simple (harmless) change to see if SVN is still with me.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19042 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Trying to commit a copy of this for Tom.

For some reason he seems unable to svn cp this file ('bad gateway' error)



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19043 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Move the runtime utilities into a standard module.

To move chpl_rt_utils.chpl into a module (now called ChapelRT.chpl) was almost
trivial.  It was necessary to place the calls to .add and .remove in serial
statements, to prevent deadlock on pthread_mutex_lock().

The rest of this change involves getting the initialization order correct.  To
do this, I added another compiler pass called addInitGuards, which makes each of
the compiler-generated module initialization functions idempotent.  That allows
us to call chpl__init_chpl__Program in main.c before inserting the first task
into the task table.

TODO: It would be better to call just chpl__init_ChapelRT() in main.c, but to do
this, we have to make each module call the initializtion code for all of the
modules it uses.  Then, calling just the init for the module we're using will
suffice.
I didn't remove the fRuntime flag.  But after this checkin, we don't use it
anymore, so it can be removed.
Internal modules are still being treated specially, but need not be.  After
adding an init call for each use to the init function for the module using it,
all we have to do is implicitly use ChapelStandard at the beginning of the main module.

DETAILS:
Moved chpl_rt_util.chpl from runtime/src/common-chpl to modules/internal/ChapelRT.chpl.
Added the new compiler pass addInitGuards:
    For each module initialization function, add and test an initialized?
    predicate to determine whether to run the initialization code.  This makes
    each such function idempotent.
    As a bookkeeping task, add the function chpl__init_preInit() and populate it
    with code that set each of the initialization predicates initially to
    false.  This routine must be (and is) called before any of the module
    initialization functions.
Instead of calling chpl_int_chpl_rt_utils() in main.c, we call
chpl__init_chpl__Program().  This is a sledgehammer to make sure that
chpl__init_ChapelRT() and any modules on which it depends are called before the
first task is inserted in the task table.  After the above TODOs are completed,
we will call just chpl__init_ChapelRT(), and it will "just work".

Factored initTheProgram() out of initPrimitiveTypes(); the latter was becoming a
catch-all.



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19044 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Fix CHPL_TASKS=none!

When I reverted "none" threading back to its previous vacuous state in r18969,
I failed to make the appropriate changes to "none" tasking.  Do so now.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19045 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Small updates to kernel 4.

- Added comments regarding the barriers and use of replicated level
  sets

- Implemented a small optimization that avoided the reduction over
  Active_Remaining for the first iteration

- Added other comment regarding possible improvements




git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19046 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Fixing a bug in fixing up internal module dependences

Vass pointed out a bug today in our internal module parsing in which
modules that were used indirectly by a standard or internal module
were never parsed in due to a poor assumption that a single pass over
unresolved modules would catch them all.  This change fixes that by
putting the code that resolves internal modules into a loop until no
unsatisfied dependences are found.



git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19047 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Do various tcmalloc related cleanup.

Add tcmalloc to the clean and clobber targets.  Remove some leftover status
reporting.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19048 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Make some of the tasking interface functional earlier.

In the FIFO tasking implementation, several of the interface functions require
that thread private storage be set up in order for them to work.  Among these
are chpl_task_getId(), chpl_task_getSerial(), and chpl_task_setSerial().  We
have been setting up thread private storage in chpl_task_callMain().  But
there has been recent demand for being able to call chpl_task_getId() earlier,
and the movement of the runtime utility code into a standard module (r19044)
also requires chpl_task_setSerial() earlier.  So, now we initialize the thread
private data in chpl_task_init().


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19049 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Added some debugging facilities to the compiler.

* Added a flag to invoke gdbShouldBreakHere() when removing a node
  with the desired ID (see trace_remove() below).
  Added the corresponding variable.

    --break-on-delete-id
    breakOnDeleteID

* Added a flag to log the IDs of deleted nodes into a file.
  (See trace_remove() below.)

    --log-deleted-ids-to <filename>

* Added a logging function to be invoked whenever an AST node is deleted.
  It records the node ID, address, as well as the flag and currentPassNo,
  in the log file (if requested) and invokes gdbShouldBreakHere()
  if the node ID matches breakOnDeleteID (if specified).
  This function is for debugging purposes only.

    void trace_remove(BaseAST* ast, char flag)

  Currently it is invoked in several cases, with the following flags:

    r  Expr::remove()
    R  Expr::remove() but this node has already been remove'ed before
    p  Expr::replace() (this node is removed there)
    x  cleanAst() - garbage-collecting the nodes not in AST
    z  destroyAst() - deleting the entire AST at compiler tear-down

  This could be used with valgrind to convert the memory addresses it reports
  to node IDs.

* Added two global variables that track the current pass, for debugging
  (and perhaps for HTML generation). They are kept current by runPasses().

    int currentPassNo;
    const char* currentPassName;

* Upon --print-passes, also print the current max AST id at the end of each
  pass, when in the developer mode.

* Moved logfile open/close operations into separate functions:
  from runPasses() into setupLocales(), teardownLogfiles().


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19050 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Replace maxThreadsPerLocale with numThreadsPerLocale

This commit replaces the config const 'maxThreadsPerLocale' with
'numThreadsPerLocale' in order to make it more useful for tasking
layers that don't have a varying number of tasks per locale, such as
qthreads and nanox.  For tasking layers that do, such as fifo, it
still serves as a user-specified maximum.  Updated README.tasks (and
generally took a pass over its contents) and README.executing to
reflect this change and document the default behavior for each tasking
layer.

There is still internally a maxThreadsPerLocale-style variable that is
used to store the upper limit on the number of threads to used,
currently as defined by the communication layer (e.g., GASNet's limit
of 255 or 127 threads).  To mark it as internal, I renamed it to
chpl__maxThreadsPerLocale, however.  I also removed the functions in
the tasking layer that let it weigh in on what the maximum should be
because (a) the tasking layers typically did not use these routines,
and (b) if they do have a preference, they can take it into account
directly in chpl_task_init(), rather than passing it to themselves via
a somewhat convoluted path.

To that end, I updated the chpl_task_init() call to take these two
values, as well as an indication of whether the communication layer
was planning on making a polling task, and the pre-existing
callStackSize constant.  Updated all tasking layers and communication
layers to reflect these interface changes.  Our threading layer
interface also changed to take the numThreads/maxThreads values so I
updated those as well.

Updated all tests that referred to maxThreadsPerLocale to refer to
numThreadsPerLocale instead.

Removed primitives that used to be used for these purposes, replacing
them with extern routines.

Renamed chpl_numCoresPerLocale() to chpl_numCoresOnThisLocale() to
reflect the fact that different locales may have different numbers of
processors.  I also moved the caching of this value to avoid calling
into the function too often into the C code for direct calls to it
(such as those we make in the runtime).

Switched instances of _extern in the modules I was touching to extern.




git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19051 3a8e244f-b0f2-452b-bcba-4c88e055c3ca
Remove fRuntime flag from compiler.

DETAILS:
    compiler/include/driver.h
    compiler/main/driver.cpp
Remove fRuntime global. 

    compiler/include/files.h
    compiler/passes/codegen.cpp
    compiler/util/files.cpp
Remove runtime flag from openCFile() and appendCFile().

    compiler/AST/symbol.cpp
    compiler/AST/expr.cpp
    compiler/AST/type.cpp
    compiler/passes/normalize.cpp
    compiler/passes/codegen.cpp
    compiler/passes/filesToAST.cpp
    compiler/passes/parallel.cpp
    compiler/passes/buildDefaultFunctions.cpp
    compiler/resolution/callDestructors.cpp
    compiler/util/files.cpp
Replace fRuntime global with "false" and reduce logical expressions.

    make/Makefile.base
Remove RUNTIME_CHPL_FLAGS.


git-svn-id: http://svn.code.sf.net/p/chapel/code/trunk@19052 3a8e244f-b0f2-452b-bcba-4c88e055c3ca