This research has been supported by research grants from the Natural Sciences and Engineering Research Council of Canada and from the Information Technology Research Centre of Ontario. The data parallel programming model has been widely used in developing scientific applications. These languages provide global name-spaces for ease of programming, but require the programmers to carefully determine data distributions for good performance. On distribut- ed memory multiprocessors, the compiler translates references to global arrays into references to smaller local arrays stored in processors' local memory modules and generates communications for nonlocal accesses. The performance of these programs thus depends on the effectiveness of optimizing communications and global-local index mapping.
Memory locality is thus essential for good performance.
In order to achieve high locality, data distributions must match loop partitioning. Data parallel programming on NUMA machines raises issues dif- ferent from UMA and distributed memory MIMD machines. On UMA machines, memory locality is not an issue so an implementation for an UMA system is not suitable for NUMA machines. The owner computes rule is usually used for gen- erating communications. On NUMA machines, neither explicit com- munications nor the owner computes rule are needed.