mit-crpg/openmc — pull request #208 by paulromano (2013-08-29 01:33:43+00:00)
Title: Shared memory parallelism with OpenMP

@siegela67 and I recently revisited the old 'threading' branch we had worked on circa December 2011 and have created this branch to bring forward those changes to the current version of OpenMC. The major things that were needed to implement OpenMP threading were:

- Add `!$omp parallel do` around the particle loop. Note that we use `schedule(static)`.
- Add the `save` attribute to all pointers created on the stack within a parallel region and additionally make them threadprivate. This is required per the OpenMP standard (see section 2.9.2 of the OpenMP standard).
- Incrementing of tallies and global tallies are enclosed in `!$omp critical` blocks. Originally, we had used atomics since the incrementing operation is a single line, but evidently gfortran does not obey them! Or else I'm just crazy and that's not valid... but I feel like it's the former.
- The `matching_bins` component of TallyObject is now a global variable that is threadprivate. To my knowledge, there is no way to make a single component of a derived type threadprivate.
- Each thread has its own fission bank and a new subroutine `join_bank_from_threads` is used to combine them into a single master fission bank which is then copied to thread 0.
- Make variables that changes during a single particle history threadprivate (e.g., see global, energy_grid, random_lcg modules)
- Make active tally lists threadprivate -- while this doesn't seem necessary at first glance, the problem is that internally the List class keeps track of the last index used... without those lists being threadprivate, there would be a race conditoin on the last index.
- Added OPENMP variable in Makefile that sets proper compile/link flags.

All test results match those generated without threading (Note that two tests fail with Intel compiler, but inspection of the results shows that the answers are off in the very last significant digit).