Respect rank/thread maximum in CPU reruns

CPU-only reruns done after GPU-enabled tests didn't respect MPI rank
and OpenMP thread count limits.

Minor refactorings to re-use variables that contain the names of the
files that limit attempted parallelism, and to clarify how the
rerun sets up its directories.

Fixes #1624

Change-Id: I58210e37129d725ef2fe2e7b1ff3c2b5ecfb5d74