Fixed testing isolator and tests to properly handle multiple executors.

Review: https://reviews.apache.org/r/14097