Fix parallel state exit bug and add a test case