Only cleanup thread array lists if they have been allocated.

Fixes a crash when close and start happen back to back.