Join worker thread before destroying