Fix intermittent failure in conversion_worker_spec