Fix a race that could crash a few threads in the input thread pool