Fix crash when trying to clear the worker list

A bug was introduced in f900c2a0.

Fixes #1023.