Fix crash when trying to clear the worker list.

Fixes #1023.
Pass an array to srem, but only if non-empty.
Use !empty? instead of any?
Merge pull request #1024 from kek/master

Fix crash when trying to clear the worker list