Remove InfiniteWorker from basicClient worker choice.

Basically the server worker factory will fill up with infinite workers which
can't take any of the BasicWorker jobs which makes it look like the server
has stalled but it really hasn't.