Avoid core dump if rpc port is in use.

The cleanup code needs to check for NULL rpcworkers thread group.