Fix: poller shouldn't remove things from the buildqueue when restarting