Delete old peers when someone asks for them