Add a "pause" option that re-adds removed backends after a confirmation