Only remove groups that have not been started