Fix --dry-run when always_yes is set