added logic for deleting adapters once loaded
Author: itsskofficial
Pull request: yes

What does this PR do?
=====================

This PR introduces new functionality to support deletion of adapter layers in PEFT (Parameter-Efficient Fine-Tuning) integrations within the Hugging Face Transformers library. Specifically, it adds methods to remove individual or multiple adapters, allowing for more flexible and modular management of adapters.

### Description

The `delete_adapter_layers` and `delete_adapters` functions remove specified LoRA (Low-Rank Adaptation) adapter layers from a model. This feature addresses cases where users need to manage memory and computational efficiency by unloading adapters that are no longer in use. Additionally, the PR includes logic to update the `peft_config` attribute, ensuring that the configuration reflects the current state of loaded adapters and that any unnecessary configurations are cleared when all adapters are deleted.

### Motivation

This functionality supports scenarios where users load multiple adapters during fine-tuning or inference but want to unload specific adapters without reloading the model entirely. This enhancement is particularly beneficial for memory-constrained environments, such as inference pipelines on limited hardware.

### Related Issues

#34649

### Dependencies

This feature requires a PEFT version greater than 0.6.1 for compatibility with the `BaseTunerLayer` interface.

### Example Usage

```python
peft_base_model.load_adapter("adapters/context-classification", adapter_name="context_classification")
peft_base_model.set_adapter("context_classification")

classification_prompt = context_classification_prompt_template.format(message)
input_ids = peft_tokenizer(classification_prompt, return_tensors="pt").to("cuda")
outputs = peft_base_model.generate(**input_ids, max_new_tokens=64, use_cache = False)
classification = peft_tokenizer.decode(outputs[0], skip_special_tokens=True)

peft_base_model.delete_adapters("context_classification")
```

Before submitting
-----------------

*    This PR has been discussed and reviewed to ensure its compatibility with PEFT integrations.
*    Documentation has been updated to reflect the new adapter deletion functionality.
*    Tests have been written to validate that adapters are deleted successfully, and configurations update as expected.

Who can review?
---------------

Community members and maintainers experienced with PEFT, LoRA, or model integrations are invited to review.

*   @zucchini-nlp for LoRA integration and adapter management
*   @Rocketknight1 for PEFT and integration review

Comments:
@vasqu sorry for the oversight. It seems I was using a bit older version of  transformers. I have retained the current working code and merged the two functions into one. But doing that unexpectedly breaks the tests which were passing before

@itsskofficial No worries! Thanks for working on this so quickly. I think only the import is an issue atm.

Could you also add some test that checks if the config/modules are handled properly after deletion? 

Yes sure @vasqu. Will review your changes and add some checks

Alright @BenjaminBossan, thanks for the suggestions. Will check them and make changes accordingly

@BenjaminBossan I have made changes as requested in the main file. Also, I have added a test for deleting adapter(s) referring the earlier ones as recommended by you. Please check and feel free to make any improvements

Can you run `make style`, should fix the code quality in the ci. The other failure seems unrelated to me.

Thx for working on this 🫡 

> Can you run `make style`, should fix the code quality in the ci. The other failure seems unrelated to me.
> 
> Thx for working on this 🫡

@vasqu I applied formatting with black, and resolved the code quality errors. Still, the test fails with the following error

![image](https://github.com/user-attachments/assets/d96942aa-315c-4352-9a28-bfbc2565657a)

> > Can you run `make style`, should fix the code quality in the ci. The other failure seems unrelated to me.
> > Thx for working on this 🫡
> 
> @vasqu I applied formatting with black, and resolved the code quality errors. Still, the test fails with the following error
> 
> ![image](https://private-user-images.githubusercontent.com/65887545/390318242-d96942aa-315c-4352-9a28-bfbc2565657a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzI2OTY0MTcsIm5iZiI6MTczMjY5NjExNywicGF0aCI6Ii82NTg4NzU0NS8zOTAzMTgyNDItZDk2OTQyYWEtMzE1Yy00MzUyLTlhMjgtYmZiYzI1NjU2NTdhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDExMjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMTI3VDA4MjgzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA4NjU4YzY1MzVkY2JhYTFhNzA2ZTMwOTRkMzFiMmM2N2EwOTUyYTU3ZjQ5MjM3NGVhYTA5ZGVhZDI3Y2NiNTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.YOGJHwt-GxxjmlEtaXhNL4-1K48irkZzwKOZbATmtcg)

This is done, a new error is troubling me now 
```
=================================== FAILURES ===================================

_______ FuyuModelTest.test_prompt_lookup_decoding_matches_greedy_search ________

[gw1] linux -- Python 3.10.15 /usr/local/bin/python3



self = <tests.models.fuyu.test_modeling_fuyu.FuyuModelTest testMethod=test_prompt_lookup_decoding_matches_greedy_search>



    @pytest.mark.generate

    def test_prompt_lookup_decoding_matches_greedy_search(self):

        # This test ensures that the prompt lookup generation does not introduce output changes over greedy search.

        # This test is mostly a copy of test_assisted_decoding_matches_greedy_search

    

        for model_class in self.all_generative_model_classes:

            if model_class._is_stateful:

                self.skipTest(reason="Stateful models don't support assisted generation")

            if any(model_name in model_class.__name__.lower() for model_name in ["fsmt", "reformer"]):

                self.skipTest(reason="Won't fix: old model with different cache format")

            if any(

                model_name in model_class.__name__.lower()

                for model_name in [

                    "bigbirdpegasus",

                    "led",

                    "mega",

                    "moshi",

                    "speech2text",

                    "git",

                    "prophetnet",

                    "seamlessm4t",

                    "clvp",

                ]

            ):

                self.skipTest(reason="May fix in the future: need model-specific fixes")

    

            # enable cache

            config, inputs_dict = self.prepare_config_and_inputs_for_generate(batch_size=1)

    

            # NOTE: assisted generation only works with cache on at the moment.

            if not hasattr(config, "use_cache"):

                self.skipTest(reason=f"{model_class.__name__} doesn't support caching")

    

            config.is_decoder = True

            model = model_class(config).to(torch_device).eval()

            # Sets assisted generation arguments such that:

            # a) no EOS is generated, to ensure generation doesn't break early

            # b) the prompt lookup tries to give the model 2 tokens, to ensure the input preparation of

            #    prompt lookup is correct

            # c) there are at least two forward passes in the main model, to ensure the input preparation of

            #    the main model is correct

            generation_kwargs = {

                "eos_token_id": -1,  # see a)

                "max_new_tokens": 4,  # see c)

                "num_beams": 1,

                "do_sample": False,

                "output_scores": True,

                "output_logits": True,

                "output_hidden_states": True,

                "output_attentions": self.has_attentions,

                "return_dict_in_generate": True,

                "use_cache": True,

            }

    

            output_greedy = model.generate(**generation_kwargs, **inputs_dict)

    

            generation_kwargs.update({"prompt_lookup_num_tokens": 2})  # see b)

>           output_prompt_lookup = model.generate(**generation_kwargs, **inputs_dict)



tests/generation/test_utils.py:1233: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py:116: in decorate_context

    return func(*args, **kwargs)

/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py:2197: in generate

    result = self._assisted_decoding(

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 



self = FuyuForCausalLM(

  (language_model): PersimmonForCausalLM(

    (model): PersimmonModel(

      (embed_tokens): Embeddin...es=32, out_features=99, bias=False)

  )

  (vision_embed_tokens): Linear(in_features=2700, out_features=32, bias=True)

)

input_ids = tensor([[69, 74, 19, 46, 11, 12, 36, 89, 36]])

candidate_generator = <transformers.generation.candidate_generator.PromptLookupCandidateGenerator object at 0x7fa9e287e590>

logits_processor = []

stopping_criteria = [<transformers.generation.stopping_criteria.MaxLengthCriteria object at 0x7fa9e287e2c0>, <transformers.generation.stopping_criteria.EosTokenCriteria object at 0x7fa9e287c100>]

generation_config = <[TypeError('Object of type Tensor is not JSON serializable') raised in repr()] GenerationConfig object at 0x7fa9e287d390>

synced_gpus = False, streamer = None

model_kwargs = {'attention_mask': tensor([[1, 1, 1, 1, 0, 1, 0, 1, 1]]), 'cache_position': tensor([8]), 'past_key_values': ((tensor([...          [ 0.0793,  0.0195, -0.0202, -0.0133,  0.0630, -0.0188, -0.1492,

           -0.1713]]]]))), 'use_cache': True}

do_sample = False, output_attentions = True, output_hidden_states = True

output_scores = True, output_logits = True, return_dict_in_generate = True

scores = (tensor([[-1.1893e-01, -1.9729e-01, -4.0989e-02, -9.5035e-02, -7.9376e-02,

         -7.2277e-02,  5.4078e-02,  4.0602e...         -0.0534,  0.0827,  0.1233, -0.1471,  0.2120, -0.0927,  0.0287, -0.0936,

          0.1425, -0.0945,  0.1491]]))



    def _assisted_decoding(

        self,

        input_ids: torch.LongTensor,

        candidate_generator: CandidateGenerator,

        logits_processor: LogitsProcessorList,

        stopping_criteria: StoppingCriteriaList,

        generation_config: GenerationConfig,

        synced_gpus: bool,

        streamer: Optional["BaseStreamer"],

        **model_kwargs,

    ) -> Union[GenerateNonBeamOutput, torch.LongTensor]:

        r"""

        Generates sequences of token ids for models with a language modeling head using **greedy decoding** or

        **sample** (depending on do_sample), assisted by candidate sequences. Assisted generation is an example of a

        candidate decoding strategy. Can be used for text-decoder, text-to-text, speech-to-text, and vision-to-text

        models.

    

        Parameters:

            input_ids (torch.LongTensor of shape (batch_size, sequence_length)):

                The sequence used as a prompt for the generation.

            candidate_generator (CandidateGenerator):

                A derived instance of [CandidateGenerator] that defines how candidate sequences are generated. For

                more information, the documentation of [CandidateGenerator] should be read.

            logits_processor (LogitsProcessorList):

                An instance of [LogitsProcessorList]. List of instances of class derived from [LogitsProcessor]

                used to modify the prediction scores of the language modeling head applied at each generation step.

            stopping_criteria (StoppingCriteriaList):

                An instance of [StoppingCriteriaList]. List of instances of class derived from [StoppingCriteria]

                used to tell if the generation loop should stop.

            generation_config ([~generation.GenerationConfig]):

                The generation configuration to be used as parametrization of the decoding method.

            synced_gpus (bool):

                Whether to continue running the while loop until max_length (needed to avoid deadlocking with

                FullyShardedDataParallel and DeepSpeed ZeRO Stage 3).

            streamer (BaseStreamer, *optional*):

                Streamer object that will be used to stream the generated sequences. Generated tokens are passed

                through streamer.put(token_ids) and the streamer is responsible for any further processing.

            model_kwargs:

                Additional model specific keyword arguments will be forwarded to the forward function of the model.

                If model is an encoder-decoder model the kwargs should include encoder_outputs.

    

        Return:

            [~generation.GenerateDecoderOnlyOutput], [~generation.GenerateEncoderDecoderOutput] or

            torch.LongTensor: A torch.LongTensor containing the generated tokens (default behaviour) or a

            [~generation.GenerateDecoderOnlyOutput] if model.config.is_encoder_decoder=False and

            return_dict_in_generate=True or a [~generation.GenerateEncoderDecoderOutput] if

            model.config.is_encoder_decoder=True.

        """

        # init values

        do_sample = generation_config.do_sample

        output_attentions = generation_config.output_attentions

        output_hidden_states = generation_config.output_hidden_states

        output_scores = generation_config.output_scores

        output_logits = generation_config.output_logits

        return_dict_in_generate = generation_config.return_dict_in_generate

    

        # init attention / hidden states / scores tuples

        scores = () if (return_dict_in_generate and output_scores) else None

        raw_logits = () if (return_dict_in_generate and output_logits) else None

        decoder_attentions = () if (return_dict_in_generate and output_attentions) else None

        cross_attentions = () if (return_dict_in_generate and output_attentions) else None

        decoder_hidden_states = () if (return_dict_in_generate and output_hidden_states) else None

    

        # if model is an encoder-decoder, retrieve encoder attention weights and hidden states

        if return_dict_in_generate and self.config.is_encoder_decoder:

            encoder_attentions = model_kwargs["encoder_outputs"].get("attentions") if output_attentions else None

            encoder_hidden_states = (

                model_kwargs["encoder_outputs"].get("hidden_states") if output_hidden_states else None

            )

    

        # keep track of which sequences are already finished

        batch_size = input_ids.shape[0]

        unfinished_sequences = torch.ones(batch_size, dtype=torch.long, device=input_ids.device)

        model_kwargs = self._get_initial_cache_position(input_ids, model_kwargs)

    

        this_peer_finished = False

        is_first_iteration = True  # to preserve the same API in the output as other generation methods

        while self._has_unfinished_sequences(this_peer_finished, synced_gpus, device=input_ids.device):

            cur_len = input_ids.shape[-1]

    

            #  1. Fetch candidate sequences from a CandidateGenerator

            candidate_input_ids, candidate_logits = candidate_generator.get_candidates(input_ids)

    

            if candidate_logits is not None:

                candidate_logits = candidate_logits.to(self.device)

    

            candidate_length = candidate_input_ids.shape[1] - input_ids.shape[1]

            is_done_candidate = stopping_criteria(candidate_input_ids, None)

    

            # 2. Use the original model to obtain the next token logits given the candidate sequence. We obtain

            # candidate_length + 1 relevant logits from this process: in the event that all candidates are correct,

            # we use this forward pass to also pick the subsequent logits in the original model.

    

            # 2.1. Prepare the model inputs

            candidate_kwargs = copy.copy(model_kwargs)

            candidate_kwargs = _prepare_attention_mask(

                candidate_kwargs, candidate_input_ids.shape[1], self.config.is_encoder_decoder

            )

            candidate_kwargs = _prepare_token_type_ids(candidate_kwargs, candidate_input_ids.shape[1])

            if "cache_position" in candidate_kwargs:

                candidate_kwargs["cache_position"] = torch.cat(

                    (

                        candidate_kwargs["cache_position"],

                        torch.arange(cur_len, cur_len + candidate_length, device=input_ids.device, dtype=torch.long),

                    ),

                    dim=0,

                )

    

            model_inputs = self.prepare_inputs_for_generation(candidate_input_ids, **candidate_kwargs)

            if "num_logits_to_keep" in model_inputs:

                model_inputs["num_logits_to_keep"] = candidate_length + 1

    

            # 2.2. Run a forward pass on the candidate sequence

            # prepare variable output controls (note: some models won't accept all output controls)

            model_inputs.update({"output_attentions": output_attentions} if output_attentions else {})

            model_inputs.update({"output_hidden_states": output_hidden_states} if output_hidden_states else {})

    

            outputs = self(**model_inputs)

    

            # 2.3. Process the new logits

            # .float() is needed to retain precision for later logits manipulations

            new_logits = outputs.logits[:, -candidate_length - 1 :].float()  # excludes the input prompt if present

            new_logits = new_logits.to(input_ids.device)

            next_token_logits = new_logits.clone()

            if len(logits_processor) > 0:

                for i in range(candidate_length + 1):

                    new_logits[:, i, :] = logits_processor(candidate_input_ids[:, : cur_len + i], new_logits[:, i, :])

    

            # 3. Select the accepted tokens. There are two possible cases:

            # Case 1: do_sample=True and we have logits for the candidates (originally from speculative decoding)

            # 👉 Apply algorithm 1 from the speculative decoding paper (https://arxiv.org/pdf/2211.17192.pdf).

            if do_sample and candidate_logits is not None:

                valid_tokens, n_matches = _speculative_sampling(

                    candidate_input_ids,

                    candidate_logits,

                    candidate_length,

                    new_logits,

                    is_done_candidate,

                )

    

            # Case 2: all other cases (originally from assisted generation) 👉 Compare the tokens selected from the

            # original model logits with the candidate tokens. We can keep the candidate tokens until the first

            # mismatch, or until the max length is reached.

            else:

                if do_sample:

                    probs = new_logits.softmax(dim=-1)

                    selected_tokens = torch.multinomial(probs[0, :, :], num_samples=1).squeeze(1)[None, :]

                else:

                    selected_tokens = new_logits.argmax(dim=-1)

    

                candidate_new_tokens = candidate_input_ids[:, cur_len:]

>               n_matches = ((~(candidate_new_tokens == selected_tokens[:, :-1])).cumsum(dim=-1) < 1).sum()

E               RuntimeError: The size of tensor a (2) must match the size of tensor b (0) at non-singleton dimension 1



/usr/local/lib/python3.10/site-packages/transformers/generation/utils.py:4336: RuntimeError

----------------------------- Captured stderr call -----------------------------

eos_token_id should consist of positive integers, but is tensor([-1]). Your generation will not stop until the maximum length is reached. Depending on other flags, it may even crash.

eos_token_id should consist of positive integers, but is tensor([-1]). Your generation will not stop until the maximum length is reached. Depending on other flags, it may even crash.

=============================== warnings summary ===============================

tests/models/olmo/test_modeling_olmo.py::OlmoModelTest::test_beam_search_low_memory

tests/models/gpt_neo/test_modeling_gpt_neo.py::GPTNeoModelTest::test_beam_search_low_memory

tests/models/mllama/test_modeling_mllama.py::MllamaForCausalLMModelTest::test_beam_search_low_memory

tests/models/phi/test_modeling_phi.py::PhiModelTest::test_beam_search_low_memory

tests/models/llama/test_modeling_llama.py::LlamaModelTest::test_beam_search_low_memory

tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationModelTest::test_beam_search_low_memory

  /usr/local/lib/python3.10/site-packages/transformers/generation/utils.py:4694: FutureWarning: num_hidden_layers is deprecated and will be removed in version 4.47.0 for DynamicCache.from_batch_splits.

    return DynamicCache.from_batch_splits(data, num_hidden_layers=num_hidden_layers)



tests/models/biogpt/test_modeling_biogpt.py::BioGptModelTest::test_generate_continue_from_past_key_values

tests/models/olmo/test_modeling_olmo.py::OlmoModelTest::test_generate_continue_from_past_key_values

tests/models/gpt_neo/test_modeling_gpt_neo.py::GPTNeoModelTest::test_generate_continue_from_past_key_values

tests/models/phi/test_modeling_phi.py::PhiModelTest::test_generate_continue_from_past_key_values

tests/models/mllama/test_modeling_mllama.py::MllamaForCausalLMModelTest::test_generate_continue_from_past_key_values

tests/models/llama/test_modeling_llama.py::LlamaModelTest::test_generate_continue_from_past_key_values

tests/models/fuyu/test_modeling_fuyu.py::FuyuModelTest::test_generate_continue_from_past_key_values

  /usr/local/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:582: UserWarning: pad_token_id should be positive but got -1. This will cause errors when batch generating, if there is padding. Please set pad_token_id explicitly as model.generation_config.pad_token_id=PAD_TOKEN_ID to avoid errors in generation

    warnings.warn(



tests/models/switch_transformers/test_modeling_switch_transformers.py::SwitchTransformersModelTest::test_beam_search_low_memory

  /usr/local/lib/python3.10/site-packages/transformers/generation/utils.py:4696: FutureWarning: num_hidden_layers is deprecated and will be removed in version 4.47.0 for EncoderDecoderCache.from_batch_splits.

    return EncoderDecoderCache.from_batch_splits(data, num_hidden_layers=num_hidden_layers)



-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html```


This is the test that has been failing

How do I pass this failing check @BenjaminBossan @vasqu?

> How do I pass this failing check

I think this test is failing for unrelated reasons and can be ignored, but I'm not super familiar with the transformers CI.

Tbh the failure seems unrelated to me, possibly flaky tests - you could rebase/merge and check if you get lucky.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34650). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This is ready to merge right @BenjaminBossan?

Yes, I think so @itsskofficial. However, I'm not a core maintainer of transformers, so we need to wait for the approval of one of the core maintainers before merging. Pinging @ArthurZucker for review or for suggesting another maintainer to review.

Ping @Rocketknight1.

@Rocketknight1 are these changes good to merge?

Will review, sorry for the delayed updates! 🤗 

> Will review, sorry for the delayed updates! 🤗

No problem, thanks for approving

@Rocketknight1

Merging since we have an approving review!

> # What does this PR do?
> This PR introduces new functionality to support deletion of adapter layers in PEFT (Parameter-Efficient Fine-Tuning) integrations within the Hugging Face Transformers library. Specifically, it adds methods to remove individual or multiple adapters, allowing for more flexible and modular management of adapters.

Thanks @Rocketknight1 for the merge. Also thank you to @vasqu @BenjaminBossan and @ArthurZucker for helping me out. Looking forward to more opportunities to collaborate