Repository: huggingface/peft

Issue:
PeftModel.disable_adapter bug
### System Info

Python 3.10.12
peft 0.8.2, transformers 4.37.2, accelerate 0.27.2


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig, TrainingArguments, TrainerCallback
import torch
import numpy as np
import evaluate
import random
from datasets import load_dataset
import re
from transformers.trainer_callback import TrainerControl, TrainerState
from trl import SFTTrainer, DataCollatorForCompletionOnlyLM
from trl.extras.dataset_formatting import get_formatting_func_from_dataset
from peft import get_peft_model, PromptTuningInit, PromptTuningConfig, TaskType

device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "TheBloke/CodeLlama-13B-Instruct-GPTQ"

tokenizer_kwargs = {} 
model_kwargs = {'quantization_config': GPTQConfig(bits=4, exllama_config={"version":2}),
                # 'attn_implementation': 'eager', 
                'device_map': 'auto',
                'torch_dtype': torch.float16,
                'trust_remote_code': False, 
                'revision': "gptq-4bit-32g-actorder_True",
                'do_sample': False}
tokenizer = AutoTokenizer.from_pretrained(model_name, **tokenizer_kwargs)
model = AutoModelForCausalLM.from_pretrained(
model_name,
**model_kwargs
)
tokenizer.pad_token_id = model.config.pad_token_id 

initial_instruction = "Output should be all uppercase."
peft_config = PromptTuningConfig(
    task_type=TaskType.CAUSAL_LM,
    prompt_tuning_init=PromptTuningInit.TEXT,
    num_virtual_tokens=len(tokenizer.tokenize(initial_instruction)),
    prompt_tuning_init_text=initial_instruction,
    tokenizer_name_or_path=model_name,
)
model = get_peft_model(model, peft_config)

messages = [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'Who is Harry Potter?'}]
input_ids1 = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
output = model.generate(input_ids1, do_sample=False, max_new_tokens=20)
print(tokenizer.decode(output[0, input_ids1.shape[-1]:].to("cpu")))

with model.disable_adapter():
    input_text = initial_instruction + ' ' + tokenizer.apply_chat_template(messages, tokenize=False)
    input_ids2 = tokenizer(input_text, return_tensors="pt")["input_ids"].to("cuda")
    output = model.generate(input_ids2, do_sample=False, max_new_tokens=20)
    print(tokenizer.decode(output[0, input_ids2.shape[-1]:].to("cpu")))

input_ids3 = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
output = model.generate(input_ids3, do_sample=False, max_new_tokens=20)
print(tokenizer.decode(output[0, input_ids3.shape[-1]:].to("cpu")))
```

### Expected behavior

All three outputs should be the same: 
"HARRY POTTER is a fictional character in the Harry Potter book series by".
However, after using the disable_adapter context manager prepare_inputs_for_generation is not restored correctly and therefore we get these printouts:

"HARRY POTTER is a fictional character in the Harry Potter book series by"
"HARRY POTTER is a fictional character in the Harry Potter book series by"
"Harry Potter is a fictional character in the Harry Potter book series by J.K"

This seems to be due to a type in the disable_adapter:
```python
			finally:
				...
                self.old_prepare_inputs_for_generation = old_prepare_inputs_for_generation
```

should be this:
```python
			finally:
				...
				self.prepare_inputs_for_generation = old_prepare_inputs_for_generation
```

https://github.com/huggingface/peft/blob/bc9426f10b584a895082ec18ccfd1d56211b0dfa/src/peft/peft_model.py#L573C24-L573C64


Discussion:

Resolving patch:
diff --git a/src/peft/peft_model.py b/src/peft/peft_model.py
--- a/src/peft/peft_model.py
+++ b/src/peft/peft_model.py
@@ -570,7 +570,7 @@ def disable_adapter(self):
         finally:
             if self.peft_config[self.active_adapter].is_prompt_learning:
                 self.forward = old_forward
-                self.old_prepare_inputs_for_generation = old_prepare_inputs_for_generation
+                self.prepare_inputs_for_generation = old_prepare_inputs_for_generation
             else:
                 self.base_model.enable_adapter_layers()
 


Test patch:
diff --git a/tests/testing_common.py b/tests/testing_common.py
--- a/tests/testing_common.py
+++ b/tests/testing_common.py
@@ -1282,6 +1282,11 @@ def get_output(model):
                 output_peft_disabled = get_output(peft_model)
             assert torch.allclose(output_before, output_peft_disabled, atol=1e-6, rtol=1e-6)
 
+            # after leaving the disable_adapter context, the output should be the same as with enabled adapter again
+            # see #1501
+            output_peft_after_disabled = get_output(peft_model)
+            assert torch.allclose(output_peft, output_peft_after_disabled, atol=1e-6, rtol=1e-6)
+
         # TODO: add tests to check if disabling adapters works after calling merge_adapter
 
     def _test_adding_multiple_adapters_with_bias_raises(self, model_id, config_cls, config_kwargs):