get_peft_model with PEFT Finetuning for Tensorflow fails
Author: SrikanthChellappa
Pull request: no

### System Info

Transformers Version:  4.34.1
Python: 3.11.3

### Who can help?

@Rocketknight1  @Gante

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

My code
model_name='google/flan-t5-base'
original_model = TFAutoModelForSeq2SeqLM.from_pretrained(model_name, dtype=tf.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_name)
from peft import LoraConfig, get_peft_model, TaskType
lora_config = LoraConfig(
    r=32, # Rank
    lora_alpha=32,
    target_modules=["q", "v"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.SEQ_2_SEQ_LM # FLAN-T5
)
peft_model = get_peft_model(original_model, lora_config)
print(print_number_of_trainable_model_parameters(peft_model))


Error Stack

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[8], line 1
----> 1 peft_model = get_peft_model(original_model, lora_config)
      2 #print(print_number_of_trainable_model_parameters(peft_model))

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\mapping.py:106, in get_peft_model(model, peft_config, adapter_name)
    104 if peft_config.is_prompt_learning:
    105     peft_config = _prepare_prompt_learning_config(peft_config, model_config)
--> 106 return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name)

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\peft_model.py:1056, in PeftModelForSeq2SeqLM.__init__(self, model, peft_config, adapter_name)
   1055 def __init__(self, model, peft_config: PeftConfig, adapter_name="default"):
-> 1056     super().__init__(model, peft_config, adapter_name)
   1057     self.base_model_prepare_inputs_for_generation = self.base_model.prepare_inputs_for_generation
   1058     self.base_model_prepare_encoder_decoder_kwargs_for_generation = (
   1059         self.base_model._prepare_encoder_decoder_kwargs_for_generation
   1060     )

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\peft_model.py:111, in PeftModel.__init__(self, model, peft_config, adapter_name)
    109 if not peft_config.is_prompt_learning:
    110     self.peft_config[adapter_name] = peft_config
--> 111     self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type](
    112         self.base_model, self.peft_config, adapter_name
    113     )
    114     self.set_additional_trainable_modules(peft_config, adapter_name)
    115 else:

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\tuners\lora.py:274, in LoraModel.__init__(self, model, config, adapter_name)
    273 def __init__(self, model, config, adapter_name) -> None:
--> 274     super().__init__(model, config, adapter_name)

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\tuners\tuners_utils.py:88, in BaseTuner.__init__(self, model, peft_config, adapter_name)
     85 if not hasattr(self, "config"):
     86     self.config = {"model_type": "custom"}
---> 88 self.inject_adapter(self.model, adapter_name)
     90 # Copy the peft_config in the injected model.
     91 self.model.peft_config = self.peft_config

File C:\ProgramData\anaconda3\envs\tfenv\Lib\site-packages\peft\tuners\tuners_utils.py:199, in BaseTuner.inject_adapter(self, model, adapter_name)
    196 self._check_new_adapter_config(peft_config)
    198 is_target_modules_in_base_model = False
--> 199 key_list = [key for key, _ in model.named_modules()]
    201 model_config = getattr(model, "config", {"model_type": "custom"})
    202 if hasattr(model_config, "to_dict"):

AttributeError: 'TFT5ForConditionalGeneration' object has no attribute 'named_modules'


### Expected behavior

A peft model should be created

Comments:
Hi @SrikanthChellappa, thanks for raising this issue! 

I believe that PEFT is currently only supported for PyTorch models, but @younesbelkada can confirm

Yes I second what @amyeroberts said, you can't use PEFT + Tensorfllow, PEFT only supports PyTorch

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Yes I second what @amyeroberts said, you can't use PEFT + Tensorfllow, PEFT only supports PyTorch

can we expect tf support sooner?

Same question. Can we expect tf support sooner?