Issue 1:
System Info

transformers version: 4.30.0.dev0

Platform: Linux-5.15.107+-x86_64-with-glibc2.31

Python version: 3.10.11

Huggingface_hub version: 0.14.1

Safetensors version: 0.3.1

PyTorch version (GPU?): 2.0.1+cu118 (True)

Tensorflow version (GPU?): 2.12.0 (True)

Flax version (CPU?/GPU?/TPU?): 0.6.9 (gpu)

Jax version: 0.4.10

JaxLib version: 0.4.10

Using GPU in script?:

Using distributed or parallel set-up in script?:

Who can help?

@ArthurZucker @younesbelkada

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

import torch
from transformers import AutoModelForCausalLM, AutoConfig
from transformers import BitsAndBytesConfig

model_name = 'mosaicml/mpt-7b-instruct'

nf4_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16
)

model_nf4 = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, offload_folder="offload",
offload_state_dict = True, quantization_config=nf4_config)

Error

ValueError: MPTForCausalLM does not support `device_map='auto'` yet.

I saw a similar issue #22188 for XGLMForCausalLM. I couldn't find MPTForCausalLM in the repository if the MPT model is not supported currently, is there any hack that I can use to get acclerate support ? Than

Expected behavior

Model gets loaded without any errors

Issue 2:
System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

transformers version: 4.24.0

Platform: Linux-5.15.0-69-generic-x86_64-with-glibc2.31

Python version: 3.10.9

Huggingface_hub version: 0.10.1

PyTorch version (GPU?): 1.12.1 (False)

Tensorflow version (GPU?): not installed (NA)

Flax version (CPU?/GPU?/TPU?): not installed (NA)

Jax version: not installed

JaxLib version: not installed

Using GPU in script?: yes

Using distributed or parallel set-up in script?: yes

Who can help?

@ArthurZucker @younesbelkada

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

Loaded the InstructCodeT5+ model as per https://huggingface.co/Salesforce/instructcodet5p-16b

Tried to use LoRA to fine-tune InstructCodeT5+ on NL to code translation task. Code and Traceback given below:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch
token_id="Salesforce/instructcodet5p-16b"
tokenizer = AutoTokenizer.from_pretrained(token_id)
model = AutoModelForSeq2SeqLM.from_pretrained(token_id,
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
trust_remote_code=True, decoder_start_token_id=1, pad_token_id=-100, load_in_8bit=True, device_map='auto').to(device)

Error:

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-12-1436d2a64ffb> in <module>
10
11 # load model from the hub
---> 12 model = AutoModelForSeq2SeqLM.from_pretrained(model_id,
13 torch_dtype=torch.float16,
14 low_cpu_mem_usage=True,

~/.local/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
460 class_ref, pretrained_model_name_or_path, **hub_kwargs, **kwargs
461 )
--> 462 return model_class.from_pretrained(
463 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
464 )

~/.cache/huggingface/modules/transformers_modules/instructcodet5p-16b/modeling_codet5p.py in from_pretrained(cls, *args, **kwargs)
855 )
856 kwargs["_fast_init"] = False
--> 857 return super().from_pretrained(*args, **kwargs)
858
859 def forward(

~/.local/lib/python3.8/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
2683
2684 if model._no_split_modules is None:
-> 2685 raise ValueError(f"{model.__class__.__name__} does not support `device_map='{device_map}'` yet.")
2686 no_split_modules = model._no_split_modules
2687 if device_map not in ["auto", "balanced", "balanced_low_0", "sequential"]:

ValueError: CodeT5pEncoderDecoderModel does not support `device_map='auto'` yet.

Seems Like this functionality is not included yet. When to expect it to be added? Thanks in advance!

Expected behavior

Expect the model to get loaded without any error