AttributeError: module 'torch' has no attribute 'float8_e4m3fn'
Author: Perpetue237
Pull request: no
Labels: name: bug

### System Info

AutoModelForCausalLM.from_pretrained now returns the above error.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

pip install transformers

from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained(
            model_path,
            cache_dir= cache_dir, 
            device_map="auto",
            quantization_config=bnb_config,
        )


### Expected behavior

Expected to load the model.

Comments:
Hi @Perpetue237, thanks for raising an issue!

Please make sure to follow the issue template and provide: 
* The running environment: run `transformers-cli env` in the terminal and copy-paste the output
* A minimal reproducible code snippet. In this case, as `model_path`, and `bnb_config` are undefined, we cannot run the code to reproduce the error
* All information relevant to the error, including full error traceback 

**1. transformers-cli env**
  
  Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.
  
  - `transformers` version: 4.43.1
  - Platform: Linux-5.4.0-146-generic-x86_64-with-glibc2.31
  - Python version: 3.10.11
  - Huggingface_hub version: 0.24.0
  - Safetensors version: 0.4.3
  - Accelerate version: 0.32.1
  - Accelerate config:    not found
  - PyTorch version (GPU?): 2.0.1 (True)
  - Tensorflow version (GPU?): not installed (NA)
  - Flax version (CPU?/GPU?/TPU?): not installed (NA)
  - Jax version: not installed
  - JaxLib version: not installed
  - Using distributed or parallel set-up in script?: <fill in>
  - Using GPU in script?: <fill in>
  - GPU type: NVIDIA A100-SXM4-80GB
  
  **2. Code snippet:**
```
 from transformers import BitsAndBytesConfig, AutoModelForCausalLM
 import torch

 model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct"
 bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
 model = AutoModelForCausalLM.from_pretrained(
     model_name,
     device_map="auto",
     quantization_config=bnb_config
)
```

3. The code works well with transformers==4.42.4


@Perpetue237 Can you share the full error traceback as well please? 

cc @ArthurZucker  


---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], [line 8](vscode-notebook-cell:?execution_count=1&line=8)
      [4](vscode-notebook-cell:?execution_count=1&line=4) model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct"
      [7](vscode-notebook-cell:?execution_count=1&line=7) bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
----> [8](vscode-notebook-cell:?execution_count=1&line=8) model = AutoModelForCausalLM.from_pretrained(
      [9](vscode-notebook-cell:?execution_count=1&line=9)     model_name,
     [10](vscode-notebook-cell:?execution_count=1&line=10)     device_map="auto",
     [11](vscode-notebook-cell:?execution_count=1&line=11)     quantization_config=bnb_config
     [12](vscode-notebook-cell:?execution_count=1&line=12) )

File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:564, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    [562](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:562) elif type(config) in cls._model_mapping.keys():
    [563](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:563)     model_class = _get_model_class(config, cls._model_mapping)
--> [564](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:564)     return model_class.from_pretrained(
    [565](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:565)         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    [566](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:566)     )
    [567](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:567) raise ValueError(
    [568](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:568)     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    [569](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:569)     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    [570](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:570) )

File /opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3903, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   [3893](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3893)     if dtype_orig is not None:
   [3894](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3894)         torch.set_default_dtype(dtype_orig)
   [3896](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3896)     (
   [3897](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3897)         model,
   [3898](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3898)         missing_keys,
   [3899](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3899)         unexpected_keys,
   [3900](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3900)         mismatched_keys,
   [3901](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3901)         offload_index,
   [3902](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3902)         error_msgs,
-> [3903](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3903)     ) = cls._load_pretrained_model(
   [3904](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3904)         model,
   [3905](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3905)         state_dict,
   [3906](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3906)         loaded_state_dict_keys,  # XXX: rename?
   [3907](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3907)         resolved_archive_file,
   [3908](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3908)         pretrained_model_name_or_path,
   [3909](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3909)         ignore_mismatched_sizes=ignore_mismatched_sizes,
   [3910](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3910)         sharded_metadata=sharded_metadata,
   [3911](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3911)         _fast_init=_fast_init,
   [3912](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3912)         low_cpu_mem_usage=low_cpu_mem_usage,
   [3913](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3913)         device_map=device_map,
   [3914](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3914)         offload_folder=offload_folder,
   [3915](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3915)         offload_state_dict=offload_state_dict,
   [3916](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3916)         dtype=torch_dtype,
   [3917](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3917)         hf_quantizer=hf_quantizer,
   [3918](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3918)         keep_in_fp32_modules=keep_in_fp32_modules,
   [3919](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3919)         gguf_path=gguf_path,
   [3920](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3920)     )
   [3922](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3922) # make sure token embedding weights are still tied if needed
   [3923](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:3923) model.tie_weights()

File /opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4377, in PreTrainedModel._load_pretrained_model(cls, model, state_dict, loaded_keys, resolved_archive_file, pretrained_model_name_or_path, ignore_mismatched_sizes, sharded_metadata, _fast_init, low_cpu_mem_usage, device_map, offload_folder, offload_state_dict, dtype, hf_quantizer, keep_in_fp32_modules, gguf_path)
   [4373](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4373)                 set_module_tensor_to_device(
   [4374](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4374)                     model_to_load, key, "cpu", torch.empty(*param.size(), dtype=dtype)
   [4375](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4375)                 )
   [4376](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4376)     else:
-> [4377](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4377)         new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
   [4378](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4378)             model_to_load,
   [4379](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4379)             state_dict,
   [4380](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4380)             loaded_keys,
   [4381](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4381)             start_prefix,
   [4382](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4382)             expected_keys,
   [4383](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4383)             device_map=device_map,
   [4384](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4384)             offload_folder=offload_folder,
   [4385](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4385)             offload_index=offload_index,
   [4386](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4386)             state_dict_folder=state_dict_folder,
   [4387](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4387)             state_dict_index=state_dict_index,
   [4388](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4388)             dtype=dtype,
   [4389](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4389)             hf_quantizer=hf_quantizer,
   [4390](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4390)             is_safetensors=is_safetensors,
   [4391](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4391)             keep_in_fp32_modules=keep_in_fp32_modules,
   [4392](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4392)             unexpected_keys=unexpected_keys,
   [4393](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4393)         )
   [4394](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4394)         error_msgs += new_error_msgs
   [4395](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4395) else:
   [4396](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:4396)     # Sharded checkpoint or whole but low_cpu_mem_usage==True

File /opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:871, in _load_state_dict_into_meta_model(model, state_dict, loaded_state_dict_keys, start_prefix, expected_keys, device_map, offload_folder, offload_index, state_dict_folder, state_dict_index, dtype, hf_quantizer, is_safetensors, keep_in_fp32_modules, unexpected_keys)
    [867](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:867) set_module_kwargs = {}
    [869](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:869) # We convert floating dtypes to the `dtype` passed. We want to keep the buffers/params
    [870](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:870) # in int/uint/bool and not cast them.
--> [871](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:871) if dtype is not None and torch.is_floating_point(param) and param.dtype != torch.float8_e4m3fn:
    [872](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:872)     if (
    [873](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:873)         keep_in_fp32_modules is not None
    [874](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:874)         and any(
   (...)
    [877](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:877)         and dtype == torch.float16
    [878](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:878)     ):
    [879](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f6b756574652f66726f6e742d656e642d726167222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c2270617468223a222f686f6d652f6b756574652f66726f6e742d656e642d7261672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a227673636f64652d66696c65486f7374227d7d-0040ssh-002dremote-002belektra-002eai-002etu-002ddarmstadt-002ede.vscode-resource.vscode-cdn.net/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:879)         param = param.to(torch.float32)

AttributeError: module 'torch' has no attribute 'float8_e4m3fn'

cc @SunMarc, as this logic appears to have come from https://github.com/huggingface/transformers/pull/32047

Oh my bad, I will fix it ! It is only available after torch 2.1

LMK if the above PR fixes it ! @Perpetue237. We will patch it soon ! 

how to update???

Same problem, How can I get new version?

> how to update???

Solved it by run `pip install transformers==4.43.2`

Thank you! It worked. I must have missed the new version.

pip install transformers==4.43.2 has run, but error exist? torch version
```
torch                         2.0.1+cu118
torchaudio                    2.0.2
torchdiffeq                   0.2.3
torchmetrics                  1.0.3
torchsde                      0.2.6
torchvision                   0.15.2+cu118
```

Did you reboot your kernel after the install? 

same error happened when I tried to load llama3.1 405B with fp8, with torch 2.4.0 (as the issue mentioned above). Tried transformers 4.43.2 but it seems not solved..

I works for me now with the newest version of transformer. I restarted the kernel and clear the cache. And where I am working in a docker container I rebuilt the container. 

@Perpetue237 Which cuda version are you using?

cuda 11.7

Was it the cuda version too old?

yes @kungfu-eric 

> yes @kungfu-eric

I am using cuda_12.1 but I still have this problem. I find one solution https://www.reddit.com/r/comfyui/comments/1ettg44/comment/lipkvex/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button 

Meet this problem again,  I am using,  transformers              4.45.2 , cuda_12.2 , torch 12.1 
upgrade torch to 2.4.1 can work  
`pip install --upgrade torch torchvision` 

torch 2.4.1+cu124
the error still exists here.

What's the error you are getting with the traceback @LukeLIN-web @cckuailong ? It's strange that you are getting it if you've installed the latest transformers as I switched the check to a `hasattr` call . 

> What's the error you are getting with the traceback @LukeLIN-web @cckuailong ? It's strange that you are getting it if you've installed the latest transformers as I switched the check to a `hasattr` call .

I meet `AttributeError: module 'torch' has no attribute 'float8_e4m3fn'` and then 
`pip install --upgrade torch torchvision` upgrade to 2.4.1 torch solves my problem. 

> AttributeError: module 'torch' has no attribute 'float8_e4m3fn'

Then, i'm pretty sure this is because you are using an older version of transformers. If you sent me the full traceback, I will be able to confirm that! 

I can only use torch==2.0.1 . Then how to solve it

Just install the latest transformers @Boltzmachine ! 

> pip install transformers==4.46.0 
 and torch                             2.0.1+cu117
torchao                           0.6.1
torchaudio                        2.0.2+cu117
torchvision                       0.15.2+cu117  the error still exists here.


> I can only use torch==2.0.1 . Then how to solve it

did you resolved? i usetransformers==4.46.0 the error still exists 


> Just install the latest transformers [@Boltzmachine](https://github.com/Boltzmachine) !

Honestly, I don't know what to do. I have a similar issue, I want to train the model found here: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix. I've installed everything in the requirements.txt, and run in to issue of version incompatibility, I upgrade, downgrade, nothing seems to work, and it's becoming frustrating. I am new to this sort of thing, been spending couple of days now redoing everything. What should I do? 

Can you show me the traceback you get with the transformers version you are using ? 

> Can you show me the traceback you get with the transformers version you are using ?

1. When I use: (img2img-turbo) PS E:\projects\img2img-turbo-main> accelerate launch src/train_pix2pix_turbo.py \

C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\najaf\anaconda3\envs\img2img-turbo\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\najaf\anaconda3\envs\img2img-turbo\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Traceback (most recent call last):
  File "E:\projects\img2img-turbo-main\src\train_pix2pix_turbo.py", line 16, in <module>
    import diffusers
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\__init__.py", line 5, in <module>
    from .utils import (
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\utils\__init__.py", line 37, in <module>
    from .dynamic_modules_utils import get_class_from_dynamic_module
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\utils\dynamic_modules_utils.py", line 28, in <module>
    from huggingface_hub import cached_download, hf_hub_download, model_info
ImportError: cannot import name 'cached_download' from 'huggingface_hub' (C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\huggingface_hub\__init__.py)
Traceback (most recent call last):
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\najaf\Anaconda3\envs\img2img-turbo\Scripts\accelerate.exe\__main__.py", line 7, in <module>
    sys.exit(main())
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
    args.func(args)
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\launch.py", line 1168, in launch_command
    simple_launcher(args)
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\launch.py", line 763, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\najaf\\anaconda3\\envs\\img2img-turbo\\python.exe', 'src/train_pix2pix_turbo.py', '\\']' returned non-zero exit status 1.

2. When I use: (img2img-turbo) PS E:\projects\img2img-turbo-main> accelerate launch src/train_pix2pix_turbo.py `
>>     --pretrained_model_name_or_path="stabilityai/sd-turbo" `
>>     --output_dir="output/pix2pix_turbo/realestate" `
>>     --dataset_folder="data/realestate" `
>>     --resolution=512 `
>>     --train_batch_size=2 `
>>     --enable_xformers_memory_efficient_attention `
>>     --viz_freq 25 `
>>     --track_val_fid `
>>     --report_to "wandb" `
>>     --tracker_project_name "pix2pix_turbo_realestate"

C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\najaf\anaconda3\envs\img2img-turbo\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\najaf\anaconda3\envs\img2img-turbo\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Traceback (most recent call last):
  File "E:\projects\img2img-turbo-main\src\train_pix2pix_turbo.py", line 16, in <module>
    import diffusers
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\__init__.py", line 5, in <module>
    from .utils import (
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\utils\__init__.py", line 37, in <module>
    from .dynamic_modules_utils import get_class_from_dynamic_module
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\diffusers\utils\dynamic_modules_utils.py", line 28, in <module>
    from huggingface_hub import cached_download, hf_hub_download, model_info
ImportError: cannot import name 'cached_download' from 'huggingface_hub' (C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\huggingface_hub\__init__.py)
Traceback (most recent call last):
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\najaf\Anaconda3\envs\img2img-turbo\Scripts\accelerate.exe\__main__.py", line 7, in <module>
    sys.exit(main())
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
    args.func(args)
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\launch.py", line 1168, in launch_command
    simple_launcher(args)
  File "C:\Users\najaf\anaconda3\envs\img2img-turbo\lib\site-packages\accelerate\commands\launch.py", line 763, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\najaf\\anaconda3\\envs\\img2img-turbo\\python.exe', 'src/train_pix2pix_turbo.py', '--pretrained_model_name_or_path=stabilityai/sd-turbo', '--output_dir=output/pix2pix_turbo/realestate', '--dataset_folder=data/realestate', '--resolution=512', '--train_batch_size=2', '--enable_xformers_memory_efficient_attention', '--viz_freq', '25', '--track_val_fid', '--report_to', 'wandb', '--tracker_project_name', 'pix2pix_turbo_realestate']' returned non-zero exit status 1.

What I've done:
- conda activate img2img-turbo
- conda install -c conda-forge libjpeg libpng
- pip install torchvision
- pip install diffusers==0.25.1 huggingface-hub==0.12.1
- reinstall different versions of python,
- pip install -r requirements.txt

And still I don't know what I'm missing or doing wrong :/

Your issue is not related to this specific tread. So please open a new issue next time. To fix your issue, make sure to upgrade diffusers and huggingface. See issue [here](https://github.com/easydiffusion/easydiffusion/issues/1851)