save_pretrained 4bits/8bits model
Author: jameswu2014
Pull request: no

### System Info

I have a BitsAndBytes Quantization 4/8bitsmodels，How to save it? I invoked save_pretrained api，however， I get error：AttributeError: 'str' object has no attribute 'numel'。

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

   bnb_config = BitsAndBytesConfig()
    bnb_config.load_in_8bit = True
    model = AutoModelForCausalLM.from_pretrained(
        model_path,
        load_in_8bit=True,
        device_map="auto",
        trust_remote_code=True
    ).eval()
    model.save_pretrained("BitsAndBytesQuant8/")

### Expected behavior

saved successfully

Comments:
cc @younesbelkada and @SunMarc 

hi @jameswu2014 
Thanks for the issue, what is the transformers version you are using? Can you try again with the latest version of `transformers`? Also can you share the full traceback and a reproducible handy snippet? Thanks

having the same issue, believe because there are string values in the state dicts, that are not really tensors

Hi @psinger,
I just checked our daily CI and we do have a CI test to check int8 serialization works correctly here: https://github.com/huggingface/transformers/blob/main/tests/bnb/test_mixed_int8.py#L286-L311 and the test is passing.
I believe maybe this PR: https://github.com/huggingface/transformers/pull/24416 fixed your issues, can you try to install transformers from source and use the latest version of `bitsandbytes` ?

```bash
pip uninstall transformers bitsandbytes
pip install git+https://github.com/huggingface/transformers.git
pip install --upgrade bitsandbytes
```

Thanks, I actually misspoke, my error is:
`AttributeError: 'str' object has no attribute 'device'
`

Installing transformers from source indeed seems to solve it.

> Hi @psinger, I just checked our daily CI and we do have a CI test to check int8 serialization works correctly here: https://github.com/huggingface/transformers/blob/main/tests/bnb/test_mixed_int8.py#L286-L311 and the test is passing. I believe maybe this PR: #24416 fixed your issues, can you try to install transformers from source and use the latest version of `bitsandbytes` ?
> 
> ```shell
> pip uninstall transformers bitsandbytes
> pip install git+https://github.com/huggingface/transformers.git
> pip install --upgrade bitsandbytes
> ```

Whether 4bits supported? I need  both 4bits and 8bits saved.

Hi @jameswu2014 
Thanks for the heads up, as stated above, currently 4bit saving is not supported yet, feel free to raise an issue on bitsandbytes repository to request this feature

Is there a work-around for 4-bit models?  Can I convert the model to something like float16 and then save it?  Or is 4-bit fine-tuning not really usable?  Thanks.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing as this is not related with transformers but bitsandbytes, please track the issue: https://github.com/TimDettmers/bitsandbytes/issues/695