Error with BetterTransformer Optimizations in Transformers Library with Starcoderplus Model"
Author: Taishi-N324
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-4.18.0-193.el8.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.13
- Huggingface_hub version: 0.20.1
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.2+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@fxmarty

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I'm trying to run a script using PyTorch and the transformers library from Hugging Face to leverage a model called "starcoderplus" for text generation. Here's the script I'm using:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoderplus")
model = AutoModelForCausalLM.from_pretrained("bigcode/starcoderplus", torch_dtype=torch.float16).to("cuda")
# convert the model to BetterTransformer
model.to_bettertransformer()

input_text = "Hello my dog is cute and"
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")

with torch.backends.cuda.sdp_kernel(enable_flash=True, enable_math=False, enable_mem_efficient=False):
    outputs = model.generate(**inputs)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
from https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-and-memory-efficient-attention-through-pytorchs-scaleddotproductattention

 I encounter the following error:

```
ValueError: Transformers now supports natively BetterTransformer optimizations (torch.nn.functional.scaled_dot_product_attention) for the model type gpt_bigcode. Please upgrade to transformers>=4.36 and torch>=2.1.1 to use it. Details: https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-and-memory-efficient-attention-through-pytorchs-scaleddotproductattention
```

However, I am already using torch version 2.1.2+cu118 and transformers version 4.36.2.

### Expected behavior

Better transformer available in starcoderplus

Comments:
I think the idea is to remove the call to `model.to_bettertransformer()` and `with torch.backends.cuda.sdp_kernel(enable_flash=True, enable_math=False, enable_mem_efficient=False):` . The error should be updated to mention this! would you like to open a PR?

I'm sorry, I don't quite understand. What do you mean?

Are you saying to remove both `model.to_bettertransformer()` and `with torch.backends.cuda.sdp_kernel(enable_flash=True, enable_math=False, enable_mem_efficient=False):`?

Originally, what I want to do is to wrap GPTBigCode with a bettertransformer and then wrap it with fsdp to train it. However, I encounter this error during the process. I was trying with a minimal configuration, but I still encounter errors.

It seems that GPTBigCode was supported during https://github.com/huggingface/optimum/pull/1252, but in the latest branch, it appears to no longer be supported according to https://github.com/huggingface/optimum/tree/main/optimum/bettertransformer/models.

The error means that now transformers supports sdpa natively so you should not need to use better transformer for the usage you are looking for 🤗 

Thank you for your kindness!
I understand now.
Thank you for your assistance.