AttributeError: 'DistilBertModel' object has no attribute '_use_flash_attention_2'
Author: javilonso
Pull request: no

### System Info

Obtaining this error in last transformers 4.37.2, but works correctly in transformers 4.35.2
Simple inference with a finetuned distilbert model.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Install transformers 4.37.2
2. Perform inference with model https://huggingface.co/distilbert/distilbert-base-uncased

### Expected behavior

Inference should go through without errors

Comments:
Hi @javilonso ! 
I quickly tried on transformers main: 
```python
from transformers import pipeline

unmasker = pipeline('fill-mask', model='distilbert-base-uncased')
unmasker("Hello I'm a [MASK] model.")
```
But I did not managed to repro, can you share a snippet to reproduce the issue?
I also tried:
```python
from transformers import DistilBertTokenizer, DistilBertModel

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained("distilbert-base-uncased")
text = "Replace me by any text you'd like."

encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```
Can you also try on transformers main?

Hi, I am also facing the same issue, the code @younesbelkada gave works well on my system. However, I get
 `AttributeError: DistilBertModel' object has no attribute '_use_flash_attention_2` 
when running my prediction with my finetuned distilbert model. I am also running transformers 4.37.2. It works fine on 4.35.2.

The error happens when trying to perform the prediction with a local copy of the model:
```python
inputs = tokenizer.encode(text, return_tensors="pt").to(device)
logits = model(inputs).logits
```

This is how I loaded the tokenizer:
```python
tokenizer = AutoTokenizer.from_pretrained(
'models/tokenizer', add_prefix_space=True)
```

I am also loading a saved local copy of the model here:
```python
model = torch.load(model_name, map_location=torch.device(device))
```

Hope the information provided is enough! Thanks in advance.
     

This issue does not seem to occur when I finetune my model again on transformers 4.38.0.
I guess the solution would be to update the transformers package.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The problem is still there.

If you want any help you need to share a reproducer. We are unable to reproduce

I am getting the same error in transformers 4.40.0 while loading a fine-tuned DistilBERT model from a bin file and performing predictions using it. The model seems to load correctly, however, when predicting, I get this error. Following is the code I am using:

```python3
import html
from pathlib import Path
import os
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification,PreTrainedModel,DistilBertModel #importing distilbert tokenizer+model
import torch
import os
from torch import softmax

class DistilBERTClass(PreTrainedModel):
    def __init__(self):
        super(DistilBERTClass, self).__init__()
        self.l1 = DistilBertModel.from_pretrained("distilbert-base-uncased")
        self.pre_classifier = torch.nn.Linear(768, 768)
        self.dropout = torch.nn.Dropout(0.1) # to avoid overfitting
        self.classifier = torch.nn.Linear(768, 3)

    def forward(self, input_ids, attention_mask, token_type_ids):
        output_1 = self.l1(input_ids=input_ids, attention_mask=attention_mask)
        hidden_state = output_1[0]
        pooler = hidden_state[:, 0]
        pooler = self.pre_classifier(pooler)
        pooler = torch.nn.Tanh()(pooler)
        pooler = self.dropout(pooler)
        output = self.classifier(pooler)
        return output

MODEL_PATH =Path("model")
#Load the model from the saved path
predictor = torch.load(MODEL_PATH/"pytorch_model.bin", map_location="cpu")
#Load the tokenizer
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
#print(predictor)
#The model is now in prediction mode
predictor.eval()  

params =  "This is a good product"
            
# predict method used to get the prediction
#response = WebScraperConfig.predictor.predict(sentence)
input_ids= tokenizer.encode(params,return_tensors='pt')
attention_mask = torch.ones(input_ids.shape, dtype=torch.long)  
token_type_ids = torch.zeros(input_ids.shape, dtype=torch.long)
with torch.no_grad():
    logits = predictor(input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids)[0]

# Apply softmax to get probabilities
probs = softmax(logits, dim=-1).tolist()[0]

# Identify the predicted sentiment class
predicted_class = torch.argmax(logits).item()

# Map the predicted class to sentiment labels
sentiment_labels = ['negative', 'neutral', 'positive']
predicted_sentiment = sentiment_labels[predicted_class]

# Print results
print(f"Predicted sentiment: {predicted_sentiment}")
print(f"Class probabilities: {probs}")
```

Please forgive the lengthy code, because I couldn't reproduce the issue otherwise.

I downgraded from version 4.40.0 to 4.35,2 and now my code works. It looks like this is an issue with newer versions of transformers, since this code does not work on neither 4.38.2 nor 4.40.0.

In the snippet you share, you do not use the custom class you declare, since I do not have the `.bin` file, there is nothing I can do for you. 
1. You are using a custom DistillBert class that does not inherit from the original distillBert class
2. You are using torch.load, which is not safe, but also not the `transformers` api. We recommend `save_pretrained` and `from_pretrained`. 
3. We need a full stacktrace 😉