Model inside docker gives different results
Author: Winvoker
Pull request: no

- `transformers` version: 4.9.2
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.8.11
- PyTorch version (GPU?): 1.9.0 (True)
- Tensorflow version (GPU?): 2.4.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

Models:
- Bert : @LysandreJik 

I am using a bert model I fine-tuned and everything runs fine. I do not get any error etc. However , inside of a docker it gives different scores(from pipeline). For example, I fine-tuned a model  and I use example scripts and use a pipeline. I get 0.94 score in my local machine. However , when i run same model inside a docker with exact same inputs , it gives 0.75 score. 
You can see a related issue here : https://stackoverflow.com/questions/66797173/issue-while-using-transformers-package-inside-the-docker-image

I am trying this model with a flask endpoints , and in the link they say it is something related to threading , however i could not solve my problem. 

Scripts I use to load model  : 

```
tokenizer = BertTokenizer.from_pretrained("models/local_tokenizer")
model = BertForSequenceClassification.from_pretrained("models/model-v6")

logger.info("Model yüklendi")
print("Model Yüklendi")

productionPipeline = pipeline(
  "text-classification", model=model.to("cpu"), tokenizer=tokenizer
)
```

Scripts used while training (slightly edited IMDB tutorial):
```
class SentimentDataset(torch.utils.data.Dataset):
    def __init__(self, dataframe, tokenizer, max_len):
        self.tokenizer = tokenizer
        self.data = dataframe
        self.text = dataframe.text
        self.targets = self.data.label
        self.max_len = max_len

    def __len__(self):
        return len(self.text)

    def __getitem__(self, index):
        text = str(self.text[index])
        text = " ".join(text.split())

        inputs = self.tokenizer.encode_plus(
            text,
            None,
            add_special_tokens=True,
            max_length=self.max_len,
            pad_to_max_length=True,
            return_token_type_ids=True,
            truncation=True
        )
        ids = inputs['input_ids']
        mask = inputs['attention_mask']
        token_type_ids = inputs["token_type_ids"]

        return {
            'input_ids': torch.tensor(ids, dtype=torch.long),
            'attention_mask': torch.tensor(mask, dtype=torch.long),
            'token_type_ids': torch.tensor(token_type_ids, dtype=torch.long),
            'labels': torch.tensor(self.targets[index], dtype=torch.float)
        }
train_dataset = SentimentDataset(train_df, tokenizer, 50)
test_dataset = SentimentDataset(test_df, tokenizer, 50)

from transformers import AdamW
from torch.utils.data import DataLoader
from tqdm import tqdm

model.train()

optim = AdamW(model.parameters(), lr=5e-6)

for epoch in range(0,2):
    step = epoch + 1

    train_loss = 0
    print("Epoch starts")
    for batch in tqdm(train_loader):
        optim.zero_grad()

        input_ids = batch['input_ids'].to(device)
        attention_mask = batch['attention_mask'].to(device)
        targets = batch['labels'].to(device)

        outputs = model(input_ids, attention_mask=attention_mask, labels=targets)
        
        loss = outputs[0]
        loss.backward()
        optim.step()
        train_loss += loss.item()

```




Comments:
I solved my problem , for anyone who experiences this problem  : 
My local transformers version was : 4.9.2 , and i was installing the latest transformers version which was : 4.10.3. 
Installing 4.9.2 inside my docker resolved the issue however I think it can be added a warning. That we can experience problems related to model results. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.