Trainer evaluate returns empty dictionary
Author: adamwawrzynski
Pull request: no

## Environment info

- `transformers` version: 3.1.0
- Platform: Linux-5.4.0-45-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
 Trainer: @sgugger

## Information

Model I am using RoBERT:

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. `python3 finetune_roberta.py -m result/ -d dataset.txt -t dataset.txt`

```python
from transformers import (BertForNextSentencePrediction,
                          BertTokenizer,
                          RobertaModel, RobertaTokenizer, Trainer,
                          TrainingArguments)
from transformers.data.datasets.language_modeling import TextDatasetForNextSentencePrediction
from transformers.data.data_collator import DataCollatorForNextSentencePrediction
from argparse import ArgumentParser


def parse_args():
    parser = ArgumentParser("Fine-tune RoBERTa in Next Sentence Prediction.")
    parser.add_argument("-m", "--model_path", dest="model_path", required=True, help="Path to RoBERTa model.")
    parser.add_argument("-d", "--dataset_path", dest="dataset_path", required=True, help="Path to dataset.")
    parser.add_argument("-t", "--test_dataset_path", dest="test_dataset_path", required=True, help="Path to test dataset.")
    args = parser.parse_args()
    return args


if __name__ == "__main__":
    args = parse_args()
    tokenizer = RobertaTokenizer.from_pretrained(args.model_path)
    finetune_model = BertForNextSentencePrediction.from_pretrained(args.model_path)

    training_args = TrainingArguments(
        output_dir=args.output_path,
        num_train_epochs=3,
        per_device_train_batch_size=1,
        per_device_eval_batch_size=1,
        warmup_steps=500,
        weight_decay=0.01,
        logging_dir='./logs',
    )

    data_collator = DataCollatorForNextSentencePrediction(
        tokenizer=tokenizer,
        mlm=False,
        block_size=512,
        nsp_probability=0.5,
      )

    train_dataset = TextDatasetForNextSentencePrediction(
        tokenizer=tokenizer,
        file_path=args.dataset_path,
        block_size=512,
    )

    test_dataset = TextDatasetForNextSentencePrediction(
        tokenizer=tokenizer,
        file_path=args.test_dataset_path,
        block_size=512,
    )

    trainer = Trainer(
        model=finetune_model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=test_dataset,
        data_collator=data_collator,
    )

    print(trainer.evaluate(test_dataset))
```
Output in terminal:
```bash
python3 finetune_roberta.py -m result/ -d dataset_fixed_alior.txt -t dataset_fixed_alior.txt -o results_test/
Special tokens have been added in the vocabulary, make sure the associated word emebedding are fine-tuned or trained.
Some weights of the model checkpoint at result/ were not used when initializing RobertaModel: ['bert.embeddings.position_ids', 'bert.embeddings.word_embeddings.weight', 'bert.embeddings.position_embeddings.weight', 'bert.embeddings.token_type_embeddings.weight', 'bert.embeddings.LayerNorm.weight', 'bert.embeddings.LayerNorm.bias', 'bert.encoder.layer.0.attention.self.query.weight', 'bert.encoder.layer.0.attention.self.query.bias', 'bert.encoder.layer.0.attention.self.key.weight', 'bert.encoder.layer.0.attention.self.key.bias', 'bert.encoder.layer.0.attention.self.value.weight', 'bert.encoder.layer.0.attention.self.value.bias', 'bert.encoder.layer.0.attention.output.dense.weight', 'bert.encoder.layer.0.attention.output.dense.bias', 'bert.encoder.layer.0.attention.output.LayerNorm.weight', 'bert.encoder.layer.0.attention.output.LayerNorm.bias', 'bert.encoder.layer.0.intermediate.dense.weight', 'bert.encoder.layer.0.intermediate.dense.bias', 'bert.encoder.layer.0.output.dense.weight', 'bert.encoder.layer.0.output.dense.bias', 'bert.encoder.layer.0.output.LayerNorm.weight', 'bert.encoder.layer.0.output.LayerNorm.bias', 'bert.encoder.layer.1.attention.self.query.weight', 'bert.encoder.layer.1.attention.self.query.bias', 'bert.encoder.layer.1.attention.self.key.weight', 'bert.encoder.layer.1.attention.self.key.bias', 'bert.encoder.layer.1.attention.self.value.weight', 'bert.encoder.layer.1.attention.self.value.bias', 'bert.encoder.layer.1.attention.output.dense.weight', 'bert.encoder.layer.1.attention.output.dense.bias', 'bert.encoder.layer.1.attention.output.LayerNorm.weight', 'bert.encoder.layer.1.attention.output.LayerNorm.bias', 'bert.encoder.layer.1.intermediate.dense.weight', 'bert.encoder.layer.1.intermediate.dense.bias', 'bert.encoder.layer.1.output.dense.weight', 'bert.encoder.layer.1.output.dense.bias', 'bert.encoder.layer.1.output.LayerNorm.weight', 'bert.encoder.layer.1.output.LayerNorm.bias', 'bert.encoder.layer.2.attention.self.query.weight', 'bert.encoder.layer.2.attention.self.query.bias', 'bert.encoder.layer.2.attention.self.key.weight', 'bert.encoder.layer.2.attention.self.key.bias', 'bert.encoder.layer.2.attention.self.value.weight', 'bert.encoder.layer.2.attention.self.value.bias', 'bert.encoder.layer.2.attention.output.dense.weight', 'bert.encoder.layer.2.attention.output.dense.bias', 'bert.encoder.layer.2.attention.output.LayerNorm.weight', 'bert.encoder.layer.2.attention.output.LayerNorm.bias', 'bert.encoder.layer.2.intermediate.dense.weight', 'bert.encoder.layer.2.intermediate.dense.bias', 'bert.encoder.layer.2.output.dense.weight', 'bert.encoder.layer.2.output.dense.bias', 'bert.encoder.layer.2.output.LayerNorm.weight', 'bert.encoder.layer.2.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.self.query.weight', 'bert.encoder.layer.3.attention.self.query.bias', 'bert.encoder.layer.3.attention.self.key.weight', 'bert.encoder.layer.3.attention.self.key.bias', 'bert.encoder.layer.3.attention.self.value.weight', 'bert.encoder.layer.3.attention.self.value.bias', 'bert.encoder.layer.3.attention.output.dense.weight', 'bert.encoder.layer.3.attention.output.dense.bias', 'bert.encoder.layer.3.attention.output.LayerNorm.weight', 'bert.encoder.layer.3.attention.output.LayerNorm.bias', 'bert.encoder.layer.3.intermediate.dense.weight', 'bert.encoder.layer.3.intermediate.dense.bias', 'bert.encoder.layer.3.output.dense.weight', 'bert.encoder.layer.3.output.dense.bias', 'bert.encoder.layer.3.output.LayerNorm.weight', 'bert.encoder.layer.3.output.LayerNorm.bias', 'bert.encoder.layer.4.attention.self.query.weight', 'bert.encoder.layer.4.attention.self.query.bias', 'bert.encoder.layer.4.attention.self.key.weight', 'bert.encoder.layer.4.attention.self.key.bias', 'bert.encoder.layer.4.attention.self.value.weight', 'bert.encoder.layer.4.attention.self.value.bias', 'bert.encoder.layer.4.attention.output.dense.weight', 'bert.encoder.layer.4.attention.output.dense.bias', 'bert.encoder.layer.4.attention.output.LayerNorm.weight', 'bert.encoder.layer.4.attention.output.LayerNorm.bias', 'bert.encoder.layer.4.intermediate.dense.weight', 'bert.encoder.layer.4.intermediate.dense.bias', 'bert.encoder.layer.4.output.dense.weight', 'bert.encoder.layer.4.output.dense.bias', 'bert.encoder.layer.4.output.LayerNorm.weight', 'bert.encoder.layer.4.output.LayerNorm.bias', 'bert.encoder.layer.5.attention.self.query.weight', 'bert.encoder.layer.5.attention.self.query.bias', 'bert.encoder.layer.5.attention.self.key.weight', 'bert.encoder.layer.5.attention.self.key.bias', 'bert.encoder.layer.5.attention.self.value.weight', 'bert.encoder.layer.5.attention.self.value.bias', 'bert.encoder.layer.5.attention.output.dense.weight', 'bert.encoder.layer.5.attention.output.dense.bias', 'bert.encoder.layer.5.attention.output.LayerNorm.weight', 'bert.encoder.layer.5.attention.output.LayerNorm.bias', 'bert.encoder.layer.5.intermediate.dense.weight', 'bert.encoder.layer.5.intermediate.dense.bias', 'bert.encoder.layer.5.output.dense.weight', 'bert.encoder.layer.5.output.dense.bias', 'bert.encoder.layer.5.output.LayerNorm.weight', 'bert.encoder.layer.5.output.LayerNorm.bias', 'bert.encoder.layer.6.attention.self.query.weight', 'bert.encoder.layer.6.attention.self.query.bias', 'bert.encoder.layer.6.attention.self.key.weight', 'bert.encoder.layer.6.attention.self.key.bias', 'bert.encoder.layer.6.attention.self.value.weight', 'bert.encoder.layer.6.attention.self.value.bias', 'bert.encoder.layer.6.attention.output.dense.weight', 'bert.encoder.layer.6.attention.output.dense.bias', 'bert.encoder.layer.6.attention.output.LayerNorm.weight', 'bert.encoder.layer.6.attention.output.LayerNorm.bias', 'bert.encoder.layer.6.intermediate.dense.weight', 'bert.encoder.layer.6.intermediate.dense.bias', 'bert.encoder.layer.6.output.dense.weight', 'bert.encoder.layer.6.output.dense.bias', 'bert.encoder.layer.6.output.LayerNorm.weight', 'bert.encoder.layer.6.output.LayerNorm.bias', 'bert.encoder.layer.7.attention.self.query.weight', 'bert.encoder.layer.7.attention.self.query.bias', 'bert.encoder.layer.7.attention.self.key.weight', 'bert.encoder.layer.7.attention.self.key.bias', 'bert.encoder.layer.7.attention.self.value.weight', 'bert.encoder.layer.7.attention.self.value.bias', 'bert.encoder.layer.7.attention.output.dense.weight', 'bert.encoder.layer.7.attention.output.dense.bias', 'bert.encoder.layer.7.attention.output.LayerNorm.weight', 'bert.encoder.layer.7.attention.output.LayerNorm.bias', 'bert.encoder.layer.7.intermediate.dense.weight', 'bert.encoder.layer.7.intermediate.dense.bias', 'bert.encoder.layer.7.output.dense.weight', 'bert.encoder.layer.7.output.dense.bias', 'bert.encoder.layer.7.output.LayerNorm.weight', 'bert.encoder.layer.7.output.LayerNorm.bias', 'bert.encoder.layer.8.attention.self.query.weight', 'bert.encoder.layer.8.attention.self.query.bias', 'bert.encoder.layer.8.attention.self.key.weight', 'bert.encoder.layer.8.attention.self.key.bias', 'bert.encoder.layer.8.attention.self.value.weight', 'bert.encoder.layer.8.attention.self.value.bias', 'bert.encoder.layer.8.attention.output.dense.weight', 'bert.encoder.layer.8.attention.output.dense.bias', 'bert.encoder.layer.8.attention.output.LayerNorm.weight', 'bert.encoder.layer.8.attention.output.LayerNorm.bias', 'bert.encoder.layer.8.intermediate.dense.weight', 'bert.encoder.layer.8.intermediate.dense.bias', 'bert.encoder.layer.8.output.dense.weight', 'bert.encoder.layer.8.output.dense.bias', 'bert.encoder.layer.8.output.LayerNorm.weight', 'bert.encoder.layer.8.output.LayerNorm.bias', 'bert.encoder.layer.9.attention.self.query.weight', 'bert.encoder.layer.9.attention.self.query.bias', 'bert.encoder.layer.9.attention.self.key.weight', 'bert.encoder.layer.9.attention.self.key.bias', 'bert.encoder.layer.9.attention.self.value.weight', 'bert.encoder.layer.9.attention.self.value.bias', 'bert.encoder.layer.9.attention.output.dense.weight', 'bert.encoder.layer.9.attention.output.dense.bias', 'bert.encoder.layer.9.attention.output.LayerNorm.weight', 'bert.encoder.layer.9.attention.output.LayerNorm.bias', 'bert.encoder.layer.9.intermediate.dense.weight', 'bert.encoder.layer.9.intermediate.dense.bias', 'bert.encoder.layer.9.output.dense.weight', 'bert.encoder.layer.9.output.dense.bias', 'bert.encoder.layer.9.output.LayerNorm.weight', 'bert.encoder.layer.9.output.LayerNorm.bias', 'bert.encoder.layer.10.attention.self.query.weight', 'bert.encoder.layer.10.attention.self.query.bias', 'bert.encoder.layer.10.attention.self.key.weight', 'bert.encoder.layer.10.attention.self.key.bias', 'bert.encoder.layer.10.attention.self.value.weight', 'bert.encoder.layer.10.attention.self.value.bias', 'bert.encoder.layer.10.attention.output.dense.weight', 'bert.encoder.layer.10.attention.output.dense.bias', 'bert.encoder.layer.10.attention.output.LayerNorm.weight', 'bert.encoder.layer.10.attention.output.LayerNorm.bias', 'bert.encoder.layer.10.intermediate.dense.weight', 'bert.encoder.layer.10.intermediate.dense.bias', 'bert.encoder.layer.10.output.dense.weight', 'bert.encoder.layer.10.output.dense.bias', 'bert.encoder.layer.10.output.LayerNorm.weight', 'bert.encoder.layer.10.output.LayerNorm.bias', 'bert.encoder.layer.11.attention.self.query.weight', 'bert.encoder.layer.11.attention.self.query.bias', 'bert.encoder.layer.11.attention.self.key.weight', 'bert.encoder.layer.11.attention.self.key.bias', 'bert.encoder.layer.11.attention.self.value.weight', 'bert.encoder.layer.11.attention.self.value.bias', 'bert.encoder.layer.11.attention.output.dense.weight', 'bert.encoder.layer.11.attention.output.dense.bias', 'bert.encoder.layer.11.attention.output.LayerNorm.weight', 'bert.encoder.layer.11.attention.output.LayerNorm.bias', 'bert.encoder.layer.11.intermediate.dense.weight', 'bert.encoder.layer.11.intermediate.dense.bias', 'bert.encoder.layer.11.output.dense.weight', 'bert.encoder.layer.11.output.dense.bias', 'bert.encoder.layer.11.output.LayerNorm.weight', 'bert.encoder.layer.11.output.LayerNorm.bias', 'bert.pooler.dense.weight', 'bert.pooler.dense.bias', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of RobertaModel were not initialized from the model checkpoint at result/ and are newly initialized: ['embeddings.word_embeddings.weight', 'embeddings.position_embeddings.weight', 'embeddings.token_type_embeddings.weight', 'embeddings.LayerNorm.weight', 'embeddings.LayerNorm.bias', 'encoder.layer.0.attention.self.query.weight', 'encoder.layer.0.attention.self.query.bias', 'encoder.layer.0.attention.self.key.weight', 'encoder.layer.0.attention.self.key.bias', 'encoder.layer.0.attention.self.value.weight', 'encoder.layer.0.attention.self.value.bias', 'encoder.layer.0.attention.output.dense.weight', 'encoder.layer.0.attention.output.dense.bias', 'encoder.layer.0.attention.output.LayerNorm.weight', 'encoder.layer.0.attention.output.LayerNorm.bias', 'encoder.layer.0.intermediate.dense.weight', 'encoder.layer.0.intermediate.dense.bias', 'encoder.layer.0.output.dense.weight', 'encoder.layer.0.output.dense.bias', 'encoder.layer.0.output.LayerNorm.weight', 'encoder.layer.0.output.LayerNorm.bias', 'encoder.layer.1.attention.self.query.weight', 'encoder.layer.1.attention.self.query.bias', 'encoder.layer.1.attention.self.key.weight', 'encoder.layer.1.attention.self.key.bias', 'encoder.layer.1.attention.self.value.weight', 'encoder.layer.1.attention.self.value.bias', 'encoder.layer.1.attention.output.dense.weight', 'encoder.layer.1.attention.output.dense.bias', 'encoder.layer.1.attention.output.LayerNorm.weight', 'encoder.layer.1.attention.output.LayerNorm.bias', 'encoder.layer.1.intermediate.dense.weight', 'encoder.layer.1.intermediate.dense.bias', 'encoder.layer.1.output.dense.weight', 'encoder.layer.1.output.dense.bias', 'encoder.layer.1.output.LayerNorm.weight', 'encoder.layer.1.output.LayerNorm.bias', 'encoder.layer.2.attention.self.query.weight', 'encoder.layer.2.attention.self.query.bias', 'encoder.layer.2.attention.self.key.weight', 'encoder.layer.2.attention.self.key.bias', 'encoder.layer.2.attention.self.value.weight', 'encoder.layer.2.attention.self.value.bias', 'encoder.layer.2.attention.output.dense.weight', 'encoder.layer.2.attention.output.dense.bias', 'encoder.layer.2.attention.output.LayerNorm.weight', 'encoder.layer.2.attention.output.LayerNorm.bias', 'encoder.layer.2.intermediate.dense.weight', 'encoder.layer.2.intermediate.dense.bias', 'encoder.layer.2.output.dense.weight', 'encoder.layer.2.output.dense.bias', 'encoder.layer.2.output.LayerNorm.weight', 'encoder.layer.2.output.LayerNorm.bias', 'encoder.layer.3.attention.self.query.weight', 'encoder.layer.3.attention.self.query.bias', 'encoder.layer.3.attention.self.key.weight', 'encoder.layer.3.attention.self.key.bias', 'encoder.layer.3.attention.self.value.weight', 'encoder.layer.3.attention.self.value.bias', 'encoder.layer.3.attention.output.dense.weight', 'encoder.layer.3.attention.output.dense.bias', 'encoder.layer.3.attention.output.LayerNorm.weight', 'encoder.layer.3.attention.output.LayerNorm.bias', 'encoder.layer.3.intermediate.dense.weight', 'encoder.layer.3.intermediate.dense.bias', 'encoder.layer.3.output.dense.weight', 'encoder.layer.3.output.dense.bias', 'encoder.layer.3.output.LayerNorm.weight', 'encoder.layer.3.output.LayerNorm.bias', 'encoder.layer.4.attention.self.query.weight', 'encoder.layer.4.attention.self.query.bias', 'encoder.layer.4.attention.self.key.weight', 'encoder.layer.4.attention.self.key.bias', 'encoder.layer.4.attention.self.value.weight', 'encoder.layer.4.attention.self.value.bias', 'encoder.layer.4.attention.output.dense.weight', 'encoder.layer.4.attention.output.dense.bias', 'encoder.layer.4.attention.output.LayerNorm.weight', 'encoder.layer.4.attention.output.LayerNorm.bias', 'encoder.layer.4.intermediate.dense.weight', 'encoder.layer.4.intermediate.dense.bias', 'encoder.layer.4.output.dense.weight', 'encoder.layer.4.output.dense.bias', 'encoder.layer.4.output.LayerNorm.weight', 'encoder.layer.4.output.LayerNorm.bias', 'encoder.layer.5.attention.self.query.weight', 'encoder.layer.5.attention.self.query.bias', 'encoder.layer.5.attention.self.key.weight', 'encoder.layer.5.attention.self.key.bias', 'encoder.layer.5.attention.self.value.weight', 'encoder.layer.5.attention.self.value.bias', 'encoder.layer.5.attention.output.dense.weight', 'encoder.layer.5.attention.output.dense.bias', 'encoder.layer.5.attention.output.LayerNorm.weight', 'encoder.layer.5.attention.output.LayerNorm.bias', 'encoder.layer.5.intermediate.dense.weight', 'encoder.layer.5.intermediate.dense.bias', 'encoder.layer.5.output.dense.weight', 'encoder.layer.5.output.dense.bias', 'encoder.layer.5.output.LayerNorm.weight', 'encoder.layer.5.output.LayerNorm.bias', 'encoder.layer.6.attention.self.query.weight', 'encoder.layer.6.attention.self.query.bias', 'encoder.layer.6.attention.self.key.weight', 'encoder.layer.6.attention.self.key.bias', 'encoder.layer.6.attention.self.value.weight', 'encoder.layer.6.attention.self.value.bias', 'encoder.layer.6.attention.output.dense.weight', 'encoder.layer.6.attention.output.dense.bias', 'encoder.layer.6.attention.output.LayerNorm.weight', 'encoder.layer.6.attention.output.LayerNorm.bias', 'encoder.layer.6.intermediate.dense.weight', 'encoder.layer.6.intermediate.dense.bias', 'encoder.layer.6.output.dense.weight', 'encoder.layer.6.output.dense.bias', 'encoder.layer.6.output.LayerNorm.weight', 'encoder.layer.6.output.LayerNorm.bias', 'encoder.layer.7.attention.self.query.weight', 'encoder.layer.7.attention.self.query.bias', 'encoder.layer.7.attention.self.key.weight', 'encoder.layer.7.attention.self.key.bias', 'encoder.layer.7.attention.self.value.weight', 'encoder.layer.7.attention.self.value.bias', 'encoder.layer.7.attention.output.dense.weight', 'encoder.layer.7.attention.output.dense.bias', 'encoder.layer.7.attention.output.LayerNorm.weight', 'encoder.layer.7.attention.output.LayerNorm.bias', 'encoder.layer.7.intermediate.dense.weight', 'encoder.layer.7.intermediate.dense.bias', 'encoder.layer.7.output.dense.weight', 'encoder.layer.7.output.dense.bias', 'encoder.layer.7.output.LayerNorm.weight', 'encoder.layer.7.output.LayerNorm.bias', 'encoder.layer.8.attention.self.query.weight', 'encoder.layer.8.attention.self.query.bias', 'encoder.layer.8.attention.self.key.weight', 'encoder.layer.8.attention.self.key.bias', 'encoder.layer.8.attention.self.value.weight', 'encoder.layer.8.attention.self.value.bias', 'encoder.layer.8.attention.output.dense.weight', 'encoder.layer.8.attention.output.dense.bias', 'encoder.layer.8.attention.output.LayerNorm.weight', 'encoder.layer.8.attention.output.LayerNorm.bias', 'encoder.layer.8.intermediate.dense.weight', 'encoder.layer.8.intermediate.dense.bias', 'encoder.layer.8.output.dense.weight', 'encoder.layer.8.output.dense.bias', 'encoder.layer.8.output.LayerNorm.weight', 'encoder.layer.8.output.LayerNorm.bias', 'encoder.layer.9.attention.self.query.weight', 'encoder.layer.9.attention.self.query.bias', 'encoder.layer.9.attention.self.key.weight', 'encoder.layer.9.attention.self.key.bias', 'encoder.layer.9.attention.self.value.weight', 'encoder.layer.9.attention.self.value.bias', 'encoder.layer.9.attention.output.dense.weight', 'encoder.layer.9.attention.output.dense.bias', 'encoder.layer.9.attention.output.LayerNorm.weight', 'encoder.layer.9.attention.output.LayerNorm.bias', 'encoder.layer.9.intermediate.dense.weight', 'encoder.layer.9.intermediate.dense.bias', 'encoder.layer.9.output.dense.weight', 'encoder.layer.9.output.dense.bias', 'encoder.layer.9.output.LayerNorm.weight', 'encoder.layer.9.output.LayerNorm.bias', 'encoder.layer.10.attention.self.query.weight', 'encoder.layer.10.attention.self.query.bias', 'encoder.layer.10.attention.self.key.weight', 'encoder.layer.10.attention.self.key.bias', 'encoder.layer.10.attention.self.value.weight', 'encoder.layer.10.attention.self.value.bias', 'encoder.layer.10.attention.output.dense.weight', 'encoder.layer.10.attention.output.dense.bias', 'encoder.layer.10.attention.output.LayerNorm.weight', 'encoder.layer.10.attention.output.LayerNorm.bias', 'encoder.layer.10.intermediate.dense.weight', 'encoder.layer.10.intermediate.dense.bias', 'encoder.layer.10.output.dense.weight', 'encoder.layer.10.output.dense.bias', 'encoder.layer.10.output.LayerNorm.weight', 'encoder.layer.10.output.LayerNorm.bias', 'encoder.layer.11.attention.self.query.weight', 'encoder.layer.11.attention.self.query.bias', 'encoder.layer.11.attention.self.key.weight', 'encoder.layer.11.attention.self.key.bias', 'encoder.layer.11.attention.self.value.weight', 'encoder.layer.11.attention.self.value.bias', 'encoder.layer.11.attention.output.dense.weight', 'encoder.layer.11.attention.output.dense.bias', 'encoder.layer.11.attention.output.LayerNorm.weight', 'encoder.layer.11.attention.output.LayerNorm.bias', 'encoder.layer.11.intermediate.dense.weight', 'encoder.layer.11.intermediate.dense.bias', 'encoder.layer.11.output.dense.weight', 'encoder.layer.11.output.dense.bias', 'encoder.layer.11.output.LayerNorm.weight', 'encoder.layer.11.output.LayerNorm.bias', 'pooler.dense.weight', 'pooler.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Evaluation:   0%|                                                                                                                                                               | 0/7385 [00:00<?, ?it/s]
...
[[[206, 3, 3, 47, 3, 4224, 163, 3, 2197, 3, 3, 3, 91, 324, 3, 2920, 3, 3, 47, 3, 3, 170, 21822, 3, 3, 14139, 3, 301, 3, 1210, 1692, 3, 67, 3, 25690, 358, 3, 258, 84, 3, 447, 3, 2980, 3, 43750, 3, 7388, 3, 15679, 4121, 3, 254, 3, 15390, 223, 21822, 3, 12135, 3, 269, 3, 254, 3, 10577, 3, 88, 611, 90, 3, 4729, 15616, 3, 92, 3, 336, 163, 21822, 3, 526, 3, 269, 3, 98, 3048, 3, 4224, 3, 215, 21822, 3, 88, 358, 3, 88, 611, 84], [254, 3, 67, 3, 88, 611, 74, 21822, 3, 3, 88, 611, 74, 21822, 3, 3, 206, 3, 3, 2048, 74, 21822, 3, 3, 92, 3, 336, 163, 21822, 3, 526]], [[254, 3, 67, 3, 88, 611, 74, 21822, 3, 3, 88, 611, 74, 21822, 3, 3, 206, 3, 3, 2048, 74, 21822, 3, 3, 92, 3, 336, 163, 21822, 3, 526], [1598, 3, 3, 56, 2726, 3, 5704, 3, 2980, 4847, 3, 336, 826, 237, 3, 4224, 163, 3, 254, 3, 10577, 3, 25690, 3, 3412, 23962, 567]], [[301, 6393, 324, 3, 36461, 118, 238, 3, 56, 3, 269, 3, 26672, 84, 3, 93, 3, 169, 210, 3, 3968, 368, 3, 7388, 3, 477, 3, 3, 91, 3, 301, 21176, 3, 94, 3, 94, 3, 301, 21176, 3, 7388, 3, 67, 3, 26605, 110, 21822, 3, 3, 15191, 114, 21822, 3, 3, 3119, 3, 2197, 3, 7208, 3, 95, 661, 3, 227, 3, 3, 3, 2554, 3, 2197, 4847, 3, 2197, 21822, 3, 3, 477, 3, 3, 9746, 3, 4224, 21822, 3, 3, 91, 41872, 311, 3, 1068, 25393, 3, 353, 3, 23635, 3, 75, 3, 1146, 173, 3, 3, 5379, 3, 237, 2736, 3, 3, 93, 3, 1068, 105, 818, 3, 3, 3, 91, 324, 3, 258, 90, 3, 324, 3, 3, 56, 3, 301, 3, 1210, 1692, 3, 25690, 431, 3, 3, 3, 91, 324, 3, 94, 3, 1844, 3, 3, 3, 1146, 163, 21822, 3, 3, 3, 56, 3, 3, 56, 3, 2197, 3, 6073, 3, 3, 29623], [3112, 130, 3, 284, 3, 5704, 3, 94, 3, 95, 661, 3, 2197, 4847]], [[3112, 130, 3, 284, 3, 5704, 3, 94, 3, 95, 661, 3, 2197, 4847], [301, 6393, 324, 3, 36461, 118, 238, 3, 93, 3, 2197, 938, 3, 3190, 75, 91, 3, 388, 3, 336, 3, 4224, 110, 21822, 3, 3, 76, 2018, 3, 215, 518, 74, 3, 3, 93, 88, 21822, 3, 3, 92, 3, 848, 535, 614, 3, 4516, 25307, 3, 353, 3, 95, 94, 3, 10032, 324, 3, 317, 21822, 3, 3, 575, 3, 4224, 3, 92, 7278, 3, 3, 47, 3, 93, 3, 3313, 3, 88, 9813, 1964, 3, 477, 3, 3, 67, 3, 317, 21822, 3, 3, 6494, 114, 21822, 3, 3, 488, 664, 2197, 2518, 21822, 3, 3, 388, 3, 95, 661, 3, 477, 3, 3, 9746, 3, 4224, 21822, 3, 3, 93, 8714, 892, 3, 2197, 12334, 3, 93, 3079, 1451, 3, 15227]], [[301, 6393, 324, 3, 36461, 118, 238, 3, 93, 3, 2197, 938, 3, 3190, 75, 91, 3, 388, 3, 336, 3, 4224, 110, 21822, 3, 3, 76, 2018, 3, 215, 518, 74, 3, 3, 93, 88, 21822, 3, 3, 92, 3, 848, 535, 614, 3, 4516, 25307, 3, 353, 3, 95, 94, 3, 10032, 324, 3, 317, 21822, 3, 3, 575, 3, 4224, 3, 92, 7278, 3, 3, 47, 3, 93, 3, 3313, 3, 88, 9813, 1964, 3, 477, 3, 3, 67, 3, 317, 21822, 3, 3, 6494, 114, 21822, 3, 3, 488, 664, 2197, 2518, 21822, 3, 3, 388, 3, 95, 661, 3, 477, 3, 3, 9746, 3, 4224, 21822, 3, 3, 93, 8714, 892, 3, 2197, 12334, 3, 93, 3079, 1451, 3, 15227], [1569, 359, 3, 404, 3, 3, 3, 7850, 219, 3, 92, 3, 3313, 3, 2854, 81, 3, 3412, 1459, 567, 3, 3412, 1459, 567, 3, 8142, 3, 3412, 1459, 567, 3, 4516, 25307, 98, 21822, 3, 3, 92, 3, 19638, 3, 3968, 176, 3, 2197, 3, 93, 3120, 95, 207, 3, 3, 238, 3, 239, 1881, 132, 3, 10577, 3, 299, 308, 3350, 360, 3, 75, 3, 95, 661, 3, 353, 3, 1068, 7183, 200, 3, 4761, 3, 22443, 3, 11031, 3, 3, 490, 21822, 3, 3, 169, 5252, 4082, 21822, 3, 3, 269, 3, 477, 149, 21822, 3, 3, 227, 3190, 3, 26605, 110, 21822, 3, 3, 383, 98, 21822, 3, 21822, 3, 3, 2197, 3, 2197, 3, 301, 1210, 85, 21822, 3, 3, 1611, 3, 3, 3, 95, 3, 3, 215, 21822, 3, 3, 92, 4655, 3, 47, 3, 575, 3, 12992, 3, 353, 16035, 95, 3, 2923, 21530, 3, 17248, 8268, 3, 47, 24072, 3, 4224, 86, 3, 2197, 3, 10577, 3, 45820, 67, 3, 17248, 5766, 3, 570, 21822, 3, 8325, 21822, 3, 3, 75, 3, 11517, 6412, 21822, 3, 3, 169, 5252, 4082, 21822, 3, 3, 47, 3, 1145, 3, 3, 131, 3322, 3, 1709, 3, 16581, 544, 3, 1328, 683, 3, 388, 3, 1210, 85, 21822, 3, 3, 22443, 3, 27921, 173, 21822, 3, 3, 301, 94, 183, 3, 3, 3, 301, 388, 215, 3, 3, 56, 3, 317, 21822, 3, 3, 21592, 3, 477, 3, 3, 9746, 3, 4224, 163, 3, 67, 3, 46, 2728, 943, 3, 43750, 3, 29317, 5571, 3, 4224, 163, 3, 47, 3482, 84, 3, 2777, 3, 88, 21822, 3, 380, 3, 7850, 4049, 3, 353, 14227, 78, 3, 3, 4224, 2139, 88, 3, 3, 93, 3, 92, 3, 3968, 176, 3, 1709, 3, 227, 3, 3, 3, 488, 664, 19586, 28602, 3, 169, 210, 3, 15390, 223, 21822, 3, 1326, 98, 21822, 3, 206, 3, 673, 3, 1146, 173, 3, 3, 5379, 3, 591, 3, 258, 84, 3, 138, 546, 3, 3, 238, 3, 383, 679, 90, 21822, 3, 3, 269, 3, 1617, 3, 3, 258, 3, 4224, 3, 92, 3, 1642, 3, 2774, 261, 3, 324, 3, 3, 56, 324, 3, 4761, 3, 22443, 3, 301, 5777, 67]], [[1569, 359, 3, 404, 3, 3, 3, 7850, 219, 3, 92, 3, 3313, 3, 2854, 81, 3, 3412, 1459, 567, 3, 3412, 1459, 567, 3, 8142, 3, 3412, 1459, 567, 3, 4516, 25307, 98, 21822, 3, 3, 92, 3, 19638, 3, 3968, 176, 3, 2197, 3, 93, 3120, 95, 207, 3, 3, 238, 3, 239, 1881, 132, 3, 10577, 3, 299, 308, 3350, 360, 3, 75, 3, 95, 661, 3, 353, 3, 1068, 7183, 200, 3, 4761, 3, 22443, 3, 11031, 3, 3, 490, 21822, 3, 3, 169, 5252, 4082, 21822, 3, 3, 269, 3, 477, 149, 21822, 3, 3, 227, 3190, 3, 26605, 110, 21822, 3, 3, 383, 98, 21822, 3, 21822, 3, 3, 2197, 3, 2197, 3, 301, 1210, 85, 21822, 3, 3, 1611, 3, 3, 3, 95, 3, 3, 215, 21822, 3, 3, 92, 4655, 3, 47, 3, 575, 3, 12992, 3, 353, 16035, 95, 3, 2923, 21530, 3, 17248, 8268, 3, 47, 24072, 3, 4224, 86, 3, 2197, 3, 10577, 3, 45820, 67, 3, 17248, 5766, 3, 570, 21822, 3, 8325, 21822, 3, 3, 75, 3, 11517, 6412, 21822, 3, 3, 169, 5252, 4082, 21822, 3, 3, 47, 3, 1145, 3, 3, 131, 3322, 3, 1709, 3, 16581, 544, 3, 1328, 683, 3, 388, 3, 1210, 85, 21822, 3, 3, 22443, 3, 27921, 173, 21822, 3, 3, 301, 94, 183, 3, 3, 3, 301, 388, 215, 3, 3, 56, 3, 317, 21822, 3, 3, 21592, 3, 477, 3, 3, 9746, 3, 4224, 163, 3, 67, 3, 46, 2728, 943, 3, 43750, 3, 29317, 5571, 3, 4224, 163, 3, 47, 3482, 84, 3, 2777, 3, 88, 21822, 3, 380, 3, 7850, 4049, 3, 353, 14227, 78, 3, 3, 4224, 2139, 88, 3, 3, 93, 3, 92, 3, 3968, 176, 3, 1709, 3, 227, 3, 3, 3, 488, 664, 19586, 28602, 3, 169, 210, 3, 15390, 223, 21822, 3, 1326, 98, 21822, 3, 206, 3, 673, 3, 1146, 173, 3, 3, 5379, 3, 591, 3, 258, 84, 3, 138, 546, 3, 3, 238, 3, 383, 679, 90, 21822, 3, 3, 269, 3, 1617, 3, 3, 258, 3, 4224, 3, 92, 3, 1642, 3, 2774, 261, 3, 324, 3, 3, 56, 324, 3, 4761, 3, 22443, 3, 301, 5777, 67], [477, 3, 3, 91, 3, 94, 3, 2980, 3, 67, 3, 3190, 75, 91, 3, 254, 3, 7208, 3, 67, 3, 3190, 75, 91, 3, 94, 3, 94, 3, 94, 3, 308, 4540, 3, 383, 43129, 252, 3, 501]], [[477, 3, 3, 91, 3, 94, 3, 2980, 3, 67, 3, 3190, 75, 91, 3, 254, 3, 7208, 3, 67, 3, 3190, 75, 91, 3, 94, 3, 94, 3, 94, 3, 308, 4540, 3, 383, 43129, 252, 3, 501], [7388, 3, 36461, 118, 238, 3, 2932, 3, 9216, 3, 3, 33118, 3, 1500, 19586, 23671, 3, 673, 3, 4592, 3, 3, 2447, 3, 353, 14227, 132, 3, 94, 3, 93, 1413, 855, 21822, 3, 3, 93, 3, 9486, 21822, 3]], []]
Evaluation: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7385/7385 [37:58<00:00,  3.24it/s]
{}
```

## Expected behavior

Trainer returns dictionary with statistics about model performance.


Comments:
You did not provide any metrics to your `Trainer` and it looks like your dataset has no labels. `Trainer.evaluate` thus can't return anything useful.