[testing] the test suite is many times slower than 2 weeks ago

We are going to have a CI-side running reports when this is merged https://github.com/huggingface/transformers/pull/7884, but we can already start looking at what caused a 4-5 times slowdown in the test suite about 10 days ago. I'm not sure the exact moment, but I checked a few reports and it appears that the change happened around Oct 8th +/- a few days.
e.g. before:
https://app.circleci.com/pipelines/github/huggingface/transformers/13323/workflows/5984ea0e-e280-4a41-bc4a-b4a3d72fc411/jobs/95699
after:
https://app.circleci.com/pipelines/github/huggingface/transformers/13521/workflows/d235c864-66fa-4408-a787-2efab850a781/jobs/97329

@sshleifer suggested a diagnostic to resolve this by adding a pytorch `--durations=N` flag, except if it's a missing `@slow` it won't work on my machine because I already have all the models pre-downloaded, so the following is just the slow execution:

Here is the report on my machine running all tests normally
```
$ pytest -n 3 --durations=0 tests
[...]
76.92s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_train_pipeline_custom_model
54.38s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_graph_mode
49.85s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_compile_tf_model
48.98s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_save_pretrained
44.11s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_compile_tf_model
38.42s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_graph_mode
35.94s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_tokenization_python_rust_equals
35.86s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_create_token_type_ids
35.81s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_embeded_special_tokens
35.58s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_max_length_equal
35.54s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_padding
35.36s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_is_fast
35.14s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_pretokenized_inputs
35.10s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_special_tokens_map_equal
35.07s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_num_special_tokens_to_add_equal
35.02s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_build_inputs_with_special_tokens
34.94s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_prepare_for_model
31.60s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_compile_tf_model
31.03s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_train_pipeline_custom_model
29.11s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_compile_tf_model
29.10s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_train_pipeline_custom_model
27.62s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_pt_tf_model_equivalence
26.36s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_compile_tf_model
25.12s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_save_pretrained
24.85s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_graph_mode
24.66s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_compile_tf_model
24.04s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_tokenization_python_rust_equals
23.15s call     tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_from_pretrained
23.10s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_graph_mode
23.08s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_padding
22.99s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_compile_tf_model
22.78s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_train_pipeline_custom_model
22.69s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_keras_save_load
22.67s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_train_pipeline_custom_model
22.43s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_pretokenized_inputs
22.38s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_create_token_type_ids
22.35s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_prepare_for_model
22.28s call     tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
22.25s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_max_length_equal
22.19s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_embeded_special_tokens
22.06s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_special_tokens_map_equal
21.95s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_is_fast
21.92s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_build_inputs_with_special_tokens
21.85s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_num_special_tokens_to_add_equal
21.61s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_graph_mode
21.49s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_add_special_tokens
21.32s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_add_tokens
21.21s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_alignement_methods
21.09s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_batch_encode_dynamic_overflowing
21.06s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_fast_only_inputs
20.95s call     tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
20.86s call     tests/test_tokenization_fast.py::SentencePieceFastTokenizerTest::test_offsets_mapping
20.06s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_model_outputs_equivalence
20.01s call     tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_train_pipeline_custom_model
19.62s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_attention_outputs
19.39s call     tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
18.78s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_save_pretrained
18.63s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_train_pipeline_custom_model
18.36s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_compile_tf_model
18.08s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_tokenization_python_rust_equals
17.85s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_save_load
17.54s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_train_pipeline_custom_model
17.39s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_train_pipeline_custom_model
17.28s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_embeded_special_tokens
17.25s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_special_tokens_map_equal
16.88s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_train_pipeline_custom_model
16.84s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_graph_mode
16.74s call     tests/test_modeling_electra.py::ElectraModelTest::test_torchscript
16.73s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_padding
16.63s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_max_length_equal
16.56s call     tests/test_modeling_fsmt.py::FSMTModelTest::test_lm_head_model_random_beam_search_generate
16.55s call     tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
16.53s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_build_inputs_with_special_tokens
16.53s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_is_fast
16.49s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_create_token_type_ids
16.45s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_num_special_tokens_to_add_equal
16.43s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_graph_mode
16.42s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_prepare_for_model
16.09s call     tests/test_tokenization_albert.py::AlbertTokenizationTest::test_pretokenized_inputs
16.02s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_model_outputs_equivalence
15.80s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_train_pipeline_custom_model
15.50s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_translation
15.30s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_train_pipeline_custom_model
15.00s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_graph_mode
14.96s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_compile_tf_model
14.07s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_train_pipeline_custom_model
14.03s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_pt_tf_model_equivalence
13.77s call     tests/test_modeling_rag.py::RagDPRT5Test::test_model_generate
13.29s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_hidden_states_output
13.10s call     tests/test_modeling_gpt2.py::GPT2ModelTest::test_model_outputs_equivalence
12.69s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_model_outputs_equivalence
12.43s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_save_pretrained
11.76s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_compile_tf_model
11.73s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_graph_mode
11.66s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_pt_tf_model_equivalence
11.63s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_special_tokens_map_equal
11.60s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_batch_encode_dynamic_overflowing
11.51s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_add_special_tokens
11.50s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_compile_tf_model
11.36s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_prepare_for_model
11.34s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_add_tokens
11.23s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_tokenization_python_rust_equals
11.19s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_fast_only_inputs
11.17s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_offsets_mapping
11.09s call     tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_xla
11.05s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_alignement_methods
11.04s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_is_fast
10.95s call     tests/test_tokenization_fast.py::WordPieceFastTokenizerTest::test_offsets_with_special_characters
10.81s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_embeded_special_tokens
10.71s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_max_length_equal
10.59s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_build_inputs_with_special_tokens
10.59s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_num_special_tokens_to_add_equal
10.56s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_pt_tf_model_equivalence
10.42s call     tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_hidden_state
10.39s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_create_token_type_ids
10.36s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_compile_tf_model
10.34s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_attention_outputs
10.31s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_model_outputs_equivalence
10.25s call     tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
10.15s call     tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_attentions
9.78s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_pt_tf_model_equivalence
9.76s call     tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
9.76s call     tests/test_modeling_bert.py::BertModelTest::test_model_outputs_equivalence
9.72s call     tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
9.50s call     tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
9.37s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_pt_tf_model_equivalence
9.31s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_batch_encode_dynamic_overflowing
9.31s call     tests/test_modeling_albert.py::AlbertModelTest::test_model_outputs_equivalence
9.30s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_save_load
9.10s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_model_outputs_equivalence
9.01s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_keras_save_load
8.88s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_add_tokens
8.81s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_offsets_mapping
8.80s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_keras_save_load
8.73s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_add_special_tokens
8.66s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_fast_only_inputs
8.60s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_pt_tf_model_equivalence
8.59s call     tests/test_tokenization_fast.py::RobertaFastTokenizerTest::test_alignement_methods
8.57s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_model_outputs_equivalence
8.50s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_compile_tf_model
8.34s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_summarization
8.33s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_lm_head_model_random_beam_search_generate
8.17s call     tests/test_modeling_albert.py::AlbertModelTest::test_torchscript
8.01s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_model_outputs_equivalence
7.96s call     tests/test_modeling_mobilebert.py::MobileBertModelTest::test_model_outputs_equivalence
7.88s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_attention_outputs
7.87s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_graph_mode
7.84s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_attention_outputs
7.84s call     tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
7.58s call     tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_attentions
7.57s call     tests/test_modeling_tf_transfo_xl.py::TFTransfoXLModelTest::test_train_pipeline_custom_model
7.51s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_save_load
7.47s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_attention_outputs
7.46s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_keyword_and_dict_args
7.36s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_save_load
7.36s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_determinism
7.32s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_graph_mode
7.23s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_attention_outputs
7.16s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_text_generation
7.05s call     tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_attentions
7.04s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_resize_token_embeddings
7.02s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_lm_head_model_random_beam_search_generate
6.90s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_train_pipeline_custom_model
6.88s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_hidden_states_output
6.82s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_save_load
6.73s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_lm_head_model_random_beam_search_generate
6.70s call     tests/test_modeling_flaubert.py::FlaubertModelTest::test_model_outputs_equivalence
6.60s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_save_load
6.57s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_keras_save_load
6.48s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_graph_mode
6.47s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_attention_outputs
6.44s call     tests/test_modeling_encoder_decoder.py::GPT2EncoderDecoderModelTest::test_encoder_decoder_model_generate
6.44s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_inputs_embeds
6.35s call     tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_compile_tf_model
6.25s call     tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_compile_tf_model
6.25s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_lm_head_model_random_beam_search_generate
6.11s call     tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_loss_computation
6.05s call     tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_attentions
5.98s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_save_load
5.93s call     tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_train_pipeline_custom_model
5.77s call     tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_compile_tf_model
5.72s call     tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
5.69s call     tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_hidden_state
5.65s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_fast_only_inputs
5.64s call     tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_multigpu_data_parallel_forward
5.60s call     tests/test_modeling_blenderbot.py::Blenderbot90MIntegrationTests::test_90_generation_from_short_input
5.58s call     tests/test_modeling_electra.py::ElectraModelTest::test_model_outputs_equivalence
5.57s call     tests/test_modeling_rag.py::RagDPRBartTest::test_model_with_encoder_outputs
5.56s call     tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
5.54s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_attention_outputs
5.54s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_padding
5.51s call     tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_attentions
5.46s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_alignement_methods
5.46s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_offsets_mapping
5.44s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_add_special_tokens
5.40s call     tests/test_tokenization_fast.py::NoPaddingTokenFastTokenizerMatchingTest::test_add_tokens
5.33s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_keras_save_load
5.30s call     tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
5.27s call     tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_graph_mode
5.22s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_hidden_states_output
5.20s call     tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_graph_mode
5.12s call     tests/test_pipelines.py::NerPipelineTests::test_tf_only_ner
5.11s call     tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
5.10s call     tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
5.09s call     tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_hidden_state
5.07s call     tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_graph_mode
5.05s call     tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_maximum_encoding_length_pair_input
5.04s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_graph_mode
5.01s call     tests/test_modeling_distilbert.py::DistilBertModelTest::test_torchscript_output_hidden_state
4.99s call     tests/test_modeling_fsmt.py::FSMTHeadTests::test_generate_fp16
4.94s call     tests/test_modeling_distilbert.py::DistilBertModelTest::test_model_outputs_equivalence
4.93s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_hidden_states_output
4.90s call     tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_hidden_state
4.85s call     tests/test_modeling_roberta.py::RobertaModelTest::test_torchscript
4.84s call     tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_train_pipeline_custom_model
4.82s call     tests/test_pipelines.py::QAPipelineTests::test_tf_question_answering
4.81s call     tests/test_modeling_encoder_decoder.py::BertEncoderDecoderModelTest::test_save_and_load_from_encoder_decoder_pretrained
4.78s call     tests/test_modeling_roberta.py::RobertaModelTest::test_torchscript_output_hidden_state
4.76s call     tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_maximum_encoding_length_single_input
4.73s call     tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_pretokenized_inputs
4.72s call     tests/test_pipelines.py::QAPipelineTests::test_torch_question_answering
4.68s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_pt_tf_model_equivalence
4.63s call     tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_add_special_tokens
4.60s call     tests/test_modeling_mobilebert.py::MobileBertModelTest::test_save_load
4.59s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_model_outputs_equivalence
4.59s call     tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_identifier_non_existent
4.57s call     tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_encoder_decoder_with_configs
4.56s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_hidden_states_output
4.51s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_text2text
4.48s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_keras_save_load
4.48s call     tests/test_tokenization_marian.py::MarianTokenizationTest::test_tokenizer_equivalence_en_de
4.47s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_keras_save_load
4.44s call     tests/test_modeling_flaubert.py::FlaubertModelTest::test_attention_outputs
4.43s call     tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_hidden_states_output
4.41s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_keyword_and_dict_args
4.39s call     tests/test_modeling_marian.py::TestMarian_en_ROMANCE::test_pipeline
4.34s call     tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_save_load
4.33s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_hidden_states_output
4.28s call     tests/test_tokenization_fsmt.py::FSMTTokenizationTest::test_pickle_tokenizer
4.28s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_text_generation
4.27s call     tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
4.16s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_resize_token_embeddings
4.12s call     tests/test_modeling_tf_t5.py::TFT5ModelTest::test_compile_tf_model
4.10s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_keras_save_load
4.09s call     tests/test_modeling_tf_t5.py::TFT5ModelTest::test_train_pipeline_custom_model
4.09s call     tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_save_load
4.08s call     tests/test_modeling_openai.py::OpenAIGPTModelTest::test_head_pruning_integration
4.07s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_keras_save_load
4.06s call     tests/test_modeling_roberta.py::RobertaModelTest::test_torchscript_output_attentions
4.04s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_fill_mask
3.96s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_determinism
3.96s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_pt_tf_model_equivalence
3.95s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_keras_save_load
3.95s setup    tests/test_modeling_marian.py::TestMarian_FR_EN::test_batch_generation_fr_en
3.93s call     tests/test_modeling_reformer.py::ReformerLocalAttnModelTest::test_model_outputs_equivalence
3.88s call     tests/test_pipelines.py::NerPipelineTests::test_ner_grouped
3.87s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_determinism
3.86s call     tests/test_pipelines.py::NerPipelineTests::test_torch_ner
3.86s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_feature_extraction
3.85s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_sentiment_analysis
3.82s call     tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
3.75s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_loss_computation
3.75s call     tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
3.73s call     tests/test_modeling_tf_t5.py::TFT5ModelTest::test_lm_head_model_random_beam_search_generate
3.72s call     tests/test_modeling_tf_distilbert.py::TFDistilBertModelTest::test_keras_save_load
3.70s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_fill_mask_with_targets
3.69s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_hidden_states_output
3.64s call     tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
3.62s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_tf_fill_mask_with_targets
3.60s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_feature_extraction
3.60s call     tests/test_pipelines.py::NerPipelineTests::test_tf_ner
3.60s call     tests/test_pipelines.py::ZeroShotClassificationPipelineTests::test_torch_zero_shot_classification
3.59s call     tests/test_modeling_bert.py::BertModelTest::test_torchscript
3.58s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_sentiment_analysis
3.57s setup    tests/test_modeling_marian.py::TestMarian_EN_DE_More::test_auto_config
3.53s call     tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_resize_token_embeddings
3.50s call     tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_fill_mask
3.50s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_keras_save_load
3.50s call     tests/test_modeling_dpr.py::DPRModelTest::test_torchscript
3.46s call     tests/test_modeling_bart.py::BARTModelTest::test_tiny_model
3.46s call     tests/test_modeling_tf_bert.py::TFBertModelTest::test_inputs_embeds
3.44s call     tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_hidden_state
3.42s call     tests/test_modeling_xlnet.py::XLNetModelTest::test_model_outputs_equivalence
3.42s call     tests/test_pipelines.py::ZeroShotClassificationPipelineTests::test_tf_zero_shot_classification
3.42s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_pt_tf_model_equivalence
3.40s setup    tests/test_modeling_marian.py::TestMarian_en_ROMANCE::test_batch_generation_en_ROMANCE_multi
3.40s call     tests/test_tokenization_albert.py::AlbertTokenizationTest::test_maximum_encoding_length_pair_input
3.40s call     tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_model_outputs_equivalence
3.39s call     tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_keyword_and_dict_args
3.36s call     tests/test_modeling_dpr.py::DPRModelTest::test_model_outputs_equivalence
3.35s setup    tests/test_modeling_marian.py::TestMarian_EN_DE_More::test_batch_generation_en_de
3.33s call     tests/test_modeling_bart.py::BARTModelTest::test_model_outputs_equivalence
3.32s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_determinism
3.32s call     tests/test_pipelines.py::NerPipelineTests::test_tf_ner_grouped
3.31s setup    tests/test_modeling_marian.py::TestMarian_en_ROMANCE::test_tokenizer_handles_empty
3.30s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_resize_token_embeddings
3.29s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_pt_tf_model_equivalence
3.28s call     tests/test_modeling_bert.py::BertModelTest::test_head_pruning_integration
3.28s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_model_outputs_equivalence
3.27s call     tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_attentions
3.26s call     tests/test_modeling_openai.py::OpenAIGPTModelTest::test_model_outputs_equivalence
3.23s setup    tests/test_modeling_marian.py::TestMarian_en_zh::test_batch_generation_eng_zho
3.23s setup    tests/test_modeling_marian.py::TestMarian_EN_FR::test_batch_generation_en_fr
3.22s call     tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
3.20s call     tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_attention_outputs
3.20s setup    tests/test_modeling_marian.py::TestMarian_RU_FR::test_batch_generation_ru_fr
3.19s call     tests/test_modeling_tf_ctrl.py::TFCTRLModelTest::test_lm_head_model_random_no_beam_search_generate
3.18s call     tests/test_modeling_tf_gpt2.py::TFGPT2ModelTest::test_keras_save_load
3.17s call     tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_pt_tf_model_equivalence
3.15s call     tests/test_modeling_tf_t5.py::TFT5ModelTest::test_graph_mode
3.13s call     tests/test_modeling_tf_roberta.py::TFRobertaModelTest::test_pt_tf_model_equivalence
3.13s setup    tests/test_modeling_marian.py::TestMarian_MT_EN::test_batch_generation_mt_en
3.12s call     tests/test_modeling_tf_transfo_xl.py::TFTransfoXLModelTest::test_compile_tf_model
3.12s setup    tests/test_modeling_marian.py::TestMarian_EN_DE_More::test_forward
3.11s call     tests/test_modeling_tf_openai.py::TFOpenAIGPTModelTest::test_model_outputs_equivalence
3.10s call     tests/test_modeling_tf_electra.py::TFElectraModelTest::test_keyword_and_dict_args
```
I made a 3-sec cut-off for this listing.

@sshleifer, @sgugger, @LysandreJik, @thomwolf


[CIs] report slow tests add --durations=0 to some pytest jobs

It appears that over the last 10 days or so the CIs have gone 4-5 times slower. From 2-3 minutes to 10-11 minutes for `run_tests_torch`.

This PR adds a report, generated by `pytest`, that lists the slowest tests so that we could quickly detect such regressions in our test suite's performance.

As suggested by @sshleifer, pytest can do the diagnostics using `--durations=` flag. I'm not sure what the best number to set it to - let's try with 50:

I propose adding this to:
* `run_tests_torch_and_tf` job - as it runs all non-slow tests in "real-time"
* all scheduled jobs that run slow tests

**edit**: Several fixes have been merged since this slowdown was found, so we are much better off and there is an effort to perform a major clean up here https://github.com/huggingface/transformers/pull/7895 so support this effort let's start with reporting the runtime of all tests, that is `--durations=0` (`pytest` currently doesn't have an option to report only tests slower than a certain run time) and once the clean up has been done, then we dial down to `--durations=50` to only monitor any new outliers.

@sshleifer, @sgugger, @LysandreJik


style: fix typo

# What does this PR do?

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.


style: fix typo in the README

# What does this PR do?

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 examples/token-classification: @stefan-it
 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* my own modified scripts: (give details below):

I am trying to run the following script
```
from transformers import BertTokenizer, BertConfig
from transformers import BertForTokenClassification, AdamW
import torch
import argparse
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
parser = argparse.ArgumentParser(description='BERT Keyword Extractor')
parser.add_argument('--sentence', type=str, default=' ',
                    help='sentence to get keywords')
parser.add_argument('--path', type=str, default='model.pt',
                    help='path to load model')
args = parser.parse_args()
tag2idx = {'B': 0, 'I': 1, 'O': 2}
tags_vals = ['B', 'I', 'O']
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)
model = BertForTokenClassification.from_pretrained("bert-base-uncased", num_labels=len(tag2idx))

def keywordextract(sentence, path):
    text = sentence
    tkns = tokenizer.tokenize(text)
    indexed_tokens = tokenizer.convert_tokens_to_ids(tkns)
    segments_ids = [0] * len(tkns)
    tokens_tensor = torch.tensor([indexed_tokens]).to(device)
    segments_tensors = torch.tensor([segments_ids]).to(device)
    model.load_state_dict(torch.load(path))
    model.eval()
    prediction = []
    logit = model(tokens_tensor, token_type_ids=None,
                                  attention_mask=segments_tensors)
    logit = logit.detach().cpu().numpy()
    prediction.extend([list(p) for p in np.argmax(logit, axis=2)])
    for k, j in enumerate(prediction[0]):
        if j==1 or j==0:
            print(tokenizer.convert_ids_to_tokens(tokens_tensor[0].to('cpu').numpy())[k], j)
keywordextract(args.sentence, args.path)
```
The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* my own task or dataset: (give details below)
SemEval2017 task 10

## To reproduce

Steps to reproduce the behavior:

1. run this script with any string ("Huggingface is a blessing for transfer learning")

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
The error is
```
Traceback (most recent call last):
  File "keyword-extractor.py", line 40, in <module>
    keywordextract(args.sentence, args.path)
  File "keyword-extractor.py", line 28, in keywordextract
    model.load_state_dict(torch.load(path))
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for BertForTokenClassification:
	size mismatch for classifier.weight: copying a param with shape torch.Size([2, 768]) from checkpoint, the shape in current model is torch.Size([3, 768]).
	size mismatch for classifier.bias: copying a param with shape torch.Size([2]) from checkpoint, the shape in current model is torch.Size([3]).
```
If I put in a model directly downloaded from huggingface archive, the error changes to
```
Traceback (most recent call last):
  File "keyword-extractor.py", line 40, in <module>
    keywordextract(args.sentence, args.path)
  File "keyword-extractor.py", line 28, in keywordextract
    model.load_state_dict(torch.load(path))
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for BertForTokenClassification:
	Missing key(s) in state_dict: "bert.embeddings.position_ids", "bert.embeddings.LayerNorm.weight", "bert.embeddings.LayerNorm.bias", "bert.encoder.layer.0.attention.output.LayerNorm.weight", "bert.encoder.layer.0.attention.output.LayerNorm.bias", "bert.encoder.layer.0.output.LayerNorm.weight", "bert.encoder.layer.0.output.LayerNorm.bias", "bert.encoder.layer.1.attention.output.LayerNorm.weight", "bert.encoder.layer.1.attention.output.LayerNorm.bias", "bert.encoder.layer.1.output.LayerNorm.weight", "bert.encoder.layer.1.output.LayerNorm.bias", "bert.encoder.layer.2.attention.output.LayerNorm.weight", "bert.encoder.layer.2.attention.output.LayerNorm.bias", "bert.encoder.layer.2.output.LayerNorm.weight", "bert.encoder.layer.2.output.LayerNorm.bias", "bert.encoder.layer.3.attention.output.LayerNorm.weight", "bert.encoder.layer.3.attention.output.LayerNorm.bias", "bert.encoder.layer.3.output.LayerNorm.weight", "bert.encoder.layer.3.output.LayerNorm.bias", "bert.encoder.layer.4.attention.output.LayerNorm.weight", "bert.encoder.layer.4.attention.output.LayerNorm.bias", "bert.encoder.layer.4.output.LayerNorm.weight", "bert.encoder.layer.4.output.LayerNorm.bias", "bert.encoder.layer.5.attention.output.LayerNorm.weight", "bert.encoder.layer.5.attention.output.LayerNorm.bias", "bert.encoder.layer.5.output.LayerNorm.weight", "bert.encoder.layer.5.output.LayerNorm.bias", "bert.encoder.layer.6.attention.output.LayerNorm.weight", "bert.encoder.layer.6.attention.output.LayerNorm.bias", "bert.encoder.layer.6.output.LayerNorm.weight", "bert.encoder.layer.6.output.LayerNorm.bias", "bert.encoder.layer.7.attention.output.LayerNorm.weight", "bert.encoder.layer.7.attention.output.LayerNorm.bias", "bert.encoder.layer.7.output.LayerNorm.weight", "bert.encoder.layer.7.output.LayerNorm.bias", "bert.encoder.layer.8.attention.output.LayerNorm.weight", "bert.encoder.layer.8.attention.output.LayerNorm.bias", "bert.encoder.layer.8.output.LayerNorm.weight", "bert.encoder.layer.8.output.LayerNorm.bias", "bert.encoder.layer.9.attention.output.LayerNorm.weight", "bert.encoder.layer.9.attention.output.LayerNorm.bias", "bert.encoder.layer.9.output.LayerNorm.weight", "bert.encoder.layer.9.output.LayerNorm.bias", "bert.encoder.layer.10.attention.output.LayerNorm.weight", "bert.encoder.layer.10.attention.output.LayerNorm.bias", "bert.encoder.layer.10.output.LayerNorm.weight", "bert.encoder.layer.10.output.LayerNorm.bias", "bert.encoder.layer.11.attention.output.LayerNorm.weight", "bert.encoder.layer.11.attention.output.LayerNorm.bias", "bert.encoder.layer.11.output.LayerNorm.weight", "bert.encoder.layer.11.output.LayerNorm.bias", "classifier.weight", "classifier.bias".
	Unexpected key(s) in state_dict: "cls.predictions.bias", "cls.predictions.transform.dense.weight", "cls.predictions.transform.dense.bias", "cls.predictions.transform.LayerNorm.gamma", "cls.predictions.transform.LayerNorm.beta", "cls.predictions.decoder.weight", "cls.seq_relationship.weight", "cls.seq_relationship.bias", "bert.pooler.dense.weight", "bert.pooler.dense.bias", "bert.embeddings.LayerNorm.gamma", "bert.embeddings.LayerNorm.beta", "bert.encoder.layer.0.attention.output.LayerNorm.gamma", "bert.encoder.layer.0.attention.output.LayerNorm.beta", "bert.encoder.layer.0.output.LayerNorm.gamma", "bert.encoder.layer.0.output.LayerNorm.beta", "bert.encoder.layer.1.attention.output.LayerNorm.gamma", "bert.encoder.layer.1.attention.output.LayerNorm.beta", "bert.encoder.layer.1.output.LayerNorm.gamma", "bert.encoder.layer.1.output.LayerNorm.beta", "bert.encoder.layer.2.attention.output.LayerNorm.gamma", "bert.encoder.layer.2.attention.output.LayerNorm.beta", "bert.encoder.layer.2.output.LayerNorm.gamma", "bert.encoder.layer.2.output.LayerNorm.beta", "bert.encoder.layer.3.attention.output.LayerNorm.gamma", "bert.encoder.layer.3.attention.output.LayerNorm.beta", "bert.encoder.layer.3.output.LayerNorm.gamma", "bert.encoder.layer.3.output.LayerNorm.beta", "bert.encoder.layer.4.attention.output.LayerNorm.gamma", "bert.encoder.layer.4.attention.output.LayerNorm.beta", "bert.encoder.layer.4.output.LayerNorm.gamma", "bert.encoder.layer.4.output.LayerNorm.beta", "bert.encoder.layer.5.attention.output.LayerNorm.gamma", "bert.encoder.layer.5.attention.output.LayerNorm.beta", "bert.encoder.layer.5.output.LayerNorm.gamma", "bert.encoder.layer.5.output.LayerNorm.beta", "bert.encoder.layer.6.attention.output.LayerNorm.gamma", "bert.encoder.layer.6.attention.output.LayerNorm.beta", "bert.encoder.layer.6.output.LayerNorm.gamma", "bert.encoder.layer.6.output.LayerNorm.beta", "bert.encoder.layer.7.attention.output.LayerNorm.gamma", "bert.encoder.layer.7.attention.output.LayerNorm.beta", "bert.encoder.layer.7.output.LayerNorm.gamma", "bert.encoder.layer.7.output.LayerNorm.beta", "bert.encoder.layer.8.attention.output.LayerNorm.gamma", "bert.encoder.layer.8.attention.output.LayerNorm.beta", "bert.encoder.layer.8.output.LayerNorm.gamma", "bert.encoder.layer.8.output.LayerNorm.beta", "bert.encoder.layer.9.attention.output.LayerNorm.gamma", "bert.encoder.layer.9.attention.output.LayerNorm.beta", "bert.encoder.layer.9.output.LayerNorm.gamma", "bert.encoder.layer.9.output.LayerNorm.beta", "bert.encoder.layer.10.attention.output.LayerNorm.gamma", "bert.encoder.layer.10.attention.output.LayerNorm.beta", "bert.encoder.layer.10.output.LayerNorm.gamma", "bert.encoder.layer.10.output.LayerNorm.beta", "bert.encoder.layer.11.attention.output.LayerNorm.gamma", "bert.encoder.layer.11.attention.output.LayerNorm.beta", "bert.encoder.layer.11.output.LayerNorm.gamma", "bert.encoder.layer.11.output.LayerNorm.beta".

```
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
To extract the keywords from a string


Fix bug in _sorted_checkpoints

I'm using transformers 3.3.1 and run a training script with `--save_total_limit 3`. I hit the exception below, and after debugging the code found that it wrongly tries to index into the `best_model_checkpoint`'s *str* rather than the `sorted_checkpoints` array. When running without the fix I got this exception:

```
Traceback (most recent call last):
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 921, in _save_training
    self._rotate_checkpoints(use_mtime=True)
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1283, in _rotate_checkpoints
    checkpoints_sorted = self._sorted_checkpoints(use_mtime=use_mtime)
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1274, in _sorted_checkpoints
    checkpoints_sorted[best_model_index],
TypeError: 'str' object does not support item assignment
```

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


question about `add_special_tokens` and embedding layer

# ❓ Questions & Help

when I call add_special_tokens in the tokniezer, new token that is not in the existing vocab is added. At this time, the size is different from the existing embedding layer, what happens inside?

1. Abandon the existing embedding layer and train a new.
2. train only new tokens (expand existing embedding layer)


[multiple models] skip saving/loading deterministic state_dict keys

Implementing https://github.com/huggingface/transformers/issues/7296

* [x] make testing save/load of `keys_to_never_save` a common test by moving it from `fsmt` modeling test
* [x] implement skipping for all models with static positional embeddings: mbart, pegasus, marian (fsmt had this feature in first place)

Fixes: #7296

@LysandreJik, @sshleifer, @sgugger


[wip] [pegasus] encode s/\r?\n/<n>/g + test

As discussed in https://github.com/huggingface/transformers/issues/7743

* [x] On encode: `s/\r?\n/<n>/g` + test
* [ ] On decode: `s/<n>/\n/g` - not sure what to do since `run_eval.py` can't work with multiline outputs as it uses readline to get each record and inserting `\n` would break the records if they are now multiline.

@sshleifer

Fixes: #7743


xlm-mlm-17-1280 & xlm-mlm-100-1280 include languages?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
Where is the list showing what languages are supported by each of these two models?


How to do categorical sequence classification?

Whenever I have a sequence of max length y of batch size x: (x, y) and categorical (z) classification (x, z) I get the error that batch size isn't the same x vs x*z, how do I fix this?


[Rag] extend_enc_output fails when number of retrieved documents not equal to RagConfig.n_docs

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Rag

The problem arises when using:
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: `dummy_dataset`

## To reproduce

Steps to reproduce the behavior:

1. Use caling retriever seperately example and but modify number of retrieved document parameter `n_docs` as follows -
```
docs_dict = retriever(input_ids.numpy(), question_hidden_states.detach().numpy(), return_tensors="pt", n_docs=2)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
Code snippet -
```
tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq")
retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=True)
model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq", use_dummy_dataset=True)

input_dict = tokenizer.prepare_seq2seq_batch("What is capital of France?", return_tensors="pt")
input_ids = input_dict["input_ids"]

# Caling retriever seperately
question_hidden_states = model.question_encoder(input_ids)[0]

# Default value of RagConfig.n_docs is 5 hence change n_docs to different than default value
docs_dict = retriever(input_ids.numpy(), question_hidden_states.detach().numpy(), return_tensors="pt", n_docs=2)
doc_scores = torch.bmm(question_hidden_states.unsqueeze(1), docs_dict["retrieved_doc_embeds"].float().transpose(1, 2)).squeeze(1)

outputs = model.generate(context_input_ids=docs_dict["context_input_ids"], context_attention_mask=docs_dict["context_attention_mask"], doc_scores=doc_scores)
generated_string = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(generated_string)
```
Stacktrace -
```
/usr/local/lib/python3.6/dist-packages/transformers/modeling_rag.py in generate(self, input_ids, attention_mask, context_input_ids, context_attention_mask, doc_scores, max_length, min_length, early_stopping, use_cache, num_beams, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, bad_words_ids, num_return_sequences, decoder_start_token_id, **kwargs)
   1353
   1354         # correctly extend last_hidden_state and attention mask
-> 1355         context_attention_mask = extend_enc_output(context_attention_mask, num_beams=num_beams)
   1356         encoder_outputs["last_hidden_state"] = extend_enc_output(last_hidden_state, num_beams=num_beams)
   1357

/usr/local/lib/python3.6/dist-packages/transformers/modeling_rag.py in extend_enc_output(tensor, num_beams)
   1346         def extend_enc_output(tensor, num_beams=None):
   1347             # split into `batch_size`, `num_beams`, `num_docs`
-> 1348             tensor = tensor[None, None, :].reshape((batch_size, 1, self.config.n_docs) + tensor.shape[1:])
   1349             # repeat same last hidden states over `num_beams` dimension
   1350             tensor = tensor.expand((batch_size, num_beams, self.config.n_docs) + tensor.shape[3:])

RuntimeError: shape '[0, 1, 5, 300]' is invalid for input of size 600
```
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I think `extend_enc_output` should use tensor dimension instead of  `self.config.n_docs` as follows -
```
tensor = tensor[None, None, :].reshape((batch_size, 1, tensor.shape[0:]) + tensor.shape[1:])
```
Let me know if proposed fix is fine, I can raise PR.


can't set evaluation_strategy to "epoch"

## Environment info

- `transformers` version: 3.3.1

### Who can help
@sgugger

## Information
Seems like we can't use the evaluation_strategy=epoch option. The default value of evaluate_during_training is boolean (False). In transformers/training_args.py, __post_init__ function, there is a check - `if self.evaluate_during_training is not None:` (line 326). If I'm not mistaken, it will always result in True. I think that if you change the line into `if self.evaluate_during_training:` the problem will be solved.

Thank you so much for your awesome work :)


Fix Rag example docstring

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7829

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


RAG generate function uses input_ids even when context_input_ids are given.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-5.4.0-51-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.8
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

I think @patrickvonplaten has been checking RAG issues

## Information

Model I am using RagTokenForGeneration:

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details below)

## To reproduce

One can use the demo for RAG [currently in PR](https://github.com/huggingface/transformers/pull/7455) but it will happen in any case.

1. Load a RagTokenForGeneration model.
2. Generate the context_input_ids (in the demo done doing a forward pass.
3. use the generate function without giving `input_ids` which is supposed to be an optional input.
4. The function check the `batch_size` using `input_ids` and breaks since they are a None in this line: https://github.com/huggingface/transformers/blob/9f7b2b243230a0ff7e61f48f852e3a5b6f6d86fa/src/transformers/modeling_rag.py#L1311

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

```
    query = "My question"
    tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq")
    rag_conf = RagConfig.from_pretrained("facebook/rag-token-nq")
    retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", question_encoder_tokenizer = tokenizer.question_encoder, generator_tokenizer = tokenizer.generator, index_name="custom", indexed_dataset=dataset)
    model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq", retriever=retriever)
    device = "cuda:0"
    input_ids = tokenizer(query, return_tensors="pt").input_ids.to(device)
    with torch.no_grad():
        # retrieve support docs
        retrieved_outputs = model(input_ids, labels=None, output_retrieved=True)
        dl_scores = retrieved_outputs.doc_scores[0].tolist()
        dp_scores = retrieved_outputs.doc_scores.softmax(dim=-1)[0].tolist()
        doc_dicts = retriever.index.get_doc_dicts(retrieved_outputs.retrieved_doc_ids)[0]
        support_docs = [
            {"score": ls, "proba": ns, "title": ti, "text": te}
            for ls, ns, ti, te in zip(dl_scores, dp_scores, doc_dicts["title"], doc_dicts["text"])
        ]
        # generate answers
        generated_ids = model.generate(
            context_input_ids=retrieved_outputs.context_input_ids,
            context_attention_mask=retrieved_outputs.context_attention_mask,
            doc_scores=retrieved_outputs.doc_scores,
            num_beams=4,
            num_return_sequences=4,
            min_length=2,
            max_length=64,
            length_penalty=1.0,
        )
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The batch size should be obtained differently. For instance `batch_size = doc_scores.shape[0]`.


Add missing comma

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Raise error when using AMP on non-CUDA device

The trainer currently implements native AMP in such a way that a GradScaler is always used. AFAIK, and as supported by [this post](https://github.com/pytorch/pytorch/issues/36169#issuecomment-611261781) and [this report on the forums](https://discuss.huggingface.co/t/training-gpt2-on-cpus/1603), this will only work on CUDA devices. Therefore, an error should probably be thrown when a user tries to use `--fp16` alongside a non-CUDA device.

This PR is currently very brief because I am uncertain about a few things:

- I am not sure if the current implementation of `--fp16` in the trainer works with TPUs;
- I am not sure about differences in behaviour between using `apex` and native AMP in this respect;
- I am not entirely sure whether `_post_init` is the right place to do an argument sanity check. Particularly because I am now calling `self.device` which will set the device through `_set_devices`, even though you may (?) want to delay that as long as possible until the arguments are used in the trainer.

cc @sgugger


Julibert model card

Related to pretrained model cards

Model card for Julibert Catalan model


Add AI-SOCO models

Related to pretrained model cards

# What does this PR do?

Adding AI-SOCO language and classification models.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@julien-c.


AttributeError: module 'tensorflow_core.keras.activations' has no attribute 'swish'

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

tensorflow 2.0.0
after "pip install transformers", i import transformers, meet this error.

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


labels and decoder_input_ids

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
what is the difference between labels  and decoder_input_ids for EncoderDecoderModel in the text summarization task?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Create model card for pre-trained NLI models.

Related to pretrained model cards


[testing] rename skip targets + docs

As discussed in https://github.com/huggingface/transformers/issues/6349 this PR brings consistency to skip decorators, so now we will have:

* `require_torch` - this test will run only under torch
* `require_torch_gpu` - as `require_torch` plus requires at least 1 GPU
* `require_torch_multigpu` - as `require_torch` plus requires at least 2 GPUs
* `require_torch_non_multigpu` - as `require_torch` plus requires 0 or 1 GPUs
* `require_torch_tpu` - as `require_torch` plus requires at least 1 TPU

Documentation updated and expanded.

The main change was done by running:
```
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|require_multigpu|require_torch_multigpu|g' {} \;
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|require_torch_and_cuda|require_torch_gpu|g' {} \;
find . -type d -name ".git" -prune -o -type f -exec perl -pi -e 's|require_non_multigpu|require_torch_non_multigpu|g' {} \;
```

Fixes: #6349

@LysandreJik, @sgugger, @sshleifer


unshared Albert

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
How can I remove ALBERT's parameter sharing during fine-tuning? (just use the albert as bert)
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


[testing] remove USE_CUDA

As discussed at https://github.com/huggingface/transformers/issues/6349 this PR removes `USE_CUDA`.

I don't think I needed to add `CUDA_VISIBLE_DEVICES=""` to `.circleci` config files since those CIs have no gpus anyway.

@LysandreJik, @sgugger, @sshleifer

fixes: #6349


[fsmt test] basic config test with online model + super tiny model

This PR does:

* [x] Seeing the issue of https://github.com/huggingface/transformers/pull/7659 I realized fsmt didn't have a basic non-slow test that loads the tokenizer via online model files. Luckily a totally unrelated examples test caught this issue in that PR, so adding a very simple quick test in the main test suite, so that it runs by the normal CI.
* [x] while I was building this test, I needed a new tiny model, so I refined the script https://github.com/huggingface/transformers/blob/master/scripts/fsmt/fsmt-make-tiny-model.py and made a new one that creates a 50 times smaller model, so we are now at 60KB, instead of 3MB.

@LysandreJik, @sshleifer


[s2s testing] turn all to unittests, use auto-delete temp dirs

Currently in `examples/seq2seq/test_seq2seq_examples.py` many tmp dirs remain uncleaned by the end of the test:

This PR does:
* Now that we have `parameterized`, convert all tests into unittests
* Now that we have unittests, we can use auto-delete tmp dirs easily using `get_auto_remove_tmp_dir()`
* convert 4 test files
* moved out the broken multi_gpu test - working on it separately here https://github.com/huggingface/transformers/pull/7281 - will re-add when it's complete.

@sshleifer


Trainer with Iterable Dataset

# What does this PR do?

Fixes #5990
Follows #5995 (closed because stale).
Merges all commits from master.

* accomodate iterable dataset without predefined length
* set it as 1 use case: provide max_steps, and NO num_epochs
* Is a merge of master and PR 5995

- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?
@sgugger


Create README.md

Related to pretrained model cards

# What does this PR do?

Creare model card for `cambridgeltl/BioRedditBERT-uncased`.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Remove duplicated mish activation function

# What does this PR do?
The mish activation function was repeated in the file. I removed the duplicate.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Model card for German BERT fine-tuned for LER/NER

Related to pretrained model cards


Fixing issue #7810

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # 7810

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.
@LysandreJik you suggested to open this PR here: https://github.com/huggingface/transformers/issues/7810

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


SequenceSummary class in modeling_utils.py

Hello,
I have a question about the documentation strings provided for the forward function of the SequenceSummary class from modeling_utils.py:
https://github.com/huggingface/transformers/blob/dc552b9b7025ea9c38717f30ad3d69c2a972049d/src/transformers/modeling_utils.py#L1484

So when `cls_index` is not specified as the argument in SequenceSummary() statement, is the last token of the sequence used for the classification task? The entire sentence for the description is somewhat awkward...

Thanks,


Upgrade PyTorch Lightning to 1.0.2

Updates Pytorch Lightning to 1.0.2, and moves early stopping callback into the general callbacks due to deprecation.


Trainer accepts iterable datasets

Fixes #5990
Follows PR #5995 (got closed for being stale...)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?
@sgugger @LysandreJik


OperatorNotAllowedInGraphError in dbmdz/bert-base-italian-cased for Token Classification

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-5.4.0-1028-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

@LysandreJik @stefan-it

## Information

Model I am using (Bert, XLNet ...):

Bert (dbmdz/bert-base-italian-cased)

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
I am following the tutorial in https://huggingface.co/transformers/custom_datasets.html#token-classification-with-w-nut-emerging-entities in order to fine-tune a custom Named Entity Recognition NN.

1. Load the tokenizer and tokenize the data
```
 tokenizer = BertTokenizerFast.from_pretrained('dbmdz/bert-base-italian-cased')
```
following the steps from [tutorial](https://huggingface.co/transformers/custom_datasets.html#token-classification-with-w-nut-emerging-entities) to obtain `train_dataset` and `val_dataset` .

2. Load the Token Classifier with pretrained weights from the model
 ```
from transformers import TFBertForTokenClassification
model = TFBertForTokenClassification.from_pretrained('dbmdz/bert-base-italian-cased', num_labels=len(unique_tags))
```
As a side note, this gives the following warning which I understand is to be expected since the loaded model was not fine tuned for Token Classification:
```
Some weights of the model checkpoint at dbmdz/bert-base-italian-cased were not used when initializing TFBertForTokenClassification: ['mlm___cls', 'nsp___cls']
- This IS expected if you are initializing TFBertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing TFBertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of TFBertForTokenClassification were not initialized from the model checkpoint at dbmdz/bert-base-italian-cased and are newly initialized: ['dropout_37', 'classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```
4. Train
```
optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)
model.compile(optimizer=optimizer, loss=model.compute_loss)
model.fit(train_dataset.shuffle(100).batch(16), epochs=3, batch_size=16)
```
This raises the following error:

```
Epoch 1/3

---------------------------------------------------------------------------
OperatorNotAllowedInGraphError            Traceback (most recent call last)
<ipython-input-23-5eb528629965> in <module>
      1 optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)
      2 model.compile(optimizer=optimizer, loss=model.compute_loss)
----> 3 model.fit(train_dataset.shuffle(100).batch(16), epochs=3, batch_size=16)

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
     64   def _method_wrapper(self, *args, **kwargs):
     65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
     67
     68     # Running inside `run_distribute_coordinator` already.

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
    851                 batch_size=batch_size):
    852               callbacks.on_train_batch_begin(step)
--> 853               tmp_logs = train_function(iterator)
    854               # Catch OutOfRangeError for Datasets of unknown size.
    855               # This blocks until the batch has finished executing.

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    578         xla_context.Exit()
    579     else:
--> 580       result = self._call(*args, **kwds)
    581
    582     if tracing_count == self._get_tracing_count():

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    625       # This is the first call of __call__, so we have to initialize.
    626       initializers = []
--> 627       self._initialize(args, kwds, add_initializers_to=initializers)
    628     finally:
    629       # At this point we know that the initialization is complete (or less

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
    504     self._concrete_stateful_fn = (
    505         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 506             *args, **kwds))
    507
    508     def invalid_creator_scope(*unused_args, **unused_kwds):

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2444       args, kwargs = None, None
   2445     with self._lock:
-> 2446       graph_function, _, _ = self._maybe_define_function(args, kwargs)
   2447     return graph_function
   2448

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   2775
   2776       self._function_cache.missed.add(call_context_key)
-> 2777       graph_function = self._create_graph_function(args, kwargs)
   2778       self._function_cache.primary[cache_key] = graph_function
   2779       return graph_function, args, kwargs

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   2665             arg_names=arg_names,
   2666             override_flat_arg_shapes=override_flat_arg_shapes,
-> 2667             capture_by_value=self._capture_by_value),
   2668         self._function_attributes,
   2669         # Tell the ConcreteFunction to clean up its graph once it goes out of

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    979         _, original_func = tf_decorator.unwrap(python_func)
    980
--> 981       func_outputs = python_func(*func_args, **func_kwargs)
    982
    983       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
    439         # __wrapped__ allows AutoGraph to swap in a converted function. We give
    440         # the function a weak reference to itself to avoid a reference cycle.
--> 441         return weak_wrapped_fn().__wrapped__(*args, **kwds)
    442     weak_wrapped_fn = weakref.ref(wrapped_fn)
    443

~/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

OperatorNotAllowedInGraphError: in user code:

    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:533 train_step  **
        y, y_pred, sample_weight, regularization_losses=self.losses)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/engine/compile_utils.py:205 __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:143 __call__
        losses = self.call(y_true, y_pred)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/keras/losses.py:246 call
        return self.fn(y_true, y_pred, **self._fn_kwargs)
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/transformers/modeling_tf_utils.py:178 compute_loss
        if tf.math.reduce_any(labels == -1):
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:778 __bool__
        self._disallow_bool_casting()
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:545 _disallow_bool_casting
        "using a `tf.Tensor` as a Python `bool`")
    /home/ubuntu/anaconda3/envs/doc_backoffice/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:532 _disallow_when_autograph_enabled
        " decorating it directly with @tf.function.".format(task))

    OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did not convert this function. Try decorating it directly with @tf.function.
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I would expect the model to train without errors, thank you for your kind help!


how to save and load fine-tuned model?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
```python
class MyModel(nn.Module):
  def __init__(self, num_classes):
    super(MyModel, self).__init__()
    self.bert = BertModel.from_pretrained('hfl/chinese-roberta-wwm-ext', return_dict=True).to(device)
    self.fc = nn.Linear(768, num_classes, bias=False)

  def forward(self, x_input_ids, x_type_ids, attn_mask):
    outputs = self.bert(x_input_ids, token_type_ids=x_type_ids, attention_mask=attn_mask)
    pred = self.fc(outputs.pooler_output)
    return pred

model = MyModel(num_classes).to(device)
# save

# load

```
I have defined my model via huggingface, but I don't know how to save and load the model, hopefully someone can help me out, thanks!