make test failing
Author: merleyc
Pull request: no

## Environment info
- `transformers` version: 4.9.2
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.0
- PyTorch version (GPU?): 1.9.0+cu102 (False)
- Tensorflow version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

Documentation: @sgugger

## Information

Found this issue while following the [instructions](https://huggingface.co/transformers/contributing.html) on how to install transformers as [dev].
The [dev] command _pip install -e .[dev]_ gives me 58 failing tests.
This happens with python=3.8.0 and python=3.8.8. I am using py=3.8.0 because of this [related issue](https://github.com/huggingface/transformers/issues/9410).

The problem arises when using:
* [ X ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

conda create -n hf_py380 python=3.8.0
conda activate hf_py380
git clone https://github.com/myuser/transformers.git
cd transformers/
git checkout -b exploration
pip uninstall transformers
git clone https://github.com/huggingface/datasets
cd datasets
pip install -e ".[dev]"
cd ..
python -m pytest -n 3 --dist=loadfile -s -v ./tests/

As results, I am getting 60 failed tests. Log file below:

> -- Docs: https://docs.pytest.org/en/stable/warnings.html
=================================================== short test summary info ===================================================
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_encoder_decoder_with_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_encoder_decoder_with_configs - AssertionError: unexpected...
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_model_no_architectures - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_eager - AssertionError: unexpectedly None
FAILED tests/test_file_utils.py::GetFromCacheTests::test_bogus_url - requests.exceptions.ProxyError: HTTPSConnectionPool(hos...
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_with_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_graph - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_only_pretrain - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_eager - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_train_encoder_decoder_with_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_graph - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_train_no_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark.py::BenchmarkTest::test_train_with_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs - AssertionError: unexpectedly None
FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs - AssertionError: unexpectedly None
FAILED tests/test_generation_utils.py::GenerationIntegrationTests::test_beam_search_warning_if_max_length_is_passed - OSErro...
FAILED tests/test_modeling_tf_longformer.py::TFLongformerModelIntegrationTest::test_layer_attn_probs - OSError: Can't load w...
FAILED tests/test_modeling_tf_longformer.py::TFLongformerModelIntegrationTest::test_layer_global_attn - OSError: Can't load ...
FAILED tests/test_tokenization_blenderbot.py::Blenderbot3BTokenizerTests::test_3B_tokenization_same_as_parlai - requests.exc...
FAILED tests/test_tokenization_bart.py::TestTokenizationBart::test_tokenization_python_rust_equals - requests.exceptions.Pro...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_build_inputs_with_special_tokens - requests.except...
FAILED tests/test_tokenization_bart.py::TestTokenizationBart::test_tokenizer_mismatch_warning - IndexError: list index out o...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_compare_add_special_tokens - requests.exceptions.P...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_add_tokens - requests.exceptions.ProxyError: HT...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_compare_prepare_for_model - requests.exceptions.Pr...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_alignement_methods - requests.exceptions.ProxyE...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_batch_encode_dynamic_overflowing - requests.exc...
FAILED tests/test_tokenization_byt5.py::ByT5TokenizationTest::test_empty_target_text - requests.exceptions.ProxyError: HTTPS...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_compare_pretokenized_inputs - requests.exceptions....
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_create_token_type_ids - requests.exceptions.ProxyE...
FAILED tests/test_tokenization_byt5.py::ByT5TokenizationTest::test_eos_treatment - requests.exceptions.ProxyError: HTTPSConn...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_embeded_special_tokens - requests.exceptions.Proxy...
FAILED tests/test_tokenization_byt5.py::ByT5TokenizationTest::test_max_length_integration - requests.exceptions.ProxyError: ...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_fast_only_inputs - requests.exceptions.ProxyError:...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_is_fast - requests.exceptions.ProxyError: HTTPSCon...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_max_length_equal - requests.exceptions.ProxyError:...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_compare_prepare_for_model - requests.exceptions...
FAILED tests/test_tokenization_canine.py::CanineTokenizationTest::test_encoding_keys - requests.exceptions.ProxyError: HTTPS...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_num_special_tokens_to_add_equal - requests.excepti...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_compare_pretokenized_inputs - requests.exceptio...
FAILED tests/test_tokenization_big_bird.py::BigBirdTokenizationTest::test_padding - requests.exceptions.ProxyError: HTTPSCon...
FAILED tests/test_tokenization_camembert.py::CamembertTokenizationTest::test_create_token_type_ids - requests.exceptions.Pro...
FAILED tests/test_tokenization_squeezebert.py::SqueezeBertTokenizationTest::test_batch_encode_dynamic_overflowing - requests...
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_padding - requests.exceptions.ProxyError: HTTPSConnectionPool...
FAILED tests/test_tokenization_xlm_roberta.py::XLMRobertaTokenizationTest::test_alignement_methods - requests.exceptions.Pro...
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_padding_different_model_input_name - requests.exceptions.Prox...
FAILED tests/test_tokenization_xlm_roberta.py::XLMRobertaTokenizationTest::test_batch_encode_dynamic_overflowing - requests....
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_prepare_batch - requests.exceptions.ProxyError: HTTPSConnecti...
FAILED tests/test_tokenization_xlm_roberta.py::XLMRobertaTokenizationTest::test_build_inputs_with_special_tokens - requests....
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_save_pretrained - requests.exceptions.ProxyError: HTTPSConnec...
FAILED tests/test_tokenization_xlm_roberta.py::XLMRobertaTokenizationTest::test_compare_add_special_tokens - requests.except...
FAILED tests/test_tokenization_squeezebert.py::SqueezeBertTokenizationTest::test_build_inputs_with_special_tokens - requests...
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_special_tokens_initialization - requests.exceptions.ProxyErro...
FAILED tests/test_tokenization_squeezebert.py::SqueezeBertTokenizationTest::test_compare_add_special_tokens - requests.excep...
FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_special_tokens_map_equal - requests.exceptions.ProxyError: HT...
FAILED tests/test_tokenization_xlm_roberta.py::XLMRobertaTokenizationTest::test_compare_prepare_for_model - requests.excepti...
FAILED tests/test_trainer.py::TrainerIntegrationTest::test_mem_metrics - AssertionError: 'init_mem_cpu_alloc_delta' not foun...
ERROR tests/test_modeling_bart.py
ERROR tests/test_modeling_encoder_decoder.py
ERROR tests/test_modeling_flax_bart.py
ERROR tests/test_modeling_flax_marian.py
ERROR tests/test_modeling_flax_mbart.py
ERROR tests/test_modeling_fsmt.py
ERROR tests/test_modeling_rag.py
ERROR tests/test_skip_decorators.py
ERROR tests/deepspeed/test_deepspeed.py
ERROR tests/deepspeed/test_model_zoo.py
ERROR tests/extended/test_trainer_ext.py
ERROR tests/sagemaker/test_multi_node_data_parallel.py
ERROR tests/sagemaker/test_multi_node_model_parallel.py
ERROR tests/sagemaker/test_single_node_gpu.py
===================== 60 failed, 7773 passed, 2260 skipped, 653 warnings, 14 errors in 7663.28s (2:07:43) =====================

## Expected behavior

Since I am not changing the code, just cloning repo etc, I expetected having all tests as PASSED. What am I doing wrong?

Thank you!


Comments:
Just to complement my issue above, I got some errors and warnings when calling _transformers-cli env_
I believe it is not related with the issue since it is cuda related messages but I am sharing it anyway.

```
$ transformers-cli env
2021-08-24 08:21:49.993817: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-08-24 08:21:49.993881: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From /home/yuzhou/miniconda3/envs/hf_py380/lib/python3.8/site-packages/transformers/commands/env.py:50: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2021-08-24 08:21:52.426412: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-08-24 08:21:52.437177: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-08-24 08:21:52.437216: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2021-08-24 08:21:52.437251: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (sr507): /proc/driver/nvidia/version does not exist

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.9.2
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.0
- PyTorch version (GPU?): 1.9.0+cu102 (False)
- Tensorflow version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

```

Hello @merleyc! I can't see the full stack trace of your error logs, but it seems that you're having connection issues? It seems that nearly all errors are `requests.exceptions.ProxyError`

Thanks for observing that, @LysandreJik !
I was able to successfully use the commands below without setting proxies:
```
git remote add upstream https://github.com/huggingface/transformers.git
git pull upstream master
```
But I set the http_proxy, https_proxy and ftp_proxy and I am running again the tests. They will take >2h to complete. At least until now I don't see proxy related errors but already see FAILING tests, like:
```
24 [gw1] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_encoder_decoder_with_configs
82 [gw1] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs
83 tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain
84 [gw1] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain
```
Any idea why are these tests failing?
I will paste the entire log here once the tests are finished.

Thanks!

Hi,
I got 49 failing tests when running the test after setting the proxies. Below are the lines that contains FAILED on it. Please see the entire log file attached.

`
34:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_encoder_decoder_with_configs
82:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs
84:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain
148:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_model_no_architectures
264:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_encoder_decoder_with_configs
266:[gw0] FAILED tests/test_file_utils.py::GetFromCacheTests::test_bogus_url
268:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
272:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_with_configs
274:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_eager
282:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_graph
286:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_train_encoder_decoder_with_configs
290:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_only_pretrain
294:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_train_no_configs
298:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_eager
366:[gw2] FAILED tests/test_benchmark.py::BenchmarkTest::test_train_with_configs
454:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_graph
506:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs
528:[gw1] FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
14436:[gw2] FAILED tests/test_tokenization_distilbert.py::BertTokenizationTest::test_padding
14438:[gw0] FAILED tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_tokenizer_mismatch_warning
14450:[gw1] FAILED tests/test_tokenization_dpr.py::BertTokenizationTest::test_padding
18032:[gw2] FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_special_tokens_initialization
18036:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_add_special_tokens
18038:[gw2] FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_special_tokens_map_equal
18054:[gw2] FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_tokenization_python_rust_equals
18060:[gw2] FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_tokenizer_mismatch_warning
18222:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_num_special_tokens_to_add_equal
18258:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_prepare_for_model
18262:[gw2] FAILED tests/test_tokenization_small_blenderbot.py::BlenderbotSmallTokenizerTest::test_empty_word_small_tok
18330:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_pretokenized_inputs
18518:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_padding
18538:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_padding_different_model_input_name
18566:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_create_token_type_ids
18568:[gw2] FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_alignement_methods
18586:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_embeded_special_tokens
18602:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_save_pretrained
18614:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_equivalence_to_orig_tokenizer
18616:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_special_tokens_initialization
18626:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_is_fast
18646:[gw1] FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_tokenization_python_rust_equals
18650:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_max_length_equal
18654:[gw2] FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_build_inputs_with_special_tokens
18668:[gw2] FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_add_special_tokens
18674:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_num_special_tokens_to_add_equal
18688:[gw2] FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_prepare_for_model
18692:[gw1] FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_add_tokens
18698:[gw2] FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_pretokenized_inputs
18704:[gw0] FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_padding
20268:[gw2] FAILED tests/test_trainer.py::TrainerIntegrationTest::test_mem_metrics
59432:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_encoder_decoder_with_configs
59433:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs - As...
59434:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain
59435:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_no_model_no_architectures
59436:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_encoder_decoder_with_configs
59437:FAILED tests/test_file_utils.py::GetFromCacheTests::test_bogus_url - requests...
59438:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript - A...
59439:FAILED tests/test_benchmark.py::BenchmarkTest::test_inference_with_configs - ...
59440:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_eager
59441:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_graph
59442:FAILED tests/test_benchmark.py::BenchmarkTest::test_train_encoder_decoder_with_configs
59443:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_no_configs_only_pretrain
59444:FAILED tests/test_benchmark.py::BenchmarkTest::test_train_no_configs - Assert...
59445:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_eager
59446:FAILED tests/test_benchmark.py::BenchmarkTest::test_train_with_configs - Asse...
59447:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_inference_with_configs_graph
59448:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs - A...
59449:FAILED tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
59450:FAILED tests/test_tokenization_distilbert.py::BertTokenizationTest::test_padding
59451:FAILED tests/test_tokenization_deberta.py::DebertaTokenizationTest::test_tokenizer_mismatch_warning
59452:FAILED tests/test_tokenization_dpr.py::BertTokenizationTest::test_padding - r...
59453:FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_special_tokens_initialization
59454:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_add_special_tokens
59455:FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_special_tokens_map_equal
59456:FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_tokenization_python_rust_equals
59457:FAILED tests/test_tokenization_reformer.py::ReformerTokenizationTest::test_tokenizer_mismatch_warning
59458:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_num_special_tokens_to_add_equal
59459:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_prepare_for_model
59460:FAILED tests/test_tokenization_small_blenderbot.py::BlenderbotSmallTokenizerTest::test_empty_word_small_tok
59461:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_compare_pretokenized_inputs
59462:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_padding
59463:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_padding_different_model_input_name
59464:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_create_token_type_ids
59465:FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_alignement_methods
59466:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_embeded_special_tokens
59467:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_save_pretrained
59468:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_equivalence_to_orig_tokenizer
59469:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_special_tokens_initialization
59470:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_is_fast
59471:FAILED tests/test_tokenization_roberta.py::RobertaTokenizationTest::test_tokenization_python_rust_equals
59472:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_max_length_equal
59473:FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_build_inputs_with_special_tokens
59474:FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_add_special_tokens
59475:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_num_special_tokens_to_add_equal
59476:FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_prepare_for_model
59477:FAILED tests/test_tokenization_t5.py::T5TokenizationTest::test_add_tokens - r...
59478:FAILED tests/test_tokenization_squeezebert.py::BertTokenizationTest::test_compare_pretokenized_inputs
59479:FAILED tests/test_tokenization_pegasus.py::BigBirdPegasusTokenizationTest::test_padding
59480:FAILED tests/test_trainer.py::TrainerIntegrationTest::test_mem_metrics - Asse...`

[results_cli3_v2.txt](https://github.com/huggingface/transformers/files/7042503/results_cli3_v2.txt)

I'd appreciate any help! :)
Thanks!

Any thought on this issue, @sgugger and @LysandreJik ? Thanks!

It's impossible to know what went wrong without having the whole output of the tests. The log file does not contain the logs, just which test passed and which did not.

Hi @sgugger ,
Apologies but how can I get the log file ?
To get the results I sent to you in the file "results_cli3_v2.txt" I run this command:
`python -m pytest -n 3 --dist=loadfile -s -v ./tests/ >> results_cli3_v2.txt`

Hi @sgugger ,
I have tried this command, which includes `--tb=long `  :
`python -m pytest --tb=long -n 8 --dist=loadfile -s -v ./tests/ > ~/resultsCLI.txt`

Is this the log file that you mentioned that should contain the _whole output of the tests_? If not, please advise.

After runnning the mentioned command, I got 3 failing tests and the error
```
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/dsession.py", line 256, in worker_collectionfinish
INTERNALERROR>     self.sched.schedule()
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/scheduler/loadscope.py", line 341, in schedule
INTERNALERROR>     self._reschedule(node)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/scheduler/loadscope.py", line 323, in _reschedule
INTERNALERROR>     self._assign_work_unit(node)
INTERNALERROR>   File "/home/mypath/miniconda3/envs/hf-dev-py380_v2/lib/python3.8/site-packages/xdist/scheduler/loadscope.py", line 261, in _assign_work_unit
INTERNALERROR>     worker_collection = self.registered_collections[node]
INTERNALERROR> KeyError: <WorkerController gw10>

```
as showed in this output file: [resultsCLI-pytestv625.txt](https://github.com/huggingface/transformers/files/7140353/resultsCLI-pytestv625.txt)

I downgraded the pytest version from 6.2.5 to 6.2.2 as stated [here](https://stackoverflow.com/questions/66803324/how-can-i-resolve-an-error-running-pytest-in-parallel-via-xdist-in-bitbucket-pip), but didn't help it. The output file with pytest v6.2.2 is: [resultsCLI-pytestv622.txt](https://github.com/huggingface/transformers/files/7140352/resultsCLI-pytestv622.txt)

Please advise.
Thanks!

We need the stack trace and the error message of the failing test to understand what is going on, this is not it.

@sgugger 
Could you please tell me what is the command to get what you are looking for?
I didn’t find it in the documentation .
Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.