Errors while running pytest
Author: archanray
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

Hi!

I followed the installation procedures from source for a conda environment.
`make test-examples` returns with the following error:

```
FAILED examples/token-classification/test_ner_examples.py::ExamplesTests::test_run_ner - AssertionError: 2.1329751014709473 not less than 1.5

FAILED examples/test_examples.py::ExamplesTests::test_run_glue - AssertionError: 0.5 not greater than or equal to 0.75
```

I also tried `make test` but get the following errors:

```
FAILED tests/test_modeling_distilbert.py::DistilBertModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.

FAILED tests/test_modeling_electra.py::ElectraModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather input tensors must have the same number of dimensions: got 1, ...

FAILED tests/test_modeling_roberta.py::RobertaModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.

FAILED tests/test_modeling_bert.py::BertModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.

FAILED tests/test_modeling_albert.py::AlbertModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.

FAILED tests/test_modeling_xlnet.py::XLNetModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.
```

I followed the setup steps mentioned in the original readme. I am running the tests in a 8 GPU machine.
Please let me know how to fix this.

output of `trasnformers-cli env`

```
- `transformers` version: 2.11.0
- Platform: Linux-5.3.0-1023-aws-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes
```

Thanks!!

Comments:
Hi, could you provide the versions of your software, like asked in the issue template?

Namely tansformer version, python version, pytorch version, tensorflow version ...

updated

I still don't know what's your transformer version, which is arguably the most important version of the list :sweat_smile: 

Do you mind running `transformers-cli env` in your environment? It should output something along the lines of:

```
- `transformers` version: 2.11.0
- Platform: Linux-5.6.15-1-MANJARO-x86_64-with-arch-Manjaro-Linux
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

apologies :( , updated again.

I met the same error with the same environment.


@LousyLory Hi, have you solved this problem?
I have try python3.6/3.7 + torch 1.5/1.4 in venv/conda.
And I have checked my environment using `transformers-cli env`, which output  same with @LousyLory .
All of them fail the test.
For conda + python3.6 + torch 1.4:
the failure case is: (The other cases are just like this)
```
======================================================================================================================== short test summary info =========================================================================================================================
FAILED tests/test_modeling_electra.py::ElectraModelTest::test_multigpu_data_parallel_forward - RuntimeError: tensor.ndimension() == static_cast<int64_t>(expected_size.size()) INTERNAL ASSERT FAILED at /opt/conda/conda-bld/pytorch_1579022034529/work/torch/csrc/cud...
FAILED tests/test_modeling_gpt2.py::GPT2ModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather got an input of invalid size: got [2, 2, 4, 7, 8], but expected [2, 4, 4, 7, 8] (gather at /opt/conda/conda-bld/pytorch_1579022034529/work/torch/csrc/cud...
FAILED tests/test_modeling_xlnet.py::XLNetModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather got an input of invalid size: got [7, 2, 32], but expected [7, 4, 32] (gather at /opt/conda/conda-bld/pytorch_1579022034529/work/torch/csrc/cuda/comm.c...
FAILED tests/test_modeling_ctrl.py::CTRLModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather got an input of invalid size: got [2, 2, 4, 7, 8], but expected [2, 4, 4, 7, 8] (gather at /opt/conda/conda-bld/pytorch_1579022034529/work/torch/csrc/cud...
FAILED tests/test_modeling_mobilebert.py::MobileBertModelTest::test_multigpu_data_parallel_forward - RuntimeError: Caught RuntimeError in replica 0 on device 0.
================================================================================================= 5 failed, 1054 passed, 560 skipped, 302 warnings in 253.59s (0:04:13) ==================================================================================================
```


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.