Importing unittests using python unittest framework
Author: nikhil1008
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1034-azure-x86_64-with-debian-buster-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0a0+8f84ded (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@sgugger


 

## Information

I am trying to run unit tests after cloning fresh repo.  I use python unittest framework and try to load the tests using "unittest.defaultTestLoader". Most of the tests pass, but I get few errors. All the errors are same - "**cannot initialize type "_CudaDeviceProperties": an object with that name is already defined**". This problem goes away if I user python unittest command line . 

The problem arises when using:
* [ ] my own modified scripts: (give details below)
I use following script to load tests. 

```
import os
import glob
import unittest
test_files_path='./tests/*'
test_files = [os.path.basename(x) for x in glob.glob(test_files_path)]
module_strings = ['tests.'+test_file[0:len(test_file)-3] for test_file in test_files]
print(module_strings)
suites = [unittest.defaultTestLoader.loadTestsFromName(test_file) for test_file in module_strings]
test_suite = unittest.TestSuite(suites)
test_runner = unittest.TextTestRunner().run(test_suite)
```

## To reproduce

Steps to reproduce the behavior:
1. Paste above code in a python file under transformers directory. 
2.Run python file. 


## Expected behavior

All tests should pass since I just cloned a fresh repo. 

Comments:
Does the problem go away if you install the latest stable pytorch 1.6.0, see https://pytorch.org/get-started/locally/?

Also the test suite uses `pytest` to run the tests - have you tried using it?

```
pytest tests
```
Do you get the same issue running it this way?

Is it possible that your script ends up running tests in a different order from how they are normally tested on CI and uncovers some bug where several tests are dependent on each other and can only be run in a particular order? If so it might be helpful to find out the minimal sequence that leads to this error. 

I'm asking since the test suite is not 100% perfect, and I saw just recently that if you change the order of tests things start to fail.


I have run your script and didn't get this error - but got 15 other errors, most `RuntimeError: Physical devices cannot be modified after being initialized` in tf tests.
My env is:
```
- `transformers` version: 3.1.0
- Platform: Linux-4.15.0-112-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0.dev20200910 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.