use `@unittest.skipIf` decorators inside tokenizer's tests instead of `if ...: return` 
Author: SaulLu
Pull request: no

### Feature request

Currently in tokenizer testing, many tests coded in `test_tokenization_common.py` are not relevant for all tokenizers. In most cases, when the test is not relevant, it is still run but no verification is done in it. See the snippet below for example:
https://github.com/huggingface/transformers/blob/114295c010dd9c94d48add7a0f091ba6ebdf482b/tests/test_tokenization_common.py#L384-L396

I would like to propose to replace these if tests in the test methods by `@unittest.skipIf` decorators. On the previous example it would give:
```python
    @unittest.skipIf(not test_sentencepiece, "Not testing sentencepiece")
    def test_subword_regularization_tokenizer(self) -> None:
        # Subword regularization is only available for the slow tokenizer.
        sp_model_kwargs = {"enable_sampling": True, "alpha": 0.1, "nbest_size": -1}
        tokenizer = self.get_tokenizer(sp_model_kwargs=sp_model_kwargs)

        self.assertTrue(hasattr(tokenizer, "sp_model_kwargs"))
        self.assertIsNotNone(tokenizer.sp_model_kwargs)
        self.assertTrue(isinstance(tokenizer.sp_model_kwargs, dict))
        self.assertEqual(tokenizer.sp_model_kwargs, sp_model_kwargs)
        self.check_subword_sampling(tokenizer)
```

### Motivation

The problem with the current method is that we don't have a view on the number of tests actually performed on each type of tokenizers. If errors are made in the configuration of the test classes, we can have a green check for all the tests but in reality nothing has been checked

### Your contribution

If you ever find it relevant, I can make the changes or let someone else who would be available to do it before me.

Comments:
Let me ping you @ydshieh as it seems to me that you have a great knowledge of testing and your opinion on it could be really appreciated! (and cc @LysandreJik and @sgugger for visibility)

Would `test_sentencepiece` be an attribute of the common tester? Not sure where it comes from in your code sample ;-)

Oh yes you're right, I should have mentioned that! 

Indeed `test_sentencepiece` is an attribute of the test class (if you want to see how this decorator works on a very simple test class, you can see an example [here](https://www.tutorialspoint.com/unittest_framework/unittest_framework_skip_test.htm)). 

In `TokenizerTesterMixin` it is set to `False` by default but is sometimes overridden to `True` by the test class of a particular tokenizer.

I love the usage of `@unittest.skipIf`!

However, I see a problem with `@unittest.skipIf(not test_sentencepiece, "Not testing sentencepiece")`, as `test_sentencepiece` wouldn't be defined at the time this decoration (and the condition) is evaluated.

We will need to think of a solution :-)

FYI, the same exists in `transformers`, for example

```
    def test_tie_model_weights(self):
        if not self.test_torchscript:
            return
```

> I love the usage of @unittest.skipIf!

Yeah :raised_hands:!
 
> However, I see a problem with @unittest.skipIf(not test_sentencepiece, "Not testing sentencepiece"), as test_sentencepiece wouldn't be defined at the time this decoration (and the condition) is evaluated.

For the case of `test_sentencepiece` I think they will be defined before as they are defined here:

https://github.com/huggingface/transformers/blob/114295c010dd9c94d48add7a0f091ba6ebdf482b/tests/test_tokenization_common.py#L142

https://github.com/huggingface/transformers/blob/19420fd99e1f08a052a1d0d267f3496002d03618/tests/models/xlm_prophetnet/test_tokenization_xlm_prophetnet.py#L33

You are right, @SaulLu! I didn't know that the decoration will be evaluated with the class attributes :-) You know better than me 😆 So It makes the change much easier!


This works!
```python
import unittest


class DummyTest(unittest.TestCase):
    test_dummy = False

    @unittest.skipIf(not test_dummy, "not test dummy")
    def test_me(self):
        assert 1 == 2
```

Well, I am quite cautious, and found we have something more to deal with. The following will test both method. It looks like it only uses the value in `DummyTestMixin`, not the one overridden in the subclasses.

(it will work if we override `test_me` methods too with the skipIf) 

```
import unittest


class DummyTestMixin:
    test_dummy = True

    @unittest.skipIf(not test_dummy, "not test dummy")
    def test_me(self):
        assert 1 == 2


class DummyTest(DummyTestMixin, unittest.TestCase):
    test_dummy = True


class DummyNotTest(DummyTestMixin, unittest.TestCase):
    test_dummy = False

```

Oh no! It would have been so great to have! But cool that you saw it quickly @ydshieh 

We can do some research though. I can find some time to see if there is any approach.

From a brief look, it looks like the solution adopted in the model tester (testing the class variable at the beginning of the test and exiting early) is the easiest one.

I have one last suggestion: it seems to me that it is possible to skip a test from inside the test using `self.skipTest` ([doc](https://docs.python.org/3/library/unittest.html?highlight=skiptest#unittest.TestCase.skipTest) - it was introduced in python 3.1).

On the toy example it seems to work well:
```python
import unittest


class DummyTestMixin:
    test_dummy = True
    a =  2

    def test_me(self):
        if not self.test_dummy:
            self.skipTest("not test dummy")
        assert self.a == 2


class DummyTestPass(DummyTestMixin, unittest.TestCase):
    test_dummy = True


class DummyNotTestPass(DummyTestMixin, unittest.TestCase):
    test_dummy = False


class DummyTestNotPass(DummyTestMixin, unittest.TestCase):
    test_dummy = True
    a = 1


class DummyNotTestNotPass(DummyTestMixin, unittest.TestCase):
    test_dummy = False
    a = 1

if __name__ == "__main__":
    unittest.main()
```

What do you think about this?

This is awesome, @SaulLu ! Thank you :-). I would love this new approach to skip. Leave @sgugger and @LysandreJik for a final confirmation.

That works for me!

So cool! I'll try to take care of these changes today or tomorrow

Oops, it completely slipped my mind. I'll try to do this next week.