Skipping more high mem tests - Wav2Vec2 Hubert
Author: amyeroberts
Pull request: yes

# What does this PR do?

In #21643 there's some tests which I forgot to skip e.g. for Wav2Vec2 I skipped the high memory tests for `TFWav2Vec2ModelTest` but didn't added them to the other class `TFWav2Vec2RobustModelTest`. This means some tests on circleci still fail with processes crashing - cf [this run](https://app.circleci.com/pipelines/github/huggingface/transformers/57822/workflows/479318b1-d4b2-4f72-8d98-7b23dde142e8/jobs/702252). This PR adds a `unittest.skip` decorator to the missed tests.

On this run,  `tests/models/wav2vec2_phoneme/test_tokenization_wav2vec2_phoneme.py::Wav2Vec2PhonemeCTCTokenizerTest::test_number_of_added_tokens` also failed with a crashing process. After inspecting, the following tests were also run on this process (`gw2`): 

```
tests/models/wav2vec2_phoneme/test_tokenization_wav2vec2_phoneme.py 
TFBartModelTest.test_keras_fit 
TFBartModelTest.test_keras_save_load 
TFGPTJModelTest.test_keras_save_load 
TFMBartModelTest.test_keras_save_load 
TFOpenAIGPTModelTest.test_keras_save_load 
TFRemBertModelTest.test_keras_save_load 
TFSwinModelTest.test_keras_save_load 
Wav2Vec2PhonemeCTCTokenizerTest.test_add_tokens_tokenizer 
Wav2Vec2PhonemeCTCTokenizerTest.test_added_token_are_matched_longest_first 
Wav2Vec2PhonemeCTCTokenizerTest.test_batch_encode_plus_batch_sequence_length 
Wav2Vec2PhonemeCTCTokenizerTest.test_batch_encode_plus_overflowing_tokens 
Wav2Vec2PhonemeCTCTokenizerTest.test_batch_encode_plus_padding 
Wav2Vec2PhonemeCTCTokenizerTest.test_call 
Wav2Vec2PhonemeCTCTokenizerTest.test_case_insensitive 
Wav2Vec2PhonemeCTCTokenizerTest.test_change_phonemizer_lang 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode_decode 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode_decode_with_del 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode_decode_with_del_filter 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode_plus_with_padding 
Wav2Vec2PhonemeCTCTokenizerTest.test_encode_with_del 
```

None of these models - PyTorch Wav2Vec2 Phoneme, TFOpenAIGPT, TFMBart, TFGPTJ, TFRemBert, TFSwin should have been affected by the PR #21502 which has cause the recent memory issues with Hubert and Wav2Vec2. I'm therefore unsure if resolving upstream issues with wav2vec2 and hubert will resolve this unfortunately. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
cc @ydshieh 

_The documentation is not available anymore as the PR was closed or merged._