[Examples] Fixes inconsistency around eval vs val and predict vs test
Author: bhadreshpsavani
Pull request: yes

# What does this PR do?

Fixes #10165 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@stas00 @sgugger 


Comments:
Hi @sgugger and @stas00,
I have made changes in the following way,
| Earlier | Now |
| ---- | ---- |
| test | predict |
| test_examples | predict_examples |
| test_dataset | predict_dataset |
| max_test_samples | max_predict_samples |
| val | eval |
| val_examples | eval_examples |
| val_dataset | eval_dataset |
| max_val_samples | max_eval_samples |

* I have also made changes in the trainer code for the above variables
* I have modified the template file accordingly
* `examples/pytorch/question-answering/run_qa_no_trainer.py` don't have code complete code for predict stage. Shall we need to add? 



Hi @sgugger,
I was trying to test [run_qa_no_trainer.py](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_qa_no_trainer.py) and I came to know that when I passed `--test_file` it was giving me an error since we don't have such [argument](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_qa_no_trainer.py#L87) in the file.

When i give `--do_predict` it gives an error at this [line](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_qa_no_trainer.py#L507) since I was using squad data and it doesn't have `test` set

But as you said it will work perfectly for the dataset with `testset`.

Ah! now I get it. We can add support for a `--test_file` in another PR, yes!

I am exploring other files, I will add more changes in docs and other files

You're doing a very painful task, @bhadreshpsavani, as @sgugger commented we unfortunately can't normalize these 2 names fully both because of the backcompat and also where in some cases something is called validation/test split :( Thank you for doing this important work and bearing with all these setbacks.

Hi @stas00, 
It's totally fine. 
I was expecting more suggestions because I did a lot of changes for all files in a single go. I am enjoying this coding work that's important for me!

Hi @stas00,
Can we use this command?
`git push --force-with-lease origin myfeature`
I read that this is safe than `--force` and [many organizations even using this.](https://stackoverflow.com/questions/41283955/github-keeps-saying-this-branch-is-x-commits-ahead-y-commits-behind)

In general all PRs are isolated until they are merged, so if you make a mistake on your own PR, in the worst case you will make a mess of your own changes, but it won't impact the master. So feel free to experiment.

**edit:** this is for sure for when you don't have write access to upstream master, or if you work in your own fork. I'm not sure if it's the same when one does have a write access and is working directly on the source. I find it much safer to always do all the work in my own fork and merge upstream via PRs.

I have never used this particular flag before, so if it's safer go for it.

`run_tests_torch` CI job has been flakey as of recent, You can always force a CI restart if you see one or more CI jobs are failing unrelated to your commit with an empty commit:

```
git commit --allow-empty -m "Trigger CI"
git push
```

This is a cool command. I will definitely need this. I will note this down!
Thanks

Hi @sgugger and @stas00 ,
I have reverted trainer changes and updated the example pytorch readme.
Please let me know if we need to make more changes 

@sgugger, should these not be covered too?

```
$ grep -Ir max_val_samples
examples/tensorflow/text-classification/run_text_classification.py:    max_val_samples: Optional[int] = field(
examples/tensorflow/text-classification/run_text_classification.py:        if data_args.max_val_samples is not None:
examples/tensorflow/text-classification/run_text_classification.py:            eval_dataset = eval_dataset.select(range(data_args.max_val_samples))
examples/research_projects/wav2vec2/run_common_voice.py:    max_val_samples: Optional[int] = field(
examples/research_projects/wav2vec2/run_common_voice.py:    if data_args.max_val_samples is not None:
examples/research_projects/wav2vec2/run_common_voice.py:        eval_dataset = eval_dataset.select(range(data_args.max_val_samples))
examples/research_projects/wav2vec2/run_common_voice.py:        max_val_samples = data_args.max_val_samples if data_args.max_val_samples is not None else len(eval_dataset)
examples/research_projects/wav2vec2/run_common_voice.py:        metrics["eval_samples"] = min(max_val_samples, len(eval_dataset))
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:    max_val_samples: Optional[int] = field(
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:        if data_args.max_val_samples is not None:
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:            eval_dataset = eval_dataset.select(range(data_args.max_val_samples))
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:            max_val_samples = data_args.max_val_samples if data_args.max_val_samples is not None else len(eval_dataset)
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:            metrics["eval_samples"] = min(max_val_samples, len(eval_dataset))
```

```
$ grep -Ir max_test_samples
examples/tensorflow/text-classification/run_text_classification.py:    max_test_samples: Optional[int] = field(
examples/tensorflow/text-classification/run_text_classification.py:        if data_args.max_test_samples is not None:
examples/tensorflow/text-classification/run_text_classification.py:            test_dataset = test_dataset.select(range(data_args.max_test_samples))
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:    max_test_samples: Optional[int] = field(
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:        if data_args.max_test_samples is not None:
tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py:            test_dataset = test_dataset.select(range(data_args.max_test_samples))
```

This PR should be rebased on master and deal with `examples/tensorflow/text-classification/run_text_classification.py` that was added recently yes.

`examples/research_projects/wav2vec2/run_common_voice.py` is a research-project so is not actively maintained and pinned to a version of transformers, I would leave it out of this PR.

For `tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py`, I wouldn't touch it since it's a test script (which will ultimately be replaced by a TF example) but I'll let @philschmid decide on this one.

> For `tests/sagemaker/scripts/pytorch/run_glue_model_parallelism.py`, I wouldn't touch it since it's a test script (which will ultimately be replaced by a TF example) but I'll let @philschmid decide on this one.

Is a custom version for `SageMakerTrainer`, this will be removed after the deprecation of `SageMakerTrainer`, there is no need for adjustments. 

Thanks for adjusting! I think this is good to merge, @stas00 if you agree I'll let you click on the button :-)