CI workflow for performed test regressions
Author: ahadnagy
Pull request: yes

# What does this PR do?

This PR adds a new workflow to the CI that dumps a report about the differences in performed tests. We noticed that the test counts sometimes fluctuate quite a bit between CI runs, and this report will help in narrowing down the root cause and find regressions (removed test, catastrophic failure during the execution, CI flakiness, runner error etc. ).

The report looks like the following:
```
=== Diff for job: albert ===
--- Absent in current run:
    - SKIPPED [1] tests/test_modeling_common.py:4029
    - SKIPPED [1] tests/test_modeling_common.py:4085
    - SKIPPED [1] tests/test_modeling_common.py:4162
    - SKIPPED [1] tests/test_modeling_common.py:4233
+++ Appeared in current run:
    + PASSED tests/models/albert/test_modeling_albert.py::AlbertModelTest::test_flash_attention_2_padding_matches_padding_free_with_position_ids
    + PASSED tests/models/albert/test_modeling_albert.py::AlbertModelTest::test_flash_attention_2_padding_matches_padding_free_with_position_ids_and_fa_kwargs
    + PASSED tests/models/albert/test_modeling_albert.py::AlbertModelTest::test_flash_attn_2_fp32_ln
    + PASSED tests/models/albert/test_modeling_albert.py::AlbertModelTest::test_flash_attn_2_from_config
```


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
@ydshieh 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39198). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ahadnagy Great, thanks! I can run it, and I will work on the remaining part.

I'am not sure why, but I get something like, a single line

> === Diff for job: albert === /n--- Absent in current run: /n    - SKIPPED [1] tests/test_pipeline_mixin.py:800: ResNetModelTest::test_pipeline_zero_shot_object_detection_float16 is skipped: `zero-shot-object-detection` is not in `self.pipeline_model_mapping` for `ResNetModelTest`. /n+++ Appeared in current run: /n    + PASSED tests/models/resnet/test_modeling_resnet.py::ResNetModelTest::test_determinism_2 /n /n=== Diff for job: bloom === /n--- Absent in current run: /n    - SKIPPED [1] tests/test_pipeline_mixin.py:800: ResNetModelTest::test_pipeline_zero_shot_object_detection_float16 is skipped: `zero-shot-object-detection` is not in `self.pipeline_model_mapping` for `ResNetModelTest`. /n+++ Appeared in current run: /n    + PASSED tests/models/resnet/test_modeling_resnet.py::ResNetModelTest::test_determinism_2 /n


(on windows/Linux)

Merge. Hope it won't break everything!