Try to make push CI less noisy on commit pages.
Author: ydshieh
Pull request: yes

# What does this PR do?

Try to make push CI less noisy on commit pages.
(Current commit page shows all push CI jobs (more than 256 now) status, which is noisy to check which tests failed.)

### Idea

  1. push to `main` -> trigger a workflow, that push to another branch `push-ci`
  2. push to `push-ci` -> trigger the actual push CI
~~- **TODO**: try to get failures in `2`, and add them to `1`. Fail the job if there is any failure.~~

Example run
[caller workflow run] https://github.com/huggingface/transformers/actions/runs/2358695597
[actual push CI run] https://github.com/huggingface/transformers/actions/runs/2358698408



Comments:
@LysandreJik This is still in WIP (just need to add back the failure report in the caller), but I think you can review it already :).

I follow [this guide](https://medium.com/prompt/trigger-another-github-workflow-without-using-a-personal-access-token-f594c21373ef) to add 2 keys in our `Settings`.

_The documentation is not available anymore as the PR was closed or merged._

@sgugger Is it necessary to show the push CI (non Circle CI ones) results (simplified version) on the commit history page?

The current approach needs to find a way to wait (on a GitHub hosted CPU machine) the `actual push CI` run finish and get back the results. This is not obvious, as the 2 workflow are somehow `independent`. I will try in this week if this is necessary.

Since those are reported on slack, I don't think so. Just having the circle CI results (since those are not on slack) is good IMO.

Hi @sgugger , @LysandreJik 

Sorry, I should have checked the results after this PR being merged.

It turns out that **the whole push CI jobs are still shown** in the dropdown menu when we click the green/red check/cross icons, see
https://github.com/huggingface/transformers/commit/39e146146b5545c89d3bc3cd5a0befd491757473

- It seems to me this check status **relies on the commit SHA**, rather than the branches where that workflow is triggered.
- We can use **on: workflow_run**, but we will lose important information in the workflow run page, see https://github.com/huggingface/accelerate/actions/workflows/on-merge.yml
- Changing the workflow/job names won't alter the order in the check status list

I will come back to this after the nightly PyTorch CI and past CI tasks, if it is OK for @sgugger .



I think it's more important to have something less noisy to debug when a break happened, personally.

It's indeed important to make debugging easier. But could you let me know which following works for you 🙏 

- changing the (push CI) trigger event to `on: workflow_run`: so we can see clearly on commit history page (for CircleCI tests) what go wrong, and we don't really care the push CI workflow run pages (less informative) - we rely on Slack push CI report
  - this could be done quickly (if everything is working fine)
- We should keep `on: push`, but try to run the whole tests as a single job
  - this will take more time - as we also like to keep the Slack report as the current format
  - so the question becomes if I should work on the past CI first, which is already delayed for a few month now.

And one remark: it seems to me that the `ci/circleci` checks are always at the end - after all Github Actions check status

The first option is what I asked for at the beginning :-). It's impossible to even see the full title of the GitHub Action failing jobs, so seeing their failures in the commit is completely useless IMO. The slack CI reports are great and more than enough for those failures.

> And one remark: it seems to me that the ci/circleci checks are always at the end - after all Github Actions check status

Yes, you still have to scroll through 200 checks on several commits when trying to debug where the break happened, so leaving as is is not a viable solution.