Transfer CI to GitHub Actions
Opened by albertvillanova (closed)

This PR transfers CI from CircleCI to GitHub Actions. The implementation in GitHub Actions tries to be as faithful as possible to the implementation in CircleCI and get the same output results (exceptions below). 

**IMPORTANT NOTE**: The fast-fail policy (described below) is not finally implemented, so that:
- we can continue merging PRs with CI in red because of some random error returned by the Hub
- it is not annoying for maintainers to have to relaunch failed CI jobs

See comments here: https://github.com/huggingface/datasets/pull/4659#discussion_r918802348

Differences in the implementation in GitHub Actions compared to the CircleCI one:
- This PR introduces some *fail-fast* mechanisms to significantly reduce the total time CI is running, both because of environmental impact and because CI in GitHub Actions billing depends on the minutes per month running time (see [About billing for GitHub Actions](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions)):
  - All tests *depend* on `check_code_quality` job: only if `check_code_quality` passes, then the other test jobs are launched
  - The tests are implemented with a matrix strategy (cross-product: OS and PyArrow versions) and fail-fast: if any of the 4 processes fails, the others are cancelled
- OS dependencies for Linux (see table below)

   | OS dependencies | Passed tests | Skipped tests |
   | --- | ---: | ---: |
   | libsndfile1-dev | 4786 | 3119 |
   | libsndfile1 | 4786 | 3119 |
   | libsndfile1, sox | 4788 | 3117 |

  - This PR replaces `libsndfile1-dev` with `libsndfile1`: the same number of passing tests but less packages installed
  - This PR adds `sox`: required by MP3 tests (2 more tests are passed: 4788 instead of 4786)
- For tests using PyArrow 6, this PR uses 6.0.1 instead of 6.0.0

TO DO:
- [ ] Remove old CircleCI CI: kept for the moment to compare stability and performance

Close #4658.

## Comparison between CircleCI and GitHub Actions

| | | CircleCI | GitHub Actions |
| --- | --- | ---: | ---: |
| Ubuntu, pyarrow-latest ||||
|| Passed tests | 4786 | 4788 |
|| Duration | 11m 0s | 10m 10s |
| Windows, pyarrow-latest ||||
|| Passed tests | 4783 | 4783 |
|| Duration | 29m 59s | 22m 56s |

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot @albertvillanova ! I hope we're finally done with flakiness on windows ^^

Also thanks for paying extra attention to billing and avoiding running unnecessary jobs. Though for certain aspects (see my comments), I think it's worth having the extra jobs to make our life easier

~@lhoestq I think you forgot to add your comments?~

I had missed it among all the other comments...

@lhoestq, I'm specially enthusiastic with the fail-fast policy: it was in my TODO list for a long time. I really think it will have a positive impact (I would love to know the spent time saving it will enable, besides the carbon footprint reduction). :wink: 

So yes, as you said above, let's give it a try at least. If we encounter any inconvenience, we can easily disable it.

Question: I guess I have to disable CircleCI CI before merging this PR?