Migrate from `setup.cfg` to `pyproject.toml`
Opened by mariosasko (closed)

Based on https://github.com/huggingface/huggingface_hub/pull/1971 in `hfh`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6619). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005066 / 0.011353 (-0.006287) | 0.003678 / 0.011008 (-0.007330) | 0.063057 / 0.038508 (0.024549) | 0.031250 / 0.023109 (0.008140) | 0.248856 / 0.275898 (-0.027042) | 0.266932 / 0.323480 (-0.056548) | 0.003814 / 0.007986 (-0.004172) | 0.002843 / 0.004328 (-0.001485) | 0.049210 / 0.004250 (0.044959) | 0.041514 / 0.037052 (0.004462) | 0.264874 / 0.258489 (0.006385) | 0.288834 / 0.293841 (-0.005007) | 0.027457 / 0.128546 (-0.101089) | 0.011071 / 0.075646 (-0.064575) | 0.206433 / 0.419271 (-0.212839) | 0.035381 / 0.043533 (-0.008152) | 0.246829 / 0.255139 (-0.008310) | 0.271094 / 0.283200 (-0.012106) | 0.017790 / 0.141683 (-0.123893) | 1.134618 / 1.452155 (-0.317536) | 1.182600 / 1.492716 (-0.310116) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.094970 / 0.018006 (0.076964) | 0.306438 / 0.000490 (0.305949) | 0.000212 / 0.000200 (0.000012) | 0.000044 / 0.000054 (-0.000010) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.017786 / 0.037411 (-0.019625) | 0.060652 / 0.014526 (0.046127) | 0.072619 / 0.176557 (-0.103937) | 0.119460 / 0.737135 (-0.617676) | 0.073580 / 0.296338 (-0.222759) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.279304 / 0.215209 (0.064095) | 2.747179 / 2.077655 (0.669524) | 1.438291 / 1.504120 (-0.065829) | 1.313405 / 1.541195 (-0.227789) | 1.354569 / 1.468490 (-0.113921) | 0.578375 / 4.584777 (-4.006402) | 2.424576 / 3.745712 (-1.321136) | 2.831513 / 5.269862 (-2.438348) | 1.756062 / 4.565676 (-2.809614) | 0.064460 / 0.424275 (-0.359815) | 0.005065 / 0.007607 (-0.002542) | 0.335003 / 0.226044 (0.108958) | 3.310500 / 2.268929 (1.041571) | 1.778017 / 55.444624 (-53.666607) | 1.504743 / 6.876477 (-5.371734) | 1.532843 / 2.142072 (-0.609229) | 0.662110 / 4.805227 (-4.143118) | 0.118239 / 6.500664 (-6.382425) | 0.042135 / 0.075469 (-0.033335) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.945650 / 1.841788 (-0.896137) | 11.623179 / 8.074308 (3.548871) | 10.927315 / 10.191392 (0.735923) | 0.131050 / 0.680424 (-0.549374) | 0.014725 / 0.534201 (-0.519476) | 0.290716 / 0.579283 (-0.288567) | 0.272357 / 0.434364 (-0.162007) | 0.323274 / 0.540337 (-0.217064) | 0.426692 / 1.386936 (-0.960244) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005478 / 0.011353 (-0.005875) | 0.003618 / 0.011008 (-0.007390) | 0.049599 / 0.038508 (0.011091) | 0.030814 / 0.023109 (0.007705) | 0.273663 / 0.275898 (-0.002235) | 0.292099 / 0.323480 (-0.031381) | 0.004196 / 0.007986 (-0.003790) | 0.002779 / 0.004328 (-0.001550) | 0.047812 / 0.004250 (0.043562) | 0.045095 / 0.037052 (0.008043) | 0.286288 / 0.258489 (0.027799) | 0.314125 / 0.293841 (0.020284) | 0.047940 / 0.128546 (-0.080606) | 0.010714 / 0.075646 (-0.064932) | 0.057453 / 0.419271 (-0.361819) | 0.033482 / 0.043533 (-0.010051) | 0.273391 / 0.255139 (0.018252) | 0.284936 / 0.283200 (0.001736) | 0.017805 / 0.141683 (-0.123878) | 1.148303 / 1.452155 (-0.303852) | 1.185268 / 1.492716 (-0.307448) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.092442 / 0.018006 (0.074436) | 0.309908 / 0.000490 (0.309418) | 0.000213 / 0.000200 (0.000013) | 0.000043 / 0.000054 (-0.000012) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.022874 / 0.037411 (-0.014537) | 0.078238 / 0.014526 (0.063712) | 0.088844 / 0.176557 (-0.087713) | 0.127054 / 0.737135 (-0.610081) | 0.089809 / 0.296338 (-0.206530) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.292360 / 0.215209 (0.077151) | 2.842700 / 2.077655 (0.765045) | 1.571071 / 1.504120 (0.066951) | 1.450773 / 1.541195 (-0.090422) | 1.467090 / 1.468490 (-0.001400) | 0.583529 / 4.584777 (-4.001248) | 2.469284 / 3.745712 (-1.276428) | 2.844426 / 5.269862 (-2.425435) | 1.773336 / 4.565676 (-2.792341) | 0.064585 / 0.424275 (-0.359690) | 0.005098 / 0.007607 (-0.002509) | 0.342816 / 0.226044 (0.116771) | 3.363309 / 2.268929 (1.094381) | 1.922834 / 55.444624 (-53.521790) | 1.649702 / 6.876477 (-5.226774) | 1.672727 / 2.142072 (-0.469345) | 0.665015 / 4.805227 (-4.140212) | 0.124764 / 6.500664 (-6.375900) | 0.041564 / 0.075469 (-0.033905) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.988970 / 1.841788 (-0.852818) | 12.148983 / 8.074308 (4.074675) | 11.132697 / 10.191392 (0.941305) | 0.131596 / 0.680424 (-0.548828) | 0.015700 / 0.534201 (-0.518501) | 0.288819 / 0.579283 (-0.290464) | 0.276692 / 0.434364 (-0.157672) | 0.330260 / 0.540337 (-0.210078) | 0.421612 / 1.386936 (-0.965324) |

</details>
</details>

![](https://cml.dev/watermark.png#d627fb8357f39d78d79e704712609c7b34bdeba4 "CML watermark")