Title: Fix style with ruff 0.0.278
Author: lhoestq (MEMBER)
State: closed
Created: 2023-07-13T12:34:24Z


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6026). All of your documentation changes will be reflected on that endpoint.

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006444 / 0.011353 (-0.004909) | 0.003768 / 0.011008 (-0.007240) | 0.079625 / 0.038508 (0.041117) | 0.064490 / 0.023109 (0.041381) | 0.313858 / 0.275898 (0.037960) | 0.350810 / 0.323480 (0.027330) | 0.004804 / 0.007986 (-0.003182) | 0.002904 / 0.004328 (-0.001425) | 0.061728 / 0.004250 (0.057477) | 0.052265 / 0.037052 (0.015213) | 0.321246 / 0.258489 (0.062757) | 0.353873 / 0.293841 (0.060032) | 0.027510 / 0.128546 (-0.101036) | 0.007942 / 0.075646 (-0.067704) | 0.260518 / 0.419271 (-0.158754) | 0.045686 / 0.043533 (0.002153) | 0.316821 / 0.255139 (0.061682) | 0.337086 / 0.283200 (0.053886) | 0.022188 / 0.141683 (-0.119495) | 1.427345 / 1.452155 (-0.024810) | 1.476059 / 1.492716 (-0.016657) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.189640 / 0.018006 (0.171634) | 0.429724 / 0.000490 (0.429235) | 0.005314 / 0.000200 (0.005114) | 0.000076 / 0.000054 (0.000021) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.024412 / 0.037411 (-0.013000) | 0.073488 / 0.014526 (0.058962) | 0.083843 / 0.176557 (-0.092714) | 0.147849 / 0.737135 (-0.589286) | 0.085465 / 0.296338 (-0.210873) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.405314 / 0.215209 (0.190105) | 4.071471 / 2.077655 (1.993816) | 1.916252 / 1.504120 (0.412132) | 1.721616 / 1.541195 (0.180422) | 1.807187 / 1.468490 (0.338697) | 0.498045 / 4.584777 (-4.086732) | 3.057526 / 3.745712 (-0.688187) | 4.451424 / 5.269862 (-0.818437) | 2.764020 / 4.565676 (-1.801656) | 0.057665 / 0.424275 (-0.366610) | 0.006679 / 0.007607 (-0.000928) | 0.485733 / 0.226044 (0.259688) | 4.844367 / 2.268929 (2.575438) | 2.435359 / 55.444624 (-53.009265) | 2.111478 / 6.876477 (-4.764999) | 2.377448 / 2.142072 (0.235375) | 0.587997 / 4.805227 (-4.217230) | 0.125545 / 6.500664 (-6.375120) | 0.061509 / 0.075469 (-0.013960) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.229210 / 1.841788 (-0.612577) | 18.553994 / 8.074308 (10.479686) | 14.037877 / 10.191392 (3.846485) | 0.144230 / 0.680424 (-0.536194) | 0.016891 / 0.534201 (-0.517310) | 0.329039 / 0.579283 (-0.250244) | 0.357269 / 0.434364 (-0.077095) | 0.384222 / 0.540337 (-0.156115) | 0.521292 / 1.386936 (-0.865644) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006359 / 0.011353 (-0.004994) | 0.003721 / 0.011008 (-0.007287) | 0.062047 / 0.038508 (0.023539) | 0.065267 / 0.023109 (0.042158) | 0.360164 / 0.275898 (0.084266) | 0.402292 / 0.323480 (0.078812) | 0.005603 / 0.007986 (-0.002382) | 0.002966 / 0.004328 (-0.001363) | 0.062580 / 0.004250 (0.058330) | 0.053634 / 0.037052 (0.016582) | 0.362210 / 0.258489 (0.103721) | 0.404285 / 0.293841 (0.110444) | 0.027567 / 0.128546 (-0.100979) | 0.008119 / 0.075646 (-0.067528) | 0.067577 / 0.419271 (-0.351694) | 0.042867 / 0.043533 (-0.000666) | 0.361576 / 0.255139 (0.106437) | 0.389061 / 0.283200 (0.105862) | 0.021923 / 0.141683 (-0.119760) | 1.446259 / 1.452155 (-0.005895) | 1.490724 / 1.492716 (-0.001992) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.206433 / 0.018006 (0.188427) | 0.424178 / 0.000490 (0.423688) | 0.002340 / 0.000200 (0.002140) | 0.000069 / 0.000054 (0.000015) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.024955 / 0.037411 (-0.012456) | 0.077446 / 0.014526 (0.062920) | 0.088540 / 0.176557 (-0.088017) | 0.141225 / 0.737135 (-0.595910) | 0.089747 / 0.296338 (-0.206592) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.443738 / 0.215209 (0.228529) | 4.208887 / 2.077655 (2.131233) | 2.155127 / 1.504120 (0.651007) | 2.028178 / 1.541195 (0.486983) | 2.084903 / 1.468490 (0.616413) | 0.497530 / 4.584777 (-4.087247) | 3.069012 / 3.745712 (-0.676700) | 3.025184 / 5.269862 (-2.244678) | 1.904687 / 4.565676 (-2.660990) | 0.057526 / 0.424275 (-0.366749) | 0.006482 / 0.007607 (-0.001125) | 0.494692 / 0.226044 (0.268647) | 4.944437 / 2.268929 (2.675508) | 2.655989 / 55.444624 (-52.788635) | 2.331677 / 6.876477 (-4.544800) | 2.382396 / 2.142072 (0.240324) | 0.582019 / 4.805227 (-4.223209) | 0.125866 / 6.500664 (-6.374799) | 0.062908 / 0.075469 (-0.012561) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.294612 / 1.841788 (-0.547176) | 19.016152 / 8.074308 (10.941844) | 14.088828 / 10.191392 (3.897436) | 0.160842 / 0.680424 (-0.519582) | 0.017054 / 0.534201 (-0.517146) | 0.333647 / 0.579283 (-0.245636) | 0.348094 / 0.434364 (-0.086270) | 0.394970 / 0.540337 (-0.145367) | 0.551141 / 1.386936 (-0.835795) |

</details>
</details>

![](https://cml.dev/watermark.png#9e9cfe886792b30b5000808072a0f91ec8536749 "CML watermark")


<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007442 / 0.011353 (-0.003911) | 0.004302 / 0.011008 (-0.006707) | 0.087159 / 0.038508 (0.048651) | 0.095094 / 0.023109 (0.071985) | 0.315422 / 0.275898 (0.039524) | 0.346672 / 0.323480 (0.023192) | 0.005811 / 0.007986 (-0.002174) | 0.003597 / 0.004328 (-0.000731) | 0.066400 / 0.004250 (0.062150) | 0.065947 / 0.037052 (0.028894) | 0.323269 / 0.258489 (0.064780) | 0.353309 / 0.293841 (0.059468) | 0.032268 / 0.128546 (-0.096278) | 0.008696 / 0.075646 (-0.066950) | 0.291486 / 0.419271 (-0.127786) | 0.054609 / 0.043533 (0.011076) | 0.321061 / 0.255139 (0.065922) | 0.336907 / 0.283200 (0.053707) | 0.027338 / 0.141683 (-0.114345) | 1.496442 / 1.452155 (0.044287) | 1.576946 / 1.492716 (0.084229) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.229140 / 0.018006 (0.211134) | 0.487500 / 0.000490 (0.487010) | 0.002425 / 0.000200 (0.002225) | 0.000089 / 0.000054 (0.000034) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029351 / 0.037411 (-0.008060) | 0.089610 / 0.014526 (0.075084) | 0.097880 / 0.176557 (-0.078676) | 0.155947 / 0.737135 (-0.581189) | 0.098593 / 0.296338 (-0.197745) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.382911 / 0.215209 (0.167702) | 3.820363 / 2.077655 (1.742708) | 1.866385 / 1.504120 (0.362265) | 1.712910 / 1.541195 (0.171716) | 1.813863 / 1.468490 (0.345373) | 0.484884 / 4.584777 (-4.099893) | 3.678911 / 3.745712 (-0.066801) | 5.249908 / 5.269862 (-0.019953) | 3.099614 / 4.565676 (-1.466063) | 0.057449 / 0.424275 (-0.366826) | 0.007728 / 0.007607 (0.000120) | 0.462123 / 0.226044 (0.236078) | 4.603942 / 2.268929 (2.335014) | 2.380957 / 55.444624 (-53.063668) | 2.059621 / 6.876477 (-4.816856) | 2.293764 / 2.142072 (0.151691) | 0.636471 / 4.805227 (-4.168756) | 0.150112 / 6.500664 (-6.350552) | 0.063705 / 0.075469 (-0.011764) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.358099 / 1.841788 (-0.483689) | 20.193750 / 8.074308 (12.119442) | 14.297350 / 10.191392 (4.105958) | 0.164477 / 0.680424 (-0.515947) | 0.018259 / 0.534201 (-0.515942) | 0.399010 / 0.579283 (-0.180273) | 0.417306 / 0.434364 (-0.017058) | 0.456961 / 0.540337 (-0.083377) | 0.631068 / 1.386936 (-0.755868) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007324 / 0.011353 (-0.004028) | 0.004463 / 0.011008 (-0.006545) | 0.066148 / 0.038508 (0.027640) | 0.093909 / 0.023109 (0.070799) | 0.399122 / 0.275898 (0.123224) | 0.430226 / 0.323480 (0.106746) | 0.005505 / 0.007986 (-0.002481) | 0.003579 / 0.004328 (-0.000749) | 0.066529 / 0.004250 (0.062278) | 0.063471 / 0.037052 (0.026418) | 0.406351 / 0.258489 (0.147862) | 0.439987 / 0.293841 (0.146146) | 0.032640 / 0.128546 (-0.095906) | 0.008770 / 0.075646 (-0.066877) | 0.072592 / 0.419271 (-0.346680) | 0.050429 / 0.043533 (0.006896) | 0.390873 / 0.255139 (0.135734) | 0.412438 / 0.283200 (0.129239) | 0.027113 / 0.141683 (-0.114570) | 1.458281 / 1.452155 (0.006126) | 1.536819 / 1.492716 (0.044103) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.228309 / 0.018006 (0.210303) | 0.454042 / 0.000490 (0.453552) | 0.000387 / 0.000200 (0.000187) | 0.000055 / 0.000054 (0.000001) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029573 / 0.037411 (-0.007838) | 0.086433 / 0.014526 (0.071907) | 0.097992 / 0.176557 (-0.078565) | 0.152464 / 0.737135 (-0.584671) | 0.099901 / 0.296338 (-0.196437) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.413807 / 0.215209 (0.198598) | 4.126395 / 2.077655 (2.048740) | 2.113544 / 1.504120 (0.609424) | 1.967829 / 1.541195 (0.426635) | 2.037123 / 1.468490 (0.568633) | 0.489403 / 4.584777 (-4.095374) | 3.689508 / 3.745712 (-0.056204) | 3.503909 / 5.269862 (-1.765952) | 2.113812 / 4.565676 (-2.451864) | 0.057988 / 0.424275 (-0.366287) | 0.007336 / 0.007607 (-0.000271) | 0.490840 / 0.226044 (0.264795) | 4.885040 / 2.268929 (2.616112) | 2.627864 / 55.444624 (-52.816760) | 2.231467 / 6.876477 (-4.645010) | 2.251307 / 2.142072 (0.109235) | 0.577370 / 4.805227 (-4.227857) | 0.131770 / 6.500664 (-6.368895) | 0.061313 / 0.075469 (-0.014156) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.362052 / 1.841788 (-0.479735) | 21.332694 / 8.074308 (13.258386) | 15.562019 / 10.191392 (5.370627) | 0.170874 / 0.680424 (-0.509550) | 0.019226 / 0.534201 (-0.514975) | 0.400311 / 0.579283 (-0.178972) | 0.423060 / 0.434364 (-0.011304) | 0.469946 / 0.540337 (-0.070391) | 0.647745 / 1.386936 (-0.739191) |

</details>
</details>

![](https://cml.dev/watermark.png#aec567c2f224f192e6e1f9799e3afc755eb517b2 "CML watermark")