Title: Bump fsspec from 2021.11.1 to 2022.3.0
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2023-07-28T09:37:15Z

Fix https://github.com/huggingface/datasets/issues/6087

(Colab installs 2023.6.0, so we should be good)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006640 / 0.011353 (-0.004713) | 0.004077 / 0.011008 (-0.006931) | 0.084905 / 0.038508 (0.046397) | 0.074004 / 0.023109 (0.050895) | 0.315968 / 0.275898 (0.040070) | 0.351594 / 0.323480 (0.028114) | 0.005623 / 0.007986 (-0.002362) | 0.003476 / 0.004328 (-0.000852) | 0.065089 / 0.004250 (0.060839) | 0.054683 / 0.037052 (0.017631) | 0.314983 / 0.258489 (0.056494) | 0.371776 / 0.293841 (0.077935) | 0.031727 / 0.128546 (-0.096819) | 0.008786 / 0.075646 (-0.066860) | 0.289905 / 0.419271 (-0.129367) | 0.053340 / 0.043533 (0.009807) | 0.311802 / 0.255139 (0.056663) | 0.351927 / 0.283200 (0.068727) | 0.024453 / 0.141683 (-0.117229) | 1.491727 / 1.452155 (0.039572) | 1.585027 / 1.492716 (0.092310) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.238757 / 0.018006 (0.220750) | 0.557691 / 0.000490 (0.557202) | 0.005158 / 0.000200 (0.004958) | 0.000204 / 0.000054 (0.000149) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.028435 / 0.037411 (-0.008977) | 0.082219 / 0.014526 (0.067693) | 0.096932 / 0.176557 (-0.079625) | 0.153802 / 0.737135 (-0.583333) | 0.098338 / 0.296338 (-0.198001) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.383448 / 0.215209 (0.168238) | 3.816074 / 2.077655 (1.738420) | 1.835111 / 1.504120 (0.330991) | 1.662326 / 1.541195 (0.121131) | 1.720202 / 1.468490 (0.251712) | 0.483107 / 4.584777 (-4.101669) | 3.648528 / 3.745712 (-0.097184) | 4.020929 / 5.269862 (-1.248932) | 2.433141 / 4.565676 (-2.132536) | 0.057081 / 0.424275 (-0.367194) | 0.007303 / 0.007607 (-0.000304) | 0.461366 / 0.226044 (0.235322) | 4.609090 / 2.268929 (2.340162) | 2.355940 / 55.444624 (-53.088684) | 1.989833 / 6.876477 (-4.886644) | 2.201451 / 2.142072 (0.059378) | 0.586156 / 4.805227 (-4.219071) | 0.133486 / 6.500664 (-6.367178) | 0.060062 / 0.075469 (-0.015407) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.247845 / 1.841788 (-0.593942) | 19.624252 / 8.074308 (11.549944) | 14.305975 / 10.191392 (4.114583) | 0.168687 / 0.680424 (-0.511737) | 0.018075 / 0.534201 (-0.516126) | 0.393859 / 0.579283 (-0.185424) | 0.407272 / 0.434364 (-0.027092) | 0.463760 / 0.540337 (-0.076578) | 0.629930 / 1.386936 (-0.757006) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006760 / 0.011353 (-0.004593) | 0.004345 / 0.011008 (-0.006663) | 0.064379 / 0.038508 (0.025871) | 0.078295 / 0.023109 (0.055186) | 0.364532 / 0.275898 (0.088633) | 0.395852 / 0.323480 (0.072372) | 0.005659 / 0.007986 (-0.002327) | 0.003515 / 0.004328 (-0.000813) | 0.065030 / 0.004250 (0.060780) | 0.059950 / 0.037052 (0.022898) | 0.375420 / 0.258489 (0.116931) | 0.411579 / 0.293841 (0.117738) | 0.031575 / 0.128546 (-0.096972) | 0.008737 / 0.075646 (-0.066910) | 0.070350 / 0.419271 (-0.348922) | 0.050607 / 0.043533 (0.007075) | 0.359785 / 0.255139 (0.104646) | 0.382638 / 0.283200 (0.099438) | 0.025533 / 0.141683 (-0.116150) | 1.564379 / 1.452155 (0.112225) | 1.620642 / 1.492716 (0.127925) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.212779 / 0.018006 (0.194773) | 0.563827 / 0.000490 (0.563337) | 0.003767 / 0.000200 (0.003567) | 0.000103 / 0.000054 (0.000049) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.030275 / 0.037411 (-0.007136) | 0.088108 / 0.014526 (0.073582) | 0.102454 / 0.176557 (-0.074103) | 0.156107 / 0.737135 (-0.581028) | 0.103961 / 0.296338 (-0.192378) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.421395 / 0.215209 (0.206186) | 4.204935 / 2.077655 (2.127280) | 2.144929 / 1.504120 (0.640809) | 1.999341 / 1.541195 (0.458147) | 2.066966 / 1.468490 (0.598476) | 0.486135 / 4.584777 (-4.098642) | 3.628139 / 3.745712 (-0.117573) | 5.652683 / 5.269862 (0.382821) | 3.216721 / 4.565676 (-1.348956) | 0.057513 / 0.424275 (-0.366762) | 0.007553 / 0.007607 (-0.000055) | 0.494470 / 0.226044 (0.268426) | 4.949343 / 2.268929 (2.680414) | 2.654222 / 55.444624 (-52.790402) | 2.322257 / 6.876477 (-4.554220) | 2.555633 / 2.142072 (0.413561) | 0.588355 / 4.805227 (-4.216872) | 0.134481 / 6.500664 (-6.366183) | 0.062415 / 0.075469 (-0.013054) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.377578 / 1.841788 (-0.464209) | 19.805201 / 8.074308 (11.730893) | 14.128536 / 10.191392 (3.937144) | 0.164343 / 0.680424 (-0.516081) | 0.018553 / 0.534201 (-0.515648) | 0.398191 / 0.579283 (-0.181093) | 0.414268 / 0.434364 (-0.020096) | 0.462270 / 0.540337 (-0.078068) | 0.608497 / 1.386936 (-0.778439) |

</details>
</details>

![](https://cml.dev/watermark.png#3af05ba487f361fae90a4c80af72de5c4ed70162 "CML watermark")


<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006966 / 0.011353 (-0.004387) | 0.004339 / 0.011008 (-0.006669) | 0.086682 / 0.038508 (0.048174) | 0.086143 / 0.023109 (0.063034) | 0.316106 / 0.275898 (0.040208) | 0.351422 / 0.323480 (0.027942) | 0.005916 / 0.007986 (-0.002069) | 0.003630 / 0.004328 (-0.000698) | 0.066980 / 0.004250 (0.062730) | 0.060031 / 0.037052 (0.022979) | 0.317487 / 0.258489 (0.058998) | 0.356280 / 0.293841 (0.062439) | 0.031816 / 0.128546 (-0.096730) | 0.008797 / 0.075646 (-0.066849) | 0.289848 / 0.419271 (-0.129424) | 0.055431 / 0.043533 (0.011898) | 0.318881 / 0.255139 (0.063742) | 0.332315 / 0.283200 (0.049116) | 0.025946 / 0.141683 (-0.115737) | 1.472904 / 1.452155 (0.020749) | 1.577973 / 1.492716 (0.085257) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.239056 / 0.018006 (0.221050) | 0.565406 / 0.000490 (0.564917) | 0.003606 / 0.000200 (0.003406) | 0.000080 / 0.000054 (0.000025) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029771 / 0.037411 (-0.007640) | 0.085534 / 0.014526 (0.071008) | 0.107008 / 0.176557 (-0.069548) | 0.631583 / 0.737135 (-0.105552) | 0.104210 / 0.296338 (-0.192128) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.390675 / 0.215209 (0.175466) | 3.898746 / 2.077655 (1.821091) | 1.933048 / 1.504120 (0.428928) | 1.792162 / 1.541195 (0.250967) | 1.958045 / 1.468490 (0.489555) | 0.488632 / 4.584777 (-4.096144) | 3.696306 / 3.745712 (-0.049406) | 3.454600 / 5.269862 (-1.815262) | 2.176292 / 4.565676 (-2.389385) | 0.057617 / 0.424275 (-0.366658) | 0.007603 / 0.007607 (-0.000004) | 0.467843 / 0.226044 (0.241798) | 4.672928 / 2.268929 (2.404000) | 2.441096 / 55.444624 (-53.003529) | 2.133506 / 6.876477 (-4.742970) | 2.431167 / 2.142072 (0.289095) | 0.588567 / 4.805227 (-4.216661) | 0.136070 / 6.500664 (-6.364594) | 0.063395 / 0.075469 (-0.012074) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.255003 / 1.841788 (-0.586784) | 20.587656 / 8.074308 (12.513348) | 15.147817 / 10.191392 (4.956425) | 0.152039 / 0.680424 (-0.528384) | 0.018815 / 0.534201 (-0.515386) | 0.397458 / 0.579283 (-0.181825) | 0.431433 / 0.434364 (-0.002931) | 0.487890 / 0.540337 (-0.052448) | 0.675367 / 1.386936 (-0.711569) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007209 / 0.011353 (-0.004144) | 0.004372 / 0.011008 (-0.006636) | 0.066288 / 0.038508 (0.027780) | 0.091776 / 0.023109 (0.068667) | 0.390724 / 0.275898 (0.114826) | 0.434711 / 0.323480 (0.111231) | 0.005790 / 0.007986 (-0.002196) | 0.003562 / 0.004328 (-0.000767) | 0.066155 / 0.004250 (0.061904) | 0.062459 / 0.037052 (0.025406) | 0.406622 / 0.258489 (0.148133) | 0.433976 / 0.293841 (0.140135) | 0.032590 / 0.128546 (-0.095957) | 0.008856 / 0.075646 (-0.066790) | 0.072327 / 0.419271 (-0.346945) | 0.049958 / 0.043533 (0.006426) | 0.400164 / 0.255139 (0.145025) | 0.413339 / 0.283200 (0.130139) | 0.025283 / 0.141683 (-0.116399) | 1.487668 / 1.452155 (0.035514) | 1.537679 / 1.492716 (0.044962) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.257814 / 0.018006 (0.239808) | 0.571741 / 0.000490 (0.571251) | 0.000412 / 0.000200 (0.000212) | 0.000056 / 0.000054 (0.000002) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.033893 / 0.037411 (-0.003518) | 0.094533 / 0.014526 (0.080008) | 0.105876 / 0.176557 (-0.070680) | 0.158675 / 0.737135 (-0.578460) | 0.107790 / 0.296338 (-0.188548) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.425796 / 0.215209 (0.210587) | 4.229159 / 2.077655 (2.151505) | 2.239613 / 1.504120 (0.735493) | 2.073830 / 1.541195 (0.532635) | 2.185508 / 1.468490 (0.717018) | 0.483984 / 4.584777 (-4.100793) | 3.645575 / 3.745712 (-0.100137) | 3.454767 / 5.269862 (-1.815095) | 2.141387 / 4.565676 (-2.424290) | 0.057570 / 0.424275 (-0.366705) | 0.007901 / 0.007607 (0.000294) | 0.501160 / 0.226044 (0.275116) | 5.012283 / 2.268929 (2.743355) | 2.701267 / 55.444624 (-52.743357) | 2.465409 / 6.876477 (-4.411068) | 2.696812 / 2.142072 (0.554739) | 0.587160 / 4.805227 (-4.218067) | 0.134175 / 6.500664 (-6.366489) | 0.062028 / 0.075469 (-0.013441) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.345632 / 1.841788 (-0.496155) | 21.077279 / 8.074308 (13.002971) | 14.700826 / 10.191392 (4.509434) | 0.156191 / 0.680424 (-0.524233) | 0.018991 / 0.534201 (-0.515210) | 0.400413 / 0.579283 (-0.178870) | 0.420597 / 0.434364 (-0.013767) | 0.486534 / 0.540337 (-0.053804) | 0.646606 / 1.386936 (-0.740330) |

</details>
</details>

![](https://cml.dev/watermark.png#5bb8fabb135ca8adf47151ad3de050e3a258ccab "CML watermark")