Unpin fsspec
State: closed | Author: albertvillanova (MEMBER)

In `fsspec--2023.4.0` default value for clobber when registering an implementation was changed from True to False. See:
- https://github.com/fsspec/filesystem_spec/pull/1237

This PR recovers previous behavior by passing clobber True when registering mock implementations.

This PR also removes the temporary pin introduced by:
- #5731

Fix #5734.

Comments: _The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006240 / 0.011353 (-0.005113) | 0.004392 / 0.011008 (-0.006616) | 0.097276 / 0.038508 (0.058768) | 0.027262 / 0.023109 (0.004153) | 0.303203 / 0.275898 (0.027305) | 0.331878 / 0.323480 (0.008398) | 0.004706 / 0.007986 (-0.003279) | 0.004428 / 0.004328 (0.000100) | 0.074666 / 0.004250 (0.070416) | 0.036154 / 0.037052 (-0.000899) | 0.302997 / 0.258489 (0.044508) | 0.340350 / 0.293841 (0.046509) | 0.031011 / 0.128546 (-0.097535) | 0.011616 / 0.075646 (-0.064031) | 0.323671 / 0.419271 (-0.095601) | 0.042062 / 0.043533 (-0.001471) | 0.311381 / 0.255139 (0.056242) | 0.324697 / 0.283200 (0.041498) | 0.084248 / 0.141683 (-0.057435) | 1.471651 / 1.452155 (0.019496) | 1.533414 / 1.492716 (0.040697) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.193555 / 0.018006 (0.175549) | 0.393452 / 0.000490 (0.392962) | 0.002348 / 0.000200 (0.002148) | 0.000071 / 0.000054 (0.000016) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.022523 / 0.037411 (-0.014889) | 0.096552 / 0.014526 (0.082026) | 0.101746 / 0.176557 (-0.074810) | 0.163145 / 0.737135 (-0.573990) | 0.106417 / 0.296338 (-0.189921) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.448589 / 0.215209 (0.233380) | 4.467803 / 2.077655 (2.390148) | 2.178745 / 1.504120 (0.674625) | 1.983339 / 1.541195 (0.442145) | 2.056554 / 1.468490 (0.588064) | 0.697571 / 4.584777 (-3.887206) | 3.363967 / 3.745712 (-0.381745) | 1.872526 / 5.269862 (-3.397336) | 1.258245 / 4.565676 (-3.307432) | 0.082954 / 0.424275 (-0.341321) | 0.012306 / 0.007607 (0.004699) | 0.545096 / 0.226044 (0.319052) | 5.468706 / 2.268929 (3.199777) | 2.645333 / 55.444624 (-52.799292) | 2.287659 / 6.876477 (-4.588818) | 2.346768 / 2.142072 (0.204696) | 0.803730 / 4.805227 (-4.001497) | 0.151037 / 6.500664 (-6.349627) | 0.066404 / 0.075469 (-0.009065) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.192982 / 1.841788 (-0.648806) | 13.631225 / 8.074308 (5.556917) | 13.830053 / 10.191392 (3.638661) | 0.141901 / 0.680424 (-0.538523) | 0.016500 / 0.534201 (-0.517701) | 0.373268 / 0.579283 (-0.206015) | 0.380123 / 0.434364 (-0.054241) | 0.430786 / 0.540337 (-0.109551) | 0.512669 / 1.386936 (-0.874267) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006161 / 0.011353 (-0.005192) | 0.004399 / 0.011008 (-0.006609) | 0.076210 / 0.038508 (0.037702) | 0.026791 / 0.023109 (0.003681) | 0.341523 / 0.275898 (0.065625) | 0.370400 / 0.323480 (0.046920) | 0.004495 / 0.007986 (-0.003491) | 0.003204 / 0.004328 (-0.001125) | 0.075444 / 0.004250 (0.071194) | 0.035914 / 0.037052 (-0.001138) | 0.343806 / 0.258489 (0.085317) | 0.384320 / 0.293841 (0.090479) | 0.031438 / 0.128546 (-0.097109) | 0.011253 / 0.075646 (-0.064393) | 0.085364 / 0.419271 (-0.333908) | 0.041407 / 0.043533 (-0.002126) | 0.338831 / 0.255139 (0.083692) | 0.364357 / 0.283200 (0.081158) | 0.087417 / 0.141683 (-0.054266) | 1.520624 / 1.452155 (0.068470) | 1.572432 / 1.492716 (0.079716) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.232403 / 0.018006 (0.214396) | 0.388187 / 0.000490 (0.387698) | 0.001158 / 0.000200 (0.000958) | 0.000069 / 0.000054 (0.000014) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.024596 / 0.037411 (-0.012816) | 0.101203 / 0.014526 (0.086677) | 0.105243 / 0.176557 (-0.071314) | 0.158215 / 0.737135 (-0.578920) | 0.110277 / 0.296338 (-0.186061) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.435661 / 0.215209 (0.220452) | 4.350151 / 2.077655 (2.272496) | 2.072372 / 1.504120 (0.568252) | 1.870675 / 1.541195 (0.329480) | 1.910883 / 1.468490 (0.442393) | 0.697384 / 4.584777 (-3.887393) | 3.399377 / 3.745712 (-0.346335) | 2.685008 / 5.269862 (-2.584854) | 1.476843 / 4.565676 (-3.088834) | 0.083177 / 0.424275 (-0.341098) | 0.012413 / 0.007607 (0.004806) | 0.542543 / 0.226044 (0.316498) | 5.431422 / 2.268929 (3.162494) | 2.506419 / 55.444624 (-52.938206) | 2.166342 / 6.876477 (-4.710135) | 2.164421 / 2.142072 (0.022348) | 0.800609 / 4.805227 (-4.004618) | 0.150527 / 6.500664 (-6.350137) | 0.065780 / 0.075469 (-0.009689) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.293409 / 1.841788 (-0.548379) | 13.814898 / 8.074308 (5.740590) | 13.940416 / 10.191392 (3.749024) | 0.149377 / 0.680424 (-0.531047) | 0.016462 / 0.534201 (-0.517739) | 0.393748 / 0.579283 (-0.185535) | 0.384327 / 0.434364 (-0.050037) | 0.489900 / 0.540337 (-0.050437) | 0.574608 / 1.386936 (-0.812328) |

</details>
</details>

![](https://cml.dev/watermark.png#f2607935c4e45c70c44fcb698db0363ca7ba83d4 "CML watermark")