Mark tests that require librosa
State: closed | Author: albertvillanova (MEMBER)

Mark tests that require `librosa`.

Note that `librosa` is an optional dependency (installed with `audio` option) and we should be able to test environments without that library installed. This is the case if we want to test Numpy 2.0, which is currently incompatible with `librosa` due to its dependency on `soxr`:
- https://github.com/dofuuz/python-soxr/issues/28

Comments: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7044). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005797 / 0.011353 (-0.005556) | 0.004017 / 0.011008 (-0.006991) | 0.063829 / 0.038508 (0.025321) | 0.031329 / 0.023109 (0.008220) | 0.249388 / 0.275898 (-0.026510) | 0.273129 / 0.323480 (-0.050351) | 0.004250 / 0.007986 (-0.003736) | 0.002821 / 0.004328 (-0.001507) | 0.049250 / 0.004250 (0.044999) | 0.046175 / 0.037052 (0.009123) | 0.252040 / 0.258489 (-0.006449) | 0.296537 / 0.293841 (0.002696) | 0.030579 / 0.128546 (-0.097967) | 0.012436 / 0.075646 (-0.063210) | 0.205829 / 0.419271 (-0.213443) | 0.036979 / 0.043533 (-0.006554) | 0.251354 / 0.255139 (-0.003785) | 0.272262 / 0.283200 (-0.010938) | 0.019047 / 0.141683 (-0.122636) | 1.112410 / 1.452155 (-0.339745) | 1.137445 / 1.492716 (-0.355271) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.097270 / 0.018006 (0.079264) | 0.309329 / 0.000490 (0.308839) | 0.000221 / 0.000200 (0.000021) | 0.000053 / 0.000054 (-0.000001) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.019021 / 0.037411 (-0.018390) | 0.066801 / 0.014526 (0.052276) | 0.075280 / 0.176557 (-0.101276) | 0.122499 / 0.737135 (-0.614637) | 0.077424 / 0.296338 (-0.218914) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.279469 / 0.215209 (0.064259) | 2.787511 / 2.077655 (0.709856) | 1.411389 / 1.504120 (-0.092731) | 1.285796 / 1.541195 (-0.255399) | 1.354252 / 1.468490 (-0.114238) | 0.735341 / 4.584777 (-3.849436) | 2.418557 / 3.745712 (-1.327155) | 2.983406 / 5.269862 (-2.286455) | 2.005853 / 4.565676 (-2.559823) | 0.080440 / 0.424275 (-0.343835) | 0.005242 / 0.007607 (-0.002365) | 0.343557 / 0.226044 (0.117513) | 3.358984 / 2.268929 (1.090055) | 1.816709 / 55.444624 (-53.627915) | 1.500225 / 6.876477 (-5.376252) | 1.715405 / 2.142072 (-0.426667) | 0.829054 / 4.805227 (-3.976174) | 0.138352 / 6.500664 (-6.362312) | 0.043709 / 0.075469 (-0.031760) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.969135 / 1.841788 (-0.872652) | 12.510750 / 8.074308 (4.436442) | 10.140368 / 10.191392 (-0.051024) | 0.133117 / 0.680424 (-0.547307) | 0.015775 / 0.534201 (-0.518426) | 0.302203 / 0.579283 (-0.277080) | 0.268214 / 0.434364 (-0.166150) | 0.347041 / 0.540337 (-0.193296) | 0.456095 / 1.386936 (-0.930841) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006255 / 0.011353 (-0.005098) | 0.004453 / 0.011008 (-0.006555) | 0.052298 / 0.038508 (0.013790) | 0.034808 / 0.023109 (0.011699) | 0.274723 / 0.275898 (-0.001175) | 0.297199 / 0.323480 (-0.026281) | 0.004499 / 0.007986 (-0.003486) | 0.003086 / 0.004328 (-0.001242) | 0.051315 / 0.004250 (0.047065) | 0.042764 / 0.037052 (0.005712) | 0.285636 / 0.258489 (0.027147) | 0.321819 / 0.293841 (0.027978) | 0.033350 / 0.128546 (-0.095196) | 0.013457 / 0.075646 (-0.062189) | 0.063930 / 0.419271 (-0.355342) | 0.034537 / 0.043533 (-0.008996) | 0.272630 / 0.255139 (0.017491) | 0.289245 / 0.283200 (0.006045) | 0.018910 / 0.141683 (-0.122773) | 1.153064 / 1.452155 (-0.299091) | 1.207065 / 1.492716 (-0.285651) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.093008 / 0.018006 (0.075002) | 0.301313 / 0.000490 (0.300823) | 0.000214 / 0.000200 (0.000014) | 0.000054 / 0.000054 (-0.000000) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.023168 / 0.037411 (-0.014244) | 0.080837 / 0.014526 (0.066312) | 0.089667 / 0.176557 (-0.086889) | 0.135849 / 0.737135 (-0.601286) | 0.092082 / 0.296338 (-0.204257) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.298933 / 0.215209 (0.083723) | 2.847736 / 2.077655 (0.770082) | 1.550268 / 1.504120 (0.046148) | 1.425675 / 1.541195 (-0.115520) | 1.469251 / 1.468490 (0.000761) | 0.720446 / 4.584777 (-3.864331) | 0.976149 / 3.745712 (-2.769563) | 3.081804 / 5.269862 (-2.188057) | 1.982797 / 4.565676 (-2.582880) | 0.078598 / 0.424275 (-0.345677) | 0.005229 / 0.007607 (-0.002379) | 0.345475 / 0.226044 (0.119430) | 3.421312 / 2.268929 (1.152384) | 1.929034 / 55.444624 (-53.515590) | 1.631523 / 6.876477 (-5.244953) | 1.671996 / 2.142072 (-0.470077) | 0.776916 / 4.805227 (-4.028311) | 0.133966 / 6.500664 (-6.366699) | 0.042183 / 0.075469 (-0.033286) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.993023 / 1.841788 (-0.848764) | 12.981642 / 8.074308 (4.907334) | 10.610457 / 10.191392 (0.419065) | 0.146748 / 0.680424 (-0.533676) | 0.016556 / 0.534201 (-0.517645) | 0.303613 / 0.579283 (-0.275670) | 0.132671 / 0.434364 (-0.301693) | 0.344786 / 0.540337 (-0.195552) | 0.443049 / 1.386936 (-0.943887) |

</details>
</details>

![](https://cml.dev/watermark.png#8419c40a085d67eb5832cecebf3ef8213112857d "CML watermark")