Make prepare_split more robust if errors in metadata dataset_info splits
Opened by albertvillanova (closed)

This PR uses `split_generator.split_info` as default value for `split_info` if any exception is raised while trying to get `split_generator.name` from `self.info.splits` (this may happen if there is any error in the metadata dataset_info splits).

Please note that `split_info` is only used by the logger.

Fix #5895 if passed `verification_mode="no_checks"`:
```python
ds = load_dataset(
    "ArmelR/stack-exchange-instruction", 
    data_dir="data/finetune", 
    split="train", 
    verification_mode="no_checks", 
    revision="c609f1caade5cfbf3b9fe9cfa17d7cb000b457bd",
)
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008809 / 0.011353 (-0.002544) | 0.005641 / 0.011008 (-0.005367) | 0.124986 / 0.038508 (0.086477) | 0.037311 / 0.023109 (0.014202) | 0.388915 / 0.275898 (0.113017) | 0.430123 / 0.323480 (0.106643) | 0.007447 / 0.007986 (-0.000538) | 0.009593 / 0.004328 (0.005264) | 0.099148 / 0.004250 (0.094898) | 0.052393 / 0.037052 (0.015341) | 0.399779 / 0.258489 (0.141290) | 0.439109 / 0.293841 (0.145268) | 0.043409 / 0.128546 (-0.085137) | 0.016286 / 0.075646 (-0.059360) | 0.431198 / 0.419271 (0.011927) | 0.064932 / 0.043533 (0.021400) | 0.390650 / 0.255139 (0.135511) | 0.432883 / 0.283200 (0.149684) | 0.110978 / 0.141683 (-0.030705) | 1.796121 / 1.452155 (0.343967) | 1.960097 / 1.492716 (0.467381) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.286292 / 0.018006 (0.268286) | 0.659495 / 0.000490 (0.659005) | 0.008294 / 0.000200 (0.008094) | 0.000485 / 0.000054 (0.000431) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029325 / 0.037411 (-0.008086) | 0.125454 / 0.014526 (0.110928) | 0.136459 / 0.176557 (-0.040097) | 0.221075 / 0.737135 (-0.516060) | 0.140281 / 0.296338 (-0.156058) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.602401 / 0.215209 (0.387192) | 6.124553 / 2.077655 (4.046898) | 2.453141 / 1.504120 (0.949021) | 2.038611 / 1.541195 (0.497416) | 2.073611 / 1.468490 (0.605121) | 0.938040 / 4.584777 (-3.646737) | 5.755972 / 3.745712 (2.010260) | 4.450935 / 5.269862 (-0.818926) | 2.337219 / 4.565676 (-2.228457) | 0.107118 / 0.424275 (-0.317157) | 0.015201 / 0.007607 (0.007594) | 0.785833 / 0.226044 (0.559788) | 7.732984 / 2.268929 (5.464055) | 3.236892 / 55.444624 (-52.207733) | 2.696402 / 6.876477 (-4.180074) | 2.805036 / 2.142072 (0.662964) | 1.108612 / 4.805227 (-3.696616) | 0.221067 / 6.500664 (-6.279597) | 0.085538 / 0.075469 (0.010068) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.600311 / 1.841788 (-0.241476) | 18.528118 / 8.074308 (10.453810) | 21.107199 / 10.191392 (10.915807) | 0.219489 / 0.680424 (-0.460934) | 0.028927 / 0.534201 (-0.505274) | 0.503446 / 0.579283 (-0.075837) | 0.619833 / 0.434364 (0.185469) | 0.582454 / 0.540337 (0.042117) | 0.709154 / 1.386936 (-0.677782) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008516 / 0.011353 (-0.002837) | 0.006090 / 0.011008 (-0.004918) | 0.104574 / 0.038508 (0.066066) | 0.042676 / 0.023109 (0.019566) | 0.458623 / 0.275898 (0.182725) | 0.568479 / 0.323480 (0.244999) | 0.008374 / 0.007986 (0.000389) | 0.004677 / 0.004328 (0.000349) | 0.105946 / 0.004250 (0.101695) | 0.055256 / 0.037052 (0.018204) | 0.511036 / 0.258489 (0.252547) | 0.598383 / 0.293841 (0.304542) | 0.043612 / 0.128546 (-0.084934) | 0.014707 / 0.075646 (-0.060940) | 0.116350 / 0.419271 (-0.302921) | 0.061413 / 0.043533 (0.017880) | 0.477785 / 0.255139 (0.222646) | 0.542643 / 0.283200 (0.259443) | 0.120431 / 0.141683 (-0.021252) | 1.994083 / 1.452155 (0.541928) | 2.100600 / 1.492716 (0.607883) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.298480 / 0.018006 (0.280474) | 0.601921 / 0.000490 (0.601432) | 0.000445 / 0.000200 (0.000245) | 0.000086 / 0.000054 (0.000032) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034784 / 0.037411 (-0.002627) | 0.133555 / 0.014526 (0.119029) | 0.138541 / 0.176557 (-0.038015) | 0.203114 / 0.737135 (-0.534021) | 0.153477 / 0.296338 (-0.142861) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.780484 / 0.215209 (0.565275) | 7.150876 / 2.077655 (5.073222) | 3.168590 / 1.504120 (1.664470) | 2.698746 / 1.541195 (1.157552) | 2.695678 / 1.468490 (1.227188) | 1.037706 / 4.584777 (-3.547071) | 5.672631 / 3.745712 (1.926918) | 2.798137 / 5.269862 (-2.471725) | 1.738588 / 4.565676 (-2.827088) | 0.111160 / 0.424275 (-0.313115) | 0.013878 / 0.007607 (0.006271) | 0.800191 / 0.226044 (0.574146) | 8.546676 / 2.268929 (6.277748) | 4.116852 / 55.444624 (-51.327773) | 3.331271 / 6.876477 (-3.545206) | 3.307410 / 2.142072 (1.165337) | 1.191019 / 4.805227 (-3.614208) | 0.248953 / 6.500664 (-6.251711) | 0.086632 / 0.075469 (0.011162) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.795057 / 1.841788 (-0.046730) | 18.038785 / 8.074308 (9.964476) | 21.865566 / 10.191392 (11.674174) | 0.211058 / 0.680424 (-0.469366) | 0.026956 / 0.534201 (-0.507245) | 0.518855 / 0.579283 (-0.060428) | 0.618105 / 0.434364 (0.183741) | 0.569227 / 0.540337 (0.028889) | 0.705431 / 1.386936 (-0.681505) |

</details>
</details>

![](https://cml.dev/watermark.png#074925b9b7c1dfd33b8675aa99c07cc26375665c "CML watermark")


<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008900 / 0.011353 (-0.002453) | 0.005726 / 0.011008 (-0.005283) | 0.131747 / 0.038508 (0.093239) | 0.040585 / 0.023109 (0.017476) | 0.420531 / 0.275898 (0.144633) | 0.459430 / 0.323480 (0.135950) | 0.007642 / 0.007986 (-0.000344) | 0.006750 / 0.004328 (0.002421) | 0.099147 / 0.004250 (0.094897) | 0.055852 / 0.037052 (0.018799) | 0.423653 / 0.258489 (0.165164) | 0.453304 / 0.293841 (0.159463) | 0.045247 / 0.128546 (-0.083300) | 0.016034 / 0.075646 (-0.059612) | 0.443115 / 0.419271 (0.023843) | 0.078853 / 0.043533 (0.035320) | 0.417508 / 0.255139 (0.162369) | 0.440936 / 0.283200 (0.157736) | 0.115603 / 0.141683 (-0.026080) | 1.844610 / 1.452155 (0.392456) | 1.998497 / 1.492716 (0.505781) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.272622 / 0.018006 (0.254616) | 0.598045 / 0.000490 (0.597556) | 0.007088 / 0.000200 (0.006888) | 0.000159 / 0.000054 (0.000105) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.032976 / 0.037411 (-0.004436) | 0.143970 / 0.014526 (0.129444) | 0.142172 / 0.176557 (-0.034384) | 0.216747 / 0.737135 (-0.520389) | 0.146004 / 0.296338 (-0.150334) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.687507 / 0.215209 (0.472298) | 6.549524 / 2.077655 (4.471870) | 2.924142 / 1.504120 (1.420022) | 2.504471 / 1.541195 (0.963277) | 2.496280 / 1.468490 (1.027790) | 0.959054 / 4.584777 (-3.625723) | 5.851742 / 3.745712 (2.106030) | 4.983357 / 5.269862 (-0.286504) | 2.627403 / 4.565676 (-1.938274) | 0.112955 / 0.424275 (-0.311320) | 0.016206 / 0.007607 (0.008599) | 0.819158 / 0.226044 (0.593114) | 8.416949 / 2.268929 (6.148020) | 3.776765 / 55.444624 (-51.667859) | 3.002397 / 6.876477 (-3.874080) | 3.158852 / 2.142072 (1.016779) | 1.197099 / 4.805227 (-3.608129) | 0.280654 / 6.500664 (-6.220010) | 0.099471 / 0.075469 (0.024002) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.687007 / 1.841788 (-0.154781) | 19.411976 / 8.074308 (11.337668) | 22.053482 / 10.191392 (11.862090) | 0.228038 / 0.680424 (-0.452386) | 0.028226 / 0.534201 (-0.505975) | 0.527695 / 0.579283 (-0.051588) | 0.635911 / 0.434364 (0.201547) | 0.618205 / 0.540337 (0.077868) | 0.735164 / 1.386936 (-0.651772) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009450 / 0.011353 (-0.001903) | 0.006566 / 0.011008 (-0.004442) | 0.108919 / 0.038508 (0.070411) | 0.050010 / 0.023109 (0.026900) | 0.505168 / 0.275898 (0.229270) | 0.552190 / 0.323480 (0.228710) | 0.007569 / 0.007986 (-0.000417) | 0.006807 / 0.004328 (0.002478) | 0.116621 / 0.004250 (0.112371) | 0.060374 / 0.037052 (0.023321) | 0.515165 / 0.258489 (0.256676) | 0.572125 / 0.293841 (0.278284) | 0.046561 / 0.128546 (-0.081986) | 0.016159 / 0.075646 (-0.059487) | 0.114568 / 0.419271 (-0.304704) | 0.064689 / 0.043533 (0.021157) | 0.497870 / 0.255139 (0.242731) | 0.567332 / 0.283200 (0.284132) | 0.126254 / 0.141683 (-0.015429) | 1.954074 / 1.452155 (0.501919) | 2.057682 / 1.492716 (0.564966) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.013857 / 0.018006 (-0.004149) | 0.601561 / 0.000490 (0.601071) | 0.002897 / 0.000200 (0.002697) | 0.000108 / 0.000054 (0.000053) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.038480 / 0.037411 (0.001069) | 0.142480 / 0.014526 (0.127954) | 0.160479 / 0.176557 (-0.016077) | 0.217942 / 0.737135 (-0.519194) | 0.159908 / 0.296338 (-0.136431) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.697926 / 0.215209 (0.482717) | 6.869754 / 2.077655 (4.792100) | 3.125463 / 1.504120 (1.621343) | 2.729123 / 1.541195 (1.187928) | 2.855747 / 1.468490 (1.387257) | 1.015345 / 4.584777 (-3.569432) | 5.839176 / 3.745712 (2.093463) | 5.019678 / 5.269862 (-0.250184) | 2.080489 / 4.565676 (-2.485187) | 0.118884 / 0.424275 (-0.305391) | 0.021381 / 0.007607 (0.013774) | 0.877847 / 0.226044 (0.651803) | 8.714561 / 2.268929 (6.445633) | 3.933399 / 55.444624 (-51.511226) | 3.281809 / 6.876477 (-3.594668) | 3.330342 / 2.142072 (1.188269) | 1.235005 / 4.805227 (-3.570222) | 0.239686 / 6.500664 (-6.260978) | 0.093546 / 0.075469 (0.018077) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.787916 / 1.841788 (-0.053872) | 20.094828 / 8.074308 (12.020520) | 22.902101 / 10.191392 (12.710709) | 0.249315 / 0.680424 (-0.431109) | 0.028058 / 0.534201 (-0.506143) | 0.524960 / 0.579283 (-0.054323) | 0.643881 / 0.434364 (0.209517) | 0.621203 / 0.540337 (0.080866) | 0.723337 / 1.386936 (-0.663599) |

</details>
</details>

![](https://cml.dev/watermark.png#074925b9b7c1dfd33b8675aa99c07cc26375665c "CML watermark")