Title: Delete `task_templates` in `IterableDataset` when they are no longer valid
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2023-07-13T13:16:17Z

Fix #6025

Comments:
_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008698 / 0.011353 (-0.002655) | 0.005250 / 0.011008 (-0.005758) | 0.104101 / 0.038508 (0.065593) | 0.085021 / 0.023109 (0.061912) | 0.426653 / 0.275898 (0.150755) | 0.460449 / 0.323480 (0.136969) | 0.005222 / 0.007986 (-0.002763) | 0.006280 / 0.004328 (0.001951) | 0.083458 / 0.004250 (0.079207) | 0.066132 / 0.037052 (0.029079) | 0.433416 / 0.258489 (0.174927) | 0.482718 / 0.293841 (0.188877) | 0.048872 / 0.128546 (-0.079675) | 0.013699 / 0.075646 (-0.061948) | 0.365660 / 0.419271 (-0.053611) | 0.071008 / 0.043533 (0.027475) | 0.428688 / 0.255139 (0.173549) | 0.443554 / 0.283200 (0.160354) | 0.035901 / 0.141683 (-0.105782) | 1.829296 / 1.452155 (0.377141) | 1.862351 / 1.492716 (0.369635) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.236284 / 0.018006 (0.218278) | 0.584075 / 0.000490 (0.583585) | 0.004634 / 0.000200 (0.004434) | 0.000125 / 0.000054 (0.000070) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034723 / 0.037411 (-0.002688) | 0.100989 / 0.014526 (0.086464) | 0.113722 / 0.176557 (-0.062834) | 0.187659 / 0.737135 (-0.549477) | 0.113937 / 0.296338 (-0.182401) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.587500 / 0.215209 (0.372291) | 5.847371 / 2.077655 (3.769716) | 2.599691 / 1.504120 (1.095571) | 2.246187 / 1.541195 (0.704992) | 2.419126 / 1.468490 (0.950636) | 0.847327 / 4.584777 (-3.737450) | 5.230438 / 3.745712 (1.484726) | 7.539021 / 5.269862 (2.269160) | 4.617473 / 4.565676 (0.051797) | 0.103620 / 0.424275 (-0.320655) | 0.009195 / 0.007607 (0.001588) | 0.714247 / 0.226044 (0.488203) | 7.331621 / 2.268929 (5.062693) | 3.416575 / 55.444624 (-52.028049) | 2.649467 / 6.876477 (-4.227009) | 2.928091 / 2.142072 (0.786018) | 1.002155 / 4.805227 (-3.803072) | 0.210790 / 6.500664 (-6.289874) | 0.081303 / 0.075469 (0.005834) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.655431 / 1.841788 (-0.186357) | 24.069595 / 8.074308 (15.995287) | 20.923766 / 10.191392 (10.732374) | 0.232021 / 0.680424 (-0.448403) | 0.026355 / 0.534201 (-0.507846) | 0.496830 / 0.579283 (-0.082453) | 0.582620 / 0.434364 (0.148257) | 0.551227 / 0.540337 (0.010890) | 0.756389 / 1.386936 (-0.630547) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009329 / 0.011353 (-0.002024) | 0.005045 / 0.011008 (-0.005964) | 0.082116 / 0.038508 (0.043608) | 0.082420 / 0.023109 (0.059311) | 0.502513 / 0.275898 (0.226615) | 0.526098 / 0.323480 (0.202618) | 0.007468 / 0.007986 (-0.000517) | 0.005477 / 0.004328 (0.001148) | 0.082617 / 0.004250 (0.078367) | 0.070292 / 0.037052 (0.033239) | 0.503290 / 0.258489 (0.244801) | 0.541631 / 0.293841 (0.247790) | 0.050826 / 0.128546 (-0.077721) | 0.014699 / 0.075646 (-0.060948) | 0.094441 / 0.419271 (-0.324830) | 0.065034 / 0.043533 (0.021501) | 0.486778 / 0.255139 (0.231639) | 0.516907 / 0.283200 (0.233707) | 0.045140 / 0.141683 (-0.096543) | 1.831676 / 1.452155 (0.379521) | 1.910865 / 1.492716 (0.418149) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.286818 / 0.018006 (0.268812) | 0.558621 / 0.000490 (0.558131) | 0.002830 / 0.000200 (0.002630) | 0.000148 / 0.000054 (0.000094) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.036716 / 0.037411 (-0.000696) | 0.107830 / 0.014526 (0.093305) | 0.116368 / 0.176557 (-0.060188) | 0.178401 / 0.737135 (-0.558734) | 0.124729 / 0.296338 (-0.171609) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.633557 / 0.215209 (0.418348) | 6.423135 / 2.077655 (4.345480) | 2.981883 / 1.504120 (1.477763) | 2.755592 / 1.541195 (1.214398) | 2.769337 / 1.468490 (1.300847) | 0.836219 / 4.584777 (-3.748558) | 5.302030 / 3.745712 (1.556318) | 7.463960 / 5.269862 (2.194098) | 4.427254 / 4.565676 (-0.138422) | 0.095990 / 0.424275 (-0.328285) | 0.009264 / 0.007607 (0.001657) | 0.770642 / 0.226044 (0.544597) | 7.779667 / 2.268929 (5.510739) | 3.799115 / 55.444624 (-51.645509) | 3.212560 / 6.876477 (-3.663917) | 3.281657 / 2.142072 (1.139584) | 1.044981 / 4.805227 (-3.760246) | 0.210693 / 6.500664 (-6.289971) | 0.079466 / 0.075469 (0.003997) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.793155 / 1.841788 (-0.048632) | 24.691127 / 8.074308 (16.616819) | 22.083150 / 10.191392 (11.891758) | 0.242246 / 0.680424 (-0.438178) | 0.028001 / 0.534201 (-0.506200) | 0.494061 / 0.579283 (-0.085222) | 0.599288 / 0.434364 (0.164924) | 0.552101 / 0.540337 (0.011764) | 0.784093 / 1.386936 (-0.602843) |

</details>
</details>

![](https://cml.dev/watermark.png#cd429c39604af34bc3a3ba1f463329b23fcbc1e3 "CML watermark")


<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006658 / 0.011353 (-0.004695) | 0.004044 / 0.011008 (-0.006965) | 0.085844 / 0.038508 (0.047336) | 0.077147 / 0.023109 (0.054038) | 0.344387 / 0.275898 (0.068489) | 0.376718 / 0.323480 (0.053238) | 0.005537 / 0.007986 (-0.002448) | 0.003452 / 0.004328 (-0.000876) | 0.065326 / 0.004250 (0.061076) | 0.057639 / 0.037052 (0.020587) | 0.352363 / 0.258489 (0.093873) | 0.378939 / 0.293841 (0.085098) | 0.031259 / 0.128546 (-0.097287) | 0.008464 / 0.075646 (-0.067183) | 0.289076 / 0.419271 (-0.130195) | 0.052991 / 0.043533 (0.009459) | 0.346053 / 0.255139 (0.090914) | 0.362761 / 0.283200 (0.079561) | 0.023501 / 0.141683 (-0.118182) | 1.478312 / 1.452155 (0.026157) | 1.545437 / 1.492716 (0.052721) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.202964 / 0.018006 (0.184957) | 0.534793 / 0.000490 (0.534303) | 0.006025 / 0.000200 (0.005825) | 0.000225 / 0.000054 (0.000171) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029418 / 0.037411 (-0.007993) | 0.084297 / 0.014526 (0.069771) | 0.096702 / 0.176557 (-0.079855) | 0.157355 / 0.737135 (-0.579781) | 0.097858 / 0.296338 (-0.198480) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.380728 / 0.215209 (0.165519) | 3.787712 / 2.077655 (1.710057) | 1.836393 / 1.504120 (0.332273) | 1.678415 / 1.541195 (0.137220) | 1.781800 / 1.468490 (0.313310) | 0.478677 / 4.584777 (-4.106100) | 3.614080 / 3.745712 (-0.131632) | 3.255637 / 5.269862 (-2.014225) | 2.063642 / 4.565676 (-2.502035) | 0.056470 / 0.424275 (-0.367805) | 0.007408 / 0.007607 (-0.000199) | 0.459155 / 0.226044 (0.233111) | 4.586679 / 2.268929 (2.317750) | 2.305737 / 55.444624 (-53.138888) | 1.954755 / 6.876477 (-4.921721) | 2.190809 / 2.142072 (0.048737) | 0.572426 / 4.805227 (-4.232802) | 0.130349 / 6.500664 (-6.370315) | 0.059346 / 0.075469 (-0.016124) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.253671 / 1.841788 (-0.588117) | 19.509015 / 8.074308 (11.434706) | 13.951349 / 10.191392 (3.759957) | 0.171038 / 0.680424 (-0.509386) | 0.018826 / 0.534201 (-0.515375) | 0.394642 / 0.579283 (-0.184642) | 0.419614 / 0.434364 (-0.014750) | 0.470931 / 0.540337 (-0.069406) | 0.643858 / 1.386936 (-0.743078) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006765 / 0.011353 (-0.004587) | 0.003955 / 0.011008 (-0.007053) | 0.064377 / 0.038508 (0.025869) | 0.076980 / 0.023109 (0.053871) | 0.368675 / 0.275898 (0.092777) | 0.403746 / 0.323480 (0.080267) | 0.005303 / 0.007986 (-0.002683) | 0.003257 / 0.004328 (-0.001072) | 0.064154 / 0.004250 (0.059903) | 0.056975 / 0.037052 (0.019923) | 0.376718 / 0.258489 (0.118229) | 0.416291 / 0.293841 (0.122450) | 0.031444 / 0.128546 (-0.097102) | 0.008532 / 0.075646 (-0.067115) | 0.070455 / 0.419271 (-0.348816) | 0.049032 / 0.043533 (0.005499) | 0.361413 / 0.255139 (0.106274) | 0.384648 / 0.283200 (0.101448) | 0.024050 / 0.141683 (-0.117633) | 1.514330 / 1.452155 (0.062176) | 1.585424 / 1.492716 (0.092708) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.214701 / 0.018006 (0.196695) | 0.447706 / 0.000490 (0.447216) | 0.000373 / 0.000200 (0.000173) | 0.000058 / 0.000054 (0.000003) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.031007 / 0.037411 (-0.006404) | 0.090545 / 0.014526 (0.076019) | 0.100611 / 0.176557 (-0.075945) | 0.154847 / 0.737135 (-0.582289) | 0.102864 / 0.296338 (-0.193475) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.427740 / 0.215209 (0.212531) | 4.273143 / 2.077655 (2.195488) | 2.294906 / 1.504120 (0.790786) | 2.138460 / 1.541195 (0.597265) | 2.274126 / 1.468490 (0.805636) | 0.486559 / 4.584777 (-4.098218) | 3.565554 / 3.745712 (-0.180158) | 3.377659 / 5.269862 (-1.892202) | 2.029883 / 4.565676 (-2.535793) | 0.057303 / 0.424275 (-0.366972) | 0.007314 / 0.007607 (-0.000293) | 0.504263 / 0.226044 (0.278219) | 5.041196 / 2.268929 (2.772268) | 2.819273 / 55.444624 (-52.625351) | 2.421479 / 6.876477 (-4.454998) | 2.503063 / 2.142072 (0.360991) | 0.581467 / 4.805227 (-4.223760) | 0.133532 / 6.500664 (-6.367132) | 0.062504 / 0.075469 (-0.012965) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.328765 / 1.841788 (-0.513022) | 20.131672 / 8.074308 (12.057363) | 14.312895 / 10.191392 (4.121503) | 0.191199 / 0.680424 (-0.489225) | 0.018522 / 0.534201 (-0.515679) | 0.393121 / 0.579283 (-0.186162) | 0.413122 / 0.434364 (-0.021242) | 0.469312 / 0.540337 (-0.071026) | 0.633140 / 1.386936 (-0.753796) |

</details>
</details>

![](https://cml.dev/watermark.png#dbf6c103f5844de40431478e7e4a64fbf2c2c067 "CML watermark")