Add pyarrow-hotfix to release docs
State: closed | Author: albertvillanova (MEMBER)
Labels: name: maintenance

Add `pyarrow-hotfix` to release docs.

Comments: <details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.004755 / 0.011353 (-0.006598) | 0.002683 / 0.011008 (-0.008325) | 0.061701 / 0.038508 (0.023193) | 0.030123 / 0.023109 (0.007013) | 0.238186 / 0.275898 (-0.037712) | 0.266570 / 0.323480 (-0.056910) | 0.002898 / 0.007986 (-0.005088) | 0.002381 / 0.004328 (-0.001948) | 0.048033 / 0.004250 (0.043782) | 0.044529 / 0.037052 (0.007477) | 0.246728 / 0.258489 (-0.011761) | 0.302066 / 0.293841 (0.008225) | 0.024008 / 0.128546 (-0.104539) | 0.006626 / 0.075646 (-0.069020) | 0.202000 / 0.419271 (-0.217272) | 0.056492 / 0.043533 (0.012959) | 0.243417 / 0.255139 (-0.011722) | 0.263947 / 0.283200 (-0.019253) | 0.020481 / 0.141683 (-0.121202) | 1.130635 / 1.452155 (-0.321520) | 1.180570 / 1.492716 (-0.312146) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.095541 / 0.018006 (0.077535) | 0.306152 / 0.000490 (0.305662) | 0.000217 / 0.000200 (0.000017) | 0.000044 / 0.000054 (-0.000011) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.018593 / 0.037411 (-0.018818) | 0.063029 / 0.014526 (0.048503) | 0.074312 / 0.176557 (-0.102245) | 0.119882 / 0.737135 (-0.617254) | 0.074066 / 0.296338 (-0.222273) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.275409 / 0.215209 (0.060200) | 2.727061 / 2.077655 (0.649407) | 1.415632 / 1.504120 (-0.088488) | 1.294922 / 1.541195 (-0.246273) | 1.341636 / 1.468490 (-0.126854) | 0.403250 / 4.584777 (-4.181527) | 2.384657 / 3.745712 (-1.361055) | 2.604131 / 5.269862 (-2.665731) | 1.558888 / 4.565676 (-3.006789) | 0.046008 / 0.424275 (-0.378267) | 0.004819 / 0.007607 (-0.002789) | 0.331046 / 0.226044 (0.105002) | 3.340950 / 2.268929 (1.072021) | 1.801077 / 55.444624 (-53.643548) | 1.479162 / 6.876477 (-5.397315) | 1.503713 / 2.142072 (-0.638359) | 0.474931 / 4.805227 (-4.330296) | 0.101869 / 6.500664 (-6.398795) | 0.041946 / 0.075469 (-0.033523) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.955641 / 1.841788 (-0.886147) | 11.441032 / 8.074308 (3.366724) | 10.267731 / 10.191392 (0.076339) | 0.128735 / 0.680424 (-0.551689) | 0.013942 / 0.534201 (-0.520259) | 0.266620 / 0.579283 (-0.312663) | 0.262334 / 0.434364 (-0.172029) | 0.302713 / 0.540337 (-0.237624) | 0.430323 / 1.386936 (-0.956613) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.004670 / 0.011353 (-0.006683) | 0.002671 / 0.011008 (-0.008338) | 0.048949 / 0.038508 (0.010441) | 0.052520 / 0.023109 (0.029411) | 0.272614 / 0.275898 (-0.003284) | 0.292618 / 0.323480 (-0.030862) | 0.004016 / 0.007986 (-0.003969) | 0.002430 / 0.004328 (-0.001899) | 0.048313 / 0.004250 (0.044063) | 0.038647 / 0.037052 (0.001595) | 0.279893 / 0.258489 (0.021404) | 0.305371 / 0.293841 (0.011530) | 0.023710 / 0.128546 (-0.104836) | 0.006999 / 0.075646 (-0.068648) | 0.053315 / 0.419271 (-0.365956) | 0.032417 / 0.043533 (-0.011115) | 0.272066 / 0.255139 (0.016927) | 0.291717 / 0.283200 (0.008518) | 0.018127 / 0.141683 (-0.123556) | 1.173611 / 1.452155 (-0.278544) | 1.183659 / 1.492716 (-0.309057) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.094831 / 0.018006 (0.076824) | 0.304911 / 0.000490 (0.304421) | 0.000225 / 0.000200 (0.000025) | 0.000049 / 0.000054 (-0.000006) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.020948 / 0.037411 (-0.016463) | 0.070255 / 0.014526 (0.055729) | 0.081371 / 0.176557 (-0.095186) | 0.118932 / 0.737135 (-0.618203) | 0.082207 / 0.296338 (-0.214132) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.294067 / 0.215209 (0.078858) | 2.856981 / 2.077655 (0.779326) | 1.598392 / 1.504120 (0.094273) | 1.479093 / 1.541195 (-0.062102) | 1.509495 / 1.468490 (0.041005) | 0.396303 / 4.584777 (-4.188473) | 2.429077 / 3.745712 (-1.316635) | 2.525037 / 5.269862 (-2.744824) | 1.503332 / 4.565676 (-3.062345) | 0.046191 / 0.424275 (-0.378084) | 0.004858 / 0.007607 (-0.002750) | 0.349528 / 0.226044 (0.123484) | 3.401451 / 2.268929 (1.132522) | 1.989613 / 55.444624 (-53.455012) | 1.664528 / 6.876477 (-5.211949) | 1.669076 / 2.142072 (-0.472997) | 0.467090 / 4.805227 (-4.338137) | 0.098137 / 6.500664 (-6.402527) | 0.040448 / 0.075469 (-0.035021) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.969578 / 1.841788 (-0.872210) | 12.064705 / 8.074308 (3.990396) | 10.991438 / 10.191392 (0.800046) | 0.130149 / 0.680424 (-0.550275) | 0.015357 / 0.534201 (-0.518844) | 0.266567 / 0.579283 (-0.312717) | 0.270619 / 0.434364 (-0.163744) | 0.305978 / 0.540337 (-0.234359) | 0.411164 / 1.386936 (-0.975772) |

</details>
</details>

![](https://cml.dev/watermark.png#86a2cf3174c55899535ee5f1707892a430ee53bc "CML watermark")


_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009810 / 0.011353 (-0.001543) | 0.005411 / 0.011008 (-0.005598) | 0.111670 / 0.038508 (0.073162) | 0.050288 / 0.023109 (0.027179) | 0.415625 / 0.275898 (0.139727) | 0.479382 / 0.323480 (0.155902) | 0.005104 / 0.007986 (-0.002882) | 0.007122 / 0.004328 (0.002793) | 0.079626 / 0.004250 (0.075375) | 0.079421 / 0.037052 (0.042369) | 0.406722 / 0.258489 (0.148233) | 0.461511 / 0.293841 (0.167670) | 0.053812 / 0.128546 (-0.074734) | 0.014315 / 0.075646 (-0.061331) | 0.389636 / 0.419271 (-0.029636) | 0.111859 / 0.043533 (0.068326) | 0.411703 / 0.255139 (0.156564) | 0.457072 / 0.283200 (0.173872) | 0.039807 / 0.141683 (-0.101876) | 1.744064 / 1.452155 (0.291909) | 1.968321 / 1.492716 (0.475604) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.341839 / 0.018006 (0.323833) | 0.628083 / 0.000490 (0.627593) | 0.023787 / 0.000200 (0.023587) | 0.000601 / 0.000054 (0.000547) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034170 / 0.037411 (-0.003241) | 0.091159 / 0.014526 (0.076633) | 0.108993 / 0.176557 (-0.067563) | 0.186906 / 0.737135 (-0.550229) | 0.109753 / 0.296338 (-0.186586) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.684138 / 0.215209 (0.468929) | 6.634852 / 2.077655 (4.557198) | 3.102870 / 1.504120 (1.598750) | 2.831023 / 1.541195 (1.289828) | 2.831597 / 1.468490 (1.363107) | 0.903584 / 4.584777 (-3.681193) | 5.503341 / 3.745712 (1.757629) | 4.970283 / 5.269862 (-0.299579) | 3.139413 / 4.565676 (-1.426264) | 0.109848 / 0.424275 (-0.314427) | 0.008501 / 0.007607 (0.000894) | 0.823815 / 0.226044 (0.597770) | 7.963355 / 2.268929 (5.694426) | 4.002010 / 55.444624 (-51.442614) | 3.229390 / 6.876477 (-3.647087) | 3.166413 / 2.142072 (1.024341) | 1.030313 / 4.805227 (-3.774914) | 0.219394 / 6.500664 (-6.281270) | 0.077760 / 0.075469 (0.002291) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.580309 / 1.841788 (-0.261479) | 24.279185 / 8.074308 (16.204877) | 22.305293 / 10.191392 (12.113901) | 0.235711 / 0.680424 (-0.444713) | 0.030342 / 0.534201 (-0.503859) | 0.498137 / 0.579283 (-0.081146) | 0.619173 / 0.434364 (0.184809) | 0.529904 / 0.540337 (-0.010434) | 0.822547 / 1.386936 (-0.564389) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009375 / 0.011353 (-0.001978) | 0.006009 / 0.011008 (-0.004999) | 0.074080 / 0.038508 (0.035572) | 0.089454 / 0.023109 (0.066345) | 0.473458 / 0.275898 (0.197560) | 0.462558 / 0.323480 (0.139078) | 0.006415 / 0.007986 (-0.001571) | 0.004777 / 0.004328 (0.000448) | 0.076563 / 0.004250 (0.072313) | 0.062793 / 0.037052 (0.025741) | 0.455860 / 0.258489 (0.197371) | 0.485281 / 0.293841 (0.191440) | 0.052966 / 0.128546 (-0.075580) | 0.021600 / 0.075646 (-0.054046) | 0.090407 / 0.419271 (-0.328864) | 0.063951 / 0.043533 (0.020418) | 0.487561 / 0.255139 (0.232422) | 0.479958 / 0.283200 (0.196758) | 0.039263 / 0.141683 (-0.102420) | 1.727215 / 1.452155 (0.275061) | 1.962039 / 1.492716 (0.469323) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.296267 / 0.018006 (0.278261) | 0.604982 / 0.000490 (0.604493) | 0.007842 / 0.000200 (0.007642) | 0.000096 / 0.000054 (0.000041) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034317 / 0.037411 (-0.003094) | 0.097796 / 0.014526 (0.083270) | 0.126034 / 0.176557 (-0.050522) | 0.180873 / 0.737135 (-0.556262) | 0.125410 / 0.296338 (-0.170928) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.608278 / 0.215209 (0.393069) | 6.154006 / 2.077655 (4.076351) | 2.822342 / 1.504120 (1.318222) | 2.568263 / 1.541195 (1.027068) | 2.518545 / 1.468490 (1.050055) | 0.863186 / 4.584777 (-3.721591) | 5.367969 / 3.745712 (1.622257) | 4.737691 / 5.269862 (-0.532170) | 2.917620 / 4.565676 (-1.648056) | 0.100731 / 0.424275 (-0.323544) | 0.008611 / 0.007607 (0.001004) | 0.735523 / 0.226044 (0.509479) | 7.552790 / 2.268929 (5.283862) | 3.821835 / 55.444624 (-51.622789) | 2.878259 / 6.876477 (-3.998217) | 2.957686 / 2.142072 (0.815613) | 0.964630 / 4.805227 (-3.840598) | 0.207098 / 6.500664 (-6.293566) | 0.084215 / 0.075469 (0.008746) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.711020 / 1.841788 (-0.130768) | 24.034122 / 8.074308 (15.959814) | 21.378504 / 10.191392 (11.187112) | 0.233433 / 0.680424 (-0.446990) | 0.037214 / 0.534201 (-0.496987) | 0.511952 / 0.579283 (-0.067332) | 0.591486 / 0.434364 (0.157123) | 0.606549 / 0.540337 (0.066211) | 0.833773 / 1.386936 (-0.553163) |

</details>
</details>

![](https://cml.dev/watermark.png#671f9b32fc559a35996c1b9070fad1a2647a7fef "CML watermark")