Issue: Fix time measuring snippet in docs
State: closed
Author: mariosasko

Fix https://discuss.huggingface.co/t/attributeerror-enter/60509

Comments:
_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007683 / 0.011353 (-0.003670) | 0.004159 / 0.011008 (-0.006849) | 0.097017 / 0.038508 (0.058509) | 0.074216 / 0.023109 (0.051107) | 0.323115 / 0.275898 (0.047217) | 0.412836 / 0.323480 (0.089356) | 0.005151 / 0.007986 (-0.002834) | 0.004037 / 0.004328 (-0.000292) | 0.067881 / 0.004250 (0.063631) | 0.051395 / 0.037052 (0.014342) | 0.356391 / 0.258489 (0.097901) | 0.386744 / 0.293841 (0.092903) | 0.043571 / 0.128546 (-0.084975) | 0.012844 / 0.075646 (-0.062803) | 0.369440 / 0.419271 (-0.049832) | 0.056944 / 0.043533 (0.013411) | 0.316159 / 0.255139 (0.061020) | 0.435530 / 0.283200 (0.152330) | 0.033622 / 0.141683 (-0.108061) | 1.379602 / 1.452155 (-0.072553) | 1.766400 / 1.492716 (0.273683) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.304151 / 0.018006 (0.286145) | 0.616365 / 0.000490 (0.615875) | 0.013588 / 0.000200 (0.013389) | 0.000441 / 0.000054 (0.000387) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.032812 / 0.037411 (-0.004600) | 0.100914 / 0.014526 (0.086388) | 0.124004 / 0.176557 (-0.052552) | 0.195087 / 0.737135 (-0.542048) | 0.124388 / 0.296338 (-0.171951) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.575649 / 0.215209 (0.360440) | 5.665461 / 2.077655 (3.587806) | 2.474892 / 1.504120 (0.970773) | 2.142687 / 1.541195 (0.601492) | 2.254962 / 1.468490 (0.786472) | 0.816635 / 4.584777 (-3.768141) | 5.044279 / 3.745712 (1.298567) | 4.566728 / 5.269862 (-0.703134) | 2.867146 / 4.565676 (-1.698531) | 0.092994 / 0.424275 (-0.331281) | 0.008395 / 0.007607 (0.000788) | 0.680346 / 0.226044 (0.454302) | 6.909875 / 2.268929 (4.640946) | 3.275602 / 55.444624 (-52.169022) | 2.556000 / 6.876477 (-4.320477) | 2.581337 / 2.142072 (0.439264) | 0.997883 / 4.805227 (-3.807344) | 0.204109 / 6.500664 (-6.296555) | 0.069705 / 0.075469 (-0.005764) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.504573 / 1.841788 (-0.337215) | 22.219363 / 8.074308 (14.145055) | 19.078040 / 10.191392 (8.886648) | 0.234970 / 0.680424 (-0.445454) | 0.027324 / 0.534201 (-0.506877) | 0.427960 / 0.579283 (-0.151323) | 0.570258 / 0.434364 (0.135894) | 0.502335 / 0.540337 (-0.038003) | 0.788078 / 1.386936 (-0.598858) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.008370 / 0.011353 (-0.002982) | 0.004573 / 0.011008 (-0.006435) | 0.073080 / 0.038508 (0.034572) | 0.068752 / 0.023109 (0.045643) | 0.439648 / 0.275898 (0.163750) | 0.499700 / 0.323480 (0.176220) | 0.006119 / 0.007986 (-0.001866) | 0.004300 / 0.004328 (-0.000028) | 0.073173 / 0.004250 (0.068923) | 0.055676 / 0.037052 (0.018624) | 0.464152 / 0.258489 (0.205663) | 0.476954 / 0.293841 (0.183113) | 0.046335 / 0.128546 (-0.082211) | 0.013373 / 0.075646 (-0.062274) | 0.092006 / 0.419271 (-0.327265) | 0.054802 / 0.043533 (0.011269) | 0.456594 / 0.255139 (0.201455) | 0.491931 / 0.283200 (0.208732) | 0.034021 / 0.141683 (-0.107662) | 1.575200 / 1.452155 (0.123045) | 1.689742 / 1.492716 (0.197026) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.299432 / 0.018006 (0.281426) | 0.605643 / 0.000490 (0.605153) | 0.006280 / 0.000200 (0.006080) | 0.000120 / 0.000054 (0.000066) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.028414 / 0.037411 (-0.008997) | 0.085812 / 0.014526 (0.071286) | 0.109142 / 0.176557 (-0.067414) | 0.163458 / 0.737135 (-0.573677) | 0.100837 / 0.296338 (-0.195501) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.615557 / 0.215209 (0.400348) | 6.051599 / 2.077655 (3.973944) | 2.872353 / 1.504120 (1.368234) | 2.508322 / 1.541195 (0.967128) | 2.550073 / 1.468490 (1.081583) | 0.835793 / 4.584777 (-3.748983) | 5.208484 / 3.745712 (1.462772) | 4.361846 / 5.269862 (-0.908016) | 2.776164 / 4.565676 (-1.789513) | 0.090831 / 0.424275 (-0.333444) | 0.007320 / 0.007607 (-0.000287) | 0.725533 / 0.226044 (0.499488) | 7.051321 / 2.268929 (4.782393) | 3.515464 / 55.444624 (-51.929160) | 2.798193 / 6.876477 (-4.078284) | 3.022512 / 2.142072 (0.880440) | 0.986744 / 4.805227 (-3.818484) | 0.198050 / 6.500664 (-6.302615) | 0.069200 / 0.075469 (-0.006269) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.623759 / 1.841788 (-0.218029) | 22.269700 / 8.074308 (14.195392) | 19.577429 / 10.191392 (9.386037) | 0.215990 / 0.680424 (-0.464434) | 0.033005 / 0.534201 (-0.501196) | 0.436848 / 0.579283 (-0.142435) | 0.591442 / 0.434364 (0.157078) | 0.547701 / 0.540337 (0.007364) | 0.741695 / 1.386936 (-0.645241) |

</details>
</details>

![](https://cml.dev/watermark.png#7e17e139b1323aca3321a5d2c2da40d82c458bae "CML watermark")


CI failures are unrelated

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009027 / 0.011353 (-0.002326) | 0.006118 / 0.011008 (-0.004890) | 0.118939 / 0.038508 (0.080431) | 0.089979 / 0.023109 (0.066869) | 0.412425 / 0.275898 (0.136527) | 0.455706 / 0.323480 (0.132227) | 0.006762 / 0.007986 (-0.001224) | 0.004409 / 0.004328 (0.000080) | 0.088002 / 0.004250 (0.083751) | 0.063708 / 0.037052 (0.026656) | 0.417373 / 0.258489 (0.158884) | 0.489582 / 0.293841 (0.195741) | 0.050222 / 0.128546 (-0.078324) | 0.014386 / 0.075646 (-0.061260) | 0.435363 / 0.419271 (0.016092) | 0.069375 / 0.043533 (0.025842) | 0.410242 / 0.255139 (0.155103) | 0.436439 / 0.283200 (0.153239) | 0.039318 / 0.141683 (-0.102365) | 1.857574 / 1.452155 (0.405419) | 1.919402 / 1.492716 (0.426686) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.343916 / 0.018006 (0.325910) | 0.633639 / 0.000490 (0.633150) | 0.014756 / 0.000200 (0.014557) | 0.000707 / 0.000054 (0.000652) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.031983 / 0.037411 (-0.005429) | 0.097222 / 0.014526 (0.082697) | 0.114644 / 0.176557 (-0.061912) | 0.187787 / 0.737135 (-0.549348) | 0.120595 / 0.296338 (-0.175743) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.605861 / 0.215209 (0.390652) | 6.039318 / 2.077655 (3.961664) | 2.699251 / 1.504120 (1.195132) | 2.436398 / 1.541195 (0.895203) | 2.493653 / 1.468490 (1.025163) | 0.889423 / 4.584777 (-3.695354) | 5.384769 / 3.745712 (1.639056) | 5.033033 / 5.269862 (-0.236829) | 3.056894 / 4.565676 (-1.508783) | 0.100683 / 0.424275 (-0.323592) | 0.009103 / 0.007607 (0.001495) | 0.737066 / 0.226044 (0.511021) | 7.370485 / 2.268929 (5.101556) | 3.422670 / 55.444624 (-52.021954) | 2.830392 / 6.876477 (-4.046084) | 2.985789 / 2.142072 (0.843717) | 0.999239 / 4.805227 (-3.805989) | 0.203506 / 6.500664 (-6.297158) | 0.076135 / 0.075469 (0.000666) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.697001 / 1.841788 (-0.144787) | 24.653975 / 8.074308 (16.579667) | 22.241622 / 10.191392 (12.050230) | 0.257075 / 0.680424 (-0.423349) | 0.029159 / 0.534201 (-0.505041) | 0.493329 / 0.579283 (-0.085954) | 0.596661 / 0.434364 (0.162297) | 0.569431 / 0.540337 (0.029094) | 0.812231 / 1.386936 (-0.574705) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.009815 / 0.011353 (-0.001538) | 0.005136 / 0.011008 (-0.005872) | 0.078224 / 0.038508 (0.039716) | 0.103276 / 0.023109 (0.080166) | 0.512742 / 0.275898 (0.236844) | 0.544010 / 0.323480 (0.220530) | 0.007957 / 0.007986 (-0.000029) | 0.004629 / 0.004328 (0.000300) | 0.074983 / 0.004250 (0.070733) | 0.071831 / 0.037052 (0.034778) | 0.542752 / 0.258489 (0.284262) | 0.573176 / 0.293841 (0.279335) | 0.053939 / 0.128546 (-0.074607) | 0.015007 / 0.075646 (-0.060640) | 0.085389 / 0.419271 (-0.333882) | 0.063587 / 0.043533 (0.020055) | 0.509580 / 0.255139 (0.254441) | 0.563374 / 0.283200 (0.280174) | 0.037575 / 0.141683 (-0.104108) | 1.840740 / 1.452155 (0.388585) | 1.836414 / 1.492716 (0.343698) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.310188 / 0.018006 (0.292182) | 0.641478 / 0.000490 (0.640988) | 0.011057 / 0.000200 (0.010857) | 0.000173 / 0.000054 (0.000119) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.043280 / 0.037411 (0.005869) | 0.109256 / 0.014526 (0.094730) | 0.126701 / 0.176557 (-0.049856) | 0.199172 / 0.737135 (-0.537963) | 0.123584 / 0.296338 (-0.172755) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.649272 / 0.215209 (0.434063) | 6.487501 / 2.077655 (4.409846) | 3.170330 / 1.504120 (1.666210) | 2.960912 / 1.541195 (1.419718) | 3.024531 / 1.468490 (1.556041) | 0.905112 / 4.584777 (-3.679665) | 5.560961 / 3.745712 (1.815249) | 4.920463 / 5.269862 (-0.349399) | 3.158989 / 4.565676 (-1.406687) | 0.095444 / 0.424275 (-0.328831) | 0.008264 / 0.007607 (0.000657) | 0.819292 / 0.226044 (0.593247) | 7.982695 / 2.268929 (5.713767) | 4.098704 / 55.444624 (-51.345921) | 3.442330 / 6.876477 (-3.434147) | 3.763426 / 2.142072 (1.621354) | 1.065464 / 4.805227 (-3.739763) | 0.215089 / 6.500664 (-6.285575) | 0.085280 / 0.075469 (0.009811) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.881770 / 1.841788 (0.039983) | 25.671479 / 8.074308 (17.597171) | 22.367019 / 10.191392 (12.175627) | 0.241377 / 0.680424 (-0.439047) | 0.033555 / 0.534201 (-0.500646) | 0.501786 / 0.579283 (-0.077497) | 0.596376 / 0.434364 (0.162012) | 0.579674 / 0.540337 (0.039337) | 0.855534 / 1.386936 (-0.531402) |

</details>
</details>

![](https://cml.dev/watermark.png#c9c1166e1cf81d38534020f9c167b326585339e5 "CML watermark")