Title: Remove redundant condition in builders
Author: albertvillanova (MEMBER)
State: closed
Created: 2023-11-10T14:56:43Z

Minor refactoring to remove redundant condition.

Comments:
<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.004475 / 0.011353 (-0.006878) | 0.002840 / 0.011008 (-0.008168) | 0.061544 / 0.038508 (0.023036) | 0.031237 / 0.023109 (0.008128) | 0.243270 / 0.275898 (-0.032628) | 0.271903 / 0.323480 (-0.051577) | 0.002906 / 0.007986 (-0.005080) | 0.003118 / 0.004328 (-0.001210) | 0.047362 / 0.004250 (0.043112) | 0.047840 / 0.037052 (0.010788) | 0.244044 / 0.258489 (-0.014445) | 0.279310 / 0.293841 (-0.014531) | 0.023408 / 0.128546 (-0.105138) | 0.007110 / 0.075646 (-0.068536) | 0.207328 / 0.419271 (-0.211943) | 0.058463 / 0.043533 (0.014930) | 0.245631 / 0.255139 (-0.009508) | 0.267755 / 0.283200 (-0.015445) | 0.018147 / 0.141683 (-0.123536) | 1.086877 / 1.452155 (-0.365278) | 1.155380 / 1.492716 (-0.337337) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.091925 / 0.018006 (0.073919) | 0.299858 / 0.000490 (0.299368) | 0.000232 / 0.000200 (0.000032) | 0.000047 / 0.000054 (-0.000007) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.018416 / 0.037411 (-0.018995) | 0.062608 / 0.014526 (0.048082) | 0.073897 / 0.176557 (-0.102660) | 0.120216 / 0.737135 (-0.616919) | 0.075788 / 0.296338 (-0.220550) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.287823 / 0.215209 (0.072614) | 2.797546 / 2.077655 (0.719891) | 1.470878 / 1.504120 (-0.033242) | 1.347497 / 1.541195 (-0.193698) | 1.363837 / 1.468490 (-0.104653) | 0.400069 / 4.584777 (-4.184708) | 2.338870 / 3.745712 (-1.406842) | 2.564075 / 5.269862 (-2.705787) | 1.568454 / 4.565676 (-2.997222) | 0.047103 / 0.424275 (-0.377172) | 0.004783 / 0.007607 (-0.002824) | 0.345244 / 0.226044 (0.119200) | 3.407752 / 2.268929 (1.138823) | 1.826552 / 55.444624 (-53.618073) | 1.536714 / 6.876477 (-5.339763) | 1.543138 / 2.142072 (-0.598934) | 0.478996 / 4.805227 (-4.326232) | 0.099580 / 6.500664 (-6.401085) | 0.041994 / 0.075469 (-0.033475) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.947106 / 1.841788 (-0.894682) | 11.391262 / 8.074308 (3.316954) | 10.531141 / 10.191392 (0.339749) | 0.141497 / 0.680424 (-0.538927) | 0.014214 / 0.534201 (-0.519987) | 0.269346 / 0.579283 (-0.309937) | 0.268129 / 0.434364 (-0.166235) | 0.309496 / 0.540337 (-0.230841) | 0.429207 / 1.386936 (-0.957729) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.004770 / 0.011353 (-0.006583) | 0.002878 / 0.011008 (-0.008130) | 0.048248 / 0.038508 (0.009740) | 0.051068 / 0.023109 (0.027959) | 0.272076 / 0.275898 (-0.003822) | 0.292423 / 0.323480 (-0.031057) | 0.004016 / 0.007986 (-0.003970) | 0.002522 / 0.004328 (-0.001807) | 0.047617 / 0.004250 (0.043367) | 0.038168 / 0.037052 (0.001115) | 0.275236 / 0.258489 (0.016746) | 0.303811 / 0.293841 (0.009970) | 0.023816 / 0.128546 (-0.104730) | 0.007177 / 0.075646 (-0.068469) | 0.053453 / 0.419271 (-0.365818) | 0.032425 / 0.043533 (-0.011108) | 0.271620 / 0.255139 (0.016481) | 0.289618 / 0.283200 (0.006418) | 0.017986 / 0.141683 (-0.123697) | 1.154225 / 1.452155 (-0.297930) | 1.224244 / 1.492716 (-0.268472) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.090477 / 0.018006 (0.072471) | 0.299461 / 0.000490 (0.298971) | 0.000224 / 0.000200 (0.000024) | 0.000053 / 0.000054 (-0.000002) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.022043 / 0.037411 (-0.015369) | 0.070327 / 0.014526 (0.055801) | 0.080132 / 0.176557 (-0.096425) | 0.120007 / 0.737135 (-0.617128) | 0.083037 / 0.296338 (-0.213301) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.294538 / 0.215209 (0.079329) | 2.882791 / 2.077655 (0.805136) | 1.582923 / 1.504120 (0.078803) | 1.457091 / 1.541195 (-0.084104) | 1.536149 / 1.468490 (0.067659) | 0.401539 / 4.584777 (-4.183238) | 2.440919 / 3.745712 (-1.304793) | 2.503108 / 5.269862 (-2.766753) | 1.509216 / 4.565676 (-3.056460) | 0.046267 / 0.424275 (-0.378008) | 0.004790 / 0.007607 (-0.002817) | 0.336137 / 0.226044 (0.110093) | 3.331655 / 2.268929 (1.062726) | 1.954228 / 55.444624 (-53.490396) | 1.686637 / 6.876477 (-5.189840) | 1.650278 / 2.142072 (-0.491794) | 0.473895 / 4.805227 (-4.331333) | 0.096908 / 6.500664 (-6.403756) | 0.040387 / 0.075469 (-0.035082) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.972999 / 1.841788 (-0.868789) | 11.978367 / 8.074308 (3.904059) | 10.861092 / 10.191392 (0.669699) | 0.129054 / 0.680424 (-0.551369) | 0.015988 / 0.534201 (-0.518213) | 0.268827 / 0.579283 (-0.310456) | 0.271714 / 0.434364 (-0.162649) | 0.304045 / 0.540337 (-0.236293) | 0.413158 / 1.386936 (-0.973778) |

</details>
</details>

![](https://cml.dev/watermark.png#9e4348a233a75907c305b3159ac9cb183cf30ea5 "CML watermark")


_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005286 / 0.011353 (-0.006067) | 0.002860 / 0.011008 (-0.008149) | 0.062449 / 0.038508 (0.023941) | 0.035346 / 0.023109 (0.012237) | 0.241685 / 0.275898 (-0.034213) | 0.268116 / 0.323480 (-0.055364) | 0.003050 / 0.007986 (-0.004935) | 0.003134 / 0.004328 (-0.001194) | 0.048818 / 0.004250 (0.044567) | 0.049187 / 0.037052 (0.012135) | 0.247395 / 0.258489 (-0.011094) | 0.280301 / 0.293841 (-0.013540) | 0.023801 / 0.128546 (-0.104745) | 0.007653 / 0.075646 (-0.067994) | 0.204185 / 0.419271 (-0.215087) | 0.071251 / 0.043533 (0.027718) | 0.244409 / 0.255139 (-0.010730) | 0.262363 / 0.283200 (-0.020836) | 0.018631 / 0.141683 (-0.123052) | 1.110152 / 1.452155 (-0.342003) | 1.165093 / 1.492716 (-0.327624) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.099536 / 0.018006 (0.081530) | 0.309598 / 0.000490 (0.309109) | 0.000207 / 0.000200 (0.000007) | 0.000050 / 0.000054 (-0.000004) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.019213 / 0.037411 (-0.018198) | 0.069296 / 0.014526 (0.054770) | 0.074752 / 0.176557 (-0.101804) | 0.121314 / 0.737135 (-0.615822) | 0.081274 / 0.296338 (-0.215065) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.281345 / 0.215209 (0.066136) | 2.755435 / 2.077655 (0.677780) | 1.453358 / 1.504120 (-0.050762) | 1.328222 / 1.541195 (-0.212973) | 1.392281 / 1.468490 (-0.076209) | 0.410539 / 4.584777 (-4.174238) | 2.452072 / 3.745712 (-1.293640) | 2.777757 / 5.269862 (-2.492105) | 1.656719 / 4.565676 (-2.908958) | 0.046844 / 0.424275 (-0.377431) | 0.004785 / 0.007607 (-0.002822) | 0.336567 / 0.226044 (0.110522) | 3.317564 / 2.268929 (1.048635) | 1.830737 / 55.444624 (-53.613888) | 1.528464 / 6.876477 (-5.348013) | 1.620527 / 2.142072 (-0.521545) | 0.480662 / 4.805227 (-4.324565) | 0.100819 / 6.500664 (-6.399845) | 0.042501 / 0.075469 (-0.032968) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.962593 / 1.841788 (-0.879195) | 12.508048 / 8.074308 (4.433740) | 11.117398 / 10.191392 (0.926006) | 0.131265 / 0.680424 (-0.549159) | 0.014469 / 0.534201 (-0.519732) | 0.271627 / 0.579283 (-0.307656) | 0.274966 / 0.434364 (-0.159398) | 0.313260 / 0.540337 (-0.227077) | 0.444741 / 1.386936 (-0.942195) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.004974 / 0.011353 (-0.006379) | 0.003383 / 0.011008 (-0.007626) | 0.048792 / 0.038508 (0.010284) | 0.052821 / 0.023109 (0.029712) | 0.267123 / 0.275898 (-0.008775) | 0.293604 / 0.323480 (-0.029876) | 0.003968 / 0.007986 (-0.004018) | 0.002594 / 0.004328 (-0.001735) | 0.047690 / 0.004250 (0.043439) | 0.040236 / 0.037052 (0.003183) | 0.267805 / 0.258489 (0.009315) | 0.310543 / 0.293841 (0.016702) | 0.025707 / 0.128546 (-0.102839) | 0.008012 / 0.075646 (-0.067634) | 0.054460 / 0.419271 (-0.364812) | 0.033545 / 0.043533 (-0.009988) | 0.270166 / 0.255139 (0.015027) | 0.285965 / 0.283200 (0.002765) | 0.019391 / 0.141683 (-0.122292) | 1.144991 / 1.452155 (-0.307164) | 1.198491 / 1.492716 (-0.294225) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.094757 / 0.018006 (0.076751) | 0.306712 / 0.000490 (0.306222) | 0.000218 / 0.000200 (0.000018) | 0.000055 / 0.000054 (0.000000) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.020995 / 0.037411 (-0.016417) | 0.070293 / 0.014526 (0.055767) | 0.081441 / 0.176557 (-0.095116) | 0.119538 / 0.737135 (-0.617597) | 0.081454 / 0.296338 (-0.214885) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.293451 / 0.215209 (0.078242) | 2.880378 / 2.077655 (0.802723) | 1.572547 / 1.504120 (0.068427) | 1.439172 / 1.541195 (-0.102023) | 1.506343 / 1.468490 (0.037853) | 0.402764 / 4.584777 (-4.182013) | 2.501341 / 3.745712 (-1.244371) | 2.538494 / 5.269862 (-2.731367) | 1.524306 / 4.565676 (-3.041371) | 0.046401 / 0.424275 (-0.377874) | 0.004781 / 0.007607 (-0.002826) | 0.349448 / 0.226044 (0.123404) | 3.416181 / 2.268929 (1.147252) | 1.964204 / 55.444624 (-53.480420) | 1.648564 / 6.876477 (-5.227912) | 1.675977 / 2.142072 (-0.466095) | 0.475717 / 4.805227 (-4.329511) | 0.098416 / 6.500664 (-6.402248) | 0.041212 / 0.075469 (-0.034257) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.975928 / 1.841788 (-0.865860) | 12.066648 / 8.074308 (3.992340) | 10.943181 / 10.191392 (0.751789) | 0.149687 / 0.680424 (-0.530736) | 0.015107 / 0.534201 (-0.519094) | 0.268950 / 0.579283 (-0.310333) | 0.280419 / 0.434364 (-0.153945) | 0.305263 / 0.540337 (-0.235074) | 0.408486 / 1.386936 (-0.978450) |

</details>
</details>

![](https://cml.dev/watermark.png#344086a7a1707ef20b57399f813ef64ce679e956 "CML watermark")