Deprecate `errors` param in favor of `encoding_errors` in text builder
Opened by mariosasko (closed)

For consistency with the JSON builder and Pandas

Comments:
_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006518 / 0.011353 (-0.004835) | 0.004121 / 0.011008 (-0.006887) | 0.103350 / 0.038508 (0.064842) | 0.045030 / 0.023109 (0.021920) | 0.351670 / 0.275898 (0.075772) | 0.408110 / 0.323480 (0.084630) | 0.003883 / 0.007986 (-0.004102) | 0.003352 / 0.004328 (-0.000977) | 0.078786 / 0.004250 (0.074535) | 0.063977 / 0.037052 (0.026925) | 0.369759 / 0.258489 (0.111270) | 0.415103 / 0.293841 (0.121262) | 0.033069 / 0.128546 (-0.095477) | 0.008863 / 0.075646 (-0.066783) | 0.353660 / 0.419271 (-0.065611) | 0.055714 / 0.043533 (0.012181) | 0.350458 / 0.255139 (0.095319) | 0.369505 / 0.283200 (0.086305) | 0.022822 / 0.141683 (-0.118861) | 1.537588 / 1.452155 (0.085433) | 1.590569 / 1.492716 (0.097853) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.206826 / 0.018006 (0.188819) | 0.471625 / 0.000490 (0.471135) | 0.005188 / 0.000200 (0.004988) | 0.000316 / 0.000054 (0.000261) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.028148 / 0.037411 (-0.009263) | 0.111941 / 0.014526 (0.097415) | 0.122106 / 0.176557 (-0.054451) | 0.181127 / 0.737135 (-0.556009) | 0.127534 / 0.296338 (-0.168805) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.409520 / 0.215209 (0.194311) | 4.098455 / 2.077655 (2.020800) | 1.852447 / 1.504120 (0.348327) | 1.657036 / 1.541195 (0.115842) | 1.709624 / 1.468490 (0.241134) | 0.542806 / 4.584777 (-4.041970) | 3.809352 / 3.745712 (0.063640) | 1.855412 / 5.269862 (-3.414449) | 1.109180 / 4.565676 (-3.456497) | 0.066801 / 0.424275 (-0.357474) | 0.011832 / 0.007607 (0.004225) | 0.518338 / 0.226044 (0.292293) | 5.190108 / 2.268929 (2.921179) | 2.320602 / 55.444624 (-53.124023) | 1.991416 / 6.876477 (-4.885060) | 2.106989 / 2.142072 (-0.035084) | 0.668914 / 4.805227 (-4.136313) | 0.145325 / 6.500664 (-6.355340) | 0.065145 / 0.075469 (-0.010324) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.254706 / 1.841788 (-0.587082) | 14.707264 / 8.074308 (6.632956) | 14.615423 / 10.191392 (4.424031) | 0.170764 / 0.680424 (-0.509659) | 0.017905 / 0.534201 (-0.516296) | 0.435606 / 0.579283 (-0.143677) | 0.434648 / 0.434364 (0.000284) | 0.520813 / 0.540337 (-0.019524) | 0.633902 / 1.386936 (-0.753034) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007212 / 0.011353 (-0.004141) | 0.004301 / 0.011008 (-0.006707) | 0.080767 / 0.038508 (0.042258) | 0.051949 / 0.023109 (0.028840) | 0.398473 / 0.275898 (0.122575) | 0.465038 / 0.323480 (0.141558) | 0.005580 / 0.007986 (-0.002406) | 0.003556 / 0.004328 (-0.000773) | 0.080682 / 0.004250 (0.076431) | 0.059517 / 0.037052 (0.022464) | 0.421171 / 0.258489 (0.162682) | 0.459752 / 0.293841 (0.165911) | 0.032960 / 0.128546 (-0.095586) | 0.009107 / 0.075646 (-0.066539) | 0.086382 / 0.419271 (-0.332889) | 0.056053 / 0.043533 (0.012520) | 0.393357 / 0.255139 (0.138218) | 0.412972 / 0.283200 (0.129772) | 0.031115 / 0.141683 (-0.110568) | 1.576961 / 1.452155 (0.124806) | 1.627249 / 1.492716 (0.134533) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.227618 / 0.018006 (0.209612) | 0.444640 / 0.000490 (0.444150) | 0.004376 / 0.000200 (0.004176) | 0.000092 / 0.000054 (0.000038) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.030826 / 0.037411 (-0.006586) | 0.117587 / 0.014526 (0.103062) | 0.127467 / 0.176557 (-0.049089) | 0.184440 / 0.737135 (-0.552695) | 0.133664 / 0.296338 (-0.162675) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.443183 / 0.215209 (0.227974) | 4.408312 / 2.077655 (2.330658) | 2.132487 / 1.504120 (0.628367) | 1.923632 / 1.541195 (0.382438) | 1.967882 / 1.468490 (0.499392) | 0.552954 / 4.584777 (-4.031823) | 3.777701 / 3.745712 (0.031989) | 1.857686 / 5.269862 (-3.412176) | 1.104847 / 4.565676 (-3.460829) | 0.068350 / 0.424275 (-0.355925) | 0.012437 / 0.007607 (0.004830) | 0.559258 / 0.226044 (0.333214) | 5.593258 / 2.268929 (3.324330) | 2.648059 / 55.444624 (-52.796565) | 2.277428 / 6.876477 (-4.599049) | 2.351685 / 2.142072 (0.209612) | 0.678750 / 4.805227 (-4.126477) | 0.145550 / 6.500664 (-6.355114) | 0.066556 / 0.075469 (-0.008913) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.327128 / 1.841788 (-0.514659) | 15.649079 / 8.074308 (7.574771) | 14.478659 / 10.191392 (4.287267) | 0.147633 / 0.680424 (-0.532791) | 0.018502 / 0.534201 (-0.515699) | 0.438556 / 0.579283 (-0.140727) | 0.433381 / 0.434364 (-0.000983) | 0.514367 / 0.540337 (-0.025970) | 0.618347 / 1.386936 (-0.768589) |

</details>
</details>

![](https://cml.dev/watermark.png#16aa1c886c5b499641a4bb3d8ce4a4f7de8244b7 "CML watermark")


<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006078 / 0.011353 (-0.005275) | 0.003914 / 0.011008 (-0.007095) | 0.102039 / 0.038508 (0.063531) | 0.037660 / 0.023109 (0.014551) | 0.348963 / 0.275898 (0.073065) | 0.407284 / 0.323480 (0.083804) | 0.004661 / 0.007986 (-0.003324) | 0.003253 / 0.004328 (-0.001076) | 0.078276 / 0.004250 (0.074025) | 0.054144 / 0.037052 (0.017091) | 0.376715 / 0.258489 (0.118225) | 0.418499 / 0.293841 (0.124658) | 0.027627 / 0.128546 (-0.100919) | 0.008494 / 0.075646 (-0.067152) | 0.316894 / 0.419271 (-0.102377) | 0.046560 / 0.043533 (0.003027) | 0.339835 / 0.255139 (0.084696) | 0.374628 / 0.283200 (0.091428) | 0.020729 / 0.141683 (-0.120954) | 1.502769 / 1.452155 (0.050615) | 1.548756 / 1.492716 (0.056040) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.229192 / 0.018006 (0.211186) | 0.426245 / 0.000490 (0.425756) | 0.005190 / 0.000200 (0.004990) | 0.000081 / 0.000054 (0.000026) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.024271 / 0.037411 (-0.013140) | 0.098869 / 0.014526 (0.084343) | 0.105079 / 0.176557 (-0.071477) | 0.164707 / 0.737135 (-0.572428) | 0.110337 / 0.296338 (-0.186002) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.426593 / 0.215209 (0.211383) | 4.293977 / 2.077655 (2.216323) | 1.928502 / 1.504120 (0.424382) | 1.728623 / 1.541195 (0.187428) | 1.792084 / 1.468490 (0.323594) | 0.568737 / 4.584777 (-4.016040) | 3.438534 / 3.745712 (-0.307178) | 1.797798 / 5.269862 (-3.472063) | 1.054078 / 4.565676 (-3.511598) | 0.068711 / 0.424275 (-0.355564) | 0.011250 / 0.007607 (0.003643) | 0.529299 / 0.226044 (0.303255) | 5.283965 / 2.268929 (3.015037) | 2.358274 / 55.444624 (-53.086350) | 2.012818 / 6.876477 (-4.863659) | 2.109923 / 2.142072 (-0.032149) | 0.679556 / 4.805227 (-4.125671) | 0.138346 / 6.500664 (-6.362318) | 0.066349 / 0.075469 (-0.009120) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.193994 / 1.841788 (-0.647794) | 14.073158 / 8.074308 (5.998850) | 13.488525 / 10.191392 (3.297133) | 0.144536 / 0.680424 (-0.535888) | 0.016748 / 0.534201 (-0.517453) | 0.362703 / 0.579283 (-0.216580) | 0.389511 / 0.434364 (-0.044853) | 0.427296 / 0.540337 (-0.113041) | 0.513227 / 1.386936 (-0.873709) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.006215 / 0.011353 (-0.005138) | 0.003834 / 0.011008 (-0.007174) | 0.078001 / 0.038508 (0.039493) | 0.036537 / 0.023109 (0.013428) | 0.369724 / 0.275898 (0.093826) | 0.426761 / 0.323480 (0.103281) | 0.003602 / 0.007986 (-0.004383) | 0.003001 / 0.004328 (-0.001327) | 0.075989 / 0.004250 (0.071739) | 0.048618 / 0.037052 (0.011566) | 0.374296 / 0.258489 (0.115807) | 0.430330 / 0.293841 (0.136489) | 0.028299 / 0.128546 (-0.100247) | 0.008537 / 0.075646 (-0.067109) | 0.083275 / 0.419271 (-0.335997) | 0.043136 / 0.043533 (-0.000397) | 0.359072 / 0.255139 (0.103933) | 0.387391 / 0.283200 (0.104192) | 0.021202 / 0.141683 (-0.120481) | 1.520832 / 1.452155 (0.068677) | 1.567030 / 1.492716 (0.074313) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.230944 / 0.018006 (0.212938) | 0.422159 / 0.000490 (0.421669) | 0.003447 / 0.000200 (0.003247) | 0.000125 / 0.000054 (0.000071) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.025442 / 0.037411 (-0.011969) | 0.103944 / 0.014526 (0.089418) | 0.110577 / 0.176557 (-0.065979) | 0.161393 / 0.737135 (-0.575743) | 0.113482 / 0.296338 (-0.182857) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.485765 / 0.215209 (0.270556) | 4.845737 / 2.077655 (2.768083) | 2.556732 / 1.504120 (1.052612) | 2.348638 / 1.541195 (0.807443) | 2.379289 / 1.468490 (0.910799) | 0.561261 / 4.584777 (-4.023516) | 3.482468 / 3.745712 (-0.263244) | 3.061319 / 5.269862 (-2.208543) | 1.483938 / 4.565676 (-3.081738) | 0.067584 / 0.424275 (-0.356691) | 0.011333 / 0.007607 (0.003726) | 0.594342 / 0.226044 (0.368297) | 5.935477 / 2.268929 (3.666548) | 3.025029 / 55.444624 (-52.419595) | 2.687032 / 6.876477 (-4.189445) | 2.752470 / 2.142072 (0.610398) | 0.674470 / 4.805227 (-4.130757) | 0.136777 / 6.500664 (-6.363887) | 0.068335 / 0.075469 (-0.007134) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.336456 / 1.841788 (-0.505332) | 14.376007 / 8.074308 (6.301699) | 14.171375 / 10.191392 (3.979983) | 0.159620 / 0.680424 (-0.520804) | 0.016685 / 0.534201 (-0.517516) | 0.364344 / 0.579283 (-0.214939) | 0.395358 / 0.434364 (-0.039006) | 0.424876 / 0.540337 (-0.115461) | 0.513267 / 1.386936 (-0.873669) |

</details>
</details>

![](https://cml.dev/watermark.png#6ed837325cb539a5deb99129e5ad181d0269e050 "CML watermark")