Fix sequence of array support for most dtype

Fixes #5936 
Also, a related fix to #5927 


_The documentation is not available anymore as the PR was closed or merged._

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007220 / 0.011353 (-0.004133) | 0.004558 / 0.011008 (-0.006451) | 0.116647 / 0.038508 (0.078139) | 0.046845 / 0.023109 (0.023736) | 0.352429 / 0.275898 (0.076531) | 0.429739 / 0.323480 (0.106259) | 0.006620 / 0.007986 (-0.001366) | 0.003731 / 0.004328 (-0.000597) | 0.088683 / 0.004250 (0.084433) | 0.070583 / 0.037052 (0.033530) | 0.366699 / 0.258489 (0.108210) | 0.420730 / 0.293841 (0.126889) | 0.037342 / 0.128546 (-0.091204) | 0.010041 / 0.075646 (-0.065605) | 0.383477 / 0.419271 (-0.035795) | 0.060279 / 0.043533 (0.016746) | 0.349988 / 0.255139 (0.094849) | 0.371423 / 0.283200 (0.088224) | 0.026725 / 0.141683 (-0.114958) | 1.736886 / 1.452155 (0.284731) | 1.812874 / 1.492716 (0.320157) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.253256 / 0.018006 (0.235250) | 0.563470 / 0.000490 (0.562980) | 0.010475 / 0.000200 (0.010275) | 0.000164 / 0.000054 (0.000110) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.030518 / 0.037411 (-0.006893) | 0.133324 / 0.014526 (0.118798) | 0.137095 / 0.176557 (-0.039461) | 0.202227 / 0.737135 (-0.534909) | 0.144195 / 0.296338 (-0.152143) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.480870 / 0.215209 (0.265661) | 4.822713 / 2.077655 (2.745058) | 2.124183 / 1.504120 (0.620064) | 1.910733 / 1.541195 (0.369538) | 1.970266 / 1.468490 (0.501776) | 0.624695 / 4.584777 (-3.960082) | 4.459659 / 3.745712 (0.713947) | 2.210123 / 5.269862 (-3.059739) | 1.300520 / 4.565676 (-3.265157) | 0.077096 / 0.424275 (-0.347180) | 0.013333 / 0.007607 (0.005726) | 0.596841 / 0.226044 (0.370797) | 5.917397 / 2.268929 (3.648469) | 2.699397 / 55.444624 (-52.745228) | 2.274833 / 6.876477 (-4.601644) | 2.525376 / 2.142072 (0.383304) | 0.755718 / 4.805227 (-4.049510) | 0.163587 / 6.500664 (-6.337077) | 0.072817 / 0.075469 (-0.002653) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.524306 / 1.841788 (-0.317481) | 18.843312 / 8.074308 (10.769004) | 15.694644 / 10.191392 (5.503252) | 0.177400 / 0.680424 (-0.503024) | 0.020104 / 0.534201 (-0.514097) | 0.466421 / 0.579283 (-0.112862) | 0.537274 / 0.434364 (0.102910) | 0.576920 / 0.540337 (0.036583) | 0.718889 / 1.386936 (-0.668047) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007671 / 0.011353 (-0.003682) | 0.004850 / 0.011008 (-0.006158) | 0.090085 / 0.038508 (0.051576) | 0.052023 / 0.023109 (0.028914) | 0.508575 / 0.275898 (0.232677) | 0.590024 / 0.323480 (0.266544) | 0.004564 / 0.007986 (-0.003422) | 0.005345 / 0.004328 (0.001017) | 0.087904 / 0.004250 (0.083653) | 0.064446 / 0.037052 (0.027394) | 0.525625 / 0.258489 (0.267136) | 0.584307 / 0.293841 (0.290466) | 0.037221 / 0.128546 (-0.091325) | 0.010588 / 0.075646 (-0.065059) | 0.098612 / 0.419271 (-0.320659) | 0.059597 / 0.043533 (0.016064) | 0.488064 / 0.255139 (0.232925) | 0.522330 / 0.283200 (0.239131) | 0.030004 / 0.141683 (-0.111679) | 1.732512 / 1.452155 (0.280357) | 1.809027 / 1.492716 (0.316310) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.218741 / 0.018006 (0.200735) | 0.494946 / 0.000490 (0.494456) | 0.004580 / 0.000200 (0.004380) | 0.000104 / 0.000054 (0.000049) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.034916 / 0.037411 (-0.002495) | 0.133695 / 0.014526 (0.119169) | 0.147964 / 0.176557 (-0.028592) | 0.213210 / 0.737135 (-0.523926) | 0.148850 / 0.296338 (-0.147488) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.508855 / 0.215209 (0.293646) | 5.065088 / 2.077655 (2.987433) | 2.473110 / 1.504120 (0.968990) | 2.259765 / 1.541195 (0.718570) | 2.359189 / 1.468490 (0.890699) | 0.639082 / 4.584777 (-3.945695) | 4.768195 / 3.745712 (1.022482) | 2.253803 / 5.269862 (-3.016059) | 1.442996 / 4.565676 (-3.122680) | 0.078761 / 0.424275 (-0.345514) | 0.013936 / 0.007607 (0.006329) | 0.625977 / 0.226044 (0.399933) | 6.260817 / 2.268929 (3.991888) | 3.149640 / 55.444624 (-52.294985) | 2.753555 / 6.876477 (-4.122921) | 2.831872 / 2.142072 (0.689799) | 0.781294 / 4.805227 (-4.023933) | 0.169109 / 6.500664 (-6.331555) | 0.075810 / 0.075469 (0.000341) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.533282 / 1.841788 (-0.308506) | 19.460579 / 8.074308 (11.386271) | 17.250424 / 10.191392 (7.059032) | 0.193485 / 0.680424 (-0.486939) | 0.020650 / 0.534201 (-0.513551) | 0.472110 / 0.579283 (-0.107173) | 0.532276 / 0.434364 (0.097912) | 0.613152 / 0.540337 (0.072814) | 0.684684 / 1.386936 (-0.702252) |

</details>
</details>

![](https://cml.dev/watermark.png#650a86ee122209d4a8c8e8068c01ebfd3ba553f5 "CML watermark")