Transpose images with EXIF Orientation tag
State: closed | Author: mariosasko (COLLABORATOR)

Closes https://github.com/huggingface/datasets/issues/6252

Comments: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6739). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005295 / 0.011353 (-0.006058) | 0.003402 / 0.011008 (-0.007606) | 0.062860 / 0.038508 (0.024352) | 0.029627 / 0.023109 (0.006518) | 0.238359 / 0.275898 (-0.037539) | 0.262940 / 0.323480 (-0.060540) | 0.003077 / 0.007986 (-0.004909) | 0.002676 / 0.004328 (-0.001652) | 0.048731 / 0.004250 (0.044480) | 0.043989 / 0.037052 (0.006936) | 0.255702 / 0.258489 (-0.002787) | 0.282667 / 0.293841 (-0.011174) | 0.028019 / 0.128546 (-0.100527) | 0.010195 / 0.075646 (-0.065451) | 0.205472 / 0.419271 (-0.213800) | 0.036551 / 0.043533 (-0.006982) | 0.243282 / 0.255139 (-0.011857) | 0.261925 / 0.283200 (-0.021274) | 0.020506 / 0.141683 (-0.121177) | 1.137228 / 1.452155 (-0.314927) | 1.183935 / 1.492716 (-0.308782) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.100290 / 0.018006 (0.082284) | 0.316279 / 0.000490 (0.315790) | 0.000239 / 0.000200 (0.000039) | 0.000043 / 0.000054 (-0.000011) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.017979 / 0.037411 (-0.019432) | 0.061616 / 0.014526 (0.047090) | 0.072989 / 0.176557 (-0.103568) | 0.118667 / 0.737135 (-0.618468) | 0.074266 / 0.296338 (-0.222072) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.287971 / 0.215209 (0.072762) | 2.845235 / 2.077655 (0.767581) | 1.501983 / 1.504120 (-0.002137) | 1.389824 / 1.541195 (-0.151370) | 1.415616 / 1.468490 (-0.052874) | 0.568727 / 4.584777 (-4.016050) | 2.368330 / 3.745712 (-1.377382) | 2.844329 / 5.269862 (-2.425532) | 1.809038 / 4.565676 (-2.756639) | 0.063699 / 0.424275 (-0.360576) | 0.004972 / 0.007607 (-0.002635) | 0.340092 / 0.226044 (0.114048) | 3.369146 / 2.268929 (1.100217) | 1.863423 / 55.444624 (-53.581201) | 1.608334 / 6.876477 (-5.268142) | 1.624479 / 2.142072 (-0.517594) | 0.632439 / 4.805227 (-4.172788) | 0.116862 / 6.500664 (-6.383802) | 0.042558 / 0.075469 (-0.032911) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.967922 / 1.841788 (-0.873866) | 11.730612 / 8.074308 (3.656304) | 9.321333 / 10.191392 (-0.870059) | 0.142604 / 0.680424 (-0.537819) | 0.013934 / 0.534201 (-0.520267) | 0.285992 / 0.579283 (-0.293292) | 0.267639 / 0.434364 (-0.166724) | 0.324972 / 0.540337 (-0.215365) | 0.427077 / 1.386936 (-0.959859) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.005806 / 0.011353 (-0.005547) | 0.003771 / 0.011008 (-0.007237) | 0.049542 / 0.038508 (0.011034) | 0.030182 / 0.023109 (0.007073) | 0.303923 / 0.275898 (0.028025) | 0.325623 / 0.323480 (0.002143) | 0.004327 / 0.007986 (-0.003659) | 0.002818 / 0.004328 (-0.001510) | 0.048237 / 0.004250 (0.043987) | 0.047490 / 0.037052 (0.010437) | 0.316556 / 0.258489 (0.058067) | 0.348352 / 0.293841 (0.054512) | 0.029444 / 0.128546 (-0.099102) | 0.010544 / 0.075646 (-0.065102) | 0.057382 / 0.419271 (-0.361890) | 0.056210 / 0.043533 (0.012677) | 0.305495 / 0.255139 (0.050356) | 0.321570 / 0.283200 (0.038370) | 0.019546 / 0.141683 (-0.122137) | 1.141732 / 1.452155 (-0.310423) | 1.223626 / 1.492716 (-0.269091) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.093864 / 0.018006 (0.075858) | 0.309715 / 0.000490 (0.309226) | 0.000217 / 0.000200 (0.000017) | 0.000053 / 0.000054 (-0.000002) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.022047 / 0.037411 (-0.015364) | 0.074885 / 0.014526 (0.060359) | 0.088440 / 0.176557 (-0.088117) | 0.127033 / 0.737135 (-0.610103) | 0.089048 / 0.296338 (-0.207290) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.292624 / 0.215209 (0.077415) | 2.877592 / 2.077655 (0.799937) | 1.607036 / 1.504120 (0.102916) | 1.487819 / 1.541195 (-0.053376) | 1.517318 / 1.468490 (0.048828) | 0.553321 / 4.584777 (-4.031456) | 2.415577 / 3.745712 (-1.330135) | 2.691411 / 5.269862 (-2.578450) | 1.743395 / 4.565676 (-2.822282) | 0.062187 / 0.424275 (-0.362088) | 0.005073 / 0.007607 (-0.002534) | 0.342907 / 0.226044 (0.116863) | 3.402054 / 2.268929 (1.133126) | 1.979481 / 55.444624 (-53.465143) | 1.702885 / 6.876477 (-5.173592) | 1.868279 / 2.142072 (-0.273794) | 0.640095 / 4.805227 (-4.165132) | 0.117138 / 6.500664 (-6.383526) | 0.042197 / 0.075469 (-0.033272) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.007495 / 1.841788 (-0.834292) | 12.037309 / 8.074308 (3.963001) | 10.227670 / 10.191392 (0.036278) | 0.149533 / 0.680424 (-0.530891) | 0.015282 / 0.534201 (-0.518919) | 0.287357 / 0.579283 (-0.291926) | 0.285109 / 0.434364 (-0.149255) | 0.324027 / 0.540337 (-0.216311) | 0.442482 / 1.386936 (-0.944454) |

</details>
</details>

![](https://cml.dev/watermark.png#19b40860acf3b3ba8db727fcf3b1b99ebb8d7e33 "CML watermark")