Title: [docs] Redirects, migrated from nginx
Author: julien-c (MEMBER)
State: closed
Created: 2023-05-12T19:19:27Z


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@mishig25 note that it's not exactly the same behavior as in nginx as here it interacts a bit with the `version` and the `language`

Should be close enough, though.

<details>
<summary>Show benchmarks</summary>

PyArrow==8.0.0

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007212 / 0.011353 (-0.004141) | 0.005125 / 0.011008 (-0.005883) | 0.098460 / 0.038508 (0.059952) | 0.034040 / 0.023109 (0.010931) | 0.320203 / 0.275898 (0.044305) | 0.357787 / 0.323480 (0.034307) | 0.006000 / 0.007986 (-0.001986) | 0.005644 / 0.004328 (0.001316) | 0.072654 / 0.004250 (0.068403) | 0.049393 / 0.037052 (0.012341) | 0.345686 / 0.258489 (0.087196) | 0.362345 / 0.293841 (0.068504) | 0.036597 / 0.128546 (-0.091949) | 0.012303 / 0.075646 (-0.063343) | 0.334374 / 0.419271 (-0.084897) | 0.062010 / 0.043533 (0.018477) | 0.312547 / 0.255139 (0.057408) | 0.336021 / 0.283200 (0.052821) | 0.112304 / 0.141683 (-0.029378) | 1.446706 / 1.452155 (-0.005449) | 1.523256 / 1.492716 (0.030540) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.217658 / 0.018006 (0.199652) | 0.449208 / 0.000490 (0.448718) | 0.002878 / 0.000200 (0.002679) | 0.000091 / 0.000054 (0.000036) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.025735 / 0.037411 (-0.011676) | 0.105876 / 0.014526 (0.091350) | 0.114887 / 0.176557 (-0.061669) | 0.170984 / 0.737135 (-0.566152) | 0.121420 / 0.296338 (-0.174918) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.419670 / 0.215209 (0.204461) | 4.189453 / 2.077655 (2.111798) | 1.938236 / 1.504120 (0.434116) | 1.769747 / 1.541195 (0.228553) | 1.910919 / 1.468490 (0.442429) | 0.705046 / 4.584777 (-3.879730) | 3.783774 / 3.745712 (0.038062) | 2.096504 / 5.269862 (-3.173358) | 1.339265 / 4.565676 (-3.226412) | 0.086670 / 0.424275 (-0.337605) | 0.012243 / 0.007607 (0.004636) | 0.524701 / 0.226044 (0.298657) | 5.240689 / 2.268929 (2.971760) | 2.473622 / 55.444624 (-52.971003) | 2.170568 / 6.876477 (-4.705909) | 2.289653 / 2.142072 (0.147581) | 0.848913 / 4.805227 (-3.956314) | 0.168332 / 6.500664 (-6.332332) | 0.064926 / 0.075469 (-0.010543) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.193614 / 1.841788 (-0.648173) | 14.920403 / 8.074308 (6.846095) | 14.475059 / 10.191392 (4.283667) | 0.164458 / 0.680424 (-0.515966) | 0.017613 / 0.534201 (-0.516588) | 0.426311 / 0.579283 (-0.152972) | 0.431478 / 0.434364 (-0.002886) | 0.520280 / 0.540337 (-0.020057) | 0.627738 / 1.386936 (-0.759198) |

</details>
PyArrow==latest

<details>
<summary>Show updated benchmarks!</summary>

### Benchmark: benchmark_array_xd.json

| metric | read_batch_formatted_as_numpy after write_array2d | read_batch_formatted_as_numpy after write_flattened_sequence | read_batch_formatted_as_numpy after write_nested_sequence | read_batch_unformated after write_array2d | read_batch_unformated after write_flattened_sequence | read_batch_unformated after write_nested_sequence | read_col_formatted_as_numpy after write_array2d | read_col_formatted_as_numpy after write_flattened_sequence | read_col_formatted_as_numpy after write_nested_sequence | read_col_unformated after write_array2d | read_col_unformated after write_flattened_sequence | read_col_unformated after write_nested_sequence | read_formatted_as_numpy after write_array2d | read_formatted_as_numpy after write_flattened_sequence | read_formatted_as_numpy after write_nested_sequence | read_unformated after write_array2d | read_unformated after write_flattened_sequence | read_unformated after write_nested_sequence | write_array2d | write_flattened_sequence | write_nested_sequence |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.007458 / 0.011353 (-0.003895) | 0.005363 / 0.011008 (-0.005645) | 0.076713 / 0.038508 (0.038205) | 0.034189 / 0.023109 (0.011079) | 0.359938 / 0.275898 (0.084040) | 0.395532 / 0.323480 (0.072052) | 0.005977 / 0.007986 (-0.002008) | 0.004263 / 0.004328 (-0.000065) | 0.075971 / 0.004250 (0.071721) | 0.051924 / 0.037052 (0.014871) | 0.362818 / 0.258489 (0.104329) | 0.409897 / 0.293841 (0.116056) | 0.035494 / 0.128546 (-0.093053) | 0.012399 / 0.075646 (-0.063247) | 0.088335 / 0.419271 (-0.330937) | 0.047968 / 0.043533 (0.004435) | 0.355744 / 0.255139 (0.100606) | 0.376339 / 0.283200 (0.093139) | 0.104542 / 0.141683 (-0.037141) | 1.464826 / 1.452155 (0.012672) | 1.600665 / 1.492716 (0.107948) |

### Benchmark: benchmark_getitem\_100B.json

| metric | get_batch_of\_1024\_random_rows | get_batch_of\_1024\_rows | get_first_row | get_last_row |
|--------|---|---|---|---|
| new / old (diff) | 0.220841 / 0.018006 (0.202834) | 0.446444 / 0.000490 (0.445954) | 0.000392 / 0.000200 (0.000192) | 0.000057 / 0.000054 (0.000003) |

### Benchmark: benchmark_indices_mapping.json

| metric | select | shard | shuffle | sort | train_test_split |
|--------|---|---|---|---|---|
| new / old (diff) | 0.029402 / 0.037411 (-0.008009) | 0.116511 / 0.014526 (0.101986) | 0.122959 / 0.176557 (-0.053598) | 0.171674 / 0.737135 (-0.565462) | 0.129871 / 0.296338 (-0.166468) |

### Benchmark: benchmark_iterating.json

| metric | read 5000 | read 50000 | read_batch 50000 10 | read_batch 50000 100 | read_batch 50000 1000 | read_formatted numpy 5000 | read_formatted pandas 5000 | read_formatted tensorflow 5000 | read_formatted torch 5000 | read_formatted_batch numpy 5000 10 | read_formatted_batch numpy 5000 1000 | shuffled read 5000 | shuffled read 50000 | shuffled read_batch 50000 10 | shuffled read_batch 50000 100 | shuffled read_batch 50000 1000 | shuffled read_formatted numpy 5000 | shuffled read_formatted_batch numpy 5000 10 | shuffled read_formatted_batch numpy 5000 1000 |
|--------|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 0.450411 / 0.215209 (0.235202) | 4.471859 / 2.077655 (2.394205) | 2.229439 / 1.504120 (0.725319) | 2.053308 / 1.541195 (0.512114) | 2.142476 / 1.468490 (0.673986) | 0.708299 / 4.584777 (-3.876478) | 3.797830 / 3.745712 (0.052118) | 2.142509 / 5.269862 (-3.127352) | 1.333357 / 4.565676 (-3.232320) | 0.086837 / 0.424275 (-0.337439) | 0.012102 / 0.007607 (0.004495) | 0.548428 / 0.226044 (0.322384) | 5.490611 / 2.268929 (3.221682) | 2.713882 / 55.444624 (-52.730742) | 2.399638 / 6.876477 (-4.476839) | 2.481549 / 2.142072 (0.339477) | 0.839812 / 4.805227 (-3.965415) | 0.168890 / 6.500664 (-6.331774) | 0.065564 / 0.075469 (-0.009906) |

### Benchmark: benchmark_map_filter.json

| metric | filter | map fast-tokenizer batched | map identity | map identity batched | map no-op batched | map no-op batched numpy | map no-op batched pandas | map no-op batched pytorch | map no-op batched tensorflow |
|--------|---|---|---|---|---|---|---|---|---|
| new / old (diff) | 1.275507 / 1.841788 (-0.566281) | 14.896343 / 8.074308 (6.822035) | 13.159701 / 10.191392 (2.968309) | 0.172065 / 0.680424 (-0.508359) | 0.017507 / 0.534201 (-0.516694) | 0.420031 / 0.579283 (-0.159252) | 0.438835 / 0.434364 (0.004471) | 0.490597 / 0.540337 (-0.049741) | 0.583952 / 1.386936 (-0.802984) |

</details>
</details>

![](https://cml.dev/watermark.png#48c9755d0ae9abe4c4d6cd8c1ce76eff849f0e5c "CML watermark")