DOC: Update redirects
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: https://github.com/pandas-dev/pandas/pull/55626#issuecomment-1855410365

Opening this up as a draft. I've generated the new paths based on what was in the redirects.csv, but I'd like to find some way to test this.

<details>
<summary>Script for testing redirects</summary>

```
from pathlib import Path
import os

# Download each of the versions of docs listed below as zip files, unzip into a
# directory with subdirectories 1.1, 1.2, etc.
base_old = Path(f"/home/richard/Downloads/docs/")
base_pr = Path("/home/richard/dev/pandas/doc/build/html")

# We'll make sure redirects don't exist for any methods that have been removed
removed_methods = [".pad.", ".mad.", ".backfill.", ".tshift.", ".sem."]

def strip_str(haystack, needle, before):
    idx = haystack.find(needle)
    assert idx >= 0
    if before:
        result = haystack[idx + len(needle):]
    else:
        result = haystack[:idx]
    return result

for version in ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "2.0", "2.1", "2.2"]:
    base_old_api = base_old / version / "reference" / "api"
    base_pr_api = base_pr / "reference" / "api"

    print('Version:', version)
    for e in base_old_api.rglob("*.html"):
        relative_path = e.relative_to(base_old_api)
        
        if not str(relative_path).startswith("pandas.core"):
            # Only looking at pandas.core redirects
            continue
        
        target_exists = os.path.exists(base_pr_api / relative_path)
        
        if any(e in str(relative_path) for e in removed_methods):
            if target_exists:
                print(f'  {relative_path}: Removed method has redirect')
            continue
            
        if not target_exists:
            print(f'  {relative_path}: Does not exist')
            continue

        contents = open(base_pr_api / relative_path).read()
        
        if "The page has been moved to" not in contents:
            # There is no redirect
            continue

        # Get the redirect path
        contents = strip_str(contents, "0;URL=", before=True)
        path = Path(strip_str(contents, '"/>\n', before=False))
        
        if not os.path.exists(base_pr_api / path):
            print(f'  {relative_path}: Redirect does not exist')
            continue

        contents = open(base_pr_api / path).read()
        if "The page has been moved to" in contents:
            print(f'  {relative_path}: Redirect is another redirect')
            
    print()
    print('-' * 20)
    print()
```

</details>

Labels:
name: Bug
name: Docs

Comments:
@datapythonista - to test the redirects, I'm planning to write a crawler that gets the API urls from previous versions of pandas and tries them out on this PR. I just want to make sure something that tests the redirects doesn't exist so I'm not reinventing the wheel.
/preview
No preview found for PR #56511. Did the docs build complete?
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/56511/
> @datapythonista - to test the redirects, I'm planning to write a crawler that gets the API urls from previous versions of pandas and tries them out on this PR. I just want to make sure something that tests the redirects doesn't exist so I'm not reinventing the wheel.

I don't think we implemented any automated testing for the redirects when we added them (this is the commit: https://github.com/pandas-dev/pandas/commit/6a745d826d9d5f5f46e208b0dad4d4ce0524defe). And I'm not aware of any test added later either.

Your idea sounds good. I guess we could also analyze the logs of our server, and see for which pages we return a 404 (probably not in an automated way, but to make sure we don't have anything broken as of today). I guess there will be a lot of noise from bots trying paths if we do that, but we can have a look if you want.
Thanks @datapythonista - I got a script for checking the redirects added to the OP. I'll post results here when this is ready.
I reworked the script in the OP to check the redirects to work just using the filesystem. It checks for

 - URLs in older versions that don't resolve in the PR docs
 - Redirects in the PR docs that go to URLs that don't exist
 - Redirects in the PR docs that redirect again
 - Removed methods that still have redirects

Right now I'm just limiting the output to `pandas.core` - the script in the OP doesn't flag any issues. There are other redirect issues that I plan to tackle in a separate PR.
@datapythonista - With #55632, I think it's likely we'll be reverting #55626 and this will be closed in that case. If that does happen, I plan on putting up a PR fixing the redirects separately.