DOC: Implementing redirect system, and adding user_guide redirects
State: closed

- [X] closes #24705
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=h1) Report
> Merging [#24715](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d46d2a5ea900c59439dff58942e79ea8b39e918d?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24715/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24715   +/-   ##
=======================================
  Coverage   92.38%   92.38%           
=======================================
  Files         166      166           
  Lines       52321    52321           
=======================================
  Hits        48337    48337           
  Misses       3984     3984
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.8% <ø> (ø)` | :arrow_up: |
| #single | `43.06% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=footer). Last update [d46d2a5...c5d3e9e](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=h1) Report
> Merging [#24715](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4b937ff59127bf0edec8e07a667fa222bf2e3b4e?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24715/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24715   +/-   ##
=======================================
  Coverage   92.39%   92.39%           
=======================================
  Files         166      166           
  Lines       52413    52413           
=======================================
  Hits        48425    48425           
  Misses       3988     3988
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.81% <ø> (ø)` | :arrow_up: |
| #single | `42.89% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=footer). Last update [4b937ff...0e6770b](https://codecov.io/gh/pandas-dev/pandas/pull/24715?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Also wondering: is there a technical reason you went with a custom approach in make.py instead of using the current logic in conf.py using sphinx's `additional_html_pages` (but then generating the old/new links from the file instead of the current list in the file) ?
I prefer to remove the previous redirects in a follow up, after this is working. I think it should be feasible. This new system won't introspect classes to redirect all the methods at runtime, which will make the list of redirects longer, but I still think it's better this way.

`additional_html_pages` could be used too, but I like this approach. I think it's better keeping the system separate from sphinx (sphinx is already too complex to mix other things there). And the context for all the additional html pages is the same, which means that we need to add all the redirects to that context, and not just pass the required data. After adding the title of the new location, that would be two somehow big dictionaries, and some trickiness in the templates.

Regarding the absolute redirects, the current/previous system only redirects to the same directory. In out case that's not the case. And our docs live in a subdirectory (`/pandas-docs/stable/`), which means that we can't use absolute url without domain (`/user_guide/missing_data/`). This leaves us with two options, using relative urls from the old to the new (e.g. `../../reference/api`), or using absolute urls. I think the relative are too complex. And knowing that there are plans to use `pandas.io`, I think an absolute url is better, as we could redirect from the old website to the new, if that make sense (not sure if it does, there are different options there).

May be we could not always use the stable documentation, and redirect the master documentation to the master. But I don't think those redirects are actually useful in master, and I wouldn't make things more complex for that. If having the master docs redirecting to stable causes confusion, I'd simply add an option `--no-redirects` (or the opposite) to `make.py`, and not generate them for master.




I'm not sure I follow the relative vs. absolute link discussion. If I go to `<...>/advanced.html`, then why doesn't using `./user_guide/advanced` for the redirect? Then this will work for the dev docs as well (useful for testing).

> And knowing that there are plans to use pandas.io, I think an absolute url is better, as we could redirect from the old website to the new, if that make sense

Ideally that'll be done at the DNS layer, before the webserver actually sees the request.

About redirecting the user_guide, you're right, is as simple as removing the url. But for the api it'd be `../api/generated`, and I was thinking on possibly even more complex redirects, but on a second thought, I don't think they are likely to happen.

I made the redirects relative, I guess we're all happy with it.
#24677 needs to be merged first. The redirects would overwrite the documentation pages if we merge the redirects first.

I added a check that should fail the CI here (the docs build, which is optional actually), so we're in the safe side, and that never happens.
I finally added here the redirects for the 3 restructing PRs (getting started, user guide, development).

I think that'll make things easier, I'll merge those 3 PRs first, and then this one, when everything is green.
@TomAugspurger I merged the 3 PRs with the restructuring, and added all the redirects here. Restarted the CI, to make sure the doc build finish on green (it doesn't if the redirects overwrite any file).

Feel free to merge on green, if you're happy.
Thanks! https://travis-ci.org/pandas-dev/pandas/jobs/483537121 is the builder from master.
That build finished. So visiting, e.g. http://pandas-docs.github.io/pandas-docs-travis/contributing.html will redirect to http://pandas-docs.github.io/pandas-docs-travis/development/contributing.html