DOC: Implement javascript to redirect old whatsnew pages to new ones
State: closed

- [x] closes #23695 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Javascript file to reformat / redirect 'whatsnew' href links in documentation.

Labels:
name: Docs

Comments:
@ismailuddin : Can you rebase your PR on `master` ?  You have a lot of old Circle CI builds in the test modal that we do not use anymore.
Hello @ismailuddin! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`doc/source/conf.py`](https://github.com/ismailuddin/pandas/blob/58157c7e109ccab2bac492ec658c94126596b261/doc/source/conf.py) !


# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=h1) Report
> Merging [#23708](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c55057f79b0cbe47cf73f0fd8141e6c84f1e7370?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23708/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23708      +/-   ##
==========================================
+ Coverage   92.24%   92.25%   +<.01%     
==========================================
  Files         161      161              
  Lines       51336    51383      +47     
==========================================
+ Hits        47357    47404      +47     
  Misses       3979     3979
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.64% <ø> (ø)` | :arrow_up: |
| #single | `42.31% <ø> (-0.03%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `97.87% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.58% <0%> (ø)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.63% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `93.87% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.82% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.34% <0%> (ø)` | :arrow_up: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.4% <0%> (ø)` | :arrow_up: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.55% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `98.47% <0%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.02% <0%> (ø)` | :arrow_up: |
| ... and [4 more](https://codecov.io/gh/pandas-dev/pandas/pull/23708/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=footer). Last update [c55057f...7b7315d](https://codecov.io/gh/pandas-dev/pandas/pull/23708?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Just a bump regarding @gfyoung 's comment about checking version; do I need to check the version number since all new links will be of the new format and hence not match the regex pattern? Or will they not be? See above for discussion.. 
IIUC, you don't need to worry about @gfyoung's concern. We aren't rebuilding the old docs with this JS applied.
Just tried this out locally, didn't quite work. For a URL like

`<root>/whatsnew.html#whatsnew-0190-enhancements-other`

I basically get a 404, since the page `whatsnew.html` no longer exists. So, do we need to drop in a blank page there?
@ismailuddin do you have time to review the mentioned problem, and see if you can fix it.
@datapythonista Ah yes sorry, I thought Tom was asking someone also about this. I'll look into it! 

---

OK, so if I'm understanding this correctly, the broken links are arising due to old links let's say on Google pointing to this `whatsnew.html` page. So even if a blank page is added in, the user still won't really find the page they want? In which case, one solution could be to use nginx's rewrite flag (http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite) as follows:

```
rewrite (http://pandas.pydata.org/pandas-docs/stable/)whatsnew.html#whatsnew-(\d{1})(\d{1,2})(\d{1})-((\w+-*)+) $1whatsnew/v$2.$3.$4.html#$5
```

This approach would only change the /stable/ links which point to the old whatsnew.html pages, which should be the only problem?
didn't check in detail, but what I understand is that even for old version of pandas, we changed the urls in this way:
```
before: http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0190-enhancements-other
after:  http://pandas.pydata.org/pandas-docs/stable/whatsnew/v0.19.0.html#other-enhancements
```
So, we need to add javascript code to `http://pandas.pydata.org/pandas-docs/stable/whatsnew.html`, that checks what's after the `#`, and generates a redirect to the new URL
@datapythonista I believe that's correct. Though note that there isn't a `/whatsnew.html` anymore that's generated by sphinx. http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html. So if want to add JS to that page we'll need to generate an empty HTML page.
Implementing it shouldn't be difficult, we can just add the `whatsnew.html` to `html_additional_pages` in `doc/source/conf.py`, and have the javascript in its content.

We already have the system for the API redirects, but that won't work, even if we make it generic to handle non-api pages, as in this case we have a 1 to N redirect.

But personally, I'd prefer to have a nice 404 page, with a link to the home page, and not add more complexity with redirects. I think we should reorganize several pages (have a directory for the user guide, another for the contributor docs...), and having redirects for those changes will be quite complex.
Do you expect this to be complex? This one is quite limited in scope I
think.

On Thu, Dec 20, 2018 at 4:58 PM Marc Garcia <notifications@github.com>
wrote:

> Implementing it shouldn't be difficult, we can just add the whatsnew.html
> to html_additional_pages in doc/source/conf.py, and have the javascript
> in its content.
>
> We already have the system for the API redirects, but that won't work,
> even if we make it generic to handle non-api pages, as in this case we have
> a 1 to N redirect.
>
> But personally, I'd prefer to have a nice 404 page, with a link to the
> home page, and not add more complexity with redirects. I think we should
> reorganize several pages (have a directory for the user guide, another for
> the contributor docs...), and having redirects for those changes will be
> quite complex.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/23708#issuecomment-449162792>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIh5P5tVt_B5sxi0nZZcBl0W2cnqNks5u7BYIgaJpZM4Yepgt>
> .
>

No, not this particular case, I think redirecting the whatsnew it's quite simple, what I described in my last comment. What I think will be complex enough to not be worth, is redirecting if we reorganize the documentation. So, I was wondering if it was worth adding redirects every time we move pages, or just improve the 404, and let the users find the new content.
Ahh yes I see what you mean, and I think I agree.

On Fri, Dec 21, 2018 at 9:35 AM Marc Garcia <notifications@github.com>
wrote:

> No, not this particular case, I think redirecting the whatsnew it's quite
> simple, what I described in my last comment. What I think will be complex
> enough to not be worth, is redirecting if we reorganize the documentation.
> So, I was wondering if it was worth adding redirects every time we move
> pages, or just improve the 404, and let the users find the new content.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/23708#issuecomment-449419602>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIn0YY9TwniDkYQ8qPqci4w1TRmKBks5u7P_PgaJpZM4Yepgt>
> .
>

Closing this, as it's not solving the problem.

@ismailuddin , as discussed, not sure if it's worth to add the redirects. But feel free to open a new PR (or reopen this), if you want to implement the approach described in https://github.com/pandas-dev/pandas/pull/23708#issuecomment-449162792

Thanks!