Repository: pandas-dev/pandas

Issue:
DOC: Redirect moved pages
Following  up the discussion in https://github.com/pandas-dev/pandas/issues/24451#issuecomment-450862635 and addressing the concerns of merging the PRs of #24499.

The problem is that we've moved many documentation pages (all the API pages, `release.rst`...) we'd like to move many more (almost every page). And we've got two options if we've been discussion so far are:
- Break all the links in search engines, stack overflow, blog posts... linking to the old locations.
- Add more complexity to the build simiar to https://github.com/pandas-dev/pandas/blob/master/doc/source/conf.py#L246 (which IMO is already quite complex, and just works for specific cases)

After thinking more on it, I think we can build a simpler redirect system than what we have, and keep redirects while reducing the complexity of the build.

What I'd do is create a file (e.g. `doc/source/redirects.csv`) containing the urls of the the old and new url (a line could look like `io,user_guide/io` or `generated/pandas.DataFrame.to_parquet,api/reference/DataFrame/pandas.DataFrame.to_parquet`)

Then, in `doc/source/conf.py` we open and iterate the file, and create a simple html in the old location with a link to the new. May be later we can use a nice template and a javascript that automatically redirects, but I'd keep it simple to start with.

If a page is splitted, we can have more than two columns in the csv (e.g. `io,user_guide/io_plain,user_guide/io_binary`), and show the two links in the redirect page.

I'd also move the current API redirect system to this, and we'd simplify `conf.py`. 

@jorisvandenbossche @TomAugspurger does this sound good?


Discussion:
I think that something like

```html
<html>
    <head>
        <meta http-equiv="refresh" content="0;URL={{ destination }}.html />
    </head>
    <body>
    </body>
</html>
```

will automatically redirect users visiting the old page to the destination URL.

Resolving patch:
diff --git a/doc/make.py b/doc/make.py
--- a/doc/make.py
+++ b/doc/make.py
@@ -15,15 +15,18 @@
 import sys
 import os
 import shutil
+import csv
 import subprocess
 import argparse
 import webbrowser
+import docutils
+import docutils.parsers.rst
 
 
 DOC_PATH = os.path.dirname(os.path.abspath(__file__))
 SOURCE_PATH = os.path.join(DOC_PATH, 'source')
 BUILD_PATH = os.path.join(DOC_PATH, 'build')
-BUILD_DIRS = ['doctrees', 'html', 'latex', 'plots', '_static', '_templates']
+REDIRECTS_FILE = os.path.join(DOC_PATH, 'redirects.csv')
 
 
 class DocBuilder:
@@ -139,6 +142,77 @@ def _open_browser(self, single_doc_html):
                            single_doc_html)
         webbrowser.open(url, new=2)
 
+    def _get_page_title(self, page):
+        """
+        Open the rst file `page` and extract its title.
+        """
+        fname = os.path.join(SOURCE_PATH, '{}.rst'.format(page))
+        option_parser = docutils.frontend.OptionParser(
+            components=(docutils.parsers.rst.Parser,))
+        doc = docutils.utils.new_document(
+            '<doc>',
+            option_parser.get_default_values())
+        with open(fname) as f:
+            data = f.read()
+
+        parser = docutils.parsers.rst.Parser()
+        # do not generate any warning when parsing the rst
+        with open(os.devnull, 'a') as f:
+            doc.reporter.stream = f
+            parser.parse(data, doc)
+
+        section = next(node for node in doc.children
+                       if isinstance(node, docutils.nodes.section))
+        title = next(node for node in section.children
+                     if isinstance(node, docutils.nodes.title))
+
+        return title.astext()
+
+    def _add_redirects(self):
+        """
+        Create in the build directory an html file with a redirect,
+        for every row in REDIRECTS_FILE.
+        """
+        html = '''
+        <html>
+            <head>
+                <meta http-equiv="refresh" content="0;URL={url}"/>
+            </head>
+            <body>
+                <p>
+                    The page has been moved to <a href="{url}">{title}</a>
+                </p>
+            </body>
+        <html>
+        '''
+        with open(REDIRECTS_FILE) as mapping_fd:
+            reader = csv.reader(mapping_fd)
+            for row in reader:
+                if not row or row[0].strip().startswith('#'):
+                    continue
+
+                path = os.path.join(BUILD_PATH,
+                                    'html',
+                                    *row[0].split('/')) + '.html'
+
+                try:
+                    title = self._get_page_title(row[1])
+                except Exception:
+                    # the file can be an ipynb and not an rst, or docutils
+                    # may not be able to read the rst because it has some
+                    # sphinx specific stuff
+                    title = 'this page'
+
+                if os.path.exists(path):
+                    raise RuntimeError((
+                        'Redirection would overwrite an existing file: '
+                        '{}').format(path))
+
+                with open(path, 'w') as moved_page_fd:
+                    moved_page_fd.write(
+                        html.format(url='{}.html'.format(row[1]),
+                                    title=title))
+
     def html(self):
         """
         Build HTML documentation.
@@ -150,6 +224,8 @@ def html(self):
 
         if self.single_doc_html is not None:
             self._open_browser(self.single_doc_html)
+        else:
+            self._add_redirects()
         return ret_code
 
     def latex(self, force=False):


Test patch: