Repository: mkdocs/mkdocs

Issue:
relative url's can cause bad links
I am deploying to s3, and then have a hidden 404 error mkdocs page - when the users enters a long (incorrect url), then the error page is rendered using that url - thus all the menu links are broken.



Discussion:
FWIW, relative URLs have also caused a ton of issues with implementing #222.

I'm not really a fan of relative URLs, it is only really useful for people that view the docs in static mode and I'm not sure how popular a use case that is.

The only way I can think of solving this would be by having a config option - but I'm somewhat wary of this.

@SvenDowideit I assume you already use this approach in your forked version?

Is there an example for this?

I use relative interlinking all the way through the REST framework docs. (Eg see links at the bottom of this document... https://raw.githubusercontent.com/tomchristie/django-rest-framework/master/docs/topics/rest-hypermedia-hateoas.md )

What doesn't work?

Consider for example the search index, it is generated once for all plages. However, if you display the  results in a modal the links need to be relative (or they wont work in static mode). To make this work in #222 I had to do this bit of gross magic, I'm trying to come up with a better solution.

https://github.com/d0ugal/mkdocs/blob/search/mkdocs/themes/mkdocs/js/tipuesearch/tipuesearch_content.js
https://github.com/d0ugal/mkdocs/blob/search/mkdocs/themes/mkdocs/base.html#L65

> it is only really useful for people that view the docs in static mode

Maybe I'm just misunderstanding, but being able to click between docs in my editor is pretty neat.

> Consider for example a search index

Are we just talking about relative links in the theme then? No problem from me with not supporting that.

Updated my comment as I rushed it on my phone. So yeah, I think for me the issue is supporting it at the theme level for me.

I think with @SvenDowideit's example you have a similar issue when the 404 could be rendered at `/invalid-path/` or `/invalid/path/` but I may be missunderstanding that.

It is worth noting that we don't have this issue with the DRF theme, but I'm not sure if that is specific to that theme. The menu works fine: http://www.django-rest-framework.org/testing/testing/testing/

I'm still surprised tho - I've never used anything _except_ relative links with MkDocs,
and put a stack of time into making them work initially.
Be helpful to see a broken example case.

I'm further confused because this report seems to be competely at odds with https://github.com/tomchristie/mkdocs/issues/192

If you can't use relative _or_ absolute URLs how are the rest of you folks getting by? :)

Hah, so I'd always use relative for document linking. I don't think absolute URLs work (although, with the strict mode disabled they may happen to work).

I chimed in as I've had issues with #222 but that is a specific theme case so I may have just confused things. Maybe Sven can clarify the original issue.

> we don't have this issue with the DRF theme

Yeah just noticed that the nav on the 404 page there is hardcoded in the theme :p

http://docs.docker.com/ is an example - we also publish a version of the same docs to http://docs.docker.com/v1.2/ etc

_and_ at the same time, there are webserver based redirects that may display a page, but come from a totally different place in the url structure.

so for (a madeup) eg http://yoursite/one/two/three/index.html may also be shown when the user requests http://yoursite/other/place.index.html

so the relative link is going to be at the wrong place, and the wrong level.

I've obviously made a change in my fork for this... :)

mmm, iirc, its also very relevant if you use the same 'search' page for all 404's, which we do.

@d0ugal What exactly do you mean with

> I'm not really a fan of relative URLs, it is only really useful for people that view the docs in **static mode** and I'm not sure how popular a use case that is.

Static mode?

@ePirat We possibly need a better name for this, basically when you browse the documentation in your browser on the local filesystem. So links like "/" don't work as that isn't the root of the documentation. So, for example `mkdocs build && firefox site/index.html`.

The report in #1035 brought me to this issue. This is the first time I've taken a close look at it. It appears from my reading that there is some confusion about this issue. As I see it, there are two separate (unresolved) issues with relative URLs. This one and the one in #192. The issue in #192 applies only to URL munging within Markdown and is not related to this issue.

In this issue the OP encountered the same problem as encountered in #1035 (which I closed as a duplicate, but it clearly defines the user's problem and helps clear up the issue). Namely, a template renders a relative URL which is hard coded into the page. Normally, not a problem as each page is always only ever at one location. However the problem exists when the same page can exist in multiple different locations. For example, error pages are served for any URL in which an error occurs. The location of any other page relative to the current URL will be different for most cases--especially now that we allow multi-level nesting. 

Of course, in dynamic servers, the error page is often regenerated each time (from a template), in which case the current URL is accounted for when rendering the template, avoiding this problem. But we don't have this option as our error page is rendered without knowledge of the requested URL. Therefore, we need nav items, images, js, css, etc, to point to the proper location regardless of the location of the current page. 

The easy way to do that is via absolute URLs. But, of course, we cannot assume that the server root is the same as the MkDocs root. For example, consider documentation hosted on http://pythonhosted.org as explained in [this comment](https://github.com/mkdocs/mkdocs/issues/192#issuecomment-75322787). The point is, we can't just assume `/` is the documentation root on the server. It may be `/projectname/` or `/username/projectname/` or some other thing.

Previous comments have mentioned adding some sort of new config setting. That should not be necessary. We already have the [`site_url`](http://www.mkdocs.org/user-guide/configuration/#site_url) setting. If a project's documentation root is at `http://example.com/projectname/`, then that is going to be the value assigned to `site_url`. We can extrapolate from that that any absolute URL should have `/projectname/` prepended to it to be a proper absolute URL.

Of course, there is the issue of the dev server (livereload or not) in which the server root is always the same as MkDocs root. But that should not be a problem as we also have the [`dev_addr`](http://www.mkdocs.org/user-guide/configuration/#dev_addr) setting. If that is set, just extrapolate the root from that rather than `site_url`. Coincidentally, I find it strange that this config setting exists. Yes, it should exist as an option on the `serve` command, but shouldn't the value just be passed into the config as an override of the `site_url` setting during development? But I digress.

So my proposal would be that the [`base_url`](http://www.mkdocs.org/user-guide/custom-themes/#base_url) template variable be the absolute path from the server root to the MkDocs root. To illustrate, here's some examples of what the `base_url` template variable should contain depending on the value of the `site_url` config setting:

| `site_url` | `base_url` |
| --- | --- |
| `http://example.com/` | `/` |
| `http://example.com` | `/` |
| `http://example.com/foo/` | `/foo/` |
| `http://example.com/foo` | `/foo/` |
| `http://example.com/foo/bar/baz/` | `/foo/bar/baz/` |
| `http://127.0.0.1:8000` (from `dev_addr`) | `/` |
| `http://0.0.0.0:80` (from `dev_addr`) | `/` |

For completeness, I should point out that browsing files via the local file system (`file://`) will break with absolute URLs every time. Of course, currently it is broken anyway (search won't work), but if we want to support that, then absolute URLs are out. In that case, I would propose using absolute URLs on error pages only (as described in my previous comment). Error pages are server specific and don't get returned by the local file system on an error, so it should be okay if they are broken in that instance. The trick is how to address the nav on error pages. If the nav contains relative URLs, those URLs still need to be absolute on the error page. Perhaps the error page should only contain a link to the homepage, not the entire nav.

Warning, the following is a complicated explanation of how the code works. Sorry, but explaining it is part of how I work out solutions, which you'll find at the end.

I think the key to fixing this issue is to understand how the URLContext works. The 'short' explanation is that in `mkdocs.commands.build.build_pages` we have the loop: `for page in site_navigation.walk_pages():` The key is in the `walk_pages` method. That method sets a page as "active" and yields it. `build_pages` then takes that page and builds it (renders Markdown and template, then writes to disc). The `url` property of each page is a method which makes a call to the `URLContext` using the "active" page to create URLs which are relative to that "active" page (the entire loop through the nav in the template happens with this one page set as "active"). After the page is built, the loop (in `build_pages`) continues, asking `walk_pages` for the next page. `walk_pages` then deactivates the current page and activates the next page before yielding it. The entire thing is dependent on the global "active" page and URLContext and the fact that we are in a synchronous system (it would never work in an asynchronous environment). That is how each page is generated with a complete nav where all the URLs are relative to that page with absolutely no consideration for the location of the current page in the template. Its a clever piece of code which removes the need for the template to care about relative URLs.

However, there are pages which do not get this treatment. Specifically, any page which is not listed in the `pages` config setting. As it happens, all those pages are template based pages, such as error pages, `sitemap.xml`, `search.html` and anything in the `extra_templates` config setting. The key difference is that for these pages there is no "active" page, in which case the URLContext defaults to assuming an "active" URL of `/`. Of course, `search.html` is at the site root (as it `sitemap.xml`) so the relative paths in the nav are correct. However, `extra_templates` could be nested at any location (I assume this is an unreported bug as its a little used feature). Presumably, setting the "active" page in the URLContext to the path of the current template would build a proper nav for each page in `extra_templates` (untested, but I think it should work). 

But for error pages it's a little more tricky. As discussed previously, error pages could be served at any URL nested at any level by the server. There it no correct "active" page to make nav items relative to. Therefore, I propose that if there is no "active" page, rather than the default being to generate a URL for a page relative to `/`, instead generate an absolute URL with the base extrapolated from `site_url`. This absolute URL should then be returned by the `mkdocs.nav.Page.url` method when the error template builds it's nav.


Resolving patch:
diff --git a/mkdocs/commands/build.py b/mkdocs/commands/build.py
--- a/mkdocs/commands/build.py
+++ b/mkdocs/commands/build.py
@@ -297,7 +297,16 @@ def resolve(self, key):
     env.filters['tojson'] = filters.tojson
     search_index = search.SearchIndex()
 
+    # Force absolute URLs in the nav of error pages and account for the
+    # possability that the docs root might be different than the server root.
+    # See https://github.com/mkdocs/mkdocs/issues/77
+    site_navigation.url_context.force_abs_urls = True
+    default_base = site_navigation.url_context.base_path
+    site_navigation.url_context.base_path = utils.urlparse(config['site_url']).path
     build_template('404.html', env, config, site_navigation)
+    # Reset nav behavior to the default
+    site_navigation.url_context.force_abs_urls = False
+    site_navigation.url_context.base_path = default_base
 
     if not build_template('search.html', env, config, site_navigation):
         log.debug("Search is enabled but the theme doesn't contain a "
diff --git a/mkdocs/nav.py b/mkdocs/nav.py
--- a/mkdocs/nav.py
+++ b/mkdocs/nav.py
@@ -82,6 +82,7 @@ class URLContext(object):
 
     def __init__(self):
         self.base_path = '/'
+        self.force_abs_urls = False
 
     def set_current_url(self, current_url):
         self.base_path = os.path.dirname(current_url)
@@ -91,6 +92,10 @@ def make_relative(self, url):
         Given a URL path return it as a relative URL,
         given the context of the current page.
         """
+        if self.force_abs_urls:
+            abs_url = '%s/%s' % (self.base_path.rstrip('/'), utils.path_to_url(url.lstrip('/')))
+            return abs_url.rstrip('/')
+
         suffix = '/' if (url.endswith('/') and len(url) > 1) else ''
         # Workaround for bug on `os.path.relpath()` in Python 2.6
         if self.base_path == '/':


Test patch:
diff --git a/mkdocs/tests/nav_tests.py b/mkdocs/tests/nav_tests.py
--- a/mkdocs/tests/nav_tests.py
+++ b/mkdocs/tests/nav_tests.py
@@ -271,6 +271,12 @@ def test_generate_site_navigation(self):
 
         self.assertEqual([n.title for n in nav_items],
                          ['Home', 'Running', 'Notes', 'License'])
+        self.assertEqual([n.url for n in nav_items], [
+            '.',
+            'api-guide/running/',
+            'about/notes/',
+            'about/sub/license/'
+        ])
         self.assertEqual([p.title for p in pages],
                          ['Home', 'Running', 'Notes', 'License'])
 
@@ -291,9 +297,66 @@ def test_generate_site_navigation_windows(self):
 
         self.assertEqual([n.title for n in nav_items],
                          ['Home', 'Running', 'Notes', 'License'])
+        self.assertEqual([n.url for n in nav_items], [
+            '.',
+            'api-guide/running/',
+            'about/notes/',
+            'about/sub/license/'
+        ])
         self.assertEqual([p.title for p in pages],
                          ['Home', 'Running', 'Notes', 'License'])
 
+    def test_force_abs_urls(self):
+        """
+        Verify force absolute URLs
+        """
+
+        pages = [
+            'index.md',
+            'api-guide/running.md',
+            'about/notes.md',
+            'about/sub/license.md',
+        ]
+
+        url_context = nav.URLContext()
+        url_context.force_abs_urls = True
+        nav_items, pages = nav._generate_site_navigation(pages, url_context)
+
+        self.assertEqual([n.title for n in nav_items],
+                         ['Home', 'Running', 'Notes', 'License'])
+        self.assertEqual([n.url for n in nav_items], [
+            '',
+            '/api-guide/running',
+            '/about/notes',
+            '/about/sub/license'
+        ])
+
+    def test_force_abs_urls_with_base(self):
+        """
+        Verify force absolute URLs
+        """
+
+        pages = [
+            'index.md',
+            'api-guide/running.md',
+            'about/notes.md',
+            'about/sub/license.md',
+        ]
+
+        url_context = nav.URLContext()
+        url_context.force_abs_urls = True
+        url_context.base_path = '/foo/'
+        nav_items, pages = nav._generate_site_navigation(pages, url_context)
+
+        self.assertEqual([n.title for n in nav_items],
+                         ['Home', 'Running', 'Notes', 'License'])
+        self.assertEqual([n.url for n in nav_items], [
+            '/foo',
+            '/foo/api-guide/running',
+            '/foo/about/notes',
+            '/foo/about/sub/license'
+        ])
+
     def test_invalid_pages_config(self):
 
         bad_pages = [