Repository: mkdocs/mkdocs

Issue:
Wrong paths in 404 page on mkdocs.org
**Good:**

https://www.mkdocs.org/foo

![www mkdocs org_foo](https://user-images.githubusercontent.com/103362/60422299-0598d500-9bec-11e9-9d40-ea82e337d58a.png)

**Bad:**

* https://www.mkdocs.org/foo/
* https://www.mkdocs.org/user-guide/foo
* https://www.mkdocs.org/user-guide/foo/

![www mkdocs org_foo_](https://user-images.githubusercontent.com/103362/60422317-0fbad380-9bec-11e9-8b96-27673ee09463.png)

The problem are the paths because it points to the wrong target. Examples:

```html
<link href="css/bootstrap-custom.min.css" rel="stylesheet">
<script src="js/bootstrap-3.0.3.min.js" defer></script>
…
<a href="user-guide/writing-your-docs/">…</a>
```

The template filter URL does not help here.

```html
<link href="{{ 'css/bootstrap.min.css'|url }}" rel="stylesheet">
```
https://github.com/mkdocs/mkdocs/blob/master/mkdocs/themes/mkdocs/base.html#L20

Any ideas to prevent the problem on the mkdocs.org and also in the used theme? Thank you in advance!


Discussion:
Is this perhaps a duplicate of #1598? Wether it is a duplicate or not, it is certainly related as that issue discusses many extra considerations when generating error pages and it links to additional considerations from earlier discussions.

In any event, your example doe snot make the problem clear to me. Could you provide an example which contains all of the following information: 

1. What is the "bad" URL found in the built error page? 
2. What URL should be used instead? 
3. What is the URL of the error page when the "bad" URL is incorrect?
4. What template syntax generates the "bad" URL?
5. What is `site_url` set to in the `mkdocs.yaml` config file?

Please provide the URL as it appears in the HTML source. In other words, use a text editor to open the built error page (site/`404.html`) and copy the URL from there.
@waylan 
I'm sorry, but my first report was unclear. The report is related to mkdocs.org - your own homepage. Because the problem of the wrong path is included there. (I updated my post.)
And everyone who used the same theme or the same code from the theme to included CSS and JS files has the same problem.

All called and _wrong_ generated URLs can be found above. Also the used code.
@waylan it would be good if this could be mentioned in the official documentation. It can be quite confusing to run into and try identify what is going wrong.

Either MkDocs should detect a root domain at build time (no need for JS), or if resolving the problem isn't viable, the workaround/gotcha (root domain for `site_url` requires a trailing slash) should be documented for the [`site_url`](https://www.mkdocs.org/user-guide/configuration/#site_url) or [404 page](https://www.mkdocs.org/user-guide/deploying-your-docs/#404-pages) issue.

Updating the `mkdocs.yml` of this project to add the trailing slash to `site_url` should also resolve the problem depicted above in images and relevant link examples of the bug.

---

IIRC, not providing a `site_url` can also result in `sitemap.xml` being generated with empty entries instead of the actual paths, but that's a different issue.
> Updating the `mkdocs.yml` of this project to add the trailing slash to `site_url` should also resolve the problem depicted above in images and relevant link examples of the bug.

This is a tricky one to debug because the dev server overrides the setting and makes it impossible to test locally. I haven't found the time to work out what was causing this issue so thanks for that. We should be catching that in config validation and presumably adding the trailing slash if it is missing. In fact, that would be the fix for this issue--to update the validation code.

> IIRC, not providing a `site_url` can also result in `sitemap.xml` being generated with empty entries instead of the actual paths, but that's a different issue.

Yes, we are ware of the many issues with `site_url` not being defined. See [this comment][1] to #2189 where I provide a proposal for making `site_url` a required setting. Implementing that proposal includes updating the documentation.

[1]: https://github.com/mkdocs/mkdocs/issues/2189#issuecomment-718143015

Resolving patch:
diff --git a/mkdocs/config/config_options.py b/mkdocs/config/config_options.py
--- a/mkdocs/config/config_options.py
+++ b/mkdocs/config/config_options.py
@@ -1,7 +1,7 @@
 import os
 from collections import namedtuple
 from collections.abc import Sequence
-from urllib.parse import urlparse
+from urllib.parse import urlparse, urlunparse
 import ipaddress
 import markdown
 
@@ -269,7 +269,8 @@ class URL(OptionallyRequired):
     Validate a URL by requiring a scheme is present.
     """
 
-    def __init__(self, default='', required=False):
+    def __init__(self, default='', required=False, is_dir=False):
+        self.is_dir = is_dir
         super().__init__(default, required)
 
     def run_validation(self, value):
@@ -282,7 +283,9 @@ def run_validation(self, value):
             raise ValidationError("Unable to parse the URL.")
 
         if parsed_url.scheme:
-            return value
+            if self.is_dir and not parsed_url.path.endswith('/'):
+                parsed_url = parsed_url._replace(path=f'{parsed_url.path}/')
+            return urlunparse(parsed_url)
 
         raise ValidationError(
             "The URL isn't valid, it should include the http:// (scheme)")
diff --git a/mkdocs/config/defaults.py b/mkdocs/config/defaults.py
--- a/mkdocs/config/defaults.py
+++ b/mkdocs/config/defaults.py
@@ -24,7 +24,7 @@ def get_schema():
         ('pages', config_options.Nav()),
 
         # The full URL to where the documentation will be hosted
-        ('site_url', config_options.URL()),
+        ('site_url', config_options.URL(is_dir=True)),
 
         # A description for the documentation project that will be added to the
         # HTML meta tags.


Test patch:
diff --git a/mkdocs/tests/config/config_options_tests.py b/mkdocs/tests/config/config_options_tests.py
--- a/mkdocs/tests/config/config_options_tests.py
+++ b/mkdocs/tests/config/config_options_tests.py
@@ -217,6 +217,20 @@ def test_invalid(self):
         self.assertRaises(config_options.ValidationError,
                           option.validate, 1)
 
+    def test_url_is_dir(self):
+        url = "https://mkdocs.org/"
+
+        option = config_options.URL(is_dir=True)
+        value = option.validate(url)
+        self.assertEqual(value, url)
+
+    def test_url_transform_to_dir(self):
+        url = "https://mkdocs.org"
+
+        option = config_options.URL(is_dir=True)
+        value = option.validate(url)
+        self.assertEqual(value, f'{url}/')
+
 
 class RepoURLTest(unittest.TestCase):