Repository: mkdocs/mkdocs

Issue:
Static resources (especially CSS/JS!) shouldn't be automatically included
(Pulling this out from #972.)

Currently, MkDocs automatically includes all static resources from the docs dir in a build. This can cause issues if some files aren't meant to be included but are located in the docs dir. For example, this could happen with files that get preprocessed by another tool into `.md` files for MkDocs (which was how I planned on doing auto-generation of docs from code for MkDocs if I ever get around to it).

However, there's a more-significant issue: all CSS and JS files are automatically included in the generated HTML itself (except when hosting on RTD, apparently). This means that any CSS/JS used for static HTML files will get erroneously included in the MkDocs-generated HTML by default. The inconsistency with RTD adds another wrinkle to this. I think the behavior should be the same no matter what, and as Python says, "Explicit is better than implicit."

At minimum, I think CSS and JS shouldn't be automatically added to the generated HTML, but I'd prefer to see MkDocs only include files that are explicitly listed in `mkdocs.yml`. Doing so would make things more flexible when people want to do unusual things with MkDocs, and with globbing or the ability to include whole subdirectories of static resources, the configuration wouldn't be much more complex than it is now.

Start the deprecation of auto-populating extra_css and extra_javascript
See #986 

This is perhaps a bit rough, and could probably do with a test addition. I had to make an addition to the validation workflow so that warnings can be added in the post validation stage. At the same time I added support for this in the pre-validation stage too, since the logic is the same.

Outstanding bits:
- [ ] Add test
- [ ] Update the documentation to note this pending change
- [ ] Possibly change the wording of the warning



Discussion:
I have to admit that I have always found the the `extra_css` and `extra_javascript` settings to be odd (same for `extra_templates`, although that works slightly differently). We provide a way for users to insert their own CSS and JavaScript from the `theme_dir`, but we also allow the same thing through these "extra" settings from the `docs_dir`. Why are we offering users two ways to accomplish the same thing?

I'm guessing the answer is historical reasons. My assumption is that the "extra" settings were added first and the `theme_dir` came later (I haven't actually checked the commit history). But if the "extra" settings are there only for historical reasons, perhaps they should be deprecated altogether. Especially now that the `theme_dir` is being more fully documented and its use and purpose more fully defined (see #911 & #947 to be available in version 0.16). And, hey, it meets our goal of reducing the number of settings.

Suppose for a moment that I wanted to host a JavaScript file on my site for my readers to download as a separate JavaScript file. For the sake of argument, suppose I have a good reason for this file not being hosted elsewhere (such as a git repo). My first inclination would be to just include it next to my Markdown file which describes and links to it--the same way an image or PDF would work. But instead, the file gets added to `extra_javascript` and loaded as a script on the page. Maybe nothing breaks, but is my file even at the same path now? Or maybe is does break things by interfering with some script that is supposed to be on the page (variable name collision, etc...)

I see two possible solutions:
1. Leave the "extra" settings in place, but make them **not** auto-populating. Only scripts, and CSS files that the user explicitly lists get included. In other words, make `extra_css` and `extra_javascript` work the same way `extra_templates` does now. I believe this is @jimporter's proposal.
2. Deprecate the `extra` settings and require users to provide custom CSS and JavaScript via the `theme_dir` only. I would also suggest deprecating `extra-templates`.

Both solution's are backward incompatible for any user who relies on these settings being auto-populated. Personally, if we are going to make a change, I would suggest  going all the way with option 2. And now is the time to do it as we approach version 1.0.

For completeness, I should mention that some other static site generators have solved this type of problem by using a naming convention for "ignored" files. For example, any file or directory with a name that starts with an underscore is ignored. The settings file, the `theme_dir`, and the `site_dir` can then all be nested inside the `docs_dir` as long as their names all start with an underscore. Personally, I found is refreshing when I first realized MkDocs does not use that model, but I have to admit that is would make some of this stuff easier. But do we really want our static resources to all be in files that start with underscores? Uhg.

For what it's worth, I use the `extra_css` to add a few minor styling things that I reference in Markdown via the Attributes extension. It looks like I could do this via customizing the styles block in the theme, but that's a bit more work since I think I'd have to copy over all the default style tags into my override. The `extra_css` is nice since it's a low-friction way of adding a few custom rules to every file.

One way of handling ignored files would be to pick a reasonable default for static files to include (maybe everything that doesn't start with a dot?), but allow users to override this behavior by passing a set of files/globs to include/exclude. For instance:

``` yaml
static_resources:
  - include:
    - sample.txt
    - images/*.png
  - exclude:
    - _*
```

This would make it possible for people to pick what gets included or not while still retaining the default behavior of including everything. I would strongly prefer it if files were _only_ included if you explicitly listed them, but this would be an ok compromise. In any case, the current strategy of automatically adding all CSS/JS files to the theme seems a bit too error-prone to me.

I'd also encourage a way of warning users if MkDocs is including files in the build that aren't explicitly referenced in `mkdocs.yml`. This doesn't have to be on by default, though. It could be part of a "strict" mode in MkDocs to help prevent mistakes when building your docs. I know I'd enable that for all my docs builds (but then, I'm the sort of person who always compiles with `-Wall -Wextra -Werror -pedantic` in GCC).

> The inconsistency with RTD adds another wrinkle to this.

RTD adds their own layer of restrictions by overwriting settings and they add theme customisations. We can't really control what they do with MkDocs.

> It looks like I could do this via customizing the styles block in the theme, but that's a bit more work since I think I'd have to copy over all the default style tags into my override. 

All you need to do is create a CSS file in your `theme_dir` (probably in a  `css` sub- directory), and that file gets included with the Template files.

Although, in retrospect, it wouldn't get linked from the HTML. But you can use a [Super Block](http://jinja.pocoo.org/docs/dev/templates/#super-blocks) to avoid replicating the entire `styles` block:

```
{% block styles %}
    {{ super() }}
    <link href="{{ base_url }}/css/my-custom-styles.css" rel="stylesheet">
{% endblock %}
```

We should probably add that to the documentation somewhere. 

Granted, that is more involved than adding an item to the `extra_css` setting in the config file. Personally, I am for deprecating the "extra" settings altogether, but I understand if the decision is only to make them not auto-populate.

> I'm guessing the answer is historical reasons. My assumption is that the "extra" settings were added first and the theme_dir came later (I haven't actually checked the commit history). But if the "extra" settings are there only for historical reasons, perhaps they should be deprecated altogether.

I'm not sure, I think it was this way when I first got involved in the project. 

> 1. Leave the "extra" settings in place, but make them not auto-populating. Only scripts, and CSS files that the user explicitly lists get included. In other words, make extra_css and extra_javascript work the same way extra_templates does now. I believe this is @jimporter's proposal.

I think this is the option I would choose. Removing the option entirely makes adding small tweaks quite a bit harder - as you then need a `theme_dir` and an html file. The overhead and understanding required to then add some CSS is much higher.

> One way of handling ignored files would be to pick a reasonable default for static files to include (maybe everything that doesn't start with a dot?), but allow users to override this behavior by passing a set of files/globs to include/exclude. For instance:

[We already skip any files starting with `.`](https://github.com/mkdocs/mkdocs/blob/master/mkdocs/utils/__init__.py#L143-L145). I am not sure if this is documented.

> Removing the option entirely makes adding small tweaks quite a bit harder - as you then need a `theme_dir` and an html file. The overhead and understanding required to then add some CSS is much higher.

I can live with that. This issue has been bothering me from the beginning and I've been able to manage without saying anything so far. I just don't have to use it myself.

I think we are really talking about two things again - but this time they are probably related enough to be in the same issue.
1. The issue of automatically populating `extra_css` and `extra_javascript`. Sure, let's stop that. We should add a warning in 0.16 if any files are added automatically, then in 1.0 they will need to be specified manually.
2. Copying of all other static assets. I really don't have a problem with this, I feel like everything I put in my `docs_dir` should be included. I'd much rather a simpler barrier like this than having to look at a mkdocs.yml to see what is copied over.

Yeah, I'm happy to compromise and leave (2) as it currently is so long as (1) gets fixed. However, I personally prefer being very explicit about what gets included in my docs, so I'd certainly be happy to see (2) fixed as well.

That said, I don't have any current need for (2), so it's really just an academic question for me right now.

It's interesting to note, that many people are [already specifying the files](https://github.com/search?l=yaml&q=extra_css&type=Code&utf8=%E2%9C%93). This may be due to the difference on RTD. Unfortunately there isn't an easy way to search for users that are using it implicitly.

> We already skip any files starting with `.`. I am not sure if this is documented.

I always forget about that. I wonder if people who really want their `site_dir` nested in their `docs_dir` can make that work by naming their `site_dir` starting with a period? Never tried that myself.

> Copying of all other static assets. I really don't have a problem with this, I feel like everything I put in my `docs_dir` should be included. I'd much rather a simpler barrier like this than having to look at a mkdocs.yml to see what is copied over.

I agree here.

> That said, I don't have any current need for (2), so it's really just an academic question for me right now.

This is basically how I feel about it. If we can come up with a compelling use-case then I would be more interested in the effort required for a disruptive change.

It is interesting to note that `extra_templates` is a feature that [doesn't see much use](https://github.com/search?l=yaml&q=extra_templates&type=Code&utf8=%E2%9C%93), it doesn't auto-populate either. 

> I always forget about that. I wonder if people who really want their site_dir nested in their docs_dir can make that work by naming their site_dir starting with a period? Never tried that myself.

haha, yes, I think that would work. I really don't want to encourage this however, I think that could be highly confusing 😄 

Okay, setting milestone 0.16 to add a warning about css or javascript that is automatically included in the HTML. We can then have a think about copying assets in general.



Resolving patch:
diff --git a/mkdocs/config/base.py b/mkdocs/config/base.py
--- a/mkdocs/config/base.py
+++ b/mkdocs/config/base.py
@@ -64,24 +64,47 @@ def _validate(self):
 
     def _pre_validate(self):
 
+        failed, warnings = [], []
+
         for key, config_option in self._schema:
-            config_option.pre_validation(self, key_name=key)
+            try:
+                config_option.pre_validation(self, key_name=key)
+                warnings.extend([(key, w) for w in config_option.warnings])
+                config_option.reset_warnings()
+            except ValidationError as e:
+                failed.append((key, e))
+
+        return failed, warnings
 
     def _post_validate(self):
 
+        failed, warnings = [], []
+
         for key, config_option in self._schema:
-            config_option.post_validation(self, key_name=key)
+            try:
+                config_option.post_validation(self, key_name=key)
+                warnings.extend([(key, w) for w in config_option.warnings])
+                config_option.reset_warnings()
+            except ValidationError as e:
+                failed.append((key, e))
+
+        return failed, warnings
 
     def validate(self):
 
-        self._pre_validate()
+        failed, warnings = self._pre_validate()
+
+        run_failed, run_warnings = self._validate()
 
-        failed, warnings = self._validate()
+        failed.extend(run_failed)
+        warnings.extend(run_warnings)
 
         # Only run the post validation steps if there are no failures, warnings
         # are okay.
         if len(failed) == 0:
-            self._post_validate()
+            post_failed, post_warnings = self._post_validate()
+            failed.extend(post_failed)
+            warnings.extend(post_warnings)
 
         return failed, warnings
 
diff --git a/mkdocs/config/config_options.py b/mkdocs/config/config_options.py
--- a/mkdocs/config/config_options.py
+++ b/mkdocs/config/config_options.py
@@ -393,6 +393,16 @@ def post_validation(self, config, key_name):
 
         config[key_name] = extras
 
+        if not extras:
+            return
+
+        self.warnings.append((
+            'The following files have been automatically included in the '
+            'documentation build and will be added to the HTML: {0}. This '
+            'behavior is deprecated. In version 1.0 and later they will '
+            "need to be explicitly listed in the '{1}' config setting."
+        ).format(','.join(extras), key_name))
+
 
 class Pages(Extras):
     """


Test patch:
diff --git a/mkdocs/tests/build_tests.py b/mkdocs/tests/build_tests.py
--- a/mkdocs/tests/build_tests.py
+++ b/mkdocs/tests/build_tests.py
@@ -28,6 +28,8 @@ def load_config(cfg=None):
         cfg['site_name'] = 'Example'
     if 'config_file_path' not in cfg:
         cfg['config_file_path'] = os.path.join(os.path.abspath('.'), 'mkdocs.yml')
+    if 'extra_css' not in cfg:
+        cfg['extra_css'] = ['css/extra.css']
     conf = config.Config(schema=config.DEFAULT_SCHEMA)
     conf.load_dict(cfg)
 
diff --git a/mkdocs/tests/config/base_tests.py b/mkdocs/tests/config/base_tests.py
--- a/mkdocs/tests/config/base_tests.py
+++ b/mkdocs/tests/config/base_tests.py
@@ -5,6 +5,7 @@
 
 from mkdocs import exceptions
 from mkdocs.config import base, defaults
+from mkdocs.config.config_options import BaseConfigOption
 
 
 class ConfigBaseTests(unittest.TestCase):
@@ -125,3 +126,111 @@ def test_load_missing_required(self):
                               base.load_config, config_file=config_file.name)
         finally:
             os.remove(config_file.name)
+
+    def test_pre_validation_error(self):
+        class InvalidConfigOption(BaseConfigOption):
+            def pre_validation(self, config, key_name):
+                raise base.ValidationError('pre_validation error')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 1)
+        self.assertEqual(errors[0][0], 'invalid_option')
+        self.assertEqual(str(errors[0][1]), 'pre_validation error')
+        self.assertTrue(isinstance(errors[0][1], base.ValidationError))
+        self.assertEqual(len(warnings), 0)
+
+    def test_run_validation_error(self):
+        class InvalidConfigOption(BaseConfigOption):
+            def run_validation(self, value):
+                raise base.ValidationError('run_validation error')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 1)
+        self.assertEqual(errors[0][0], 'invalid_option')
+        self.assertEqual(str(errors[0][1]), 'run_validation error')
+        self.assertTrue(isinstance(errors[0][1], base.ValidationError))
+        self.assertEqual(len(warnings), 0)
+
+    def test_post_validation_error(self):
+        class InvalidConfigOption(BaseConfigOption):
+            def post_validation(self, config, key_name):
+                raise base.ValidationError('post_validation error')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 1)
+        self.assertEqual(errors[0][0], 'invalid_option')
+        self.assertEqual(str(errors[0][1]), 'post_validation error')
+        self.assertTrue(isinstance(errors[0][1], base.ValidationError))
+        self.assertEqual(len(warnings), 0)
+
+    def test_pre_and_run_validation_errors(self):
+        """ A pre_validation error does not stop run_validation from running. """
+        class InvalidConfigOption(BaseConfigOption):
+            def pre_validation(self, config, key_name):
+                raise base.ValidationError('pre_validation error')
+
+            def run_validation(self, value):
+                raise base.ValidationError('run_validation error')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 2)
+        self.assertEqual(errors[0][0], 'invalid_option')
+        self.assertEqual(str(errors[0][1]), 'pre_validation error')
+        self.assertTrue(isinstance(errors[0][1], base.ValidationError))
+        self.assertEqual(errors[1][0], 'invalid_option')
+        self.assertEqual(str(errors[1][1]), 'run_validation error')
+        self.assertTrue(isinstance(errors[1][1], base.ValidationError))
+        self.assertEqual(len(warnings), 0)
+
+    def test_run_and_post_validation_errors(self):
+        """ A run_validation error stops post_validation from running. """
+        class InvalidConfigOption(BaseConfigOption):
+            def run_validation(self, value):
+                raise base.ValidationError('run_validation error')
+
+            def post_validation(self, config, key_name):
+                raise base.ValidationError('post_validation error')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 1)
+        self.assertEqual(errors[0][0], 'invalid_option')
+        self.assertEqual(str(errors[0][1]), 'run_validation error')
+        self.assertTrue(isinstance(errors[0][1], base.ValidationError))
+        self.assertEqual(len(warnings), 0)
+
+    def test_validation_warnings(self):
+        class InvalidConfigOption(BaseConfigOption):
+            def pre_validation(self, config, key_name):
+                self.warnings.append('pre_validation warning')
+
+            def run_validation(self, value):
+                self.warnings.append('run_validation warning')
+
+            def post_validation(self, config, key_name):
+                self.warnings.append('post_validation warning')
+
+        c = base.Config(schema=(('invalid_option', InvalidConfigOption()), ))
+
+        errors, warnings = c.validate()
+
+        self.assertEqual(len(errors), 0)
+        self.assertEqual(warnings, [
+            ('invalid_option', 'pre_validation warning'),
+            ('invalid_option', 'run_validation warning'),
+            ('invalid_option', 'post_validation warning'),
+        ])
diff --git a/mkdocs/tests/config/config_options_tests.py b/mkdocs/tests/config/config_options_tests.py
--- a/mkdocs/tests/config/config_options_tests.py
+++ b/mkdocs/tests/config/config_options_tests.py
@@ -289,7 +289,7 @@ def test_invalid(self):
         self.assertRaises(config_options.ValidationError,
                           option.validate, {})
 
-    def test_talk(self):
+    def test_walk(self):
 
         option = config_options.Extras(utils.is_markdown_file)
 
diff --git a/mkdocs/tests/config/config_tests.py b/mkdocs/tests/config/config_tests.py
--- a/mkdocs/tests/config/config_tests.py
+++ b/mkdocs/tests/config/config_tests.py
@@ -198,7 +198,6 @@ def test_doc_dir_in_site_dir(self):
         )
 
         conf = {
-            'site_name': 'Example',
             'config_file_path': j(os.path.abspath('..'), 'mkdocs.yml')
         }
 
@@ -211,7 +210,11 @@ def test_doc_dir_in_site_dir(self):
             c = config.Config(schema=(
                 ('docs_dir', config_options.Dir(default='docs')),
                 ('site_dir', config_options.SiteDir(default='site')),
+                ('config_file_path', config_options.Type(utils.string_types))
             ))
             c.load_dict(patch)
 
-            self.assertRaises(config_options.ValidationError, c.validate)
+            errors, warnings = c.validate()
+
+            self.assertEqual(len(errors), 1)
+            self.assertEqual(warnings, [])