Repository: mkdocs/mkdocs

Issue:
Change to prevent `0.0.0.0` makes containerized usage painful
The change in 1.1.1 to prevent the usage of `0.0.0.0` makes using MkDocs in a container painful. We have a local development environment used by our developers which spins up a documentation repository that they can live-edit, with MkDocs hosted inside a Docker container. This is now significantly more difficult.


Discussion:
We do not support using the server on anything except the local machine. We never have and never will. The only change is that we are now enforcing it. In other words, you have not been using the server in a supported way. We have never guaranteed that it would continue to work for you. If you want to use a server on anything other than the local machine, then we have always suggested that you use an appropriate third-party server.
These are used by developers on local machines, just within a Docker container to keep MkDocs isolated from the rest of the environment. We properly use `mkdocs build` when documentation is released, and you are completely correct in your assertion that it should not be used outside a local machine.

This change means we (and anyone else using `mkdocs serve` in a container) need to either maintain a modified version of MkDocs, a configuration file with mangled addresses, or a (moderately, but not extremely) complex script to inspect the local network and determine exactly the right address to bind to every time the container spins up.

Further, I believe this change is fragile - it does not prevent the use of (for example) `0.0.0.00` or `[::0]` as a wildcard bind (even though neither is the most reduced form).

If security is the primary impetus behind this change, adding a banner to the default theme when a wildcard bind is detected (that must then be removed by customizing the theme) or requiring a matching environment variable to permit such binds would probably be better solutions.
> to keep MkDocs isolated from the rest of the environment

If that is all you are doing, a Docker image seems like overkill to me. A Python [venv][1] works great and it what I use.

[1]: https://docs.python.org/3.7/library/venv.html
It's actually part of a larger set of developer tooling:

Developers edit OpenAPI YAML files, which are run through source code generators as well as a [Widdershins](https://github.com/Mermade/widdershins) watcher which generates Markdown. The Markdown is injected into MkDocs so as developers modify API specifications they can review the resulting stubs and documentation before it is committed.

Our developers work on several different platforms and our docker-compose environment ensures everyone can easily have all the tooling at hand to work with our code base - which is not as large as many out there; I'm sure there are many developers using MkDocs' flexibility and and live editing in more complex ways than this as part of a documentation pipeline.

Compatibility with containers is important in modern environments; being flexible on this will endear you to a lot of developers who may well look for another tool if they have to struggle to get this working inside Docker when they want to try MkDocs out.
I agree with this issue -- this makes a docker-compose bind of `127.0.0.1:8080:8080`, which is both non-production and not (external) network accessible, fail to start.

We already use docker for several things, so our limited (1-2 person) multi-hat IT staff already give out machines with docker ready... making it a simple `git clone` and `docker-compose up`... doesn't matter what OS.

while venv is perfectly suitable for programers and IT Ops people, it's not the best for business staff that don't even understand Markdown before approaching this.
If you require a complex environment like that, and have the expertise to get it all working, then adding a third party server to your container should be little trouble for you.

<sub>Sent with <a href="http://githawk.com">GitHawk</a></sub>
Why the hostility towards containerization?  It's here and it governs the world of many in the this industry.  Having it in a Docker along with our other 10+ services that are spun up in our docker-compose environment means that it is just one of a dozen services that are ready to go with a single, simple `docker-compose up`.
> If you require a complex environment like that, and have the expertise to get it all working, then adding a third party server to your container should be little trouble for you.

It's not massive trouble but it's more trouble than zero, which is the way it was until now. I'm kinda speechless that it would be so hard to understand that the feature useful for a lot of people, and the level of hostility in the responses. Using python `venv` is not a solution for a product that has nothing to do with python, but it's about writing docs. In our company we want any engineer to be able to contribute to the docs, and for this they should be able to 1-click start the project, which until now was easily doable with this Dockerfile

```dockerfile
FROM python:3.7-stretch

WORKDIR /usr/src/app

# Install dependencies and fire server w/ automatic reloading
CMD [ "sh", "-c", "pip install --cache-dir=.pip -r requirements.txt; mkdocs serve -a 0.0.0.0:8000" ] 
```

Now I will need to create a docker-compose with some some workaround or nginx just to fake the host which makes zero sense. The only reason to put a `0.0.0.0` validation is to help devs not make mistakes, yet you're making life harder for a subset. 
@inakianduaga I just made a similar change for my employees... Apparently putting it behind a reverse proxy breaks live reloading... They were a bit disappointed.

If you (or anyone) figures out a solution to work around this, I'd love to see how to preserve it behind a reverse proxy
@VTLee If you're on linux you can simply run the container using [host networking](https://docs.docker.com/network/host/) and it should work. We are on macs so that doesn't work. I guess the easiest is to lookup the docker host ip on the host and pass it as an ENV variable to the container on the `docker run` command (passing `host.docker.internal` also doesn't work because it also breaks mkdocs built-in validation)
This change prohibits any usage of the MkDocs dev server inside of Docker and I strongly advocate against this change. Providing MkDocs inside of a Docker image with dependencies preinstalled is a very important and real use case. In fact, the Material for MkDocs [Docker image](https://hub.docker.com/r/squidfunk/mkdocs-material/) has been pulled over 800k times.

Another use case I frequently encounter is when I want to preview the docs on my local machine, as well as on my mobile phone, which `0.0.0.0` makes very convenient.

Please revert to the old behavior. This will cause huge frustration among developers and a flood of incoming issues. I don't understand why this change is necessary.
>  putting it behind a reverse proxy breaks live reloading

This has never worked and results in a constant stream of requests from users. The change under discussion was intended to curtail this requests by informing the user that this is not supported.

> This will cause huge frustration among developers and a flood of incoming issues. I don't understand why this change is necessary.

If a solution can be provided which allows docker containers but does not permit any other use outside of the local machine, I'm willing to listen. Personally, I have never saw any benefit to docker containers and the support requests are a contact burden and annoyance to me. I would prefer that you instead use a third party server.

The only alternate approach I have considered thus far is removing the server altogether. I certainly don't need it, as it is easy enough to use a third party solution. But I'm open to other suggestions.
> the level of hostility in the responses

Sorry, I don't mean to be hostile, but I am extremely annoyed and frustrated that users continue to insist on using the sever in a way that we continue to state is not supported. We have continually insisted that a third party server should be used and yet it has fallen on deaf ears. Personally, I don't have time to support this and no-one else has made themselves available to do so.
> Personally, I have never saw any benefit to docker containers and the support requests are a contact burden and annoyance to me. I would prefer that you instead use a third party server.

You underestimate how many people are not proficient in Python and look for a one-stop solution. There's a large community of technical writers (as [my survey](https://twitter.com/squidfunk/status/1259832077668810752) shows), and from my own experience with Python I can say that setting up a clean environment is far from being easy. There're several methods like `virtualenv`, `pipenv` etc., all with their caveats. Sadly, it's not as easy as `npm`.

Also, how should MkDocs be integrated with a third-party server for serving assets and live-reloading? If you could give some guidance, we can work towards a solution, but just taking this feature away from the users is not so nice.

> Sorry, I don't mean to be hostile, but I am extremely annoyed and frustrated that users continue to insist on using the sever in a way that we continue to state is not supported.

Where's this frustration coming from? Users want to put the development server in production? How about just printing a warning when running on 0.0.0.0 "Warning, this development server is not meant for production use."?
Fully agree with @squidfunk. This change just broke my local (docker-based) development environment without warning. I had been using the same setup for over a year now, working flawlessly.

This change is outright hostile to docker-based setups, and not really effective anyway in what it is trying to do, as @acaos pointed out above. Simply using `--dev-addr 0.0.0.00:8000` (or similar variations) instead still does work.

However, I do understand your frustration and not everyone is into Docker either. Therefor I'd like to suggest to put this change behind another parameter and printing out a big fat warning when used:

~~~
$ mkdocs serve --livereload --dev-addr 0.0.0.0:8000 --i-know-what-i-am-doing-and-innocent-kittens-may-get-killed-because-of-it
#################################################################
### The MkDocs' server is intended for development purposes   ###
### only. Therefore, '0.0.0.0' is not a supported IP address. ###
### Please use a third party production-ready server instead. ###
#################################################################
INFO    -  Building documentation...
~~~

The default behavior then could still be the current one.
I'd be fine with a completely obnoxious flag over a flat-out 'crash'...

`mkdocs serve --allow-non-production-and-unsupported-bind-to-all-ips`

really call it what you want -- but if you require them to use a flag like this, I would _hope_ it's fairly obvious that 1) It's unsupported, and 2) it's not for production use

I also would combined this with @michaelkrupp 's suggested output when used
@squidfunk you make some good arguments, however...

> You underestimate how many people... look for a one-stop solution. 

This is part of the problem. We don't offer a one-stop solution. We never have. We offer a local development solution only. Anything outside of that has never been part of MkDocs. The fact that some users have hacked it to be a a risk they took. 

> ... from my own experience with Python I can say that setting up a clean environment is far from being easy.

And a Docker Container is easy? Seems to me like venv is worlds easier to work out that docker. So long as we are talking about docker, this argument will never hold for me.

> There're several methods like virtualenv, pipenv etc., all with their caveats.

And only `venv` is included in the standard lib. Why would you consider anything else? But I digress.

> Also, how should MkDocs be integrated with a third-party server for serving assets and live-reloading? 

I haven't researched the options myself, but the live-reload server is based on a standard (which is not unique to nor originates with our server). If `npm` is so easy, I'm sure there are multiple node servers which would work quite well for this. And they would be better supported. As you are well aware, the python based server we use is currently not being maintained. If we change servers (due to lack of maintenance) we likely would loose the live-reload feature as I'm not aware of any other Python solutions and I don't have the resources to maintain one myself. If that were to happen, I suspect everyone involved in this conversation would move to a third party server anyway. Think of this as a push in that direction.

> Users want to put the development server in production? 

Yes, that is a very common request. We could issue a warning, but given how those users react when they are informed we don't support production use, they would just ignore the warning. Therefore, I was trying to force the issue.

Truth be told, I expected there to be some backlash, but not from those using docker containers. Admittedly, as I don't use docker, it hadn't occurred to me that that would be an issue.

That said, I don't see why we should support them. Our server is not to be used outside the local machine. From one perspective a docker container is not the local machine.

@VTLee 

> 
> 
> @inakianduaga I just made a similar change for my employees... Apparently putting it behind a reverse proxy breaks live reloading... They were a bit disappointed.
> 
> If you (or anyone) figures out a solution to work around this, I'd love to see how to preserve it behind a reverse proxy

You can do this with nginx with a block similar to this:

```
    location / {
        add_header Access-Control-Allow-Origin $http_origin;
        add_header Access-Control-Allow-Methods "GET, POST, OPTIONS";

        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Port $server_port;

        set $backend "http://127.0.0.1:8080";
        proxy_pass $backend;
        proxy_http_version 1.1;
        proxy_redirect off;
    }
```

That or just use `mkdocs serve -a 0.0.0.00:8080`.
I've given this some more thought and I see a few possible solutions, in order of (increasing) severity, although not necessarily order of preference:

- Someone else steps up and volunteers to maintain the server code. Their first commit can be changing the error to a warning. But I expect them to honor their word and stay with the project.
- The current behavior remains.
- The server is completely dropped and users are responsible to go find their own solution. Perhaps we could offer a few recommendations in the docs.
- I walk away from MkDocs.

To be fair, I'm not completely satisfied with any of those myself. But I don't have any other ideas.
> You can do this with nginx with a block similar to this:

@acaos you are perpetuating the problem. We do not support using proxy server with the included server. Every time someone comments with  a "working solution" that is just one more date point which reinforces the need to forcefully disable it.
@waylan you don't understand that people want to run this in a Docker on their local development system SO THAT they don't have to worry about even having Python installed.  With Docker, I can trivially run a local development version on a Windows box using Docker Desktop -- without having to know anything about that Python Jazz.  There are probably another twenty "why this is good to allow us to run your awesome tool in Docker" that we can come up with...

Leave your new behavior as default and add a `--force-insecure-bind-all` command line parameter that allows those of us who know what we're doing to do continue to use it within a Docker on our local development systems.  It's a really simple solution that allows you to have the default behavior *and* you even get to call the feature "insecure" so that nobody can be surprised by it.
> @waylan you don't understand that people want to run this in a Docker on their local development system SO THAT they don't have to worry about even having Python installed. 

If you have such an aversion to Python, then why are you using a Python based tool? There are certainly more than enough competing products to chose from developed in any number of languages. 

In looking at the various options out there, I came across a number of products where the docs basically said: just download and launch this docker container and you get a one-stop solution. All I saw was a mysterious black box. When I found MkDocs, I understood how it worked. I found a product I was willing to use. It seems some of you want the black box. I can't identify with that mindset.

> Leave your new behavior as default and add a `--force-insecure-bind-all` command line parameter that allows those of us who know what we're doing to do continue to use it within a Docker on our local development systems. It's a really simple solution that allows you to have the default behavior and you even get to call the feature "insecure" so that nobody can be surprised by it.

The issue is not that it is "insecure." A more appropriate name for the flag might be `--enable-proxy-connections` which I have repeatedly stated that we have never supported. Either we support a feature or we don't. The fact that a flag exists to enable a behavior suggests that we support it.  And no one is going to accept the 'you can only use that feature if you know what you are doing' response. In other words, adding a flag makes it worse not better as we can no longer claim we don't support that use case.

As I think about it, it occurs to me that anyone who is using a docker container has been using a proxy. Therefore this has always been unsupported. The fact that it worked as all was a coincidence. Even if we removed the forced error, the entire thing could break at any time for any number of reasons. It would be a disservice to our users to suggest otherwise. 

> If you have such an aversion to Python, then why are you using a Python based tool? There are certainly more than enough competing products to chose from developed in any number of languages.

You fell into the common trap thinking that people choose solutions because of their underlying technologies. This might be true for developers, who question every layer of the stack, but MkDocs solves problems of non-technical people, i.e. technical writers, who mostly don't care about the underlying technology. Personally, I'm not a fan of Python. I dislike the syntax and the whole environment. However, I built one of the most popular third party themes for a Python-based static site generator, because I believe in its values – not in its technology.

> In looking at the various options out there, I came across a number of products where the docs basically said: just download and launch this docker container and you get a one-stop solution. All I saw was a mysterious black box. When I found MkDocs, I understood how it worked. I found a product I was willing to use. It seems some of you want the black box. I can't identify with that mindset.

It's fine to speak of your own expectations evaluating a new technology, but a project with 10k stars should also consider the expectations of users. To you, Docker is a black box. To me, it is Python. I'm actually more familiar with Docker than Python. Still, I love MkDocs.
> As I think about it, it occurs to me that anyone who is using a docker container has been using a proxy.

This is inaccurate. In fact, in our use, in a Docker, on our local development systems, is doing nothing more than simple port forwarding.  There's no HTTP proxy server in the middle.  This is the standard way for using Docker locally for development and many other things.

This change isn't preventing proxies at all either -- proxies can explicitly route to loopback...  In fact, you're expressly pushing people towards a proxy solution as a work around.

I would like to plead with you to just add `-some-command-line-option` to disable this new default behavior.  It will end a lot of pain for people today and for a long time to come.
@waylan Please reconsider your decision. As stated above, I do understand that you are not familiar with Docker and do not want to use it personally. That is fair, and I have been on the same track couple of years ago.

Fact is though, that docker has become the de-facto standard for many of us. For someone like me it has become indispensable by now, as I do not develop in just one or two language on a daily basis, but somewhere in the ballpark of 25. Maintaining 25 tool-chains on 4 devices is just not practical. With docker I can abstract all of this and just not care about how the internals work. Everything is just a simple `docker pull` away.

Python is my favorite language (and has ever been since 2.3) by a huge margin - I absolutely love it. But its ecosystem currently is just a huge mess and a major pain in the a** to work with. Docker solves this problem for the most part.

The change you made will just spawn work-arounds, of which two have already been posted here.
Another one would be to simply monkey-patch that config validator method via a mkdocs-plugin. All of those are arguably worse than a clean `--iknowhatiamdoing` command line switch.

People who want to run it in production will do so, no matter how hard you try to stop them. Please do not throw us Docker users under the bus and force us to use ugly hacks just because you get annoyed with people running unsupported configurations.

We all love MkDocs, and we want to keep using it in our Docker environments :heartpulse:
I will add my voice to the requests to please enable `0.0.0.0` as a bind address once again. I work in a polyglot environment and use Docker to provide a consistent developer environment for all team members, regardless of technical background. If my team members are required to become Python developers in order to use mkdocs, they will reject `mkdocs`.
I've created a MkDocs + BrowserSync Docker configuration which can be used for automated reloading. Hopefully this will provide an alternative or template to those who are having issues with `mkdocs serve` in a Docker, and also a backup in case the `serve` functionality breaks or is entirely removed.

It can be found at: <https://github.com/acaos/mkdocs-browsersync>

I still believe it would be ideal to revert this change.
Also: This changes a behavior which is offered by the kernel itself, and people are used to.

Even if it wasn't for to the wide amount of people using docker nowadays, it still makes sense to reconsider your decision and remove this validation (considering it is not covering any edge-cases as you can see here: https://github.com/mkdocs/mkdocs/issues/2108#issuecomment-627562040) as it introduces more frustration, confusion and pain. 

Maybe a sentence or two in the README or github issue-templates would be more beneficial, where you can put an emphasis on the matter, that anything other than localhost is not supported.

@waylan it might be tempting to think that such a change will ease maintenance by clearly stating that this behavior is not supported, but as we can see from this issue you'll be explaining the reasoning behind this decision over and over again. It will lead to _more_ issues.
> @waylan it might be tempting to think that such a change will ease maintenance by clearly stating that this behavior is not supported, but as we can see from this issue you'll be explaining the reasoning behind this decision over and over again. It will lead to more issues.

Yes I can see that. I completely underestimated the popularity of docker containers (probably because I have never had a legitimate need for using them). As I stated previously, these are the options I'm willing to consider:

> - Someone else steps up and volunteers to maintain the server code. Their first commit can be changing the error to a warning. But I expect them to honor their word and stay with the project.
> - The current behavior remains.
> - The server is completely dropped and users are responsible to go find their own solution. Perhaps we could offer a few recommendations in the docs.
> - I walk away from MkDocs.

I am leaning very heavily toward the last option. I'm willing to make one final release which restores the behavior and step away. This issue has made it clear that MkDocs is causing me too much stress. The change under discussion was an attempt to reduce that stress but has instead increased it. I don't have any good reason to stay here. I can perhaps use the time to focus on a few other ideas I have't had time to explore.

To be clear, this is not meant as an empty threat to force someone into something. I am simply tired of it. Perhaps after a break I'll come back refreshed, but as of now it is looking like MkDocs could become an unmaintained project.

Resolving patch:
diff --git a/mkdocs/config/config_options.py b/mkdocs/config/config_options.py
--- a/mkdocs/config/config_options.py
+++ b/mkdocs/config/config_options.py
@@ -236,7 +236,8 @@ def run_validation(self, value):
 
         if host != 'localhost':
             try:
-                ipaddress.ip_address(host)
+                # Validate and normalize IP Address
+                host = str(ipaddress.ip_address(host))
             except ValueError as e:
                 raise ValidationError(e)
 
@@ -254,10 +255,11 @@ def __str__(self):
     def post_validation(self, config, key_name):
         host = config[key_name].host
         if key_name == 'dev_addr' and host in ['0.0.0.0', '::']:
-            raise ValidationError(
-                ("The MkDocs' server is intended for development purposes only. "
-                 "Therefore, '{}' is not a supported IP address. Please use a "
-                 "third party production-ready server instead.").format(host)
+            self.warnings.append(
+                ("The use of the IP address '{}' suggests a production environment "
+                 "or the use of a proxy to connect to the MkDocs server. However, "
+                 "the MkDocs' server is intended for local development purposes only. "
+                 "Please use a third party production-ready server instead.").format(host)
             )
 
 


Test patch:
diff --git a/mkdocs/tests/config/config_options_tests.py b/mkdocs/tests/config/config_options_tests.py
--- a/mkdocs/tests/config/config_options_tests.py
+++ b/mkdocs/tests/config/config_options_tests.py
@@ -131,6 +131,14 @@ def test_default_address(self):
         self.assertEqual(value.host, '127.0.0.1')
         self.assertEqual(value.port, 8000)
 
+    def test_IP_normalization(self):
+        addr = '127.000.000.001:8000'
+        option = config_options.IpAddress(default=addr)
+        value = option.validate(None)
+        self.assertEqual(str(value), '127.0.0.1:8000')
+        self.assertEqual(value.host, '127.0.0.1')
+        self.assertEqual(value.port, 8000)
+
     def test_invalid_address_range(self):
         option = config_options.IpAddress()
         self.assertRaises(
@@ -166,25 +174,17 @@ def test_invalid_address_missing_port(self):
             option.validate, '127.0.0.1'
         )
 
-    def test_disallowed_address(self):
+    def test_unsupported_address(self):
         option = config_options.IpAddress()
         value = option.validate('0.0.0.0:8000')
-        self.assertRaises(
-            config_options.ValidationError,
-            option.post_validation,
-            {'dev_addr': value},
-            'dev_addr'
-        )
+        option.post_validation({'dev_addr': value}, 'dev_addr')
+        self.assertEqual(len(option.warnings), 1)
 
-    def test_disallowed_IPv6_address(self):
+    def test_unsupported_IPv6_address(self):
         option = config_options.IpAddress()
         value = option.validate(':::8000')
-        self.assertRaises(
-            config_options.ValidationError,
-            option.post_validation,
-            {'dev_addr': value},
-            'dev_addr'
-        )
+        option.post_validation({'dev_addr': value}, 'dev_addr')
+        self.assertEqual(len(option.warnings), 1)
 
     def test_invalid_IPv6_address(self):
         # The server will error out with this so we treat it as invalid.