Repository: mkdocs/mkdocs (Python)

Issue:
mkdocs 1.2.x livereload no longer working
mkdocs==1.2 and latest no longer works
mkdocs==1.1.2 works

MacOS 11.4
Docker version 20.10.7, build f0df350
python:3.8-alpine (3.9.5 and 10-rc same behavior)

I checked the Docker container and the markdown is updating but the generated html is not. This seems to be associated with the 1.2 change with watcher.

LMK what logs/config/diag I can provide to help RCA the issue.


Discussion:
Ah, I recently experienced issues with watchdog + Docker/Kubernetes setups. Basically the observer based on inotify is a bit limited on Docker setups, I did not dig deep into it but it seems the inotify events on directories within a volume are only available to one container. I solved it by using the observer that polls the file system instead. Do you happen to have your docs files in a volume mounted on several containers?
We mount our source docs to /tmp in the container. We do have several containers that run this way but we observed the issue in a singleton environment (compose down, prune, and a single compose up).

Happy to help troubleshoot. LMK.
Could you try to add "dependencies" (not sure if this is the right term) to your docker-compose configuration so as to make sure the container running MkDocs starts **last**, and tell us if livereload still does not work?
I understand now. The docker compose has a single container only. 

In summary, I see the markdown update but not the HTML file change when observed inside the container with docker exec -it /bin/sh

I tested with mkdoc -v serve and saw no output after a change. 
Users of Material for MkDocs are also reporting this new behavior, i.e. `mkdocs serve` not working properly. It would be great if we could somehow ensure that it also works from within Docker, because many users use the Docker image for previewing.
A quick test would be to patch https://github.com/mkdocs/mkdocs/blob/master/mkdocs/livereload/__init__.py#L65 to use `PollingObserver` instead of `Observer` to see if it fixes the issue.

```python
from watchdog.observers.polling import PollingObserver
```
@oprypin hi, what's your point of view on this please mate?
> A quick test would be to patch https://github.com/mkdocs/mkdocs/blob/master/mkdocs/livereload/__init__.py#L65 to use `PollingObserver` instead of `Observer` to see if it fixes the issue.
> 
> ```python
> from watchdog.observers.polling import PollingObserver
> ```

I'm using docker compose with a single container and this patch fixed the issue for me.
Does reproducing this issue require running MkDocs in Docker on Mac? Or are there perhaps other conditions to this?

I don't know how to reproduce it, because just [running in Docker on Linux](https://squidfunk.github.io/mkdocs-material/creating-your-site/#previewing-as-you-write) works.
I saw the issue when running Docker on Windows, with the docs files mounted from the Windows file system.

However, using Docker on WSL2 with the docs files mounted from the Linux file system, I'm not able to reproduce the issue.
I am running Docker on Mac and see this issue too. Some initial logs:

`
INFO     -  Documentation built in 51.21 seconds
DEBUG    -  [13:26:54] Watching directory '/docs/docs'
DEBUG    -  [13:26:54] Watching file '/docs/mkdocs.yml' through directory '/docs'
INFO     -  [13:26:54] Serving on http://0.0.0.0:8000/
DEBUG    -  [13:26:56] "GET /livereload/47543439/47625824 HTTP/1.1" code 200
`

We run the docker container as follows:

`
docker build -t mkdocs-serve-local:latest mkdocs/ && docker run --rm -it -p 8000:8000 -v ${PWD}:/docs mkdocs-serve-local:latest
`

The Dockerfile CMD is `CMD ["serve", "-v", "--dev-addr=0.0.0.0:8000"]`

These are the dependencies we use when building the image:

```
mkdocs==1.2.1
Pygments>=2.4 
markdown>=3.2 
pymdown-extensions>=7.0 
mkdocs-material-extensions>=1.0
mkdocs-git-revision-date-localized-plugin>=0.7.3 
mkdocs-material>=7.1.8
```
I have a proposed fix in #2477.
I did not figure out any hybrid approach that would let us still benefit from the fast implementation in most cases.

Please try it and let me know if something's wrong.
```bash
pip install git+https://github.com/mkdocs/mkdocs.git@refs/pull/2477/head
```
@oprypin, I can confirm `git+https://github.com/mkdocs/mkdocs.git@refs/pull/2477/head` fixes the issue in docker compose on MacOS.
The fix is now included in the [latest release](https://github.com/mkdocs/mkdocs/releases/tag/1.2.2).
Time for mkdocs to move to [watchfiles](https://watchfiles.helpmanual.io/) ? :eyes:

_(Admission: watchfiles is me)_



Patch:
diff --git a/mkdocs/livereload/__init__.py b/mkdocs/livereload/__init__.py
index 5ac36e7343..a599b73575 100644
--- a/mkdocs/livereload/__init__.py
+++ b/mkdocs/livereload/__init__.py
@@ -4,7 +4,6 @@
 import mimetypes
 import os
 import os.path
-import pathlib
 import re
 import socketserver
 import threading
@@ -13,7 +12,7 @@
 import wsgiref.simple_server
 
 import watchdog.events
-import watchdog.observers
+import watchdog.observers.polling
 
 
 class _LoggerAdapter(logging.LoggerAdapter):
@@ -35,7 +34,7 @@ def __init__(
         port,
         root,
         mount_path="/",
-        build_delay=0.25,
+        polling_interval=0.5,
         shutdown_delay=0.25,
         **kwargs,
     ):
@@ -45,7 +44,7 @@ def __init__(
         self.root = os.path.abspath(root)
         self.mount_path = ("/" + mount_path.lstrip("/")).rstrip("/") + "/"
         self.url = f"http://{self.server_name}:{self.server_port}{self.mount_path}"
-        self.build_delay = build_delay
+        self.build_delay = 0.1
         self.shutdown_delay = shutdown_delay
         # To allow custom error pages.
         self.error_handler = lambda code: None
@@ -62,7 +61,7 @@ def __init__(
 
         self._shutdown = False
         self.serve_thread = threading.Thread(target=lambda: self.serve_forever(shutdown_delay))
-        self.observer = watchdog.observers.Observer(timeout=shutdown_delay)
+        self.observer = watchdog.observers.polling.PollingObserver(timeout=polling_interval)
 
     def watch(self, path, func=None, recursive=True):
         """Add the 'path' to watched paths, call the function and reload when any file changes under it."""
@@ -77,57 +76,18 @@ def watch(self, path, func=None, recursive=True):
                 stacklevel=2,
             )
 
-        def callback(event, allowed_path=None):
-            if isinstance(event, watchdog.events.DirCreatedEvent):
+        def callback(event):
+            if event.is_directory:
                 return
-            if allowed_path is not None and event.src_path != allowed_path:
-                return
-            # Text editors always cause a "file close" event in addition to "modified" when saving
-            # a file. Some editors also have "swap" functionality that keeps writing into another
-            # file that's never closed. Prevent such write events from causing a rebuild.
-            if isinstance(event, watchdog.events.FileModifiedEvent):
-                # But FileClosedEvent is implemented only on Linux, otherwise we mustn't skip this:
-                if type(self.observer).__name__ == "InotifyObserver":
-                    return
             log.debug(str(event))
             with self._rebuild_cond:
                 self._to_rebuild[func] = True
                 self._rebuild_cond.notify_all()
 
-        dir_handler = watchdog.events.FileSystemEventHandler()
-        dir_handler.on_any_event = callback
-
-        seen = set()
-
-        def schedule(path):
-            seen.add(path)
-            if path.is_file():
-                # Watchdog doesn't support watching files, so watch its directory and filter by path
-                handler = watchdog.events.FileSystemEventHandler()
-                handler.on_any_event = lambda event: callback(event, allowed_path=os.fspath(path))
-
-                parent = path.parent
-                log.debug(f"Watching file '{path}' through directory '{parent}'")
-                self.observer.schedule(handler, parent)
-            else:
-                log.debug(f"Watching directory '{path}'")
-                self.observer.schedule(dir_handler, path, recursive=recursive)
-
-        schedule(pathlib.Path(path).resolve())
-
-        def watch_symlink_targets(path_obj):  # path is os.DirEntry or pathlib.Path
-            if path_obj.is_symlink():
-                path_obj = pathlib.Path(path_obj).resolve()
-                if path_obj in seen or not path_obj.exists():
-                    return
-                schedule(path_obj)
-
-            if path_obj.is_dir() and recursive:
-                with os.scandir(os.fspath(path_obj)) as scan:
-                    for entry in scan:
-                        watch_symlink_targets(entry)
-
-        watch_symlink_targets(pathlib.Path(path))
+        handler = watchdog.events.FileSystemEventHandler()
+        handler.on_any_event = callback
+        log.debug(f"Watching '{path}'")
+        self.observer.schedule(handler, path, recursive=recursive)
 
     def serve(self):
         self.observer.start()


Test patch:
diff --git a/mkdocs/tests/livereload_tests.py b/mkdocs/tests/livereload_tests.py
index c4d256973c..c7450587e6 100644
--- a/mkdocs/tests/livereload_tests.py
+++ b/mkdocs/tests/livereload_tests.py
@@ -38,7 +38,7 @@ def testing_server(root, builder=lambda: None, mount_path="/"):
             port=0,
             root=root,
             mount_path=mount_path,
-            build_delay=0.1,
+            polling_interval=0.2,
             bind_and_activate=False,
         )
         server.setup_environ()
@@ -146,7 +146,7 @@ def test_rebuild_after_rename(self, site_dir):
             self.assertTrue(started_building.wait(timeout=10))
 
     @tempdir()
-    def test_no_rebuild_on_edit(self, site_dir):
+    def test_rebuild_on_edit(self, site_dir):
         started_building = threading.Event()
 
         with open(Path(site_dir, "test"), "wb") as f:
@@ -159,7 +159,7 @@ def test_no_rebuild_on_edit(self, site_dir):
                 f.write(b"hi\n")
                 f.flush()
 
-                self.assertFalse(started_building.wait(timeout=0.2))
+                self.assertTrue(started_building.wait(timeout=10))
 
     @tempdir({"foo.docs": "a"})
     @tempdir({"foo.site": "original"})
@@ -448,15 +448,15 @@ def wait_for_build():
             server.watch(Path(origin_dir, "mkdocs.yml"))
             time.sleep(0.01)
 
+            Path(origin_dir, "unrelated.md").write_text("foo")
+            self.assertFalse(started_building.wait(timeout=0.5))
+
             Path(tmp_dir, "mkdocs.yml").write_text("edited")
             self.assertTrue(wait_for_build())
 
             Path(dest_docs_dir, "subdir", "foo.md").write_text("edited")
             self.assertTrue(wait_for_build())
 
-            Path(origin_dir, "unrelated.md").write_text("foo")
-            self.assertFalse(started_building.wait(timeout=0.2))
-
     @tempdir(["file_dest_1.md", "file_dest_2.md", "file_dest_unused.md"], prefix="tmp_dir")
     @tempdir(["file_under.md"], prefix="dir_to_link_to")
     @tempdir()
@@ -495,7 +495,7 @@ def wait_for_build():
             self.assertTrue(wait_for_build())
 
             Path(tmp_dir, "file_dest_unused.md").write_text("edited")
-            self.assertFalse(started_building.wait(timeout=0.2))
+            self.assertFalse(started_building.wait(timeout=0.5))
 
     @tempdir(prefix="site_dir")
     @tempdir(["docs/unused.md", "README.md"], prefix="origin_dir")