Repository: mkdocs/mkdocs

Issue:
Permission denied when copying a template from a read-only path
There are two things uncommon about my setup that cause the error.

1. I have installed mkdocs using [Nix](https://nixos.org/nix/). Long story short, all the mkdocs files (the ones in `lib/python2.7/site-packages/mkdocs`) have mode 0444 (that is, read-only).
2. I have `theme_dir` set in `mkdocs.yml` and I use it to overwrite one of the theme files, namely `js/highlight.pack.js`.

This is what I get:

~~~~
$ mkdocs build
WARNING -  Config value: 'extra_javascript'. Warning: The following files have been automatically included in the documentation build and will be added to the HTML: highlight/theme/js/highlight.pack.js. This behavior is deprecated. In version 1.0 and later they will need to be explicitly listed in the 'extra_javascript' config setting.
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: <project path>/build/site
Traceback (most recent call last):
  File "/nix/store/2zkwsgan90gl63pqnq01vrdrpf11fm1m-mkdocs-0.16.3/bin/.mkdocs-wrapped", line 12, in <module>
    sys.exit(cli())
  File "/nix/store/cjhms7xja78pbh5gnh9ii7hlxizq2iy7-python2.7-click-6.7/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/cjhms7xja78pbh5gnh9ii7hlxizq2iy7-python2.7-click-6.7/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/nix/store/cjhms7xja78pbh5gnh9ii7hlxizq2iy7-python2.7-click-6.7/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/nix/store/cjhms7xja78pbh5gnh9ii7hlxizq2iy7-python2.7-click-6.7/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/cjhms7xja78pbh5gnh9ii7hlxizq2iy7-python2.7-click-6.7/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/2zkwsgan90gl63pqnq01vrdrpf11fm1m-mkdocs-0.16.3/lib/python2.7/site-packages/mkdocs/__main__.py", line 156, in build_command
    ), dirty=not clean)
  File "/nix/store/2zkwsgan90gl63pqnq01vrdrpf11fm1m-mkdocs-0.16.3/lib/python2.7/site-packages/mkdocs/commands/build.py", line 373, in build
    theme_dir, config['site_dir'], exclude=['*.py', '*.pyc', '*.html'], dirty=dirty
  File "/nix/store/2zkwsgan90gl63pqnq01vrdrpf11fm1m-mkdocs-0.16.3/lib/python2.7/site-packages/mkdocs/utils/__init__.py", line 175, in copy_media_files
    copy_file(source_path, output_path)
  File "/nix/store/2zkwsgan90gl63pqnq01vrdrpf11fm1m-mkdocs-0.16.3/lib/python2.7/site-packages/mkdocs/utils/__init__.py", line 110, in copy_file
    shutil.copy(source_path, output_path)
  File "/nix/store/w8zld7z4gq4b36z0szgrh6yv5zi30915-python-2.7.13/lib/python2.7/shutil.py", line 119, in copy
    copyfile(src, dst)
  File "/nix/store/w8zld7z4gq4b36z0szgrh6yv5zi30915-python-2.7.13/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: u'<project path>/build/site/js/highlight.pack.js'

$ ls -l build/site/js/
total 396
-r--r--r-- 1 kirelagin staff 300764 Sep 26 16:03 highlight.pack.js
-r--r--r-- 1 kirelagin staff  84245 Sep 26 16:03 jquery-2.1.1.min.js
-r--r--r-- 1 kirelagin staff  11084 Sep 26 16:03 modernizr-2.8.3.min.js
-r--r--r-- 1 kirelagin staff   2676 Sep 26 16:03 theme.js

$ ls -ld build/site/js/
drwxr-xr-x 6 kirelagin staff 204 Sep 26 16:03 build/site/js/
~~~~

What happens is, the built-in theme files get copied with their permissions preserved, so `site/js/highlight.pack.js` ends up having mode 0444. Next mkdocs tries to overwrite this file with the one from the `theme_dir` and at this point `shutil.copyfile` fails, because that’s how it works.

I’m not really sure what to do with that. Probably, catching the exception and adjusting the permissions would make sense.


Discussion:
Hmm, I'm not sure how we should deal with this either. That said, it is reasonable to expect that a user could have read-only permissions to any files in site-packages. For example, on a system with multiple users, only the root user might have write access to site-packages and therefore permission to install/update Python libs. Just because the user running MkDocs isn't the root user shouldn't mean they can't have access to all of MkDocs' features. So, yeah, this is a bug.

As a workaround, you could create a virtualenv with your user account and install MkDocs in there. Then you would have full access. You just need to remember to "activate" the venv before running MkDocs.  Actually, I always run MkDocs from a venv; I don't even have it installed as a system Python lib.
So, it appears that the problem is that we [use][1] [shutil.copy][2] rather than [shutil.copyfile][3]. The difference is that  `shutil.copy` also copyies permissions. However, if we used `shutil.copyfile`, then a new file would be created (with permissions for the current user) and only the contents would be copied. As, `shutil.copy` just calls `shutil.copyfile` under the hood (followed by `shutil.copymode` which we don't want), there is no reason why we can't just use `shutil.copyfile`. We might also need to add a line or two to check for directories, which `shutil.copy` [includes][4].

[1]: https://github.com/mkdocs/mkdocs/blob/master/mkdocs/utils/__init__.py#L111
[2]: https://docs.python.org/2/library/shutil.html#shutil.copy
[3]: https://docs.python.org/2/library/shutil.html#shutil.copyfile
[4]: https://github.com/python/cpython/blob/2.7/Lib/shutil.py#L131-L132
Yes, looks like you are right. 👍 

Resolving patch:
diff --git a/mkdocs/utils/__init__.py b/mkdocs/utils/__init__.py
--- a/mkdocs/utils/__init__.py
+++ b/mkdocs/utils/__init__.py
@@ -103,12 +103,15 @@ def reduce_list(data_set):
 def copy_file(source_path, output_path):
     """
     Copy source_path to output_path, making sure any parent directories exist.
-    """
 
+    The output_path may be a directory.
+    """
     output_dir = os.path.dirname(output_path)
     if not os.path.exists(output_dir):
         os.makedirs(output_dir)
-    shutil.copy(source_path, output_path)
+    if os.path.isdir(output_path):
+        output_path = os.path.join(output_path, os.path.basename(source_path))
+    shutil.copyfile(source_path, output_path)
 
 
 def write_file(content, output_path):


Test patch:
diff --git a/mkdocs/tests/utils/utils_tests.py b/mkdocs/tests/utils/utils_tests.py
--- a/mkdocs/tests/utils/utils_tests.py
+++ b/mkdocs/tests/utils/utils_tests.py
@@ -6,6 +6,9 @@
 import mock
 import os
 import unittest
+import tempfile
+import shutil
+import stat
 
 from mkdocs import nav, utils, exceptions
 from mkdocs.tests.base import dedent
@@ -248,3 +251,71 @@ def test_unicode_yaml(self):
         config = utils.yaml_load(yaml_src)
         self.assertTrue(isinstance(config['key'], utils.text_type))
         self.assertTrue(isinstance(config['key2'][0], utils.text_type))
+
+    def test_copy_files(self):
+        src_paths = [
+            'foo.txt',
+            'bar.txt',
+            'baz.txt',
+        ]
+        dst_paths = [
+            'foo.txt',
+            'foo/',             # ensure src filename is appended
+            'foo/bar/baz.txt'   # ensure missing dirs are created
+        ]
+        expected = [
+            'foo.txt',
+            'foo/bar.txt',
+            'foo/bar/baz.txt',
+        ]
+
+        src_dir = tempfile.mkdtemp()
+        dst_dir = tempfile.mkdtemp()
+
+        try:
+            for i, src in enumerate(src_paths):
+                src = os.path.join(src_dir, src)
+                with open(src, 'w') as f:
+                    f.write('content')
+                dst = os.path.join(dst_dir, dst_paths[i])
+                utils.copy_file(src, dst)
+                self.assertTrue(os.path.isfile(os.path.join(dst_dir, expected[i])))
+        finally:
+            shutil.rmtree(src_dir)
+            shutil.rmtree(dst_dir)
+
+    def test_copy_files_without_permissions(self):
+        src_paths = [
+            'foo.txt',
+            'bar.txt',
+            'baz.txt',
+        ]
+        expected = [
+            'foo.txt',
+            'bar.txt',
+            'baz.txt',
+        ]
+
+        src_dir = tempfile.mkdtemp()
+        dst_dir = tempfile.mkdtemp()
+
+        try:
+            for i, src in enumerate(src_paths):
+                src = os.path.join(src_dir, src)
+                with open(src, 'w') as f:
+                    f.write('content')
+                # Set src file to read-only
+                os.chmod(src, stat.S_IRUSR)
+                utils.copy_file(src, dst_dir)
+                self.assertTrue(os.path.isfile(os.path.join(dst_dir, expected[i])))
+                self.assertNotEqual(os.stat(src).st_mode, os.stat(os.path.join(dst_dir, expected[i])).st_mode)
+                # While src was read-only, dst must remain writable
+                self.assertTrue(os.access(os.path.join(dst_dir, expected[i]), os.W_OK))
+        finally:
+            for src in src_paths:
+                # Undo read-only so we can delete temp files
+                src = os.path.join(src_dir, src)
+                if os.path.exists(src):
+                    os.chmod(src, stat.S_IRUSR | stat.S_IWUSR)
+            shutil.rmtree(src_dir)
+            shutil.rmtree(dst_dir)