Repository: mkdocs/mkdocs

Issue:
Modify stale files warning to ignore .htaccess, .gitignore and .git
I am not sure if this is the same as issue #167 or not, or if there a better workaround than the setup I am using.

I have the mkdocs project in one folder, and I am building to a sibling folder ( `site_dir: ../site` )
My site dir contains its own git repo, and .htaccess and .gitignore, which I want to keep there, but I get that stale warning.

This causes two issues:
1. Someone may actually run this clean command and delete files I want to keep
2. I cannot run `--clean`

Perhaps the behavior of that warning (and of `--clean`) should be changed to accommodate such scenario? 

If it cannot be solved automatically, then perhaps by a directive in the config? `build_ignore: [file, folder]`



Discussion:
Related - maybe we should ignore '.[something]'  files in the docs dir (ie. don't copy them across to the built site)

Wait - will it now copy any file in mkdocs to the build folder? This is a bigger problem then. `.gitignore` files are different, and I sure hope it will not copy and overwrite my `.git` folder... (but so far, I am loving mkdocs, the simplicity and thought put into it is amazing).

You will need to be careful here, we copy everything in the `docs_dir` (defaults to `docs`) to the `site_dir`, unless it is markdown (then it is processed first) or HTML. We also expect to be building to a directory that MkDocs fully controlls. If you want to merge it into another I'd suggest that you build to a temp location and then move the result into the final location.

#167 is related, at the moment we warn you if files are there. With #167 being done we would warn you only if files are going to be overwritten.

Otherwise, yeah, I would be +1 for not copying .dotfiles that are in the `docs_dir`. Should that only be at the top level or in the full tree? Thinking of things like `.DS_Store` that would be all over.

Everywhere.

I see. This definitely needs to be addressed.... I guess I am not the only one following the practice of building the docs to a separate "docroot" with its own git repo that is intended to be pushed as is, directly to production. If there is any other way for me to overcome the issue, I am of course open to it. For now, I will try the recommendation I hear from you - build directly to a clean folder, copy manually to my docroot.

I would hope that you don't put anything in your `docs_dir` that you don't want in production... Typically the git repository for the documentation would be in the same location as the `mkdocs.yml` and thus up one level from the `docs_dir`. But yeah, better safe than sorry.

Yeah, absolutely. The .git is at the same level as the config file. But just to avoid confusion and to align myself with how it was designed to work now, I have reverted to build to the default "site" folder, and I will copy all manually to the actual docroot. No big deal at all.

Great, I do agree that it isn't ideal but glad it is workable for now.


Resolving patch:
diff --git a/mkdocs/utils.py b/mkdocs/utils.py
--- a/mkdocs/utils.py
+++ b/mkdocs/utils.py
@@ -54,6 +54,13 @@ def copy_media_files(from_dir, to_dir):
         relative_path = os.path.relpath(source_dir, from_dir)
         output_dir = os.path.normpath(os.path.join(to_dir, relative_path))
 
+        # Filter filenames starting with a '.'
+        filenames = [f for f in filenames if not f.startswith('.')]
+
+        # Filter the dirnames that start with a '.' and update the list in
+        # place to prevent us walking these.
+        dirnames[:] = [d for d in dirnames if not d.startswith('.')]
+
         for filename in filenames:
             if not is_markdown_file(filename) and not is_html_file(filename):
                 source_path = os.path.join(source_dir, filename)


Test patch:
diff --git a/mkdocs/test.py b/mkdocs/test.py
--- a/mkdocs/test.py
+++ b/mkdocs/test.py
@@ -596,6 +596,34 @@ def test_markdown_duplicate_custom_extension(self):
         html_ext, _, _ = build.convert_markdown(md_input, ['toc'])
         self.assertEqual(html_ext.strip(), '<p>foo</p>')
 
+    def test_copying_media(self):
+
+        docs_dir = tempfile.mkdtemp()
+        site_dir = tempfile.mkdtemp()
+        try:
+            # Create a markdown file, image, dot file and dot directory.
+            open(os.path.join(docs_dir, 'index.md'), 'w').close()
+            open(os.path.join(docs_dir, 'img.jpg'), 'w').close()
+            open(os.path.join(docs_dir, '.hidden'), 'w').close()
+            os.mkdir(os.path.join(docs_dir, '.git'))
+            open(os.path.join(docs_dir, '.git/hidden'), 'w').close()
+
+            conf = config.validate_config({
+                'site_name': 'Example',
+                'docs_dir': docs_dir,
+                'site_dir': site_dir
+            })
+            build.build(conf)
+
+            # Verify only the markdown (coverted to html) and the image are copied.
+            self.assertTrue(os.path.isfile(os.path.join(site_dir, 'index.html')))
+            self.assertTrue(os.path.isfile(os.path.join(site_dir, 'img.jpg')))
+            self.assertFalse(os.path.isfile(os.path.join(site_dir, '.hidden')))
+            self.assertFalse(os.path.isfile(os.path.join(site_dir, '.git/hidden')))
+        finally:
+            shutil.rmtree(docs_dir)
+            shutil.rmtree(site_dir)
+
 # class IntegrationTests(unittest.TestCase):
 #     def test_mkdocs_site(self):
 #         """