Repository: getpelican/pelican

Issue:
clean should not remove .git metadata
I am deploying my pelican-generated site via heroku, so the target directory is a git repository. make clean deletes everything at the output directory, so I had to hack up my makefile to stop that behaviour. 



Discussion:
@msoulier Can you share your solution please (or point me to the source)?

@amanjeev 

``` make
clean:
    find $(OUTPUTDIR) -mindepth 1 -name '.git*' -prune -o -delete
```

This will also exclude `.gitignore`, `.gitconfig` etc.

@rupert: In my testing, those parameters to `find` preserves the .git directory itself but nukes everything inside it.

Ah sorry, I've updated the `find` command above.

``` bash
$ find output
output
output/.git
output/.git/hello
output/.gitignore
output/foo
output/foo/bar
output/hello
$ find output -mindepth 1 -name '.git*' -prune -o -print
output/foo
output/foo/bar
output/hello
```

Great. Any idea as to the full command to delete everything but the `.git` directory and its contents?

Delete everything except `.git`:

``` bash
$ find $(OUTPUTDIR) -mindepth 1 -name .git -prune -o -delete
```

Delete everything except `.git`, `.gitignore` etc: 

``` bash
$ find $(OUTPUTDIR) -mindepth 1 -name '.git*' -prune -o -delete
```

In my testing, those commands do the same as I noted previously: preserves the .git directory itself but nukes everything inside it.

I wrote the commands for Mac OS X `find`. I've since found that the GNU version of `find` doesn't support using `-prune` with `-delete`.

@justinmayer, can you try:

``` bash
$ find $(OUTPUTDIR) -mindepth 1 -name '.git*' -prune -o -print0 | xargs -0 rm -rf
```

``` bash
$ find output
output
output/.git
output/.git/hello
output/.gitignore
output/foo
output/foo/bar
output/hello
$ make clean
find /Users/rupert/rupertb.com/output -mindepth 1 -name '.git*' -prune -o -print0 | xargs -0 rm -rf
$ find output
output
output/.git
output/.git/hello
output/.gitignore
```

I actually did my testing on Mac OS X 10.8.2 to produce the behavior described above.

I thought about `xargs`, but once we get to the point of piping things to `rm -rf`, I think we've crossed the line over into... a potentially bad place. ;^)

Honestly, if folks want to avoid nuking `.git`, `.hg`, and other files/directories in the output folder, they are probably better off installing and using `ack` instead of wrangling with `find`. Perhaps someone would care to write up an appropriate line for the Makefile using `ack` and then add it to our [Tips](https://github.com/getpelican/pelican/blob/master/docs/tips.rst) page?


Resolving patch:
diff --git a/pelican/__init__.py b/pelican/__init__.py
--- a/pelican/__init__.py
+++ b/pelican/__init__.py
@@ -49,6 +49,7 @@ def __init__(self, settings):
         self.markup = settings['MARKUP']
         self.ignore_files = settings['IGNORE_FILES']
         self.delete_outputdir = settings['DELETE_OUTPUT_DIRECTORY']
+        self.output_retention = settings['OUTPUT_RETENTION']
 
         self.init_path()
         self.init_plugins()
@@ -175,7 +176,7 @@ def run(self):
         # explicitely asked
         if (self.delete_outputdir and not
                 os.path.realpath(self.path).startswith(self.output_path)):
-            clean_output_dir(self.output_path)
+            clean_output_dir(self.output_path, self.output_retention)
 
         writer = self.get_writer()
 
diff --git a/pelican/settings.py b/pelican/settings.py
--- a/pelican/settings.py
+++ b/pelican/settings.py
@@ -54,6 +54,7 @@
     'NEWEST_FIRST_ARCHIVES': True,
     'REVERSE_CATEGORY_ORDER': False,
     'DELETE_OUTPUT_DIRECTORY': False,
+    'OUTPUT_RETENTION': (),
     'ARTICLE_URL': '{slug}.html',
     'ARTICLE_SAVE_AS': '{slug}.html',
     'ARTICLE_LANG_URL': '{slug}-{lang}.html',
diff --git a/pelican/utils.py b/pelican/utils.py
--- a/pelican/utils.py
+++ b/pelican/utils.py
@@ -298,8 +298,8 @@ def copy(path, source, destination, destination_path=None, overwrite=False):
         logger.warning('skipped copy %s to %s' % (source_, destination_))
 
 
-def clean_output_dir(path):
-    """Remove all the files from the output directory"""
+def clean_output_dir(path, retention):
+    """Remove all files from output directory except those in retention list"""
 
     if not os.path.exists(path):
         logger.debug("Directory already removed: %s" % path)
@@ -312,10 +312,13 @@ def clean_output_dir(path):
             logger.error("Unable to delete file %s; %s" % (path, str(e)))
         return
 
-    # remove all the existing content from the output folder
+    # remove existing content from output folder unless in retention list
     for filename in os.listdir(path):
         file = os.path.join(path, filename)
-        if os.path.isdir(file):
+        if any(filename == retain for retain in retention):
+            logger.debug("Skipping deletion; %s is on retention list: %s" \
+                         % (filename, file))
+        elif os.path.isdir(file):
             try:
                 shutil.rmtree(file)
                 logger.debug("Deleted directory %s" % file)


Test patch:
diff --git a/pelican/tests/test_utils.py b/pelican/tests/test_utils.py
--- a/pelican/tests/test_utils.py
+++ b/pelican/tests/test_utils.py
@@ -193,28 +193,31 @@ def test_watchers(self):
             shutil.rmtree(empty_path, True)
 
     def test_clean_output_dir(self):
+        retention = ()
         test_directory = os.path.join(os.path.dirname(__file__),
                                       'clean_output')
         content = os.path.join(os.path.dirname(__file__), 'content')
         shutil.copytree(content, test_directory)
-        utils.clean_output_dir(test_directory)
+        utils.clean_output_dir(test_directory, retention)
         self.assertTrue(os.path.isdir(test_directory))
         self.assertListEqual([], os.listdir(test_directory))
         shutil.rmtree(test_directory)
 
     def test_clean_output_dir_not_there(self):
+        retention = ()
         test_directory = os.path.join(os.path.dirname(__file__),
                                       'does_not_exist')
-        utils.clean_output_dir(test_directory)
+        utils.clean_output_dir(test_directory, retention)
         self.assertFalse(os.path.exists(test_directory))
 
     def test_clean_output_dir_is_file(self):
+        retention = ()
         test_directory = os.path.join(os.path.dirname(__file__),
                                       'this_is_a_file')
         f = open(test_directory, 'w')
         f.write('')
         f.close()
-        utils.clean_output_dir(test_directory)
+        utils.clean_output_dir(test_directory, retention)
         self.assertFalse(os.path.exists(test_directory))
 
     def test_strftime(self):