Repository: getnikola/nikola

Issue:
Make github pages deploy seamless
I am not knowledgeable enough of git to handle this, all recipes I've seen look like a complicated mess (but it's all on me! really!).

So, let's fix it the right way: why not have a `nikola github-deploy` command that does The Right Thing (tm) using just a github username or whatever in conf.py?

Looking forward to it :-)



Discussion:
I think the easiest an cleanest way is to use [git-subtree](https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt). For example, suppose your nikola output folder is `output` and your remote repo on github is named `origin`, this is all you'll need: 

``` bash
git subtree push --prefix output origin gh-pages
```

It means: 

> push the subree with the prefix output (i.e the folder `output`) as a branch called `gh-pages` (the branch Github Pages statically serves for project repos) in the repository `origin`. 

Pros: 
- You don't need to handle any dark git plumbing like copy-tree. Moreover, you don't need local branches at all!
- It's so obvious than doesn't seem git

Cons: 
- The only drawback is that `git-subtree` isn't built-in in git until the version 1.7.11 (not very recent one, but "still there" . However it's installable since 1.5+, it's a [simple bash script](https://github.com/git/git/blob/master/contrib/subtree/git-subtree.sh): put it as `git-subtree` with +x in your path, and voilá. 

tiny addendum: 
- some people doing pretty the same for a similar purpose https://gist.github.com/cobyism/4730490
- for user/organization pages (like my abandoned [blog](https://github.com/mgaitan/mgaitan.github.com/) the branch Github Pages serves is `master` instead of `gh-page`. So, the "nikola root blog" must to live in another branch, and be pushed using `git subtree push --prefix output origin master`

I agree with @mgaitan, for reference check my deploy commands:

https://github.com/damianavila/damian_blog/blob/master/conf.py#L202

or the last update of this post I wrote some time ago:

http://www.damian.oquanta.info/posts/one-line-deployment-of-your-site-to-gh-pages.html

Pretty easy in this way :wink: 

I'm partial to my approach https://github.com/asmeurer/blog/blob/master/conf.py#L241. Just rsyncs to gh-pages. No git magic. Very clean separation of source and rendered pages in git. 

I think regardless you need different strategies for username.github.io pages and gh-pages. 

By the way, github doesn't even work with older versions of git. 

With that being said, the rsync itself is a bit fragile. If you get the excludes wrong, it will wipe the .git directory. It's probably possible to do it using only git, though. 

I tried the subtree approach, but found that I don't like having my output tracked by git.  
So, FWIW, here's my new version of deploy commands. 
https://github.com/punchagan/mumbaiultimate.in/blob/master/conf.py#L178

A lot of ways to achieve the same objective... nice to see the other approaches.

@punchagan why don't you just use git add -A? Other than that, I like your approach of pulling in things with git checkout better than mine. 

If you are going to use `git add -A` make sure first there's no random stuff on output via `nikola orphans` or something similar.

So... a dozen comments. Anyone writing this? :-)


Resolving patch:
diff --git a/nikola/plugins/command/github_deploy.py b/nikola/plugins/command/github_deploy.py
new file mode 100644
--- /dev/null
+++ b/nikola/plugins/command/github_deploy.py
@@ -0,0 +1,248 @@
+# -*- coding: utf-8 -*-
+
+# Copyright © 2014 Puneeth Chaganti and others.
+
+# Permission is hereby granted, free of charge, to any
+# person obtaining a copy of this software and associated
+# documentation files (the "Software"), to deal in the
+# Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the
+# Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice
+# shall be included in all copies or substantial portions of
+# the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
+# OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+from __future__ import print_function
+import os
+import shutil
+import subprocess
+import sys
+from textwrap import dedent
+
+from nikola.plugin_categories import Command
+from nikola.plugins.command.check import real_scan_files
+from nikola.utils import ask_yesno, get_logger
+from nikola.__main__ import main
+from nikola import __version__
+
+
+class CommandGitHubDeploy(Command):
+    """ Deploy site to GitHub pages. """
+    name = 'github_deploy'
+
+    doc_usage = ''
+    doc_purpose = 'deploy the site to GitHub pages'
+    doc_description = dedent(
+        """\
+        This command can be used to deploy your site to GitHub pages.
+        It performs the following actions:
+
+        1. Ensure that your site is a git repository, and git is on the PATH.
+        2. Check for changes, and prompt the user to continue, if required.
+        3. Build the site
+        4. Clean any files that are "unknown" to Nikola.
+        5. Create a deploy branch, if one doesn't exist.
+        6. Commit the output to this branch.  (NOTE: Any untracked source
+           files, may get committed at this stage, on the wrong branch!)
+        7. Push and deploy!
+
+        NOTE: This command needs your site to be a git repository, with a
+        master branch (or a different branch, configured using
+        GITHUB_SOURCE_BRANCH if you are pushing to user.github
+        .io/organization.github.io pages) containing the sources of your
+        site.  You also, obviously, need to have `git` on your PATH,
+        and should be able to push to the repository specified as the remote
+        (origin, by default).
+        """
+    )
+
+    logger = None
+
+    _deploy_branch = ''
+    _source_branch = ''
+    _remote_name = ''
+
+    def _execute(self, command, args):
+
+        self.logger = get_logger(
+            CommandGitHubDeploy.name, self.site.loghandlers
+        )
+        self._source_branch = self.site.config.get(
+            'GITHUB_SOURCE_BRANCH', 'master'
+        )
+        self._deploy_branch = self.site.config.get(
+            'GITHUB_DEPLOY_BRANCH', 'gh-pages'
+        )
+        self._remote_name = self.site.config.get(
+            'GITHUB_REMOTE_NAME', 'origin'
+        )
+
+        self._ensure_git_repo()
+
+        if not self._prompt_continue():
+            return
+
+        build = main(['build'])
+        if build != 0:
+            self.logger.error('Build failed, not deploying to GitHub')
+            sys.exit(build)
+
+        only_on_output, _ = real_scan_files(self.site)
+        for f in only_on_output:
+            os.unlink(f)
+
+        self._checkout_deploy_branch()
+
+        self._copy_output()
+
+        self._commit_and_push()
+
+        return
+
+    def _commit_and_push(self):
+        """ Commit all the files and push. """
+
+        deploy = self._deploy_branch
+        source = self._source_branch
+        remote = self._remote_name
+
+        source_commit = subprocess.check_output(['git', 'rev-parse', source])
+        commit_message = (
+            'Nikola auto commit.\n\n'
+            'Source commit: %s'
+            'Nikola version: %s' % (source_commit, __version__)
+        )
+
+        commands = [
+            ['git', 'add', '-A'],
+            ['git', 'commit', '-m', commit_message],
+            ['git', 'push', '-f', remote, '%s:%s' % (deploy, deploy)],
+            ['git', 'checkout', source],
+        ]
+
+        for command in commands:
+            self.logger.info("==> {0}".format(command))
+            try:
+                subprocess.check_call(command)
+            except subprocess.CalledProcessError as e:
+                self.logger.error(
+                    'Failed GitHub deployment — command {0} '
+                    'returned {1}'.format(e.cmd, e.returncode)
+                )
+                sys.exit(e.returncode)
+
+    def _copy_output(self):
+        """ Copy all output to the top level directory. """
+        output_folder = self.site.config['OUTPUT_FOLDER']
+        for each in os.listdir(output_folder):
+            if os.path.exists(each):
+                if os.path.isdir(each):
+                    shutil.rmtree(each)
+
+                else:
+                    os.unlink(each)
+
+            shutil.move(os.path.join(output_folder, each), '.')
+
+    def _checkout_deploy_branch(self):
+        """ Check out the deploy branch
+
+        Creates an orphan branch if not present.
+
+        """
+
+        deploy = self._deploy_branch
+
+        try:
+            subprocess.check_call(
+                [
+                    'git', 'show-ref', '--verify', '--quiet',
+                    'refs/heads/%s' % deploy
+                ]
+            )
+        except subprocess.CalledProcessError:
+            self._create_orphan_deploy_branch()
+        else:
+            subprocess.check_call(['git', 'checkout', deploy])
+
+    def _create_orphan_deploy_branch(self):
+        """ Create an orphan deploy branch """
+
+        result = subprocess.check_call(
+            ['git', 'checkout', '--orphan', self._deploy_branch]
+        )
+        if result != 0:
+            self.logger.error('Failed to create a deploy branch')
+            sys.exit(1)
+
+        result = subprocess.check_call(['git', 'rm', '-rf', '.'])
+        if result != 0:
+            self.logger.error('Failed to create a deploy branch')
+            sys.exit(1)
+
+        with open('.gitignore', 'w') as f:
+            f.write('%s\n' % self.site.config['OUTPUT_FOLDER'])
+            f.write('%s\n' % self.site.config['CACHE_FOLDER'])
+            f.write('*.pyc\n')
+            f.write('*.db\n')
+
+        subprocess.check_call(['git', 'add', '.gitignore'])
+        subprocess.check_call(['git', 'commit', '-m', 'Add .gitignore'])
+
+    def _ensure_git_repo(self):
+        """ Ensure that the site is a git-repo.
+
+        Also make sure that a remote with the specified name exists.
+
+        """
+
+        try:
+            remotes = subprocess.check_output(['git', 'remote'])
+
+        except subprocess.CalledProcessError as e:
+            self.logger.notice('github_deploy needs a git repository!')
+            sys.exit(e.returncode)
+
+        except OSError as e:
+            import errno
+            self.logger.error('Running git failed with {0}'.format(e))
+            if e.errno == errno.ENOENT:
+                self.logger.notice('Is git on the PATH?')
+            sys.exit(1)
+
+        else:
+            if self._remote_name not in remotes:
+                self.logger.error(
+                    'Need a remote called "%s" configured' % self._remote_name
+                )
+                sys.exit(1)
+
+    def _prompt_continue(self):
+        """ Show uncommitted changes, and ask if user wants to continue. """
+
+        changes = subprocess.check_output(['git', 'status', '--porcelain'])
+        if changes.strip():
+            changes = subprocess.check_output(['git', 'status']).strip()
+            message = (
+                "You have the following changes:\n%s\n\n"
+                "Anything not committed, and unknown to Nikola may be lost, "
+                "or committed onto the wrong branch. Do you wish to continue?"
+            ) % changes
+            proceed = ask_yesno(message, False)
+
+        else:
+            proceed = True
+
+        return proceed
diff --git a/nikola/plugins/command/init.py b/nikola/plugins/command/init.py
--- a/nikola/plugins/command/init.py
+++ b/nikola/plugins/command/init.py
@@ -26,23 +26,23 @@
 
 from __future__ import print_function
 import os
-import sys
 import shutil
 import codecs
 import json
 import textwrap
 import datetime
-import dateutil.tz
 
+import dateutil.tz
 from mako.template import Template
 from pkg_resources import resource_filename
 
 import nikola
 from nikola.nikola import DEFAULT_TRANSLATIONS_PATTERN, DEFAULT_INDEX_READ_MORE_LINK, DEFAULT_RSS_READ_MORE_LINK, LEGAL_VALUES
 from nikola.plugin_categories import Command
-from nikola.utils import get_logger, makedirs, STDERR_HANDLER, load_messages
+from nikola.utils import ask, ask_yesno, get_logger, makedirs, STDERR_HANDLER, load_messages
 from nikola.winutils import fix_git_symlinked
 
+
 LOGGER = get_logger('init', STDERR_HANDLER)
 
 SAMPLE_CONF = {
@@ -254,30 +254,6 @@ def create_empty_site(cls, target):
     @staticmethod
     def ask_questions(target):
         """Ask some questions about Nikola."""
-        def ask(query, default):
-            """Ask a question."""
-            if default:
-                default_q = ' [{0}]'.format(default)
-            else:
-                default_q = ''
-            inpf = raw_input if sys.version_info[0] == 2 else input
-            inp = inpf("{query}{default_q}: ".format(query=query, default_q=default_q)).strip()
-            return inp if inp else default
-
-        def ask_yesno(query, default=None):
-            if default is None:
-                default_q = ' [y/n]'
-            elif default is True:
-                default_q = ' [Y/n]'
-            elif default is False:
-                default_q = ' [y/N]'
-            inpf = raw_input if sys.version_info[0] == 2 else input
-            inp = inpf("{query}{default_q} ".format(query=query, default_q=default_q)).strip()
-            if inp:
-                return inp.lower().startswith('y')
-            else:
-                return default
-
         def lhandler(default, toconf, show_header=True):
             if show_header:
                 print("We will now ask you to provide the list of languages you want to use.")
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -145,6 +145,7 @@ def req_missing(names, purpose, python=True, optional=False):
     bytes_str = bytes
     unicode_str = str
     unichr = chr
+    raw_input = input
     from imp import reload as _reload
 else:
     bytes_str = str
@@ -164,7 +165,8 @@ def req_missing(names, purpose, python=True, optional=False):
            '_reload', 'unicode_str', 'bytes_str', 'unichr', 'Functionary',
            'TranslatableSetting', 'TemplateHookRegistry', 'LocaleBorg',
            'sys_encode', 'sys_decode', 'makedirs', 'get_parent_theme_name',
-           'demote_headers', 'get_translation_candidate', 'write_metadata']
+           'demote_headers', 'get_translation_candidate', 'write_metadata',
+           'ask', 'ask_yesno']
 
 # Are you looking for 'generic_rss_renderer'?
 # It's defined in nikola.nikola.Nikola (the site object).
@@ -1144,3 +1146,34 @@ def write_metadata(data):
     meta.append('')
 
     return '\n'.join(meta)
+
+
+def ask(query, default=None):
+    """Ask a question."""
+    if default:
+        default_q = ' [{0}]'.format(default)
+    else:
+        default_q = ''
+    inp = raw_input("{query}{default_q}: ".format(query=query, default_q=default_q)).strip()
+    if inp or default is None:
+        return inp
+    else:
+        return default
+
+
+def ask_yesno(query, default=None):
+    """Ask a yes/no question."""
+    if default is None:
+        default_q = ' [y/n]'
+    elif default is True:
+        default_q = ' [Y/n]'
+    elif default is False:
+        default_q = ' [y/N]'
+    inp = raw_input("{query}{default_q} ".format(query=query, default_q=default_q)).strip()
+    if inp:
+        return inp.lower().startswith('y')
+    elif default is not None:
+        return default
+    else:
+        # Loop if no answer and no default.
+        return ask_yesno(query, default)


Test patch: