Repository: ansible/ansible

Issue:
Support `--separate-git-dir` option in Git module
##### SUMMARY
<!--- Explain the problem briefly -->
This is a follow up for https://github.com/ansible/ansible/pull/38016#pullrequestreview-118799688:
Git module should support `--separate-git-dir`:
* we need to support `separate-git-dir` module argument
* have docs/examples for that
* cover all of that with tests including negative ones

##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
git
##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
devel
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
N/A
##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
N/A
##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->
```yaml
git:
  repo: git://...
  dest: /opt/app/web-app-xxx
  separate-git-dir: /var/db/git/web-app-xxx.git
```
<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
It would run `git clone --separate-git-dir` under the hood (or a fallback for older Git versions)

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
N/A. It doesn't work now, however if cloned manually subsequent updates work as of #38016.


Discussion:
@HD650 I think you could do this

-label needs_triage
@webknjaz rager that!

Resolving patch:
diff --git a/lib/ansible/modules/source_control/git.py b/lib/ansible/modules/source_control/git.py
--- a/lib/ansible/modules/source_control/git.py
+++ b/lib/ansible/modules/source_control/git.py
@@ -161,6 +161,12 @@
               all git servers support git archive.
         version_added: "2.4"
 
+    separate_git_dir:
+        description:
+            - The path to place the cloned repository. If specified, Git repository
+              can be separated from working tree.
+        version_added: "2.7"
+
 requirements:
     - git>=1.7.1 (the command line tool)
 
@@ -210,6 +216,11 @@
     dest: /src/ansible-examples
     archive: /tmp/ansible-examples.zip
 
+# Example clone a repo with separate git directory
+- git:
+    repo: https://github.com/ansible/ansible-examples.git
+    dest: /src/ansible-examples
+    separate_git_dir: /src/ansible-examples.git
 '''
 
 RETURN = '''
@@ -233,6 +244,16 @@
     returned: error
     type: string
     sample: Your git version is too old to fully support the depth argument. Falling back to full checkouts.
+git_dir_now:
+    description: Contains the new path of .git directory if it's changed
+    returned: success
+    type: string
+    sample: /path/to/new/git/dir
+git_dir_before:
+    description: Contains the original path of .git directory if it's changed
+    returned: success
+    type: string
+    sample: /path/to/old/git/dir
 '''
 
 import filecmp
@@ -250,6 +271,24 @@
 from ansible.module_utils._text import to_native
 
 
+def relocate_repo(module, result, repo_dir, old_repo_dir, worktree_dir):
+    if os.path.exists(repo_dir):
+        module.fail_json(msg='Separate-git-dir path %s already exists.' % repo_dir)
+    if worktree_dir:
+        dot_git_file_path = os.path.join(worktree_dir, '.git')
+        try:
+            shutil.move(old_repo_dir, repo_dir)
+            with open(dot_git_file_path, 'w') as dot_git_file:
+                dot_git_file.write('gitdir: %s' % repo_dir)
+            result['git_dir_before'] = old_repo_dir
+            result['git_dir_now'] = repo_dir
+        except (IOError, OSError) as err:
+            # if we already moved the .git dir, roll it back
+            if os.path.exists(repo_dir):
+                shutil.move(repo_dir, old_repo_dir)
+            module.fail_json(msg='Unable to move git dir. %s' % str(err))
+
+
 def head_splitter(headfile, remote, module=None, fail_on_error=False):
     '''Extract the head reference'''
     # https://github.com/ansible/ansible-modules-core/pull/907
@@ -404,9 +443,8 @@ def get_submodule_versions(git_path, module, dest, version='HEAD'):
 
 
 def clone(git_path, module, repo, dest, remote, depth, version, bare,
-          reference, refspec, verify_commit):
+          reference, refspec, verify_commit, separate_git_dir, result):
     ''' makes a new git repo if it does not already exist '''
-
     dest_dirname = os.path.dirname(dest)
     try:
         os.makedirs(dest_dirname)
@@ -432,8 +470,23 @@ def clone(git_path, module, repo, dest, remote, depth, version, bare,
                         "HEAD, branches, tags or in combination with refspec.")
     if reference:
         cmd.extend(['--reference', str(reference)])
+    needs_separate_git_dir_fallback = False
+
+    if separate_git_dir:
+        git_version_used = git_version(git_path, module)
+        if git_version_used is None:
+            module.fail_json(msg='Can not find git executable at %s' % git_path)
+        if git_version_used < LooseVersion('1.7.5'):
+            # git before 1.7.5 doesn't have separate-git-dir argument, do fallback
+            needs_separate_git_dir_fallback = True
+        else:
+            cmd.append('--separate-git-dir=%s' % separate_git_dir)
+
     cmd.extend([repo, dest])
     module.run_command(cmd, check_rc=True, cwd=dest_dirname)
+    if needs_separate_git_dir_fallback:
+        relocate_repo(module, result, separate_git_dir, os.path.join(dest, ".git"), dest)
+
     if bare and remote != 'origin':
         module.run_command([git_path, 'remote', 'add', remote, repo], check_rc=True, cwd=dest)
 
@@ -972,7 +1025,9 @@ def main():
             track_submodules=dict(default='no', type='bool'),
             umask=dict(default=None, type='raw'),
             archive=dict(type='path'),
+            separate_git_dir=dict(type='path'),
         ),
+        mutually_exclusive=[('separate_git_dir', 'bare')],
         supports_check_mode=True
     )
 
@@ -993,6 +1048,7 @@ def main():
     ssh_opts = module.params['ssh_opts']
     umask = module.params['umask']
     archive = module.params['archive']
+    separate_git_dir = module.params['separate_git_dir']
 
     result = dict(changed=False, warnings=list())
 
@@ -1023,6 +1079,9 @@ def main():
     # call run_command()
     module.run_command_environ_update = dict(LANG='C', LC_ALL='C', LC_MESSAGES='C', LC_CTYPE='C')
 
+    if separate_git_dir:
+        separate_git_dir = os.path.realpath(separate_git_dir)
+
     gitconfig = None
     if not dest and allow_clone:
         module.fail_json(msg="the destination directory must be specified unless clone=no")
@@ -1030,6 +1089,11 @@ def main():
         dest = os.path.abspath(dest)
         try:
             repo_path = get_repo_path(dest, bare)
+            if separate_git_dir and os.path.exists(repo_path) and separate_git_dir != repo_path:
+                result['changed'] = True
+                if not module.check_mode:
+                    relocate_repo(module, result, separate_git_dir, repo_path, dest)
+                    repo_path = separate_git_dir
         except (IOError, ValueError) as err:
             # No repo path found
             """``.git`` file does not have a valid format for detached Git dir."""
@@ -1073,7 +1137,7 @@ def main():
                     result['diff'] = diff
             module.exit_json(**result)
         # there's no git config, so clone
-        clone(git_path, module, repo, dest, remote, depth, version, bare, reference, refspec, verify_commit)
+        clone(git_path, module, repo, dest, remote, depth, version, bare, reference, refspec, verify_commit, separate_git_dir, result)
     elif not update:
         # Just return having found a repo already in the dest path
         # this does no checking that the repo is the actual repo


Test patch:
diff --git a/test/integration/targets/git/tasks/main.yml b/test/integration/targets/git/tasks/main.yml
--- a/test/integration/targets/git/tasks/main.yml
+++ b/test/integration/targets/git/tasks/main.yml
@@ -37,5 +37,3 @@
 - include_tasks: ambiguous-ref.yml
 - include_tasks: archive.yml
 - include_tasks: separate-git-dir.yml
-  when:
-    - git_version.stdout is version("1.7.5", '>=')
diff --git a/test/integration/targets/git/tasks/separate-git-dir.yml b/test/integration/targets/git/tasks/separate-git-dir.yml
--- a/test/integration/targets/git/tasks/separate-git-dir.yml
+++ b/test/integration/targets/git/tasks/separate-git-dir.yml
@@ -7,17 +7,85 @@
     state: absent
     path: '{{ checkout_dir }}'
 
-- name: create a tempdir for separate git dir
-  local_action: shell mktemp -du
-  register: tempdir
+- name: SEPARATE-GIT-DIR | make a pre-exist repo dir
+  file:
+    state: directory
+    path: '{{ separate_git_dir }}'
 
 - name: SEPARATE-GIT-DIR | clone with a separate git dir
-  command: git clone {{ repo_format1 }} {{ checkout_dir }} --separate-git-dir={{ tempdir.stdout }}
+  git:
+    repo: '{{ repo_format1 }}'
+    dest: '{{ checkout_dir }}'
+    separate_git_dir: '{{ separate_git_dir }}'
+  ignore_errors: yes
+  register: result
+
+- name: SEPARATE-GIT-DIR | the clone will fail due to pre-exist dir
+  assert:
+    that: 'result is failed'
+
+- name: SEPARATE-GIT-DIR | delete pre-exist dir
+  file:
+    state: absent
+    path: '{{ separate_git_dir }}'
+
+- name: SEPARATE-GIT-DIR | clone again with a separate git dir
+  git:
+    repo: '{{ repo_format1 }}'
+    dest: '{{ checkout_dir }}'
+    separate_git_dir: '{{ separate_git_dir }}'
+
+- name: SEPARATE-GIT-DIR | check the stat of git dir
+  stat:
+    path: '{{ separate_git_dir }}'
+  register: stat_result
+
+- name: SEPARATE-GIT-DIR | the git dir should exist
+  assert:
+    that: 'stat_result.stat.exists == True'
 
 - name: SEPARATE-GIT-DIR | update repo the usual way
   git:
-    repo: "{{ repo_format1 }}"
-    dest: "{{ checkout_dir }}"
+    repo: '{{ repo_format1 }}'
+    dest: '{{ checkout_dir }}'
+    separate_git_dir: '{{ separate_git_dir }}'
+  register: result
+
+- name: SEPARATE-GIT-DIR | update should not fail
+  assert:
+    that:
+      - result is not failed
+
+- name: SEPARATE-GIT-DIR | move the git dir to new place
+  git:
+    repo: '{{ repo_format1 }}'
+    dest: '{{ checkout_dir }}'
+    separate_git_dir: '{{ separate_git_dir }}_new'
+  register: result
+
+- name: SEPARATE-GIT-DIR | the movement should not failed
+  assert:
+    that: 'result is not failed'
+
+- name: SEPARATE-GIT-DIR | check the stat of new git dir
+  stat:
+    path: '{{ separate_git_dir }}_new'
+  register: stat_result
+
+- name: SEPARATE-GIT-DIR | the new git dir should exist
+  assert:
+    that: 'stat_result.stat.exists == True'
+
+- name: SEPARATE-GIT-DIR | test the update
+  git:
+    repo: '{{ repo_format1 }}'
+    dest: '{{ checkout_dir }}'
+  register: result
+
+- name: SEPARATE-GIT-DIR | the update should not failed
+  assert:
+    that:
+      - result is not failed
 
 - name: SEPARATE-GIT-DIR | set git dir to non-existent dir
   shell: "echo gitdir: /dev/null/non-existent-dir > .git"
@@ -56,7 +124,7 @@
 - name: SEPARATE-GIT-DIR | clear separate git dir
   file:
     state: absent
-    path: "{{ tempdir.stdout }}"
+    path: "{{ separate_git_dir }}_new"
 
 - name: SEPARATE-GIT-DIR | clear checkout_dir
   file:
diff --git a/test/integration/targets/git/vars/main.yml b/test/integration/targets/git/vars/main.yml
--- a/test/integration/targets/git/vars/main.yml
+++ b/test/integration/targets/git/vars/main.yml
@@ -11,6 +11,7 @@ git_archive_extensions:
 
 checkout_dir: '{{ output_dir }}/git'
 repo_dir: '{{ output_dir }}/local_repos'
+separate_git_dir: '{{ output_dir }}/sep_git_dir'
 repo_format1: 'https://github.com/jimi-c/test_role'
 repo_format2: 'git@github.com:jimi-c/test_role.git'
 repo_format3: 'ssh://git@github.com/jimi-c/test_role.git'