Repository: spack/spack

Issue:
Git tags not pulled in full before checkout
For repositories with tag commits that aren't reachable from the current head, the checkout of a tag in the git fetch strategy can fail.  This came up with the Go repository, which doesn't get its release version tags with the initial clone.

Two options, I'll put in either one but though it best to get some feedback first:
1. Depend on using `--branch`, this requires a newer version of git, but allows it to put `--depth 1` and drop download bandwidth and time by a significant amount
2. Add a `pull --tags` before the checkout.  In principle `fetch --tags` would also work, but the behavior is dependent on git version.  Either way, this completely prevents using a shallow clone.

Speaking of, is there a reason `--depth 1` isn't in the standard flags?



Discussion:
@trws: I'm pro option 2.  I'd rather use the new features, but I already had to back `--branch` out for compatibility -- see b2f46c7 or SPACK-87 in LC JIRA.  One thing I wouldn't be opposed to, though, is querying the git version, and doing something better if the version is newer.

Spack could really use a notion of "fetch dependencies" and the ability to balance between finding a builtin version of git or building a newer one.  Lacking that I think option 2 is better for compatibility.

For `--depth 1`, I think it's not there because I wasn't sure what versions of git support it. If it is supported back to 1.7 or 1.6, I think I would be ok with it.  But we still have machines around here with really old default git versions, and I think other sites have older ones.

That makes sense.  The `--depth` option has been around since at least 1.7.1.  I don't have an older version handy, or the documentation for that matter, to check any farther back.  Unfortunately using the fetch method to get tags will result in the full history being grabbed in order to resolve the tags, but I don't know a way to avoid that if `--branch` support can't be expected.

What do you think of having it try `clone --branch`, and if it gets an exit code have it retry without the `--branch` and do the `pull --tags` after?  It seems to end with an exit code of 129 on invalid argument, and 128 on failure to find something, but `--branch` may just fail to find a branch of the given tag name, so it would probably have to just be a check for non-zero or either of those values.

When did `--branch` start working?  You can always check the git version first (I think the fetch strategy already has a git_version() method on it) and use `--branch` if it's new enough...

I'm not sure when it started working for tags, but I can find out.

On 12 Jun 2015, at 17:10, Todd Gamblin wrote:

> When did `--branch` start working?  You can always check the git 
> version first (I think the fetch strategy already has a git_version() 
> method on it) and use `--branch` if it's new enough...
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/scalability-llnl/spack/issues/64#issuecomment-111645172

Kind of like this:  https://github.com/scalability-llnl/spack/blob/develop/lib/spack/spack/fetch_strategy.py#L423


Resolving patch:
diff --git a/lib/spack/spack/fetch_strategy.py b/lib/spack/spack/fetch_strategy.py
--- a/lib/spack/spack/fetch_strategy.py
+++ b/lib/spack/spack/fetch_strategy.py
@@ -417,12 +417,18 @@ def fetch(self):
             # If we want a particular branch ask for it.
             if self.branch:
                 args.extend(['--branch', self.branch])
+            elif self.tag and self.git_version >= ver('1.8.5.2'):
+                    args.extend(['--branch', self.tag])
 
             # Try to be efficient if we're using a new enough git.
             # This checks out only one branch's history
             if self.git_version > ver('1.7.10'):
                 args.append('--single-branch')
 
+            # Yet more efficiency, only download a 1-commit deep tree
+            if self.git_version >= ver('1.7.1'):
+                args.extend(['--depth','1'])
+
             args.append(self.url)
             self.git(*args)
             self.stage.chdir_to_source()
@@ -430,7 +436,8 @@ def fetch(self):
             # For tags, be conservative and check them out AFTER
             # cloning.  Later git versions can do this with clone
             # --branch, but older ones fail.
-            if self.tag:
+            if self.tag and self.git_version < ver('1.8.5.2'):
+                self.git('pull', '--tags')
                 self.git('checkout', self.tag)
 
 


Test patch: