Repository: iterative/dvc

Issue:
Minimalistic way of `dvc pull` usage in ci/cd
I'm using dvc pull in ci pipeline and not so long ago (after a new release) it broke due to lack of git executable in the container (python:3.6-slim-jessie). It was easily fixed by including of additional stage to Dockerfile with git-core installation, but this approach does not meet minimalistic principals of ci. I think that some dvc methods such as dvc pull need to be considered as production-ready with no additional requirements.


Discussion:
Related #1487 . @puhoshville have you seen that ticket? Would be great if you read it and let us know what you think.
It makes sense, because `git` is usually a _buildtime dependency_, but it is hardly used during _run time_.
Also, having `git` as a requirement for `dvc pull` breaks compatibility with the `--no-scm` flag.

Steps to reproduce with Docker:
```dockerfile
FROM python

RUN apt-get remove git --yes
RUN pip install dvc
RUN mkdir -p /tmp/example \
      && mkdir -p /tmp/dvc-storage \
      && cd /tmp/example \
      && dvc init --no-scm \
      && dvc remote add -d dvc-storage /tmp/dvc-storage \
      && echo "hello" > hello \
      && dvc add hello \
      && dvc push \
      && rm -rf .dvc/cache \
      && dvc pull
```
Traceback:
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/git/__init__.py", line 83, in <module>
    refresh()
  File "/usr/local/lib/python3.7/site-packages/git/__init__.py", line 73, in refresh
    if not Git.refresh(path=path):
  File "/usr/local/lib/python3.7/site-packages/git/cmd.py", line 290, in refresh
    raise ImportError(err)
ImportError: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dvc", line 6, in <module>
    from dvc.main import main
  File "/usr/local/lib/python3.7/site-packages/dvc/main.py", line 6, in <module>
    from dvc.cli import parse_args
  File "/usr/local/lib/python3.7/site-packages/dvc/cli.py", line 12, in <module>
    import dvc.command.pkg as pkg
  File "/usr/local/lib/python3.7/site-packages/dvc/command/pkg.py", line 8, in <module>
    from dvc.repo.pkg import PackageParams
  File "/usr/local/lib/python3.7/site-packages/dvc/repo/__init__.py", line 16, in <module>
    class Repo(object):
  File "/usr/local/lib/python3.7/site-packages/dvc/repo/__init__.py", line 35, in Repo
    from dvc.repo.pkg import install_pkg
  File "/usr/local/lib/python3.7/site-packages/dvc/repo/pkg.py", line 5, in <module>
    from git.cmd import Git
  File "/usr/local/lib/python3.7/site-packages/git/__init__.py", line 85, in <module>
    raise ImportError('Failed to initialize: {0}'.format(exc))
ImportError: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet
```
@mroutis Got it! Thank you! Will send a fix ASAP.
@shcheklein I think that this issue is not quite relevant to the dataset storage task. exactly same `ImportError: Failed to initialize: Bad git executable.` I got while tried to load pretrained model to container not dataset.
@mroutis thank you for such a detailed explanation! It's exactly what I was talking about :)
@puhoshville yeah, it is a simple import bug in one of unrelated parts of dvc. I have my patch going, should be ready today. Thanks for the feedback! 🙂 

Resolving patch:
diff --git a/dvc/repo/pkg.py b/dvc/repo/pkg.py
--- a/dvc/repo/pkg.py
+++ b/dvc/repo/pkg.py
@@ -2,12 +2,11 @@
 
 import os
 import shutil
-from git.cmd import Git
 
 import dvc.logger as logger
 from dvc.exceptions import DvcException
 from dvc.stage import Stage
-from dvc.temp_git_repo import TempGitRepo
+from dvc.scm.git.temp_git_repo import TempGitRepo
 
 
 class PackageParams(object):
@@ -60,6 +59,8 @@ class GitPackage(Package):
     DEF_DVC_FILE_PREFIX = "mod_"
 
     def install_or_update(self, parent_repo, pkg_params):
+        from git.cmd import Git
+
         if not self.is_in_root():
             raise DvcException(
                 "This command can be run only from a repository root"
diff --git a/dvc/temp_git_repo.py b/dvc/scm/git/temp_git_repo.py
similarity index 99%
rename from dvc/temp_git_repo.py
rename to dvc/scm/git/temp_git_repo.py
--- a/dvc/temp_git_repo.py
+++ b/dvc/scm/git/temp_git_repo.py
@@ -2,8 +2,6 @@
 import shutil
 import tempfile
 
-import git
-
 from dvc import logger
 from dvc.exceptions import DvcException
 
@@ -70,6 +68,8 @@ def __enter__(self):
         return self
 
     def _clone_to_temp_dir(self):
+        import git
+
         result = tempfile.mktemp(
             prefix=self._tmp_mod_prefix, dir=self.modules_dir
         )


Test patch: