Repository: iterative/dvc

Issue:
`import`: get from source project cache if `url/path` are in the local system
UPDATE: Jump to https://github.com/iterative/dvc/issues/2599#issuecomment-566346857 (notice also https://github.com/iterative/dvc/issues/2599#issuecomment-700337141)

```
DVC version: 0.62.1
Python version: 3.7.3
Platform: Darwin-18.7.0-x86_64-i386-64bit
Binary: False
Filesystem type (workspace): ('apfs', '/dev/disk1s1')
```

I create a DVC project in `/path/to/data-reg` and `dvc add` a `data/` directory in it.

Then I create another DVC project in `/path/to/project` in the same file system and try to import `data/` but get:

```console
$ dvc import /path/to/data-reg data
Importing 'data (../data-reg-test)' -> 'data'
Missing cache for directory '../../../../private/var/folders/_c/3mt_xn_d4xl2ddsx2m98h_r40000gn/T/tmpkq6tyek3dvc-repo/data'.
Cache for files inside will be lost. Would you like to continue? Use '-f' to force. [y/n] y
ERROR: failed to import 'data' from '/path/to/data-reg'. - config file error: no remote specified. Setup default remote with
    dvc config core.remote <name>
or use:
    dvc pull -r <name>


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
```

A few problems:

- ~~The first part of the message is completely mysterious to me. I don't understand what cache dir is missing and obviously that path isn't friendly.~~
- ~~I also don't know what it's asking me to confirm so I just say `y` and move on 😋~~
- The "no remote specified" is also kind of cryptic (doesn't specify which project is missing a remote) but OK, I get it: I didn't push `data/` to a remote in the first project, so the new project can't download it. However (**main point of this issue**) it could simply copy it from the source project's cache, since they're on the same file system. This would be a nice-to-have feature I think. In fact it should probably try this before the default remote, for performance.
- ~~There's 2 new lines after `dvc pull -r <name>` in the error output.~~

UPDATE: Moved all the UI-related stuff to #2602 (per Ivan's https://github.com/iterative/dvc/issues/2599#issuecomment-541292359 below).


Discussion:
@jorgeorpinel I would probably split this into two separate issues - UI and optimization. It would be easier to track and discuss them.
Let's start with local, as it makes the most sense and is a natural thing to do.  We need to at least show a warning that we are automatically using project's cache as a remote or maybe we need an explicit flag that would tell `import/get` that this is okay to do. On one hand, warning is a low-friction approach, but might be dangerous if user doesn't realize that he is not using a remote and then some colleague on another machine tries to `dvc pull` and there is no remote to pull imported data from. But, in that case the url of the git is a local url, that is not accessible on another machine, so user will run into that anyways. So maybe warning is enough. On another hand, an explicit flag would be a pain on `dvc pull`, as we will have to explicitly allow pulling from cache. Seems like the first approach is better.
> Let's start with local

You mean use local repo and/or cache as default place to get/import from? I would agree with this.

> show a warning that we are automatically using project's cache... might be dangerous if user doesn't realize that he is not using a remote...

Maybe either way output where the import is coming from as a regular INFO message. But honestly, the possibility that people won't pay attention to the output is there anyway. Also, in some other recent discussion about adding output warnings someone brought up that they can break shell scripts and cause other problems unless hidden under the `-v` output.
For the record: to implement this, we need to look into https://github.com/iterative/dvc/blob/0.71.0/dvc/external_repo.py#L68, where we need to check if the url we are cloning from is a local directory and if there is no default remote (core.remote in config) setup, in which case add default remote that will point to the original repo's cache location. Here is a quick test for this feature:
```
#!/bin/bash

set -x
set -e

rm -rf mytest
mkdir mytest
cd mytest

mkdir erepo
pushd erepo
git init
dvc init
echo foo > foo
dvc add foo
git add .
git commit -m "init"
popd

mkdir repo
pushd repo
git init
dvc init
dvc get ../erepo foo
popd
```

Resolving patch:
diff --git a/dvc/command/remote.py b/dvc/command/remote.py
--- a/dvc/command/remote.py
+++ b/dvc/command/remote.py
@@ -19,6 +19,10 @@ def __init__(self, args):
 
 class CmdRemoteAdd(CmdRemoteConfig):
     def run(self):
+        if self.args.default:
+            logger.info(
+                "Setting '{}' as a default remote.".format(self.args.name)
+            )
         self.remote_config.add(
             self.args.name,
             self.args.url,
diff --git a/dvc/external_repo.py b/dvc/external_repo.py
--- a/dvc/external_repo.py
+++ b/dvc/external_repo.py
@@ -7,8 +7,9 @@
 
 from funcy import retry
 
-from dvc.config import NoRemoteError
+from dvc.config import NoRemoteError, ConfigError
 from dvc.exceptions import NoRemoteInExternalRepoError
+from dvc.remote import RemoteConfig
 from dvc.exceptions import NoOutputInExternalRepoError
 from dvc.exceptions import OutputNotFoundError
 from dvc.utils.fs import remove
@@ -69,6 +70,22 @@ def _external_repo(url=None, rev=None, cache_dir=None):
 
     repo = Repo(new_path)
     try:
+        # check if the URL is local and no default remote is present
+        # add default remote pointing to the original repo's cache location
+        if os.path.isdir(url):
+            rconfig = RemoteConfig(repo.config)
+            if not _default_remote_set(rconfig):
+                original_repo = Repo(url)
+                try:
+                    rconfig.add(
+                        "auto-generated-upstream",
+                        original_repo.cache.local.cache_dir,
+                        default=True,
+                        level=Config.LEVEL_LOCAL,
+                    )
+                finally:
+                    original_repo.close()
+
         if cache_dir is not None:
             cache_config = CacheConfig(repo.config)
             cache_config.set_dir(cache_dir, level=Config.LEVEL_LOCAL)
@@ -113,3 +130,19 @@ def _clone_repo(url, path):
 
     git = Git.clone(url, path)
     git.close()
+
+
+def _default_remote_set(rconfig):
+    """
+    Checks if default remote config is present.
+    Args:
+        rconfig: a remote config
+
+    Returns:
+        True if the default remote config is set, else False
+    """
+    try:
+        rconfig.get_default()
+        return True
+    except ConfigError:
+        return False
diff --git a/dvc/remote/config.py b/dvc/remote/config.py
--- a/dvc/remote/config.py
+++ b/dvc/remote/config.py
@@ -105,7 +105,6 @@ def add(self, name, url, default=False, force=False, level=None):
             level=level,
         )
         if default:
-            logger.info("Setting '{}' as a default remote.".format(name))
             self.config.set(
                 Config.SECTION_CORE,
                 Config.SECTION_CORE_REMOTE,


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -7,7 +7,6 @@
 from git import Repo
 from git.exc import GitCommandNotFound
 
-from dvc.remote.config import RemoteConfig
 from dvc.remote.ssh.connection import SSHConnection
 from dvc.repo import Repo as DvcRepo
 from dvc.utils.compat import cast_bytes_py2
@@ -148,11 +147,6 @@ def erepo(repo_dir):
         repo.dvc.scm.add([stage_foo.path, stage_bar.path, stage_data_dir.path])
         repo.dvc.scm.commit("init repo")
 
-        rconfig = RemoteConfig(repo.dvc.config)
-        rconfig.add("upstream", repo.dvc.cache.local.cache_dir, default=True)
-        repo.dvc.scm.add([repo.dvc.config.config_file])
-        repo.dvc.scm.commit("add remote")
-
         repo.create("version", "master")
         repo.dvc.add("version")
         repo.dvc.scm.add([".gitignore", "version.dvc"])