Repository: iterative/dvc

Issue:
pull: Unclear error message when remote support is not installed
# Bug Report

## Description

If you've installed DVC via pip without the remote support you need (for example S3), the error message is very unclear.

### Reproduce

1. Setup fresh repo & `pip install dvc[s3]`
2. Copy dataset.zip to repo
3. `dvc init` & `dvc add dataset.zip`
4. Setup remote & `dvc push`
5. Move to fresh python environment
6. `pip install dvc`
7. Clone repo 
8. `dvc pull -v`

### Expected

I would've expected an error message saying something like: "the remote type (s3) is not supported in your dvc installation". 

Instead you get the following output:

```
$ dvc pull -v
2021-10-26 10:35:07,698 DEBUG: failed to pull cache for 'models'      
2021-10-26 10:35:07,699 DEBUG: failed to pull cache for 'data/evaluation'
2021-10-26 10:35:08,269 WARNING: No file hash info found for 'data/evaluation'. It won't be created.                                                                                                                                            
2021-10-26 10:35:08,304 WARNING: No file hash info found for 'models'. It won't be created.                                                                                                                                                     
2 files failed                                                                                                                                                                                                                                  
2021-10-26 10:35:08,326 ERROR: failed to pull data from the cloud - Checkout failed for following targets:
data/evaluation
models
Is your cache up to date?
<https://error.dvc.org/missing-files>
```

Took me a couple hours to figure it out I had to reinstall dvc with s3 support enabled 😅 

### Environment information

<!--
This is required to ensure that we can reproduce the bug.
-->

**Output of `dvc doctor`:**

```console
$ dvc doctor

DVC version: 2.8.2 (pip)
---------------------------------
Platform: Python 3.9.7 on Linux-4.14.246-187.474.amzn2.x86_64-x86_64-with-glibc2.31
Supports:
        webhdfs (fsspec = 2021.10.1),
        http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
        https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
Cache types: hardlink, symlink
Cache directory: nfs4 on xxxxxx.efs.eu-west-1.amazonaws.com:/data-pvc
Caches: local
Remotes: s3
Workspace directory: nfs4 on xxxxx.efs.eu-west-1.amazonaws.com:/data-pvc
Repo: dvc, git
```



Discussion:
@nielstenboom Hi. Are you sure that's full verbose log? It seems to be missing a bunch of info.
Ah I'm sorry, I forgot a step to reproduce this error, set the cache directory to something else before pulling:

7.5 `dvc cache dir /tmp`
8. `dvc pull -v` 

Results in: 

```
dvc pull -v
2021-10-26 17:56:30,068 DEBUG: failed to pull cache for 'models'
2021-10-26 17:56:30,068 DEBUG: failed to pull cache for 'data/evaluation'
2021-10-26 17:56:30,134 WARNING: No file hash info found for 'models'. It won't be created.
2021-10-26 17:56:30,154 WARNING: No file hash info found for 'data/evaluation'. It won't be created.
2 files failed
2021-10-26 17:56:30,163 ERROR: failed to pull data from the cloud - Checkout failed for following targets:
models
data/evaluation
Is your cache up to date?
<https://error.dvc.org/missing-files>
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dvc/command/data_sync.py", line 30, in run
    stats = self.repo.pull(
  File "/usr/local/lib/python3.9/site-packages/dvc/repo/__init__.py", line 50, in wrapper
    return f(repo, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dvc/repo/pull.py", line 40, in pull
    stats = self.checkout(
  File "/usr/local/lib/python3.9/site-packages/dvc/repo/__init__.py", line 50, in wrapper
    return f(repo, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dvc/repo/checkout.py", line 110, in checkout
    raise CheckoutError(stats["failed"], stats)
dvc.exceptions.CheckoutError: Checkout failed for following targets:
models
data/evaluation
Is your cache up to date?
<https://error.dvc.org/missing-files>
------------------------------------------------------------
2021-10-26 17:56:30,166 DEBUG: Analytics is enabled.
2021-10-26 17:56:30,180 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpbfgth088']'
2021-10-26 17:56:30,181 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpbfgth088']'
```

Without the cache command the expected error message does appear.
@nielstenboom The error in both logs indicates that your dvc files are missing version info, that usually happens when you forget to track `dvc.lock` with git.

If you don't install `[s3]` part during `pip install`, you'll get:

```
ERROR: failed to pull data from the cloud - URL 's3://' is supported but requires these missing dependencies: ['s3fs', 'boto3']. To install dvc with those dependencies, run:

        pip install 'dvc[s3]'

See <https://dvc.org/doc/install> for more info.
```

My guess is that maybe you've done `git pull` or something after switching the venv.
@efiop thank you for your quick response!

I do believe this is a bug though, caused by changing the local cache setting. The following happens with a version of dvc that does not have s3 installed with it:

```console
$ dvc pull
Everything is up to date.
ERROR: failed to pull data from the cloud - URL 's3://' is supported but requires these missing dependencies: ['s3fs', 'boto3']. To install dvc with those dependencies, run:

	pip install 'dvc[s3]'

See <https://dvc.org/doc/install> for more info.

$ dvc cache dir /tmp
$ dvc pull
WARNING: No file hash info found for 'data/evaluation'. It won't be created.
WARNING: No file hash info found for 'models'. It won't be created.
2 files failed
ERROR: failed to pull data from the cloud - Checkout failed for following targets:
data/evaluation
models
Is your cache up to date?
<https://error.dvc.org/missing-files>
```

The `dvc cache dir /tmp` command causes the error message to change, that does not seem like the intended behavior to me. 
Hm, the `everything is up to date` is definitely a bug though. And I can indeed reproduce that warning as well, looking into it...

EDIT:  when odb (aka cache dir) is not accessible, we fail to load the corresponding object and thus checkout throws those warnings that are clearly no longer relevant.

Resolving patch:
diff --git a/dvc/output.py b/dvc/output.py
--- a/dvc/output.py
+++ b/dvc/output.py
@@ -24,6 +24,7 @@
 from .data.transfer import transfer as otransfer
 from .data.tree import Tree
 from .fs import get_cloud_fs
+from .fs.base import RemoteMissingDepsError
 from .fs.hdfs import HDFSFileSystem
 from .fs.local import LocalFileSystem
 from .fs.s3 import S3FileSystem
@@ -871,6 +872,8 @@ def _collect_used_dir_cache(
 
         try:
             self.get_dir_cache(jobs=jobs, remote=remote)
+        except RemoteMissingDepsError:
+            raise
         except DvcException:
             logger.debug(f"failed to pull cache for '{self}'")
 


Test patch:
diff --git a/tests/unit/output/test_output.py b/tests/unit/output/test_output.py
--- a/tests/unit/output/test_output.py
+++ b/tests/unit/output/test_output.py
@@ -5,9 +5,11 @@
 from funcy import first
 from voluptuous import MultipleInvalid, Schema
 
+from dvc.fs.base import RemoteMissingDepsError
 from dvc.ignore import _no_match
 from dvc.output import CHECKSUM_SCHEMA, Output
 from dvc.stage import Stage
+from dvc.utils.fs import remove
 
 
 def test_save_missing(dvc, mocker):
@@ -100,3 +102,20 @@ def test_get_used_objs(exists, expected_message, mocker, caplog):
     with caplog.at_level(logging.WARNING, logger="dvc"):
         assert {} == output.get_used_objs()
     assert first(caplog.messages) == expected_message
+
+
+def test_remote_missing_depenency_on_dir_pull(tmp_dir, scm, dvc, mocker):
+    tmp_dir.dvc_gen({"dir": {"subfile": "file2 content"}}, commit="add dir")
+    with dvc.config.edit() as conf:
+        conf["remote"]["s3"] = {"url": "s3://bucket/name"}
+        conf["core"] = {"remote": "s3"}
+
+    remove("dir")
+    remove(dvc.odb.local.cache_dir)
+
+    with mocker.patch(
+        "dvc.data_cloud.DataCloud.get_remote_odb",
+        side_effect=RemoteMissingDepsError("remote missing"),
+    ):
+        with pytest.raises(RemoteMissingDepsError):
+            dvc.pull()