Repository: iterative/dvc

Issue:
push:  KeyError in self._cache[item] 
# Bug Report

> Context: https://discord.com/channels/485586884165107732/485596304961962003/825187339843272726

<!--
## Issue name

Issue names must follow the pattern `command: description` where the command is the dvc command that you are trying to run. The description should describe the consequence of the bug. 

Example: `repro: doesn't detect input changes`
-->

## Description
`dvc push` raises a `KeyError`

Full stacktrace
```
dvc push -v
2021-03-26 21:57:03,722 DEBUG: Check for update is enabled.
2021-03-26 21:57:03,816 DEBUG: Trying to spawn '['daemon', '-q', 'updater']'
2021-03-26 21:57:03,951 DEBUG: Spawned '['daemon', '-q', 'updater']'
2021-03-26 21:57:04,841 DEBUG: Preparing to upload data to 'azure://dvc'
2021-03-26 21:57:04,841 DEBUG: Preparing to collect status from azure://dvc
2021-03-26 21:57:04,841 DEBUG: Collecting information from local cache...
2021-03-26 21:57:04,841 DEBUG: Collecting information from remote cache...
2021-03-26 21:57:04,841 DEBUG: Matched '0' indexed hashes
2021-03-26 21:57:04,951 DEBUG: Estimated remote size: 4096 files
2021-03-26 21:57:04,966 DEBUG: Querying '9' hashes via traverse
2021-03-26 21:57:04,997 ERROR: unexpected error - 'dvc/'
------------------------------------------------------------
Traceback (most recent call last):
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\main.py", line 55, in main
    ret = cmd.run()
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\command\data_sync.py", line 56, in run
    processed_files_count = self.repo.push(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\repo\__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\repo\push.py", line 41, in push
    return len(used_run_cache) + self.cloud.push(used, jobs, remote=remote)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\data_cloud.py", line 66, in push
    return remote.push(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\remote\base.py", line 56, in wrapper
    return f(obj, *args, **kwargs)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\remote\base.py", line 466, in push
    ret = self._process(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\remote\base.py", line 323, in _process
    dir_status, file_status, dir_contents = self._status(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\remote\base.py", line 175, in _status
    self.hashes_exist(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\remote\base.py", line 132, in hashes_exist
    return indexed_hashes + self.odb.hashes_exist(list(hashes), **kwargs)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\objects\db\base.py", line 409, in hashes_exist
    remote_hashes = set(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\objects\db\base.py", line 269, in list_hashes_traverse
    yield from itertools.chain.from_iterable(in_remote)
  File "C:\Python38\lib\concurrent\futures\_base.py", line 611, in result_iterator
    yield fs.pop().result()
  File "C:\Python38\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Python38\lib\concurrent\futures\_base.py", line 388, in __get_result
    raise self._exception
  File "C:\Python38\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\objects\db\base.py", line 259, in list_with_update
    return list(
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\objects\db\base.py", line 150, in list_hashes
    for path in self._list_paths(prefix, progress_callback):
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\objects\db\base.py", line 130, in _list_paths
    for file_info in self.fs.walk_files(path_info, prefix=prefix):
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\fs\fsspec_wrapper.py", line 113, in walk_files
    for file in self.ls(path_info, recursive=True):
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\fs\fsspec_wrapper.py", line 99, in ls
    if self.isdir(path_info) and self.is_empty(path_info):
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\fs\fsspec_wrapper.py", line 80, in is_empty
    entry = self.info(path_info)
  File "c:\users\marki\doc-proc\env\lib\site-packages\dvc\fs\fsspec_wrapper.py", line 120, in info
    info = self.fs.info(self._with_bucket(path_info)).copy()
  File "c:\users\marki\doc-proc\env\lib\site-packages\adlfs\spec.py", line 534, in info
    fetch_from_azure = (path and self._ls_from_cache(path) is None) or refresh
  File "c:\users\marki\doc-proc\env\lib\site-packages\fsspec\spec.py", line 343, in _ls_from_cache
    return self.dircache[path]
  File "c:\users\marki\doc-proc\env\lib\site-packages\fsspec\dircache.py", line 62, in __getitem__
    return self._cache[item]  # maybe raises KeyError
KeyError: 'dvc/'
------------------------------------------------------------
2021-03-26 21:57:05,091 DEBUG: Version info for developers:
DVC version: 2.0.12 (pip)
---------------------------------
Platform: Python 3.8.0 on Windows-10-10.0.18362-SP0
Supports: azure, http, https
Cache types: hardlink
Cache directory: NTFS on C:\
Caches: local
Remotes: ssh, azure
Workspace directory: NTFS on C:\
Repo: dvc, git
 
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-03-26 21:57:05,107 DEBUG: Analytics is disabled.
```
### Reproduce

1. inside existing repo `dvc remote add dvc_azure azure://dvc` 
2. `dvc modify --local dvc_azure connection_string '....'`
3. `dvc push -v`

<!--
Step list of how to reproduce the bug
-->

<!--
Example:

1. dvc init
2. Copy dataset.zip to the directory
3. dvc add dataset.zip
4. dvc run -d dataset.zip -o model ./train.sh
5. modify dataset.zip
6. dvc repro
-->

### Expected

Dvc should say `pushed 9 files` or at least not crash.

### Environment information
Ran commands from Windows 10 using cmd.exe
```
2021-03-26 21:57:05,091 DEBUG: Version info for developers:
DVC version: 2.0.12 (pip)
---------------------------------
Platform: Python 3.8.0 on Windows-10-10.0.18362-SP0
Supports: azure, http, https
Cache types: hardlink
Cache directory: NTFS on C:\
Caches: local
Remotes: ssh, azure
Workspace directory: NTFS on C:\
Repo: dvc, git
```

**Output of `dvc doctor`:**

```console
$ dvc doctor

DVC version: 2.0.12 (pip)
---------------------------------
Platform: Python 3.8.0 on Windows-10-10.0.18362-SP0
Supports: azure, http, https
Cache types: hardlink
Cache directory: NTFS on C:\
Caches: local
Remotes: ssh, azure
Workspace directory: NTFS on C:\
Repo: dvc, git
```

**Additional Information (if any):**

I also had to manually edit the `config.local` because single quotes got inserted into my connection string. I think this is a quirk of how `cmd.exe` interacts with `dvc` but I don't know the details.
<!--
Please check https://github.com/iterative/dvc/wiki/Debugging-DVC on ways to gather more information regarding the issue.

If applicable, please also provide a `--verbose` output of the command, eg: `dvc add --verbose`.
If the issue is regarding the performance, please attach the profiling information and the benchmark comparisons.
-->



Discussion:
Following our discussion on discord @jorgeorpinel 
Hi @biro-mark ! Could you try installing adlfs from the upstream and seeing if that helps, please?

> pip install git+https://github.com/dask/adlfs.git

Duplicate of https://github.com/iterative/dvc/issues/5686

I got exactly the same error as @biro-mark and installing adlfs from upstream worked for me. 

Thank you for your help @efiop 
Yup that worked!
I've asked a release on the upstream, so that we can bump this up on dvc's setup when installing with `dvc[azure]`.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -98,7 +98,7 @@ def run(self):
 gs = ["gcsfs>=0.7.2"]
 gdrive = ["pydrive2>=1.8.1", "six >= 1.13.0"]
 s3 = ["boto3>=1.9.201"]
-azure = ["adlfs>=0.6.3", "azure-identity>=1.4.0", "knack"]
+azure = ["adlfs>=0.7.0", "azure-identity>=1.4.0", "knack"]
 # https://github.com/Legrandin/pycryptodome/issues/465
 oss = ["oss2==2.6.1", "pycryptodome<3.9.9"]
 ssh = ["paramiko[invoke]>=2.7.0"]


Test patch: