Repository: iterative/dvc

Issue:
add: fails when declaring a git submodule as a dependency
# Bug Report

## Description
In DVC v2 `dvc run` fails when a git submodules is provided as a dependency.

### Reproduce
```bash
#!/usr/bin/env bash
set -x
set -e
mkdir -p repo_a
mkdir -p repo_b
git init repo_a
git init repo_b
cd repo_b
echo "nothing" > nothing
git add -A
git commit -m "first commit"
cd ../repo_a
echo "nothing" > nothing
git add -A
git commit -m "first commit"
git submodule add ../repo_b modules/repo_b
git submodule update --init modules/repo_b
git commit -a -m "second commit"
dvc init
dvc run -v -n test -d nothing -d modules -o something 'echo "test" > something'
```

### Expected
`dvc run` exits normally.

### Environment information
Ubuntu 18.04, dvc v2.0.14, Python 3.6.

**Output of `dvc doctor`:**

```console
$ dvc doctor
DVC version: 2.0.14 (pip)
---------------------------------
Platform: Python 3.6.12 on Linux-4.15.0-1065-aws-x86_64-with-debian-buster-sid
Supports: http, https, s3, ssh

```

**Additional Information (if any):**
Output from above:
```console
+ set -e
+ mkdir -p repo_a
+ mkdir -p repo_b
+ git init repo_a
Initialized empty Git repository in /home/asarroff/tmp/dvc-test/repo_a/.git/
+ git init repo_b
Initialized empty Git repository in /home/asarroff/tmp/dvc-test/repo_b/.git/
+ cd repo_b
+ echo nothing
+ git add -A
+ git commit -m 'first commit'
[master (root-commit) 1f99a56] first commit
 1 file changed, 1 insertion(+)
 create mode 100644 nothing
+ cd ../repo_a
+ echo nothing
+ git add -A
+ git commit -m 'first commit'
[master (root-commit) f671379] first commit
 1 file changed, 1 insertion(+)
 create mode 100644 nothing
+ git submodule add ../repo_b modules/repo_b
Cloning into '/home/asarroff/tmp/dvc-test/repo_a/modules/repo_b'...
done.
+ git submodule update --init modules/repo_b
+ git commit -a -m 'second commit'
[master 175fdcb] second commit
 2 files changed, 4 insertions(+)
 create mode 100644 .gitmodules
 create mode 160000 modules/repo_b
+ dvc init
Initialized DVC repository.

You can now commit the changes to git.

What's next?
------------
- Check out the documentation: <https://dvc.org/doc>
- Get help and share ideas: <https://dvc.org/chat>
- Star us on GitHub: <https://github.com/iterative/dvc>
+ dvc run -v -n test -d nothing -d modules -o something 'echo "test" > something'
2021-03-30 21:14:47,194 DEBUG: Check for update is enabled.
2021-03-30 21:14:47,278 DEBUG: Trying to spawn '['daemon', '-q', 'updater']'
2021-03-30 21:14:47,279 DEBUG: Spawned '['daemon', '-q', 'updater']'
2021-03-30 21:14:50,834 DEBUG: Removing output 'something' of stage: 'test'.
2021-03-30 21:14:50,834 DEBUG: Removing 'something'
2021-03-30 21:14:51,186 DEBUG: state save (14035717747750367173, 1617153265880000000, 8) 3618634fc7650e537697fd7f542002c0
2021-03-30 21:14:51,270 DEBUG: state save (14035717747750367173, 1617153265880000000, 8) 3618634fc7650e537697fd7f542002c0
2021-03-30 21:14:51,516 DEBUG: state save (2807698749563520249, 1617153267732000000, 8) 3618634fc7650e537697fd7f542002c0
2021-03-30 21:14:51,686 ERROR: unexpected error - [Errno 2] No such file or directory: PosixPathInfo: 'modules/repo_b/.git'
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/main.py", line 55, in main
    ret = cmd.run()
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/command/run.py", line 64, in run
    desc=self.args.desc,
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/repo/scm_context.py", line 14, in run
    return method(repo, *args, **kw)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/repo/run.py", line 33, in run
    stage.run(no_commit=no_commit, run_cache=run_cache)
  File "/home/asarroff/.local/lib/python3.6/site-packages/funcy/decorators.py", line 39, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/home/asarroff/.local/lib/python3.6/site-packages/funcy/decorators.py", line 60, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/stage/__init__.py", line 508, in run
    run_stage(self, dry, force, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/stage/run.py", line 154, in run_stage
    stage.repo.stage_cache.restore(stage, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/stage/cache.py", line 179, in restore
    if not _can_hash(stage):
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/stage/cache.py", line 38, in _can_hash
    if not (dep.scheme == "local" and dep.def_path and dep.get_hash()):
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/output/base.py", line 196, in get_hash
    self.fs.PARAM_CHECKSUM,
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 177, in stage
    obj = _get_tree_obj(path_info, fs, name, odb, state, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 116, in _get_tree_obj
    tree = _build_tree(path_info, fs, name, state, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 84, in _build_tree
    for fi, hi in _iter_hashes(path_info, fs, name, state, **kwargs):
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 76, in _iter_hashes
    yield from _calculate_hashes(path_info, fs, name, state, **kwargs).items()
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 62, in _calculate_hashes
    return dict(pairs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/progress.py", line 129, in wrapped
    res = fn(*args, **kwargs)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 54, in _hash
    return file_info, get_file_hash(file_info, fs, name, state)
  File "/home/asarroff/miniconda3/envs/tensorflow-stable/lib/python3.6/site-packages/dvc/objects/stage.py", line 40, in get_file_hash
    errno.ENOENT, os.strerror(errno.ENOENT), path_info
FileNotFoundError: [Errno 2] No such file or directory: PosixPathInfo: 'modules/repo_b/.git'
------------------------------------------------------------
2021-03-30 21:14:58,081 DEBUG: Version info for developers:
DVC version: 2.0.14 (pip)
---------------------------------
Platform: Python 3.6.12 on Linux-4.15.0-1065-aws-x86_64-with-debian-buster-sid
Supports: http, https, s3, ssh
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: nfs4 on 127.0.0.1:/
Repo: dvc, git
```


Discussion:
Issue happens because submodules contain a flat file named `.git`, but our default ignore pattern only contains a pattern for the directory `.git/`.

In `fs.local.walk`, for an actual file we only test against file ignore patterns, and we yield the path to the file `.git`. However, in `fs.local.exists`, we always check against both file and directory patterns (without checking to see of our path is a file or directory), which ends up returning that the file `.git` does not exist, since it technically matches the directory pattern. This mismatch causes the eventual exception - `walk` yields a file which later appears to be ignored/not exist.

Basically, we should also be including the file pattern `.git` in our default ignores so that we do not get this behavior mismatch between `walk` and `exists` (`walk` should not be yielding `.git` file in submodules).

Resolving patch:
diff --git a/dvc/ignore.py b/dvc/ignore.py
--- a/dvc/ignore.py
+++ b/dvc/ignore.py
@@ -183,7 +183,12 @@ class DvcIgnoreFilter:
     def __init__(self, fs, root_dir):
         from dvc.repo import Repo
 
-        default_ignore_patterns = [".hg/", ".git/", "{}/".format(Repo.DVC_DIR)]
+        default_ignore_patterns = [
+            ".hg/",
+            ".git/",
+            ".git",
+            "{}/".format(Repo.DVC_DIR),
+        ]
 
         self.fs = fs
         self.root_dir = root_dir


Test patch:
diff --git a/tests/func/test_ignore.py b/tests/func/test_ignore.py
--- a/tests/func/test_ignore.py
+++ b/tests/func/test_ignore.py
@@ -112,7 +112,7 @@ def test_ignore_collecting_dvcignores(tmp_dir, dvc, dname):
     assert (
         DvcIgnorePatterns(
             *merge_patterns(
-                _to_pattern_info_list([".hg/", ".git/", ".dvc/"]),
+                _to_pattern_info_list([".hg/", ".git/", ".git", ".dvc/"]),
                 os.fspath(tmp_dir),
                 _to_pattern_info_list([os.path.basename(dname)]),
                 top_ignore_path,
@@ -344,7 +344,7 @@ def test_pattern_trie_fs(tmp_dir, dvc):
     )
 
     base_pattern = (
-        _to_pattern_info_list([".hg/", ".git/", ".dvc/"]),
+        _to_pattern_info_list([".hg/", ".git/", ".git", ".dvc/"]),
         os.fspath(tmp_dir),
     )
     first_pattern = merge_patterns(