Repository: iterative/dvc

Issue:
metrics diff --show-md master: 'NoneType' object is not subscriptable  
# Bug Report

Within master branch:
```sh
dvc metrics diff --show-md master 

ERROR: unexpected error - 'NoneType' object is not subscriptable      

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
```

This also happens with dvc plots
```sh
dvc plots diff --target loss.csv --show-vega master

ERROR: unexpected error - 'NoneType' object is not subscriptable

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
```

## Description

Doing that OP within master throws that error. This is undesired for CI/CD environments.
 
### Reproduce

```ssh
git clone https://github.com/DavidGOrtega/cml-dvc-basic.git
cd cml-dvc-basic
dvc init
dvc remote add -d myremote s3://your-remote
python get_data.py
dvc add data
```
```ssh
dvc push --run-cache

WARNING: Output 'metrics.json'(stage: 'mystage') is missing version info. Cache for it will not be collected. Use `dvc repro` to get your pipeline up to date.
WARNING: Output 'loss.csv'(stage: 'mystage') is missing version info. Cache for it will not be collected. Use `dvc repro` to get your pipeline up to date.
5 files pushed  
```

```ssh
git add --all
git commit -m 'ci ready' 
```

At this point should suffice for CML looking for doing repro within the CI but unfortunately it does not work 

```ssh
dvc repro --pull

'data.dvc' didn't change, skipping                                    
Running stage 'mystage':
> python train.py
/usr/local/lib/python2.7/dist-packages/sklearn/neural_network/multilayer_perceptron.py:562: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (23) reached and the optimization hasn't converged yet.
  % self.max_iter, ConvergenceWarning)
Generating lock file 'dvc.lock'                                                                                                    
Updating lock file 'dvc.lock'

To track the changes with git, run:

        git add .gitignore dvc.lock
Use `dvc push` to send your updates to remote storage.
```

```ssh
dvc metrics diff -v --show-md -v HEAD HEAD

2021-03-28 16:27:10,627 TRACE: Namespace(a_rev='HEAD', all=False, b_rev='HEAD', cd='.', cmd='diff', cprofile=False, cprofile_dump=None, func=<class 'dvc.command.metrics.CmdMetricsDiff'>, instrument=False, instrument_open=False, no_path=False, pdb=False, precision=None, quiet=0, recursive=False, show_json=False, show_md=True, targets=None, verbose=2, version=None)
2021-03-28 16:27:10,963 DEBUG: Check for update is enabled.
2021-03-28 16:27:11,069 TRACE: params.yaml does not exist, it won't be used in parametrization
2021-03-28 16:27:11,070 TRACE: Context during resolution of stage mystage:
{}
2021-03-28 16:27:11,414 TRACE: params.yaml does not exist, it won't be used in parametrization
2021-03-28 16:27:11,415 TRACE: Context during resolution of stage mystage:
{}
2021-03-28 16:27:11,416 DEBUG: Lockfile for 'dvc.yaml' not found
2021-03-28 16:27:11,582 ERROR: unexpected error - 'NoneType' object is not subscriptable
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/main.py", line 50, in main
  File "dvc/command/metrics.py", line 149, in run
  File "dvc/repo/metrics/__init__.py", line 13, in diff
  File "dvc/repo/metrics/diff.py", line 23, in diff
  File "dvc/repo/metrics/diff.py", line 8, in _get_metrics
  File "dvc/repo/metrics/__init__.py", line 8, in show
  File "dvc/repo/__init__.py", line 49, in wrapper
  File "dvc/repo/metrics/show.py", line 120, in show
  File "dvc/repo/metrics/show.py", line 82, in _read_metrics
  File "dvc/utils/serialize/_yaml.py", line 20, in load_yaml
  File "dvc/utils/serialize/_common.py", line 50, in _load_data
  File "dvc/fs/repo.py", line 143, in open
  File "dvc/fs/dvc.py", line 81, in open
  File "dvc/objects/db/base.py", line 68, in hash_to_path_info
TypeError: 'NoneType' object is not subscriptable
------------------------------------------------------------
2021-03-28 16:27:12,462 DEBUG: Version info for developers:
DVC version: 2.0.6 (deb)
---------------------------------
Platform: Python 3.7.10 on Linux-4.19.76-linuxkit-x86_64-with-debian-buster-sid
Supports: All remotes
Cache types: hardlink, symlink
Cache directory: fuse.grpcfuse on grpcfuse
Caches: local
Remotes: s3
Workspace directory: fuse.grpcfuse on grpcfuse
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-03-28 16:27:12,473 DEBUG: Analytics is enabled.
2021-03-28 16:27:12,661 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpjo47u5rj']'
2021-03-28 16:27:12,667 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpjo47u5rj']'
[18640] Failed to execute script __main__
```

plots does not work either
```ssh
dvc plots diff -v --target loss.csv --show-vega HEAD HEAD
2021-03-28 16:38:29,322 DEBUG: Check for update is enabled.
2021-03-28 16:38:29,442 ERROR: 'loss.csv' does not exist.
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/command/plots.py", line 37, in run
  File "dvc/command/plots.py", line 80, in _func
  File "dvc/repo/plots/__init__.py", line 173, in diff
  File "dvc/repo/plots/diff.py", line 16, in diff
  File "dvc/repo/plots/__init__.py", line 160, in show
dvc.exceptions.MetricDoesNotExistError: 'loss.csv' does not exist.
------------------------------------------------------------
2021-03-28 16:38:29,450 DEBUG: Analytics is enabled.
2021-03-28 16:38:29,506 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpjufpxbo2']'
2021-03-28 16:38:29,508 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpjufpxbo2']'
root@97f8cc35d780:/iterative/cml-dvc-basic# [18685] Failed to execute script __main__
dvc plots diff -v --target loss.csv --show-vega HEAD
2021-03-28 16:38:47,913 DEBUG: Check for update is enabled.
2021-03-28 16:38:48,008 DEBUG: Lockfile for 'dvc.yaml' not found
2021-03-28 16:38:48,332 DEBUG: Lockfile for 'dvc.yaml' not found
2021-03-28 16:38:48,482 ERROR: unexpected error - 'NoneType' object is not subscriptable
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/main.py", line 50, in main
  File "dvc/command/plots.py", line 37, in run
  File "dvc/command/plots.py", line 80, in _func
  File "dvc/repo/plots/__init__.py", line 173, in diff
  File "dvc/repo/plots/diff.py", line 16, in diff
  File "dvc/repo/plots/__init__.py", line 149, in show
  File "dvc/repo/plots/__init__.py", line 86, in collect
  File "dvc/fs/repo.py", line 143, in open
  File "dvc/fs/dvc.py", line 81, in open
  File "dvc/objects/db/base.py", line 68, in hash_to_path_info
TypeError: 'NoneType' object is not subscriptable
------------------------------------------------------------
2021-03-28 16:38:49,413 DEBUG: Version info for developers:
DVC version: 2.0.6 (deb)
---------------------------------
Platform: Python 3.7.10 on Linux-4.19.76-linuxkit-x86_64-with-debian-buster-sid
Supports: All remotes
Cache types: hardlink, symlink
Cache directory: fuse.grpcfuse on grpcfuse
Caches: local
Remotes: s3
Workspace directory: fuse.grpcfuse on grpcfuse
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-03-28 16:38:49,420 DEBUG: Analytics is enabled.
2021-03-28 16:38:49,597 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpiyh0i0y5']'
2021-03-28 16:38:49,601 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpiyh0i0y5']'
root@97f8cc35d780:/iterative/cml-dvc-basic# [18702] Failed to execute script __main__
```

If we add dvc.lock diff will work

```ssh
git add dvc.lock
git commit -m 'dvc.lock'

dvc metrics diff -v --show-md -v HEAD HEAD
2021-03-28 16:27:48,548 TRACE: Namespace(a_rev='HEAD', all=False, b_rev='HEAD', cd='.', cmd='diff', cprofile=False, cprofile_dump=None, func=<class 'dvc.command.metrics.CmdMetricsDiff'>, instrument=False, instrument_open=False, no_path=False, pdb=False, precision=None, quiet=0, recursive=False, show_json=False, show_md=True, targets=None, verbose=2, version=None)
2021-03-28 16:27:48,864 DEBUG: Check for update is enabled.
2021-03-28 16:27:48,971 TRACE: params.yaml does not exist, it won't be used in parametrization
2021-03-28 16:27:48,972 TRACE: Context during resolution of stage mystage:
{}
2021-03-28 16:27:49,332 TRACE: params.yaml does not exist, it won't be used in parametrization
2021-03-28 16:27:49,333 TRACE: Context during resolution of stage mystage:
{}
2021-03-28 16:27:49,341 TRACE: Assuming '/iterative/cml-dvc-basic/.dvc/cache/be/3703304dfc59f95af156f8a354e68a' is unchanged since it is read-only
| Path   | Metric   | Old   | New   | Change   |
|--------|----------|-------|-------|----------|

2021-03-28 16:27:49,420 DEBUG: Analytics is enabled.
2021-03-28 16:27:49,507 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpdzwzsmo5']'
2021-03-28 16:27:49,511 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpdzwzsmo5']'
[18675] Failed to execute script __main__
```

Additionally plots does not work with another error
```ssh
dvc plots diff -v --target loss.csv --show-vega HEAD HEAD
2021-03-28 16:43:30,479 DEBUG: Check for update is enabled.
2021-03-28 16:43:30,593 ERROR: 'loss.csv' does not exist.
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/command/plots.py", line 37, in run
  File "dvc/command/plots.py", line 80, in _func
  File "dvc/repo/plots/__init__.py", line 173, in diff
  File "dvc/repo/plots/diff.py", line 16, in diff
  File "dvc/repo/plots/__init__.py", line 160, in show
dvc.exceptions.MetricDoesNotExistError: 'loss.csv' does not exist.
------------------------------------------------------------
2021-03-28 16:43:30,608 DEBUG: Analytics is enabled.
2021-03-28 16:43:30,663 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp22ehhoy5']'
2021-03-28 16:43:30,665 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp22ehhoy5']'
root@97f8cc35d780:/iterative/cml-dvc-basic# [18722] Failed to execute script __main__
```


### Expected

No diff message instead of that non descriptive error.

### Environment information

```console
$ dvc doctor

DVC version: 2.0.6 (deb)
---------------------------------
Platform: Python 3.7.10 on Linux-4.19.76-linuxkit-x86_64-with-debian-buster-sid
Supports: All remotes
Cache types: hardlink, symlink
Cache directory: fuse.grpcfuse on grpcfuse
Caches: local
Remotes: s3
Workspace directory: fuse.grpcfuse on grpcfuse
Repo: dvc, git
```


Discussion:
related to #5204 #5685 ?
@DavidGOrtega Could you please add `-v` flag and post full log? That will contain a traceback, which makes looking into issues much easier.
@efiop I have updated the description. Seems that dvc diff can work without dvc.lock commited and dvc plots diff can not be compared with itself?
If there is no git committed `dvc.lock`, then the output does not exist in that git commit (in this case `HEAD`), some error in that case would be expected.
Thanks @pmrowla for the feedback we have some things still here:
 - I  think that a descriptive error with something like ```have you commited dvc.lock?``` would help the user?
 - for a CML point of view --run-cache is nice but you have to prepare the workflow to at least commit one dvc.lock or do the first run locally.
 - dvc plots diff -v --target loss.csv --show-vega HEAD HEAD does not work even dvc.lock has commited. seems that it does not like to diff itself.

What do you think?
This is related to some extent to #5525. In general, we have discussed having these `plots/metrics/params` commands continue even where they encounter errors (as well as providing more informative error messages/hints).

What's the specific problem this causes for CML? Does the CI pipeline need to depend on `dvc metrics diff` exiting without error? Or does the output look ugly for demo purposes or something?
> Does the CI pipeline need to depend on dvc metrics diff exiting without error? 

Yes the CI short-circuits every non 0 exit so in CML there must be lots of (|| true) to survive.
Also as I stated plots can not render HEAD twice and thats bad for master workflow unless we quite complex pipeline.

However the biggest issue is the need of dvc.lock committed. We could do it in the CI but many users don't want to do that 
Thanks. Just trying to get context to best determine how to solve and prioritize.

It seems like everything you are raising are expected failures (diffing against nothing, self-diffing), but you'd like them to be handled to avoid non-0 exit? So it seems like the suggestion in #5525 would be sufficient, right?

> I think that we should gather the exceptions preventing us from displaying particular file at particular revision and return them as part of result and let the error handling happen after doing as much as possible. Errors can still be logged to debug, and in case of dvc plots/metrics command we could issue single warning, like "10 out of 1000 plots could not be shown. Use -v to see why".
> self-diffing

Its a must. Master branch would still generate loss plot in example within it, not only not failing is desirable. As I stated if not done that way the CI workflow would be much complex having to ask the branch in which we are to do a HEAD vs HEAD~1 if possible, etc...
So to summarize your needs:

* `diff` commands should not return non-0 exit codes
* in self-diff scenario, behavior should mimic `show` commands

Does that sound right?
@DavidGOrtega thanks for the detailed description 🙏 could you please elaborate a bit on self-diffing. I can't wrap my mind around this command `dvc plots diff -v --target loss.csv --show-vega HEAD HEAD` - could show a workflow example for us to understand better what's happening here. Or may be @dberenbaum could explain this. I'm personally lost a bit ...

> for a CML point of view --run-cache is nice but you have to prepare the workflow to at least commit one dvc.lock or do the first run locally.

is it related to this ticket? also not sure I understand where it comes from. Would be really great if you could expand a bit on this.
> is it related to this ticket? also not sure I understand where it comes from. Would be really great if you could expand a bit on this.

@shcheklein it is in the sense of ```diff``` needing dvc.lock committed in order to be able to work as I stated in the description (search "At this point should suffice for CML looking for doing repro"). 
This make --run-cache still be dependant of having dvc.lock to be committed. That means that the first ```repro``` must be done in the CI with a complex workflow to commit dvc.lock the first time or ```repro``` being run locally first to commit dvc.lock and then everything works.

The need of self diff is because the master branch with a common workflow will diff with itself. Here is a common CML scenario assuming unfortunately dvc.lock committed at least once. 
As I stated in the description dvc metrics diff does NOT have any issue diffing itself while plots have. If you look at the workflow dvc is diffing against master hence failing when the workflow runs against master.

So the ticket scope is:
 - diff needs dvc.lock committed to not fail
 - dvc plots diff fails with itself while dvc metrics does not 


```yaml
name: train-my-model

on: [push]

jobs:
  run:
    runs-on: [ubuntu-latest]
  
    steps:
      - uses: actions/checkout@v2

      - uses: iterative/setup-cml@v1

      - uses: iterative/setup-dvc@v1

      - uses: actions/setup-python@v2
        with:
          python-version: '3.x'

      - name: cml
        shell: bash
        env:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        run: |
          git fetch --prune --unshallow
          
          pip install -r requirements.txt

          # Pull dataset with DVC 
          (dvc pull --run-cache -v) || true
          
          # Reproduce pipeline if any changes detected in dependencies
          dvc repro --pull

          echo 'DVC push cache'
          dvc push --run-cache -v

          echo 'CML Report' > report.md

          echo 'Use DVC metrics diff to compare metrics to master'
          (dvc metrics diff -v --show-md master >> report.md ) || echo 'Failed diff'
          
          echo 'Add figure to report'
          (dvc plots diff -v --target loss.csv --show-vega master > vega.json && \
          vl2png vega.json -s 1.3 | cml-publish --md >> report.md) || echo 'Failed plots diff'

          echo 'Send report'
          cml-send-comment report.md 
```


> * diff needs dvc.lock committed to not fail
> 
> * dvc plots diff fails with itself while dvc metrics does not

👍 

@DavidGOrtega I'm clear on these problems, but I'm less clear on what you need for solutions. 

> not only not failing is desirable

Does this mean there is some expected output you want in these scenarios? Could you explain what you would like to see as a result in each scenario (for example, returning nothing, returning some warning, returning the current metrics/plots).
> @DavidGOrtega I'm clear on these problems, but I'm less clear on what you need for solutions.

My 2 cents - nothing special is required. It should not fail (I assume `'NoneType' object..` is a fail), should show the diff (can be empty) and should return no zero status. @DavidGOrtega might have a different opinion or ideas.
So, this one `dvc plots diff fails with itself while dvc metrics does not` can be considered a bug, a separate bug, not related specifically to this ticket. If I got it right.

> This make --run-cache still be dependant of having dvc.lock to be committed. 

Yes, I understand that we need it in the master (the commit that we'll be comparing the new run with). This is probably expected, as @pmrowla dvc.lock pretty much is the key to get the right versions of the plots/metrics files in this case.

When repro is done we should have `dvc.lock` in the workspace and that should be enough now to compare everything, right?

Also, in a repo like this, I would not expect commits to not have the `dvc.lock` file. Master (HEAD) should have it, when you do a branch you should see it automatically in a brach. Or if you do a new commit you should also see it.

I agree that errors could be handled better.


> So, this one dvc plots diff fails with itself while dvc metrics does not can be considered a bug, a separate bug, not related specifically to this ticket. If I got it right.

I don't know exactly if it should or it should not, as I showed without dvc.lock committed first the all diffs are failing. Which is very misleading since someone can try to do diff without committing first and dvc.lock present locally having that difficult to identify issue.

> Also, in a repo like this, I would not expect commits to not have the dvc.lock file

Theoretically --run-cache could enable us to not have to commit dvc.lock. We have users that:
 - Need to run every single repo in the CI and not locally
 - Do not want to do commit in the CI
 


> I don't know exactly if it should or it should not, as I showed without dvc.lock committed first the all diffs are failing.

I was referring specifically to self diff. And the discrepancy that it fails in one case and doesn't in another one. No matter dvc.lock is present or not. This specific self diff behavior looks like a separate potential bug to me.

> Theoretically --run-cache could enable us to not have to commit dvc.lock. 

Okay, I see now. The idea is that for commands that need access to DVC-tracked files/directories we could not only check cache (based on `dvc.lock`), but also first could check `run-cache` and "restore" the `dvc.lock` to know what files we need to read. It one additional level of "indirection" when we access artifacts, but it's probably possible. 

Not sure what are the consequences for this to be honest. For now my perception is that `run-cache` is more of a cache it's not built to be reliable source of historic data (like Git). At least we'll need to protect entries to some extent (by using commit hash for example in the key).

> Theoretically --run-cache could enable us to not have to commit dvc.lock. We have users that:
> 
> * Need to run every single repo in the CI and not locally
> * Do not want to do commit in the CI

Run-cache does not allow you to diff against older revisions though. It works for repro & CI runs because you have the actual data for your current revision in your workspace, so DVC can still hash all of your data if needed with or without a dvc.lock hash.

For comparing older revisions (solely from git), there is no data for DVC to hash. DVC has no idea what version of some dependency existed in some specific git commit without the `dvc.lock` file containing that hash.

Run-cache is just a cache of stages that says "these input/dependency hashes will give these output hashes". DVC still has to have the set of dependency hashes to be able to use run-cache.

Not git committing `dvc.lock` is the pipeline equivalent of not git committing the `.dvc` file for a pure data dependency. Doing so basically disables any DVC features related to git history at all, including being able to actually version your data, and being able to retrieve past revisions of your data.
> I don't know exactly if it should or it should not, as I showed without dvc.lock committed first the all diffs are failing. Which is very misleading since someone can try to do diff without committing first and dvc.lock present locally having that difficult to identify issue.

On this point, we could consider assuming that if `dvc.lock` is not present in a git commit, then the hashes for all data should be `None`. In this case, the diff for any git commit without `dvc.lock` and the user's workspace would show all of the workspace data as entirely new (and nonexistent in the git commit). But this also seems like unexpected behavior from the user's perspective.

It sounds like there is some general misunderstanding here about how `dvc.lock` and run-cache work in DVC?
> My 2 cents - nothing special is required. It should not fail (I assume `'NoneType' object..` is a fail), should show the diff (can be empty) and should return no zero status. @DavidGOrtega might have a different opinion or ideas.

Since @dmpetrov and @DavidGOrtega seem fine with any solution that generates 0 exit codes, do the other details matter? It still seems to me that prioritizing #5525 should fix all of the problems in this ticket unless I'm missing something.
> Since @dmpetrov and @DavidGOrtega seem fine with any solution that generates 0 exit codes, do the other details matter?

plots should not fail comparing with itself. It should return at least one plot like metrics returns an empty table.

> It sounds like there is some general misunderstanding here about how dvc.lock and run-cache work in DVC?

Probably. I have been always told that dvc.lock would not be needed with --run-cache.
Here's a summary of my understanding from the discussions in this ticket and some talks with those who have been participating:

# self-diff

### Bug

`dvc metrics diff HEAD HEAD` (which outputs an empty table) and `dvc plots diff HEAD HEAD` (which throws an error) are inconsistent for self-diffs. Not sure what expected behavior is here, but it should be consistent, and if there is an error, it should be more informative. Unifying the various diff commands is on the dvc roadmap, so while I don't know when we will get to this, it is well aligned with dvc priorities already.

### CML impact

@DavidGOrtega I'm curious whether you are seeing this issue in your actual CML pipelines even after `dvc.lock` is committed, since the example you provided in https://github.com/iterative/dvc/issues/5692#issuecomment-811804906 isn't quite a self-diff:

`dvc plots diff -v --target loss.csv --show-vega master`

This is subtly different from:

`dvc plots diff -v --target loss.csv --show-vega HEAD HEAD`

In testing, it looks like the first command will work once you have a committed `dvc.lock`, even if there are no differences between the workspace and `master` (and I would expect this to be a common CML workflow). It's only when hard-coding the same revision multiple times that an error is thrown. Similarly, `metrics diff master` should return actual metrics (instead of the empty table from `metrics diff HEAD HEAD`).

# dvc.lock

For the general case where there is no `dvc.lock` in one of the revisions, expected behavior for `dvc metrics diff` and `dvc plots diff` is unclear. Options include:
1. Throw an error with a more informative message/hint and return a non-0 exit code (see suggestion in https://github.com/iterative/dvc/issues/5692#issuecomment-811253143).
2. Show an informative warning instead of error and return no other output and a 0 exit code.
3. Return a table/plot with the data for the revision where `dvc.lock` exists and no data for the revision where `dvc.lock` is missing and a 0 exit code.
One question I have for the dvc devs in this scenario is what happens when the metrics/plots are Git-tracked but no `dvc.lock` exists or there is a mismatch? Like the self-diffing, this is already well aligned with current dvc priorities.

# run-cache

Using the run-cache with the Git-committed data to recreate the expected outputs is an interesting idea. As I understand the CML scenario:
1. The `dvc.yaml` and `train.py` and `data.dvc` dependencies are committed and tracked by Git in the master branch.
2. CML runs `dvc repro`, which populates the run-cache.
3. To get the metrics or other outputs, it should be possible to need only the run-cache and the Git-tracked files in the master branch (all other info that would be found in `dvc.lock` can be reconstructed from this).

If I'm understanding that right, it's an interesting idea. This would be a big change (for example, not sure how `always-changed` stages would work) that is not currently prioritized, although we can continue to discuss and might come back to it later.
@DavidGOrtega What's the status on this? Would it make sense to find some time to discuss? Feels like the discussion thread has gotten pretty long for anyone to follow and includes multiple issues. Even if it's not a blocker for you, it's probably worth making sure the workflow makes sense for the typical CML user.
 > Would it make sense to find some time to discuss? Feels like the discussion thread has gotten pretty long for anyone to follow and includes multiple issues.
 
Sorry for the late reply. A review is also very good for me. We are preparing ```cml-pr``` which makes a PR with the needed files aside as a solution for dvc.lock and other non tracked dvc files as metrics, etc...
So it would potentially solve if not all, at least a lot of the issues. However dvc its going still to throw the errors that we know breaking the workflow. We can review this together. 



Resolving patch:
diff --git a/dvc/fs/dvc.py b/dvc/fs/dvc.py
--- a/dvc/fs/dvc.py
+++ b/dvc/fs/dvc.py
@@ -68,13 +68,17 @@ def open(  # type: ignore
             raise IsADirectoryError
 
         out = outs[0]
+
+        if not out.hash_info:
+            raise FileNotFoundError
+
         if out.changed_cache(filter_info=path):
             from dvc.config import NoRemoteError
 
             try:
                 remote_obj = self.repo.cloud.get_remote(remote)
-            except NoRemoteError:
-                raise FileNotFoundError
+            except NoRemoteError as exc:
+                raise FileNotFoundError from exc
             if out.is_dir_checksum:
                 checksum = self._get_granular_hash(path, out).value
             else:


Test patch:
diff --git a/tests/unit/fs/test_dvc.py b/tests/unit/fs/test_dvc.py
--- a/tests/unit/fs/test_dvc.py
+++ b/tests/unit/fs/test_dvc.py
@@ -3,10 +3,12 @@
 
 import pytest
 
+from dvc.config import NoRemoteError
 from dvc.fs.dvc import DvcFileSystem
 from dvc.hash_info import HashInfo
 from dvc.objects.stage import stage
 from dvc.path_info import PathInfo
+from dvc.utils.fs import remove
 
 
 def test_exists(tmp_dir, dvc):
@@ -39,6 +41,18 @@ def test_open_dirty_hash(tmp_dir, dvc):
         assert fobj.read() == "file"
 
 
+def test_open_no_remote(tmp_dir, dvc):
+    tmp_dir.dvc_gen("file", "file")
+    (tmp_dir / "file").unlink()
+    remove(dvc.odb.local.cache_dir)
+
+    fs = DvcFileSystem(repo=dvc)
+    with pytest.raises(FileNotFoundError) as exc_info:
+        with fs.open(PathInfo(tmp_dir) / "file", "r"):
+            pass
+    assert isinstance(exc_info.value.__cause__, NoRemoteError)
+
+
 def test_open_dirty_no_hash(tmp_dir, dvc):
     tmp_dir.gen("file", "file")
     (tmp_dir / "file.dvc").write_text("outs:\n- path: file\n")