Repository: iterative/dvc

Issue #6594: diff: use object diff
Created: 2021-09-10T20:16:40Z
Currently dvc/repo/diff.py is quite a bit outdated, we didn't touch it since before introducing objects, so it is still based on hash_info, which makes it suboptimal and very confusing. What we should do instead is introduce dvc/objects/diff.py and implement generic diffing logic for Tree objects, that could then be used in dvc diff. Also it is important to remember that even though right now we don't provide one Tree for the whole repo (kinda like git commits do), we will do that soon, which will again simplify dvc diff logic, that will then use objects.diff even more directly.
Btw, we are already doing basic object diff when merging trees in our git merge-driver (see merge in dvc/objects/tree.py).

Duplicate issue #6720: diff: ERROR: unexpected error - 'not in cache' 
Created: 2021-09-30T16:09:46Z
Description
When I tried to compare two commits I get an error.
The command is: dvc diff f0c0269 16ba2cd
I'm using your sample repo: https://github.com/iterative/example-get-started
Reproduce
Command 1 (compare commit with previous "dvc add" commit):
dvc diff f0c0269 16ba2cd

Output:
ERROR: unexpected error - 'not in cache'                              

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

Command 2 (compare commit with a commit before "dvc add" commit):
dvc diff 3e07290 16ba2cd

Output:
Added:                                                                
    data/data.xml

files summary: 1 added

How to reproduce it:
git clone https://github.com/iterative/example-get-started
cd example-get-started/
dvc diff 16ba2cd f0c0269

Expected
These are the first commits in your sample repo:
* 08a3b89 - (tag: 4-import-data) Import raw data (overwrite) (2021-05-20 23:56:52 -0400) <Olivaw Owlet>
* 16ba2cd - (tag: 3-config-remote) Configure default remote (2021-05-19 20:10:12 -0400) <Olivaw Owlet>
* f0c0269 - (tag: 2-track-data) Add raw data (2021-05-18 16:23:32 -0400) <Olivaw Owlet>
* 3e07290 - (tag: 1-dvc-init) Initialize DVC project (2021-05-17 12:36:52 -0400) <Olivaw Owlet>
* 90b2aea - (tag: 0-git-init) Initialize Git repository (2021-05-16 08:50:12 -0400) <Olivaw Owlet>

If you show changes on the third commit f0c0269 with git show f0c0269 you can see:
commit f0c026993d3dc8aa4d79426acb59b81dbb63e5a2 (tag: 2-track-data)
Author: Olivaw Owlet <64868532+iterative-olivaw@users.noreply.github.com>
Date:   Tue May 18 16:23:32 2021 -0400

    Add raw data

diff --git a/data/.gitignore b/data/.gitignore
new file mode 100644
index 0000000..fc12be5
--- /dev/null
+++ b/data/.gitignore
@@ -0,0 +1 @@
+/data.xml
diff --git a/data/data.xml.dvc b/data/data.xml.dvc
new file mode 100644
index 0000000..8fab3a9
--- /dev/null
+++ b/data/data.xml.dvc
@@ -0,0 +1,5 @@
+outs:
+- md5: a304afb96060aad90176268345e10355
+  size: 37891850
+  path: data.xml
+  desc: Initial XML StackOverflow dataset (raw data)

I suppose that means that was the commit where the file data\data.xml was added using dvc.
What I expect:
If I create a commit adding a file with dvc add and I compare the next commit with the previous one I expect the dvc diff two show that the file was added.
That means I expect the command 1 to have to same output as command 2 because the file was added in the commit f0c0269
Environment information
Output of `dvc diff -v f0c0269 16ba2cd:
2021-09-30 17:03:09,704 ERROR: unexpected error - 'not in cache'      
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/josecelano/anaconda3/lib/python3.8/site-packages/dvc/main.py", line 55, in main
    ret = cmd.do_run()
  File "/home/josecelano/anaconda3/lib/python3.8/site-packages/dvc/command/base.py", line 45, in do_run
    return self.run()
  File "/home/josecelano/anaconda3/lib/python3.8/site-packages/dvc/command/diff.py", line 131, in run
    del diff["not in cache"]
KeyError: 'not in cache'
------------------------------------------------------------
2021-09-30 17:03:09,891 DEBUG: Version info for developers:
DVC version: 2.7.3 (pip)
---------------------------------
Platform: Python 3.8.8 on Linux-5.10.0-1045-oem-x86_64-with-glibc2.10
Supports:
	azure (adlfs = 2021.9.1, knack = 0.8.2, azure-identity = 1.6.1),
	http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
	https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5)
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: https
Workspace directory: ext4 on /dev/mapper/vgubuntu-root
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-09-30 17:03:09,892 DEBUG: Analytics is enabled.
2021-09-30 17:03:09,926 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpx7a6ek24']'
2021-09-30 17:03:09,927 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpx7a6ek24']'

Closing comment (2022-02-17T19:06:51Z): That means I expect the command 1 to have to same output as command 2 because the file was added in the commit f0c0269

Since the data.xml.dvc exists in f0c0269 but not in 3e07290, these shouldn't have the same output, right @josecelano? I would expect dvc diff f0c0269 16ba2cd to return nothing. I think nautilus-cyberneering/chinese-ideographs#38 sums it up: dvc diff fails when no files have been changed.
For example:
$ dvc diff 16ba2cd 08a3b89
2022-02-17 13:51:57,963 ERROR: unexpected error - 'not in cache'
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dave/Code/dvc/dvc/cli/__init__.py", line 78, in main
    ret = cmd.do_run()
  File "/Users/dave/Code/dvc/dvc/cli/command.py", line 22, in do_run
    return self.run()
  File "/Users/dave/Code/dvc/dvc/commands/diff.py", line 131, in run
    del diff["not in cache"]
KeyError: 'not in cache'
------------------------------------------------------------

@efiop The error isn't related to #6594 or dvc/repo/diff.py.

  
    
      dvc/dvc/commands/diff.py
    
    
         Line 131
      in
      0b18efb
    
  
  
    

        
          
           del diff["not in cache"] 
        
    
  


This line simply doesn't handle the case where that key doesn't exist. Do you want me to submit a PR?