Repository: iterative/dvc

Issue:
dvc add ignore unprotected files that have not been modified
DVC version 0.29.0 installed via pip on Linux

Following the sequence of commands
dvc unprotect data.dat
(modify data.dat)
dvc add data.dat

Correctly replaces dvc.dat with the correct kind of file link
but
dvc unprotect data.dat
(not modifying)
dvc add data.dat

Causes add to skip data.dat because it is already cached and leaves the file on disk without linking. Presumably the ideal behavior is to have dvc add check what kind of link the cache should use and update the file to that link even if the file was not modified. This way modified and unmodified data have the same behavior.

Currently the workaround is to rm the data file and checkout which is somewhat awkward.



Discussion:
Looks like it is a side effect of https://github.com/iterative/dvc/blob/0.29.0/dvc/stage.py#L423 . We should probably call `save()` on that stage if we see that it is already cached, just to ensure that all the cache is optimized. 
@efiop I am having problems with the work around of rm'ing the file and checking it out. If I have more than one such file, "dvc checkout" will only checkout file1.dat: and then quit with an error message "Error: unexpected error - [errno 1] Operation not permitted: file2.dat". If I call dvc checkout n'times where n is the number of files I can get all of them but this is very very tedious.
@rmcavoy So sorry for the delay! Notification got lost in the flow :slightly_frowning_face: . Does that issue still persist?
Closed accidentally.


Resolving patch:
diff --git a/dvc/stage.py b/dvc/stage.py
--- a/dvc/stage.py
+++ b/dvc/stage.py
@@ -394,7 +394,21 @@ def is_cached(self):
             out.pop(RemoteLOCAL.PARAM_CHECKSUM, None)
             out.pop(RemoteS3.PARAM_CHECKSUM, None)
 
-        return old_d == new_d
+        if old_d != new_d:
+            return False
+
+        # NOTE: committing to prevent potential data duplication. For example
+        #
+        #    $ dvc config cache.type hardlink
+        #    $ echo foo > foo
+        #    $ dvc add foo
+        #    $ rm -f foo
+        #    $ echo foo > foo
+        #    $ dvc add foo # should replace foo with a link to cache
+        #
+        old.commit()
+
+        return True
 
     @staticmethod
     def create(


Test patch:
diff --git a/tests/func/test_add.py b/tests/func/test_add.py
--- a/tests/func/test_add.py
+++ b/tests/func/test_add.py
@@ -475,3 +475,24 @@ def test(self):
         created_stages_filenames = stage_creator_spy.mock.call_args[0][0]
         for fname in created_stages_filenames:
             self.assertNotIn(".git", fname)
+
+
+class TestAddUnprotected(TestDvc):
+    def test(self):
+        ret = main(["config", "cache.type", "hardlink"])
+        self.assertEqual(ret, 0)
+
+        ret = main(["config", "cache.protected", "true"])
+        self.assertEqual(ret, 0)
+
+        ret = main(["add", self.FOO])
+        self.assertEqual(ret, 0)
+
+        ret = main(["unprotect", self.FOO])
+        self.assertEqual(ret, 0)
+
+        ret = main(["add", self.FOO])
+        self.assertEqual(ret, 0)
+
+        self.assertFalse(os.access(self.FOO, os.W_OK))
+        self.assertTrue(System.is_hardlink(self.FOO))