Repository: iterative/dvc

Issue:
import: Doesn't remove temporary files when download failed
*DVC version*: 0.23.2

`dvc import 'https://en.wikipedia.org/wiki/Main_Page'`

```
Importing 'https://en.wikipedia.org/wiki/Main_Page' -> '/home/mroutis/tmp/Main_Page'
[##############################] 100% Main_Page
Error: failed to import https://en.wikipedia.org/wiki/Main_Page - could not find an ETag for 'https://en.wikipedia.org/wiki/Main_Page'
```

`ls ~/tmp` -> `Main_Page`

---

`dvc import https://s3-us-west-2.amazonaws.com/dvc-share/get-started/data.xml`

```
Importing 'https://s3-us-west-2.amazonaws.com/dvc-share/get-started/data.xml' -> '/home/mroutis/tmp/data.xml'
[##                            ] 7% data.xml^CError: interrupted by the user

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
```

`ls ~/tmp` -> `data.xml.e5f6cf5a-a919-43bf-949b-dfd3754227f1`


Discussion:
Although, it would be great if you could resume "importing" big files :thinking: 
Also, there's the oportunity to show a better message, like: `failed te import x... You can add it with dvc add` or something more acitonable
I don't think we should delete them though. tmp will be cleaned up on the next reboot(might differ on your distro), so it is not a big deal. And yes, when we will be able to resume downloads(which is not a tiny task, since the implementation is remote type specific), we would not want to delete these files as well.
> tmp will be cleaned up on the next reboot(might differ on your distro), so it is not a big deal

The temporary files are created in the working directory, in this case, mine was `/home/mroutis/tmp`, sorry if it caused confusion :sweat_smile: 

> And yes, when we will be able to resume downloads(which is not a tiny task, since the implementation is remote type specific), we would not want to delete these files as well.

Should we re-open https://github.com/iterative/dvc/issues/108 and close this one, @efiop ?
Right, we are using <pre>fname + <font color="#CC0000">&apos;.&apos;</font> + <font color="#06989A">str</font>(uuid.uuid4())</pre> as a temp file name, so it is even more visible, since it places a not-hidden temp file with a meaningful name right in your working directory :)

Thanks for linking #108 ! :slightly_smiling_face: We used to have `--continue` support, but it got lost during the redesign :slightly_frowning_face:  Sure, let's reopen it.

@pared is working on improving the error message, so let's leave this issue opened for now.

Resolving patch:
diff --git a/dvc/state.py b/dvc/state.py
--- a/dvc/state.py
+++ b/dvc/state.py
@@ -245,24 +245,19 @@ def dump(self):
 
     @staticmethod
     def mtime_and_size(path):
-        size = 0
+        size = os.path.getsize(path)
         mtime = os.path.getmtime(path)
 
         if os.path.isdir(path):
             for root, dirs, files in os.walk(path):
-                for dname in dirs:
-                    p = os.path.join(root, dname)
-                    m = os.path.getmtime(p)
-                    if m > mtime:
-                        mtime = m
-
-                for fname in files:
-                    p = os.path.join(root, fname)
+                for name in dirs + files:
+                    p = os.path.join(root, name)
                     st = os.stat(p)
                     size += st.st_size
                     m = st.st_mtime
                     if m > mtime:
                         mtime = m
+
         # State of files handled by dvc is stored in db as TEXT.
         # We cast results to string for later comparisons with stored values.
         return str(int(nanotime.timestamp(mtime))), str(size)


Test patch:
diff --git a/tests/unit/state.py b/tests/unit/state.py
--- a/tests/unit/state.py
+++ b/tests/unit/state.py
@@ -1,13 +1,23 @@
+import os
+
 from dvc.state import State
 from tests.basic_env import TestDvc
 
 
-class TestMtimeAndSizeReturnType(TestDvc):
+class TestMtimeAndSize(TestDvc):
     def test(self):
         file_time, file_size = State.mtime_and_size(self.DATA)
         dir_time, dir_size = State.mtime_and_size(self.DATA_DIR)
 
+        actual_file_size = os.path.getsize(self.DATA)
+        actual_dir_size = (os.path.getsize(self.DATA_DIR)
+                           + os.path.getsize(self.DATA)
+                           + os.path.getsize(self.DATA_SUB_DIR)
+                           + os.path.getsize(self.DATA_SUB))
+
         self.assertIs(type(file_time), str)
         self.assertIs(type(file_size), str)
+        self.assertEqual(file_size, str(actual_file_size))
         self.assertIs(type(dir_time), str)
         self.assertIs(type(dir_size), str)
+        self.assertEqual(dir_size, str(actual_dir_size))