Repository: iterative/dvc

Issue:
add: --to-remote error: 'HashedStreamReader' object has no attribute 'tell'
# Bug Report

<!--
## add: --to-remote error: 'HashedStreamReader' object has no attribute 'tell'

Issue names must follow the pattern `command: description` where the command is the dvc command that you are trying to run. The description should describe the consequence of the bug. 

Example: `repro: doesn't detect input changes`
-->

## Description

<!--
A clear and concise description of what the bug is.
-->

Running `dvc add` with the `--to-remote` option results in an error: 
```AttributeError: 'HashedStreamReader' object has no attribute 'tell'```

### Reproduce

<!--
Step list of how to reproduce the bug
-->

<!--
Example:

1. dvc init
2. Copy dataset.zip to the directory
3. dvc add dataset.zip
4. dvc run -d dataset.zip -o model ./train.sh
5. modify dataset.zip
6. dvc repro
-->

1. Run `dvc add --to-remote -o datasets/my-dataset/ --file my-dataset.dvc afile.zip`

### Expected

<!--
A clear and concise description of what you expect to happen.
-->

Successful execution with no errors.

### Environment information

<!--
This is required to ensure that we can reproduce the bug.
-->

**Output of `dvc version`:**

```console
$ dvc version
DVC version: 2.0.0a0+27c240
---------------------------------
Platform: Python 3.7.3 on Darwin-20.3.0-x86_64-i386-64bit
Supports: All remotes
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: gs
Workspace directory: apfs on /dev/disk1s5s1
Repo: dvc, git
```

**Additional Information (if any):**

<!--
If applicable, please also provide a `--verbose` output of the command, eg: `dvc add --verbose`.
-->

Verbose output:
```
dvc add --to-remote -o datasets/my-dataset/ --file my-dataset.dvc afile.zip --verbose
2021-02-23 08:28:44,338 DEBUG: Check for update is enabled.
Adding...
2021-02-23 08:28:46,408 ERROR: unexpected error - 'HashedStreamReader' object has no attribute 'tell'
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/main.py", line 50, in main
    ret = cmd.run()
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/command/add.py", line 32, in run
    jobs=self.args.jobs,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/repo/scm_context.py", line 14, in run
    return method(repo, *args, **kw)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/repo/add.py", line 131, in add
    **kwargs,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/repo/add.py", line 172, in _process_stages
    command="add",
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/data_cloud.py", line 111, in transfer
    return remote.transfer(from_fs, from_fs.path_info, jobs=jobs)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/remote/base.py", line 520, in transfer
    no_progress_bar=no_progress_bar,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/objects/__init__.py", line 312, in transfer
    tmp_info, hash_info = _transfer_file(odb, from_fs, from_info)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/objects/__init__.py", line 254, in _transfer_file
    desc=from_info.name,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/fs/base.py", line 256, in upload_fobj
    self._upload_fobj(wrapped, to_info)  # pylint: disable=no-member
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/fs/gs.py", line 203, in _upload_fobj
    _upload_to_bucket(bucket, fobj, to_info)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/fs/gs.py", line 38, in wrapper
    return func(*args, chunk_size=chunk_size, **kwargs)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/dvc/fs/gs.py", line 50, in _upload_to_bucket
    blob.upload_from_file(fobj)
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1167, in upload_from_file
    client, file_obj, content_type, size, num_retries, predefined_acl
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1077, in _do_upload
    client, stream, content_type, size, num_retries, predefined_acl
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1020, in _do_resumable_upload
    predefined_acl=predefined_acl,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 969, in _initiate_resumable_upload
    stream_final=False,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/resumable_media/requests/upload.py", line 402, in initiate
    stream_final=stream_final,
  File "/usr/local/anaconda3/lib/python3.7/site-packages/google/resumable_media/_upload.py", line 463, in _prepare_initiate_request
    if stream.tell() != 0:
  File "/usr/local/anaconda3/lib/python3.7/site-packages/tqdm/utils.py", line 88, in __getattr__
    return getattr(self._wrapped, name)
AttributeError: 'HashedStreamReader' object has no attribute 'tell'
------------------------------------------------------------
2021-02-23 08:28:47,265 DEBUG: Version info for developers:
DVC version: 2.0.0a0+27c240
---------------------------------
Platform: Python 3.7.3 on Darwin-20.3.0-x86_64-i386-64bit
Supports: All remotes
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: gs
Workspace directory: apfs on /dev/disk1s5s1
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-02-23 08:28:47,267 DEBUG: Analytics is enabled.
2021-02-23 08:28:47,534 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/vx/n884kynx5ps8hksttp4mwl64010gl8/T/tmpeo2oe0g6']'
2021-02-23 08:28:47,536 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/vx/n884kynx5ps8hksttp4mwl64010gl8/T/tmpeo2oe0g6']'
```


Discussion:
Hi @KeyboardSounds !

Thank you so much for the report! Looks like we simply need to keep track of the number of bytes read in https://github.com/iterative/dvc/blob/27c2401f596d99a2dc2d5f5ba9c7482effe35bed/dvc/utils/stream.py#L82 and then return that in `tell()`.
I think we could just delegate that job to the underlying `fobj` (currently we only keep a reference to its read function though we can also keep a reference to the file like object itself) and just call `fobj.tell()`
We also additionally need to implement a few other methods (just to ensure other APIs won't make any problems) like `seekable()` etc

Resolving patch:
diff --git a/dvc/utils/stream.py b/dvc/utils/stream.py
--- a/dvc/utils/stream.py
+++ b/dvc/utils/stream.py
@@ -1,6 +1,8 @@
 import hashlib
 import io
 
+from funcy import cached_property
+
 from dvc.hash_info import HashInfo
 from dvc.istextfile import istextblock
 from dvc.utils import dos2unix
@@ -61,17 +63,30 @@ def peek(self, n):
         return self.leftover[:n]
 
 
-class HashedStreamReader:
+class HashedStreamReader(io.IOBase):
 
     PARAM_CHECKSUM = "md5"
 
     def __init__(self, fobj):
+        self.fobj = fobj
         self.md5 = hashlib.md5()
         self.is_text_file = None
-        self.reader = fobj.read1 if hasattr(fobj, "read1") else fobj.read
+        super().__init__()
+
+    def readable(self):
+        return True
+
+    def tell(self):
+        return self.fobj.tell()
+
+    @cached_property
+    def _reader(self):
+        if hasattr(self.fobj, "read1"):
+            return self.fobj.read1
+        return self.fobj.read
 
     def read(self, n=-1):
-        chunk = self.reader(n)
+        chunk = self._reader(n)
         if self.is_text_file is None:
             self.is_text_file = istextblock(chunk)
 


Test patch:
diff --git a/tests/unit/utils/test_stream.py b/tests/unit/utils/test_stream.py
--- a/tests/unit/utils/test_stream.py
+++ b/tests/unit/utils/test_stream.py
@@ -9,7 +9,15 @@ def test_hashed_stream_reader(tmp_dir):
     foo = tmp_dir / "foo"
     with open(foo, "rb") as fobj:
         stream_reader = HashedStreamReader(fobj)
-        assert stream_reader.read(3) == b"foo"
+
+        assert stream_reader.readable()
+        assert not stream_reader.seekable()
+
+        assert stream_reader.read(2) == b"fo"
+        assert stream_reader.tell() == 2
+
+        assert stream_reader.read(1) == b"o"
+        assert stream_reader.tell() == 3
 
     hex_digest = file_md5(foo, LocalFileSystem(None, {}))
     assert stream_reader.is_text_file
@@ -20,13 +28,21 @@ def test_hashed_stream_reader_as_chunks(tmp_dir):
     tmp_dir.gen({"foo": b"foo \x00" * 16})
 
     foo = tmp_dir / "foo"
+
+    actual_size = len(foo.read_bytes())
     with open(foo, "rb") as fobj:
         stream_reader = HashedStreamReader(fobj)
+
+        total_read = 0
         while True:
             chunk = stream_reader.read(16)
+            total_read += len(chunk)
+            assert stream_reader.tell() == total_read
             if not chunk:
                 break
 
+        assert stream_reader.tell() == actual_size == total_read
+
     hex_digest = file_md5(foo, LocalFileSystem(None, {}))
     assert not stream_reader.is_text_file
     assert hex_digest == stream_reader.hash_info.value