Repository: activeloopai/deeplake

Issue:
[BUG] dataset_meta.json isn't updated after create_tensor
## 🐛🐛 Bug Report

After loading an existing dataset with hub.dataset, create_tensor doesn't update the "tensors" attribute in dataset_meta.json.

### ⚗️ Current Behavior

For example, a dataset called "hub_dataset" with tensor "images" already exists. Loading it with hub.dataset then appending a new tensor, "test":

```python
ds = hub.dataset("hub_dataset")
ds.create_tensor('test')
```
The "tensors" attribute in dataset_meta.json won't contain the "test" tensor.

```json
{"version": "2.0.8", "tensors": ["images"]}
```

### ⚙️ Environment
- `Python` version(s): [3.7.11]
- `OS`: Ubuntu 18.04


Discussion:
@mccrearyd I'm also experiencing this issue. Will create a ticket internally.

Thanks for pointing this out @rahulbshrestha!
@rahulbshrestha I'm able to re-create this error when adding a tensor to an older dataset, but when I create a new dataset and do everything from scratch, it works fine. 

Are you having this issue when adding tensors to older datasets, or does it also happen to datasets that you created recently?
@istranic Yeah, this issue only occurs when adding tensors to older datasets. For new datasets, it works fine (as you mentioned).
@istranic and @rahulbshrestha , I am a beginner developer, can I take a stab at this if it's still open ?

Hey @divikshrivastava, welcome to Hub, and thank you for your willingness to contribute! Since this issue is not high priority, would you mind taking a look at the open issues for Hacktoberfest? Here's the link: https://github.com/activeloopai/Hub/projects/11. 

Resolving patch:
diff --git a/hub/core/storage/local.py b/hub/core/storage/local.py
--- a/hub/core/storage/local.py
+++ b/hub/core/storage/local.py
@@ -150,9 +150,7 @@ def _all_keys(self, refresh: bool = False) -> Set[str]:
             key_set = set()
             for root, dirs, files in os.walk(full_path):
                 for file in files:
-                    key_set.add(
-                        os.path.relpath(os.path.join(full_path, file), full_path)
-                    )
+                    key_set.add(os.path.relpath(os.path.join(root, file), full_path))
             self.files = key_set
         return self.files
 


Test patch:
diff --git a/hub/api/tests/test_api.py b/hub/api/tests/test_api.py
--- a/hub/api/tests/test_api.py
+++ b/hub/api/tests/test_api.py
@@ -48,6 +48,29 @@ def test_persist(ds_generator):
 
     assert ds_new.meta.version == hub.__version__
 
+    ds_new.create_tensor("label")
+    ds_new.label.extend([1, 2, 3, 4])
+
+    ds2 = ds_generator()
+
+    ds2.storage["dataset_meta.json"] == ds_new.storage["dataset_meta.json"]
+    assert len(ds2) == 4
+    np.testing.assert_array_equal(ds2.label.numpy(), np.array([[1], [2], [3], [4]]))
+
+
+def test_persist_keys(local_ds_generator):
+    ds = local_ds_generator()
+
+    ds.create_tensor("image")
+
+    ds_new = local_ds_generator()
+    assert set(ds_new.storage.keys()) == {
+        "dataset_meta.json",
+        "image/commit_diff",
+        "image/tensor_meta.json",
+        "image/tensor_info.json",
+    }
+
 
 def test_persist_with(local_ds_generator):
     with local_ds_generator() as ds: