Repository: iterative/dvc

Issue:
dvc exp show -v: JSONFileCurruptedError
# Bug Report

## dvc exp show -v: JSONFileCurruptedError

dvc exp show throws a JSONFileCorruptedError when a local cache is used in .dvc/config.


## Description

dvc exp show throws a JSONFileCorruptedError when a local cache is used in .dvc/config.


### Reproduce


The error can be reproduced as follows:

A) Create virtual environment:
1. virtualenv venv
2. source venv/bin/activate
3. pip install dvc[s3]

B) Create local repo folder <REPO_FOLDER>
1. mkdir <REPO_FOLDER>
2. cd <REPO_FOLDER>
3. git init
4. cd ..

C) Setup files in cloned repo:
1. git clone <REPO_FOLDER> dvc_checkout
2. cd dvc_checkout
3. dvc init
5. (add .dvc/config, evaluate.py, and dvc.yaml as shown below)
6. dvc repro
7. git add .
8. git commit -m "initial"
9. git push
10. dvc exp show -v



.dvc/config
```
[core]
    remote = store
    analytics = false
[cache]
    dir = <CACHE_FOLDER>
    shared = group
    type = reflink,symlink
    slow_link_warning = true
['remote "store"']
    url = <STORE_FOLDER>
``` 

dvc.yaml
```
stages:
  evaluate:
    cmd: python evaluate.py
    deps:
    - evaluate.py
    metrics:
      - metrics.json:
          cache: false
``` 

evaluate.py
```python
import json
def main():
    metrics_filename = "metrics.json"
    data = {"auc": 0.29}
    with open(metrics_filename, 'w') as f:
        json.dump(data, f)
    print("Finished.")
if __name__ == "__main__":
    main()
``` 
### Error
```
2023-07-07 20:01:44,746 DEBUG: v3.1.0 (pip), CPython 3.10.8 on Linux-3.10.0-1127.8.2.el7.x86_64-x86_64-with-glibc2.17
2023-07-07 20:01:44,746 DEBUG: command: /data/kpetersen/src/mambaforge/bin/dvc exp show -v
2023-07-07 20:01:45,609 DEBUG: unable to read: 'metrics.json', JSON file structure is corrupted: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/funcy/flow.py", line 84, in reraise
    yield
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/serialize/_json.py", line 19, in parse_json
    return json.loads(text, **kwargs) or {}
  File "/data/kpetersen/src/mambaforge/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/data/kpetersen/src/mambaforge/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/__init__.py", line 402, in wrapper
    vals = func(*args, **kwargs)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/repo/metrics/show.py", line 78, in _read_metric
    val = load_path(path, fs)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/serialize/__init__.py", line 26, in load_path
    return loader(fs_path, fs=fs)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/serialize/_json.py", line 14, in load_json
    return _load_data(path, parser=parse_json, fs=fs)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/serialize/_common.py", line 77, in _load_data
    return parser(fd.read(), path)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/dvc/utils/serialize/_json.py", line 18, in parse_json
    with reraise(json.JSONDecodeError, JSONFileCorruptedError(path)):
  File "/data/kpetersen/src/mambaforge/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/data/kpetersen/src/mambaforge/lib/python3.10/site-packages/funcy/flow.py", line 88, in reraise
    raise into from e
dvc.utils.serialize._json.JSONFileCorruptedError: unable to read: 'metrics.json', JSON file structure is corrupted
``` 




### Expected

I would expect that no error is thrown and that the overview table show the correct metric instead of an exclamation mark.


### Environment information

<!--
This is required to ensure that we can reproduce the bug.
-->

**Output of `dvc doctor`:**

```console
$ dvc doctor
DVC version: 3.4.0 (pip)
------------------------
Platform: Python 3.10.8 on Linux-3.10.0-1127.8.2.el7.x86_64-x86_64-with-glibc2.17
Subprojects:
	dvc_data = 2.3.3
	dvc_objects = 0.23.0
	dvc_render = 0.5.3
	dvc_task = 0.3.0
	scmrepo = 1.0.4
Supports:
	http (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
	s3 (s3fs = 2023.6.0, boto3 = 1.26.161)
Config:
	Global: /home/kpetersen/.config/dvc
	System: /etc/xdg/dvc

```



Discussion:
I'm not able to reproduce the error. Does your `metrics.json` look like `{"auc": 0.29}`? Does it work when you don't set the cache directory?
Thanks for checking! 

1) Yes, the `metrics.json` looks like `{"auc": 0.29}`. However, this content is stored in the cache and `metrics.json` is a link to that cached file even though `cache: false` was written in the `dvc.yaml` (see above).

2) It works if I remove the `[cache]` portion from the `.dvc/config`.
Thanks @kpetersen-hf! So far I have not been able to reproduce it. Is there anything special about the cache directory you are using? Is it on another filesystem?
Hi Dave, I don't think there is anything special about the cache folder. In my test, I put it in a sibling folder of `dvc_checkout`. So, same filesystem. 

I wonder what your `metrics.json` looks like when you use the cache in `.dvc/config`. Is it a link or a `json` file?

What would be the next best steps to debug this?
Okay, I can reproduce it if I drop `reflink` from the config (since my system has reflink supported). Looks like it's a bug with a `symlink` cache.
I see, thanks for investigating!
Here's a script that reliably reproduces the error:

```bash
git init -q 9713
cd 9713
dvc init -q

cat <<EOT >> .dvc/config
[cache]
    type = symlink
EOT

cat <<EOT >> dvc.yaml
stages:
  evaluate:
    cmd: python evaluate.py
    deps:
    - evaluate.py
    metrics:
      - metrics.json:
          cache: false
EOT

cat <<EOT >> evaluate.py
import json
def main():
    metrics_filename = "metrics.json"
    data = {"auc": 0.29}
    with open(metrics_filename, 'w') as f:
        json.dump(data, f)
    print("Finished.")
if __name__ == "__main__":
    main()
EOT

dvc repro -q
git add .
git commit -q -m "initial"
dvc exp show -v
```
Looks like it is a 3.x regression. @pmrowla Could you take a look please?

Resolving patch:
diff --git a/dvc/stage/cache.py b/dvc/stage/cache.py
--- a/dvc/stage/cache.py
+++ b/dvc/stage/cache.py
@@ -123,12 +123,15 @@ def _create_stage(self, cache, wdir=None):
 
     @contextmanager
     def _cache_type_copy(self):
-        cache_types = self.repo.cache.legacy.cache_types
+        cache_types = self.repo.cache.local.cache_types
+        legacy_cache_types = self.repo.cache.legacy.cache_types
+        self.repo.cache.local.cache_types = ["copy"]
         self.repo.cache.legacy.cache_types = ["copy"]
         try:
             yield
         finally:
-            self.repo.cache.legacy.cache_types = cache_types
+            self.repo.cache.local.cache_types = cache_types
+            self.repo.cache.legacy.cache_types = legacy_cache_types
 
     def _uncached_outs(self, stage, cache):
         # NOTE: using temporary stage to avoid accidentally modifying original


Test patch: