Repository: Lightning-AI/torchmetrics

Issue:
Dont bring the defect from LPIPS to torchmetrics
## 🐛 Bug

It is a common bug (specially designed feature?) from LPIPS when it is served as a loss function. 

Please refers to [This page](https://github.com/richzhang/PerceptualSimilarity/issues/121). You may observe that it is a small change by adding an eps. Unfortunately, the git owner didn't update his repo for 2 more years, including bug fixes like [this pull request](https://github.com/richzhang/PerceptualSimilarity/pull/58/commits/67fb893a8a4f9975706f7d1bf49e44a305a38010)

The latest lightning torchmetric merges the LPIPS to get rid of the requirements of `pip install lpips`. Unfortunately it also bring this unfixed bug to torchmetric. Therefore, a revisit is required. 

The output without bug fixes will be something like: 

```
315     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
316 RuntimeError: Function 'SqrtBackward0' returned nan values in its 0th output.
```

### To Reproduce

Steps to reproduce the behavior...

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

<details>
  <summary>Code sample</summary>

<!-- Ideally attach a minimal code sample to reproduce the decried issue.
Minimal means having the shortest code but still preserving the bug. -->

</details>

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

### Environment

<details>                                                                                                              
  <summary>Current environment</summary>                                                                               
                                                                                                                       
* CUDA:                                                                                                                
        - GPU:                                                                                                         
                - NVIDIA GeForce RTX 3090                                                                              
                - NVIDIA GeForce RTX 3090                                                                              
        - available:         True                                                                                      
        - version:           11.2                                                                                      
* Lightning:                                                                                                           
        - lightning:         2.0.9                                                                                     
        - lightning-cloud:   0.5.38                                                                                    
        - lightning-utilities: 0.9.0                                                                                   
        - lion-pytorch:      0.1.2                                                                                     
        - pytorch-lightning: 2.0.9                                                                                     
        - torch:             1.13.1.post200                                                                            
        - torchinfo:         1.8.0                                                                                     
        - torchmetrics:      1.1.2                                                                                     
        - torchvision:       0.14.1a0+59d9189                                                                          
* Packages:                                                                                                            
        - annotated-types:   0.5.0                                                                                     
        - anyio:             3.7.1                                                                                     
        - appdirs:           1.4.4                                                                                     
        - arrow:             1.2.3                                                                                     
        - attrs:             23.1.0                                                                                    
        - backoff:           2.2.1                                                                                     
        - backports.functools-lru-cache: 1.6.5                                                                         
        - beautifulsoup4:    4.12.2                                                                                    
        - blessed:           1.19.1                                                                                    
        - brotli:            1.1.0                                                                                     
        - build:             0.10.0                                                                                    
        - cachecontrol:      0.13.1         
       - certifi:           2023.7.22
        - cffi:              1.15.1
        - charset-normalizer: 3.2.0
        - cleo:              2.0.1
        - click:             8.1.7
        - colorama:          0.4.6
        - crashtest:         0.4.1
        - croniter:          1.4.1
        - cryptography:      41.0.4
        - dateutils:         0.6.12
        - deepdiff:          6.5.0
        - distlib:           0.3.7
        - docker-pycreds:    0.4.0
        - dulwich:           0.21.6
        - einops:            0.6.1
        - exceptiongroup:    1.1.3
        - fastapi:           0.103.1
        - filelock:          3.12.4
        - flash-attn:        2.2.5
        - fsspec:            2023.9.1
        - gitdb:             4.0.10
        - gitpython:         3.1.36
        - h11:               0.14.0
        - idna:              3.4
        - imagecodecs:       2023.9.4
        - imageio:           2.31.1
        - importlib-metadata: 6.8.0
        - importlib-resources: 6.0.1
        - inquirer:          3.1.3
        - installer:         0.7.0
        - itsdangerous:      2.1.2
        - jaraco.classes:    3.3.0
        - jeepney:           0.8.0
        - jinja2:            3.1.2
        - jsonschema:        4.17.3
        - keyring:           24.2.0
        - lazy-loader:       0.3
        - lightning:         2.0.9
        - lightning-cloud:   0.5.38
        - lightning-utilities: 0.9.0
        - lion-pytorch:      0.1.2
        - lit:               16.0.6.dev0
        - lpips:             0.1.4
        - markdown-it-py:    3.0.0
        - markupsafe:        2.1.3
        - - mdurl:             0.1.0
        - more-itertools:    10.1.0
        - msgpack:           1.0.5
        - natsort:           8.4.0
        - networkx:          3.1
        - ninja:             1.11.1
        - numpy:             1.26.0
        - opencv-python:     4.7.0
        - ordered-set:       4.1.0
        - orjson:            3.9.6
        - packaging:         23.1
        - pathtools:         0.1.2
        - pexpect:           4.8.0
        - pillow:            10.0.0
        - pip:               23.2.1
        - pkginfo:           1.9.6
        - pkgutil-resolve-name: 1.3.10
        - platformdirs:      3.10.0
        - poetry:            1.6.1
        - poetry-core:       1.7.0
        - poetry-plugin-export: 1.5.0
        - protobuf:          4.21.12
        - psutil:            5.9.5
        - ptyprocess:        0.7.0
        - pycparser:         2.21
        - pydantic:          2.1.1
        - pydantic-core:     2.4.0
        - pygments:          2.16.1
        - pyjwt:             2.8.0
        - pyopenssl:         23.2.0
        - pyproject-hooks:   1.0.0
        - pyrsistent:        0.19.3
        - pysocks:           1.7.1
        - python-dateutil:   2.8.2
        - python-editor:     1.0.4
        - python-multipart:  0.0.6
        - pytorch-lightning: 2.0.9
        - pytz:              2023.3.post1
        - pywavelets:        1.4.1
        - pyyaml:            6.0.1
        - rapidfuzz:         2.15.1
        - readchar:          4.0.5.dev0
        - requests:          2.31.0
        - requests-toolbelt: 1.0.0
        - rich:              13.5.3
        - - scikit-image:      0.21.0
        - scipy:             1.11.2
        - secretstorage:     3.3.3
        - sentry-sdk:        1.31.0
        - setproctitle:      1.3.2
        - setuptools:        68.2.2
        - shellingham:       1.5.3
        - six:               1.16.0
        - smmap:             3.0.5
        - sniffio:           1.3.0
        - soupsieve:         2.5
        - starlette:         0.27.0
        - starsessions:      1.3.0
        - tifffile:          2023.9.18
        - tomli:             2.0.1
        - tomlkit:           0.12.1
        - torch:             1.13.1.post200
        - torchinfo:         1.8.0
        - torchmetrics:      1.1.2
        - torchvision:       0.14.1a0+59d9189
        - tqdm:              4.66.1
        - traitlets:         5.10.0
        - triton:            2.0.0
        - trove-classifiers: 2023.9.19
        - typing-extensions: 4.8.0
        - urllib3:           2.0.5
        - uvicorn:           0.23.2
        - virtualenv:        20.24.4
        - wandb:             0.15.10
        - wcwidth:           0.2.6
        - websocket-client:  1.6.3
        - websockets:        11.0.3
        - wheel:             0.41.2
        - zipp:              3.17.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         x86_64
        - python:            3.10.12
        - release:           6.2.0-33-generic
        - version:           #33~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep  7 10:33:52 UTC 2
</details>

### Additional context

<!-- Add any other context about the problem here. -->



Discussion:
Hi! thanks for your contribution!, great first issue!

Resolving patch:
diff --git a/src/torchmetrics/functional/image/lpips.py b/src/torchmetrics/functional/image/lpips.py
--- a/src/torchmetrics/functional/image/lpips.py
+++ b/src/torchmetrics/functional/image/lpips.py
@@ -191,10 +191,10 @@ def _upsample(in_tens: Tensor, out_hw: Tuple[int, ...] = (64, 64)) -> Tensor:
     return nn.Upsample(size=out_hw, mode="bilinear", align_corners=False)(in_tens)
 
 
-def _normalize_tensor(in_feat: Tensor, eps: float = 1e-10) -> Tensor:
-    """Normalize tensors."""
-    norm_factor = torch.sqrt(torch.sum(in_feat**2, dim=1, keepdim=True))
-    return in_feat / (norm_factor + eps)
+def _normalize_tensor(in_feat: Tensor, eps: float = 1e-8) -> Tensor:
+    """Normalize input tensor."""
+    norm_factor = torch.sqrt(eps + torch.sum(in_feat**2, dim=1, keepdim=True))
+    return in_feat / norm_factor
 
 
 def _resize_tensor(x: Tensor, size: int = 64) -> Tensor:


Test patch: