Repository: pytorch/pytorch

Issue:
torch.var outputs inf for correction>N
### 🐛 Describe the bug

torch.var 

```
x = torch.arange(12, dtype=torch.float32).reshape(1,3,2,2)
torch.var(x, dim=2, correction=3)
```

output:
```
>>> torch.var(x, dim=2, correction=3)
tensor([[[inf, inf],
         [inf, inf],
         [inf, inf]]])
>>> 
```
In above N=2 because dim=2 and above example dim=2 is 2
Now if we increase the correction value greater than dim value i.e. 2 in this case then it result in inf for all element.
But as per formula https://pytorch.org/docs/stable/generated/torch.var.html . it should have negative of summation.

### Versions

Versions
Versions of relevant libraries:
[pip3] flake8==5.0.4
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.24.3
[pip3] pytorch-lightning==1.9.4
[pip3] torch==2.0.1
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==2.0.1
[pip3] torchdata==0.6.1
[pip3] torchmetrics==0.11.1

cc @svekars @carljparker @albanD


Discussion:
In the corresponding C++ code,  the factor is actually implemented as `1 / max(0, N - correction)`. This explains the `inf` as `1 / 0 = inf`. I guess we have 3 options here:

- Fix the docs to reflect the `max(0, ...)` part
- Report a warning if `correction >= N` and keep the current `inf` behavior
- Throw an error if `correction >= N` and exit

Additionally, the above problem also applies to `torch.std`.
Thanks for looking into this.
I think that updating the doc to have the max(0, ...) is the simplest fix here. Would you be able to send a PR with this update?
> Thanks for looking into this. I think that updating the doc to have the max(0, ...) is the simplest fix here. Would you be able to send a PR with this update?

Sure, I'll prepare a PR later.
While preparing the PR, I noticed some other functions that suffer from a similar problem.

- `torch.var`
- `torch.var_mean`
- `torch.std`
- `torch.std_mean`
- `torch.cov`

Unfortunately, the behavior is slightly inconsistent, as only `torch.cov` throws a `UserWarning` if `correction>=observations`. Maybe we should add that to the other 4 functions too.
```
>>> torch.var(torch.randn(3), correction=3)
tensor(inf)
>>> torch.std(torch.randn(3), correction=3)
tensor(inf)
>>> torch.var_mean(torch.randn(3), correction=3)
(tensor(inf), tensor(-0.4473))
>>> torch.std_mean(torch.randn(3), correction=3)
(tensor(inf), tensor(0.7958))
>>> torch.cov(torch.randn(2, 3), correction=3)
<stdin>:1: UserWarning: cov(): degrees of freedom is <= 0 (Triggered internally at ../aten/src/ATen/native/Correlation.cpp:117.)
tensor([[inf, -inf],
        [-inf, inf]])
```
throwing a warning sounds good as well. But that will require a bit more work to add the warning at the right place and add proper testing.
If you have time, please do both. If you have limited time, just updating the doc is the most important change I think.
For now, I created a PR for the docs change only. I'll try working on the warning too, but we can also create a new issue for that.

Resolving patch:
diff --git a/torch/_torch_docs.py b/torch/_torch_docs.py
--- a/torch/_torch_docs.py
+++ b/torch/_torch_docs.py
@@ -2232,22 +2232,25 @@ def merge_dicts(*dicts):
 the variance of each variable (covariance of a variable with itself). By definition, if :attr:`input` represents
 a single variable (Scalar or 1D) then its variance is returned.
 
-The unbiased sample covariance of the variables :math:`x` and :math:`y` is given by:
+The sample covariance of the variables :math:`x` and :math:`y` is given by:
 
 .. math::
-    \text{cov}_w(x,y) = \frac{\sum^{N}_{i = 1}(x_{i} - \bar{x})(y_{i} - \bar{y})}{N~-~1}
+    \text{cov}(x,y) = \frac{\sum^{N}_{i = 1}(x_{i} - \bar{x})(y_{i} - \bar{y})}{\max(0,~N~-~\delta N)}
 
-where :math:`\bar{x}` and :math:`\bar{y}` are the simple means of the :math:`x` and :math:`y` respectively.
+where :math:`\bar{x}` and :math:`\bar{y}` are the simple means of the :math:`x` and :math:`y` respectively, and
+:math:`\delta N` is the :attr:`correction`.
 
-If :attr:`fweights` and/or :attr:`aweights` are provided, the unbiased weighted covariance
+If :attr:`fweights` and/or :attr:`aweights` are provided, the weighted covariance
 is calculated, which is given by:
 
 .. math::
-    \text{cov}_w(x,y) = \frac{\sum^{N}_{i = 1}w_i(x_{i} - \mu_x^*)(y_{i} - \mu_y^*)}{\sum^{N}_{i = 1}w_i~-~1}
+    \text{cov}_w(x,y) = \frac{\sum^{N}_{i = 1}w_i(x_{i} - \mu_x^*)(y_{i} - \mu_y^*)}
+    {\max(0,~\sum^{N}_{i = 1}w_i~-~\frac{\sum^{N}_{i = 1}w_ia_i}{\sum^{N}_{i = 1}w_i}~\delta N)}
 
-where :math:`w` denotes :attr:`fweights` or :attr:`aweights` based on whichever is provided, or
-:math:`w = fweights \times aweights` if both are provided, and
-:math:`\mu_x^* = \frac{\sum^{N}_{i = 1}w_ix_{i} }{\sum^{N}_{i = 1}w_i}` is the weighted mean of the variable.
+where :math:`w` denotes :attr:`fweights` or :attr:`aweights` (``f`` and ``a`` for brevity) based on whichever is
+provided, or :math:`w = f \times a` if both are provided, and
+:math:`\mu_x^* = \frac{\sum^{N}_{i = 1}w_ix_{i} }{\sum^{N}_{i = 1}w_i}` is the weighted mean of the variable. If not
+provided, ``f`` and/or ``a`` can be seen as a :math:`\mathbb{1}` vector of appropriate size.
 
 Args:
     input (Tensor): A 2D matrix containing multiple variables and observations, or a
@@ -2260,11 +2263,11 @@ def merge_dicts(*dicts):
         will return the simple average. Defaults to ``1``.
     fweights (tensor, optional): A Scalar or 1D tensor of observation vector frequencies representing the number of
         times each observation should be repeated. Its numel must equal the number of columns of :attr:`input`.
-        Must have integral dtype. Ignored if ``None``. `Defaults to ``None``.
+        Must have integral dtype. Ignored if ``None``. Defaults to ``None``.
     aweights (tensor, optional): A Scalar or 1D array of observation vector weights.
         These relative weights are typically large for observations considered “important” and smaller for
         observations considered less “important”. Its numel must equal the number of columns of :attr:`input`.
-        Must have floating point dtype. Ignored if ``None``. `Defaults to ``None``.
+        Must have floating point dtype. Ignored if ``None``. Defaults to ``None``.
 
 Returns:
     (Tensor) The covariance matrix of the variables.
@@ -10729,7 +10732,7 @@ def merge_dicts(*dicts):
 
 The standard deviation (:math:`\sigma`) is calculated as
 
-.. math:: \sigma = \sqrt{\frac{1}{N - \delta N}\sum_{i=0}^{N-1}(x_i-\bar{x})^2}
+.. math:: \sigma = \sqrt{\frac{1}{\max(0,~N - \delta N)}\sum_{i=0}^{N-1}(x_i-\bar{x})^2}
 
 where :math:`x` is the sample set of elements, :math:`\bar{x}` is the
 sample mean, :math:`N` is the number of samples and :math:`\delta N` is
@@ -10785,7 +10788,7 @@ def merge_dicts(*dicts):
 
 The standard deviation (:math:`\sigma`) is calculated as
 
-.. math:: \sigma = \sqrt{\frac{1}{N - \delta N}\sum_{i=0}^{N-1}(x_i-\bar{x})^2}
+.. math:: \sigma = \sqrt{\frac{1}{\max(0,~N - \delta N)}\sum_{i=0}^{N-1}(x_i-\bar{x})^2}
 
 where :math:`x` is the sample set of elements, :math:`\bar{x}` is the
 sample mean, :math:`N` is the number of samples and :math:`\delta N` is
@@ -12112,7 +12115,7 @@ def merge_dicts(*dicts):
 
 The variance (:math:`\sigma^2`) is calculated as
 
-.. math:: \sigma^2 = \frac{1}{N - \delta N}\sum_{i=0}^{N-1}(x_i-\bar{x})^2
+.. math:: \sigma^2 = \frac{1}{\max(0,~N - \delta N)}\sum_{i=0}^{N-1}(x_i-\bar{x})^2
 
 where :math:`x` is the sample set of elements, :math:`\bar{x}` is the
 sample mean, :math:`N` is the number of samples and :math:`\delta N` is
@@ -12168,7 +12171,7 @@ def merge_dicts(*dicts):
 
 The variance (:math:`\sigma^2`) is calculated as
 
-.. math:: \sigma^2 = \frac{1}{N - \delta N}\sum_{i=0}^{N-1}(x_i-\bar{x})^2
+.. math:: \sigma^2 = \frac{1}{\max(0,~N - \delta N)}\sum_{i=0}^{N-1}(x_i-\bar{x})^2
 
 where :math:`x` is the sample set of elements, :math:`\bar{x}` is the
 sample mean, :math:`N` is the number of samples and :math:`\delta N` is


Test patch: