Repository: pytorch
Issue: Documentation for torch.finfo doesn't match implementation
Labels: module: docs,triaged,module: numpy

## 🐛 Bug

The output of `torch.finfo(torch.float16)` seems to be incorrect.

## To Reproduce

Steps to reproduce the behavior:

```
>>> import torch
>>> torch.finfo(torch.float16)
finfo(resolution=0.001, min=-65504, max=65504, eps=0.000976562, tiny=6.10352e-05, dtype=float16)
>>> torch.tensor([1e-7], dtype=torch.float16)
tensor([1.1921e-07], dtype=torch.float16)
>>> torch.tensor([1.0], dtype=torch.float16) + torch.tensor([0.0006], dtype=torch.float16)
tensor([1.0010], dtype=torch.float16)
```

## Expected behavior

Notice that `1e-7` is representable in fp16 but finfo indicates a `tiny` value much larger than that.
Also, `1.0 + 6e-4 != 1.0` but the reported `eps` is larger than `6e-4`.

I expect `tiny` to equal the smallest positive value that's representable in fp16 and `eps` to equal the smallest representable value such that `1.0 + eps != 1.0`.

## Environment

```
PyTorch version: 1.8.0a0+7aef199
Is debug build: False
CUDA used to build PyTorch: 11.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.19.4
Libc version: glibc-2.27

Python version: 3.8 (64-bit runtime)
Python platform: Linux-4.15.0-118-generic-x86_64-with-glibc2.10
Is CUDA available: True
CUDA runtime version: 11.2.152
GPU models and configuration:
GPU 0: GeForce RTX 3090
GPU 1: GeForce GTX 1080 Ti
GPU 2: GeForce GTX 1080 Ti
GPU 3: GeForce GTX 1080 Ti

Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.1.0
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.1.0
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.8.0a0+7aef199
[pip3] torchaudio==0.7.0a0+a853dff
[pip3] torchvision==0.8.2
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              hfd86e86_1
[conda] magma-cuda102             2.5.2                         1    pytorch
[conda] mkl                       2020.2                      256
[conda] mkl-include               2020.2                      256
[conda] mkl-service               2.3.0            py38he904b0f_0
[conda] mkl_fft                   1.2.0            py38h23d657b_0
[conda] mkl_random                1.1.1            py38h0573a6f_0
[conda] numpy                     1.19.5                   pypi_0    pypi
[conda] torch                     1.8.0a0+7aef199          pypi_0    pypi
[conda] torchaudio                0.7.0a0+a853dff          pypi_0    pypi
[conda] torchvision               0.8.2                    pypi_0    pypi
```

cc @brianjo @mruberry @rgommers @heitorschueroff