Repository: pytorch
Issue: Overflow error in torch.linalg uncaught
Labels: high priority,module: numerical-stability,triaged,module: multithreading,module: linear algebra,module: correctness (silent),module: reductions

## 🐛 Bug

The torch.linalg.norm function silently produces incorrect results.
(It seems that this is specific to the setup used in Google Colab, on my local machine it works).

## To Reproduce

```
import torch

def custom_norm(tensor):
    return torch.sqrt((tensor**2).sum())

def test_norm(N):
    A = torch.ones((N,N))
    print(f"Size: {A.shape} \t:: Custom norm: {custom_norm(A)} \t:: Torch norm {torch.linalg.norm(A)}")

test_norm(10)
test_norm(100)
test_norm(1000)
test_norm(10000)
``` 
Output is 
```
Size: torch.Size([10, 10]) 	:: Custom norm: 10.0 	:: Torch norm 10.0
Size: torch.Size([100, 100]) 	:: Custom norm: 100.0 	:: Torch norm 100.0
Size: torch.Size([1000, 1000]) 	:: Custom norm: 1000.0 	:: Torch norm 1000.0
Size: torch.Size([10000, 10000]) 	:: Custom norm: 10000.0 	:: Torch norm 4096.0
```

## Expected behavior
The torch norm is wrong in the last line, probably due to some internal overflow (?). 
Expected behavior would be to report that the capacity of the datatype is not high enough, e.g. raise some Error.

Then the user can correct it to the following, which will yield the correct output.

```
import torch
def custom_norm(tensor):
    return torch.sqrt((tensor**2).sum())
def test_norm(N):
    A = torch.ones((N,N),dtype=torch.float64)
    print(f"Size: {A.shape} \t:: Custom norm: {custom_norm(A)} \t:: Torch norm {torch.linalg.norm(A)}")
test_norm(10)
test_norm(100)
test_norm(1000)
test_norm(10000)
```
## Environment

```
Collecting environment information...
PyTorch version: 1.9.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.12.0
Libc version: glibc-2.26

Python version: 3.7.12 (default, Sep 10 2021, 00:21:48)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB
Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.9.0+cu102
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.10.0
[pip3] torchvision==0.10.0+cu102
[conda] Could not collect
```

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano