bfloat16 matmul gives incorrect result on CPU (without mkldnn)
Labels: module: cpu, triaged, module: bfloat16, module: linear algebra

### 🐛 Describe the bug

It seems that Pytorch's default CPU backend does not compute bfloat16 matmul correctly. The expected behavior is observed for certain dimensions only (the outcome is as expected for 256x256x256), but fails for others (1024x1024x1024).

In addition, it seems that the mkl-dnn backend correctly computes the result. However, it's only limited to avx512+ systems only.

```python
import torch

def check_correctness(a: torch.Tensor, b:torch.Tensor, expected: int):
    for mkldnn_flag in [True, False]:
        with torch.backends.mkldnn.flags(enabled=mkldnn_flag):
            c = torch.matmul(a, b)
            assert(torch.all(c == expected)), "Incorrect result with\n" \
                                              f"torch.backends.mkldnn.flags(enabled={mkldnn_flag}),\n" \
                                              f"and dtypes: {a.dtype}, {b.dtype}, {c.dtype}\n" \
                                              f"expected: {expected}\n" \
                                              f"got: {c}\n"


val = 1024
a = torch.ones(val, val)
b = torch.ones(val, val)

check_correctness(a, b, expected=val)

a = a.to(torch.bfloat16)
b = b.to(torch.bfloat16)

check_correctness(a, b, expected=val)
```

Executing the above code yields the following message:

```sh
Traceback (most recent call last):
  File "test_matmul.py", line 23, in <module>
    check_correctness(a, b, expected=val)
  File "test_matmul.py", line 7, in check_correctness
    assert(torch.all(c == expected)), "Incorrect result with\n" \
AssertionError: Incorrect result with
torch.backends.mkldnn.flags(enabled=False),
and dtypes: torch.bfloat16, torch.bfloat16, torch.bfloat16
expected: 1024
got: tensor([[256., 256., 256.,  ..., 256., 256., 256.],
        [256., 256., 256.,  ..., 256., 256., 256.],
        [256., 256., 256.,  ..., 256., 256., 256.],
        ...,
        [256., 256., 256.,  ..., 256., 256., 256.],
        [256., 256., 256.,  ..., 256., 256., 256.],
        [256., 256., 256.,  ..., 256., 256., 256.]], dtype=torch.bfloat16)
```

### Versions

```sh
PyTorch version: 1.12.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
Clang version: Could not collect
CMake version: version 3.24.1
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Quadro P1000
Nvidia driver version: 516.40
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.2
[pip3] torch==1.12.1+cpu
[pip3] torchaudio==0.12.1+cpu
[pip3] torchvision==0.13.1+cpu
[conda] No relevant packages
```

cc @VitalyFedyunin @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano