Conv1d step-by-step numerical error 
Labels: needs reproduction, triaged, module: macos, module: determinism, module: arm

### 🐛 Describe the bug - numerical inconsistency, conv1d, conv2d (and other tags)

When I apply `F.conv1d` with some kernel and stride I want to get same result when I do it on input feature split manually shifiting this stride. That is important because error is stacking over layers of my system and creates big noise total.
All code run on cpu only.

Here is my code which shows inconsistency (non-zero numerical error):
```python3
import torch
import torch.nn.functional as F

# params for running F.conv1d:
_hop = 1
_n_frames = 10
_window = _hop * 2
_frames_step = 1

# I also provide some deterministic values to match params above
# feature = torch.randn(_n_frames * _hop, dtype=torch.float32)
feature = torch.arange(_n_frames * _hop, dtype=torch.float32)
# _kernel = torch.randn(8, 1, _window, dtype=torch.float32)
_kernel = torch.tensor([
    [[-1.2982e-04, -2.1533e-01]],
    [[-2.2290e-01,  2.4641e-01]],
    [[-7.0638e-01,  8.7808e-01]],
    [[ 9.2179e-01,  3.0907e-01]],
    [[-2.3548e-01,  1.6494e+00]],
    [[-1.1634e+00,  2.9117e-01]],
    [[-1.6536e+00,  1.1718e+00]],
    [[ 1.0997e+00, -1.2599e+00]]
], dtype=torch.float32)

print("feature", feature.shape)
print("kernel", _kernel.shape)

with torch.no_grad():
    # original usage
    output_orig = F.conv1d(
        feature.reshape(1, 1, -1),
        _kernel,
        stride=_hop,
        padding=0,
    ).cpu().transpose(1,2)

    # with splits
    output_splits = []
    for i in range(0, _n_frames, _frames_step):
        _feat = feature[i*_hop:(i+_frames_step)*_hop + (_window - _hop)]
        if _feat.shape[0] < _window:
            continue
        output_cached_split = F.conv1d(
            _feat.reshape(1, 1, -1),
            _kernel,
            stride=_hop,
            padding=0,
        ).cpu().transpose(1,2)
        output_splits.append(output_cached_split)
    output_split = torch.cat(output_splits, dim=1)

print(output_orig.shape, output_orig.dtype, output_orig.min(), output_orig.max())
print(output_split.shape, output_split.dtype, output_split.min(), output_split.max())


error = (output_orig[0] - output_split[0]).T
error_1d = np.abs(error).mean(0)
print("Errors for each timestamp:", error_1d)

# my functions for visualization, they just print better view
from my_utils import plot_output, plot_error
plot_output(output_orig, output_split, f=None)
plt.show()
plot_error(output_orig, output_split, shift=0, f=None)
plt.show()
```

output:
```
feature torch.Size([10])
kernel torch.Size([8, 1, 2])
torch.Size([1, 9, 8]) torch.float32 tensor(-6.6867) tensor(12.9608)
torch.Size([1, 9, 8]) torch.float32 tensor(-6.6867) tensor(12.9608)
Errors for each timestamp: tensor([0.0000e+00, 0.0000e+00, 1.4901e-08, 0.0000e+00, 2.2352e-08, 1.5274e-07,
        1.4901e-08, 0.0000e+00, 1.1921e-07])
```
![image](https://github.com/pytorch/pytorch/assets/28161779/4d426065-b1d6-4f29-b4f4-78f69d415b40)
![image](https://github.com/pytorch/pytorch/assets/28161779/e761527f-01ad-4be8-a6dc-2849432b5041)

But I expect error 0.0 everywhere.


And also I've run same code but **with double precision**.
Output:
```
feature torch.Size([10])
kernel torch.Size([8, 1, 2])
torch.Size([1, 9, 8]) torch.float64 tensor(-6.6867, dtype=torch.float64) tensor(12.9608, dtype=torch.float64)
torch.Size([1, 9, 8]) torch.float64 tensor(-6.6867, dtype=torch.float64) tensor(12.9608, dtype=torch.float64)
Errors for each timestamp: tensor([0.0000e+00, 0.0000e+00, 2.7756e-17, 0.0000e+00, 1.5266e-16, 6.2450e-17,
        1.6653e-16, 0.0000e+00, 0.0000e+00], dtype=torch.float64)
```
![image](https://github.com/pytorch/pytorch/assets/28161779/3a03b2ee-0f21-45ea-9e81-ac10169b41e0)
![image](https://github.com/pytorch/pytorch/assets/28161779/956163c8-e7d1-4108-a249-999bfea1d014)

I think there should be no numerical error at all.

I've also seen other issue about it, but I think multichannel and strides supposed to be computed differently.
https://github.com/pytorch/pytorch/issues/23042

### Versions

I use homebrew

Collecting environment information...
PyTorch version: 2.0.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.3.1 (x86_64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: version 3.26.3
Libc version: N/A

Python version: 3.9.17 (main, Jun  6 2023, 14:44:03)  [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-12.3.1-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz

Versions of relevant libraries:
[pip3] numpy==1.23.1
[pip3] pytorch-lightning==1.7.4
[pip3] pytorch-ranger==0.1.1
[pip3] torch==2.0.0
[pip3] torch-audiomentations==0.10.1
[pip3] torch-optimizer==0.1.0
[pip3] torch-pitch-shift==1.2.4
[pip3] torch-stoi==0.1.2
[pip3] torch-time-stretch==1.0.2
[pip3] torchaudio==2.0.1
[pip3] torchmetrics==0.9.1
[pip3] torchsummary==1.5
[conda] Could not collect

cc @malfet @albanD @mruberry @kurtamohler