Repository: pytorch
Issue: Max pool with negative integer inputs and channels_last memory layout gives the wrong values
Labels: module: nn,triaged

### 🐛 Describe the bug

When running max pool operation on integral tensors with negative values and channels_last memory format the lowest value in the returned tensor is always zero.

``` python
import torch

t = torch.arange(24).view(1,1,4,6) - 12

expected = torch.max_pool2d(t, 2)
actual = torch.max_pool2d(t.clone(memory_format=torch.channels_last), 2)

assert torch.all(expected == actual).item()
# Note that the assert fails because
# expected is
# tensor([[[[-5, -3, -1],
#           [ 7,  9, 11]]]])
# actual is
#  tensor([[[[ 0,  0,  0],
#           [ 7,  9, 11]]]])
```

### Versions

Collecting environment information...
PyTorch version: 2.1.0.dev20230730+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.6 (https://github.com/conda-forge/clangdev-feedstock ceeebe884c3cfd7160cf5a43e147f94439fafee3)
CMake version: version 3.25.3
Libc version: glibc-2.35

Python version: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-79-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   48 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          60
On-line CPU(s) list:             0-59
Vendor ID:                       AuthenticAMD
Model name:                      AMD EPYC 7763 64-Core Processor
CPU family:                      25
Model:                           1
Thread(s) per core:              2
Core(s) per socket:              30
Socket(s):                       1
Stepping:                        1
BogoMIPS:                        4890.80
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip pku ospke vaes vpclmulqdq rdpid arch_capabilities
Virtualization:                  AMD-V
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       1.9 MiB (30 instances)
L1i cache:                       1.9 MiB (30 instances)
L2 cache:                        15 MiB (30 instances)
L3 cache:                        16 MiB (1 instance)
NUMA node(s):                    2
NUMA node0 CPU(s):               0-29
NUMA node1 CPU(s):               30-59
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] flake8==6.0.0
[pip3] flake8-bugbear==23.3.23
[pip3] flake8-comprehensions==3.12.0
[pip3] flake8-executable==2.1.3
[pip3] flake8-logging-format==0.9.0
[pip3] flake8-pyi==23.3.1
[pip3] flake8-simplify==0.19.3
[pip3] mypy==1.4.1
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.1
[pip3] torch==2.1.0.dev20230730+cpu
[pip3] torchvision==0.16.0.dev20230730+cpu
[pip3] triton==2.0.0
[conda] numpy                     1.25.2          py310ha4c1d20_0    conda-forge
[conda] torch                     2.1.0.dev20230730+cpu          pypi_0    pypi
[conda] torchvision               0.16.0.dev20230730+cpu          pypi_0    pypi

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki