Repository: pytorch
Issue: RuntimeError: derivative for aten::mps_max_pool2d_backward is not implemented
Labels: triaged,module: mps

### 🐛 Describe the bug

Hi there, I encountered the runtime error "derivative for aten::mps_max_pool2d_backward is not implemented" when trying to use ```torch.autograd.grad``` to compute the Hessian of the loss function of a ResNet18 model w.r.t. model parameters, which is done by first using ```torch.autograd.grad``` to compute the gradient itself, and then using it again to compute the gradient of the gradient to get the Hessian. (Please kindly see lines 122-131 in the full traceback message below.)

Thank you so much!



```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [18], in <cell line: 2>()
      1 Hess_eigval, \
----> 2 Hess_eigval_density = Hess.LanczosApproxSpec(init_poly_deg=64, # iterations used to compute spectrum range
      3                                              poly_deg=256)

File ~/Documents/hessian.py:248, in Hessian.LanczosApproxSpec(self, init_poly_deg, poly_deg, spectrum_margin, poly_points, log_hessian, eps, denormalize)
    244 print('LanczosApproxSpec')
    246 print('Estimating spectrum range')
--> 248 lb, ub = self.compute_lb_ub(init_poly_deg)
    249 print('Estimated spectrum range:')
    250 print('[{}\t{}]'.format(lb, ub))

File ~/Documents/hessian.py:217, in Hessian.compute_lb_ub(self, init_poly_deg)
    216 def compute_lb_ub(self, init_poly_deg):
--> 217     ritz_val, S, alp, bet = self.Lanczos(init_poly_deg)
    219     theta_1 = ritz_val[0]
    220     theta_k = ritz_val[-1]

File ~/Documents/hessian.py:340, in Hessian.Lanczos(self, M)
    337 for j in tqdm(range(M)):
    338     sys.stdout.flush()
--> 340     v_next = self.mat_vec(v)
    342     if j:
    343         v_next = self.my_sub(v_next, self.my_mult_const(v_prev,bet[j-1]))

File ~/Documents/hessian.py:145, in Hessian.mat_vec(self, v)
    144 def mat_vec(self, v):
--> 145     Av = self.Hv(v)
    147     for eigvec, eigval in zip(self.vecs, self.vals):
    148         coeff = eigval * self.my_inner(eigvec, v)

File ~/Documents/hessian.py:127, in Hessian.Hv(self, v)
    122 elif self.hessian_type == 'Hessian':
    123     grad = torch.autograd.grad(loss,
    124                                self.model.parameters(),
    125                                create_graph=True)
--> 127     Hg_ = torch.autograd.grad(grad,
    128                               self.model.parameters(),
    129                               v)
    130 else:
    131     raise Exception('Wrong hessian type!')

File ~/Coding/miniconda3/envs/torch-gpu/lib/python3.9/site-packages/torch/autograd/__init__.py:300, in grad(outputs, inputs, grad_outputs, retain_graph, create_graph, only_inputs, allow_unused, is_grads_batched)
    298     return _vmap_internals._vmap(vjp, 0, 0, allow_none_pass_through=True)(grad_outputs_)
    299 else:
--> 300     return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    301         t_outputs, grad_outputs_, retain_graph, create_graph, t_inputs,
    302         allow_unused, accumulate_grad=False)

RuntimeError: derivative for aten::mps_max_pool2d_backward is not implemented
```

### Versions
Jupyter Lab, Python 3.9, PyTorch 1.13.0 Stable, M1 Mac
```
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.0 (arm64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 18:24:45)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-13.0-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0
[pip3] torchaudio==0.13.0
[pip3] torchvision==0.14.0
[conda] numpy                     1.23.3           py39h42add53_1  
[conda] numpy-base                1.23.3           py39hadd41eb_1  
[conda] pytorch                   1.13.0                  py3.9_0    pytorch
[conda] torchaudio                0.13.0                 py39_cpu    pytorch
[conda] torchvision               0.14.0                 py39_cpu    pytorch
```

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev