Repository: pytorch
Issue: Composite Compliance Problems Tracker
Labels: module: internals,triaged,module: linear algebra,module: __torch_dispatch__

### Motivation

Operations written in a non-composite-compliant way:
- may not be supported by meta tensors
- may not be supported by `__torch_dispatch__` Tensor subclasses
- may not be supported by functorch
- etc.

We should rewrite non-composite-compliant operations to be composite compliant to support existing subsystems and future subsystems.

See https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/README.md#compositeimplicitautograd-compliance for more details.

### Improve Composite Compliance testing
- [x] @zou3519 https://github.com/pytorch/pytorch/issues/72114
- [x] @zou3519 https://github.com/pytorch/pytorch/issues/72115
- [x] https://github.com/pytorch/pytorch/issues/74678

### List of non-composite-compliant-operators

All of these are tested for composite compliance in https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/common_methods_invocations.py. 

-> linalg ops
* [x] linalg.eig (due to use of `all close`)
* [x] linalg.eigvals (special path when requires_grad false which calls data_ptr) @zou3519 
* [x] linalg.householder_product (candidate for new_zeros({a, b}) - possibility of choosing from either subclass) (@kshitij12345 )
* [x] linalg.vander (due to use of `cumprod`)

-> nn ops
* [x] nn.functional.embedding 'Allowed exemption'
* [x] nn.functional.batch_norm (ubsan failure)
* [x] nn.functional.max_pool1d (special path when requires_grad false which calls data_ptr) @zou3519 
* [x] nn.functional.ctc_loss (access data_ptr)
* [x] nn.functional.instance_norm (only fails on ROCm because "Trying to use forward AD with miopen_batch_norm that does not support it because it has not been implemented yet." -- allowed exemption due to testing suite problem)
* [x] nn.functional.embedding_bag  'Allowed exemption'
* [x] nn.functional.conv_transpose3d @zou3519 
* [ ] nn.functional.gaussian_nll_loss (data-dependent error: https://github.com/pytorch/pytorch/blob/65ea3d062161f3aa5c8969b62ca322b0518300ae/torch/nn/functional.py#L2828-L2830)
* [x] nn.functional.batch_norm_without_cudnn

-> torch ops
* [x] allclose (new_op? : Implementation `at::isclose(self, other, rtol, atol, equal_nan).all().item<uint8_t>()` )
* [x] prod (item: computation diverges)
* [x] cumprod (uses prod?)
* [ ] index_reduce (item: computation diverges) -> [uncommon pattern](https://github.com/pytorch/pytorch/blob/f5b460b200611b6697d1b21926411f8190c102f3/torch/csrc/autograd/FunctionsManual.cpp#L6513-L6521)
* [ ] scatter_reduce_prod (item: computation diverges) -> [uncommon pattern](https://github.com/pytorch/pytorch/blob/f5b460b200611b6697d1b21926411f8190c102f3/torch/csrc/autograd/FunctionsManual.cpp#L6419-L6427)
* [x] eig (item: used for checks)
* [x] istft (item : used for checks)
* [x] nanquantile (item : used for checks)
* [x] quantile (item : used for checks)
* [x] cov (item : used for checks)
* [x] corrcoef (uses cov)
* [x] put (new_op)
* [ ] tensor_split (new_op)
* [ ] narrow (converts scalar tensor to raw c++ type with `item<>`)
* [x] take (uses `put`)
* [x] matrix_exp (candidate for new_zeros({a, b}) - possibility of choosing from either subclass)
* [x] as_strided_scatter (only fails for LazyTensor -- allowing an exemption for this)


-> Masked (call to `item`)
* [x] _masked.amax
* [x] _masked.amin
* [x] _masked.argmax
* [x] _masked.argmin
* [x] _masked.cumprod (uses cumprod)
* [x] _masked.cumsum
* [x] _masked.log_softmax
* [x] _masked.logaddexp
* [x] _masked.logsumexp
* [x] _masked.mean
* [x] _masked.median
* [x] _masked.norm
* [x] _masked.prod (uses prod)
* [x] _masked.softmax
* [x] _masked.softmin
* [x] _masked.std
* [x] _masked.sum
* [x] _masked.var

cc @ezyang @gchanan @zou3519 @bdhirsh @bhosmer @smessmer @ljk53 @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano @albanD