Repository: pytorch
Issue: Codegen refactoring master task
Labels: triaged

This is the master task for all known codegen issues. The current state of our codegen is hard to maintain, it has a fair amount of bugs, hacks and hard coded logic which is not documented anywhere and requires a lot of domain knowledge. 

Known Issues:

- [ ] TensorOptions. Logic around TensorOptions manipulations is probably the hardest and the most confusing part of the whole codegen. In order to add a new entry to TensorOptions, one would have to make a big set of changes in pretty much every code gen script. It has to be refactored and cleaned up. 

- [ ] Some of the generated python bindings are wrong. As an example, take a look at `THPVariable_empty_like` or`THPVariable_randn_like` in python_torch_functions.cpp. For one of the overloads, `requires_grad` will be ignored due to wrong arg index.
```
>>> a = torch.ones(2, 3, requires_grad=True)
>>> print(torch.rand_like(a).requires_grad)
False
```

- [ ] All factory methods in native_functions.yaml should have the same schema with regards to Tensor Options. [issue](https://github.com/pytorch/pytorch/issues/30763)

- [ ] **Add `requires_grad` to native_functions.yaml and potentially to TensorOptions object**
Ideally, `requires_grad` has to be added to all function in native_functions.yaml that have TensorOptions in it. Its not the case right now so there are many places in code gen that add `requires_grad` on a fly which adds significant complexity to overall logic. [Related Issue](https://github.com/pytorch/pytorch/issues/30761)

- [ ] **Fix processing default values from native_functions.yaml** 
As an example, take a look at `tril_indices` / `triu_indices`, both have special logic around code gen to respect the default value.

- [ ] Look into how TensorOptions behaves when dtype is not set and the call to .dtype() is made. you get the current default dtype, which is questionable. This causes hack around `arange` in pybind layer (python_torch_functions.cpp)

- [ ] Move all code gen scripts and related utility scrips to one place. 

- [ ] Add high level documentation which will describe what is being used as an input, what scripts are producing what and why. Without knowledge transfer or prior experience it takes a while to figure it all out. 

- [ ] Bad overload order for zeros_like. [Issue](https://github.com/pytorch/pytorch/issues/19685)
-----------------
Once this [stack](https://github.com/pytorch/pytorch/pull/30983) lands, all these issues will make sense:
- [ ] Calling `.is_pinned()` from C++ code causes exception in case of CUDA tensor.
See basic.cpp

- [ ] **Use only optional version of tensor options when getting them from TensorOptions object**
There is a huge difference in behavior between calling `TensorOptions.dtype()` and `TensorOptions.dtype_opt()`. Thats true for all tensor options. Ideally we want to use only `.*_opt()` calls but due to legacy reasons we cant do that right away. This should be cleaned up.

- [ ] **Remove special code template for `.to` from function_wrapper.py**
When calling to `.to` we expect unset `requires_grad`. There is special template that checks for that. We should come up with a better solution.

- [ ] We should be able to calculate backend without creating a TO object. see TensorFactories.cu

- [ ] **All schemas in native_functions.yaml that have TensorOptions should be have optional ScalarType, Layout, Device and pin memory**
There are several functions that dont follow this rule which causes several hacks in the code base. This is closely related to issue: _Use only optional version of tensor options when getting them from TensorOptions object_ which is described above.

- [ ] **Overload TensorOptions `merge_in()` to accept uncollapsed TensorOptions**
As we are moving away from TensorOptions object we should introduce overload  for `merge_in()` which will work with optional dtype, layout, device and pin_memory options.
-----------------
TODO:

- Check if hard coded pieces can be removed from codegen templates. 
- Clean up dead code from codeine scripts