Repository: pytorch
Issue: [TorchScript Usability] Behaviors of used-defined subclasses of torch.tensor are silently ignored
Labels: oncall: jit,TSRootCause:InvalidCustomClass,TSUsability

**Problem description**

Behaviors of user-defined subclasses of torch.tensor are silently ignored. Instead it should report error when users try to subclass builtin nominal types 

Test case: [test_subclass.py](https://github.com/gmagogsfm/TorchScriptStudy/blob/main/tensor_subclass.py). 

This test case overwrites `__len__` of `torch.Tensor` but the overwritten method is silently ignored
```
> python test-tensor-subclass.py
Eager and jit behavior mismatching: 123456 vs 2
```

Test case: [test-tensor-subclass2.py](https://github.com/penguinwu/hodgepodge/blob/master/TorchScriptStudy/bad-examples/test-tensor-subclass2.py)

This test case adds a new method `myNumber()` to a subclass of `torch.Tensor`, but when invoking the method, the method was not recognized (because subclasses were not compiled)
```
python test-tensor-subclass2.py 
Eager:  123456
Traceback (most recent call last):
  File "test-tensor-subclass2.py", line 18, in <module>
    scripted_fn = torch.jit.script(fn)
  File "/home/pengwu/local/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/jit/_script.py", line 939, in script
    fn = torch._C._jit_script_compile(
RuntimeError: 
Tried to access nonexistent attribute or method 'myNumber' of type 'Tensor'.:
  File "test-tensor-subclass2.py", line 13
def fn(x: TensorSubClass):
    return x.myNumber()
           ~~~~~~~~~~ <--- HERE
``` 

**Root-cause**

In `try_ann_to_type()`, we simply recognize any subclass of `torch.Tensor` as `TensorType` w/o compiling the classes (if it is a user-defined class). Therefore, subclasses of `torch.Tensor` are just internalized as the built-in `TensorType`: 

```
def try_ann_to_type(ann, loc):
    if ann is None:
        return TensorType.getInferred()
    if inspect.isclass(ann) and issubclass(ann, torch.Tensor):
        return TensorType.get()
   ...
```

**How to fix it?**

We need to figure out whether we allow subclassing `torch.Tensor`. Our current implementation allows it (i.e., laxed type checkin) but does not really support it (subclass behaviors do not take effect).

- Option 1: disallow custom subclassing of tensors, but may allow builtin subclasses of `torch.Tensor` from `torch`. **Pros** consistent w/ our current rule of not supporting subclassing in custom class definition.  **Cons**: it may break existing codes (I can easily imagine that users subclass `torch.Tensor` but only use default behaviors of `torch.Tensor`. ).
- Option 2: allow custom subclassing of tensors but do not allow overwriting existing methods in `torch.Tensor` (because that would be a more elaborate change). **Pros** codes that rely on the subclass behaviors would behave correctly; **Const** also may break existing codes because subclasses of `torch.Tensor` need to be compiled to Torch IR, it is possible that some unused methods of `torch.Tensor` subclasses would fail the compilation.
- Option 3: wait to implement Option #2 until we support partially compiled TorchScript custom classes. With partially compiled TorchScript custom classes, our compilation (type-checking) surface is smaller, so less likely to break existing codes

Related to #50444 

@gmagogsfm  This one is not one of the trivial fixes, probably need some discussions. 



cc @gmagogsfm