Repository: pytorch/pytorch

Issue:
[FX] getattr default not respected after symbolic_trace
## 🐛 Bug

Is during symbolic tracing a builtin.getattr is traced and it uses a default value, that default value is lost.

## To Reproduce

Steps to reproduce the behavior:

```
import torch
import torch.nn as nn
from torch.fx import symbolic_trace

class TestModule(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, a: torch.Tensor) -> torch.Size:
        return getattr(a, "nonexistent_attr", torch.Size([1,2]))

m = TestModule()
traced = symbolic_trace(m)

# WRONG: AttributeError: 'Tensor' object has no attribute 'nonexistent_attr'
# traced(torch.rand(3, 4))

print(traced.graph)
"""
graph():
    %a : torch.Tensor [#users=1] = placeholder[target=a]
    %getattr_1 : [#users=1] = call_function[target=builtins.getattr](args = (%a, nonexistent_attr), kwargs = {})
    return getattr_1
"""
```

## Expected behavior

If you inspect `traced.graph` you'll notice that no default value for getattr is provided, although it should be `torch.Size([1, 2])`. When you run the traced module you will encounter an exception about `a` not having `nonexistent_attr`, which should not be thrown because the default value to return instead be returned/should prevent the exception.


Discussion:
Hm I'm not actually sure if it's possible to intercept the default behavior here. `getattr` is implemented as this C builtin function: https://github.com/python/cpython/blob/9af34c935185eca497617a216d141c72ffaeae9c/Python/bltinmodule.c#L1108 and the path that handles default values is entirely contained within that implementation; the default value is not visible in the dispatch implementation here https://github.com/python/cpython/blob/0fd27375cabd12e68a2f12cfeca11a2d5043429e/Objects/object.c#L944 which eventually delegates down into the object's `__getattr__` method

Potentially we could fix this externally by patching `builtins.getattr` during trace timeand installing the logic that way.

Resolving patch:
diff --git a/torch/fx/graph.py b/torch/fx/graph.py
--- a/torch/fx/graph.py
+++ b/torch/fx/graph.py
@@ -923,11 +923,13 @@ def emit_node(node : Node):
                     return
                 qualified_name = _get_qualified_name(node.target)
                 global_name = add_global(qualified_name, node.target)
+                # special case for getattr: node.args could be 2-argument or 3-argument
+                # 2-argument: attribute access; 3-argument: fall through to attrib function call with default value
                 if global_name == 'getattr' and \
                    isinstance(node.args, tuple) and \
                    isinstance(node.args[1], str) and \
-                   node.args[1].isidentifier():
-                    # pretty print attribute access
+                   node.args[1].isidentifier() and \
+                   len(node.args) == 2:
                     body.append(f'{repr(node)}{maybe_type_annotation} = {_format_target(repr(node.args[0]), node.args[1])}')
                     return
                 body.append(f'{repr(node)}{maybe_type_annotation} = {global_name}({_format_args(node.args, node.kwargs)})')


Test patch:
diff --git a/test/test_fx.py b/test/test_fx.py
--- a/test/test_fx.py
+++ b/test/test_fx.py
@@ -95,6 +95,8 @@ def a_lifted_leaf2(a, b):
 
 wrap('len')
 
+wrap('getattr')
+
 @wrap
 def wrapped_via_decorator(a):
     return a + 1
@@ -926,6 +928,14 @@ def forward(self, x):
         self.assertEqual(traced2(inp), inp + 3.0)
         self.assertIs(len, builtins.len)
 
+    def test_torch_fx_getattr(self):
+        class FXGetattrTest(torch.nn.Module):
+            def forward(self, x):
+                return getattr(x, 'nonexistent_attr', torch.Tensor([2, 3]))
+
+        traced = symbolic_trace(FXGetattrTest())
+        self.assertEqual(traced(torch.rand(3, 4)), torch.Tensor([2, 3]))
+
     def test_sqrt(self):
         class Sqrt1(torch.nn.Module):
             def forward(self, x):