Global lambda function is not properly guarded
Labels: triaged, oncall: pt2, module: dynamo

The below code alters the lambda function `toy_sum` at the 3rd iteration, but the dynamo does not catch that modification.

**Code**

```python
import torch
import torchdynamo

toy_sum = lambda tlist: sum(tlist)

def toy_example(tlist):
    ret_val = 0
    for _ in range(5):
        ret_val += toy_sum(tlist)
    return ret_val


tlist = [torch.ones(10), torch.ones(10), torch.ones(10)]

print("with dynamo")
with torchdynamo.optimize("eager"):
    for i in range(5):
        if i == 2:  # alter `toy_sum` after the third iteration
            toy_sum = lambda tlist: tlist[0]
        print(toy_example(tlist))


print("without dynamo")
toy_sum = lambda tlist: sum(tlist)
for i in range(5):
    if i == 2:  # alter `toy_sum` after the third iteration
        toy_sum = lambda tlist: tlist[0]
    print(toy_example(tlist))
```

**Result**

```
with dynamo
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
without dynamo
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
```

<details>
<summary>logs with torchdynamo.config.debug</summary>

```
with dynamo
skipping __init__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/contextlib.py
skipping __enter__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/contextlib.py

__compiled_fn_0 <eval_with_key>.1
opcode         name      target                    args                kwargs
-------------  --------  ------------------------  ------------------  --------
placeholder    tlist_0_  tlist_0_                  ()                  {}
placeholder    tlist_1_  tlist_1_                  ()                  {}
placeholder    tlist_2_  tlist_2_                  ()                  {}
call_function  add       <built-in function add>   (0, tlist_0_)       {}
call_function  add_1     <built-in function add>   (add, tlist_1_)     {}
call_function  add_2     <built-in function add>   (add_1, tlist_2_)   {}
call_function  add_3     <built-in function add>   (add_2, 0)          {}
call_function  add_4     <built-in function add>   (0, tlist_0_)       {}
call_function  add_5     <built-in function add>   (add_4, tlist_1_)   {}
call_function  add_6     <built-in function add>   (add_5, tlist_2_)   {}
call_function  iadd      <built-in function iadd>  (add_3, add_6)      {}
call_function  add_7     <built-in function add>   (0, tlist_0_)       {}
call_function  add_8     <built-in function add>   (add_7, tlist_1_)   {}
call_function  add_9     <built-in function add>   (add_8, tlist_2_)   {}
call_function  iadd_1    <built-in function iadd>  (iadd, add_9)       {}
call_function  add_10    <built-in function add>   (0, tlist_0_)       {}
call_function  add_11    <built-in function add>   (add_10, tlist_1_)  {}
call_function  add_12    <built-in function add>   (add_11, tlist_2_)  {}
call_function  iadd_2    <built-in function iadd>  (iadd_1, add_12)    {}
call_function  add_13    <built-in function add>   (0, tlist_0_)       {}
call_function  add_14    <built-in function add>   (add_13, tlist_1_)  {}
call_function  add_15    <built-in function add>   (add_14, tlist_2_)  {}
call_function  iadd_3    <built-in function iadd>  (iadd_2, add_15)    {}
output         output    output                    ((iadd_3,),)        {}

ORIGINAL BYTECODE toy_example test.py 11
 12           0 LOAD_CONST               1 (0)
              2 STORE_FAST               1 (ret_val)

 13           4 LOAD_GLOBAL              0 (range)
              6 LOAD_CONST               2 (5)
              8 CALL_FUNCTION            1
             10 GET_ITER
           12 FOR_ITER                16 (to 30)
             14 STORE_FAST               2 (_)

 14          16 LOAD_FAST                1 (ret_val)
             18 LOAD_GLOBAL              1 (toy_sum)
             20 LOAD_FAST                0 (tlist)
             22 CALL_FUNCTION            1
             24 INPLACE_ADD
             26 STORE_FAST               1 (ret_val)
             28 JUMP_ABSOLUTE           12

 15     >>   30 LOAD_FAST                1 (ret_val)
             32 RETURN_VALUE

MODIFIED BYTECODE
 11           0 LOAD_GLOBAL              2 (__compiled_fn_0)
              2 LOAD_FAST                0 (tlist)
              4 LOAD_CONST               1 (0)
              6 BINARY_SUBSCR
              8 LOAD_FAST                0 (tlist)
             10 LOAD_CONST               3 (1)
             12 BINARY_SUBSCR
             14 LOAD_FAST                0 (tlist)
             16 LOAD_CONST               4 (2)
             18 BINARY_SUBSCR
             20 CALL_FUNCTION            3
             22 UNPACK_SEQUENCE          1
             24 RETURN_VALUE


GUARDS:
 - 
            local 'tlist' LIST_LENGTH"
            {
                'guard_types': ['LIST_LENGTH'],
                'code': ['___check_type_id(tlist, 4346971272)', 'len(tlist) == 3'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fc50003bae0; to 'type' at 0x103198488 (list)>
            }
            
 - 
            local 'tlist[0]' TENSOR_MATCH"
            {
                'guard_types': ['TENSOR_MATCH'],
                'code': None,
                'obj_weakref': <weakref at 0x7fc4f17e0310; to 'Tensor' at 0x7fc4f018f090>
                'guarded_class': <weakref at 0x7fc4f088a040; to 'torch._C._TensorMeta' at 0x7fc4ea5964b0 (Tensor)>
            }
            
 - 
            local 'tlist[1]' TENSOR_MATCH"
            {
                'guard_types': ['TENSOR_MATCH'],
                'code': None,
                'obj_weakref': <weakref at 0x7fc4f17e0360; to 'Tensor' at 0x7fc4f0189e00>
                'guarded_class': <weakref at 0x7fc4f088a040; to 'torch._C._TensorMeta' at 0x7fc4ea5964b0 (Tensor)>
            }
            
 - 
            local 'tlist[2]' TENSOR_MATCH"
            {
                'guard_types': ['TENSOR_MATCH'],
                'code': None,
                'obj_weakref': <weakref at 0x7fc4f17e03b0; to 'Tensor' at 0x7fc4f0251450>
                'guarded_class': <weakref at 0x7fc4f088a040; to 'torch._C._TensorMeta' at 0x7fc4ea5964b0 (Tensor)>
            }
            
 - 
            global 'sum' BUILTIN_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
 - 
            global 'range' BUILTIN_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
 - 
            global 'toy_sum' FUNCTION_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            

skipping _fn /Users/kinetc/work/torchdynamo/torchdynamo/eval_frame.py
skipping nothing /Users/kinetc/work/torchdynamo/torchdynamo/eval_frame.py
skipping __exit__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/contextlib.py
skipping __repr__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor.py
skipping _str /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping __init__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/autograd/grad_mode.py
skipping is_scripting /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_jit_internal.py
skipping __enter__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/autograd/grad_mode.py
skipping __init__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/autograd/grad_mode.py
skipping _str_intern /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping unpack_dual /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/autograd/forward_ad.py
skipping _tensor_str /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping __init__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping __exit__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/autograd/grad_mode.py
skipping tensor_totype /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping __iter__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor.py
skipping __format__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor.py
skipping _tensor_str_with_formatter /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping _vector_str /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping width /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping <listcomp> /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping _val_formatter /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping format /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping <listcomp> /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping <listcomp> /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping _add_suffixes /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/_tensor_str.py
skipping __instancecheck__ /Users/kinetc/opt/anaconda3/envs/dynamo/lib/python3.8/site-packages/torch/nn/parameter.py
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
skipping __exit__ /Users/kinetc/work/torchdynamo/torchdynamo/eval_frame.py
without dynamo
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([15., 15., 15., 15., 15., 15., 15., 15., 15., 15.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
tensor([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])
```
</details>

Is there any resources that this kind of behavior (modifying lambda variable at a runtime) is not considered while tracing?


cc: @nunoplopes

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire