Repository: pytorch
Issue: `torch.compiler.allow_in_graph` does not create a `call_module` op in fx.Graph in torch 2.3.0
Labels: triaged,oncall: pt2,module: dynamo

### 🐛 Describe the bug


I have been testing 2.3 but noticed some diverging behaviour with `allow_in_graph`
```python
@torch.compiler.allow_in_graph                      
class AllowInGraphLayer(torch.nn.Module):                                                               
    def forward(self, x):                                                                             
        return x + x                                                                                  
                                                                                                      
def test_allow_in_graph_dynamo():                                                                     
    class Model(torch.nn.Module):                  
        def __init__(self):                        
            super().__init__()                                                                        
            self.add = AllowInGraphLayer()                                                              
                                                                                                      
        def forward(self, x):                      
            return self.add(x)                     
                                                   
                                                                                                                                                                                   
    def backend(gm, _):                                                                                                                                                                                     
        gm.graph.print_tabular()                                                                      
        return gm.forward                                                                             
                                                   
    model = Model()                                                                                   
    model = torch.compile(model, backend=backend)                                                     
    model(torch.rand(10)) 
```

# Output for torch2.2
```
opcode       name           target         args                 kwargs
-----------  -------------  -------------  -------------------  --------
placeholder  l_x_           L_x_           ()                   {}
call_module  l__self___add  L__self___add  (l_x_,)              {}
output       output         output         ((l__self___add,),)  {}
```

# Output for torch2.3
```
opcode         name    target                   args          kwargs
-------------  ------  -----------------------  ------------  --------
placeholder    l_x_    L_x_                     ()            {}
call_function  add     <built-in function add>  (l_x_, l_x_)  {}
output         output  output                   ((add,),)     {}
```

Is this expected?

### Versions

For torch2.3.
```
>>> import torch
>>> torch.__version__
'2.3.0+cu118'
```

For torch2.2
```
>>> torch.__version__
'2.2.1+cu118'
```


```
Collecting environment information...        
PyTorch version: 2.3.0+cu118                                                                          
Is debug build: False                                                                                 
CUDA used to build PyTorch: 11.8                
ROCM used to build PyTorch: N/A                                                                                                                                                                             
                                                   
OS: Ubuntu 22.04.2 LTS (x86_64)                                                                       
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0                                                    
Clang version: Could not collect                                                                                                                                                                            
CMake version: Could not collect             
Libc version: glibc-2.35                                                                              

Python version: 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.19.0-1030-gcp-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 550.54.14            
cuDNN version: Could not collect
HIP runtime version: N/A      
MIOpen runtime version: N/A     
Is XNNPACK available: True
                                                   
CPU:                                                                                                  
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   46 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) CPU @ 2.20GHz
CPU family:                      6
Model:                           79
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
Stepping:                        0
BogoMIPS:                        4399.99
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology n
onstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp f
sgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat md_clear arch_capabilities

Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       128 KiB (4 instances)
L1i cache:                       128 KiB (4 instances)
L2 cache:                        1 MiB (4 instances)
L3 cache:                        55 MiB (1 instance)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:          Mitigation; IBRS
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT Host state unknown

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.3
[pip3] onnx==1.14.1
[pip3] onnxruntime==1.15.1
[pip3] torch==2.3.0+cu118
[pip3] torchaudio==2.3.0+cu118
[pip3] torchvision==0.18.0+cu118
[pip3] triton==2.3.0
[conda] Could not collect
```


cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng