Repository: pytorch
Issue: [inductor][cpu]float32 dynamic shape maml_omniglot performance regression in 2025-01-13 nightly release
Labels: oncall: pt2,oncall: cpu inductor

### 🐛 Describe the bug

<p>dynamic shape default wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>torchbench</td>
      <td>maml_omniglot</td>
      <td>single</td>
      <td>5</td>
      <td>1.991409</td>
      <td>0.0011490320000000001</td>
      <td>0.0022881926660880004</td>
      <td>9.557705</td>
      <td>5</td>
      <td>2.569708</td>
      <td>0.000891765</td>
      <td>0.00229157565462</td>
      <td>9.459977</td>
      <td>0.77</td>
      <td>1.0</td>
      <td>0.78</td>
      <td>0.99</td>
    </tr>
  </tbody>

</table>

<p>dynamic shape cpp wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>torchbench</td>
      <td>maml_omniglot</td>
      <td>single</td>
      <td>5</td>
      <td>2.090643</td>
      <td>0.001101955</td>
      <td>0.002303794507065</td>
      <td>6.568006</td>
      <td>5</td>
      <td>2.732528</td>
      <td>0.000844825</td>
      <td>0.0023085079675999997</td>
      <td>6.547288</td>
      <td>0.77</td>
      <td>1.0</td>
      <td>0.77</td>
      <td>1.0</td>
    </tr>
  </tbody>

</table>

the last good commit: f8fcb9e7d38b82844d72ae32c27d1592db27a8e2
```
/workspace/pytorch# bash inductor_single_run.sh single inference performance torchbench maml_omniglot float32 first dynamic
Testing with dynamic shapes.
Testing with inductor.
single-thread testing....
loading model: 0it [00:00, ?it/s]
cpu  eval  maml_omniglot
running benchmark: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 249.55it/s]
1.778x
WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu]
dev,name,batch_size,speedup,abs_latency,compilation_latency,compression_ratio,eager_peak_mem,dynamo_peak_mem,calls_captured,unique_graphs,graph_breaks,unique_graph_breaks,autograd_captures,autograd_compiles,cudagraph_skips
cpu,maml_omniglot,5,1.777746,1.250640,27.412123,0.846500,47.260877,55.830938,14,1,0,0,0,0,1
```
the bad commit: 28b4992e7a60bb3fbb07c591099fa810557b4e57
```
/workspace/pytorch# bash inductor_single_run.sh single inference performance torchbench maml_omniglot float32 first dynamic
Testing with dynamic shapes.
Testing with inductor.
single-thread testing....
loading model: 0it [00:00, ?it/s]
cpu  eval  maml_omniglot
running benchmark: 100%|██████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 224.95it/s]
1.434x
WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu]
dev,name,batch_size,speedup,abs_latency,compilation_latency,compression_ratio,eager_peak_mem,dynamo_peak_mem,calls_captured,unique_graphs,graph_breaks,unique_graph_breaks,autograd_captures,autograd_compiles,cudagraph_skips
cpu,maml_omniglot,5,1.433554,1.590300,30.843010,0.770410,47.418573,61.549773,14,1,0,0,0,0,1
```

### Versions

</table><p>SW info</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>target_branch</th>
      <th>target_commit</th>
      <th>refer_branch</th>
      <th>refer_commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>torchbench</td>
      <td>main</td>
      <td>766a5e3a</td>
      <td>main</td>
      <td>766a5e3a</td>
    </tr>
    <tr>
      <td>torch</td>
      <td>main</td>
      <td>e0f67405a154e7f9ce1ca9533cbc1d156fe075d7</td>
      <td>main</td>
      <td>f2d6cfa6775601df5a038f7a4d0b37da75a53ed9</td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>main</td>
      <td>0.19.0a0+d23a6e1</td>
      <td>main</td>
      <td>0.19.0a0+d23a6e1</td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>main</td>
      <td>0.16.0a0+b0ebddc</td>
      <td>main</td>
      <td>0.16.0a0+b0ebddc</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>main</td>
      <td>2.6.0a0+b6d4675</td>
      <td>main</td>
      <td>2.6.0a0+b6d4675</td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>main</td>
      <td>0.7.1a0+0790338</td>
      <td>main</td>
      <td>0.7.1a0+0790338</td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>main</td>
      <td>nightly</td>
      <td>main</td>
      <td>nightly</td>
    </tr>
  </tbody>
</table>

</table>

Repro:
[inductor_single_run.sh](https://github.com/chuanqi129/inductor-tools/blob//main/scripts/modelbench/inductor_single_run.sh)
bash inductor_single_run.sh single inference performance torchbench maml_omniglot float32 first dynamic
Suspected guilty commit: 28b4992e7a60bb3fbb07c591099fa810557b4e57
[torchbench-maml_omniglot-inference-float32-dynamic-default-single-performance-drop_guilty_commit.log](https://github.com/user-attachments/files/18433106/torchbench-maml_omniglot-inference-float32-dynamic-default-single-performance-drop_guilty_commit.log)
cc @chauhang @penguinwu @chuanqi129