Repository: pytorch
Issue: [CPU] jx_nest_base float32 both inductor and eager performance regression in 2024-08-26 nightly release
Labels: oncall: pt2,oncall: cpu inductor

### 🐛 Describe the bug

<p>fp32 static shape default wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>timm_models</td>
      <td>jx_nest_base</td>
      <td>single</td>
      <td>1</td>
      <td>1.235563</td>
      <td>0.27360534200000003</td>
      <td>0.338056637177546</td>
      <td>62.567019</td>
      <td>1</td>
      <td>1.201868</td>
      <td>0.234643102</td>
      <td>0.282010035714536</td>
      <td>61.797824</td>
      <td>1.03</td>
      <td>0.83</td>
      <td>0.86</td>
      <td>0.99</td>
    </tr>
  </tbody>

</table>

<p>fp32 dynamic shape default wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>timm_models</td>
      <td>jx_nest_base</td>
      <td>single</td>
      <td>1</td>
      <td>1.240841</td>
      <td>0.276368383</td>
      <td>0.342929220730103</td>
      <td>62.597383</td>
      <td>1</td>
      <td>1.192174</td>
      <td>0.23566079199999998</td>
      <td>0.280948669041808</td>
      <td>61.646299</td>
      <td>1.04</td>
      <td>0.82</td>
      <td>0.85</td>
      <td>0.98</td>
    </tr>
  </tbody>

</table>

<p>fp32 static shape cpp wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>timm_models</td>
      <td>jx_nest_base</td>
      <td>single</td>
      <td>1</td>
      <td>1.278338</td>
      <td>0.26669467300000005</td>
      <td>0.34092593489347406</td>
      <td>46.669321</td>
      <td>1</td>
      <td>1.202949</td>
      <td>0.23166499200000001</td>
      <td>0.278681170461408</td>
      <td>45.100968</td>
      <td>1.06</td>
      <td>0.82</td>
      <td>0.87</td>
      <td>0.97</td>
    </tr>
  </tbody>

</table>

<p>fp32 dynamic shape cpp wrapper</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>suite</th>
      <th>name</th>
      <th>thread</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>timm_models</td>
      <td>jx_nest_base</td>
      <td>single</td>
      <td>1</td>
      <td>1.277099</td>
      <td>0.27013244200000003</td>
      <td>0.34498587154575805</td>
      <td>47.364179</td>
      <td>1</td>
      <td>1.225369</td>
      <td>0.231195754</td>
      <td>0.283300109883226</td>
      <td>45.65043</td>
      <td>1.04</td>
      <td>0.82</td>
      <td>0.86</td>
      <td>0.96</td>
    </tr>
  </tbody>

</table>

### Versions

<p>SW info</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>target_branch</th>
      <th>target_commit</th>
      <th>refer_branch</th>
      <th>refer_commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>torchbench</td>
      <td>main</td>
      <td>23512dbe</td>
      <td>main</td>
      <td>23512dbe</td>
    </tr>
    <tr>
      <td>torch</td>
      <td>main</td>
      <td>2553278bae5993bd94bae4f04bf4586fb3f30d57</td>
      <td>main</td>
      <td>b4a1673a6741e183856cf3503f0574d3ac881ce0</td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>main</td>
      <td>0.19.0a0+d23a6e1</td>
      <td>main</td>
      <td>0.19.0a0+d23a6e1</td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>main</td>
      <td>0.16.0a0+b0ebddc</td>
      <td>main</td>
      <td>0.16.0a0+b0ebddc</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>main</td>
      <td>2.4.0a0+b3f6f51</td>
      <td>main</td>
      <td>2.4.0a0+b3f6f51</td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>main</td>
      <td>0.7.1a0+0790338</td>
      <td>main</td>
      <td>0.7.1a0+0790338</td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>main</td>
      <td>nightly</td>
      <td>main</td>
      <td>nightly</td>
    </tr>
  </tbody>
</table>

</table>

Repro:
[inductor_single_run.sh](https://github.com/chuanqi129/inductor-tools/blob/main/scripts/modelbench/inductor_single_run.sh)
bash inductor_single_run.sh single inference performance timm_models jx_nest_base float32 first static cpp
Suspected guilty commit: fb26b843906bbad5e28d1edccf298c74b8e00492
[timm_models-jx_nest_base-inference-float32-static-cpp-single-performance-drop_guilty_commit.log](https://github.com/user-attachments/files/16784943/timm_models-jx_nest_base-inference-float32-static-cpp-single-performance-drop_guilty_commit.log)
cc @ezyang @chauhang @penguinwu @WeizhuoZhang-intel @chuanqi129