[inductor][cpu] [dynamic shapes][cppwrapper] performance regression
Labels: module: cpu, triaged, oncall: pt2

<p>new_perf_regression in 2023-10-08 vs 2023-09-24 </p>
<p>Note: multi threads secnario for models above first *, single thread for models between two *</p>

<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
    <tr>
      <td>pyhpc_isoneutral_mixing</td>
      <td>1</td>
      <td>45.356827</td>
      <td>5.9067e-05</td>
      <td>0.002679091700409</td>
      <td>20.751223</td>
      <td>1</td>
      <td>51.871931</td>
      <td>5.2059000000000004e-05</td>
      <td>0.002700400855929</td>
      <td>20.048833</td>
      <td>0.87</td>
      <td>1.01</td>
      <td>0.88</td>
      <td>0.97</td>
    </tr>
    <tr>
      <td>timm_efficientnet</td>
      <td>1</td>
      <td>1.373197</td>
      <td>0.014694839999999999</td>
      <td>0.020178910203479998</td>
      <td>28.523822</td>
      <td>1</td>
      <td>1.612629</td>
      <td>0.012724577</td>
      <td>0.020520021882933004</td>
      <td>29.662758</td>
      <td>0.85</td>
      <td>1.02</td>
      <td>0.87</td>
      <td>1.04</td>
    </tr>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
  </tbody>
</table>
<p>SW info</p>
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>SW</th>
      <th>Nightly commit</th>
      <th>Main commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Pytorch</td>
      <td>f90c2af</td>
      <td>459cef8</td>
    </tr>
    <tr>
      <td>Torchbench</td>
      <td>/</td>
      <td>ffbbebb9</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>475b6ae</td>
      <td>ede4309</td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>142d029</td>
      <td>45e4b8c</td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>8636bf3</td>
      <td>4ac707a</td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>eb9bf61</td>
      <td>d76d92c</td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>0200b11</td>
      <td>/</td>
    </tr>
  </tbody>
</table>
<p>Reference SW info (nightly)</p>
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>item</th>
      <th>commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>torchbench</td>
      <td>ffbbebb9</td>
    </tr>
    <tr>
      <td>torch</td>
      <td>2.2.0a0+git24d07d1</td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>0.16.0a0+8636bf3</td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>0.16.0a0+142d029</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>2.2.0a0+475b6ae</td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>0.7.0a0+eb9bf61</td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>0200b11</td>
    </tr>
  </tbody>
</table>
<p>Repro</p>
<a href=https://github.com/chuanqi129/inductor-tools/blob/yudong/aws_auto/scripts/modelbench/inductor_single_run.sh>
  inductor_single_run.sh </a>
<code>
bash inductor_single_run.sh multiple/single inference performance torchbench/huggingface/timm_models model float32 first dynamic cpp 0
</code>

<suspected first bad commit>
pyhpc_isoneutral_mixing:  d04b35e7e31547c3cdb9684e1dd0b556bf593a93 is the first bad commit
timm_efficientnet:

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @ezyang @msaroufim @wconstab @bdhirsh @anijain2305