[inductor][cpu] performance regression
Labels: triaged, oncall: pt2, module: inductor, module: cpu inductor

<p>new_perf_regression: 2023-09-20 nightly release vs 2023-09-17 nightly release</p>

<p>Note: multi threads secnario for models above first *, single thread for models between two *</p>

<p>new_perf_regression</p>
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
    <tr>
      <td>llama</td>
      <td>1</td>
      <td>0.077857</td>
      <td>0.416423326</td>
      <td>0.032421470892381996</td>
      <td>39.62569</td>
      <td>1</td>
      <td>0.318457</td>
      <td>0.102005337</td>
      <td>0.032484313605009</td>
      <td>38.0415</td>
      <td>0.24</td>
      <td>1.0</td>
      <td>0.24</td>
      <td>0.96</td>
    </tr>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
  </tbody>
</table>
<p>SW info</p>
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>SW</th>
      <th>Nightly commit</th>
      <th>Main commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Pytorch</td>
      <td>00ae5fa</td>
      <td>1b3e5b5</td>
    </tr>
    <tr>
      <td>Torchbench</td>
      <td>/</td>
      <td>ffbbebb9</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>475b6ae</td>
      <td>ede4309</td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>142d029</td>
      <td>45e4b8c</td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>8636bf3</td>
      <td>4ac707a</td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>eb9bf61</td>
      <td>d76d92c</td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>0200b11</td>
      <td>/</td>
    </tr>
  </tbody>
</table>

</table>
<p>Reference SW info</p>
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>SW</th>
      <th>Nightly commit</th>
      <th>Main commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Pytorch</td>
      <td>0de2555</td>
      <td></td>
    </tr>
    <tr>
      <td>Torchbench</td>
      <td>/</td>
      <td>ffbbebb9</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>475b6ae</td>
      <td></td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>142d029</td>
      <td></td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>8636bf3</td>
      <td></td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>eb9bf61</td>
      <td></td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>0200b11</td>
      <td>/</td>
    </tr>
  </tbody>
</table>

<p>Repro</p>
<a href=https://github.com/chuanqi129/inductor-tools/blob/yudong/aws_auto/scripts/modelbench/inductor_single_run.sh>
  inductor_single_run.sh </a>
<code>
bash inductor_single_run.sh multiple/single inference performance torchbench/huggingface/timm_models model float32 first static default 0
</code>

<p><a href=https://github.com/pytorch/pytorch/commit/1a361e4e9ff1a8dadd72c7301441cfb83d687f24#diff-cf6ca00beddc32a2a6a2933fb9913b6a2b925ffc3b745488967210e4343134ac>
  Suspected guilty commit </a></p>

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov