[inductor][cpu] performance regression
Labels: triaged, oncall: pt2, module: cpu inductor

<p>new_perf_regression compare with 2023-08-30 nightly release</p>

opacus_cifar10 perf regression occurs in mutliple threads test mode.
<table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>compilation_latency_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>compilation_latency_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
      <th>Inductor Ratio(old/new)</th>
      <th>Compilation_latency_Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>opacus_cifar10</td>
      <td>64</td>
      <td>0.613604</td>
      <td>0.011064978</td>
      <td>0.006789514760712</td>
      <td>59.939124</td>
      <td>64</td>
      <td>0.739715</td>
      <td>0.009038787</td>
      <td>0.006686126325705</td>
      <td>27.589113</td>
      <td>0.83</td>
      <td>0.98</td>
      <td>0.82</td>
      <td>0.46</td>
    </tr>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
    <tr>
      <td>basic_gnn_gcn</td>
      <td>1</td>
      <td>1.453264</td>
      <td>0.07012852100000001</td>
      <td>0.10191525494254401</td>
      <td>16.308482</td>
      <td>1</td>
      <td>4.18351</td>
      <td>0.024277587</td>
      <td>0.10156552799037</td>
      <td>12.917972</td>
      <td>0.35</td>
      <td>1.0</td>
      <td>0.35</td>
      <td>0.79</td>
    </tr>
    <tr>
      <td>basic_gnn_gin</td>
      <td>1</td>
      <td>1.102929</td>
      <td>0.053309631</td>
      <td>0.058796738009199004</td>
      <td>16.217177</td>
      <td>1</td>
      <td>3.911906</td>
      <td>0.016065617</td>
      <td>0.06284718353600201</td>
      <td>12.840029</td>
      <td>0.28</td>
      <td>1.07</td>
      <td>0.3</td>
      <td>0.79</td>
    </tr>
    <tr>
      <td>basic_gnn_sage</td>
      <td>1</td>
      <td>1.069263</td>
      <td>0.055456228</td>
      <td>0.059297292719964007</td>
      <td>14.707836</td>
      <td>1</td>
      <td>3.680068</td>
      <td>0.016367606</td>
      <td>0.060233903077207995</td>
      <td>11.36611</td>
      <td>0.29</td>
      <td>1.02</td>
      <td>0.3</td>
      <td>0.77</td>
    </tr>
    <tr>
      <td>opacus_cifar10</td>
      <td>1</td>
      <td>0.389078</td>
      <td>0.013655245</td>
      <td>0.00531295541411</td>
      <td>58.53751</td>
      <td>1</td>
      <td>0.737097</td>
      <td>0.006703775</td>
      <td>0.004941332441175</td>
      <td>26.168891</td>
      <td>0.53</td>
      <td>0.93</td>
      <td>0.49</td>
      <td>0.45</td>
    </tr>
    <tr>
      <td>cait_m36_384</td>
      <td>1</td>
      <td>0.845182</td>
      <td>4.210067068</td>
      <td>3.5582729046663757</td>
      <td>96.197948</td>
      <td>1</td>
      <td>0.97062</td>
      <td>3.666166209</td>
      <td>3.55845424577958</td>
      <td>93.097843</td>
      <td>0.87</td>
      <td>1.0</td>
      <td>0.87</td>
      <td>0.97</td>
    </tr>
    <tr>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
      <td>*</td>
    </tr>
  </tbody>
</table><p>SW info</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>SW</th>
      <th>Nightly commit</th>
      <th>Main commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Pytorch</td>
      <td>ea7e229</td>
      <td></td>
    </tr>
    <tr>
      <td>Torchbench</td>
      <td>/</td>
      <td>770d5cf7</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>dc83b38</td>
      <td></td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>c11d758</td>
      <td></td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>58366ab</td>
      <td></td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>1d231d1</td>
      <td></td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>f228c8b</td>
      <td>/</td>
    </tr>
  </tbody>
</table>

Repro:
bash [inductor_single_test.sh](https://github.com/chuanqi129/inductor-tools/blob/yudong/aws_auto/scripts/modelbench/inductor_single_run.sh) single inference performance **suite model** float32 first static default 0

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305