Repository: pytorch
Issue: [inductor][cpu]for 4 models, TORCHINDUCTOR_FREEZING=0 has better performance than TORCHINDUCTOR_FREEZING=1
Labels: triaged,oncall: cpu inductor

### 🐛 Describe the bug

This issue is separated from https://github.com/pytorch/pytorch/issues/104952, verified on 2023-07-06 nightly: https://github.com/pytorch/pytorch/commit/13763f58ad86fadf49ef7960d1836318e6480d36
</table><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>name</th>
      <th>batch_size_new</th>
      <th>speed_up_new</th>
      <th>inductor_new</th>
      <th>eager_new</th>
      <th>batch_size_old</th>
      <th>speed_up_old</th>
      <th>inductor_old</th>
      <th>eager_old</th>
      <th>Ratio Speedup(New/old)</th>
      <th>Eager Ratio(old/new)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Background_Matting</td>
      <td>1</td>
      <td>0.782193</td>
      <td>0.350721461</td>
      <td>0.274331872</td>
      <td>1</td>
      <td>1.00737</td>
      <td>0.279395653</td>
      <td>0.281454799</td>
      <td>0.78</td>
      <td>1.03</td>
    </tr>
    <tr>
      <td>functorch_dp_cifar10</td>
      <td>64</td>
      <td>0.622732</td>
      <td>0.009190167</td>
      <td>0.005723011</td>
      <td>64</td>
      <td>1.008348</td>
      <td>0.005596095</td>
      <td>0.005642811</td>
      <td>0.62</td>
      <td>0.99</td>
    </tr>
    <tr>
      <td>resmlp_12_224</td>
      <td>128</td>
      <td>0.749039</td>
      <td>0.415152565</td>
      <td>0.310965462</td>
      <td>128</td>
      <td>1.237528</td>
      <td>0.259625741</td>
      <td>0.321294124</td>
      <td>0.61</td>
      <td>1.03</td>
    </tr>
    <tr>
      <td>tnt_s_patch16_224</td>
      <td>1</td>
      <td>1.173545</td>
      <td>0.094649226</td>
      <td>0.111075126</td>
      <td>1</td>
      <td>1.367958</td>
      <td>0.081854891</td>
      <td>0.111974053</td>
      <td>0.86</td>
      <td>1.01</td>
    </tr>
</tbody>

</table>

The new is bad performance collected with TORCHINDUCTOR_FREEZING=1
The old is good performance collected with TORCHINDUCTOR_FREEZING=0
These are fP32 static default wrapper test, the first three models test with multi-thread, the last one tnt_s_patch16_224 is single thread.

### Versions

</table><p>SW info</p><table border="1" class="dataframe table">
  <thead>
    <tr style="text-align: right;">
      <th>SW</th>
      <th>Nightly commit</th>
      <th>Main commit</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Pytorch</td>
      <td>13763f5</td>
      <td></td>
    </tr>
    <tr>
      <td>Torchbench</td>
      <td>/</td>
      <td>ff42d907</td>
    </tr>
    <tr>
      <td>torchaudio</td>
      <td>b2d9c3e</td>
      <td></td>
    </tr>
    <tr>
      <td>torchtext</td>
      <td>b0ebddc</td>
      <td></td>
    </tr>
    <tr>
      <td>torchvision</td>
      <td>315f315</td>
      <td></td>
    </tr>
    <tr>
      <td>torchdata</td>
      <td>0790338</td>
      <td></td>
    </tr>
    <tr>
      <td>dynamo_benchmarks</td>
      <td>nightly</td>
      <td>/</td>
    </tr>
  </tbody>

</table>

Repro:
[inductor_single_run.sh](https://github.com/chuanqi129/inductor-tools/blob/main/scripts/modelbench/inductor_single_run.sh)
bad performance: bash inductor_single_run.sh multiple/single inference performance **suite model** float32 first static default 0 on
good performance: bash inductor_single_run.sh multiple/single inference performance **suite model** float32 first static default 0 off
cc @WeizhuoZhang-intel @chuanqi129