Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240309+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240308+cu121

Base PyTorch commit: 953c6c37cbfcc894963120666dbf066977931885

Affected PyTorch version: 2.3.0.dev20240309+cu121

Affected PyTorch commit: d482614fec5fb9bccb49bf4ee4ab561e872c0f50

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: +42.12506%
- test_eval[pytorch_stargan-cpu-eager]: +8.54433%
- test_eval[pytorch_stargan-cpu-jit]: +8.93644%
- test_eval[resnext50_32x4d-cpu-jit]: +12.90490%
- test_train[timm_regnet-cpu-eager]: -12.50678%


cc @xuzhao9

Result json: 
```
{
  "start": "953c6c37cbfcc894963120666dbf066977931885",
  "end": "d482614fec5fb9bccb49bf4ee4ab561e872c0f50",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "3d089de8518",
      "commit1_time": "2024-03-07 22:39:57 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2529209022410214
      },
      "commit2": "e0c534fe02c",
      "commit2_time": "2024-03-07 23:06:29 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2992546748369932
      }
    },
    {
      "commit1": "13366a101aa",
      "commit1_time": "2024-03-08 03:29:47 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08710294095799327,
        "test_eval[pytorch_stargan-cpu-eager]": 0.26536677218973637,
        "test_eval[pytorch_stargan-cpu-jit]": 0.249555195402354,
        "test_train[timm_regnet-cpu-eager]": 0.8135752575471997
      },
      "commit2": "96ed37ac133",
      "commit2_time": "2024-03-08 05:13:13 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.27098240768536924
      }
    },
    {
      "commit1": "9b03a06288b",
      "commit1_time": "2024-03-08 18:59:37 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09417016729712487
      },
      "commit2": "d55d8038129",
      "commit2_time": "2024-03-08 19:08:33 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08455208353698254
      }
    },
    {
      "commit1": "23ac0cd5612",
      "commit1_time": "2024-03-08 01:59:01 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29740579361096026
      },
      "commit2": "909d73d8cb6",
      "commit2_time": "2024-03-08 02:14:12 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.272232997789979
      }
    },
    {
      "commit1": "909d73d8cb6",
      "commit1_time": "2024-03-08 02:14:12 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.272232997789979
      },
      "commit2": "72dd9b24303",
      "commit2_time": "2024-03-08 02:30:49 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2520510943606496
      }
    },
    {
      "commit1": "96ed37ac133",
      "commit1_time": "2024-03-08 05:13:13 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.27098240768536924
      },
      "commit2": "e628f2cc669",
      "commit2_time": "2024-03-08 07:19:51 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29639562824741006
      }
    },
    {
      "commit1": "d55d8038129",
      "commit1_time": "2024-03-08 19:08:33 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08455208353698254
      },
      "commit2": "0f3f4f55342",
      "commit2_time": "2024-03-08 19:08:50 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07154126232489944
      }
    },
    {
      "commit1": "6791b0c09ea",
      "commit1_time": "2024-03-09 01:08:37 +0000",
      "commit1_digest": {
        "test_train[timm_regnet-cpu-eager]": 0.8462646229192614
      },
      "commit2": "d482614fec5",
      "commit2_time": "2024-03-09 01:14:18 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07603516671806573,
        "test_eval[pytorch_stargan-cpu-eager]": 0.2681995242834091,
        "test_eval[pytorch_stargan-cpu-jit]": 0.29938788460567595,
        "test_eval[resnext50_32x4d-cpu-jit]": 0.04306706661979357,
        "test_train[timm_regnet-cpu-eager]": 0.7360542310401798
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/8216052341