Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240214+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240213+cu121

Base PyTorch commit: 2502a011108b0592dd6dd34e5a61b2d52b95160e

Affected PyTorch version: 2.3.0.dev20240214+cu121

Affected PyTorch commit: 16369816a2e1c298fe5d15a439e922d10eb1894b

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: -10.04726%
- test_eval[shufflenet_v2_x1_0-cpu-eager]: -7.54493%
- test_train[timm_nfnet-cpu-jit]: -7.38260%
- test_train[timm_regnet-cpu-eager]: +9.09434%


cc @xuzhao9

Result json: 
```
{
  "start": "2502a011108b0592dd6dd34e5a61b2d52b95160e",
  "end": "16369816a2e1c298fe5d15a439e922d10eb1894b",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "2502a011108",
      "commit1_time": "2024-02-13 04:16:34 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08886424242518842,
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.1480509854387492,
        "test_train[timm_nfnet-cpu-jit]": 11.548286182293669,
        "test_train[timm_regnet-cpu-eager]": 0.7871630265377462
      },
      "commit2": "2bda6b4cb80",
      "commit2_time": "2024-02-13 04:30:45 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07714265566319227
      }
    },
    {
      "commit1": "2bda6b4cb80",
      "commit1_time": "2024-02-13 04:30:45 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07714265566319227
      },
      "commit2": "47a2e6b6b87",
      "commit2_time": "2024-02-13 05:10:19 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0709608663804829
      }
    },
    {
      "commit1": "47a2e6b6b87",
      "commit1_time": "2024-02-13 05:10:19 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0709608663804829
      },
      "commit2": "a4c476a081a",
      "commit2_time": "2024-02-13 05:18:03 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07939412356354296
      }
    },
    {
      "commit1": "ca777fbbb73",
      "commit1_time": "2024-02-13 23:15:24 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07403988395817578
      },
      "commit2": "cf117e37d55",
      "commit2_time": "2024-02-13 23:28:47 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08603780907578766
      }
    },
    {
      "commit1": "be35fc9ea7a",
      "commit1_time": "2024-02-13 23:47:52 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09047098411247134
      },
      "commit2": "b51e0246b7f",
      "commit2_time": "2024-02-13 23:50:40 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07851886847056448
      }
    },
    {
      "commit1": "b2e779868f3",
      "commit1_time": "2024-02-14 00:25:42 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07895802771672607
      },
      "commit2": "cb0886ecf29",
      "commit2_time": "2024-02-14 00:29:10 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09219151632860303
      }
    },
    {
      "commit1": "36292871512",
      "commit1_time": "2024-02-13 09:02:53 +0000",
      "commit1_digest": {
        "test_train[timm_nfnet-cpu-jit]": 11.445479934709146
      },
      "commit2": "f4b5f710e8f",
      "commit2_time": "2024-02-13 14:59:59 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.080885119875893,
        "test_train[timm_nfnet-cpu-jit]": 12.406846110615879
      }
    },
    {
      "commit1": "2536c5186ea",
      "commit1_time": "2024-02-14 01:05:58 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08670096718706191
      },
      "commit2": "16369816a2e",
      "commit2_time": "2024-02-14 01:10:40 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06978382444940508,
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.1671025016810745,
        "test_train[timm_nfnet-cpu-jit]": 12.442483476130292,
        "test_train[timm_regnet-cpu-eager]": 0.8316005552653223
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7904908157