Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.4.0.dev20240604+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240603+cu121

Base PyTorch commit: 7e97b33fbbb24fa5876500be764c97f51b74ac0e

Affected PyTorch version: 2.4.0.dev20240604+cu121

Affected PyTorch commit: 1208347d0912d1236ae43257e2914767e35c3b36

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: +7.18279%
- test_train[timm_nfnet-cpu-jit]: -10.89356%
- test_train[timm_resnest-cpu-eager]: +8.12692%
- test_train[tts_angular-cpu-eager]: +10.09725%


cc @xuzhao9

Result json: 
```
{
  "start": "7e97b33fbbb24fa5876500be764c97f51b74ac0e",
  "end": "1208347d0912d1236ae43257e2914767e35c3b36",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "7c3740d3889",
      "commit1_time": "2024-06-03 17:46:12 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12412784609477967
      },
      "commit2": "d8d0bf264a7",
      "commit2_time": "2024-06-03 17:53:48 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.14196302790660412
      }
    },
    {
      "commit1": "c35b65715ca",
      "commit1_time": "2024-06-03 18:07:57 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06910683824680745
      },
      "commit2": "3f45fa63f28",
      "commit2_time": "2024-06-03 18:10:46 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08584793200716376
      }
    },
    {
      "commit1": "db9d457a3f7",
      "commit1_time": "2024-06-03 19:33:06 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09398004622198641
      },
      "commit2": "941316f821b",
      "commit2_time": "2024-06-03 20:24:07 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08453186261467635
      }
    },
    {
      "commit1": "3efac92888f",
      "commit1_time": "2024-06-03 21:38:57 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08600447282660753
      },
      "commit2": "c27882ffa8c",
      "commit2_time": "2024-06-03 21:38:57 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07779938990715891
      }
    },
    {
      "commit1": "0f1f0d3015f",
      "commit1_time": "2024-06-03 22:14:16 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06999592003412544
      },
      "commit2": "0e7bd7feddc",
      "commit2_time": "2024-06-03 22:30:11 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0890772013925016
      }
    },
    {
      "commit1": "d8d0bf264a7",
      "commit1_time": "2024-06-03 17:53:48 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.14196302790660412
      },
      "commit2": "3437177e2bc",
      "commit2_time": "2024-06-03 18:06:31 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08192613259889185
      }
    },
    {
      "commit1": "3437177e2bc",
      "commit1_time": "2024-06-03 18:06:31 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08192613259889185
      },
      "commit2": "c35b65715ca",
      "commit2_time": "2024-06-03 18:07:57 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06910683824680745
      }
    },
    {
      "commit1": "2fc907971a0",
      "commit1_time": "2024-06-03 18:13:27 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08719911614898593
      },
      "commit2": "db9d457a3f7",
      "commit2_time": "2024-06-03 19:33:06 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09398004622198641
      }
    },
    {
      "commit1": "6faa3d5f18b",
      "commit1_time": "2024-06-03 22:14:10 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08081005206331611
      },
      "commit2": "f6ca822366e",
      "commit2_time": "2024-06-03 22:14:16 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07174320640042424
      }
    },
    {
      "commit1": "0e7bd7feddc",
      "commit1_time": "2024-06-03 22:30:11 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0890772013925016
      },
      "commit2": "ac568fc0077",
      "commit2_time": "2024-06-03 22:42:05 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12060661960858851
      }
    },
    {
      "commit1": "ac568fc0077",
      "commit1_time": "2024-06-03 22:42:05 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12060661960858851
      },
      "commit2": "b42cfcabc4e",
      "commit2_time": "2024-06-03 23:41:54 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08307855795137584
      }
    },
    {
      "commit1": "b42cfcabc4e",
      "commit1_time": "2024-06-03 23:41:54 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08307855795137584
      },
      "commit2": "dbf39a6e632",
      "commit2_time": "2024-06-04 00:39:01 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07511241473257542
      }
    },
    {
      "commit1": "05fa05cbae5",
      "commit1_time": "2024-06-04 00:49:04 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07060186236631125
      },
      "commit2": "41033a42742",
      "commit2_time": "2024-06-04 00:59:58 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0790498574031517
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/9388464567