Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.4.0.dev20240604+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240603+cu121

Base PyTorch commit: 7e97b33fbbb24fa5876500be764c97f51b74ac0e

Affected PyTorch version: 2.4.0.dev20240604+cu121

Affected PyTorch commit: 1208347d0912d1236ae43257e2914767e35c3b36

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: +17.26324%
- test_eval[pytorch_stargan-cpu-eager]: -16.18091%
- test_eval[pytorch_stargan-cpu-jit]: +12.50543%
- test_train[timm_nfnet-cpu-jit]: -11.68789%
- test_train[tts_angular-cpu-eager]: +13.63316%


cc @xuzhao9

Result json: 
```
{
  "start": "7e97b33fbbb24fa5876500be764c97f51b74ac0e",
  "end": "1208347d0912d1236ae43257e2914767e35c3b36",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "f03f8bc901a",
      "commit1_time": "2024-06-03 14:44:03 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06863791514188052
      },
      "commit2": "d05cddfe232",
      "commit2_time": "2024-06-03 15:00:21 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08177166962996125
      }
    },
    {
      "commit1": "c35b65715ca",
      "commit1_time": "2024-06-03 18:07:57 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07301771782804281
      },
      "commit2": "3f45fa63f28",
      "commit2_time": "2024-06-03 18:10:46 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07908609602600336
      }
    },
    {
      "commit1": "db9d457a3f7",
      "commit1_time": "2024-06-03 19:33:06 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12413908652961254
      },
      "commit2": "941316f821b",
      "commit2_time": "2024-06-03 20:24:07 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08142698239535093
      }
    },
    {
      "commit1": "a4064da8cac",
      "commit1_time": "2024-06-03 20:36:14 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08935926924459636
      },
      "commit2": "6d4ec9b2ecb",
      "commit2_time": "2024-06-03 21:21:55 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12676669429056348
      }
    },
    {
      "commit1": "3437177e2bc",
      "commit1_time": "2024-06-03 18:06:31 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0799752180930227
      },
      "commit2": "c35b65715ca",
      "commit2_time": "2024-06-03 18:07:57 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07301771782804281
      }
    },
    {
      "commit1": "3f45fa63f28",
      "commit1_time": "2024-06-03 18:10:46 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07908609602600336
      },
      "commit2": "2fc907971a0",
      "commit2_time": "2024-06-03 18:13:27 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12147313568275422
      }
    },
    {
      "commit1": "941316f821b",
      "commit1_time": "2024-06-03 20:24:07 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08142698239535093
      },
      "commit2": "ef9451ac8dc",
      "commit2_time": "2024-06-03 20:35:22 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07255737998057157
      }
    },
    {
      "commit1": "ef9451ac8dc",
      "commit1_time": "2024-06-03 20:35:22 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07255737998057157
      },
      "commit2": "a4064da8cac",
      "commit2_time": "2024-06-03 20:36:14 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08935926924459636
      }
    },
    {
      "commit1": "6d4ec9b2ecb",
      "commit1_time": "2024-06-03 21:21:55 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.12676669429056348
      },
      "commit2": "c6dc6246902",
      "commit2_time": "2024-06-03 21:38:50 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07777450957801194
      }
    },
    {
      "commit1": "41033a42742",
      "commit1_time": "2024-06-04 00:59:58 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-eager]": 0.26139165319036695
      },
      "commit2": "1208347d091",
      "commit2_time": "2024-06-04 01:14:34 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07538383868522942,
        "test_eval[pytorch_stargan-cpu-eager]": 0.22489399111364036,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2592265143059194,
        "test_train[timm_nfnet-cpu-jit]": 11.999537387024612,
        "test_train[tts_angular-cpu-eager]": 0.22390947612002493
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/9371386604


Closed by noise.