Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.4.0.dev20240406+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240405+cu121

Base PyTorch commit: d8717c2d68bc0a8bda7e8ad823a0d6c2d8aae934

Affected PyTorch version: 2.4.0.dev20240406+cu121

Affected PyTorch commit: 22b99871442fc3c78ec173894c017d6d98996501

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: -11.24176%
- test_eval[pytorch_stargan-cpu-eager]: +8.02621%
- test_eval[pytorch_stargan-cpu-jit]: -7.79200%
- test_train[timm_nfnet-cpu-jit]: -10.73200%


cc @xuzhao9

Result json: 
```
{
  "start": "d8717c2d68bc0a8bda7e8ad823a0d6c2d8aae934",
  "end": "22b99871442fc3c78ec173894c017d6d98996501",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "7b575f08148",
      "commit1_time": "2024-04-05 03:40:16 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07442933431593701,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2439336642390117
      },
      "commit2": "18c9d460682",
      "commit2_time": "2024-04-05 03:53:22 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08567604330601171,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2901924855890684
      }
    },
    {
      "commit1": "5b0ce8f3342",
      "commit1_time": "2024-04-05 13:02:38 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08376329938182607,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2641073085833341
      },
      "commit2": "8c7d8f0ff20",
      "commit2_time": "2024-04-05 13:34:41 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.294476368627511
      }
    },
    {
      "commit1": "aa063054ce9",
      "commit1_time": "2024-04-04 23:23:50 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0842408264055848,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2609420069376938
      },
      "commit2": "064a650b635",
      "commit2_time": "2024-04-04 23:23:50 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07846144441282377,
        "test_eval[pytorch_stargan-cpu-jit]": 0.29253241840051486
      }
    },
    {
      "commit1": "064a650b635",
      "commit1_time": "2024-04-04 23:23:50 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07846144441282377,
        "test_eval[pytorch_stargan-cpu-jit]": 0.29253241840051486
      },
      "commit2": "0c8a165b43c",
      "commit2_time": "2024-04-05 00:25:37 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07182230791077018,
        "test_eval[pytorch_stargan-cpu-jit]": 0.29366842728340997
      }
    },
    {
      "commit1": "c575e378bae",
      "commit1_time": "2024-04-05 03:29:11 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2931857754010707
      },
      "commit2": "7ffad9ab04b",
      "commit2_time": "2024-04-05 03:29:11 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2690223386511207
      }
    },
    {
      "commit1": "7ffad9ab04b",
      "commit1_time": "2024-04-05 03:29:11 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2690223386511207
      },
      "commit2": "86c5cc65593",
      "commit2_time": "2024-04-05 03:29:40 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29609531225869434
      }
    },
    {
      "commit1": "86c5cc65593",
      "commit1_time": "2024-04-05 03:29:40 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29609531225869434
      },
      "commit2": "7b575f08148",
      "commit2_time": "2024-04-05 03:40:16 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07442933431593701,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2439336642390117
      }
    },
    {
      "commit1": "cfd06bd60cc",
      "commit1_time": "2024-04-05 09:34:30 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29313635841244834
      },
      "commit2": "5b0ce8f3342",
      "commit2_time": "2024-04-05 13:02:38 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08376329938182607,
        "test_eval[pytorch_stargan-cpu-jit]": 0.2641073085833341
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/8582669493


Noise, closed