zhiminy/EvalEng huggingface dataset:
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20231227+cu118'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20231226+cu118

Base PyTorch commit: f6dfbffb3bb46ada6fe66b5da4f989f9d4d69b3c

Affected PyTorch version: 2.3.0.dev20231227+cu118

Affected PyTorch commit: 9c3ae37fc453505f5e437d1edadefdb278c2c39c

Affected Tests:
- test_eval[shufflenet_v2_x1_0-cpu-eager]: +9.68443%
- test_train[timm_resnest-cpu-eager]: +8.96711%


cc @xuzhao9

Result json: 
```
{
  "start": "f6dfbffb3bb46ada6fe66b5da4f989f9d4d69b3c",
  "end": "9c3ae37fc453505f5e437d1edadefdb278c2c39c",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "bb2a1e99413",
      "commit1_time": "2023-12-26 06:05:15 +0000",
      "commit1_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.13937065496575088
      },
      "commit2": "3c9076f070f",
      "commit2_time": "2023-12-26 06:37:55 +0000",
      "commit2_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.15668932371772826
      }
    },
    {
      "commit1": "427ecc61c02",
      "commit1_time": "2023-12-26 19:36:34 +0000",
      "commit1_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.16316868651192634
      },
      "commit2": "51eef859eb7",
      "commit2_time": "2023-12-26 19:43:43 +0000",
      "commit2_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.13596062674187123
      }
    },
    {
      "commit1": "7571511af9b",
      "commit1_time": "2023-12-26 20:25:57 +0000",
      "commit1_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.13102208042982966
      },
      "commit2": "e5bcfe205e4",
      "commit2_time": "2023-12-26 21:41:47 +0000",
      "commit2_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.14086037557572126
      }
    },
    {
      "commit1": "e5bcfe205e4",
      "commit1_time": "2023-12-26 21:41:47 +0000",
      "commit1_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.14086037557572126
      },
      "commit2": "e86636266fc",
      "commit2_time": "2023-12-26 21:52:28 +0000",
      "commit2_digest": {
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.15771865150891246
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7340405764


Noise, closed.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'ef28de986bb0540a4cd8f569f7b0b973a2ae26f6'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 50f7a03676c9c35f4d16168113febb42dabc4c51

Affected PyTorch commit: ef28de986bb0540a4cd8f569f7b0b973a2ae26f6

Affected Tests:
- test_eval[detectron2_fasterrcnn_r_50_dc5-cuda-eager]_latency: +13.33525%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/7349258944

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/7423600510

```
{
  "target_repo": "pytorch",
  "start": "9c3ae37fc453505f5e437d1edadefdb278c2c39c",
  "end": "6375eb15efa1273514695e468f68963feedc0067",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'a87500edd636dc80d1814094102e845f0c96218e'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: b41958165d875c02e0cffdba8aa01e2623a4884a

Affected PyTorch commit: a87500edd636dc80d1814094102e845f0c96218e

Affected Tests:
- test_train[detectron2_fasterrcnn_r_101_c4-cuda-eager]_latency: -10.36966%
- test_train[lennard_jones-cuda-eager]_latency: -12.85057%
- test_eval[llama-cuda-eager]_latency: -10.31637%
- test_eval[maml-cuda-eager]_latency: -10.25241%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: -13.10763%
- test_eval[shufflenet_v2_x1_0-cuda-eager]_latency: -10.82862%
- test_eval[yolov3-cuda-eager]_latency: -11.49696%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/7387867818

cc @xuzhao9


https://github.com/pytorch/benchmark/actions/runs/7429049479

```
{
  "target_repo": "pytorch",
  "start": "4bfaa6bc250f5ff5702703ea237f578a15bbe3b6",
  "end": "1ed8efa9b35328df6f0e8dbfd27578dcfde58e60",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Fix model_flops feature on PT2
Root cause: Algorithmic Error

Now the flops counter is broken, need to fix it and add a test to prevent future issues.

Run command:

```
buck2 run @mode/opt //pytorch/benchmark:run -- resnet50 -d cuda -t train --metrics model_flops --torchdynamo inductor
```

or

```
python run.py resnet50 -d cuda -t train --metrics model_flops  --torchdynamo inductor
```

We should also add a test internally to guard on this use case.
Repository: pytorch/benchmark
Issue: e10cfdd8953 regresses hf_Longformer-cuda-eager on pytorch 2024-01-16
Root cause: Algorithmic Error

e10cfdd8953 regresses  hf_Longformer-cuda-eager on pytorch 2024-01-16:

```
{
  "target_repo": "pytorch",
  "start": "8bcdde5058658cc193c94a7f1eb16660553dc35a",
  "start_version": "2.3.0.dev20240110+cu118",
  "end": "28bb31e4a588f3fa198e927439fe09c871201aab",
  "end_version": "2.3.0.dev20240116+cu118",
  "result": [
    {
      "commit1": "7e6a04e542f",
      "commit1_time": "2024-01-10 22:25:40 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "7e6a04e542ff4b1cd87dc2dba9095274604b26af",
          "pytorch_version": "2.3.0a0+git7e6a04e",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "7e6a04e542ff4b1cd87dc2dba9095274604b26af"
        },
        "metrics": {
          "test_eval[hf_Longformer-cuda-eager]_latency": 142.44164,
          "test_eval[hf_Longformer-cuda-eager]_cmem": 0.7841796875,
          "test_eval[hf_Longformer-cuda-eager]_gmem": 5.22845458984375
        }
      },
      "commit2": "e10cfdd8953",
      "commit2_time": "2024-01-10 23:16:42 +0000",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "e10cfdd89531366c5dd0b8dfa3313b2d4be06457",
          "pytorch_version": "2.3.0a0+gite10cfdd",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "e10cfdd89531366c5dd0b8dfa3313b2d4be06457"
        },
        "metrics": {
          "test_eval[hf_Longformer-cuda-eager]_latency": 183.184652,
          "test_eval[hf_Longformer-cuda-eager]_cmem": 0.7841796875,
          "test_eval[hf_Longformer-cuda-eager]_gmem": 5.23040771484375
        }
      }
    }
  ]
}
```

Before latency: 142.44 ms
After latency: 183.18 ms

cc @malfet Can you help take a look at this issue?

The bisector points to the root cause commit https://github.com/pytorch/pytorch/commit/e10cfdd89531366c5dd0b8dfa3313b2d4be06457

Reported to upstream: https://github.com/pytorch/pytorch/issues/118548

Case closed, further discussion at https://github.com/pytorch/pytorch/issues/118774
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240130+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240127+cu118

Base PyTorch commit: fe10b1800f5df432532fc91e0d553a521ec53479

Affected PyTorch version: 2.3.0.dev20240130+cu121

Affected PyTorch commit: dcc077eea2f95a9d5caa65eeca4d8c73e207f61d

Affected Tests:
- test_train[Super_SloMo-cuda-eager]: -7.47195%
- test_eval[Super_SloMo-cuda-eager]: +10.42605%
- test_train[Super_SloMo-cuda-jit]: -7.53183%
- test_eval[Super_SloMo-cuda-jit]: +10.53007%
- test_train[dcgan-cuda-eager]: -9.97524%
- test_eval[dcgan-cuda-eager]: -7.26311%
- test_train[dcgan-cuda-jit]: -10.01823%
- test_eval[hf_T5-cpu-eager]: +20.98431%
- test_eval[mnasnet1_0-cuda-jit]: -9.02097%
- test_eval[pyhpc_equation_of_state-cpu-jit]: +7.70336%
- test_train[timm_regnet-cuda-eager]: +13.83100%
- test_eval[timm_regnet-cuda-eager]: -63.66011%
- test_train[timm_regnet-cuda-jit]: +14.27366%
- test_eval[timm_regnet-cuda-jit]: -78.27843%
- test_train[yolov3-cuda-eager]: +12.31370%


cc @xuzhao9

Result json: 
```
{
  "start": "fe10b1800f5df432532fc91e0d553a521ec53479",
  "end": "dcc077eea2f95a9d5caa65eeca4d8c73e207f61d",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "0ed24cb1afb",
      "commit1_time": "2024-01-30 02:07:34 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07681977916508913
      },
      "commit2": "67d8db9252f",
      "commit2_time": "2024-01-30 02:22:42 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06479158182628453
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7715560263


False positive, closed.
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240204+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240203+cu121

Base PyTorch commit: ce40ee8ecdfdecd48ffcec4fced35f16a77f268d

Affected PyTorch version: 2.3.0.dev20240204+cu121

Affected PyTorch commit: 8f82a44a5beed0f205a7f273b9f062da43d65679

Affected Tests:
- test_train[tts_angular-cpu-eager]: -10.11409%


cc @xuzhao9

Result json: 
```
{
  "start": "ce40ee8ecdfdecd48ffcec4fced35f16a77f268d",
  "end": "8f82a44a5beed0f205a7f273b9f062da43d65679",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "ce40ee8ecdf",
      "commit1_time": "2024-02-03 03:57:29 +0000",
      "commit1_digest": {
        "test_train[tts_angular-cpu-eager]": 0.2241453909315169
      },
      "commit2": "9a8e3b07d75",
      "commit2_time": "2024-02-03 05:55:09 +0000",
      "commit2_digest": {
        "test_train[tts_angular-cpu-eager]": 0.1910615146625787
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7775598350


Noise, closed.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '2.3.0.dev20240201+cu121'
Root cause: Others

TorchBench CI has detected a performance signal or runtime regression, and bisected its result.

Control PyTorch commit: e3cde685340d2c5c752428b37449ba75f59488af
Control PyTorch version: 2.3.0.dev20240131+cu121

Treatment PyTorch commit: 4cba1dd0c3e7f625856566ae00d89d8ee2e8b990
Treatment PyTorch version: 2.3.0.dev20240201+cu121

Bisection result:

```
{
  "target_repo": "pytorch",
  "start": "e3cde685340d2c5c752428b37449ba75f59488af",
  "start_version": "2.3.0.dev20240131+cu121",
  "end": "4cba1dd0c3e7f625856566ae00d89d8ee2e8b990",
  "end_version": "2.3.0.dev20240201+cu121",
  "result": [
    {
      "commit1": "a5a0fdcae92",
      "commit1_time": "2024-01-31 18:18:17 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "a5a0fdcae928cb192c94b865f70e69c4f663bf0b",
          "pytorch_version": "2.3.0a0+gita5a0fdc",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "a5a0fdcae928cb192c94b865f70e69c4f663bf0b"
        },
        "metrics": {
          "test_eval[hf_Longformer-cuda-eager]_latency": 182.798165,
          "test_eval[hf_Longformer-cuda-eager]_cmem": 0.7626953125,
          "test_eval[hf_Longformer-cuda-eager]_gmem": 5.22613525390625
        }
      },
      "commit2": "81b55f58ce4",
      "commit2_time": "2024-01-31 18:34:16 +0000",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "81b55f58ce492f14451a93286791c0ceb2f32892",
          "pytorch_version": "2.3.0a0+git81b55f5",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "81b55f58ce492f14451a93286791c0ceb2f32892"
        },
        "metrics": {
          "test_eval[hf_Longformer-cuda-eager]_latency": 142.382033,
          "test_eval[hf_Longformer-cuda-eager]_cmem": 0.76171875,
          "test_eval[hf_Longformer-cuda-eager]_gmem": 5.22613525390625
        }
      }
    }
  ]
}
```

cc @xuzhao9


Root cause commit: https://github.com/pytorch/pytorch/commit/81b55f58ce4
This is a known speedup, closed.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '2.3.0.dev20240201+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression, and bisected its result.

Control PyTorch commit: e3cde685340d2c5c752428b37449ba75f59488af
Control PyTorch version: 2.3.0.dev20240131+cu121

Treatment PyTorch commit: 4cba1dd0c3e7f625856566ae00d89d8ee2e8b990
Treatment PyTorch version: 2.3.0.dev20240201+cu121

Bisection result:

```
[
    {
        "commit1": "a5a0fdcae92",
        "commit1_time": "2024-01-31 18:18:17 +0000",
        "commit1_digest": {
            "name": "torch-nightly",
            "environ": {
                "pytorch_git_version": "a5a0fdcae928cb192c94b865f70e69c4f663bf0b",
                "pytorch_version": "2.3.0a0+gita5a0fdc",
                "device": "NVIDIA A100-SXM4-40GB",
                "git_commit_hash": "a5a0fdcae928cb192c94b865f70e69c4f663bf0b"
            },
            "metrics": {
                "test_eval[hf_Longformer-cuda-eager]_latency": 182.607644,
                "test_eval[hf_Longformer-cuda-eager]_cmem": 0.76171875,
                "test_eval[hf_Longformer-cuda-eager]_gmem": 5.22613525390625
            }
        },
        "commit2": "81b55f58ce4",
        "commit2_time": "2024-01-31 18:34:16 +0000",
        "commit2_digest": {
            "name": "torch-nightly",
            "environ": {
                "pytorch_git_version": "81b55f58ce492f14451a93286791c0ceb2f32892",
                "pytorch_version": "2.3.0a0+git81b55f5",
                "device": "NVIDIA A100-SXM4-40GB",
                "git_commit_hash": "81b55f58ce492f14451a93286791c0ceb2f32892"
            },
            "metrics": {
                "test_eval[hf_Longformer-cuda-eager]_latency": 142.317729,
                "test_eval[hf_Longformer-cuda-eager]_cmem": 0.76171875,
                "test_eval[hf_Longformer-cuda-eager]_gmem": 5.22613525390625
            }
        }
    }
]
```

cc @xuzhao9
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240208+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240207+cu121

Base PyTorch commit: d0ca849fdf9807e730f91ce8e86d126b241e0940

Affected PyTorch version: 2.3.0.dev20240208+cu121

Affected PyTorch commit: 5f6b35915af98fac741998edb55aeeba4aaa3794

Affected Tests:
- test_train[LearningToPaint-cpu-eager]: +15.74098%
- test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]: -11.43266%
- test_eval[pytorch_stargan-cpu-eager]: +8.13840%
- test_eval[pytorch_stargan-cpu-jit]: -14.09097%
- test_eval[resnext50_32x4d-cpu-jit]: +26.43378%
- test_eval[shufflenet_v2_x1_0-cpu-eager]: -12.57871%
- test_train[tts_angular-cpu-eager]: +9.48007%


cc @xuzhao9

Result json: 
```
{
  "start": "d0ca849fdf9807e730f91ce8e86d126b241e0940",
  "end": "5f6b35915af98fac741998edb55aeeba4aaa3794",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "5f2ad407a9d",
      "commit1_time": "2024-02-07 00:41:51 +0000",
      "commit1_digest": {
        "test_train[LearningToPaint-cpu-eager]": 0.22561463890597225,
        "test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]": 0.08558876970782876
      },
      "commit2": "e9907a34460",
      "commit2_time": "2024-02-07 00:48:00 +0000",
      "commit2_digest": {
        "test_train[LearningToPaint-cpu-eager]": 0.2607216318603605,
        "test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]": 0.07227953588590026
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7833373157


Noise, closed.
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240210+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240209+cu121

Base PyTorch commit: 930b60f5aa7e17402491900a545d96822796b638

Affected PyTorch version: 2.3.0.dev20240210+cu121

Affected PyTorch commit: 42ca82dfb1eb133b287b881c346a0032c1c2a08e

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: -28.22126%
- test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]: +7.44217%
- test_eval[pytorch_stargan-cpu-jit]: -17.48634%
- test_eval[shufflenet_v2_x1_0-cpu-eager]: -8.80510%
- test_train[timm_nfnet-cpu-jit]: +8.84381%
- test_train[tts_angular-cpu-eager]: -10.42186%


cc @xuzhao9

Result json: 
```
{
  "start": "930b60f5aa7e17402491900a545d96822796b638",
  "end": "42ca82dfb1eb133b287b881c346a0032c1c2a08e",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "25a0fa6d139",
      "commit1_time": "2024-02-09 18:58:55 +0000",
      "commit1_digest": {
        "test_train[timm_nfnet-cpu-jit]": 11.139131883019582
      },
      "commit2": "01e248d6f12",
      "commit2_time": "2024-02-09 19:01:54 +0000",
      "commit2_digest": {
        "test_train[timm_nfnet-cpu-jit]": 12.787111069029198
      }
    },
    {
      "commit1": "3278b4c557b",
      "commit1_time": "2024-02-10 03:06:58 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08193294913507998
      },
      "commit2": "42ca82dfb1e",
      "commit2_time": "2024-02-10 03:22:06 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09768780632875859,
        "test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]": 0.07573601566255092,
        "test_eval[pytorch_stargan-cpu-jit]": 0.27305402685888114,
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.16107456446625293,
        "test_train[timm_nfnet-cpu-jit]": 12.663205798203126,
        "test_train[tts_angular-cpu-eager]": 0.2369374900124967
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7856320643
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240211+cu121'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240210+cu121

Base PyTorch commit: 42ca82dfb1eb133b287b881c346a0032c1c2a08e

Affected PyTorch version: 2.3.0.dev20240211+cu121

Affected PyTorch commit: 3ab08946d5052eaeda11d683d6a58e801a032755

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: -12.12817%
- test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]: -7.62186%
- test_eval[shufflenet_v2_x1_0-cpu-eager]: +11.22779%
- test_train[timm_regnet-cpu-eager]: +10.11562%
- test_train[tts_angular-cpu-eager]: +15.91174%


cc @xuzhao9

Result json: 
```
{
  "start": "42ca82dfb1eb133b287b881c346a0032c1c2a08e",
  "end": "3ab08946d5052eaeda11d683d6a58e801a032755",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "57d8f676191",
      "commit1_time": "2024-02-10 19:33:37 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.08245309875346721
      },
      "commit2": "568740f0804",
      "commit2_time": "2024-02-10 20:27:59 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07620419734157621
      }
    },
    {
      "commit1": "568740f0804",
      "commit1_time": "2024-02-10 20:27:59 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.07620419734157621
      },
      "commit2": "2c8722182eb",
      "commit2_time": "2024-02-10 21:56:00 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06864220281131565
      }
    },
    {
      "commit1": "2c8722182eb",
      "commit1_time": "2024-02-10 21:56:00 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.06864220281131565
      },
      "commit2": "6db6a1b526f",
      "commit2_time": "2024-02-10 22:35:00 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0833537692669779
      }
    },
    {
      "commit1": "d8e319a961b",
      "commit1_time": "2024-02-10 23:02:05 +0000",
      "commit1_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.0779700511135161
      },
      "commit2": "3ab08946d50",
      "commit2_time": "2024-02-10 23:04:36 +0000",
      "commit2_digest": {
        "test_eval[pyhpc_equation_of_state-cpu-jit]": 0.09115854618139566,
        "test_eval[pyhpc_turbulent_kinetic_energy-cpu-jit]": 0.07389496047981084,
        "test_eval[shufflenet_v2_x1_0-cpu-eager]": 0.16278377044945955,
        "test_train[timm_regnet-cpu-eager]": 0.8448276366107166,
        "test_train[tts_angular-cpu-eager]": 0.23655516053549946
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7863500177
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.3.0.dev20240212+cu121'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.3.0.dev20240211+cu121

Base PyTorch commit: 3ab08946d5052eaeda11d683d6a58e801a032755

Affected PyTorch version: 2.3.0.dev20240212+cu121

Affected PyTorch commit: dcb08a7044d986ceecd7252fdb2dd0e10501587a

Affected Tests:
- test_eval[pyhpc_equation_of_state-cpu-jit]: +15.48235%
- test_train[tts_angular-cpu-eager]: -11.84071%


cc @xuzhao9

Result json: 
```
{
  "start": "3ab08946d5052eaeda11d683d6a58e801a032755",
  "end": "dcb08a7044d986ceecd7252fdb2dd0e10501587a",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "95a8d5b1bc5",
      "commit1_time": "2024-02-11 19:29:19 +0000",
      "commit1_digest": {
        "test_train[tts_angular-cpu-eager]": 0.1980879666749388
      },
      "commit2": "b8e4423278b",
      "commit2_time": "2024-02-11 19:29:24 +0000",
      "commit2_digest": {
        "test_train[tts_angular-cpu-eager]": 0.21859262036159635
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/7875431400


False positive, closed