Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '8c684663042a61de20f163ab6a292cd43c67e294'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 2c3aa09cde00fe4b45307770ebc86178d61773e7

Affected PyTorch commit: 8c684663042a61de20f163ab6a292cd43c67e294

Affected Tests:
- test_eval[detectron2_fasterrcnn_r_50_fpn-cuda-eager]_latency: +10.10224%
- test_train[lennard_jones-cuda-eager]_latency: +10.20684%
- test_train[mobilenet_v3_large-cuda-eager]_latency: +10.35100%
- test_eval[mobilenet_v3_large-cuda-eager]_latency: +16.20380%
- test_train[phlippe_resnet-cuda-eager]_latency: +10.06037%
- test_train[resnet18-cuda-eager]_latency: +10.21684%
- test_eval[yolov3-cuda-eager]_latency: +11.06962%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6014744962

cc @xuzhao9


Workflow: https://github.com/pytorch/benchmark/actions/runs/6015520958

```
{
  "target_repo": "pytorch",
  "start": "138e2895d08a6517c5718b2a0118c1b23ff4664c",
  "end": "e68b3ad14f9598334a97c1aff15408e1d8f09002",
  "result": [
    {
      "commit1": "20f3808aa2a",
      "commit1_time": "2023-08-28 17:01:23 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "20f3808aa2aa65bc2e800266da9249e5b6d57874",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "20f3808aa2aa65bc2e800266da9249e5b6d57874"
        },
        "metrics": {
          "test_train[lennard_jones-cuda-eager]_latency": 5.339759,
          "test_train[lennard_jones-cuda-eager]_cmem": 0.5068359375,
          "test_train[lennard_jones-cuda-eager]_gmem": 1.53607177734375
        }
      },
      "commit2": "a1d81322108",
      "commit2_time": "2023-08-28 17:08:13 +0000",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "a1d81322108979967a6caeefc81162f0a5f1fc98",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "a1d81322108979967a6caeefc81162f0a5f1fc98"
        },
        "metrics": {
          "test_train[lennard_jones-cuda-eager]_latency": 5.958023,
          "test_train[lennard_jones-cuda-eager]_cmem": 0.5078125,
          "test_train[lennard_jones-cuda-eager]_gmem": 1.53607177734375
        }
      }
    },
    {
      "commit1": "00eed6f3675",
      "commit1_time": "2023-08-29 04:10:17 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "00eed6f36759323c70c6e05ccb1561367921eee0",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "00eed6f36759323c70c6e05ccb1561367921eee0"
        },
        "metrics": {
          "test_train[lennard_jones-cuda-eager]_latency": 5.376983,
          "test_train[lennard_jones-cuda-eager]_cmem": 0.509765625,
          "test_train[lennard_jones-cuda-eager]_gmem": 1.53607177734375
        }
      },
      "commit2": "e68b3ad14f9",
      "commit2_time": "2023-08-29 04:31:44 +0000",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "e68b3ad14f9598334a97c1aff15408e1d8f09002",
          "device": "NVIDIA A100-SXM4-40GB",
          "git_commit_hash": "e68b3ad14f9598334a97c1aff15408e1d8f09002"
        },
        "metrics": {
          "test_train[lennard_jones-cuda-eager]_latency": 6.179011,
          "test_train[lennard_jones-cuda-eager]_cmem": 0.509765625,
          "test_train[lennard_jones-cuda-eager]_gmem": 1.53607177734375
        }
      }
    }
  ]
}
```

Unrelated - noisy workload - closed.