Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '3dedcb31fce3b5e55fee44d15eb693530c6606f8'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 10b46f7c7f69f9bf705d2b6ea53efb9c59145685

Affected PyTorch commit: 3dedcb31fce3b5e55fee44d15eb693530c6606f8

Affected Tests:
- test_eval[doctr_det_predictor-cuda-eager]_cmem: +10.54131%
- test_eval[timm_resnest-cuda-eager]_gmem: +8.27724%
- test_train[torchrec_dlrm-cuda-eager]_latency: +7.70788%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5105246929

cc @xuzhao9


2023-05-28

Test workflow: https://github.com/pytorch/benchmark/actions/runs/5205215214

Output:

```
{
  "target_repo": "pytorch",
  "start": "fb468b6792213e0d8e6221b3bb51e71fcadbed30",
  "end": "c3ea8cc58beec343fef8ec69c409e0b83e60a8e9",
  "result": [
    {
      "commit1": "6c9b94dcda6",
      "commit1_time": "2023-05-26 06:35:23 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "6c9b94dcda696984552769da6cf578ccca0cfbdd",
          "git_commit_hash": "6c9b94dcda696984552769da6cf578ccca0cfbdd"
        },
        "metrics": {
          "test_eval[timm_resnest-cuda-eager]_latency": 39.873679,
          "test_eval[timm_resnest-cuda-eager]_cmem": 0.7080078125,
          "test_eval[timm_resnest-cuda-eager]_gmem": 4.2294921875
        }
      },
      "commit2": "c05a317371e",
      "commit2_time": "2023-05-26 07:01:22 -0700",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "c05a317371efc0ea6c40b447829b0cb3c86cde48",
          "git_commit_hash": "c05a317371efc0ea6c40b447829b0cb3c86cde48"
        },
        "metrics": {
          "test_eval[timm_resnest-cuda-eager]_latency": 40.165018,
          "test_eval[timm_resnest-cuda-eager]_cmem": 0.7060546875,
          "test_eval[timm_resnest-cuda-eager]_gmem": 3.6591796875
        }
      }
    },
    {
      "commit1": "87510022157",
      "commit1_time": "2023-05-26 14:57:31 +0000",
      "commit1_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "8751002215790a3a88750faa8f4366933e296693",
          "git_commit_hash": "8751002215790a3a88750faa8f4366933e296693"
        },
        "metrics": {
          "test_eval[timm_resnest-cuda-eager]_latency": 40.04428,
          "test_eval[timm_resnest-cuda-eager]_cmem": 0.705078125,
          "test_eval[timm_resnest-cuda-eager]_gmem": 3.6591796875
        }
      },
      "commit2": "0d5b74da0ca",
      "commit2_time": "2023-05-26 15:20:34 +0000",
      "commit2_digest": {
        "name": "torch-nightly",
        "environ": {
          "pytorch_git_version": "0d5b74da0cab798fbfdb9caa53fad816999c8386",
          "git_commit_hash": "0d5b74da0cab798fbfdb9caa53fad816999c8386"
        },
        "metrics": {
          "test_eval[timm_resnest-cuda-eager]_latency": 39.842427,
          "test_eval[timm_resnest-cuda-eager]_cmem": 0.705078125,
          "test_eval[timm_resnest-cuda-eager]_gmem": 3.9169921875
        }
      }
    }
  ]
}
```