Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '1.14.0.dev20221109+cu116'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal.

Base PyTorch version: 1.14.0.dev20221108+cu116

Base PyTorch commit: ed97e0aa2918e687309ee9a146c8294aefb237d2

Affected PyTorch version: 1.14.0.dev20221109+cu116

Affected PyTorch commit: a7420d2ccb62d005f2e1853cfef8d25eb7748a90

Affected Tests:
- test_train[Super_SloMo-cuda-jit]: +38.42899%
- test_eval[Super_SloMo-cuda-jit]: +58.32354%
- test_train[Super_SloMo-cuda-eager]: +38.33748%
- test_eval[Super_SloMo-cuda-eager]: +58.10627%
- test_eval[drq-cuda-eager]: +9.78706%
- test_train[maml-cuda-eager]: +43.48992%
- test_eval[maml-cuda-eager]: +44.05519%
- test_eval[maml_omniglot-cpu-eager]: -8.20316%
- test_train[maml_omniglot-cuda-eager]: +10.19391%
- test_eval[maml_omniglot-cuda-eager]: +17.63023%
- test_eval[mnasnet1_0-cpu-jit]: +29.34491%
- test_eval[opacus_cifar10-cuda-eager]: +13.29463%
- test_train[pytorch_unet-cuda-jit]: +64.45304%
- test_eval[pytorch_unet-cuda-jit]: +32.27077%
- test_train[pytorch_unet-cuda-eager]: +64.38176%
- test_eval[pytorch_unet-cuda-eager]: +30.06754%
- test_train[resnext50_32x4d-cpu-jit]: +9.15583%
- test_train[yolov3-cuda-eager]: +59.01862%
- test_eval[yolov3-cuda-eager]: +52.37508%


cc @xuzhao9

Result json: 
```
{
  "start": "ed97e0aa2918e687309ee9a146c8294aefb237d2",
  "end": "a7420d2ccb62d005f2e1853cfef8d25eb7748a90",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "5eecfcf5f3d",
      "commit1_time": "2022-11-08 18:52:56 +0000",
      "commit1_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.158440691162832,
        "test_eval[Super_SloMo-cuda-jit]": 0.6240458720945753,
        "test_train[Super_SloMo-cuda-eager]": 1.161482525523752,
        "test_eval[Super_SloMo-cuda-eager]": 0.6255389389931224,
        "test_eval[drq-cuda-eager]": 0.0021878498138657495,
        "test_train[maml-cuda-eager]": 0.6540568203898147,
        "test_eval[maml-cuda-eager]": 0.03645486057004226,
        "test_train[maml_omniglot-cuda-eager]": 4.9522784824483095,
        "test_eval[maml_omniglot-cuda-eager]": 0.0010164210845479847,
        "test_eval[opacus_cifar10-cuda-eager]": 0.005449093823560356,
        "test_train[pytorch_unet-cuda-jit]": 0.48875991369131955,
        "test_eval[pytorch_unet-cuda-jit]": 0.15588033852400257,
        "test_train[pytorch_unet-cuda-eager]": 0.48886265341425317,
        "test_eval[pytorch_unet-cuda-eager]": 0.1673918773420155,
        "test_train[yolov3-cuda-eager]": 0.8960100429947488,
        "test_eval[yolov3-cuda-eager]": 0.23439507512375712
      },
      "commit2": "d98a884b33e",
      "commit2_time": "2022-11-08 19:04:25 +0000",
      "commit2_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.6034549559350126,
        "test_eval[Super_SloMo-cuda-jit]": 0.9877922425977885,
        "test_train[Super_SloMo-cuda-eager]": 1.6061797839007341,
        "test_eval[Super_SloMo-cuda-eager]": 0.9888652526540682,
        "test_eval[drq-cuda-eager]": 0.0024137507092599326,
        "test_train[maml-cuda-eager]": 0.9438509189989418,
        "test_eval[maml-cuda-eager]": 0.0532651458401233,
        "test_train[maml_omniglot-cuda-eager]": 5.516390937683172,
        "test_eval[maml_omniglot-cuda-eager]": 0.0011943323885828508,
        "test_eval[opacus_cifar10-cuda-eager]": 0.006168293960702916,
        "test_train[pytorch_unet-cuda-jit]": 0.8044066080008634,
        "test_eval[pytorch_unet-cuda-jit]": 0.2060748871648684,
        "test_train[pytorch_unet-cuda-eager]": 0.8046210412401706,
        "test_eval[pytorch_unet-cuda-eager]": 0.2175196320633404,
        "test_train[yolov3-cuda-eager]": 1.4257998043205589,
        "test_eval[yolov3-cuda-eager]": 0.3575076060369611
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/3430364811


Known problem, closed.