zhiminy/EvalEng huggingface dataset:
Repository: pytorch/benchmark
Issue: simple-gpt usage and requirements
Root cause: Documentation Deficiency

Hi folks，

many thanks for this great work, I am at the very beginning stage of using torchbench, I want to try simple_gpt, looks it's different from nanogpt, I can't run it successfully based on the instructions of https://github.com/pytorch/benchmark/blob/main/README.md,  BTW, there is one command: https://github.com/pytorch/benchmark/pull/1867, but this command looks needs to be run on pytorch repo,  I am confused here.
1, what's the difference between torchbench and pytorch/benchmark?
2, what's the necessary's step/pre-request to run simple_gpt model?

looking forward to get feedback from you, 

thanks again. 

See my answers below:

1. `pytorch/benchmark` is used by `pytorch/pytorch` in its CI workflows. The PyTorch team uses the `torchbench.py` as a customized benchmark driver to implement their own features.
2. To run the simple_gpt model:
   a. Install torch, torchvision, and torchaudio nightly packages
   b. git clone the pytorch/pytorch repo (https://github.com/pytorch/pytorch) and cd into it.
   b. Run the command `python benchmarks/dynamo/torchbench.py -d cuda --output-directory=benchmark_logs --output=performance.csv --inference --performance --timing --print-memory --multiprocess --nothing --only simple_gpt`



thanks Xu zhao,  it's much clear. 
BTW,  regarding to run simple_gpt,  is multi_gpu the basic requirement or can we run it with single gpu?

Best Regrads

cc the model owner on more details @xmfan 

You can run `simple_gpt` with single gpu using the command in the linked PR, provided your GPU has enough memory. You can ignore the warning message:
```
WARNING:common:The use multiprocess flag is set but there are <= 1 devices available.
```

Note that this version of simple_gpt does not contain any model weights, nor the code to load them

got it, thanks, @xuzhao9 @xmfan
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'a04dd794ad694baeb257c12329c3166c6a44ae50'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 2d7dd2e800dfd6332656074bfa208e4b25cfe907

Affected PyTorch commit: a04dd794ad694baeb257c12329c3166c6a44ae50

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: +10.84122%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6773344391

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6791128142

```
{
  "target_repo": "pytorch",
  "start": "64f326097be8ac66ff057365f3bed2d64c697563",
  "end": "bd9be877e4459d7889e5e3c8051caaffdfe21c85",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: backend for --torchdynamo selection

Hi  I am trying to run compilation of --torchdynamo with different backends, there are lot's of options:
[--torchdynamo {aot_eager,aot_eager_decomp_partition,aot_eager_default_partitioner,aot_torchxla_trace_once,aot_torchxla_trivial,aot_ts,cudagraphs,dynamo_accuracy_minifier_backend,dynamo_minifier_backend,eager,eager_debug,inductor,non_leaf_compile_error_TESTING_ONLY,onnxrt,openxla,openxla_eval,pre_dispatch_eager,relu_accuracy_error_TESTING_ONLY,relu_compile_error_TESTING_ONLY,relu_runtime_error_TESTING_ONLY,torchxla_trace_once,torchxla_trivial,ts,tvm}]

most of them is the first time to heard for me,  could u share/provide some doc/guidelines on how to understand its difference?

Thanks

These are the backends that connect with torchdynamo. To understand them, you need to learn how torchdynamo works.

The backends are defined at: https://github.com/pytorch/pytorch/tree/9a909891212a73cdeed9ef379ac6d0fbe2b796db/torch/_dynamo/backends. Specifically, `torch.compile()` uses the `inductor` backend.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '2c3ba6926e38dba05bda34f0af9c092a40cff5b7'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: a04dd794ad694baeb257c12329c3166c6a44ae50

Affected PyTorch commit: 2c3ba6926e38dba05bda34f0af9c092a40cff5b7

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: -10.99237%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6814355761

cc @xuzhao9


Workflow: https://github.com/pytorch/benchmark/actions/runs/6884777196

```
{
  "target_repo": "pytorch",
  "start": "bd9be877e4459d7889e5e3c8051caaffdfe21c85",
  "end": "fbf7866ac98258a8130e14de3021d3160bc28af3",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: automatic run test benchmark with docker file
Root cause: Documentation Deficiency

Hi 

there is docker file to build the docker image, but no script to run the benchmark test within docker image automatically,  I tried many ways to run the test_bench.py  in docker container build with [torchbench-nightly.dockerfile], 
looks there is no good docker run command to run it, 

here is my command:
![image](https://github.com/pytorch/benchmark/assets/146043398/a24c7845-6f61-4076-8c41-3c958ef33319)

this command sometimes bad to work, some strange issue, such as "/runner should be mount to simpleDir", could u please provide the command flow/script to run benchmark test through docker?

many thanks.





strange to get this feedback:
![image](https://github.com/pytorch/benchmark/assets/146043398/62daebca-5142-47fe-965e-05792ded3fd8)

when direct run with command:
docker run -it --gpus all  ghcr.io/pytorch/torchbench:latest

thanks


Here is an example of running benchmark with docker: https://github.com/pytorch/benchmark/blob/main/.github/workflows/pr-a10g.yml#L56

@xuzhao9  thanks, with this workflow file, I can modify the dockerfile to access volume mounting of my user id and runner, I can run this flow smoothly now.

there is another question of running test_bench.py,  how can I get all of the output performance metrics to a file?  looks --benchmark-autosave not help.

thanks


@jinsong-mao Here is our example script to run test_bench.py: https://github.com/pytorch/benchmark/blob/main/.github/scripts/run.sh#L48

@xuzhao9 hi xuzhao, if I want to run the test_bench.py in fp16, how should i add configs? thanks


Hi @xuzhao9 ,  after running the tests and checking the output json file, it's very strange that the GPU clock frequency is only 765, 
![image](https://github.com/pytorch/benchmark/assets/146043398/2fc02296-57cc-4744-b6c5-d5d3a73cf231)

I am running test on A100, am I correct?  how to explain it?

Thanks

@jinsong-mao `test_bench.py` does not support FP16 as a global config. However, some models are running at fp16 precision by default, so some of the numbers you get from `test_bench.py` are already at FP16.

@jinsong-mao To minimize latency variation, we are pinning the A100 GPU frequency at 1215,1410: https://github.com/pytorch/benchmark/blob/main/.github/workflows/userbenchmark-a100.yml#L33

For NVIDIA T4 GPU, we pin the frequency at `5001,900`: https://github.com/pytorch/benchmark/blob/v2.0/torchbenchmark/score/configs/v2/config-v2.env#L16C15-L16C23

Sorry I am not sure why the GPU clock frequency is at 765 in your case. Please check your environment.

The GPU clock frequency may not be correct in this log. I remember it was obtained before or after the testing. Since GPUs automatically adjust their frequency, the frequency may return to the default 765 for A100 when GPUs are idle. But when GPUs are busy on testing, their frequency is as high as 1410 if you check with `nvidia-smi -q -d CLOCK`.   

You can use `sudo nvidia-smi -lgc 1410,1410` to boost the frequency to 1410MHz for A100. But the default frequency when GPUs are idle is still 765.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '63a5a14da9ef3ebd68ce0cebea4aa84e030a2cf8'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: f40306d6c4b2613c18525a274d98feeda0036473

Affected PyTorch commit: 63a5a14da9ef3ebd68ce0cebea4aa84e030a2cf8

Affected Tests:
- test_eval[yolov3-cuda-eager]_latency: +10.37931%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6841725074

cc @xuzhao9


https://github.com/pytorch/benchmark/actions/runs/6891964505
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'a45a8bf9e7e1530692f2703f8da430bc2825af7c'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 63a5a14da9ef3ebd68ce0cebea4aa84e030a2cf8

Affected PyTorch commit: a45a8bf9e7e1530692f2703f8da430bc2825af7c

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: +11.27744%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6852552207

cc @xuzhao9


https://github.com/pytorch/benchmark/actions/runs/6892250801
Repository: pytorch/benchmark
Issue: How to run test_bench.py with ROCM?
Root cause: Documentation Deficiency

Hi @xuzhao9, 

I don't know how to create a dockerfile for AMD ROCM,  is there any example?  

Best Regards


You could use the userbenchmark rocm-test: https://github.com/pytorch/benchmark/blob/main/userbenchmark/rocm-test/run.py

Note that `test_bench.py` only covers the default, PT1 eager mode runs. It does not support any of the FP16, ROCM, or PT2 Inductor.
We use  the`userbenchmark` framework to address this issue: https://github.com/pytorch/benchmark/blob/main/userbenchmark/ADDING_USERBENCHMARKS.md

For example, to test TorchInductor, you could use the `dynamo` userbenchmark: https://github.com/pytorch/benchmark/pull/1960

To test ROCM, you could use the `rocm-test` userbenchmark.

There is no current userbenchmark for FP16 right now, but we welcome you to contribute your own.

Hi @xuzhao9  thanks for you remind,  I can reproduce the userbenchmark command for llama on cuda like this:
![image](https://github.com/pytorch/benchmark/assets/146043398/9b0735ed-8825-4cef-95be-0201d6d25060)

1, the output metric is very simple, it doesn't give me any information like inference token per second, input/output sequence length, batch size(default 32),  how should I get that information?

2, I also wants to test different composition of input/output sequence length and even batch size of llama test, what should I do to achieve this?

3, the training test for llama looks failed with some issues, is it my fault or it's the current status of llama?
![image](https://github.com/pytorch/benchmark/assets/146043398/9256b3f7-d290-49a1-8ac6-438ef890b204)

many thanks for your kindly feedback. 
Best

@jinsong-mao You could read the README.md file at https://github.com/pytorch/pytorch/tree/main/benchmarks/dynamo for more details. The `dynamo` userbenchmark is owned by the PyTorch 2.0 team, so my answers might be inaccurate, but here are my takes:

1. The input/output sequence length is hard-coded and not tunable: https://github.com/pytorch/benchmark/blob/main/torchbenchmark/models/llama/__init__.py#L21. It is possible to make it a model argument, but that would need additional work. Based on my understanding, the input are tensors, not tokens, so it cannot provide information like tokens per second.
2. Since the seqlen is hardcoded right now, you can't test different seqlens. However, different batch sizes are tunable with `--batch-size` (https://github.com/pytorch/pytorch/blob/main/benchmarks/dynamo/common.py#L2905)
3. Train test is not implemented right now. We welcome the community contribution to enable it.
Repository: pytorch/benchmark
Issue: Fix V2 bisection backlog

```[tasklist]
### Tasks
- [x] bisection-gh6707630439 [link](https://github.com/pytorch/benchmark/actions/runs/6710053921) [rerun](https://github.com/pytorch/benchmark/actions/runs/6884762526)
- [x] bisection-gh6407125260 [link](https://github.com/pytorch/benchmark/actions/runs/6409249711) [rerun](https://github.com/pytorch/benchmark/actions/runs/6886585482)
- [x] bisection-gh6891812881 [link](https://github.com/pytorch/benchmark/actions/runs/6898271942)
- [x] bisection-gh6904963254 [link](https://github.com/pytorch/benchmark/actions/runs/6907107994/job/18793577498)
- [ ] bisection-gh6470437914: 2023-10-09(97287ff) to 2023-10-10(c5b2551): [link](https://github.com/pytorch/benchmark/actions/runs/6914288405)
- [ ] 2023-06-22 to 2023-09-08
```

The error is because CUDA 11.8 is required to build Kineto while V2 was still using 11.7 as default. Upgraded to 11.8.

```
{
  "start": "92fea5ae3f5307b8ebf5e9cb123d971237f8bf03",
  "end": "468a73f0e3527c52495c864c7d48dc26684f6c0b",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '690c805c8b539501aad5fbf18914ac92afb65d5a'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: a5a404865c01f86881f6b3ab0cd9a562d0b420de

Affected PyTorch commit: 690c805c8b539501aad5fbf18914ac92afb65d5a

Affected Tests:
- test_train[maml_omniglot-cuda-eager]_latency: +10.45702%
- test_train[mobilenet_v3_large-cuda-eager]_latency: +10.81443%
- test_eval[phlippe_resnet-cuda-eager]_latency: +11.99753%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: +12.63274%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6893332158

cc @xuzhao9
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '690c805c8b539501aad5fbf18914ac92afb65d5a'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: a5a404865c01f86881f6b3ab0cd9a562d0b420de

Affected PyTorch commit: 690c805c8b539501aad5fbf18914ac92afb65d5a

Affected Tests:
- test_eval[hf_Bert-cuda-eager]_latency: +10.54552%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: +12.85242%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6895705024

cc @xuzhao9
Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '2.2.0.dev20231116+cu118'
Root cause: Others


TorchBench CI has detected a performance signal.

Base PyTorch version: 2.2.0.dev20231107+cu118

Base PyTorch commit: 10a829b85d721a76ba436f2416d2d802aab6e9e0

Affected PyTorch version: 2.2.0.dev20231116+cu118

Affected PyTorch commit: a3b859fc673515f74d103494e61b432c9cc9009f

Affected Tests:
- test_eval[maml-cuda-eager]: +7.53184%
- test_train[maml_omniglot-cuda-eager]: +8.92542%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]: +9.42485%
- test_eval[pytorch_stargan-cpu-jit]: -9.59628%
- test_train[timm_regnet-cpu-eager]: -7.28432%
- test_train[tts_angular-cpu-eager]: +11.80518%


cc @xuzhao9

Result json: 
```
{
  "start": "10a829b85d721a76ba436f2416d2d802aab6e9e0",
  "end": "a3b859fc673515f74d103494e61b432c9cc9009f",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "cc11c0d11bf",
      "commit1_time": "2023-11-15 19:55:47 +0000",
      "commit1_digest": {
        "test_train[maml_omniglot-cuda-eager]": 5.461679924052442,
        "test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]": 50.496652409949455
      },
      "commit2": "12b2dd16b05",
      "commit2_time": "2023-11-15 20:26:13 +0000",
      "commit2_digest": {
        "test_train[maml_omniglot-cuda-eager]": 5.87546450990194,
        "test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]": 54.94782136425492
      }
    }
  ]
}
```
Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/6898271942


Noise, closed.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '07c9b95f45c54ec7e6134961ef00e3c4b4403f7f'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 0603dcb449ccf52baea0cc6981f8fc587566f1b3

Affected PyTorch commit: 07c9b95f45c54ec7e6134961ef00e3c4b4403f7f

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: +17.18571%
- test_train[phlippe_resnet-cuda-eager]_latency: +10.49662%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6914939521

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6935754237

Closed by empty result.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '6f045ad045b78cfb1b70d169c78cf4b02e4908fe'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 07c9b95f45c54ec7e6134961ef00e3c4b4403f7f

Affected PyTorch commit: 6f045ad045b78cfb1b70d169c78cf4b02e4908fe

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: -14.07179%
- test_eval[shufflenet_v2_x1_0-cuda-eager]_latency: -11.05462%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6921541981

cc @xuzhao9


Workflow: https://github.com/pytorch/benchmark/actions/runs/6923558121

```
{
  "target_repo": "pytorch",
  "start": "5e307417548374bb40d754518d324efa5359e8aa",
  "end": "870539670aff386d66b190b4749316e21d2ae56e",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: I encountered an issue with the `torchbenchmark` package. When installed, it is only importable within the project directory and not from other directories. This behavior seems unexpected.
Root cause: Validation Gap

ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 import torchbenchmark

File <string>:11, in _custom_import(module_name, globals, locals, fromlist, level)

File ~/miniconda/envs/python38_torch201_cuda/lib/python3.8/site-packages/torchbenchmark/__init__.py:18
     14 from urllib import request
     16 import torch
---> 18 from components._impl.tasks import base as base_task
     19 from components._impl.workers import subprocess_worker
     20 from . import models

File <string>:11, in _custom_import(module_name, globals, locals, fromlist, level)

ModuleNotFoundError: No module named 'components'




Try to import torchbenchmark from a directory outside the project.
python
Copy code
import torchbenchmark
Expected Behavior:

I expect to be able to import the torchbenchmark package from any directory after it has been installed.

Actual Behavior:

The import works only within the project directory and fails from other directories.

Environment:

Python version: [e.g., 3.8]
Operating System: [e.g., Windows, Linux]
Other relevant information: [e.g., virtual environment, package versions]
Additional Information:

Include any additional information that might be relevant to understanding or resolving the issue.

Note:

Feel free to attach any screenshots or additional details that can help in diagnosing the problem.

css
Copy code

Make sure to replace placeholders like `[e.g., 3.8]` and `[e.g., Windows, Linux]` with the spe

Right, we only tested to import `torchbenchmark.models.*` packages, not `torchbenchmark` it self.

We can work on adding this support.

Similar issue, I'm running `~/local/pytorch/benchmarks/dynamo/run_all.sh` but seeing the same `ModuleNotFoundError`.

I haven't found a workaround for this yet.

https://github.com/pytorch/benchmark/pull/2201 should fix this problem, can you help try it?
Repository: pytorch/benchmark
Issue: llama_7b model OOM issue
Root cause: Resource Mishandling

Hi 

I duplicate the llama model and rename it into llama_7b, changed the model parameters according to llama_7b specification, looks like this:
![image](https://github.com/pytorch/benchmark/assets/146043398/2f39b5c0-8396-4765-a026-5c3efdb05458)

skiped the CPU eager mode, only run the cuda model. 

it reports the following issue when running with this command:
python userbenchmark/dynamo/dynamobench/torchbench.py -dcuda --float16 -n1 --inductor --performance --inference --filter "llama" --batch_size 1 --in_slen 32 --out_slen 3 --output-dir=torchbench_llama_test_logs
![image](https://github.com/pytorch/benchmark/assets/146043398/d71d6856-8c61-46e3-9f1c-3389a03b7c1b)

If I want to run this model, how should I fix it?  my hardware is A100-40G

thanks

We only guarantee the runability of models on PT eager mode on A100 40GB in our CI. It is possible that inductor uses more GPU memory than eager mode, causing OOM. Optimizing GPU memory usage with inductor is an open question.
cc @msaroufim 

@xuzhao9 I tried to use 4xA100-40G to avoid the OOM issue,  looks torchbench.py only use one GPU's memory, I used options like --device-index or --multiprocess,  both failed. do you have any advice on multi GPU support?

thanks

@jinsong-mao Unfortunately, we don't have multi-GPU support right now due to lack of CI infra to test it. If you are interested, you could try the `--multiprocess`, `--ddp`, or `--fsdp` option in the PT2 Benchmark Runner: https://github.com/pytorch/benchmark/blob/main/userbenchmark/dynamo/dynamobench/common.py#L3164-L3174
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'df0c1b3a939b5d9be09eb28016dfec63e51204a6'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 794fd233b401def6002984c9df88fe188ce00e79

Affected PyTorch commit: df0c1b3a939b5d9be09eb28016dfec63e51204a6

Affected Tests:
- test_train[dcgan-cuda-eager]_latency: +11.06719%
- test_train[detectron2_maskrcnn-cuda-eager]_latency: +19.49786%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: +11.36724%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6996616291

cc @xuzhao9


Workflow: https://github.com/pytorch/benchmark/actions/runs/7010678041

```
{
  "target_repo": "pytorch",
  "start": "0f5e24bda9450a89ba56d2fdd471f56d97fe4546",
  "end": "d37c4c69954ad7bdccca96854105c48e93d4587e",
  "result": []
}
```