Repository: pytorch
Issue: Profiler UTF-8 decode issue
Labels: high priority,triage review,oncall: profiler

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Steps to reproduce the behavior:

1. Modify huggingface transformer's trainer.py, wrap train function with profiler.profile
2. Run a huggingface transformer's model single-node multi-gpus training, with DeepSpeed enabled

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
Will raise error after some training steps:
```
  File "./run_mlm.py", line 898, in <module>
    main()
  File "./run_mlm.py", line 849, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1989, in train
    my_profiler.step()
  File "/opt/conda/lib/python3.8/site-packages/torch/profiler/profiler.py", line 280, in step
  File "/opt/conda/lib/python3.8/site-packages/torch/autograd/profiler.py", line 1137, in parse_kineto_results
    if filter_name(kineto_event.name()):
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb9 in position 4: invalid start byte

I'm Confused about this error info, since can't get the detail about 'kineto_event'.
```

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

- PyTorch version: 1.10.0a0+ecc3718
- Is debug build: False
- CUDA used to build PyTorch: 11.4
- ROCM used to build PyTorch: N/A
- 
- OS: Ubuntu 20.04.2 LTS (x86_64)
- GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
- Clang version: Could not collect
- CMake version: version 3.21.0
- Libc version: glibc-2.31
- 
- Python version: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05)  [GCC 9.3.0] (64-bit runtime)
- Python platform: Linux-5.11.0-31-lowlatency-x86_64-with-glibc2.10
- Is CUDA available: True
- CUDA runtime version: 11.4.48
- GPU models and configuration:
- GPU 0: NVIDIA A100-SXM4-40GB
- GPU 1: NVIDIA A100-SXM4-40GB
- GPU 2: NVIDIA A100-SXM4-40GB
- GPU 3: NVIDIA A100-SXM4-40GB
- GPU 4: NVIDIA A100-SXM4-40GB
- GPU 5: NVIDIA A100-SXM4-40GB
- GPU 6: NVIDIA A100-SXM4-40GB
- GPU 7: NVIDIA A100-SXM4-40GB
- 
- Nvidia driver version: 470.57.02
- cuDNN version: Probably one of the following:
- /usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.2
- /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.2
- HIP runtime version: N/A
- MIOpen runtime version: N/A
- 
- Versions of relevant libraries:
- [pip3] numpy==1.21.1
- [pip3] nvidia-dlprof-pytorch-nvtx==1.3.0
- [pip3] pytorch-quantization==2.1.0
- [pip3] pytorch-transformers==1.1.0
- [pip3] torch==1.10.0a0+ecc3718
- [pip3] torchtext==0.11.0a0
- [pip3] torchvision==0.11.0a0
- [conda] magma-cuda110             2.5.2                         5    local
- [conda] mkl                       2019.5                      281    conda-forge
- [conda] mkl-include               2019.5                      281    conda-forge
- [conda] numpy                     1.21.1           py38h9894fe3_0    conda-forge
- [conda] nvidia-dlprof-pytorch-nvtx 1.3.0                    pypi_0    pypi
- [conda] pytorch-quantization      2.1.0                    pypi_0    pypi
- [conda] pytorch-transformers      1.1.0                    pypi_0    pypi
- [conda] torch                     1.10.0a0+ecc3718          pypi_0    pypi
- [conda] torchtext                 0.11.0a0                 pypi_0    pypi
- [conda] torchvision               0.11.0a0                 pypi_0    pypi

## Additional context

<!-- Add any other context about the problem here. -->


cc @ezyang @gchanan @zou3519 @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb @ilia-cher @gdankel @bitfort @orionr