Repository: pytorch
Issue: torch.profiler's FLOPs measure only counts operations involving '+' and '*' .
Labels: oncall: profiler

### 🐛 Describe the bug

(1) c = a - b
(2) c = a + (-b)

Two operations shown above are mathematically identical.

However, torch.profiler does not count the FLOPs of operation (1).

```python
import torch
from torch.profiler import profile

def flops(a, b, op):
    with profile(
        activities = [torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA], 
        with_flops = True) as subtraction:
        if op == '-':
            c = a - b
        elif op == '-':
            c = a + (-b)
        else:
            raise NotImplementedError

    subtraction_events = subtraction.events()
    subtraction_flops = sum([int(evt.flops) for evt in subtraction_events])
    print(subtraction_flops)
```

Now test with two tensors with six elements each.

```python
a = torch.rand((2, 3), device='cuda')
b = torch.rand((2, 3), device='cuda')
flops(a, b, '+')
flops(a, b, '-')
```

```
6
0
```

You can easily find out that the results are different.

This also happens in other operations: **, /, and other library functions like torch.pow, torch.std_mean, etc. are not counted.

I understand that torch.profiler gives 'estimated' value, but I believe this is something far from estimation.

### Versions

Collecting environment information...
PyTorch version: 1.10.1
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.27

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:58:50)  [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-4.15.0-184-generic-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: 11.2.152
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.57.02
cuDNN version: Probably one of the following:
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.1.1
/usr/local/cuda-11.2/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.1.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.10.1
[pip3] torchaudio==0.10.1
[pip3] torchvision==0.11.2
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7e14d7c_0    conda-forge
[conda] mkl_fft                   1.3.1            py39h0c7bc48_1    conda-forge
[conda] mkl_random                1.2.2            py39hde0f152_0    conda-forge
[conda] numpy                     1.22.3           py39he7a7128_0  
[conda] numpy-base                1.22.3           py39hf524024_0  
[conda] pytorch                   1.10.1          py3.9_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchaudio                0.10.1               py39_cu113    pytorch
[conda] torchvision               0.11.2               py39_cu113    pytorch

cc @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb