torch.compiled mish function is x5 slower than eager (CPU)
Labels: triaged, bug, oncall: pt2, module: cpu inductor

### 🐛 Describe the bug

I wanted to test the benefits of`torch.compile` function in Pytorch 2. As I don't have access to a powerful GPU, I tried to speed up computations on CPU. So, I tested if inductor  could match the pytorch hadnwritten [mish activation ](https://pytorch.org/docs/stable/generated/torch.nn.Mish.html).

However, the compiled function is x5 slower than the handwritten eager function or the builtin mish function. This results are the same when:
- With input tensor of 1000 or 10000000
- When using 1 or 4 cores (my laptop has 4 physical cores)

Is this the expected behaviour? Will cuda path be affected too? Thanks!


Here is the code:
```python 
import torch
from torch import nn
import torch.nn.functional as F

import time

torch.set_num_interop_threads(1)
torch.set_num_threads(1) 

def mish(x: torch.Tensor):
    return x * torch.tanh(F.softplus(x))

ops = [
    F.mish,
    mish,
    torch.jit.script(mish),
    # torch.compile(mish, fullgraph=True),
    torch.compile(mish, mode="max-autotune", fullgraph=True),
    # torch.compile(mish, mode="reduce-overhead", fullgraph=True),
]

t = torch.randn(10000000)

torch._inductor.config.debug = True

# Warm up
for _ in range(10):
    for op in ops:
        _ = op(t)

res = {}
N = 10
for i, op in enumerate(ops):
    start = time.time_ns()
    for _ in range(N):
        _ = op(t)
    end = time.time_ns()
    
    res[f"{i}_{op}"] = f"{(end - start) / 1e6 / N:.2f} ms"

print(res)
```

The result  for a input of 10000000 elements:
```
{'0_<function mish at 0x7f3f452a8f70>': '214.37 ms',   # F,mish
 '1_<function mish at 0x7f3f7a7d2e50>': '224.24 ms',  # eager mish
 '2_<torch.jit.ScriptFunction object at 0x7f3f44325f90>': '216.41 ms',  # TS mish
 '3_<function mish at 0x7f3f4266a1f0>': '1228.75 ms'}  # Inductor mish
```
If the input tensor has 1000 elements, all functions take 0.01ms except inductor that takes 0.05ms

Here is the C++ code gen:
```
from ctypes import c_void_p, c_long
import torch
import random
from torch import empty_strided, as_strided, device
from torch._inductor.codecache import AsyncCompile

aten = torch.ops.aten
assert_size_stride = torch._C._dynamo.guards.assert_size_stride
async_compile = AsyncCompile()


kernel_cpp_0 = async_compile.cpp('''
#include "/tmp/torchinductor_victor/77/c7773nj5pwikpmm2pwa62rcudlf7p3if7eyqb5k4sjsvewwje4le.h"
extern "C" void kernel(const float* __restrict__ in_ptr0,
                       float* __restrict__ out_ptr0)
{
    #pragma GCC ivdep
    for(long i0=0; i0<10000000; i0+=1)
    {
        {
            {
                auto tmp0 = in_ptr0[i0];
                auto tmp1 = static_cast<float>(20);
                auto tmp2 = tmp0 > tmp1;
                auto tmp3 = std::exp(tmp0);
                auto tmp4 = std::log1p(tmp3);
                auto tmp5 = tmp2 ? tmp0 : tmp4;
                auto tmp6 = std::tanh(tmp5);
                auto tmp7 = tmp0 * tmp6;
                out_ptr0[i0] = tmp7;
            }
        }
    }
}
''')


async_compile.wait(globals())
del async_compile

def call(args):
    arg0_1, = args
    args.clear()
    buf0 = empty_strided((10000000, ), (1, ), device='cpu', dtype=torch.float32)
    kernel_cpp_0(c_void_p(arg0_1.data_ptr()), c_void_p(buf0.data_ptr()))
    del arg0_1
    return (buf0, )


if __name__ == "__main__":
    from torch._dynamo.testing import rand_strided
    from torch._inductor.utils import print_performance
    arg0_1 = rand_strided((10000000, ), (1, ), device='cpu', dtype=torch.float32)
    print_performance(lambda: call([arg0_1]))
```

My environment is:
- Torch version: 2.0.0.dev20221231+cpu
- CPU: Intel® Core™ i5-7300HQ CPU @ 2.50GHz × 4 
- OS: Ubuntu 20.04.3 LTS

### Error logs

_No response_

### Minified repro

_No response_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh