[PT2.0 Feature Proposal] TorchInductor CPU FP32 Inference Optimization
Labels: triaged, oncall: pt2

### 🚀 The feature, motivation and pitch

As part of the PyTorch 2.0 compilation stack, TorchInductor CPU backend optimization brings notable performance improvements via graph compilation over the PyTorch eager mode. The performance improvements on various backends are tracked on the dashboard: https://github.com/pytorch/pytorch/issues/93531
There was also an earlier poster about the work: https://dev-discuss.pytorch.org/t/torchinductor-update-4-cpu-backend-started-to-show-promising-performance-boost/874

### Alternatives

_No response_

### Additional context

- Fusion and Memory Layout Optimization for Conv and GEMM
  - https://github.com/pytorch/pytorch/pull/90890
  - https://github.com/pytorch/pytorch/pull/90264
  - https://github.com/pytorch/pytorch/pull/90265
  - https://github.com/pytorch/pytorch/pull/90266
  - https://github.com/pytorch/pytorch/pull/90267
  - https://github.com/pytorch/pytorch/pull/87050
  - https://github.com/pytorch/pytorch/pull/87063
  - https://github.com/pytorch/pytorch/pull/87064
  - https://github.com/pytorch/pytorch/pull/87065
  - https://github.com/pytorch/pytorch/pull/87066
  - https://github.com/pytorch/pytorch/pull/87435
  - https://github.com/pytorch/pytorch/pull/88048
  - https://github.com/pytorch/pytorch/pull/88403
  - https://github.com/pytorch/pytorch/pull/88412
  - https://github.com/pytorch/pytorch/pull/88414
  - https://github.com/pytorch/pytorch/pull/88794
  - https://github.com/pytorch/pytorch/pull/88864
  - https://github.com/pytorch/pytorch/pull/88988
  - https://github.com/pytorch/pytorch/pull/89109
  - https://github.com/pytorch/pytorch/pull/89209
  - https://github.com/pytorch/pytorch/pull/90259
  - https://github.com/pytorch/pytorch/pull/89746
  - https://github.com/pytorch/pytorch/pull/89278
  - https://github.com/pytorch/pytorch/pull/89265
  - https://github.com/pytorch/pytorch/pull/91433
  - https://github.com/pytorch/pytorch/pull/91953
  - https://github.com/pytorch/pytorch/pull/91954
  - https://github.com/pytorch/pytorch/pull/91955
  - https://github.com/pytorch/pytorch/pull/91956
- Inductor Lowering Enhancement
  - https://github.com/pytorch/pytorch/pull/89904
  - https://github.com/pytorch/pytorch/pull/89838 
  - https://github.com/pytorch/pytorch/pull/91233
  - https://github.com/pytorch/pytorch/pull/91605
  - https://github.com/pytorch/pytorch/pull/90814
- Runtime Optimization
  - https://github.com/pytorch/pytorch/pull/87037
  - https://github.com/pytorch/pytorch/pull/89377
- Vectorization Optimization
  - https://github.com/pytorch/pytorch/pull/90283
  - https://github.com/pytorch/pytorch/pull/91074
  - https://github.com/pytorch/pytorch/pull/91076
  - https://github.com/pytorch/pytorch/pull/90750
  - https://github.com/pytorch/pytorch/pull/87068
  - https://github.com/pytorch/pytorch/pull/87356
  - https://github.com/pytorch/pytorch/pull/88160
  - https://github.com/pytorch/pytorch/pull/88482
  - https://github.com/pytorch/pytorch/pull/88736
  - https://github.com/pytorch/pytorch/pull/88298
  - https://github.com/pytorch/pytorch/pull/90270
  - https://github.com/pytorch/pytorch/pull/89837
  - https://github.com/pytorch/pytorch/pull/91613
  - https://github.com/pytorch/pytorch/pull/91755
  - https://github.com/pytorch/pytorch/pull/91532
- CPP Wrapper
  - https://github.com/pytorch/pytorch/pull/89744
  - https://github.com/pytorch/pytorch/pull/88167
  - https://github.com/pytorch/pytorch/pull/88560
  - https://github.com/pytorch/pytorch/pull/88561
  - https://github.com/pytorch/pytorch/pull/88666
  - https://github.com/pytorch/pytorch/pull/88667
  - https://github.com/pytorch/pytorch/pull/90754
- Profiling and Debuggability
  - https://github.com/pytorch/pytorch/pull/89674
  - https://github.com/pytorch/pytorch/pull/89367
  - https://github.com/pytorch/pytorch/pull/90008
- Benchmarking Enhancement
  - https://github.com/pytorch/pytorch/pull/88477
  - https://github.com/pytorch/pytorch/pull/90641
  - https://github.com/pytorch/pytorch/pull/91870
  - https://github.com/pytorch/pytorch/pull/90150

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @jansel @desertfire @Chillee @penguinwu @jgong5 @fzhao3