Repository: kubernetes
Issue: Scheduler: introduce WASM runtime to scheduler and load scheduler plugins via WASM runtime
Labels: sig/scheduling,kind/feature,lifecycle/frozen,needs-triage

### What would you like to be added?

(Disclaimer: It's just an idea now, will confirm the technical feasibility for this after getting some feedback on this idea)

**summery**: introduce WASM runtime in kubernetes scheduler. 
We can allow users to write WASM plugins and load them in the scheduler much more easily without building their custom schedulers. (Probably we just need to pass a path to the wasm files in CC)
Furthermore, we probably will be able to allow users to write their own plugins in other languages than Golang. 

---

As you may know, istio/envoy allows developers to write Filter plugins in WASM.
https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/
I've always felt it was an interesting idea and wondering if we can implement a similar one on the scheduler.
And, as far as I knew, there was no stable option to introduce WASM runtime in Golang without using CGO (= can be cross-compiled easily.).

But, I just found the one finally; that is tetratelabs/wazero. tetratelabs/wazero is the zero dependencies WebAssembly runtime for Golang and does not require CGO.
https://github.com/tetratelabs/wazero

By using it, I believe we can achieve something like we wanted to see in https://github.com/kubernetes/kubernetes/issues/100723 and https://github.com/kubernetes/kubernetes/issues/106705.

We can define a new interface to communicate with WASM plugins or may be able to use the scheduling framework's interface (e.g., `Filter()` `Score()` etc) as it is.

Seems [trivy](https://github.com/aquasecurity/trivy) is using it to provide a plugin feature which is something like what I wanna propose in this issue. 
> Under the hood Trivy leverages [wazero](https://github.com/tetratelabs/wazero) to run WebAssembly modules without CGO.
https://aquasecurity.github.io/trivy/v0.31.3/docs/advanced/modules/

Also, there has already been a tool to implement a general plugin by tetratelabs/wazero. (which also seems good)
https://github.com/knqyf263/go-plugin

---

Some concerns:
- First of all, is it be allowed to use tetratelabs/wazero? I mean I don't know if there are any rules for dependencies in kubernetes, but this feature will depend _heavily_ on tetratelabs/wazero.
- we probably need tinygo to build a plugin.

### Why is this needed?

https://github.com/kubernetes/kubernetes/issues/100723 describes a lot about the current extensions of the scheduler and its difficulties for users.

This feature will hugely contribute to the scheduler's extendability.
With this feature, users can use their own plugins without building their custom schedulers. That would be amazing.
And furthermore, users may be able to write their own scheduler plugins in languages other than Golang. That would be too amazing!

We tried to support a similar feature by Golang standard [`plugin`](https://pkg.go.dev/plugin) package in https://github.com/kubernetes/kubernetes/issues/100723 and https://github.com/kubernetes/kubernetes/issues/106705

But, as described in the discussion in https://github.com/kubernetes/kubernetes/issues/100723, we've found Golang's plugin feature has so many limitations and probably we need to give up to use it.

/assign
/sig scheduling
/cc @kubernetes/sig-scheduling-leads