Repository: rust
Issue: Tracking issue for WebAssembly atomics
Labels: O-wasm,C-tracking-issue,C-discussion

This is an issue intended to track the state of WebAssembly atomics support in Rust. For the WebAssembly target there is the [threads proposal in WebAssembly](https://github.com/webassembly/threads) which adds a number of instructions and a new kind of memory to the WebAssembly specification. The new instructions largely deal with atomic memory operations (e.g. `i32.atomic.add`), but also deal with synchronization between threads (`memory.atomic.notify`). The threads proposal does *not* add an ability to spawn threads nor does it really define what threads are, but it's largely set up to have a wasm-instance-per-thread (not that this is super relevant for the standard library).

As of the time of this writing the WebAssembly threads proposal is at [stage 2 of the phases process](https://github.com/webassembly/proposals). It is [shipping in Chrome and in Firefox](https://webassembly.org/roadmap/), however.

Rust's support for this proposal boils down to a few things:

* Primarily Rust/LLVM support the `-Ctarget-feature=+atomics` CLI flag to rustc. This causes codegen for atomic types like `std::sync::atomic` [to use the atomic instructions](https://godbolt.org/z/GGz3fq).
* Rust has support for the three synchronization intrinsics:
  * [`memory.atomic.notify`](https://doc.rust-lang.org/stable/core/arch/wasm32/fn.memory_atomic_notify.html)
  * [`memory.atomic.wait32`](https://doc.rust-lang.org/stable/core/arch/wasm32/fn.memory_atomic_wait32.html)
  * [`memory.atomic.wait64`](https://doc.rust-lang.org/stable/core/arch/wasm32/fn.memory_atomic_wait64.html)
* The Rust standard library [implements mutexes differently](https://github.com/rust-lang/rust/blob/c71248b70870960af9993de4f31d3cba9bbce7e8/library/std/src/sys/wasm/mod.rs#L50-L66) based on whether the `atomics` feature is enabled for the library at compile time. Namely it has custom implementations of:
  * [`Condvar`](https://github.com/rust-lang/rust/blob/master/library/std/src/sys/wasm/condvar_atomics.rs)
  * [`Mutex`](https://github.com/rust-lang/rust/blob/master/library/std/src/sys/wasm/mutex_atomics.rs)
  * [`RwLock`](https://github.com/rust-lang/rust/blob/master/library/std/src/sys/wasm/rwlock_atomics.rs)

In terms of toolchain, we're, as usual, inheriting a lot of the experience from LLVM as well. As usual the WebAssembly target uses LLD as the linker, but [a number of options are passed by default](https://github.com/rust-lang/rust/blob/c71248b70870960af9993de4f31d3cba9bbce7e8/compiler/rustc_codegen_ssa/src/back/linker.rs#L1041-L1072) when we're generating an executable compiled with threads (currently detected with `-Ctarget-feature=+atomics`). We instruct LLD to create a "shared" memory (which allows the memory to be shared across multiple wasm instances, how threading works on the web and in other engines), specifies a default maximum size for memory (this is required for shared memory, and normal wasm memories don't need to list a maximum), flags memory as being imported (since otherwise each instance would export a new memory and not share it!), and ensures that a few TLS/initialization-related symbols are exported.

The symbols are perhaps the most interesting part here, so to go into them in some more detail:

* `__wasm_init_memory` - this is called once which initializes all data segments of memory (e.g. copies from `data` into `memory`). This is intended to only happen once for the lifetime of a module at the beginning.
* `__wasm_init_tls` - this is a function which is intended to be called once-per-instance and initializes thread-local information from a static area. The pointer to thread-local data is passed as the first argument. The pointer must be initialized according to `__tls_size` and `__tls_align`.

Also as of today there is no dedicated target for wasm with atomics. The usage of `-Ctarget-feature=+atomics` was intended to help ship this feature ASAP on nightly Rust, but wasn't necessarily intended to be the final form of the feature. This means that if you want to use wasm and atomics you need to use Cargo's `-Zbuild-std` feature to recompiled the standard library.

---

Overall threads, wasm, and Rust I feel are not in a great spot. I'm unfortunately not certain about how best to move things forward. One thing we could do is to simply stabilize everything as-is and call it a day. As can be seen with memory initialization, imports, and TLS, lots of pieces are missing and are quite manual. Additionally `std::thread` has no hope of ever working with this model!

In addition to the drawbacks previously mentioned, there's no way for TLS destructors to get implemented with any of this runtime support. The standard library ignores destructors registered on wasm and simply never runs them. Even if a runtime has a method of running TLS destructors, they don't have a way of hooking into the standard library to run the destructors.

I personally fear that the most likely scenario here is to simply stabilize what we have, bad user experience and all. Other possible alternatives (but not great ones?) might be:

* Add new wasm target for threads, but add a target per "runtime". We might add one for wasm-bindgen, one for Wasmtime, etc. This can try to work around TLS destructor issues and make things much more seamless, but it would be an explosion of targets.
* Coordinate with C and other toolchains to try to create a standard way to deal with wasm threads. For example we could standardize with C how modules are instantiated, TLS is handled, threads are intended to be spawned/exited, etc. This AFAIK isn't happening since I believe "Emscripten does its thing" and I don't think anyone else is trying to get something done in this space. Wasm-bindgen "works" but doesn't implement TLS destructors and it's still very manual and left up to users.

I originally started writing this issue [to stabilize the `core::arch` intrinsics](https://github.com/rust-lang/stdarch/pull/926#issuecomment-704675105), but upon reflection there are so many other unanswered questions in this space I'm no longer certain this is the best course of action. In any case I wanted to write down my thoughts on the current state of things, and hopefully have a canonical place this can be discussed for Rust-related things.