Repository: bytecodealliance/wasmtime
Pull request #7475 (2023-11-03 16:14:09)

wasi-http: Remove extra task when reading bodies
This commit is in the same spirit as #7461 and is aimed at removing an extra task when interoperating between WebAssembly and Hyper. The intention here is to ensure that when data is ready immediately a context switch between Tokio tasks isn't necessary. I'm still not sure why this context switch would tank performance so much.

The goal of this commit is to remove the Tokio task associated with reading bodies. Currently a task is spawned with two channels coming out: one for frames and one for trailers. Instead the body is now read directly when data frames are read and trailers are managed similarly. The interactions here are somewhat nontrivial due to the management of resources and how the body needs to jump from the body stream object to the trailers object, but it's overall not the worst thing in the world.

Locally on Linux this takes a "hello world" component which starts out by reading its HTTP body and then responds with a string from 6.7k rps locally via `wasmtime serve` to 16.8k rps. This restores the performance of wasi-http to be in-line with the historical SDK that Fermyon offered, for example, that's based on a custom HTTP interface. Note that these performance numbers use #7461 as a baseline.

This commit has a downside, however, that if the incoming body is not being subscribed to actively by the wasm then no activity will happen on the host. This is a function of `poll` isn't being actively watched unless `subscribe`'s result is being actively watched via a wasm call to `wasi:io/poll`. Manual investigation of Hyper shows that this is probably ok for now since bodies are primarily channel-based which seems like the tasks doing the actual I/O are managed elsewhere. In that sense this shouldn't cause any immediate impact, but Hyper's situation in the future could change or my analysis could additionally be wrong. Given the performance benefits here though I'd be tempted to push in favor of this commit and handle "poll in the background" as necessary in the future if needed.

<!--
Please make sure you include the following information:

- If this work has been discussed elsewhere, please include a link to that
  conversation. If it was discussed in an issue, just mention "issue #...".

- Explain why this change is needed. If the details are in an issue already,
  this can be brief.

Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.html

Please ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->



Patch:
diff --git a/cranelift/codegen/src/isa/aarch64/abi.rs b/cranelift/codegen/src/isa/aarch64/abi.rs
index b5d6e56fc..d1279921d 100644
--- a/cranelift/codegen/src/isa/aarch64/abi.rs
+++ b/cranelift/codegen/src/isa/aarch64/abi.rs
@@ -677,43 +677,9 @@ impl ABIMachineSpec for AArch64MachineDeps {
 
         let probe_count = align_to(frame_size, guard_size) / guard_size;
         if probe_count <= PROBE_MAX_UNROLL {
-            // When manually unrolling stick an instruction that stores 0 at a
-            // constant offset relative to the stack pointer. This will
-            // turn into something like `movn tmp, #n ; stur xzr [sp, tmp]`.
-            //
-            // Note that this may actually store beyond the stack size for the
-            // last item but that's ok since it's unused stack space and if
-            // that faults accidentally we're so close to faulting it shouldn't
-            // make too much difference to fault there.
-            insts.reserve(probe_count as usize);
-            for i in 0..probe_count {
-                let offset = (guard_size * (i + 1)) as i64;
-                insts.push(Self::gen_store_stack(
-                    StackAMode::SPOffset(-offset, I8),
-                    zero_reg(),
-                    I32,
-                ));
-            }
+            Self::gen_probestack_unroll(insts, guard_size, probe_count)
         } else {
-            // The non-unrolled version uses two temporary registers. The
-            // `start` contains the current offset from sp and counts downwards
-            // during the loop by increments of `guard_size`. The `end` is
-            // the size of the frame and where we stop.
-            //
-            // Note that this emission is all post-regalloc so it should be ok
-            // to use the temporary registers here as input/output as the loop
-            // itself is not allowed to use the registers.
-            let start = writable_spilltmp_reg();
-            let end = writable_tmp2_reg();
-            // `gen_inline_probestack` is called after regalloc2, so it's acceptable to reuse
-            // `start` and `end` as temporaries in load_constant.
-            insts.extend(Inst::load_constant(start, 0, &mut |_| start));
-            insts.extend(Inst::load_constant(end, frame_size.into(), &mut |_| end));
-            insts.push(Inst::StackProbeLoop {
-                start,
-                end: end.to_reg(),
-                step: Imm12::maybe_from_u64(guard_size.into()).unwrap(),
-            });
+            Self::gen_probestack_loop(insts, frame_size, guard_size)
         }
     }
 
@@ -1210,6 +1176,52 @@ impl ABIMachineSpec for AArch64MachineDeps {
     }
 }
 
+impl AArch64MachineDeps {
+    fn gen_probestack_unroll(insts: &mut SmallInstVec<Inst>, guard_size: u32, probe_count: u32) {
+        // When manually unrolling adjust the stack pointer and then write a zero
+        // to the stack at that offset. This generates something like
+        // `sub sp, sp, #1, lsl #12` followed by `stur wzr, [sp]`.
+        //
+        // We do this because valgrind expects us to never write beyond the stack
+        // pointer and associated redzone.
+        // See: https://github.com/bytecodealliance/wasmtime/issues/7454
+        for _ in 0..probe_count {
+            insts.extend(Self::gen_sp_reg_adjust(-(guard_size as i32)));
+
+            insts.push(Self::gen_store_stack(
+                StackAMode::SPOffset(0, I8),
+                zero_reg(),
+                I32,
+            ));
+        }
+
+        // Restore the stack pointer to its original value
+        insts.extend(Self::gen_sp_reg_adjust((guard_size * probe_count) as i32));
+    }
+
+    fn gen_probestack_loop(insts: &mut SmallInstVec<Inst>, frame_size: u32, guard_size: u32) {
+        // The non-unrolled version uses two temporary registers. The
+        // `start` contains the current offset from sp and counts downwards
+        // during the loop by increments of `guard_size`. The `end` is
+        // the size of the frame and where we stop.
+        //
+        // Note that this emission is all post-regalloc so it should be ok
+        // to use the temporary registers here as input/output as the loop
+        // itself is not allowed to use the registers.
+        let start = writable_spilltmp_reg();
+        let end = writable_tmp2_reg();
+        // `gen_inline_probestack` is called after regalloc2, so it's acceptable to reuse
+        // `start` and `end` as temporaries in load_constant.
+        insts.extend(Inst::load_constant(start, 0, &mut |_| start));
+        insts.extend(Inst::load_constant(end, frame_size.into(), &mut |_| end));
+        insts.push(Inst::StackProbeLoop {
+            start,
+            end: end.to_reg(),
+            step: Imm12::maybe_from_u64(guard_size.into()).unwrap(),
+        });
+    }
+}
+
 fn select_api_key(
     isa_flags: &aarch64_settings::Flags,
     call_conv: isa::CallConv,
diff --git a/cranelift/codegen/src/isa/riscv64/abi.rs b/cranelift/codegen/src/isa/riscv64/abi.rs
index d119d0f8e..c4e16fcd4 100644
--- a/cranelift/codegen/src/isa/riscv64/abi.rs
+++ b/cranelift/codegen/src/isa/riscv64/abi.rs
@@ -336,8 +336,8 @@ impl ABIMachineSpec for Riscv64MachineDeps {
             insts.push(Inst::AluRRR {
                 alu_op: AluOPRRR::Add,
                 rd: writable_stack_reg(),
-                rs1: tmp.to_reg(),
-                rs2: stack_reg(),
+                rs1: stack_reg(),
+                rs2: tmp.to_reg(),
             });
         }
 
@@ -718,10 +718,20 @@ impl ABIMachineSpec for Riscv64MachineDeps {
         // Number of probes that we need to perform
         let probe_count = align_to(frame_size, guard_size) / guard_size;
 
+        // Must be a caller-saved register that is not an argument.
+        let tmp = match call_conv {
+            isa::CallConv::Tail => Writable::from_reg(x_reg(1)),
+            _ => Writable::from_reg(x_reg(28)), // t3
+        };
+
         if probe_count <= PROBE_MAX_UNROLL {
-            Self::gen_probestack_unroll(insts, guard_size, probe_count)
+            Self::gen_probestack_unroll(insts, tmp, guard_size, probe_count)
         } else {
-            Self::gen_probestack_loop(insts, call_conv, guard_size, probe_count)
+            insts.push(Inst::StackProbeLoop {
+                guard_size,
+                probe_count,
+                tmp,
+            });
         }
     }
 }
@@ -1052,33 +1062,41 @@ fn create_reg_enviroment() -> MachineEnv {
 }
 
 impl Riscv64MachineDeps {
-    fn gen_probestack_unroll(insts: &mut SmallInstVec<Inst>, guard_size: u32, probe_count: u32) {
-        insts.reserve(probe_count as usize);
-        for i in 0..probe_count {
-            let offset = (guard_size * (i + 1)) as i64;
+    fn gen_probestack_unroll(
+        insts: &mut SmallInstVec<Inst>,
+        tmp: Writable<Reg>,
+        guard_size: u32,
+        probe_count: u32,
+    ) {
+        // When manually unrolling adjust the stack pointer and then write a zero
+        // to the stack at that offset.
+        //
+        // We do this because valgrind expects us to never write beyond the stack
+        // pointer and associated redzone.
+        // See: https://github.com/bytecodealliance/wasmtime/issues/7454
+
+        // Store the adjust amount in a register upfront, so we don't have to
+        // reload it for each probe. It's worth loading this as a negative and
+        // using an `add` instruction since we have compressed versions of `add`
+        // but not the `sub` instruction.
+        insts.extend(Inst::load_constant_u64(tmp, (-(guard_size as i64)) as u64));
+
+        for _ in 0..probe_count {
+            insts.push(Inst::AluRRR {
+                alu_op: AluOPRRR::Add,
+                rd: writable_stack_reg(),
+                rs1: stack_reg(),
+                rs2: tmp.to_reg(),
+            });
+
             insts.push(Self::gen_store_stack(
-                StackAMode::SPOffset(-offset, I8),
+                StackAMode::SPOffset(0, I8),
                 zero_reg(),
                 I32,
             ));
         }
-    }
 
-    fn gen_probestack_loop(
-        insts: &mut SmallInstVec<Inst>,
-        call_conv: isa::CallConv,
-        guard_size: u32,
-        probe_count: u32,
-    ) {
-        // Must be a caller-saved register that is not an argument.
-        let tmp = match call_conv {
-            isa::CallConv::Tail => Writable::from_reg(x_reg(1)),
-            _ => Writable::from_reg(x_reg(28)), // t3
-        };
-        insts.push(Inst::StackProbeLoop {
-            guard_size,
-            probe_count,
-            tmp,
-        });
+        // Restore the stack pointer to its original value
+        insts.extend(Self::gen_sp_reg_adjust((guard_size * probe_count) as i32));
     }
 }
diff --git a/cranelift/codegen/src/isa/riscv64/inst/emit.rs b/cranelift/codegen/src/isa/riscv64/inst/emit.rs
index 936037d77..47e5b8a15 100644
--- a/cranelift/codegen/src/isa/riscv64/inst/emit.rs
+++ b/cranelift/codegen/src/isa/riscv64/inst/emit.rs
@@ -345,8 +345,16 @@ impl Inst {
                 rd,
                 rs1,
                 rs2,
-            } if rd.to_reg() == rs1 && rs1 != zero_reg() && rs2 != zero_reg() => {
-                sink.put2(encode_cr_type(CrOp::CAdd, rd, rs2));
+            } if (rd.to_reg() == rs1 || rd.to_reg() == rs2)
+                && rs1 != zero_reg()
+                && rs2 != zero_reg() =>
+            {
+                // Technically `c.add rd, rs` expands to `add rd, rd, rs`, but we can
+                // also swap rs1 with rs2 and we get an equivalent instruction. i.e we
+                // can also compress `add rd, rs, rd` into `c.add rd, rs`.
+                let src = if rd.to_reg() == rs1 { rs2 } else { rs1 };
+
+                sink.put2(encode_cr_type(CrOp::CAdd, rd, src));
             }
 
             // C.MV
diff --git a/cranelift/codegen/src/isa/x64/abi.rs b/cranelift/codegen/src/isa/x64/abi.rs
index d99933500..49c54d529 100644
--- a/cranelift/codegen/src/isa/x64/abi.rs
+++ b/cranelift/codegen/src/isa/x64/abi.rs
@@ -33,17 +33,23 @@ pub struct X64ABIMachineSpec;
 impl X64ABIMachineSpec {
     fn gen_probestack_unroll(insts: &mut SmallInstVec<Inst>, guard_size: u32, probe_count: u32) {
         insts.reserve(probe_count as usize);
-        for i in 0..probe_count {
-            let offset = (guard_size * (i + 1)) as i64;
+        for _ in 0..probe_count {
+            // "Allocate" stack space for the probe by decrementing the stack pointer before
+            // the write. This is required to make valgrind happy.
+            // See: https://github.com/bytecodealliance/wasmtime/issues/7454
+            insts.extend(Self::gen_sp_reg_adjust(-(guard_size as i32)));
 
             // TODO: It would be nice if we could store the imm 0, but we don't have insts for those
             // so store the stack pointer. Any register will do, since the stack is undefined at this point
             insts.push(Self::gen_store_stack(
-                StackAMode::SPOffset(-offset, I8),
+                StackAMode::SPOffset(0, I8),
                 regs::rsp(),
                 I32,
             ));
         }
+
+        // Restore the stack pointer to its original value
+        insts.extend(Self::gen_sp_reg_adjust((guard_size * probe_count) as i32));
     }
 
     fn gen_probestack_loop(
@@ -523,8 +529,8 @@ impl ABIMachineSpec for X64ABIMachineSpec {
         // Unroll at most n consecutive probes, before falling back to using a loop
         //
         // This was number was picked because the loop version is 38 bytes long. We can fit
-        // 5 inline probes in that space, so unroll if its beneficial in terms of code size.
-        const PROBE_MAX_UNROLL: u32 = 5;
+        // 4 inline probes in that space, so unroll if its beneficial in terms of code size.
+        const PROBE_MAX_UNROLL: u32 = 4;
 
         // Number of probes that we need to perform
         let probe_count = align_to(frame_size, guard_size) / guard_size;
diff --git a/cranelift/codegen/src/machinst/vcode.rs b/cranelift/codegen/src/machinst/vcode.rs
index c743fcd20..90bb2a286 100644
--- a/cranelift/codegen/src/machinst/vcode.rs
+++ b/cranelift/codegen/src/machinst/vcode.rs
@@ -1738,6 +1738,15 @@ impl VCodeConstants {
     pub fn get(&self, c: VCodeConstant) -> &VCodeConstantData {
         &self.constants[c]
     }
+
+    /// Checks if the given [VCodeConstantData] is registered as
+    /// used by the pool.
+    pub fn pool_uses(&self, constant: &VCodeConstantData) -> bool {
+        match constant {
+            VCodeConstantData::Pool(c, _) => self.pool_uses.contains_key(c),
+            _ => false,
+        }
+    }
 }
 
 /// A use of a constant by one or more VCode instructions; see [VCodeConstants].
diff --git a/cranelift/codegen/src/opts/shifts.isle b/cranelift/codegen/src/opts/shifts.isle
index acf53946c..272c5c90b 100644
--- a/cranelift/codegen/src/opts/shifts.isle
+++ b/cranelift/codegen/src/opts/shifts.isle
@@ -268,3 +268,43 @@
       (rotl ty x (isub_uextend y z)))
 (rule (simplify (rotl ty (rotr ty x y @ (value_type kty)) z @ (value_type kty)))
       (rotr ty x (isub_uextend y z)))
+
+;; Convert shifts into rotates. We always normalize into a rotate left.
+;;
+;; (bor (ishl x k1) (ushr x k2)) == (rotl x k1) if k2 == ty_bits - k1
+;; (bor (ushr x k2) (ishl x k1)) == (rotl x k1) if k2 == ty_bits - k1
+;;
+;; TODO: This rule is restricted to scalars since no backend currently
+;; supports SIMD rotates.
+(rule (simplify (bor (ty_int ty)
+                      (ishl ty x k @ (iconst _ (u64_from_imm64 k1)))
+                      (ushr ty x (iconst _ (u64_from_imm64 k2)))))
+      (if-let $true (u64_eq k2 (u64_sub (ty_bits_u64 (lane_type ty)) k1)))
+      (rotl ty x k))
+(rule (simplify (bor (ty_int ty)
+                      (ushr ty x (iconst _ (u64_from_imm64 k2)))
+                      (ishl ty x k @ (iconst _ (u64_from_imm64 k1)))))
+      (if-let $true (u64_eq k2 (u64_sub (ty_bits_u64 (lane_type ty)) k1)))
+      (rotl ty x k))
+
+;; Normalize the shift amount. Some rules can't fire unless the shift amount
+;; is normalized. This also helps us materialize fewer and smaller constants.
+;;
+;; (op x k) == (op x (and k (ty_shift_mask ty)))
+;;
+;; where `op` is one of ishl, ushr, sshr, rotl, rotr
+(rule (simplify (ishl ty x (iconst kty (u64_from_imm64 k))))
+      (if-let $false (u64_eq k (u64_and k (ty_shift_mask ty))))
+      (ishl ty x (iconst kty (imm64 (u64_and k (ty_shift_mask ty))))))
+(rule (simplify (ushr ty x (iconst kty (u64_from_imm64 k))))
+      (if-let $false (u64_eq k (u64_and k (ty_shift_mask ty))))
+      (ushr ty x (iconst kty (imm64 (u64_and k (ty_shift_mask ty))))))
+(rule (simplify (sshr ty x (iconst kty (u64_from_imm64 k))))
+      (if-let $false (u64_eq k (u64_and k (ty_shift_mask ty))))
+      (sshr ty x (iconst kty (imm64 (u64_and k (ty_shift_mask ty))))))
+(rule (simplify (rotr ty x (iconst kty (u64_from_imm64 k))))
+      (if-let $false (u64_eq k (u64_and k (ty_shift_mask ty))))
+      (rotr ty x (iconst kty (imm64 (u64_and k (ty_shift_mask ty))))))
+(rule (simplify (rotl ty x (iconst kty (u64_from_imm64 k))))
+      (if-let $false (u64_eq k (u64_and k (ty_shift_mask ty))))
+      (rotl ty x (iconst kty (imm64 (u64_and k (ty_shift_mask ty))))))
diff --git a/crates/c-api/include/wasmtime.h b/crates/c-api/include/wasmtime.h
index 44e4e3d2c..ce7d46e0f 100644
--- a/crates/c-api/include/wasmtime.h
+++ b/crates/c-api/include/wasmtime.h
@@ -181,6 +181,7 @@
 #define WASMTIME_API_H
 
 #include <wasi.h>
+// IWYU pragma: begin_exports
 #include <wasmtime/config.h>
 #include <wasmtime/engine.h>
 #include <wasmtime/error.h>
@@ -196,6 +197,7 @@
 #include <wasmtime/trap.h>
 #include <wasmtime/val.h>
 #include <wasmtime/async.h>
+// IWYU pragma: end_exports
 
 /**
  * \brief Wasmtime version string.
diff --git a/crates/wasi-http/src/body.rs b/crates/wasi-http/src/body.rs
index 25d29a539..4456fe24c 100644
--- a/crates/wasi-http/src/body.rs
+++ b/crates/wasi-http/src/body.rs
@@ -1,8 +1,12 @@
 use crate::{bindings::http::types, types::FieldMap};
 use anyhow::anyhow;
 use bytes::Bytes;
+use http_body::{Body, Frame};
 use http_body_util::combinators::BoxBody;
+use http_body_util::BodyExt;
 use std::future::Future;
+use std::mem;
+use std::task::{Context, Poll};
 use std::{
     pin::Pin,
     sync::{Arc, Mutex},
@@ -10,186 +14,198 @@ use std::{
 };
 use tokio::sync::{mpsc, oneshot};
 use wasmtime_wasi::preview2::{
-    self, AbortOnDropJoinHandle, HostInputStream, HostOutputStream, StreamError, Subscribe,
+    self, poll_noop, AbortOnDropJoinHandle, HostInputStream, HostOutputStream, StreamError,
+    Subscribe,
 };
 
 pub type HyperIncomingBody = BoxBody<Bytes, types::Error>;
 
-/// Holds onto the things needed to construct a [`HostIncomingBody`] until we are ready to build
-/// one. The HostIncomingBody spawns a task that starts consuming the incoming body, and we don't
-/// want to do that unless the user asks to consume the body.
-pub struct HostIncomingBodyBuilder {
-    pub body: HyperIncomingBody,
-    pub worker: Option<Arc<AbortOnDropJoinHandle<Result<(), types::Error>>>>,
-    pub between_bytes_timeout: Duration,
+/// Small wrapper around `BoxBody` which adds a timeout to every frame.
+struct BodyWithTimeout {
+    /// Underlying stream that frames are coming from.
+    inner: HyperIncomingBody,
+    /// Currently active timeout that's reset between frames.
+    timeout: Pin<Box<tokio::time::Sleep>>,
+    /// Whether or not `timeout` needs to be reset on the next call to
+    /// `poll_frame`.
+    reset_sleep: bool,
+    /// Maximal duration between when a frame is first requested and when it's
+    /// allowed to arrive.
+    between_bytes_timeout: Duration,
 }
 
-impl HostIncomingBodyBuilder {
-    /// Consume the state held in the [`HostIncomingBodyBuilder`] to spawn a task that will drive the
-    /// streaming body to completion. Data segments will be communicated out over the
-    /// [`HostIncomingBodyStream`], and a [`HostFutureTrailers`] gives a way to block on/retrieve
-    /// the trailers.
-    pub fn build(mut self) -> HostIncomingBody {
-        let (body_writer, body_receiver) = mpsc::channel(1);
-        let (trailer_writer, trailers) = oneshot::channel();
-
-        let worker = preview2::spawn(async move {
-            loop {
-                let frame = match tokio::time::timeout(
-                    self.between_bytes_timeout,
-                    http_body_util::BodyExt::frame(&mut self.body),
-                )
-                .await
-                {
-                    Ok(None) => break,
-
-                    Ok(Some(Ok(frame))) => frame,
-
-                    Ok(Some(Err(e))) => {
-                        match body_writer.send(Err(e)).await {
-                            Ok(_) => {}
-                            // If the body read end has dropped, then we report this error with the
-                            // trailers. unwrap and rewrap Err because the Ok side of these two Results
-                            // are different.
-                            Err(e) => {
-                                let _ = trailer_writer.send(Err(e.0.unwrap_err()));
-                            }
-                        }
-                        break;
-                    }
-
-                    Err(_) => {
-                        match body_writer
-                            .send(Err(types::Error::TimeoutError(
-                                "data frame timed out".to_string(),
-                            )
-                            .into()))
-                            .await
-                        {
-                            Ok(_) => {}
-                            Err(e) => {
-                                let _ = trailer_writer.send(Err(e.0.unwrap_err()));
-                            }
-                        }
-                        break;
-                    }
-                };
+impl BodyWithTimeout {
+    fn new(inner: HyperIncomingBody, between_bytes_timeout: Duration) -> BodyWithTimeout {
+        BodyWithTimeout {
+            inner,
+            between_bytes_timeout,
+            reset_sleep: true,
+            timeout: Box::pin(preview2::with_ambient_tokio_runtime(|| {
+                tokio::time::sleep(Duration::new(0, 0))
+            })),
+        }
+    }
+}
 
-                if frame.is_trailers() {
-                    // We know we're not going to write any more data frames at this point, so we
-                    // explicitly drop the body_writer so that anything waiting on the read end returns
-                    // immediately.
-                    drop(body_writer);
+impl Body for BodyWithTimeout {
+    type Data = Bytes;
+    type Error = types::Error;
+
+    fn poll_frame(
+        self: Pin<&mut Self>,
+        cx: &mut Context<'_>,
+    ) -> Poll<Option<Result<Frame<Bytes>, types::Error>>> {
+        let me = Pin::into_inner(self);
+
+        // If the timeout timer needs to be reset, do that now relative to the
+        // current instant. Otherwise test the timeout timer and see if it's
+        // fired yet and if so we've timed out and return an error.
+        if me.reset_sleep {
+            me.timeout
+                .as_mut()
+                .reset(tokio::time::Instant::now() + me.between_bytes_timeout);
+            me.reset_sleep = false;
+        }
 
-                    let trailers = frame.into_trailers().unwrap();
+        // Register interest in this context on the sleep timer, and if the
+        // sleep elapsed that means that we've timed out.
+        if let Poll::Ready(()) = me.timeout.as_mut().poll(cx) {
+            return Poll::Ready(Some(Err(types::Error::TimeoutError(
+                "frame timed out".to_string(),
+            ))));
+        }
 
-                    // TODO: this will fail in two cases:
-                    // 1. we've already used the channel once, which should be imposible,
-                    // 2. the read end is closed.
-                    // I'm not sure how to differentiate between these two cases, or really
-                    // if we need to do anything to handle either.
-                    let _ = trailer_writer.send(Ok(trailers));
+        // Without timeout business now handled check for the frame. If a frame
+        // arrives then the sleep timer will be reset on the next frame.
+        let result = Pin::new(&mut me.inner).poll_frame(cx);
+        me.reset_sleep = result.is_ready();
+        result
+    }
+}
 
-                    break;
-                }
+pub struct HostIncomingBody {
+    body: IncomingBodyState,
+    /// An optional worker task to keep alive while this body is being read.
+    /// This ensures that if the parent of this body is dropped before the body
+    /// then the backing data behind this worker is kept alive.
+    worker: Option<Arc<AbortOnDropJoinHandle<()>>>,
+}
 
-                assert!(frame.is_data(), "frame wasn't data");
+enum IncomingBodyState {
+    /// The body is stored here meaning that within `HostIncomingBody` the
+    /// `take_stream` method can be called for example.
+    Start(BodyWithTimeout),
 
-                let data = frame.into_data().unwrap();
+    /// The body is within a `HostIncomingBodyStream` meaning that it's not
+    /// currently owned here. The body will be sent back over this channel when
+    /// it's done, however.
+    InBodyStream(oneshot::Receiver<StreamEnd>),
+}
 
-                // If the receiver no longer exists, thats ok - in that case we want to keep the
-                // loop running to relieve backpressure, so we get to the trailers.
-                let _ = body_writer.send(Ok(data)).await;
-            }
+/// Message sent when a `HostIncomingBodyStream` is done to the
+/// `HostFutureTrailers` state.
+enum StreamEnd {
+    /// The body wasn't completely read and was dropped early. May still have
+    /// trailers, but requires reading more frames.
+    Remaining(BodyWithTimeout),
 
-            // Force the builder's worker to be owned by this task, ensuring that it's kept around
-            // until this task exits.
-            drop(self.worker);
-        });
+    /// Body was completely read and trailers were read. Here are the trailers.
+    /// Note that `None` means that the body finished without trailers.
+    Trailers(Option<FieldMap>),
+}
 
+impl HostIncomingBody {
+    pub fn new(body: HyperIncomingBody, between_bytes_timeout: Duration) -> HostIncomingBody {
+        let body = BodyWithTimeout::new(body, between_bytes_timeout);
         HostIncomingBody {
-            worker,
-            stream: Some(HostIncomingBodyStream::new(body_receiver)),
-            trailers,
+            body: IncomingBodyState::Start(body),
+            worker: None,
         }
     }
-}
 
-pub struct HostIncomingBody {
-    pub worker: AbortOnDropJoinHandle<()>,
-    pub stream: Option<HostIncomingBodyStream>,
-    pub trailers: oneshot::Receiver<Result<hyper::HeaderMap, types::Error>>,
-}
+    pub fn retain_worker(&mut self, worker: &Arc<AbortOnDropJoinHandle<()>>) {
+        assert!(self.worker.is_none());
+        self.worker = Some(worker.clone());
+    }
 
-impl HostIncomingBody {
-    pub fn into_future_trailers(self) -> HostFutureTrailers {
-        HostFutureTrailers {
-            _worker: self.worker,
-            state: HostFutureTrailersState::Waiting(self.trailers),
+    pub fn take_stream(&mut self) -> Option<HostIncomingBodyStream> {
+        match &mut self.body {
+            IncomingBodyState::Start(_) => {}
+            IncomingBodyState::InBodyStream(_) => return None,
         }
+        let (tx, rx) = oneshot::channel();
+        let body = match mem::replace(&mut self.body, IncomingBodyState::InBodyStream(rx)) {
+            IncomingBodyState::Start(b) => b,
+            IncomingBodyState::InBodyStream(_) => unreachable!(),
+        };
+        Some(HostIncomingBodyStream {
+            state: IncomingBodyStreamState::Open { body, tx },
+            buffer: Bytes::new(),
+            error: None,
+        })
+    }
+
+    pub fn into_future_trailers(self) -> HostFutureTrailers {
+        HostFutureTrailers::Waiting(self)
     }
 }
 
 pub struct HostIncomingBodyStream {
-    pub open: bool,
-    pub receiver: mpsc::Receiver<Result<Bytes, types::Error>>,
-    pub buffer: Bytes,
-    pub error: Option<anyhow::Error>,
+    state: IncomingBodyStreamState,
+    buffer: Bytes,
+    error: Option<anyhow::Error>,
 }
 
-impl HostIncomingBodyStream {
-    fn new(receiver: mpsc::Receiver<Result<Bytes, types::Error>>) -> Self {
-        Self {
-            open: true,
-            receiver,
-            buffer: Bytes::new(),
-            error: None,
-        }
-    }
+enum IncomingBodyStreamState {
+    /// The body is currently open for reading and present here.
+    ///
+    /// When trailers are read, or when this is dropped, the body is sent along
+    /// `tx`.
+    ///
+    /// This state is transitioned to `Closed` when an error happens, EOF
+    /// happens, or when trailers are read.
+    Open {
+        body: BodyWithTimeout,
+        tx: oneshot::Sender<StreamEnd>,
+    },
+
+    /// This body is closed and no longer available for reading, no more data
+    /// will come.
+    Closed,
 }
 
 #[async_trait::async_trait]
 impl HostInputStream for HostIncomingBodyStream {
     fn read(&mut self, size: usize) -> Result<Bytes, StreamError> {
-        use mpsc::error::TryRecvError;
-
-        if !self.buffer.is_empty() {
-            let len = size.min(self.buffer.len());
-            let chunk = self.buffer.split_to(len);
-            return Ok(chunk);
-        }
-
-        if let Some(e) = self.error.take() {
-            return Err(StreamError::LastOperationFailed(e));
-        }
-
-        if !self.open {
-            return Err(StreamError::Closed);
-        }
-
-        match self.receiver.try_recv() {
-            Ok(Ok(mut bytes)) => {
-                let len = bytes.len().min(size);
-                let chunk = bytes.split_to(len);
-                if !bytes.is_empty() {
-                    self.buffer = bytes;
-                }
-
+        loop {
+            // Handle buffered data/errors if any
+            if !self.buffer.is_empty() {
+                let len = size.min(self.buffer.len());
+                let chunk = self.buffer.split_to(len);
                 return Ok(chunk);
             }
 
-            Ok(Err(e)) => {
-                self.open = false;
-                return Err(StreamError::LastOperationFailed(e.into()));
+            if let Some(e) = self.error.take() {
+                return Err(StreamError::LastOperationFailed(e));
             }
 
-            Err(TryRecvError::Empty) => {
-                return Ok(Bytes::new());
-            }
+            // Extract the body that we're reading from. If present perform a
+            // non-blocking poll to see if a frame is already here. If it is
+            // then turn the loop again to operate on the results. If it's not
+            // here then return an empty buffer as no data is available at this
+            // time.
+            let body = match &mut self.state {
+                IncomingBodyStreamState::Open { body, .. } => body,
+                IncomingBodyStreamState::Closed => return Err(StreamError::Closed),
+            };
 
-            Err(TryRecvError::Disconnected) => {
-                self.open = false;
-                return Err(StreamError::Closed);
+            let future = body.frame();
+            futures::pin_mut!(future);
+            match poll_noop(future) {
+                Some(result) => {
+                    self.record_frame(result);
+                }
+                None => return Ok(Bytes::new()),
             }
         }
     }
@@ -198,56 +214,155 @@ impl HostInputStream for HostIncomingBodyStream {
 #[async_trait::async_trait]
 impl Subscribe for HostIncomingBodyStream {
     async fn ready(&mut self) {
-        if !self.buffer.is_empty() {
+        if !self.buffer.is_empty() || self.error.is_some() {
             return;
         }
 
-        if !self.open {
-            return;
+        if let IncomingBodyStreamState::Open { body, .. } = &mut self.state {
+            let frame = body.frame().await;
+            self.record_frame(frame);
         }
+    }
+}
+
+impl HostIncomingBodyStream {
+    fn record_frame(&mut self, frame: Option<Result<Frame<Bytes>, types::Error>>) {
+        match frame {
+            Some(Ok(frame)) => match frame.into_data() {
+                // A data frame was received, so queue up the buffered data for
+                // the next `read` call.
+                Ok(bytes) => {
+                    assert!(self.buffer.is_empty());
+                    self.buffer = bytes;
+                }
 
-        match self.receiver.recv().await {
-            Some(Ok(bytes)) => self.buffer = bytes,
+                // Trailers were received meaning that this was the final frame.
+                // Throw away the body and send the trailers along the
+                // `tx` channel to make them available.
+                Err(trailers) => {
+                    let trailers = trailers.into_trailers().unwrap();
+                    let tx = match mem::replace(&mut self.state, IncomingBodyStreamState::Closed) {
+                        IncomingBodyStreamState::Open { body: _, tx } => tx,
+                        IncomingBodyStreamState::Closed => unreachable!(),
+                    };
+
+                    // NB: ignore send failures here because if this fails then
+                    // no one was interested in the trailers.
+                    let _ = tx.send(StreamEnd::Trailers(Some(trailers)));
+                }
+            },
 
+            // An error was received meaning that the stream is now done.
+            // Destroy the body to terminate the stream while enqueueing the
+            // error to get returned from the next call to `read`.
             Some(Err(e)) => {
                 self.error = Some(e.into());
-                self.open = false;
+                self.state = IncomingBodyStreamState::Closed;
             }
 
-            None => self.open = false,
+            // No more frames are going to be received again, so drop the `body`
+            // and the `tx` channel we'd send the body back onto because it's
+            // not needed as frames are done.
+            None => {
+                self.state = IncomingBodyStreamState::Closed;
+            }
         }
     }
 }
 
-pub struct HostFutureTrailers {
-    _worker: AbortOnDropJoinHandle<()>,
-    pub state: HostFutureTrailersState,
+impl Drop for HostIncomingBodyStream {
+    fn drop(&mut self) {
+        // When a body stream is dropped, for whatever reason, attempt to send
+        // the body back to the `tx` which will provide the trailers if desired.
+        // This isn't necessary if the state is already closed. Additionally,
+        // like `record_frame` above, `send` errors are ignored as they indicate
+        // that the body/trailers aren't actually needed.
+        let prev = mem::replace(&mut self.state, IncomingBodyStreamState::Closed);
+        if let IncomingBodyStreamState::Open { body, tx } = prev {
+            let _ = tx.send(StreamEnd::Remaining(body));
+        }
+    }
 }
 
-pub enum HostFutureTrailersState {
-    Waiting(oneshot::Receiver<Result<hyper::HeaderMap, types::Error>>),
+pub enum HostFutureTrailers {
+    /// Trailers aren't here yet.
+    ///
+    /// This state represents two similar states:
+    ///
+    /// * The body is here and ready for reading and we're waiting to read
+    ///   trailers. This can happen for example when the actual body wasn't read
+    ///   or if the body was only partially read.
+    ///
+    /// * The body is being read by something else and we're waiting for that to
+    ///   send us the trailers (or the body itself). This state will get entered
+    ///   when the body stream is dropped for example. If the body stream reads
+    ///   the trailers itself it will also send a message over here with the
+    ///   trailers.
+    Waiting(HostIncomingBody),
+
+    /// Trailers are ready and here they are.
+    ///
+    /// Note that `Ok(None)` means that there were no trailers for this request
+    /// while `Ok(Some(_))` means that trailers were found in the request.
     Done(Result<Option<FieldMap>, types::Error>),
 }
 
 #[async_trait::async_trait]
 impl Subscribe for HostFutureTrailers {
     async fn ready(&mut self) {
-        if let HostFutureTrailersState::Waiting(rx) = &mut self.state {
-            let result = match rx.await {
-                Ok(Ok(headers)) => {
-                    if headers.is_empty() {
-                        Ok(None)
-                    } else {
-                        Ok(Some(headers))
-                    }
+        let body = match self {
+            HostFutureTrailers::Waiting(body) => body,
+            HostFutureTrailers::Done(_) => return,
+        };
+
+        // If the body is itself being read by a body stream then we need to
+        // wait for that to be done.
+        if let IncomingBodyState::InBodyStream(rx) = &mut body.body {
+            match rx.await {
+                // Trailers were read for us and here they are, so store the
+                // result.
+                Ok(StreamEnd::Trailers(t)) => *self = Self::Done(Ok(t)),
+
+                // The body wasn't fully read and was dropped before trailers
+                // were reached. It's up to us now to complete the body.
+                Ok(StreamEnd::Remaining(b)) => body.body = IncomingBodyState::Start(b),
+
+                // Technically this shouldn't be possible as the sender
+                // shouldn't get destroyed without receiving a message. Handle
+                // this just in case though.
+                Err(_) => {
+                    debug_assert!(false, "should be unreachable");
+                    *self = HostFutureTrailers::Done(Err(types::Error::ProtocolError(
+                        "stream hung up before trailers were received".to_string(),
+                    )));
                 }
-                Ok(Err(e)) => Err(types::Error::ProtocolError(format!("hyper error: {e:?}"))),
-                Err(_) => Err(types::Error::ProtocolError(
-                    "stream hung up before trailers were received".to_string(),
-                )),
-            };
-            self.state = HostFutureTrailersState::Done(result);
+            }
         }
+
+        // Here it should be guaranteed that `InBodyStream` is now gone, so if
+        // we have the body ourselves then read frames until trailers are found.
+        let body = match self {
+            HostFutureTrailers::Waiting(body) => body,
+            HostFutureTrailers::Done(_) => return,
+        };
+        let hyper_body = match &mut body.body {
+            IncomingBodyState::Start(body) => body,
+            IncomingBodyState::InBodyStream(_) => unreachable!(),
+        };
+        let result = loop {
+            match hyper_body.frame().await {
+                None => break Ok(None),
+                Some(Err(e)) => break Err(e),
+                Some(Ok(frame)) => {
+                    // If this frame is a data frame ignore it as we're only
+                    // interested in trailers.
+                    if let Ok(headers) = frame.into_trailers() {
+                        break Ok(Some(headers));
+                    }
+                }
+            }
+        };
+        *self = HostFutureTrailers::Done(result);
     }
 }
 
@@ -266,9 +381,6 @@ pub struct HostOutgoingBody {
 
 impl HostOutgoingBody {
     pub fn new() -> (Self, HyperOutgoingBody) {
-        use http_body_util::BodyExt;
-        use hyper::body::{Body, Frame};
-        use std::task::{Context, Poll};
         use tokio::sync::oneshot::error::RecvError;
         struct BodyImpl {
             body_receiver: mpsc::Receiver<Bytes>,
diff --git a/crates/wasi-http/src/types.rs b/crates/wasi-http/src/types.rs
index e962ff47c..c7041fcad 100644
--- a/crates/wasi-http/src/types.rs
+++ b/crates/wasi-http/src/types.rs
@@ -3,7 +3,7 @@
 
 use crate::{
     bindings::http::types::{self, Method, Scheme},
-    body::{HostIncomingBodyBuilder, HyperIncomingBody, HyperOutgoingBody},
+    body::{HostIncomingBody, HyperIncomingBody, HyperOutgoingBody},
 };
 use http_body_util::BodyExt;
 use hyper::header::HeaderName;
@@ -36,12 +36,11 @@ pub trait WasiHttpView: Send {
         req: hyper::Request<HyperIncomingBody>,
     ) -> wasmtime::Result<Resource<HostIncomingRequest>> {
         let (parts, body) = req.into_parts();
-        let body = HostIncomingBodyBuilder {
+        let body = HostIncomingBody::new(
             body,
-            worker: None,
             // TODO: this needs to be plumbed through
-            between_bytes_timeout: std::time::Duration::from_millis(600 * 1000),
-        };
+            std::time::Duration::from_millis(600 * 1000),
+        );
         Ok(self.table().push(HostIncomingRequest {
             parts,
             body: Some(body),
@@ -156,8 +155,12 @@ async fn handler(
             .map_err(|_| timeout_error("connection"))??;
 
             let worker = preview2::spawn(async move {
-                let _ = conn.await;
-                Ok::<_, types::Error>(())
+                match conn.await {
+                    Ok(()) => {}
+                    // TODO: shouldn't throw away this error and ideally should
+                    // surface somewhere.
+                    Err(e) => tracing::warn!("dropping error {e}"),
+                }
             });
 
             (sender, worker)
@@ -172,8 +175,11 @@ async fn handler(
         .map_err(|_| timeout_error("connection"))??;
 
         let worker = preview2::spawn(async move {
-            conn.await?;
-            Ok::<_, types::Error>(())
+            match conn.await {
+                Ok(()) => {}
+                // TODO: same as above, shouldn't throw this error away.
+                Err(e) => tracing::warn!("dropping error {e}"),
+            }
         });
 
         (sender, worker)
@@ -257,7 +263,7 @@ impl TryInto<http::Method> for types::Method {
 
 pub struct HostIncomingRequest {
     pub parts: http::request::Parts,
-    pub body: Option<HostIncomingBodyBuilder>,
+    pub body: Option<HostIncomingBody>,
 }
 
 pub struct HostResponseOutparam {
@@ -284,8 +290,8 @@ pub struct HostRequestOptions {
 pub struct HostIncomingResponse {
     pub status: u16,
     pub headers: FieldMap,
-    pub body: Option<HostIncomingBodyBuilder>,
-    pub worker: Arc<AbortOnDropJoinHandle<Result<(), types::Error>>>,
+    pub body: Option<HostIncomingBody>,
+    pub worker: Arc<AbortOnDropJoinHandle<()>>,
 }
 
 pub struct HostOutgoingResponse {
@@ -336,7 +342,7 @@ pub enum HostFields {
 
 pub struct IncomingResponseInternal {
     pub resp: hyper::Response<HyperIncomingBody>,
-    pub worker: Arc<AbortOnDropJoinHandle<Result<(), types::Error>>>,
+    pub worker: Arc<AbortOnDropJoinHandle<()>>,
     pub between_bytes_timeout: std::time::Duration,
 }
 
diff --git a/crates/wasi-http/src/types_impl.rs b/crates/wasi-http/src/types_impl.rs
index bce0384ab..266fd4c36 100644
--- a/crates/wasi-http/src/types_impl.rs
+++ b/crates/wasi-http/src/types_impl.rs
@@ -1,11 +1,11 @@
 use crate::bindings::http::types::{
     self, Error, HeaderError, Headers, Method, Scheme, StatusCode, Trailers,
 };
-use crate::body::{FinishMessage, HostFutureTrailers, HostFutureTrailersState};
+use crate::body::{FinishMessage, HostFutureTrailers};
 use crate::types::{HostIncomingRequest, HostOutgoingResponse};
 use crate::WasiHttpView;
 use crate::{
-    body::{HostIncomingBody, HostIncomingBodyBuilder, HostOutgoingBody},
+    body::{HostIncomingBody, HostOutgoingBody},
     types::{
         FieldMap, HostFields, HostFutureIncomingResponse, HostIncomingResponse,
         HostOutgoingRequest, HostResponseOutparam,
@@ -13,7 +13,7 @@ use crate::{
 };
 use anyhow::Context;
 use hyper::header::HeaderName;
-use std::{any::Any, sync::Arc};
+use std::any::Any;
 use wasmtime::component::Resource;
 use wasmtime_wasi::preview2::{
     bindings::io::streams::{InputStream, OutputStream},
@@ -307,8 +307,8 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostIncomingRequest for T {
     ) -> wasmtime::Result<Result<Resource<HostIncomingBody>, ()>> {
         let req = self.table().get_mut(&id)?;
         match req.body.take() {
-            Some(builder) => {
-                let id = self.table().push(builder.build())?;
+            Some(body) => {
+                let id = self.table().push(body)?;
                 Ok(Ok(id))
             }
 
@@ -540,8 +540,8 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostIncomingResponse for T {
             .context("[incoming_response_consume] getting response")?;
 
         match r.body.take() {
-            Some(builder) => {
-                let id = self.table().push(builder.build())?;
+            Some(body) => {
+                let id = self.table().push(body)?;
                 Ok(Ok(id))
             }
 
@@ -571,17 +571,17 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostFutureTrailers for T {
         id: Resource<HostFutureTrailers>,
     ) -> wasmtime::Result<Option<Result<Option<Resource<Trailers>>, Error>>> {
         let trailers = self.table().get_mut(&id)?;
-        match &trailers.state {
-            HostFutureTrailersState::Waiting(_) => return Ok(None),
-            HostFutureTrailersState::Done(Err(e)) => return Ok(Some(Err(e.clone()))),
-            HostFutureTrailersState::Done(Ok(None)) => return Ok(Some(Ok(None))),
-            HostFutureTrailersState::Done(Ok(Some(_))) => {}
+        match trailers {
+            HostFutureTrailers::Waiting(_) => return Ok(None),
+            HostFutureTrailers::Done(Err(e)) => return Ok(Some(Err(e.clone()))),
+            HostFutureTrailers::Done(Ok(None)) => return Ok(Some(Ok(None))),
+            HostFutureTrailers::Done(Ok(_)) => {}
         }
 
         fn get_fields(elem: &mut dyn Any) -> &mut FieldMap {
             let trailers = elem.downcast_mut::<HostFutureTrailers>().unwrap();
-            match &mut trailers.state {
-                HostFutureTrailersState::Done(Ok(Some(e))) => e,
+            match trailers {
+                HostFutureTrailers::Done(Ok(Some(e))) => e,
                 _ => unreachable!(),
             }
         }
@@ -605,7 +605,7 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostIncomingBody for T {
     ) -> wasmtime::Result<Result<Resource<InputStream>, ()>> {
         let body = self.table().get_mut(&id)?;
 
-        if let Some(stream) = body.stream.take() {
+        if let Some(stream) = body.take_stream() {
             let stream = InputStream::Host(Box::new(stream));
             let stream = self.table().push_child(stream, &id)?;
             return Ok(Ok(stream));
@@ -752,10 +752,10 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostFutureIncomingResponse f
         let resp = self.table().push(HostIncomingResponse {
             status: parts.status.as_u16(),
             headers: FieldMap::from(parts.headers),
-            body: Some(HostIncomingBodyBuilder {
-                body,
-                worker: Some(Arc::clone(&resp.worker)),
-                between_bytes_timeout: resp.between_bytes_timeout,
+            body: Some({
+                let mut body = HostIncomingBody::new(body, resp.between_bytes_timeout);
+                body.retain_worker(&resp.worker);
+                body
             }),
             worker: resp.worker,
         })?;
diff --git a/crates/wasi/src/preview2/mod.rs b/crates/wasi/src/preview2/mod.rs
index bcb116527..957a74a37 100644
--- a/crates/wasi/src/preview2/mod.rs
+++ b/crates/wasi/src/preview2/mod.rs
@@ -291,7 +291,13 @@ pub fn in_tokio<F: Future>(f: F) -> F::Output {
     }
 }
 
-fn with_ambient_tokio_runtime<R>(f: impl FnOnce() -> R) -> R {
+/// Executes the closure `f` with an "ambient Tokio runtime" which basically
+/// means that if code in `f` tries to get a runtime `Handle` it'll succeed.
+///
+/// If a `Handle` is already available, e.g. in async contexts, then `f` is run
+/// immediately. Otherwise for synchronous contexts this crate's fallback
+/// runtime is configured and then `f` is executed.
+pub fn with_ambient_tokio_runtime<R>(f: impl FnOnce() -> R) -> R {
     match tokio::runtime::Handle::try_current() {
         Ok(_) => f(),
         Err(_) => {
@@ -301,7 +307,14 @@ fn with_ambient_tokio_runtime<R>(f: impl FnOnce() -> R) -> R {
     }
 }
 
-fn poll_noop<F>(future: Pin<&mut F>) -> Option<F::Output>
+/// Attempts to get the result of a `future`.
+///
+/// This function does not block and will poll the provided future once. If the
+/// result is here then `Some` is returned, otherwise `None` is returned.
+///
+/// Note that by polling `future` this means that `future` must be re-polled
+/// later if it's to wake up a task.
+pub fn poll_noop<F>(future: Pin<&mut F>) -> Option<F::Output>
 where
     F: Future,
 {
diff --git a/crates/wasmtime/src/compiler.rs b/crates/wasmtime/src/compiler.rs
index 33856f4b2..5022ef9da 100644
--- a/crates/wasmtime/src/compiler.rs
+++ b/crates/wasmtime/src/compiler.rs
@@ -28,7 +28,7 @@ use std::collections::{btree_map, BTreeMap, BTreeSet};
 use std::{any::Any, collections::HashMap};
 use wasmtime_environ::{
     Compiler, DefinedFuncIndex, FuncIndex, FunctionBodyData, ModuleTranslation, ModuleType,
-    ModuleTypes, PrimaryMap, SignatureIndex, StaticModuleIndex, Tunables, WasmFunctionInfo,
+    ModuleTypes, PrimaryMap, SignatureIndex, StaticModuleIndex, WasmFunctionInfo,
 };
 use wasmtime_jit::{CompiledFunctionInfo, CompiledModuleInfo};
 
@@ -447,8 +447,7 @@ impl FunctionIndices {
     pub fn link_and_append_code<'a>(
         mut self,
         mut obj: object::write::Object<'static>,
-        tunables: &'a Tunables,
-        compiler: &dyn Compiler,
+        engine: &'a Engine,
         compiled_funcs: Vec<(String, Box<dyn Any + Send>)>,
         translations: PrimaryMap<StaticModuleIndex, ModuleTranslation<'_>>,
     ) -> Result<(wasmtime_jit::ObjectBuilder<'a>, Artifacts)> {
@@ -457,6 +456,8 @@ impl FunctionIndices {
         // The result is a vector parallel to `compiled_funcs` where
         // `symbol_ids_and_locs[i]` is the symbol ID and function location of
         // `compiled_funcs[i]`.
+        let compiler = engine.compiler();
+        let tunables = &engine.config().tunables;
         let symbol_ids_and_locs = compiler.append_code(
             &mut obj,
             &compiled_funcs,
@@ -559,7 +560,22 @@ impl FunctionIndices {
 
         artifacts.modules = translations
             .into_iter()
-            .map(|(module, translation)| {
+            .map(|(module, mut translation)| {
+                // If configured attempt to use static memory initialization which
+                // can either at runtime be implemented as a single memcpy to
+                // initialize memory or otherwise enabling virtual-memory-tricks
+                // such as mmap'ing from a file to get copy-on-write.
+                if engine.config().memory_init_cow {
+                    let align = compiler.page_size_align();
+                    let max_always_allowed = engine.config().memory_guaranteed_dense_image_size;
+                    translation.try_static_init(align, max_always_allowed);
+                }
+
+                // Attempt to convert table initializer segments to
+                // FuncTable representation where possible, to enable
+                // table lazy init.
+                translation.try_func_table_init();
+
                 let funcs: PrimaryMap<DefinedFuncIndex, CompiledFunctionInfo> =
                     wasm_functions_for_module(&mut wasm_functions, module)
                         .map(|(key, wasm_func_index)| {
diff --git a/crates/wasmtime/src/component/component.rs b/crates/wasmtime/src/component/component.rs
index 013f93bd7..a57165ed4 100644
--- a/crates/wasmtime/src/component/component.rs
+++ b/crates/wasmtime/src/component/component.rs
@@ -206,8 +206,7 @@ impl Component {
 
         let (mut object, compilation_artifacts) = function_indices.link_and_append_code(
             object,
-            &engine.config().tunables,
-            compiler,
+            engine,
             compiled_funcs,
             module_translations,
         )?;
@@ -473,3 +472,34 @@ impl ComponentRuntimeInfo for ComponentInner {
         }
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use crate::component::Component;
+    use crate::{Config, Engine};
+    use wasmtime_environ::MemoryInitialization;
+
+    #[test]
+    fn cow_on_by_default() {
+        let mut config = Config::new();
+        config.wasm_component_model(true);
+        let engine = Engine::new(&config).unwrap();
+        let component = Component::new(
+            &engine,
+            r#"
+                (component
+                    (core module
+                        (memory 1)
+                        (data (i32.const 100) "abcd")
+                    )
+                )
+            "#,
+        )
+        .unwrap();
+
+        for (_, module) in component.inner.static_modules.iter() {
+            let init = &module.env_module().memory_initialization;
+            assert!(matches!(init, MemoryInitialization::Static { .. }));
+        }
+    }
+}
diff --git a/crates/wasmtime/src/module.rs b/crates/wasmtime/src/module.rs
index 38a51fcd7..a5553ff94 100644
--- a/crates/wasmtime/src/module.rs
+++ b/crates/wasmtime/src/module.rs
@@ -402,7 +402,6 @@ impl Module {
         use crate::compiler::CompileInputs;
 
         let tunables = &engine.config().tunables;
-        let compiler = engine.compiler();
 
         // First a `ModuleEnvironment` is created which records type information
         // about the wasm module. This is where the WebAssembly is parsed and
@@ -437,25 +436,9 @@ impl Module {
         engine.append_compiler_info(&mut object);
         engine.append_bti(&mut object);
 
-        // If configured attempt to use static memory initialization which
-        // can either at runtime be implemented as a single memcpy to
-        // initialize memory or otherwise enabling virtual-memory-tricks
-        // such as mmap'ing from a file to get copy-on-write.
-        if engine.config().memory_init_cow {
-            let align = engine.compiler().page_size_align();
-            let max_always_allowed = engine.config().memory_guaranteed_dense_image_size;
-            translation.try_static_init(align, max_always_allowed);
-        }
-
-        // Attempt to convert table initializer segments to
-        // FuncTable representation where possible, to enable
-        // table lazy init.
-        translation.try_func_table_init();
-
         let (mut object, compilation_artifacts) = function_indices.link_and_append_code(
             object,
-            tunables,
-            compiler,
+            engine,
             compiled_funcs,
             std::iter::once(translation).collect(),
         )?;
@@ -1361,3 +1344,27 @@ fn memory_images(engine: &Engine, module: &CompiledModule) -> Result<Option<Modu
     };
     ModuleMemoryImages::new(module.module(), module.code_memory().wasm_data(), mmap)
 }
+
+#[cfg(test)]
+mod tests {
+    use crate::{Engine, Module};
+    use wasmtime_environ::MemoryInitialization;
+
+    #[test]
+    fn cow_on_by_default() {
+        let engine = Engine::default();
+        let module = Module::new(
+            &engine,
+            r#"
+                (module
+                    (memory 1)
+                    (data (i32.const 100) "abcd")
+                )
+            "#,
+        )
+        .unwrap();
+
+        let init = &module.env_module().memory_initialization;
+        assert!(matches!(init, MemoryInitialization::Static { .. }));
+    }
+}
diff --git a/src/commands/serve.rs b/src/commands/serve.rs
index 31276a1aa..927cc755b 100644
--- a/src/commands/serve.rs
+++ b/src/commands/serve.rs
@@ -373,12 +373,16 @@ impl hyper::service::Service<Request> for ProxyHandler {
                 wasmtime_wasi_http::proxy::Proxy::instantiate_pre(&mut store, &inner.instance_pre)
                     .await?;
 
-            proxy
+            if let Err(e) = proxy
                 .wasi_http_incoming_handler()
                 .call_handle(store, req, out)
-                .await?;
+                .await
+            {
+                log::error!("[{req_id}] :: {:#?}", e);
+                return Err(e);
+            }
 
-            Ok::<_, anyhow::Error>(())
+            Ok(())
         });
 
         Box::pin(async move {
diff --git a/winch/codegen/src/codegen/mod.rs b/winch/codegen/src/codegen/mod.rs
index ab688a61b..bb1a8fead 100644
--- a/winch/codegen/src/codegen/mod.rs
+++ b/winch/codegen/src/codegen/mod.rs
@@ -104,6 +104,7 @@ where
             ControlStackFrame::If {
                 reachable,
                 original_stack_len,
+                original_sp_offset,
                 ..
             } => {
                 if *reachable {
@@ -112,7 +113,12 @@ where
                     // entry, the if-else branch will be reachable.
                     self.context.reachable = true;
                     // Reset the stack to the original length and offset.
-                    Self::reset_stack(&mut self.context, *original_stack_len);
+                    Self::reset_stack(
+                        &mut self.context,
+                        self.masm,
+                        *original_stack_len,
+                        *original_sp_offset,
+                    );
                     frame.bind_else(self.masm, self.context.reachable);
                 }
             }
@@ -127,9 +133,9 @@ where
         if frame.is_next_sequence_reachable() {
             self.context.reachable = true;
 
-            let (value_stack_len, _) = frame.original_stack_len_and_sp_offset();
+            let (value_stack_len, target_sp) = frame.original_stack_len_and_sp_offset();
             // Reset the stack to the original length and offset.
-            Self::reset_stack(&mut self.context, value_stack_len);
+            Self::reset_stack(&mut self.context, self.masm, value_stack_len, target_sp);
             // If the current frame is the outermost frame, which corresponds to the
             // current function's body, only bind the exit label as we don't need to
             // push any more values to the value stack, else perform the entire `bind_end`
@@ -145,27 +151,36 @@ where
             // and SP in the expected state.  The compiler can enter
             // in this state through an infinite loop.
             let (value_stack_len, target_sp) = frame.original_stack_len_and_sp_offset();
-            Self::reset_stack(&mut self.context, value_stack_len);
-            if self.masm.sp_offset() > target_sp {
-                self.masm.free_stack(self.masm.sp_offset() - target_sp);
-            }
+            Self::reset_stack(&mut self.context, self.masm, value_stack_len, target_sp);
         }
     }
 
     /// Helper function to reset value and stack pointer to the given length and stack pointer
     /// offset respectively. This function is only used when restoring the code generation's
     /// reachabiliy state when handling an unreachable `end` or `else`.
-    pub fn reset_stack(context: &mut CodeGenContext, target_stack_len: usize) {
+    pub fn reset_stack(
+        context: &mut CodeGenContext,
+        masm: &mut M,
+        target_stack_len: usize,
+        target_sp: u32,
+    ) {
         // `CodeGenContext::reset_stack` only gets called when
         // handling unreachable end or unreachable else, so we only
         // care about freeing any registers in the provided range.
+        let mut bytes_freed = 0;
         context.drop_last(
             context.stack.len() - target_stack_len,
             |regalloc, val| match val {
                 Val::Reg(tr) => regalloc.free(tr.reg),
+                Val::Memory(m) => bytes_freed += m.slot.size,
                 _ => {}
             },
         );
+        if masm.sp_offset() > target_sp {
+            let bytes = masm.sp_offset() - target_sp;
+            assert!(bytes_freed == bytes);
+            masm.free_stack(bytes);
+        }
     }
 
     fn emit_body(
diff --git a/winch/codegen/src/isa/x64/asm.rs b/winch/codegen/src/isa/x64/asm.rs
index 98f7723c6..bd791784e 100644
--- a/winch/codegen/src/isa/x64/asm.rs
+++ b/winch/codegen/src/isa/x64/asm.rs
@@ -202,8 +202,14 @@ impl Assembler {
                 // instrunction.
                 let constant_data = pool.get(*c);
                 let data = VCodeConstantData::Pool(*c, constant_data.clone());
+                // If the constaant data is not marked as used, it will be
+                // inserted, therefore, it needs to be registered.
+                let needs_registration = !constants.pool_uses(&data);
                 let constant = constants.insert(VCodeConstantData::Pool(*c, constant_data.clone()));
-                buffer.register_constant(&constant, &data);
+
+                if needs_registration {
+                    buffer.register_constant(&constant, &data);
+                }
                 SyntheticAmode::ConstantOffset(constant)
             }
         }


Test patch:
diff --git a/cranelift/filetests/filetests/egraph/shifts.clif b/cranelift/filetests/filetests/egraph/shifts.clif
index dbe984c84..fbe308ddb 100644
--- a/cranelift/filetests/filetests/egraph/shifts.clif
+++ b/cranelift/filetests/filetests/egraph/shifts.clif
@@ -74,9 +74,9 @@ block0(v0: i8):
     v2 = sshr v0, v1
     v3 = ishl v2, v1
     return v3
-    ; check: v4 = iconst.i8 224
-    ; check: v5 = band v0, v4
-    ; check: return v5
+    ; check: v8 = iconst.i8 224
+    ; check: v9 = band v0, v8
+    ; check: return v9
 }
 
 function %sextend_shift_32_64_unsigned(i32) -> i64 {
@@ -596,9 +596,9 @@ block0(v0: i128):
     return v3
 }
 
-; check: v4 = iconst.i64 0
-; nextln: v5 = uextend.i128 v4  ; v4 = 0
-; nextln: return v5
+; check: v7 = iconst.i64 0
+; nextln: v8 = uextend.i128 v7
+; nextln: return v8
 
 
 function %i128_ishl_overflow_becomes_0(i128) -> i128 {
@@ -609,9 +609,9 @@ block0(v0: i128):
     return v3
 }
 
-; check: v4 = iconst.i64 0
-; nextln: v5 = uextend.i128 v4  ; v4 = 0
-; nextln: return v5
+; check: v7 = iconst.i64 0
+; nextln: v8 = uextend.i128 v7
+; nextln: return v8
 
 function %simd_shift_does_not_panic(i8x16) -> i8x16 {
 block0(v0: i8x16):
@@ -636,11 +636,10 @@ block0(v0: i8x16):
     return v5
 }
 
-; check: v1 = iconst.i16 30
-; nextln: v3 = sshr v0, v1  ; v1 = 30
-; nextln: v6 = iconst.i16 6
-; nextln: v7 = sshr v3, v6  ; v6 = 6
-; check: return v7
+; check: v6 = iconst.i16 6
+; check: v7 = sshr v0, v6
+; check: v9 = sshr v7, v6
+; check: return v9
 
 
 function %rotr_rotr_prop(i8) -> i8 {
@@ -758,3 +757,57 @@ block0(v0: i8, v1: i8):
 }
 
 ; check: return v0
+
+function %shifts_to_rotl(i64) -> i64 {
+block0(v0: i64):
+    v1 = iconst.i16 3
+    v2 = iconst.i32 61
+    v3 = ishl v0, v1
+    v4 = ushr v0, v2
+    v5 = bor v3, v4
+    return v5
+}
+
+; check: v1 = iconst.i16 3
+; nextln: v6 = rotl v0, v1  ; v1 = 3
+; check: return v6
+
+
+function %shifts_to_rotl_reverse(i64) -> i64 {
+block0(v0: i64):
+    v1 = iconst.i16 32
+    v2 = iconst.i32 32
+    v3 = ishl v0, v1
+    v4 = ushr v0, v2
+    v5 = bor v4, v3
+    return v5
+}
+
+; check: v1 = iconst.i16 32
+; nextln: v6 = rotl v0, v1  ; v1 = 32
+; check: return v6
+
+function %shifts_to_rotr_equivalent(i64) -> i64 {
+block0(v0: i64):
+    v1 = iconst.i16 3
+    v2 = iconst.i32 61
+    v3 = ishl v0, v2
+    v4 = ushr v0, v1
+    v5 = bor v3, v4
+    return v5
+}
+
+; check: v2 = iconst.i32 61
+; nextln: v6 = rotl v0, v2  ; v2 = 61
+; check: return v6
+
+function %ishl_normalize_amount(i64) -> i64 {
+block0(v0: i64):
+    v1 = iconst.i32 99
+    v2 = ishl v0, v1
+    return v2
+}
+
+; check: v3 = iconst.i32 35
+; check: v4 = ishl v0, v3
+; check: return v4
diff --git a/cranelift/filetests/filetests/isa/aarch64/inline-probestack.clif b/cranelift/filetests/filetests/isa/aarch64/inline-probestack.clif
index 2ec7431b9..eee45fd76 100644
--- a/cranelift/filetests/filetests/isa/aarch64/inline-probestack.clif
+++ b/cranelift/filetests/filetests/isa/aarch64/inline-probestack.clif
@@ -47,9 +47,13 @@ block0:
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
-;   movn x16, #4095 ; str wzr, [sp, x16, SXTX]
-;   movn x16, #8191 ; str wzr, [sp, x16, SXTX]
-;   movn x16, #12287 ; str wzr, [sp, x16, SXTX]
+;   sub sp, sp, #4096
+;   str wzr, [sp]
+;   sub sp, sp, #4096
+;   str wzr, [sp]
+;   sub sp, sp, #4096
+;   str wzr, [sp]
+;   add sp, sp, #12288
 ;   sub sp, sp, #12288
 ; block0:
 ;   mov x0, sp
@@ -61,14 +65,15 @@ block0:
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-;   mov x16, #-0x1000
-;   str wzr, [sp, x16, sxtx]
-;   mov x16, #-0x2000
-;   str wzr, [sp, x16, sxtx]
-;   mov x16, #-0x3000
-;   str wzr, [sp, x16, sxtx]
+;   sub sp, sp, #1, lsl #12
+;   stur wzr, [sp]
+;   sub sp, sp, #1, lsl #12
+;   stur wzr, [sp]
+;   sub sp, sp, #1, lsl #12
+;   stur wzr, [sp]
+;   add sp, sp, #3, lsl #12
 ;   sub sp, sp, #3, lsl #12
-; block1: ; offset 0x24
+; block1: ; offset 0x28
 ;   mov x0, sp
 ;   add sp, sp, #3, lsl #12
 ;   ldp x29, x30, [sp], #0x10
diff --git a/cranelift/filetests/filetests/isa/riscv64/c-inline-probestack.clif b/cranelift/filetests/filetests/isa/riscv64/c-inline-probestack.clif
new file mode 100644
index 000000000..8ab8cd3fb
--- /dev/null
+++ b/cranelift/filetests/filetests/isa/riscv64/c-inline-probestack.clif
@@ -0,0 +1,162 @@
+test compile precise-output
+set enable_probestack=true
+set probestack_strategy=inline
+; This is the default and is equivalent to a page size of 4096
+set probestack_size_log2=12
+target riscv64 has_c
+
+
+; If the stack size is just one page, we can avoid the stack probe entirely
+function %single_page() -> i64 system_v {
+ss0 = explicit_slot 2048
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   addi sp,sp,-2048
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,1
+;   addi t6,t6,-2048
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   c.addi16sp sp, -0x10
+;   c.sdsp ra, 8(sp)
+;   c.sdsp s0, 0(sp)
+;   c.mv s0, sp
+;   addi sp, sp, -0x800
+; block1: ; offset 0xc
+;   c.mv a0, sp
+;   c.lui t6, 1
+;   addi t6, t6, -0x800
+;   c.add sp, t6
+;   c.ldsp ra, 8(sp)
+;   c.ldsp s0, 0(sp)
+;   c.addi16sp sp, 0x10
+;   c.jr ra
+
+function %unrolled() -> i64 system_v {
+ss0 = explicit_slot 12288
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   lui t3,-1
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   lui t6,3
+;   add sp,sp,t6
+;   lui t6,-3
+;   add sp,sp,t6
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,3
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   c.addi16sp sp, -0x10
+;   c.sdsp ra, 8(sp)
+;   c.sdsp s0, 0(sp)
+;   c.mv s0, sp
+;   c.lui t3, 0xfffff
+;   c.add sp, t3
+;   c.swsp zero, 0(sp)
+;   c.add sp, t3
+;   c.swsp zero, 0(sp)
+;   c.add sp, t3
+;   c.swsp zero, 0(sp)
+;   c.lui t6, 3
+;   c.add sp, t6
+;   c.lui t6, 0xffffd
+;   c.add sp, t6
+; block1: ; offset 0x1e
+;   c.mv a0, sp
+;   c.lui t6, 3
+;   c.add sp, t6
+;   c.ldsp ra, 8(sp)
+;   c.ldsp s0, 0(sp)
+;   c.addi16sp sp, 0x10
+;   c.jr ra
+
+function %large() -> i64 system_v {
+ss0 = explicit_slot 100000
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   inline_stack_probe##guard_size=4096 probe_count=25 tmp=t3
+;   lui t6,-24
+;   addi t6,t6,-1696
+;   add sp,sp,t6
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,24
+;   addi t6,t6,1696
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   c.addi16sp sp, -0x10
+;   c.sdsp ra, 8(sp)
+;   c.sdsp s0, 0(sp)
+;   c.mv s0, sp
+;   c.lui t6, 0x19
+;   c.lui t3, 1
+;   bgeu t3, t6, 0x12
+;   sub t5, sp, t6
+;   sb zero, 0(t5)
+;   sub t6, t6, t3
+;   c.j -0x10
+;   c.lui t6, 0xfffe8
+;   addi t6, t6, -0x6a0
+;   c.add sp, t6
+; block1: ; offset 0x26
+;   c.mv a0, sp
+;   c.lui t6, 0x18
+;   addi t6, t6, 0x6a0
+;   c.add sp, t6
+;   c.ldsp ra, 8(sp)
+;   c.ldsp s0, 0(sp)
+;   c.addi16sp sp, 0x10
+;   c.jr ra
+
diff --git a/cranelift/filetests/filetests/isa/riscv64/inline-probestack.clif b/cranelift/filetests/filetests/isa/riscv64/inline-probestack.clif
new file mode 100644
index 000000000..25e0b9c39
--- /dev/null
+++ b/cranelift/filetests/filetests/isa/riscv64/inline-probestack.clif
@@ -0,0 +1,162 @@
+test compile precise-output
+set enable_probestack=true
+set probestack_strategy=inline
+; This is the default and is equivalent to a page size of 4096
+set probestack_size_log2=12
+target riscv64
+
+
+; If the stack size is just one page, we can avoid the stack probe entirely
+function %single_page() -> i64 system_v {
+ss0 = explicit_slot 2048
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   addi sp,sp,-2048
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,1
+;   addi t6,t6,-2048
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   addi sp, sp, -0x10
+;   sd ra, 8(sp)
+;   sd s0, 0(sp)
+;   mv s0, sp
+;   addi sp, sp, -0x800
+; block1: ; offset 0x14
+;   mv a0, sp
+;   lui t6, 1
+;   addi t6, t6, -0x800
+;   add sp, sp, t6
+;   ld ra, 8(sp)
+;   ld s0, 0(sp)
+;   addi sp, sp, 0x10
+;   ret
+
+function %unrolled() -> i64 system_v {
+ss0 = explicit_slot 12288
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   lui t3,-1
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   add sp,sp,t3
+;   sw zero,0(sp)
+;   lui t6,3
+;   add sp,sp,t6
+;   lui t6,-3
+;   add sp,sp,t6
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,3
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   addi sp, sp, -0x10
+;   sd ra, 8(sp)
+;   sd s0, 0(sp)
+;   mv s0, sp
+;   lui t3, 0xfffff
+;   add sp, sp, t3
+;   sw zero, 0(sp)
+;   add sp, sp, t3
+;   sw zero, 0(sp)
+;   add sp, sp, t3
+;   sw zero, 0(sp)
+;   lui t6, 3
+;   add sp, sp, t6
+;   lui t6, 0xffffd
+;   add sp, sp, t6
+; block1: ; offset 0x3c
+;   mv a0, sp
+;   lui t6, 3
+;   add sp, sp, t6
+;   ld ra, 8(sp)
+;   ld s0, 0(sp)
+;   addi sp, sp, 0x10
+;   ret
+
+function %large() -> i64 system_v {
+ss0 = explicit_slot 100000
+
+block0:
+  v1 = stack_addr.i64 ss0
+  return v1
+}
+
+; VCode:
+;   addi sp,sp,-16
+;   sd ra,8(sp)
+;   sd fp,0(sp)
+;   mv fp,sp
+;   inline_stack_probe##guard_size=4096 probe_count=25 tmp=t3
+;   lui t6,-24
+;   addi t6,t6,-1696
+;   add sp,sp,t6
+; block0:
+;   load_addr a0,0(nominal_sp)
+;   lui t6,24
+;   addi t6,t6,1696
+;   add sp,sp,t6
+;   ld ra,8(sp)
+;   ld fp,0(sp)
+;   addi sp,sp,16
+;   ret
+;
+; Disassembled:
+; block0: ; offset 0x0
+;   addi sp, sp, -0x10
+;   sd ra, 8(sp)
+;   sd s0, 0(sp)
+;   mv s0, sp
+;   lui t6, 0x19
+;   lui t3, 1
+;   bgeu t3, t6, 0x14
+;   sub t5, sp, t6
+;   sb zero, 0(t5)
+;   sub t6, t6, t3
+;   j -0x10
+;   lui t6, 0xfffe8
+;   addi t6, t6, -0x6a0
+;   add sp, sp, t6
+; block1: ; offset 0x38
+;   mv a0, sp
+;   lui t6, 0x18
+;   addi t6, t6, 0x6a0
+;   add sp, sp, t6
+;   ld ra, 8(sp)
+;   ld s0, 0(sp)
+;   addi sp, sp, 0x10
+;   ret
+
diff --git a/cranelift/filetests/filetests/isa/riscv64/stack-limit.clif b/cranelift/filetests/filetests/isa/riscv64/stack-limit.clif
index b0e01f1ce..be0c64af3 100644
--- a/cranelift/filetests/filetests/isa/riscv64/stack-limit.clif
+++ b/cranelift/filetests/filetests/isa/riscv64/stack-limit.clif
@@ -195,11 +195,11 @@ block0(v0: i64):
 ;   call %Probestack
 ;   lui t6,-98
 ;   addi t6,t6,1408
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   lui t6,98
 ;   addi t6,t6,-1408
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -224,11 +224,11 @@ block0(v0: i64):
 ;   jalr ra
 ;   lui t6, 0xfff9e
 ;   addi t6, t6, 0x580
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x48
 ;   lui t6, 0x62
 ;   addi t6, t6, -0x580
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -307,11 +307,11 @@ block0(v0: i64):
 ;   call %Probestack
 ;   lui t6,-98
 ;   addi t6,t6,1408
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   lui t6,98
 ;   addi t6,t6,-1408
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -338,11 +338,11 @@ block0(v0: i64):
 ;   jalr ra
 ;   lui t6, 0xfff9e
 ;   addi t6, t6, 0x580
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x50
 ;   lui t6, 0x62
 ;   addi t6, t6, -0x580
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
diff --git a/cranelift/filetests/filetests/isa/riscv64/stack.clif b/cranelift/filetests/filetests/isa/riscv64/stack.clif
index 143a13482..4174d42b2 100644
--- a/cranelift/filetests/filetests/isa/riscv64/stack.clif
+++ b/cranelift/filetests/filetests/isa/riscv64/stack.clif
@@ -59,12 +59,12 @@ block0:
 ;   call %Probestack
 ;   lui t6,-24
 ;   addi t6,t6,-1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   load_addr a0,0(nominal_sp)
 ;   lui t6,24
 ;   addi t6,t6,1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -82,12 +82,12 @@ block0:
 ;   jalr ra
 ;   lui t6, 0xfffe8
 ;   addi t6, t6, -0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x2c
 ;   mv a0, sp
 ;   lui t6, 0x18
 ;   addi t6, t6, 0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -149,12 +149,12 @@ block0:
 ;   call %Probestack
 ;   lui t6,-24
 ;   addi t6,t6,-1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   ld a0,0(nominal_sp)
 ;   lui t6,24
 ;   addi t6,t6,1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -172,12 +172,12 @@ block0:
 ;   jalr ra
 ;   lui t6, 0xfffe8
 ;   addi t6, t6, -0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x2c
 ;   ld a0, 0(sp)
 ;   lui t6, 0x18
 ;   addi t6, t6, 0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -239,12 +239,12 @@ block0(v0: i64):
 ;   call %Probestack
 ;   lui t6,-24
 ;   addi t6,t6,-1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   sd a0,0(nominal_sp)
 ;   lui t6,24
 ;   addi t6,t6,1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -262,12 +262,12 @@ block0(v0: i64):
 ;   jalr ra
 ;   lui t6, 0xfffe8
 ;   addi t6, t6, -0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x2c
 ;   sd a0, 0(sp)
 ;   lui t6, 0x18
 ;   addi t6, t6, 0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -933,13 +933,13 @@ block0(v0: i128):
 ;   call %Probestack
 ;   lui t6,-24
 ;   addi t6,t6,-1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   sd a0,0(nominal_sp)
 ;   sd a1,8(nominal_sp)
 ;   lui t6,24
 ;   addi t6,t6,1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -957,13 +957,13 @@ block0(v0: i128):
 ;   jalr ra
 ;   lui t6, 0xfffe8
 ;   addi t6, t6, -0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x2c
 ;   sd a0, 0(sp)
 ;   sd a1, 8(sp)
 ;   lui t6, 0x18
 ;   addi t6, t6, 0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -1067,13 +1067,13 @@ block0:
 ;   call %Probestack
 ;   lui t6,-24
 ;   addi t6,t6,-1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ; block0:
 ;   ld a0,0(nominal_sp)
 ;   ld a1,8(nominal_sp)
 ;   lui t6,24
 ;   addi t6,t6,1712
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -1091,13 +1091,13 @@ block0:
 ;   jalr ra
 ;   lui t6, 0xfffe8
 ;   addi t6, t6, -0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ; block1: ; offset 0x2c
 ;   ld a0, 0(sp)
 ;   ld a1, 8(sp)
 ;   lui t6, 0x18
 ;   addi t6, t6, 0x6b0
-;   add sp, t6, sp
+;   add sp, sp, t6
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
diff --git a/cranelift/filetests/filetests/isa/riscv64/wasm/zbb.wat b/cranelift/filetests/filetests/isa/riscv64/wasm/zbb.wat
index e307e6a6a..f0cc3f4cc 100644
--- a/cranelift/filetests/filetests/isa/riscv64/wasm/zbb.wat
+++ b/cranelift/filetests/filetests/isa/riscv64/wasm/zbb.wat
@@ -76,7 +76,7 @@
 ;; block0:
 ;;   j label1
 ;; block1:
-;;   roriw a0,a0,100
+;;   roriw a0,a0,4
 ;;   ret
 ;;
 ;; function u0:7:
diff --git a/cranelift/filetests/filetests/isa/riscv64/zca.clif b/cranelift/filetests/filetests/isa/riscv64/zca.clif
index 6d6620d52..c0cf006d6 100644
--- a/cranelift/filetests/filetests/isa/riscv64/zca.clif
+++ b/cranelift/filetests/filetests/isa/riscv64/zca.clif
@@ -463,7 +463,7 @@ block0:
 ;   mv a0,a5
 ;   lui t6,1
 ;   addi t6,t6,-2048
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -489,7 +489,7 @@ block0:
 ;   c.mv a0, a5
 ;   c.lui t6, 1
 ;   addi t6, t6, -0x800
-;   add sp, t6, sp
+;   c.add sp, t6
 ;   c.ldsp ra, 8(sp)
 ;   c.ldsp s0, 0(sp)
 ;   c.addi16sp sp, 0x10
@@ -515,7 +515,7 @@ block0:
 ;   load_addr a1,0(nominal_sp)
 ;   lui t6,1
 ;   addi t6,t6,-2048
-;   add sp,t6,sp
+;   add sp,sp,t6
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -533,7 +533,7 @@ block0:
 ;   c.mv a1, sp
 ;   c.lui t6, 1
 ;   addi t6, t6, -0x800
-;   add sp, t6, sp
+;   c.add sp, t6
 ;   c.ldsp ra, 8(sp)
 ;   c.ldsp s0, 0(sp)
 ;   c.addi16sp sp, 0x10
diff --git a/cranelift/filetests/filetests/isa/x64/inline-probestack-large.clif b/cranelift/filetests/filetests/isa/x64/inline-probestack-large.clif
index adeab023c..c2193fd97 100644
--- a/cranelift/filetests/filetests/isa/x64/inline-probestack-large.clif
+++ b/cranelift/filetests/filetests/isa/x64/inline-probestack-large.clif
@@ -50,9 +50,13 @@ block0:
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   movl    %esp, -65536(%rsp)
-;   movl    %esp, -131072(%rsp)
-;   movl    %esp, -196608(%rsp)
+;   subq    %rsp, $65536, %rsp
+;   movl    %esp, 0(%rsp)
+;   subq    %rsp, $65536, %rsp
+;   movl    %esp, 0(%rsp)
+;   subq    %rsp, $65536, %rsp
+;   movl    %esp, 0(%rsp)
+;   addq    %rsp, $196608, %rsp
 ;   subq    %rsp, $196608, %rsp
 ; block0:
 ;   lea     rsp(0 + virtual offset), %rax
@@ -65,11 +69,15 @@ block0:
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   movl %esp, -0x10000(%rsp)
-;   movl %esp, -0x20000(%rsp)
-;   movl %esp, -0x30000(%rsp)
+;   subq $0x10000, %rsp
+;   movl %esp, (%rsp)
+;   subq $0x10000, %rsp
+;   movl %esp, (%rsp)
+;   subq $0x10000, %rsp
+;   movl %esp, (%rsp)
+;   addq $0x30000, %rsp
 ;   subq $0x30000, %rsp
-; block1: ; offset 0x20
+; block1: ; offset 0x30
 ;   leaq (%rsp), %rax
 ;   addq $0x30000, %rsp
 ;   movq %rbp, %rsp
diff --git a/cranelift/filetests/filetests/isa/x64/inline-probestack.clif b/cranelift/filetests/filetests/isa/x64/inline-probestack.clif
index c55c87000..afb5b93fe 100644
--- a/cranelift/filetests/filetests/isa/x64/inline-probestack.clif
+++ b/cranelift/filetests/filetests/isa/x64/inline-probestack.clif
@@ -49,9 +49,13 @@ block0:
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   movl    %esp, -4096(%rsp)
-;   movl    %esp, -8192(%rsp)
-;   movl    %esp, -12288(%rsp)
+;   subq    %rsp, $4096, %rsp
+;   movl    %esp, 0(%rsp)
+;   subq    %rsp, $4096, %rsp
+;   movl    %esp, 0(%rsp)
+;   subq    %rsp, $4096, %rsp
+;   movl    %esp, 0(%rsp)
+;   addq    %rsp, $12288, %rsp
 ;   subq    %rsp, $12288, %rsp
 ; block0:
 ;   lea     rsp(0 + virtual offset), %rax
@@ -64,11 +68,15 @@ block0:
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   movl %esp, -0x1000(%rsp)
-;   movl %esp, -0x2000(%rsp)
-;   movl %esp, -0x3000(%rsp)
+;   subq $0x1000, %rsp
+;   movl %esp, (%rsp)
+;   subq $0x1000, %rsp
+;   movl %esp, (%rsp)
+;   subq $0x1000, %rsp
+;   movl %esp, (%rsp)
+;   addq $0x3000, %rsp
 ;   subq $0x3000, %rsp
-; block1: ; offset 0x20
+; block1: ; offset 0x30
 ;   leaq (%rsp), %rax
 ;   addq $0x3000, %rsp
 ;   movq %rbp, %rsp
diff --git a/crates/wasi-http/tests/all/main.rs b/crates/wasi-http/tests/all/main.rs
index 3683213df..7a106f4a3 100644
--- a/crates/wasi-http/tests/all/main.rs
+++ b/crates/wasi-http/tests/all/main.rs
@@ -268,7 +268,7 @@ async fn do_wasi_http_hash_all(override_send_request: bool) -> Result<()> {
                 let response = handle(request.into_parts().0).map(|resp| {
                     Ok(IncomingResponseInternal {
                         resp,
-                        worker: Arc::new(preview2::spawn(future::ready(Ok(())))),
+                        worker: Arc::new(preview2::spawn(future::ready(()))),
                         between_bytes_timeout,
                     })
                 });
diff --git a/tests/misc_testsuite/winch/local_get.wast b/tests/misc_testsuite/winch/local_get.wast
new file mode 100644
index 000000000..16e06113d
--- /dev/null
+++ b/tests/misc_testsuite/winch/local_get.wast
@@ -0,0 +1,103 @@
+;; Test `local.get` operator
+
+(module
+  ;; Typing
+
+  (func (export "type-local-i32") (result i32) (local i32) (local.get 0))
+  (func (export "type-local-i64") (result i64) (local i64) (local.get 0))
+  (func (export "type-local-f32") (result f32) (local f32) (local.get 0))
+  (func (export "type-local-f64") (result f64) (local f64) (local.get 0))
+
+  (func (export "type-param-i32") (param i32) (result i32) (local.get 0))
+  (func (export "type-param-i64") (param i64) (result i64) (local.get 0))
+  (func (export "type-param-f32") (param f32) (result f32) (local.get 0))
+  (func (export "type-param-f64") (param f64) (result f64) (local.get 0))
+
+  (func (export "type-mixed") (param i64 f32 f64 i32 i32)
+    (local f32 i64 i64 f64)
+    (drop (i64.eqz (local.get 0)))
+    (drop (f32.neg (local.get 1)))
+    (drop (f64.neg (local.get 2)))
+    (drop (i32.eqz (local.get 3)))
+    (drop (i32.eqz (local.get 4)))
+    (drop (f32.neg (local.get 5)))
+    (drop (i64.eqz (local.get 6)))
+    (drop (i64.eqz (local.get 7)))
+    (drop (f64.neg (local.get 8)))
+  )
+
+
+  ;; As parameter of control constructs and instructions
+
+  (func (export "as-block-value") (param i32) (result i32)
+    (block (result i32) (local.get 0))
+  )
+  (func (export "as-loop-value") (param i32) (result i32)
+    (loop (result i32) (local.get 0))
+  )
+  (func (export "as-br-value") (param i32) (result i32)
+    (block (result i32) (br 0 (local.get 0)))
+  )
+  (func (export "as-br_if-value") (param i32) (result i32)
+    (block $l0 (result i32) (br_if $l0 (local.get 0) (i32.const 1)))
+  )
+
+  (func (export "as-br_if-value-cond") (param i32) (result i32)
+    (block (result i32)
+      (br_if 0 (local.get 0) (local.get 0))
+    )
+  )
+  (func (export "as-br_table-value") (param i32) (result i32)
+    (block
+      (block
+        (block
+          (br_table 0 1 2 (local.get 0))
+          (return (i32.const 0))
+        )
+        (return (i32.const 1))
+      )
+      (return (i32.const 2))
+    )
+    (i32.const 3)
+  )
+
+  (func (export "as-return-value") (param i32) (result i32)
+    (return (local.get 0))
+  )
+
+  (func (export "as-if-then") (param i32) (result i32)
+    (if (result i32) (local.get 0) (then (local.get 0)) (else (i32.const 0)))
+  )
+  (func (export "as-if-else") (param i32) (result i32)
+    (if (result i32) (local.get 0) (then (i32.const 1)) (else (local.get 0)))
+  )
+)
+
+(assert_return (invoke "type-local-i32") (i32.const 0))
+(assert_return (invoke "type-local-i64") (i64.const 0))
+(assert_return (invoke "type-local-f32") (f32.const 0))
+(assert_return (invoke "type-local-f64") (f64.const 0))
+
+(assert_return (invoke "type-param-i32" (i32.const 2)) (i32.const 2))
+(assert_return (invoke "type-param-i64" (i64.const 3)) (i64.const 3))
+(assert_return (invoke "type-param-f32" (f32.const 4.4)) (f32.const 4.4))
+(assert_return (invoke "type-param-f64" (f64.const 5.5)) (f64.const 5.5))
+
+(assert_return (invoke "as-block-value" (i32.const 6)) (i32.const 6))
+(assert_return (invoke "as-loop-value" (i32.const 7)) (i32.const 7))
+
+(assert_return (invoke "as-br-value" (i32.const 8)) (i32.const 8))
+(assert_return (invoke "as-br_if-value" (i32.const 9)) (i32.const 9))
+(assert_return (invoke "as-br_if-value-cond" (i32.const 10)) (i32.const 10))
+(assert_return (invoke "as-br_table-value" (i32.const 1)) (i32.const 2))
+
+(assert_return (invoke "as-return-value" (i32.const 0)) (i32.const 0))
+
+(assert_return (invoke "as-if-then" (i32.const 1)) (i32.const 1))
+(assert_return (invoke "as-if-else" (i32.const 0)) (i32.const 0))
+
+(assert_return
+  (invoke "type-mixed"
+    (i64.const 1) (f32.const 2.2) (f64.const 3.3) (i32.const 4) (i32.const 5)
+  )
+)
diff --git a/tests/misc_testsuite/winch/local_set.wast b/tests/misc_testsuite/winch/local_set.wast
new file mode 100644
index 000000000..f368f7bf3
--- /dev/null
+++ b/tests/misc_testsuite/winch/local_set.wast
@@ -0,0 +1,97 @@
+;; Test `local.set` operator
+
+(module
+  ;; Typing
+
+  (func (export "type-local-i32") (local i32) (local.set 0 (i32.const 0)))
+  (func (export "type-local-i64") (local i64) (local.set 0 (i64.const 0)))
+  (func (export "type-local-f32") (local f32) (local.set 0 (f32.const 0)))
+  (func (export "type-local-f64") (local f64) (local.set 0 (f64.const 0)))
+
+  (func (export "type-param-i32") (param i32) (local.set 0 (i32.const 10)))
+  (func (export "type-param-i64") (param i64) (local.set 0 (i64.const 11)))
+  (func (export "type-param-f32") (param f32) (local.set 0 (f32.const 11.1)))
+  (func (export "type-param-f64") (param f64) (local.set 0 (f64.const 12.2)))
+
+  (func (export "type-mixed") (param i64 f32 f64 i32 i32) (local f32 i64 i64 f64)
+    (local.set 0 (i64.const 0))
+    (local.set 1 (f32.const 0))
+    (local.set 2 (f64.const 0))
+    (local.set 3 (i32.const 0))
+    (local.set 4 (i32.const 0))
+    (local.set 5 (f32.const 0))
+    (local.set 6 (i64.const 0))
+    (local.set 7 (i64.const 0))
+    (local.set 8 (f64.const 0))
+  )
+
+  ;; As parameter of control constructs and instructions
+
+  (func (export "as-block-value") (param i32)
+    (block (local.set 0 (i32.const 1)))
+  )
+  (func (export "as-loop-value") (param i32)
+    (loop (local.set 0 (i32.const 3)))
+  )
+
+  (func (export "as-br-value") (param i32)
+    (block (br 0 (local.set 0 (i32.const 9))))
+  )
+  (func (export "as-br_if-value") (param i32)
+    (block
+      (br_if 0 (local.set 0 (i32.const 8)) (i32.const 1))
+    )
+  )
+  (func (export "as-br_if-value-cond") (param i32)
+    (block
+      (br_if 0 (i32.const 6) (local.set 0 (i32.const 9)))
+    )
+  )
+  (func (export "as-br_table-value") (param i32)
+    (block
+      (br_table 0 (local.set 0 (i32.const 10)) (i32.const 1))
+    )
+  )
+
+  (func (export "as-return-value") (param i32)
+    (return (local.set 0 (i32.const 7)))
+  )
+
+  (func (export "as-if-then") (param i32)
+    (if (local.get 0) (then (local.set 0 (i32.const 3))))
+  )
+  (func (export "as-if-else") (param i32)
+    (if (local.get 0) (then) (else (local.set 0 (i32.const 1))))
+  )
+)
+
+(assert_return (invoke "type-local-i32"))
+(assert_return (invoke "type-local-i64"))
+(assert_return (invoke "type-local-f32"))
+(assert_return (invoke "type-local-f64"))
+
+(assert_return (invoke "type-param-i32" (i32.const 2)))
+(assert_return (invoke "type-param-i64" (i64.const 3)))
+(assert_return (invoke "type-param-f32" (f32.const 4.4)))
+(assert_return (invoke "type-param-f64" (f64.const 5.5)))
+
+(assert_return (invoke "as-block-value" (i32.const 0)))
+(assert_return (invoke "as-loop-value" (i32.const 0)))
+
+(assert_return (invoke "as-br-value" (i32.const 0)))
+(assert_return (invoke "as-br_if-value" (i32.const 0)))
+(assert_return (invoke "as-br_if-value-cond" (i32.const 0)))
+(assert_return (invoke "as-br_table-value" (i32.const 0)))
+
+(assert_return (invoke "as-return-value" (i32.const 0)))
+
+(assert_return (invoke "as-if-then" (i32.const 1)))
+(assert_return (invoke "as-if-else" (i32.const 0)))
+
+(assert_return
+  (invoke "type-mixed"
+    (i64.const 1) (f32.const 2.2) (f64.const 3.3) (i32.const 4) (i32.const 5)
+  )
+)
+
+
diff --git a/winch/filetests/filetests/x64/unreachable/with_spilled_local.wat b/winch/filetests/filetests/x64/unreachable/with_spilled_local.wat
new file mode 100644
index 000000000..66848d1ab
--- /dev/null
+++ b/winch/filetests/filetests/x64/unreachable/with_spilled_local.wat
@@ -0,0 +1,24 @@
+;;! target = "x86_64"
+
+(module
+  (func (export "")
+    (local i32)
+    local.get 0
+    block
+    end
+    unreachable
+  )
+)
+;;    0:	 55                   	push	rbp
+;;    1:	 4889e5               	mov	rbp, rsp
+;;    4:	 4883ec10             	sub	rsp, 0x10
+;;    8:	 48c744240800000000   	
+;; 				mov	qword ptr [rsp + 8], 0
+;;   11:	 4c89742404           	mov	qword ptr [rsp + 4], r14
+;;   16:	 448b5c240c           	mov	r11d, dword ptr [rsp + 0xc]
+;;   1b:	 4153                 	push	r11
+;;   1d:	 0f0b                 	ud2	
+;;   1f:	 4883c408             	add	rsp, 8
+;;   23:	 4883c410             	add	rsp, 0x10
+;;   27:	 5d                   	pop	rbp
+;;   28:	 c3                   	ret	
diff --git a/winch/filetests/filetests/x64/unreachable/with_spilled_local_in_if.wat b/winch/filetests/filetests/x64/unreachable/with_spilled_local_in_if.wat
new file mode 100644
index 000000000..b201d1748
--- /dev/null
+++ b/winch/filetests/filetests/x64/unreachable/with_spilled_local_in_if.wat
@@ -0,0 +1,32 @@
+;;! target = "x86_64"
+
+(module
+  (func (export "")
+    (local i32)
+    local.get 0
+    if
+      local.get 0
+      block
+      end
+      unreachable
+    else
+      nop
+    end
+  )
+)
+;;    0:	 55                   	push	rbp
+;;    1:	 4889e5               	mov	rbp, rsp
+;;    4:	 4883ec10             	sub	rsp, 0x10
+;;    8:	 48c744240800000000   	
+;; 				mov	qword ptr [rsp + 8], 0
+;;   11:	 4c89742404           	mov	qword ptr [rsp + 4], r14
+;;   16:	 8b44240c             	mov	eax, dword ptr [rsp + 0xc]
+;;   1a:	 85c0                 	test	eax, eax
+;;   1c:	 0f840d000000         	je	0x2f
+;;   22:	 448b5c240c           	mov	r11d, dword ptr [rsp + 0xc]
+;;   27:	 4153                 	push	r11
+;;   29:	 0f0b                 	ud2	
+;;   2b:	 4883c408             	add	rsp, 8
+;;   2f:	 4883c410             	add	rsp, 0x10
+;;   33:	 5d                   	pop	rbp
+;;   34:	 c3                   	ret