Repository: bytecodealliance/wasmtime
Pull request #8319 (2024-04-09 19:39:24)

Always reserve space for outgoing args
Instead of reserving stack space for stack arguments and return values at the call site that requires them, eagerly reserve the maximum amount of space needed in the function prologue. This change minimizes manipulation of the stack pointer, and in the case of tail calls only modifies the stack pointer to recover the outgoing argument space that is released by the tail callee. Additionally, eagerly reserving space for stack arguments in the non-s390x backends moves us closer to agreement on how to handle calls across all backends, as it already follows this frame layout.

This change sets us up to be able to remove the notion of a nominal SP from all backends, as the stack pointer will now be a stable point to address everything within the frame with a positive offset.

Co-authored-by: Jamey Sharp <jsharp@fastly.com>

<!--
Please make sure you include the following information:

- If this work has been discussed elsewhere, please include a link to that
  conversation. If it was discussed in an issue, just mention "issue #...".

- Explain why this change is needed. If the details are in an issue already,
  this can be brief.

Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.html

Please ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->



Patch:
diff --git a/cranelift/codegen/src/isa/aarch64/abi.rs b/cranelift/codegen/src/isa/aarch64/abi.rs
index 8139ac81f..6a091a51c 100644
--- a/cranelift/codegen/src/isa/aarch64/abi.rs
+++ b/cranelift/codegen/src/isa/aarch64/abi.rs
@@ -863,10 +863,15 @@ impl ABIMachineSpec for AArch64MachineDeps {
         }
 
         // Allocate the fixed frame below the clobbers if necessary.
-        if frame_layout.fixed_frame_storage_size > 0 {
-            insts.extend(Self::gen_sp_reg_adjust(
-                -(frame_layout.fixed_frame_storage_size as i32),
-            ));
+        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.outgoing_args_size;
+        if stack_size > 0 {
+            insts.extend(Self::gen_sp_reg_adjust(-(stack_size as i32)));
+        }
+
+        // Adjust the nominal sp to account for the outgoing argument area.
+        let sp_adj = frame_layout.outgoing_args_size as i32;
+        if sp_adj > 0 {
+            insts.push(Self::gen_nominal_sp_adj(sp_adj));
         }
 
         insts
@@ -890,10 +895,9 @@ impl ABIMachineSpec for AArch64MachineDeps {
         }
 
         // Free the fixed frame if necessary.
-        if frame_layout.fixed_frame_storage_size > 0 {
-            insts.extend(Self::gen_sp_reg_adjust(
-                frame_layout.fixed_frame_storage_size as i32,
-            ));
+        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.outgoing_args_size;
+        if stack_size > 0 {
+            insts.extend(Self::gen_sp_reg_adjust(stack_size as i32));
         }
 
         let load_vec_reg = |rd| Inst::FpuLoad64 {
diff --git a/cranelift/codegen/src/isa/riscv64/abi.rs b/cranelift/codegen/src/isa/riscv64/abi.rs
index dca2cda68..536a471a8 100644
--- a/cranelift/codegen/src/isa/riscv64/abi.rs
+++ b/cranelift/codegen/src/isa/riscv64/abi.rs
@@ -456,7 +456,9 @@ impl ABIMachineSpec for Riscv64MachineDeps {
         let mut insts = SmallVec::new();
         // Adjust the stack pointer downward for clobbers and the function fixed
         // frame (spillslots and storage slots).
-        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.clobber_size;
+        let stack_size = frame_layout.fixed_frame_storage_size
+            + frame_layout.clobber_size
+            + frame_layout.outgoing_args_size;
         if flags.unwind_info() && frame_layout.setup_area_size > 0 {
             // The *unwind* frame (but not the actual frame) starts at the
             // clobbers, just below the saved FP/LR pair.
@@ -497,6 +499,12 @@ impl ABIMachineSpec for Riscv64MachineDeps {
             }
 
             insts.extend(Self::gen_sp_reg_adjust(-(stack_size as i32)));
+
+            // Adjust the nominal sp to account for the outgoing argument area.
+            let sp_adj = frame_layout.outgoing_args_size as i32;
+            if sp_adj > 0 {
+                insts.push(Self::gen_nominal_sp_adj(sp_adj));
+            }
         }
         insts
     }
@@ -507,7 +515,9 @@ impl ABIMachineSpec for Riscv64MachineDeps {
         frame_layout: &FrameLayout,
     ) -> SmallVec<[Inst; 16]> {
         let mut insts = SmallVec::new();
-        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.clobber_size;
+        let stack_size = frame_layout.fixed_frame_storage_size
+            + frame_layout.clobber_size
+            + frame_layout.outgoing_args_size;
         if stack_size > 0 {
             insts.extend(Self::gen_sp_reg_adjust(stack_size as i32));
         }
diff --git a/cranelift/codegen/src/isa/x64/abi.rs b/cranelift/codegen/src/isa/x64/abi.rs
index a4c36a523..3dc691753 100644
--- a/cranelift/codegen/src/isa/x64/abi.rs
+++ b/cranelift/codegen/src/isa/x64/abi.rs
@@ -654,8 +654,10 @@ impl ABIMachineSpec for X64ABIMachineSpec {
         }
 
         // Adjust the stack pointer downward for clobbers and the function fixed
-        // frame (spillslots and storage slots).
-        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.clobber_size;
+        // frame (spillslots, storage slots, and argument area).
+        let stack_size = frame_layout.fixed_frame_storage_size
+            + frame_layout.clobber_size
+            + frame_layout.outgoing_args_size;
         if stack_size > 0 {
             insts.push(Inst::alu_rmi_r(
                 OperandSize::Size64,
@@ -664,9 +666,17 @@ impl ABIMachineSpec for X64ABIMachineSpec {
                 Writable::from_reg(regs::rsp()),
             ));
         }
+
+        // Adjust the nominal sp to account for the outgoing argument area.
+        let sp_adj = frame_layout.outgoing_args_size as i32;
+        if sp_adj > 0 {
+            insts.push(Self::gen_nominal_sp_adj(sp_adj));
+        }
+
         // Store each clobbered register in order at offsets from RSP,
         // placing them above the fixed frame slots.
-        let mut cur_offset = frame_layout.fixed_frame_storage_size;
+        let mut cur_offset =
+            frame_layout.fixed_frame_storage_size + frame_layout.outgoing_args_size;
         for reg in &frame_layout.clobbered_callee_saves {
             let r_reg = reg.to_reg();
             let off = cur_offset;
@@ -710,12 +720,11 @@ impl ABIMachineSpec for X64ABIMachineSpec {
     ) -> SmallVec<[Self::I; 16]> {
         let mut insts = SmallVec::new();
 
-        let stack_size = frame_layout.fixed_frame_storage_size + frame_layout.clobber_size;
-
         // Restore regs by loading from offsets of RSP. RSP will be
         // returned to nominal-RSP at this point, so we can use the
         // same offsets that we used when saving clobbers above.
-        let mut cur_offset = frame_layout.fixed_frame_storage_size;
+        let mut cur_offset =
+            frame_layout.fixed_frame_storage_size + frame_layout.outgoing_args_size;
         for reg in &frame_layout.clobbered_callee_saves {
             let rreg = reg.to_reg();
             match rreg.class() {
@@ -739,6 +748,11 @@ impl ABIMachineSpec for X64ABIMachineSpec {
                 RegClass::Vector => unreachable!(),
             }
         }
+
+        let stack_size = frame_layout.fixed_frame_storage_size
+            + frame_layout.clobber_size
+            + frame_layout.outgoing_args_size;
+
         // Adjust RSP back upward.
         if stack_size > 0 {
             insts.push(Inst::alu_rmi_r(
diff --git a/cranelift/codegen/src/isa/x64/inst/emit.rs b/cranelift/codegen/src/isa/x64/inst/emit.rs
index c99f4d142..eb465ae56 100644
--- a/cranelift/codegen/src/isa/x64/inst/emit.rs
+++ b/cranelift/codegen/src/isa/x64/inst/emit.rs
@@ -1735,8 +1735,12 @@ pub(crate) fn emit(
 
             // The total size that we're going to copy, including the return address and frame
             // pointer that are pushed on the stack already.
-            let size = i32::try_from(state.nominal_sp_to_fp()).unwrap()
-                + i32::try_from(state.frame_layout().setup_area_size).unwrap();
+            let size = i32::try_from(
+                state.frame_layout().setup_area_size
+                    + state.frame_layout().clobber_size
+                    + state.frame_layout().fixed_frame_storage_size,
+            )
+            .unwrap();
 
             debug_assert_eq!(size % 8, 0);
 
@@ -1780,8 +1784,12 @@ pub(crate) fn emit(
 
             // The total size that we're going to copy, including the return address and frame
             // pointer that are pushed on the stack alreadcy.
-            let size = i32::try_from(state.nominal_sp_to_fp()).unwrap()
-                + i32::try_from(state.frame_layout().setup_area_size).unwrap();
+            let size = i32::try_from(
+                state.frame_layout().setup_area_size
+                    + state.frame_layout().clobber_size
+                    + state.frame_layout().fixed_frame_storage_size,
+            )
+            .unwrap();
 
             debug_assert_eq!(size % 8, 0);
 
diff --git a/cranelift/codegen/src/isa/x64/lower.rs b/cranelift/codegen/src/isa/x64/lower.rs
index 2bb04215d..6c7f41134 100644
--- a/cranelift/codegen/src/isa/x64/lower.rs
+++ b/cranelift/codegen/src/isa/x64/lower.rs
@@ -175,8 +175,6 @@ fn emit_vm_call(
     let mut abi =
         X64CallSite::from_libcall(ctx.sigs(), &sig, &extname, dist, caller_conv, flags.clone());
 
-    abi.emit_stack_pre_adjust(ctx);
-
     assert_eq!(inputs.len(), abi.num_args(ctx.sigs()));
 
     for (i, input) in inputs.iter().enumerate() {
@@ -188,11 +186,12 @@ fn emit_vm_call(
     for (i, output) in outputs.iter().enumerate() {
         retval_insts.extend(abi.gen_retval(ctx, i, ValueRegs::one(*output)).into_iter());
     }
+
     abi.emit_call(ctx);
+
     for inst in retval_insts {
         ctx.emit(inst);
     }
-    abi.emit_stack_post_adjust(ctx);
 
     Ok(())
 }
diff --git a/cranelift/codegen/src/machinst/abi.rs b/cranelift/codegen/src/machinst/abi.rs
index bb65431a2..86f8792a8 100644
--- a/cranelift/codegen/src/machinst/abi.rs
+++ b/cranelift/codegen/src/machinst/abi.rs
@@ -45,7 +45,10 @@
 //! for the latter before knowing how SP might be adjusted around
 //! callsites, we implement a "nominal SP" tracking feature by which a
 //! fixup (distance between actual SP and a "nominal" SP) is known at
-//! each instruction.
+//! each instruction. When the prologue is finished, SP is expected
+//! to point at the bottom of the outgoing argument area, and will
+//! only move again directly around function calls. This allows the
+//! use of fixed offsets from SP for the rest of the function body.
 //!
 //! Note that if we ever support dynamic stack-space allocation (for
 //! `alloca`), we will need a way to reference spill slots and stack
@@ -76,7 +79,7 @@
 //!                              +---------------------------+
 //!                              |          ...              |
 //!                              | clobbered callee-saves    |
-//! unwind-frame base     ---->  | (pushed by prologue)      |
+//! unwind-frame base -------->  | (pushed by prologue)      |
 //!                              +---------------------------+
 //!                              |          ...              |
 //!                              | spill slots               |
@@ -85,11 +88,11 @@
 //!                              | stack slots               |
 //!                              | (accessed via nominal SP) |
 //! nominal SP --------------->  | (alloc'd by prologue)     |
-//! (SP at end of prologue)      +---------------------------+
+//!                              +---------------------------+
 //!                              | [alignment as needed]     |
 //!                              |          ...              |
 //!                              | args for call             |
-//! SP before making a call -->  | (pushed at callsite)      |
+//! SP ----------------------->  | (pushed at callsite)      |
 //!                              +---------------------------+
 //!
 //!   (low address)
@@ -472,6 +475,45 @@ pub trait ABIMachineSpec {
     /// Generate a meta-instruction that adjusts the nominal SP offset.
     fn gen_nominal_sp_adj(amount: i32) -> Self::I;
 
+    /// When setting up for a call, ensure that `space` bytes are available in the outgoing
+    /// argument area on the stack. The specified amount of space is the minimum required for both
+    /// arguments to that function, and any values returned through the stack. There are two
+    /// reasonable implementations which each target can choose between:
+    /// 1. At least this much space is reserved during the prologue and `StackAMode::OutgoingArg`
+    ///    refers to the bottom of the reserved area, so this method does nothing.
+    /// 2. `StackAMode::OutgoingArg` refers to the top of this area, so this method needs to adjust
+    ///    the stack pointer to trim any unused portion of the bottom of the stack frame
+    ///    immediately before the call. `gen_restore_argument_area` needs to undo any stack pointer
+    ///    changes made here.
+    fn gen_reserve_argument_area(_space: u32) -> SmallInstVec<Self::I> {
+        smallvec![]
+    }
+
+    /// When returning from a call, perform any cleanup necessary to restore the stack pointer to
+    /// just after the argument area. This ensures that we always have
+    /// [`FrameLayout::outgoing_args_size`] bytes available in the argument area.
+    ///
+    /// * `ret_space` - The space left consumed in the outgoing argument area for values returned
+    ///   by the callee.
+    /// * `arg_space` - The argument space explicitly cleaned up by the callee when it returns. A
+    ///   value of `0` indicates that the callee did not cleanup the argument area at all, while
+    ///   any other value indicates that the callee has moved the stack pointer to account for
+    ///   those arguments when it returns (as is the case for the tail calling convention).
+    fn gen_restore_argument_area(_ret_space: u32, arg_space: u32) -> SmallInstVec<Self::I> {
+        if arg_space > 0 {
+            let amount = i32::try_from(arg_space).unwrap();
+
+            // Recover the argument space by decrementing sp
+            let mut insts = Self::gen_sp_reg_adjust(-amount);
+
+            // Emit a nominal sp adjustment to ensure offsets are computed correctly
+            insts.push(Self::gen_nominal_sp_adj(amount));
+            insts
+        } else {
+            smallvec![]
+        }
+    }
+
     /// Compute a FrameLayout structure containing a sorted list of all clobbered
     /// registers that are callee-saved according to the ABI, as well as the sizes
     /// of all parts of the stack frame.  The result is used to emit the prologue
@@ -2191,31 +2233,6 @@ impl<M: ABIMachineSpec> CallSite<M> {
         frame_size
     }
 
-    /// Emit code to pre-adjust the stack, prior to argument copies and call.
-    pub fn emit_stack_pre_adjust(&self, ctx: &mut Lower<M::I>) {
-        let sig = &ctx.sigs()[self.sig];
-        let stack_space = sig.sized_stack_arg_space + sig.sized_stack_ret_space;
-        let stack_space = i32::try_from(stack_space).unwrap();
-        adjust_stack_and_nominal_sp::<M>(ctx, -stack_space);
-    }
-
-    /// Emit code to post-adjust the stack, after call return and return-value copies.
-    pub fn emit_stack_post_adjust(&self, ctx: &mut Lower<M::I>) {
-        let sig = &ctx.sigs()[self.sig];
-
-        let stack_space = if sig.call_conv() == isa::CallConv::Tail {
-            // The "tail" calling convention has callees clean up stack
-            // arguments, not callers. Callers still clean up any stack return
-            // space, however.
-            sig.sized_stack_ret_space
-        } else {
-            sig.sized_stack_arg_space + sig.sized_stack_ret_space
-        };
-        let stack_space = i32::try_from(stack_space).unwrap();
-
-        adjust_stack_and_nominal_sp::<M>(ctx, stack_space);
-    }
-
     /// Emit a copy of a large argument into its associated stack buffer, if
     /// any.  We must be careful to perform all these copies (as necessary)
     /// before setting up the argument registers, since we may have to invoke
@@ -2484,16 +2501,10 @@ impl<M: ABIMachineSpec> CallSite<M> {
                         }
                         &ABIArgSlot::Stack { offset, ty, .. } => {
                             let sig_data = &ctx.sigs()[self.sig];
-                            let ret_area_base = if sig_data.call_conv() == isa::CallConv::Tail {
-                                // The callee already popped the stack argument
-                                // space for us, so the return area is on top of
-                                // the stack.
-                                0
-                            } else {
-                                // The return area is just below the stack
-                                // argument area on the stack.
-                                sig_data.sized_stack_arg_space()
-                            };
+                            // The outgoing argument area must always be restored after a call,
+                            // ensuring that the return values will be in a consistent place after
+                            // any call.
+                            let ret_area_base = sig_data.sized_stack_arg_space();
                             insts.push(M::gen_load_stack(
                                 StackAMode::OutgoingArg(offset + ret_area_base),
                                 *into_reg,
@@ -2552,6 +2563,20 @@ impl<M: ABIMachineSpec> CallSite<M> {
             0
         };
 
+        let call_conv = sig.call_conv;
+        let ret_space = sig.sized_stack_ret_space;
+        let arg_space = sig.sized_stack_arg_space;
+
+        ctx.abi_mut()
+            .accumulate_outgoing_args_size(ret_space + arg_space);
+
+        // Any adjustment to SP to account for required outgoing arguments/stack return values must
+        // be done around the call, to ensure that SP is always in a consistent state for all other
+        // writes.
+        for inst in M::gen_reserve_argument_area(ret_space + arg_space) {
+            ctx.emit(inst);
+        }
+
         let tmp = ctx.alloc_tmp(word_type).only_reg().unwrap();
         for inst in M::gen_call(
             &self.dest,
@@ -2560,7 +2585,7 @@ impl<M: ABIMachineSpec> CallSite<M> {
             self.clobbers,
             self.opcode,
             tmp,
-            ctx.sigs()[self.sig].call_conv,
+            call_conv,
             self.caller_conv,
             callee_pop_size,
         )
@@ -2568,6 +2593,20 @@ impl<M: ABIMachineSpec> CallSite<M> {
         {
             ctx.emit(inst);
         }
+
+        // Compute the space that's reclaimed by the callee when it returns. In the case of the
+        // Tail calling convention, the callee will cleanup the arguments used in the outgoing
+        // argument area, which we will need to adjust back down to restore SP to where it was
+        // before the call.
+        let arg_space = if call_conv == isa::CallConv::Tail {
+            arg_space
+        } else {
+            0
+        };
+
+        for inst in M::gen_restore_argument_area(ret_space, arg_space) {
+            ctx.emit(inst);
+        }
     }
 }
 
diff --git a/cranelift/codegen/src/machinst/isle.rs b/cranelift/codegen/src/machinst/isle.rs
index 91462bc22..b74a60ed0 100644
--- a/cranelift/codegen/src/machinst/isle.rs
+++ b/cranelift/codegen/src/machinst/isle.rs
@@ -838,21 +838,13 @@ macro_rules! isle_prelude_method_helpers {
             mut caller: $abicaller,
             args: ValueSlice,
         ) -> InstOutput {
-            let off = self.lower_ctx.sigs()[abi].sized_stack_arg_space()
-                + self.lower_ctx.sigs()[abi].sized_stack_ret_space();
-            self.lower_ctx
-                .abi_mut()
-                .accumulate_outgoing_args_size(off as u32);
-
-            caller.emit_stack_pre_adjust(self.lower_ctx);
-
             self.gen_call_common_args(&mut caller, args);
 
             // Handle retvals prior to emitting call, so the
             // constraints are on the call instruction; but buffer the
             // instructions till after the call.
             let mut outputs = InstOutput::new();
-            let mut retval_insts: crate::machinst::abi::SmallInstVec<_> = smallvec::smallvec![];
+            let mut retval_insts = crate::machinst::abi::SmallInstVec::new();
             // We take the *last* `num_rets` returns of the sig:
             // this skips a StructReturn, if any, that is present.
             let sigdata_num_rets = self.lower_ctx.sigs().num_rets(abi);
@@ -877,8 +869,6 @@ macro_rules! isle_prelude_method_helpers {
                 self.lower_ctx.emit(inst);
             }
 
-            caller.emit_stack_post_adjust(self.lower_ctx);
-
             outputs
         }
 
diff --git a/crates/wasi-http/src/body.rs b/crates/wasi-http/src/body.rs
index 64819e027..7cba3ae6d 100644
--- a/crates/wasi-http/src/body.rs
+++ b/crates/wasi-http/src/body.rs
@@ -82,7 +82,7 @@ pub struct HostIncomingBody {
     /// An optional worker task to keep alive while this body is being read.
     /// This ensures that if the parent of this body is dropped before the body
     /// then the backing data behind this worker is kept alive.
-    worker: Option<Arc<AbortOnDropJoinHandle<()>>>,
+    worker: Option<AbortOnDropJoinHandle<()>>,
 }
 
 enum IncomingBodyState {
@@ -117,9 +117,9 @@ impl HostIncomingBody {
         }
     }
 
-    pub fn retain_worker(&mut self, worker: &Arc<AbortOnDropJoinHandle<()>>) {
+    pub fn retain_worker(&mut self, worker: AbortOnDropJoinHandle<()>) {
         assert!(self.worker.is_none());
-        self.worker = Some(worker.clone());
+        self.worker = Some(worker);
     }
 
     pub fn take_stream(&mut self) -> Option<HostIncomingBodyStream> {
diff --git a/crates/wasi-http/src/types.rs b/crates/wasi-http/src/types.rs
index eb4f2ad0f..aa58ab605 100644
--- a/crates/wasi-http/src/types.rs
+++ b/crates/wasi-http/src/types.rs
@@ -10,7 +10,6 @@ use crate::{
 use http_body_util::BodyExt;
 use hyper::header::HeaderName;
 use std::any::Any;
-use std::sync::Arc;
 use std::time::Duration;
 use tokio::net::TcpStream;
 use tokio::time::timeout;
@@ -267,7 +266,7 @@ async fn handler(
 
     Ok(IncomingResponseInternal {
         resp,
-        worker: Arc::new(worker),
+        worker,
         between_bytes_timeout,
     })
 }
@@ -358,7 +357,6 @@ pub struct HostIncomingResponse {
     pub status: u16,
     pub headers: FieldMap,
     pub body: Option<HostIncomingBody>,
-    pub worker: Arc<AbortOnDropJoinHandle<()>>,
 }
 
 pub struct HostOutgoingResponse {
@@ -409,7 +407,7 @@ pub enum HostFields {
 
 pub struct IncomingResponseInternal {
     pub resp: hyper::Response<HyperIncomingBody>,
-    pub worker: Arc<AbortOnDropJoinHandle<()>>,
+    pub worker: AbortOnDropJoinHandle<()>,
     pub between_bytes_timeout: std::time::Duration,
 }
 
diff --git a/crates/wasi-http/src/types_impl.rs b/crates/wasi-http/src/types_impl.rs
index d8dc5f3d3..001edf48d 100644
--- a/crates/wasi-http/src/types_impl.rs
+++ b/crates/wasi-http/src/types_impl.rs
@@ -846,10 +846,9 @@ impl<T: WasiHttpView> crate::bindings::http::types::HostFutureIncomingResponse f
             headers: parts.headers,
             body: Some({
                 let mut body = HostIncomingBody::new(body, resp.between_bytes_timeout);
-                body.retain_worker(&resp.worker);
+                body.retain_worker(resp.worker);
                 body
             }),
-            worker: resp.worker,
         })?;
 
         Ok(Some(Ok(Ok(resp))))
diff --git a/crates/wasi/src/preview1.rs b/crates/wasi/src/preview1.rs
index 96bab3f88..90475a7c6 100644
--- a/crates/wasi/src/preview1.rs
+++ b/crates/wasi/src/preview1.rs
@@ -83,6 +83,7 @@ use std::ops::{Deref, DerefMut};
 use std::slice;
 use std::sync::atomic::{AtomicU64, Ordering};
 use std::sync::Arc;
+use system_interface::fs::FileIoExt;
 use wasmtime::component::Resource;
 use wiggle::tracing::instrument;
 use wiggle::{GuestError, GuestPtr, GuestStrCow, GuestType};
@@ -276,6 +277,32 @@ impl BlockingMode {
             }
         }
     }
+
+    fn block_write(
+        &self,
+        file: &super::filesystem::File,
+        bytes: GuestPtr<'_, [u8]>,
+        append: bool,
+        mut pos: u64,
+    ) -> Result<usize> {
+        let buf = bytes.as_cow()?;
+        let mut buf = buf.deref();
+
+        let mut total: usize = 0;
+        while !buf.is_empty() {
+            let nwritten = if append {
+                file.file.append(buf)
+            } else {
+                file.file.write_at(buf, pos)
+            };
+            let nwritten = nwritten.or_else(|e| Err(StreamError::LastOperationFailed(e.into())))?;
+            pos = pos.checked_add(nwritten as u64).expect("add overflow");
+            (_, buf) = buf.split_at(nwritten);
+            total = total.checked_add(nwritten).expect("add overflow");
+        }
+
+        Ok(total)
+    }
 }
 
 #[derive(Debug)]
@@ -1550,37 +1577,46 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiP1Ctx {
     ) -> Result<types::Size, types::Error> {
         let t = self.transact()?;
         let desc = t.get_descriptor(fd)?;
-        let (buf, read) = match desc {
+        match desc {
             Descriptor::File(File {
                 fd,
-                blocking_mode,
+                blocking_mode: _,
                 position,
                 ..
             }) => {
                 let fd = fd.borrowed();
-                let blocking_mode = *blocking_mode;
                 let position = position.clone();
                 drop(t);
-                let Some(buf) = first_non_empty_iovec(iovs)? else {
+                let pos = position.load(Ordering::Relaxed);
+                let file = self.table().get(&fd)?.file()?;
+                let Some(iov) = first_non_empty_iovec(iovs)? else {
                     return Ok(0);
                 };
+                let is_shared_memory = iov.is_shared_memory();
+                let bytes_read = if is_shared_memory {
+                    let mut buf = vec![0; iov.len() as usize];
+                    let bytes_read = file
+                        .file
+                        .read_at(&mut buf[..], pos)
+                        .or_else(|e| Err(StreamError::LastOperationFailed(e.into())))?;
+                    iov.copy_from_slice(&buf[0..bytes_read])?;
+                    bytes_read
+                } else {
+                    let mut buf = iov.as_slice_mut()?.expect("get none");
+                    let buf = buf.deref_mut();
+                    let bytes_read = file
+                        .file
+                        .read_at(buf, pos)
+                        .or_else(|e| Err(StreamError::LastOperationFailed(e.into())))?;
+                    bytes_read
+                };
 
-                let pos = position.load(Ordering::Relaxed);
-                let stream = self.read_via_stream(fd.borrowed(), pos).map_err(|e| {
-                    e.try_into()
-                        .context("failed to call `read-via-stream`")
-                        .unwrap_or_else(types::Error::trap)
-                })?;
-                let read = blocking_mode
-                    .read(self, stream.borrowed(), buf.len().try_into()?)
-                    .await;
-                streams::HostInputStream::drop(self, stream).map_err(|e| types::Error::trap(e))?;
-                let read = read?;
-                let n = read.len().try_into()?;
-                let pos = pos.checked_add(n).ok_or(types::Errno::Overflow)?;
+                let pos = pos
+                    .checked_add(bytes_read.try_into()?)
+                    .ok_or(types::Errno::Overflow)?;
                 position.store(pos, Ordering::Relaxed);
 
-                (buf, read)
+                Ok(bytes_read.try_into()?)
             }
             Descriptor::Stdin { stream, .. } => {
                 let stream = stream.borrowed();
@@ -1591,17 +1627,16 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiP1Ctx {
                 let read = BlockingMode::Blocking
                     .read(self, stream, buf.len().try_into()?)
                     .await?;
-                (buf, read)
+                if read.len() > buf.len().try_into()? {
+                    return Err(types::Errno::Range.into());
+                }
+                let buf = buf.get_range(0..u32::try_from(read.len())?).unwrap();
+                buf.copy_from_slice(&read)?;
+                let n = read.len().try_into()?;
+                Ok(n)
             }
             _ => return Err(types::Errno::Badf.into()),
-        };
-        if read.len() > buf.len().try_into()? {
-            return Err(types::Errno::Range.into());
         }
-        let buf = buf.get_range(0..u32::try_from(read.len())?).unwrap();
-        buf.copy_from_slice(&read)?;
-        let n = read.len().try_into()?;
-        Ok(n)
     }
 
     /// Read from a file descriptor, without using and updating the file descriptor's offset.
@@ -1670,42 +1705,26 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiP1Ctx {
                 position,
             }) => {
                 let fd = fd.borrowed();
-                let fd2 = fd.borrowed();
                 let blocking_mode = *blocking_mode;
                 let position = position.clone();
+                let pos = position.load(Ordering::Relaxed);
                 let append = *append;
                 drop(t);
+                let f = self.table().get(&fd)?.file()?;
                 let Some(buf) = first_non_empty_ciovec(ciovs)? else {
                     return Ok(0);
                 };
-                let (stream, pos) = if append {
-                    let stream = self.append_via_stream(fd).map_err(|e| {
-                        e.try_into()
-                            .context("failed to call `append-via-stream`")
-                            .unwrap_or_else(types::Error::trap)
-                    })?;
-                    (stream, 0)
-                } else {
-                    let pos = position.load(Ordering::Relaxed);
-                    let stream = self.write_via_stream(fd, pos).map_err(|e| {
-                        e.try_into()
-                            .context("failed to call `write-via-stream`")
-                            .unwrap_or_else(types::Error::trap)
-                    })?;
-                    (stream, pos)
-                };
-                let n = blocking_mode.write(self, stream.borrowed(), buf).await;
-                streams::HostOutputStream::drop(self, stream).map_err(|e| types::Error::trap(e))?;
-                let n = n?;
+                let nwritten = blocking_mode.block_write(f, buf, append, pos)?;
                 if append {
-                    let len = self.stat(fd2).await?;
+                    let len = self.stat(fd).await?;
                     position.store(len.size, Ordering::Relaxed);
                 } else {
-                    let pos = pos.checked_add(n as u64).ok_or(types::Errno::Overflow)?;
+                    let pos = pos
+                        .checked_add(nwritten as u64)
+                        .ok_or(types::Errno::Overflow)?;
                     position.store(pos, Ordering::Relaxed);
                 }
-                let n = n.try_into()?;
-                Ok(n)
+                Ok(nwritten.try_into()?)
             }
             Descriptor::Stdout { stream, .. } | Descriptor::Stderr { stream, .. } => {
                 let stream = stream.borrowed();


Test patch:
diff --git a/cranelift/filetests/filetests/isa/aarch64/call.clif b/cranelift/filetests/filetests/isa/aarch64/call.clif
index 2e9956918..def87fb8b 100644
--- a/cranelift/filetests/filetests/isa/aarch64/call.clif
+++ b/cranelift/filetests/filetests/isa/aarch64/call.clif
@@ -131,10 +131,10 @@ block0(v0: i8):
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
-; block0:
-;   movz w7, #42
 ;   sub sp, sp, #16
 ;   virtual_sp_offset_adjust 16
+; block0:
+;   movz w7, #42
 ;   strb w0, [sp]
 ;   load_ext_name x8, TestCase(%g)+0
 ;   mov x0, x7
@@ -146,7 +146,6 @@ block0(v0: i8):
 ;   mov x6, x7
 ;   blr x8
 ;   add sp, sp, #16
-;   virtual_sp_offset_adjust -16
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -154,9 +153,9 @@ block0(v0: i8):
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
-;   mov w7, #0x2a
 ;   sub sp, sp, #0x10
+; block1: ; offset 0xc
+;   mov w7, #0x2a
 ;   sturb w0, [sp]
 ;   ldr x8, #0x1c
 ;   b #0x24
@@ -697,22 +696,21 @@ block0(v0: i128, v1: i64):
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
-; block0:
-;   mov x6, x2
 ;   sub sp, sp, #16
 ;   virtual_sp_offset_adjust 16
+; block0:
+;   mov x6, x2
 ;   str x0, [sp]
 ;   mov x4, x0
 ;   str x1, [sp, #8]
 ;   mov x5, x1
-;   load_ext_name x10, TestCase(%f14)+0
+;   load_ext_name x8, TestCase(%f14)+0
 ;   mov x0, x4
 ;   mov x1, x5
 ;   mov x2, x4
 ;   mov x3, x5
-;   blr x10
+;   blr x8
 ;   add sp, sp, #16
-;   virtual_sp_offset_adjust -16
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -720,14 +718,14 @@ block0(v0: i128, v1: i64):
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
-;   mov x6, x2
 ;   sub sp, sp, #0x10
+; block1: ; offset 0xc
+;   mov x6, x2
 ;   stur x0, [sp]
 ;   mov x4, x0
 ;   stur x1, [sp, #8]
 ;   mov x5, x1
-;   ldr x10, #0x28
+;   ldr x8, #0x28
 ;   b #0x30
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %f14 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
@@ -735,7 +733,7 @@ block0(v0: i128, v1: i64):
 ;   mov x1, x5
 ;   mov x2, x4
 ;   mov x3, x5
-;   blr x10
+;   blr x8
 ;   add sp, sp, #0x10
 ;   ldp x29, x30, [sp], #0x10
 ;   ret
@@ -775,22 +773,21 @@ block0(v0: i128, v1: i64):
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
-; block0:
-;   mov x6, x2
 ;   sub sp, sp, #16
 ;   virtual_sp_offset_adjust 16
+; block0:
+;   mov x6, x2
 ;   str x0, [sp]
 ;   mov x4, x0
 ;   str x1, [sp, #8]
 ;   mov x5, x1
-;   load_ext_name x10, TestCase(%f15)+0
+;   load_ext_name x8, TestCase(%f15)+0
 ;   mov x0, x4
 ;   mov x1, x5
 ;   mov x2, x4
 ;   mov x3, x5
-;   blr x10
+;   blr x8
 ;   add sp, sp, #16
-;   virtual_sp_offset_adjust -16
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -798,14 +795,14 @@ block0(v0: i128, v1: i64):
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
-;   mov x6, x2
 ;   sub sp, sp, #0x10
+; block1: ; offset 0xc
+;   mov x6, x2
 ;   stur x0, [sp]
 ;   mov x4, x0
 ;   stur x1, [sp, #8]
 ;   mov x5, x1
-;   ldr x10, #0x28
+;   ldr x8, #0x28
 ;   b #0x30
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %f15 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
@@ -813,7 +810,7 @@ block0(v0: i128, v1: i64):
 ;   mov x1, x5
 ;   mov x2, x4
 ;   mov x3, x5
-;   blr x10
+;   blr x8
 ;   add sp, sp, #0x10
 ;   ldp x29, x30, [sp], #0x10
 ;   ret
diff --git a/cranelift/filetests/filetests/isa/aarch64/tail-call-conv.clif b/cranelift/filetests/filetests/isa/aarch64/tail-call-conv.clif
index 40e1917bb..64b8b4f4e 100644
--- a/cranelift/filetests/filetests/isa/aarch64/tail-call-conv.clif
+++ b/cranelift/filetests/filetests/isa/aarch64/tail-call-conv.clif
@@ -66,6 +66,8 @@ block0:
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
+;   sub sp, sp, #16
+;   virtual_sp_offset_adjust 16
 ; block0:
 ;   movz x2, #10
 ;   movz x3, #15
@@ -93,12 +95,13 @@ block0:
 ;   movz x28, #125
 ;   movz x0, #130
 ;   movz x1, #135
-;   sub sp, sp, #16
-;   virtual_sp_offset_adjust 16
 ;   str x0, [sp]
 ;   str x1, [sp, #8]
 ;   load_ext_name x1, TestCase(%tail_callee_stack_args)+0
 ;   blr x1
+;   sub sp, sp, #16
+;   virtual_sp_offset_adjust 16
+;   add sp, sp, #16
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -106,7 +109,8 @@ block0:
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
+;   sub sp, sp, #0x10
+; block1: ; offset 0xc
 ;   mov x2, #0xa
 ;   mov x3, #0xf
 ;   mov x4, #0x14
@@ -133,7 +137,6 @@ block0:
 ;   mov x28, #0x7d
 ;   mov x0, #0x82
 ;   mov x1, #0x87
-;   sub sp, sp, #0x10
 ;   stur x0, [sp]
 ;   stur x1, [sp, #8]
 ;   ldr x1, #0x84
@@ -141,6 +144,8 @@ block0:
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %tail_callee_stack_args 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
 ;   blr x1
+;   sub sp, sp, #0x10
+;   add sp, sp, #0x10
 ;   ldp x29, x30, [sp], #0x10
 ;   ret
 
@@ -267,16 +272,15 @@ block0:
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
-; block0:
 ;   sub sp, sp, #16
 ;   virtual_sp_offset_adjust 16
+; block0:
 ;   mov x0, sp
 ;   load_ext_name x1, TestCase(%tail_callee_stack_rets)+0
 ;   blr x1
-;   ldr x13, [sp]
+;   ldr x11, [sp]
 ;   ldr x2, [sp, #8]
 ;   add sp, sp, #16
-;   virtual_sp_offset_adjust -16
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -284,15 +288,15 @@ block0:
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
 ;   sub sp, sp, #0x10
+; block1: ; offset 0xc
 ;   mov x0, sp
 ;   ldr x1, #0x18
 ;   b #0x20
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %tail_callee_stack_rets 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
 ;   blr x1
-;   ldur x13, [sp]
+;   ldur x11, [sp]
 ;   ldur x2, [sp, #8]
 ;   add sp, sp, #0x10
 ;   ldp x29, x30, [sp], #0x10
@@ -376,6 +380,8 @@ block0:
 ; VCode:
 ;   stp fp, lr, [sp, #-16]!
 ;   mov fp, sp
+;   sub sp, sp, #32
+;   virtual_sp_offset_adjust 32
 ; block0:
 ;   movz x2, #10
 ;   movz x3, #15
@@ -403,17 +409,16 @@ block0:
 ;   movz x28, #125
 ;   movz x0, #130
 ;   movz x1, #135
-;   sub sp, sp, #32
-;   virtual_sp_offset_adjust 32
 ;   str x0, [sp]
 ;   str x1, [sp, #8]
 ;   add x0, sp, #16
 ;   load_ext_name x1, TestCase(%tail_callee_stack_args_and_rets)+0
 ;   blr x1
-;   ldr x9, [sp]
-;   ldr x2, [sp, #8]
-;   add sp, sp, #16
-;   virtual_sp_offset_adjust -16
+;   sub sp, sp, #16
+;   virtual_sp_offset_adjust 16
+;   ldr x9, [sp, #16]
+;   ldr x2, [sp, #24]
+;   add sp, sp, #32
 ;   ldp fp, lr, [sp], #16
 ;   ret
 ;
@@ -421,7 +426,8 @@ block0:
 ; block0: ; offset 0x0
 ;   stp x29, x30, [sp, #-0x10]!
 ;   mov x29, sp
-; block1: ; offset 0x8
+;   sub sp, sp, #0x20
+; block1: ; offset 0xc
 ;   mov x2, #0xa
 ;   mov x3, #0xf
 ;   mov x4, #0x14
@@ -448,7 +454,6 @@ block0:
 ;   mov x28, #0x7d
 ;   mov x0, #0x82
 ;   mov x1, #0x87
-;   sub sp, sp, #0x20
 ;   stur x0, [sp]
 ;   stur x1, [sp, #8]
 ;   add x0, sp, #0x10
@@ -457,9 +462,10 @@ block0:
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %tail_callee_stack_args_and_rets 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
 ;   blr x1
-;   ldur x9, [sp]
-;   ldur x2, [sp, #8]
-;   add sp, sp, #0x10
+;   sub sp, sp, #0x10
+;   ldur x9, [sp, #0x10]
+;   ldur x2, [sp, #0x18]
+;   add sp, sp, #0x20
 ;   ldp x29, x30, [sp], #0x10
 ;   ret
 
diff --git a/cranelift/filetests/filetests/isa/riscv64/call.clif b/cranelift/filetests/filetests/isa/riscv64/call.clif
index 2c11c7712..448d100fc 100644
--- a/cranelift/filetests/filetests/isa/riscv64/call.clif
+++ b/cranelift/filetests/filetests/isa/riscv64/call.clif
@@ -171,15 +171,14 @@ block0(v0: i8):
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-;   sd s3,-8(sp)
-;   addi sp,sp,-16
+;   sd s1,-8(sp)
+;   addi sp,sp,-32
+;   virtual_sp_offset_adj +16
 ; block0:
 ;   li a7,42
-;   addi sp,sp,-16
-;   virtual_sp_offset_adj +16
-;   slli a5,a0,56; srai a5,a5,56
-;   sd a5,0(sp)
-;   load_sym s3,%g+0
+;   slli a3,a0,56; srai a3,a3,56
+;   sd a3,0(sp)
+;   load_sym s1,%g+0
 ;   mv a0,a7
 ;   mv a1,a7
 ;   mv a2,a7
@@ -187,11 +186,9 @@ block0(v0: i8):
 ;   mv a4,a7
 ;   mv a5,a7
 ;   mv a6,a7
-;   callind s3
-;   addi sp,sp,16
-;   virtual_sp_offset_adj -16
-;   addi sp,sp,16
-;   ld s3,-8(sp)
+;   callind s1
+;   addi sp,sp,32
+;   ld s1,-8(sp)
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -203,16 +200,15 @@ block0(v0: i8):
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-;   sd s3, -8(sp)
-;   addi sp, sp, -0x10
+;   sd s1, -8(sp)
+;   addi sp, sp, -0x20
 ; block1: ; offset 0x18
 ;   addi a7, zero, 0x2a
-;   addi sp, sp, -0x10
-;   slli a5, a0, 0x38
-;   srai a5, a5, 0x38
-;   sd a5, 0(sp)
-;   auipc s3, 0
-;   ld s3, 0xc(s3)
+;   slli a3, a0, 0x38
+;   srai a3, a3, 0x38
+;   sd a3, 0(sp)
+;   auipc s1, 0
+;   ld s1, 0xc(s1)
 ;   j 0xc
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %g 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
@@ -223,10 +219,9 @@ block0(v0: i8):
 ;   mv a4, a7
 ;   mv a5, a7
 ;   mv a6, a7
-;   jalr s3
-;   addi sp, sp, 0x10
-;   addi sp, sp, 0x10
-;   ld s3, -8(sp)
+;   jalr s1
+;   addi sp, sp, 0x20
+;   ld s1, -8(sp)
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -630,26 +625,23 @@ block0(v0: i128, v1: i64):
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-;   sd s3,-8(sp)
-;   addi sp,sp,-16
+;   sd s1,-8(sp)
+;   addi sp,sp,-32
+;   virtual_sp_offset_adj +16
 ; block0:
 ;   mv a6,a2
 ;   mv a7,a0
-;   addi sp,sp,-16
-;   virtual_sp_offset_adj +16
 ;   sd a1,0(sp)
 ;   mv a5,a1
-;   load_sym s3,%f14+0
+;   load_sym s1,%f14+0
 ;   mv a0,a7
 ;   mv a1,a5
 ;   mv a2,a7
 ;   mv a3,a5
 ;   mv a4,a7
-;   callind s3
-;   addi sp,sp,16
-;   virtual_sp_offset_adj -16
-;   addi sp,sp,16
-;   ld s3,-8(sp)
+;   callind s1
+;   addi sp,sp,32
+;   ld s1,-8(sp)
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -661,16 +653,15 @@ block0(v0: i128, v1: i64):
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-;   sd s3, -8(sp)
-;   addi sp, sp, -0x10
+;   sd s1, -8(sp)
+;   addi sp, sp, -0x20
 ; block1: ; offset 0x18
 ;   mv a6, a2
 ;   mv a7, a0
-;   addi sp, sp, -0x10
 ;   sd a1, 0(sp)
 ;   mv a5, a1
-;   auipc s3, 0
-;   ld s3, 0xc(s3)
+;   auipc s1, 0
+;   ld s1, 0xc(s1)
 ;   j 0xc
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %f14 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
@@ -679,10 +670,9 @@ block0(v0: i128, v1: i64):
 ;   mv a2, a7
 ;   mv a3, a5
 ;   mv a4, a7
-;   jalr s3
-;   addi sp, sp, 0x10
-;   addi sp, sp, 0x10
-;   ld s3, -8(sp)
+;   jalr s1
+;   addi sp, sp, 0x20
+;   ld s1, -8(sp)
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -733,26 +723,23 @@ block0(v0: i128, v1: i64):
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-;   sd s3,-8(sp)
-;   addi sp,sp,-16
+;   sd s1,-8(sp)
+;   addi sp,sp,-32
+;   virtual_sp_offset_adj +16
 ; block0:
 ;   mv a6,a2
 ;   mv a7,a0
-;   addi sp,sp,-16
-;   virtual_sp_offset_adj +16
 ;   sd a1,0(sp)
 ;   mv a5,a1
-;   load_sym s3,%f15+0
+;   load_sym s1,%f15+0
 ;   mv a0,a7
 ;   mv a1,a5
 ;   mv a2,a7
 ;   mv a3,a5
 ;   mv a4,a7
-;   callind s3
-;   addi sp,sp,16
-;   virtual_sp_offset_adj -16
-;   addi sp,sp,16
-;   ld s3,-8(sp)
+;   callind s1
+;   addi sp,sp,32
+;   ld s1,-8(sp)
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -764,16 +751,15 @@ block0(v0: i128, v1: i64):
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-;   sd s3, -8(sp)
-;   addi sp, sp, -0x10
+;   sd s1, -8(sp)
+;   addi sp, sp, -0x20
 ; block1: ; offset 0x18
 ;   mv a6, a2
 ;   mv a7, a0
-;   addi sp, sp, -0x10
 ;   sd a1, 0(sp)
 ;   mv a5, a1
-;   auipc s3, 0
-;   ld s3, 0xc(s3)
+;   auipc s1, 0
+;   ld s1, 0xc(s1)
 ;   j 0xc
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %f15 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
@@ -782,10 +768,9 @@ block0(v0: i128, v1: i64):
 ;   mv a2, a7
 ;   mv a3, a5
 ;   mv a4, a7
-;   jalr s3
-;   addi sp, sp, 0x10
-;   addi sp, sp, 0x10
-;   ld s3, -8(sp)
+;   jalr s1
+;   addi sp, sp, 0x20
+;   ld s1, -8(sp)
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
diff --git a/cranelift/filetests/filetests/isa/riscv64/tail-call-conv.clif b/cranelift/filetests/filetests/isa/riscv64/tail-call-conv.clif
index d44b697bc..25e41c37d 100644
--- a/cranelift/filetests/filetests/isa/riscv64/tail-call-conv.clif
+++ b/cranelift/filetests/filetests/isa/riscv64/tail-call-conv.clif
@@ -82,7 +82,8 @@ block0:
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-;   addi sp,sp,-16
+;   addi sp,sp,-64
+;   virtual_sp_offset_adj +48
 ; block0:
 ;   li s1,10
 ;   sd s1,8(nominal_sp)
@@ -112,8 +113,6 @@ block0:
 ;   li t2,125
 ;   li s1,130
 ;   li a0,135
-;   addi sp,sp,-48
-;   virtual_sp_offset_adj +48
 ;   sd t0,0(sp)
 ;   sd t1,8(sp)
 ;   sd t2,16(sp)
@@ -123,7 +122,9 @@ block0:
 ;   ld s1,8(nominal_sp)
 ;   ld a0,0(nominal_sp)
 ;   callind t0
-;   addi sp,sp,16
+;   addi sp,sp,-48
+;   virtual_sp_offset_adj +48
+;   addi sp,sp,64
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -135,12 +136,12 @@ block0:
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-;   addi sp, sp, -0x10
+;   addi sp, sp, -0x40
 ; block1: ; offset 0x14
 ;   addi s1, zero, 0xa
-;   sd s1, 8(sp)
+;   sd s1, 0x38(sp)
 ;   addi a0, zero, 0xf
-;   sd a0, 0(sp)
+;   sd a0, 0x30(sp)
 ;   addi a1, zero, 0x14
 ;   addi a2, zero, 0x19
 ;   addi a3, zero, 0x1e
@@ -165,7 +166,6 @@ block0:
 ;   addi t2, zero, 0x7d
 ;   addi s1, zero, 0x82
 ;   addi a0, zero, 0x87
-;   addi sp, sp, -0x30
 ;   sd t0, 0(sp)
 ;   sd t1, 8(sp)
 ;   sd t2, 0x10(sp)
@@ -179,7 +179,8 @@ block0:
 ;   ld s1, 0x38(sp)
 ;   ld a0, 0x30(sp)
 ;   jalr t0
-;   addi sp, sp, 0x10
+;   addi sp, sp, -0x30
+;   addi sp, sp, 0x40
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
@@ -332,19 +333,18 @@ block0:
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-; block0:
 ;   addi sp,sp,-48
 ;   virtual_sp_offset_adj +48
+; block0:
 ;   load_addr s1,0(sp)
 ;   load_sym t0,%tail_callee_stack_rets+0
 ;   callind t0
-;   ld a2,0(sp)
-;   ld a4,8(sp)
-;   ld a0,16(sp)
-;   ld a2,24(sp)
+;   ld a0,0(sp)
+;   ld a2,8(sp)
+;   ld a4,16(sp)
+;   ld a0,24(sp)
 ;   ld s1,32(sp)
 ;   addi sp,sp,48
-;   virtual_sp_offset_adj -48
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -356,8 +356,8 @@ block0:
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-; block1: ; offset 0x10
 ;   addi sp, sp, -0x30
+; block1: ; offset 0x14
 ;   mv s1, sp
 ;   auipc t0, 0
 ;   ld t0, 0xc(t0)
@@ -365,10 +365,10 @@ block0:
 ;   .byte 0x00, 0x00, 0x00, 0x00 ; reloc_external Abs8 %tail_callee_stack_rets 0
 ;   .byte 0x00, 0x00, 0x00, 0x00
 ;   jalr t0
-;   ld a2, 0(sp)
-;   ld a4, 8(sp)
-;   ld a0, 0x10(sp)
-;   ld a2, 0x18(sp)
+;   ld a0, 0(sp)
+;   ld a2, 8(sp)
+;   ld a4, 0x10(sp)
+;   ld a0, 0x18(sp)
 ;   ld s1, 0x20(sp)
 ;   addi sp, sp, 0x30
 ;   ld ra, 8(sp)
@@ -486,7 +486,8 @@ block0:
 ;   sd ra,8(sp)
 ;   sd fp,0(sp)
 ;   mv fp,sp
-;   addi sp,sp,-16
+;   addi sp,sp,-112
+;   virtual_sp_offset_adj +96
 ; block0:
 ;   li s1,10
 ;   sd s1,8(nominal_sp)
@@ -516,8 +517,6 @@ block0:
 ;   li t2,125
 ;   li s1,130
 ;   li a0,135
-;   addi sp,sp,-96
-;   virtual_sp_offset_adj +96
 ;   sd t0,0(sp)
 ;   sd t1,8(sp)
 ;   sd t2,16(sp)
@@ -529,14 +528,14 @@ block0:
 ;   ld s1,8(nominal_sp)
 ;   ld a0,0(nominal_sp)
 ;   callind t0
-;   ld a4,0(sp)
-;   ld a0,8(sp)
-;   ld a2,16(sp)
-;   ld a4,24(sp)
-;   ld s1,32(sp)
-;   addi sp,sp,48
-;   virtual_sp_offset_adj -48
-;   addi sp,sp,16
+;   addi sp,sp,-48
+;   virtual_sp_offset_adj +48
+;   ld a4,48(sp)
+;   ld a0,56(sp)
+;   ld a2,64(sp)
+;   ld a4,72(sp)
+;   ld s1,80(sp)
+;   addi sp,sp,112
 ;   ld ra,8(sp)
 ;   ld fp,0(sp)
 ;   addi sp,sp,16
@@ -548,12 +547,12 @@ block0:
 ;   sd ra, 8(sp)
 ;   sd s0, 0(sp)
 ;   mv s0, sp
-;   addi sp, sp, -0x10
+;   addi sp, sp, -0x70
 ; block1: ; offset 0x14
 ;   addi s1, zero, 0xa
-;   sd s1, 8(sp)
+;   sd s1, 0x68(sp)
 ;   addi a0, zero, 0xf
-;   sd a0, 0(sp)
+;   sd a0, 0x60(sp)
 ;   addi a1, zero, 0x14
 ;   addi a2, zero, 0x19
 ;   addi a3, zero, 0x1e
@@ -578,7 +577,6 @@ block0:
 ;   addi t2, zero, 0x7d
 ;   addi s1, zero, 0x82
 ;   addi a0, zero, 0x87
-;   addi sp, sp, -0x60
 ;   sd t0, 0(sp)
 ;   sd t1, 8(sp)
 ;   sd t2, 0x10(sp)
@@ -594,13 +592,13 @@ block0:
 ;   ld s1, 0x68(sp)
 ;   ld a0, 0x60(sp)
 ;   jalr t0
-;   ld a4, 0(sp)
-;   ld a0, 8(sp)
-;   ld a2, 0x10(sp)
-;   ld a4, 0x18(sp)
-;   ld s1, 0x20(sp)
-;   addi sp, sp, 0x30
-;   addi sp, sp, 0x10
+;   addi sp, sp, -0x30
+;   ld a4, 0x30(sp)
+;   ld a0, 0x38(sp)
+;   ld a2, 0x40(sp)
+;   ld a4, 0x48(sp)
+;   ld s1, 0x50(sp)
+;   addi sp, sp, 0x70
 ;   ld ra, 8(sp)
 ;   ld s0, 0(sp)
 ;   addi sp, sp, 0x10
diff --git a/cranelift/filetests/filetests/isa/x64/call-conv.clif b/cranelift/filetests/filetests/isa/x64/call-conv.clif
index 33d93d834..c3fb6f152 100644
--- a/cranelift/filetests/filetests/isa/x64/call-conv.clif
+++ b/cranelift/filetests/filetests/isa/x64/call-conv.clif
@@ -12,13 +12,12 @@ block0(v0: i32):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movq    %rdi, %rcx
 ;   subq    %rsp, $32, %rsp
 ;   virtual_sp_offset_adjust 32
+; block0:
+;   movq    %rdi, %rcx
 ;   call    *%rcx
 ;   addq    %rsp, $32, %rsp
-;   virtual_sp_offset_adjust -32
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -27,9 +26,9 @@ block0(v0: i32):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movq %rdi, %rcx
 ;   subq $0x20, %rsp
+; block1: ; offset 0x8
+;   movq %rdi, %rcx
 ;   callq *%rcx
 ;   addq $0x20, %rsp
 ;   movq %rbp, %rsp
@@ -49,17 +48,16 @@ block0(v0: i32, v1: f32):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movdqa  %xmm0, %xmm6
 ;   subq    %rsp, $32, %rsp
 ;   virtual_sp_offset_adjust 32
+; block0:
+;   movdqa  %xmm0, %xmm6
 ;   movq    %rdi, %rcx
 ;   movdqa  %xmm6, %xmm1
 ;   call    *%rdi
-;   addq    %rsp, $32, %rsp
-;   virtual_sp_offset_adjust -32
 ;   movdqa  %xmm6, %xmm0
 ;   call    *%rdi
+;   addq    %rsp, $32, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -68,15 +66,15 @@ block0(v0: i32, v1: f32):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movdqa %xmm0, %xmm6
 ;   subq $0x20, %rsp
+; block1: ; offset 0x8
+;   movdqa %xmm0, %xmm6
 ;   movq %rdi, %rcx
 ;   movdqa %xmm6, %xmm1
 ;   callq *%rdi
-;   addq $0x20, %rsp
 ;   movdqa %xmm6, %xmm0
 ;   callq *%rdi
+;   addq $0x20, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
@@ -194,6 +192,8 @@ block0(
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
+;   subq    %rsp, $144, %rsp
+;   virtual_sp_offset_adjust 144
 ; block0:
 ;   movq    %rcx, %rax
 ;   movq    %rdx, %rcx
@@ -204,8 +204,6 @@ block0(
 ;   movq    24(%rbp), %r11
 ;   movss   32(%rbp), %xmm11
 ;   movsd   40(%rbp), %xmm13
-;   subq    %rsp, $144, %rsp
-;   virtual_sp_offset_adjust 144
 ;   movq    %r8, 32(%rsp)
 ;   movq    %r9, 40(%rsp)
 ;   movsd   %xmm0, 48(%rsp)
@@ -225,7 +223,6 @@ block0(
 ;   movq    %rsi, %rcx
 ;   call    *%rcx
 ;   addq    %rsp, $144, %rsp
-;   virtual_sp_offset_adjust -144
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -234,7 +231,8 @@ block0(
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
+;   subq $0x90, %rsp
+; block1: ; offset 0xb
 ;   movq %rcx, %rax
 ;   movq %rdx, %rcx
 ;   movq %rsi, %rdx
@@ -244,7 +242,6 @@ block0(
 ;   movq 0x18(%rbp), %r11
 ;   movss 0x20(%rbp), %xmm11
 ;   movsd 0x28(%rbp), %xmm13
-;   subq $0x90, %rsp
 ;   movq %r8, 0x20(%rsp)
 ;   movq %r9, 0x28(%rsp)
 ;   movsd %xmm0, 0x30(%rsp)
@@ -283,18 +280,17 @@ block0(v0: i64, v1:i64, v2:i64, v3:i64, v4:i64):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
+;   subq    %rsp, $48, %rsp
+;   virtual_sp_offset_adjust 48
 ; block0:
 ;   movq    %rdx, %r11
 ;   movq    %rcx, %r9
 ;   movq    %rsi, %rdx
 ;   movq    %rdi, %rcx
-;   subq    %rsp, $48, %rsp
-;   virtual_sp_offset_adjust 48
 ;   movq    %r8, 32(%rsp)
 ;   movq    %r11, %r8
 ;   call    *%rcx
 ;   addq    %rsp, $48, %rsp
-;   virtual_sp_offset_adjust -48
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -303,12 +299,12 @@ block0(v0: i64, v1:i64, v2:i64, v3:i64, v4:i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
+;   subq $0x30, %rsp
+; block1: ; offset 0x8
 ;   movq %rdx, %r11
 ;   movq %rcx, %r9
 ;   movq %rsi, %rdx
 ;   movq %rdi, %rcx
-;   subq $0x30, %rsp
 ;   movq %r8, 0x20(%rsp)
 ;   movq %r11, %r8
 ;   callq *%rcx
@@ -327,13 +323,13 @@ block0(v0: i32, v1: f32, v2: i64, v3: f64, v4: i32, v5: i32, v6: i32, v7: f32, v
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
+;   subq    %rsp, $96, %rsp
+;   virtual_sp_offset_adjust 96
 ; block0:
 ;   movq    %rsi, %r9
 ;   movq    %rdi, %rsi
 ;   movdqa  %xmm1, %xmm6
 ;   movdqa  %xmm0, %xmm1
-;   subq    %rsp, $96, %rsp
-;   virtual_sp_offset_adjust 96
 ;   movl    %edx, 32(%rsp)
 ;   movl    %ecx, 40(%rsp)
 ;   movl    %r8d, 48(%rsp)
@@ -346,7 +342,6 @@ block0(v0: i32, v1: f32, v2: i64, v3: f64, v4: i32, v5: i32, v6: i32, v7: f32, v
 ;   movdqa  %xmm6, %xmm3
 ;   call    *%rcx
 ;   addq    %rsp, $96, %rsp
-;   virtual_sp_offset_adjust -96
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -355,12 +350,12 @@ block0(v0: i32, v1: f32, v2: i64, v3: f64, v4: i32, v5: i32, v6: i32, v7: f32, v
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
+;   subq $0x60, %rsp
+; block1: ; offset 0x8
 ;   movq %rsi, %r9
 ;   movq %rdi, %rsi
 ;   movdqa %xmm1, %xmm6
 ;   movdqa %xmm0, %xmm1
-;   subq $0x60, %rsp
 ;   movl %edx, 0x20(%rsp)
 ;   movl %ecx, 0x28(%rsp)
 ;   movl %r8d, 0x30(%rsp)
@@ -387,14 +382,13 @@ block0(v0: i32, v1: i8x16):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
 ;   subq    %rsp, $48, %rsp
 ;   virtual_sp_offset_adjust 48
+; block0:
 ;   lea     32(%rsp), %rcx
 ;   movdqu  %xmm0, 0(%rcx)
 ;   call    *%rdi
 ;   addq    %rsp, $48, %rsp
-;   virtual_sp_offset_adjust -48
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -403,8 +397,8 @@ block0(v0: i32, v1: i8x16):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
 ;   subq $0x30, %rsp
+; block1: ; offset 0x8
 ;   leaq 0x20(%rsp), %rcx
 ;   movdqu %xmm0, (%rcx)
 ;   callq *%rdi
@@ -424,19 +418,18 @@ block0(v0: i32, v1: i32, v2: i8x16, v3: i64, v4: i8x16):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
+;   subq    %rsp, $64, %rsp
+;   virtual_sp_offset_adjust 64
 ; block0:
 ;   movq    %rsi, %rcx
 ;   movq    %rdx, %r8
-;   subq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust 64
 ;   lea     32(%rsp), %rdx
 ;   movdqu  %xmm0, 0(%rdx)
 ;   lea     48(%rsp), %r9
 ;   movdqu  %xmm1, 0(%r9)
 ;   call    *%rdi
-;   addq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust -64
 ;   paddb   %xmm0, %xmm0, %xmm0
+;   addq    %rsp, $64, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -445,17 +438,17 @@ block0(v0: i32, v1: i32, v2: i8x16, v3: i64, v4: i8x16):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
+;   subq $0x40, %rsp
+; block1: ; offset 0x8
 ;   movq %rsi, %rcx
 ;   movq %rdx, %r8
-;   subq $0x40, %rsp
 ;   leaq 0x20(%rsp), %rdx
 ;   movdqu %xmm0, (%rdx)
 ;   leaq 0x30(%rsp), %r9
 ;   movdqu %xmm1, (%r9)
 ;   callq *%rdi
-;   addq $0x40, %rsp
 ;   paddb %xmm0, %xmm0
+;   addq $0x40, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
@@ -470,19 +463,18 @@ block0(v0: i32, v1: i8x16):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movq    %rdi, %r9
 ;   subq    %rsp, $64, %rsp
 ;   virtual_sp_offset_adjust 64
-;   lea     48(%rsp), %r8
-;   movdqu  %xmm0, 0(%r8)
-;   movq    %r8, 32(%rsp)
+; block0:
+;   movq    %rdi, %r9
+;   lea     48(%rsp), %rcx
+;   movdqu  %xmm0, 0(%rcx)
+;   movq    %rcx, 32(%rsp)
 ;   movq    %r9, %rcx
 ;   movq    %r9, %rdx
 ;   movq    %r9, %r8
 ;   call    *%r9
 ;   addq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust -64
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -491,12 +483,12 @@ block0(v0: i32, v1: i8x16):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movq %rdi, %r9
 ;   subq $0x40, %rsp
-;   leaq 0x30(%rsp), %r8
-;   movdqu %xmm0, (%r8)
-;   movq %r8, 0x20(%rsp)
+; block1: ; offset 0x8
+;   movq %rdi, %r9
+;   leaq 0x30(%rsp), %rcx
+;   movdqu %xmm0, (%rcx)
+;   movq %rcx, 0x20(%rsp)
 ;   movq %r9, %rcx
 ;   movq %r9, %rdx
 ;   movq %r9, %r8
@@ -516,22 +508,21 @@ block0(v0: i32, v1: i8x16):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movq    %rdi, %r9
 ;   subq    %rsp, $80, %rsp
 ;   virtual_sp_offset_adjust 80
-;   lea     48(%rsp), %r8
-;   movdqu  %xmm0, 0(%r8)
-;   movq    %r8, 32(%rsp)
-;   lea     64(%rsp), %rsi
-;   movdqu  %xmm0, 0(%rsi)
-;   movq    %rsi, 40(%rsp)
+; block0:
+;   movq    %rdi, %r9
+;   lea     48(%rsp), %rcx
+;   movdqu  %xmm0, 0(%rcx)
+;   movq    %rcx, 32(%rsp)
+;   lea     64(%rsp), %r10
+;   movdqu  %xmm0, 0(%r10)
+;   movq    %r10, 40(%rsp)
 ;   movq    %r9, %rcx
 ;   movq    %r9, %rdx
 ;   movq    %r9, %r8
 ;   call    *%r9
 ;   addq    %rsp, $80, %rsp
-;   virtual_sp_offset_adjust -80
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -540,15 +531,15 @@ block0(v0: i32, v1: i8x16):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movq %rdi, %r9
 ;   subq $0x50, %rsp
-;   leaq 0x30(%rsp), %r8
-;   movdqu %xmm0, (%r8)
-;   movq %r8, 0x20(%rsp)
-;   leaq 0x40(%rsp), %rsi
-;   movdqu %xmm0, (%rsi)
-;   movq %rsi, 0x28(%rsp)
+; block1: ; offset 0x8
+;   movq %rdi, %r9
+;   leaq 0x30(%rsp), %rcx
+;   movdqu %xmm0, (%rcx)
+;   movq %rcx, 0x20(%rsp)
+;   leaq 0x40(%rsp), %r10
+;   movdqu %xmm0, (%r10)
+;   movq %r10, 0x28(%rsp)
 ;   movq %r9, %rcx
 ;   movq %r9, %rdx
 ;   movq %r9, %r8
@@ -568,23 +559,23 @@ block0(v0: i32, v1: i8x16):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movq    %rdi, %r9
 ;   subq    %rsp, $96, %rsp
 ;   virtual_sp_offset_adjust 96
+; block0:
+;   movq    %rdi, %r10
 ;   lea     48(%rsp), %rdx
 ;   movdqu  %xmm0, 0(%rdx)
-;   lea     64(%rsp), %r11
-;   movdqu  %xmm0, 0(%r11)
-;   movq    %r11, 32(%rsp)
-;   lea     80(%rsp), %rcx
-;   movdqu  %xmm0, 0(%rcx)
-;   movq    %rcx, 40(%rsp)
+;   lea     64(%rsp), %r9
+;   movdqu  %xmm0, 0(%r9)
+;   movq    %r9, 32(%rsp)
+;   lea     80(%rsp), %rdi
+;   movdqu  %xmm0, 0(%rdi)
+;   movq    %rdi, 40(%rsp)
+;   movq    %r10, %r9
 ;   movq    %r9, %rcx
 ;   movq    %r9, %r8
 ;   call    *%r9
 ;   addq    %rsp, $96, %rsp
-;   virtual_sp_offset_adjust -96
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -593,17 +584,18 @@ block0(v0: i32, v1: i8x16):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movq %rdi, %r9
 ;   subq $0x60, %rsp
+; block1: ; offset 0x8
+;   movq %rdi, %r10
 ;   leaq 0x30(%rsp), %rdx
 ;   movdqu %xmm0, (%rdx)
-;   leaq 0x40(%rsp), %r11
-;   movdqu %xmm0, (%r11)
-;   movq %r11, 0x20(%rsp)
-;   leaq 0x50(%rsp), %rcx
-;   movdqu %xmm0, (%rcx)
-;   movq %rcx, 0x28(%rsp)
+;   leaq 0x40(%rsp), %r9
+;   movdqu %xmm0, (%r9)
+;   movq %r9, 0x20(%rsp)
+;   leaq 0x50(%rsp), %rdi
+;   movdqu %xmm0, (%rdi)
+;   movq %rdi, 0x28(%rsp)
+;   movq %r10, %r9
 ;   movq %r9, %rcx
 ;   movq %r9, %r8
 ;   callq *%r9
diff --git a/cranelift/filetests/filetests/isa/x64/fastcall.clif b/cranelift/filetests/filetests/isa/x64/fastcall.clif
index d83c9fbee..b68c8edf8 100644
--- a/cranelift/filetests/filetests/isa/x64/fastcall.clif
+++ b/cranelift/filetests/filetests/isa/x64/fastcall.clif
@@ -241,27 +241,21 @@ block0(v0: i64):
 ;   pushq   %rbp
 ;   unwind PushFrameRegs { offset_upward_to_caller_sp: 16 }
 ;   movq    %rsp, %rbp
-;   unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 16 }
-;   subq    %rsp, $16, %rsp
-;   movq    %rdi, 0(%rsp)
-;   unwind SaveReg { clobber_offset: 0, reg: p7i }
+;   unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 }
+;   subq    %rsp, $48, %rsp
+;   virtual_sp_offset_adjust 48
 ; block0:
 ;   uninit  %xmm3
 ;   xorpd   %xmm3, %xmm3, %xmm3
 ;   cvtsi2sd %xmm3, %rcx, %xmm3
-;   subq    %rsp, $48, %rsp
-;   virtual_sp_offset_adjust 48
 ;   movq    %rcx, 32(%rsp)
 ;   movq    %rcx, 40(%rsp)
 ;   movq    %rcx, %rdx
-;   load_ext_name %g+0, %rdi
+;   load_ext_name %g+0, %r11
 ;   movq    %rdx, %rcx
 ;   movdqa  %xmm3, %xmm2
-;   call    *%rdi
+;   call    *%r11
 ;   addq    %rsp, $48, %rsp
-;   virtual_sp_offset_adjust -48
-;   movq    0(%rsp), %rdi
-;   addq    %rsp, $16, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -270,22 +264,18 @@ block0(v0: i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x10, %rsp
-;   movq %rdi, (%rsp)
-; block1: ; offset 0xc
+;   subq $0x30, %rsp
+; block1: ; offset 0x8
 ;   xorpd %xmm3, %xmm3
 ;   cvtsi2sdq %rcx, %xmm3
-;   subq $0x30, %rsp
 ;   movq %rcx, 0x20(%rsp)
 ;   movq %rcx, 0x28(%rsp)
 ;   movq %rcx, %rdx
-;   movabsq $0, %rdi ; reloc_external Abs8 %g 0
+;   movabsq $0, %r11 ; reloc_external Abs8 %g 0
 ;   movq %rdx, %rcx
 ;   movdqa %xmm3, %xmm2
-;   callq *%rdi
+;   callq *%r11
 ;   addq $0x30, %rsp
-;   movq (%rsp), %rdi
-;   addq $0x10, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
diff --git a/cranelift/filetests/filetests/isa/x64/i128.clif b/cranelift/filetests/filetests/isa/x64/i128.clif
index 5310d3a4b..57354ce2c 100644
--- a/cranelift/filetests/filetests/isa/x64/i128.clif
+++ b/cranelift/filetests/filetests/isa/x64/i128.clif
@@ -1411,22 +1411,19 @@ block0(v0: i128, v1: i128):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $16, %rsp
-;   movq    %r13, 0(%rsp)
+;   subq    %rsp, $32, %rsp
+;   virtual_sp_offset_adjust 16
+;   movq    %r13, 16(%rsp)
 ; block0:
 ;   movq    %r8, %r13
-;   subq    %rsp, $16, %rsp
-;   virtual_sp_offset_adjust 16
 ;   lea     0(%rsp), %r8
 ;   load_ext_name %g+0, %r9
 ;   call    *%r9
 ;   movq    0(%rsp), %r8
-;   addq    %rsp, $16, %rsp
-;   virtual_sp_offset_adjust -16
-;   movq    %r13, %r9
-;   movq    %r8, 0(%r9)
-;   movq    0(%rsp), %r13
-;   addq    %rsp, $16, %rsp
+;   movq    %r13, %r10
+;   movq    %r8, 0(%r10)
+;   movq    16(%rsp), %r13
+;   addq    %rsp, $32, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -1435,20 +1432,18 @@ block0(v0: i128, v1: i128):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x10, %rsp
-;   movq %r13, (%rsp)
-; block1: ; offset 0xc
+;   subq $0x20, %rsp
+;   movq %r13, 0x10(%rsp)
+; block1: ; offset 0xd
 ;   movq %r8, %r13
-;   subq $0x10, %rsp
 ;   leaq (%rsp), %r8
 ;   movabsq $0, %r9 ; reloc_external Abs8 %g 0
 ;   callq *%r9
 ;   movq (%rsp), %r8
-;   addq $0x10, %rsp
-;   movq %r13, %r9
-;   movq %r8, (%r9)
-;   movq (%rsp), %r13
-;   addq $0x10, %rsp
+;   movq %r13, %r10
+;   movq %r8, (%r10)
+;   movq 0x10(%rsp), %r13
+;   addq $0x20, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
diff --git a/cranelift/filetests/filetests/isa/x64/struct-arg.clif b/cranelift/filetests/filetests/isa/x64/struct-arg.clif
index 5de74242e..00b95c41a 100644
--- a/cranelift/filetests/filetests/isa/x64/struct-arg.clif
+++ b/cranelift/filetests/filetests/isa/x64/struct-arg.clif
@@ -72,17 +72,16 @@ block0(v0: i64):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
-;   movq    %rdi, %rsi
 ;   subq    %rsp, $64, %rsp
 ;   virtual_sp_offset_adjust 64
+; block0:
+;   movq    %rdi, %rsi
 ;   lea     0(%rsp), %rdi
 ;   movl    $64, %edx
-;   load_ext_name %Memcpy+0, %r11
-;   call    *%r11
+;   load_ext_name %Memcpy+0, %r9
+;   call    *%r9
 ;   call    User(userextname0)
 ;   addq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust -64
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -91,13 +90,13 @@ block0(v0: i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
-;   movq %rdi, %rsi
 ;   subq $0x40, %rsp
+; block1: ; offset 0x8
+;   movq %rdi, %rsi
 ;   leaq (%rsp), %rdi
 ;   movl $0x40, %edx
-;   movabsq $0, %r11 ; reloc_external Abs8 %Memcpy 0
-;   callq *%r11
+;   movabsq $0, %r9 ; reloc_external Abs8 %Memcpy 0
+;   callq *%r9
 ;   callq 0x26 ; reloc_external CallPCRel4 u0:0 -4
 ;   addq $0x40, %rsp
 ;   movq %rbp, %rsp
@@ -115,22 +114,19 @@ block0(v0: i64, v1: i64):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $16, %rsp
-;   movq    %r13, 0(%rsp)
+;   subq    %rsp, $80, %rsp
+;   virtual_sp_offset_adjust 64
+;   movq    %r13, 64(%rsp)
 ; block0:
 ;   movq    %rdi, %r13
-;   subq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust 64
 ;   lea     0(%rsp), %rdi
 ;   movl    $64, %edx
-;   load_ext_name %Memcpy+0, %rax
-;   call    *%rax
+;   load_ext_name %Memcpy+0, %r10
+;   call    *%r10
 ;   movq    %r13, %rdi
 ;   call    User(userextname0)
-;   addq    %rsp, $64, %rsp
-;   virtual_sp_offset_adjust -64
-;   movq    0(%rsp), %r13
-;   addq    %rsp, $16, %rsp
+;   movq    64(%rsp), %r13
+;   addq    %rsp, $80, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -139,20 +135,18 @@ block0(v0: i64, v1: i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x10, %rsp
-;   movq %r13, (%rsp)
-; block1: ; offset 0xc
+;   subq $0x50, %rsp
+;   movq %r13, 0x40(%rsp)
+; block1: ; offset 0xd
 ;   movq %rdi, %r13
-;   subq $0x40, %rsp
 ;   leaq (%rsp), %rdi
 ;   movl $0x40, %edx
-;   movabsq $0, %rax ; reloc_external Abs8 %Memcpy 0
-;   callq *%rax
+;   movabsq $0, %r10 ; reloc_external Abs8 %Memcpy 0
+;   callq *%r10
 ;   movq %r13, %rdi
-;   callq 0x30 ; reloc_external CallPCRel4 u0:0 -4
-;   addq $0x40, %rsp
-;   movq (%rsp), %r13
-;   addq $0x10, %rsp
+;   callq 0x2e ; reloc_external CallPCRel4 u0:0 -4
+;   movq 0x40(%rsp), %r13
+;   addq $0x50, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
@@ -203,30 +197,27 @@ block0(v0: i64, v1: i64, v2: i64):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $16, %rsp
-;   movq    %r12, 0(%rsp)
-;   movq    %r14, 8(%rsp)
+;   subq    %rsp, $208, %rsp
+;   virtual_sp_offset_adjust 192
+;   movq    %r12, 192(%rsp)
+;   movq    %r14, 200(%rsp)
 ; block0:
 ;   movq    %rdi, %r12
 ;   movq    %rdx, %r14
-;   subq    %rsp, $192, %rsp
-;   virtual_sp_offset_adjust 192
 ;   lea     0(%rsp), %rdi
 ;   movl    $128, %edx
-;   load_ext_name %Memcpy+0, %rax
-;   call    *%rax
+;   load_ext_name %Memcpy+0, %r11
+;   call    *%r11
 ;   lea     128(%rsp), %rdi
 ;   movl    $64, %edx
-;   load_ext_name %Memcpy+0, %r11
+;   load_ext_name %Memcpy+0, %r9
 ;   movq    %r14, %rsi
-;   call    *%r11
+;   call    *%r9
 ;   movq    %r12, %rdi
 ;   call    User(userextname0)
-;   addq    %rsp, $192, %rsp
-;   virtual_sp_offset_adjust -192
-;   movq    0(%rsp), %r12
-;   movq    8(%rsp), %r14
-;   addq    %rsp, $16, %rsp
+;   movq    192(%rsp), %r12
+;   movq    200(%rsp), %r14
+;   addq    %rsp, $208, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -235,28 +226,26 @@ block0(v0: i64, v1: i64, v2: i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x10, %rsp
-;   movq %r12, (%rsp)
-;   movq %r14, 8(%rsp)
-; block1: ; offset 0x11
+;   subq $0xd0, %rsp
+;   movq %r12, 0xc0(%rsp)
+;   movq %r14, 0xc8(%rsp)
+; block1: ; offset 0x1b
 ;   movq %rdi, %r12
 ;   movq %rdx, %r14
-;   subq $0xc0, %rsp
 ;   leaq (%rsp), %rdi
 ;   movl $0x80, %edx
-;   movabsq $0, %rax ; reloc_external Abs8 %Memcpy 0
-;   callq *%rax
+;   movabsq $0, %r11 ; reloc_external Abs8 %Memcpy 0
+;   callq *%r11
 ;   leaq 0x80(%rsp), %rdi
 ;   movl $0x40, %edx
-;   movabsq $0, %r11 ; reloc_external Abs8 %Memcpy 0
+;   movabsq $0, %r9 ; reloc_external Abs8 %Memcpy 0
 ;   movq %r14, %rsi
-;   callq *%r11
+;   callq *%r9
 ;   movq %r12, %rdi
-;   callq 0x58 ; reloc_external CallPCRel4 u0:0 -4
-;   addq $0xc0, %rsp
-;   movq (%rsp), %r12
-;   movq 8(%rsp), %r14
-;   addq $0x10, %rsp
+;   callq 0x5c ; reloc_external CallPCRel4 u0:0 -4
+;   movq 0xc0(%rsp), %r12
+;   movq 0xc8(%rsp), %r14
+;   addq $0xd0, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
diff --git a/cranelift/filetests/filetests/isa/x64/tail-call-conv.clif b/cranelift/filetests/filetests/isa/x64/tail-call-conv.clif
index 0e055c258..e9fd2218a 100644
--- a/cranelift/filetests/filetests/isa/x64/tail-call-conv.clif
+++ b/cranelift/filetests/filetests/isa/x64/tail-call-conv.clif
@@ -69,12 +69,13 @@ block0:
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $64, %rsp
-;   movq    %rbx, 16(%rsp)
-;   movq    %r12, 24(%rsp)
-;   movq    %r13, 32(%rsp)
-;   movq    %r14, 40(%rsp)
-;   movq    %r15, 48(%rsp)
+;   subq    %rsp, $144, %rsp
+;   virtual_sp_offset_adjust 80
+;   movq    %rbx, 96(%rsp)
+;   movq    %r12, 104(%rsp)
+;   movq    %r13, 112(%rsp)
+;   movq    %r14, 120(%rsp)
+;   movq    %r15, 128(%rsp)
 ; block0:
 ;   movl    $10, %edi
 ;   movq    %rdi, rsp(0 + virtual offset)
@@ -92,8 +93,6 @@ block0:
 ;   movl    $70, %r15d
 ;   movl    $75, %ebx
 ;   movl    $80, %edi
-;   subq    %rsp, $80, %rsp
-;   virtual_sp_offset_adjust 80
 ;   movq    %r10, 0(%rsp)
 ;   movq    %r11, 8(%rsp)
 ;   movq    %rax, 16(%rsp)
@@ -103,15 +102,17 @@ block0:
 ;   movq    %r15, 48(%rsp)
 ;   movq    %rbx, 56(%rsp)
 ;   movq    %rdi, 64(%rsp)
-;   load_ext_name %tail_callee_stack_args+0, %rax
+;   load_ext_name %tail_callee_stack_args+0, %r10
 ;   movq    rsp(0 + virtual offset), %rdi
-;   call    *%rax
-;   movq    16(%rsp), %rbx
-;   movq    24(%rsp), %r12
-;   movq    32(%rsp), %r13
-;   movq    40(%rsp), %r14
-;   movq    48(%rsp), %r15
-;   addq    %rsp, $64, %rsp
+;   call    *%r10
+;   subq    %rsp, $80, %rsp
+;   virtual_sp_offset_adjust 80
+;   movq    96(%rsp), %rbx
+;   movq    104(%rsp), %r12
+;   movq    112(%rsp), %r13
+;   movq    120(%rsp), %r14
+;   movq    128(%rsp), %r15
+;   addq    %rsp, $144, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -120,15 +121,15 @@ block0:
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x40, %rsp
-;   movq %rbx, 0x10(%rsp)
-;   movq %r12, 0x18(%rsp)
-;   movq %r13, 0x20(%rsp)
-;   movq %r14, 0x28(%rsp)
-;   movq %r15, 0x30(%rsp)
-; block1: ; offset 0x21
+;   subq $0x90, %rsp
+;   movq %rbx, 0x60(%rsp)
+;   movq %r12, 0x68(%rsp)
+;   movq %r13, 0x70(%rsp)
+;   movq %r14, 0x78(%rsp)
+;   movq %r15, 0x80(%rsp)
+; block1: ; offset 0x27
 ;   movl $0xa, %edi
-;   movq %rdi, (%rsp)
+;   movq %rdi, 0x50(%rsp)
 ;   movl $0xf, %esi
 ;   movl $0x14, %edx
 ;   movl $0x19, %ecx
@@ -143,7 +144,6 @@ block0:
 ;   movl $0x46, %r15d
 ;   movl $0x4b, %ebx
 ;   movl $0x50, %edi
-;   subq $0x50, %rsp
 ;   movq %r10, (%rsp)
 ;   movq %r11, 8(%rsp)
 ;   movq %rax, 0x10(%rsp)
@@ -153,15 +153,16 @@ block0:
 ;   movq %r15, 0x30(%rsp)
 ;   movq %rbx, 0x38(%rsp)
 ;   movq %rdi, 0x40(%rsp)
-;   movabsq $0, %rax ; reloc_external Abs8 %tail_callee_stack_args 0
+;   movabsq $0, %r10 ; reloc_external Abs8 %tail_callee_stack_args 0
 ;   movq 0x50(%rsp), %rdi
-;   callq *%rax
-;   movq 0x10(%rsp), %rbx
-;   movq 0x18(%rsp), %r12
-;   movq 0x20(%rsp), %r13
-;   movq 0x28(%rsp), %r14
-;   movq 0x30(%rsp), %r15
-;   addq $0x40, %rsp
+;   callq *%r10
+;   subq $0x50, %rsp
+;   movq 0x60(%rsp), %rbx
+;   movq 0x68(%rsp), %r12
+;   movq 0x70(%rsp), %r13
+;   movq 0x78(%rsp), %r14
+;   movq 0x80(%rsp), %r15
+;   addq $0x90, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
@@ -393,30 +394,29 @@ block0:
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-; block0:
 ;   subq    %rsp, $144, %rsp
 ;   virtual_sp_offset_adjust 144
+; block0:
 ;   lea     0(%rsp), %rdi
 ;   call    TestCase(%tail_callee_stack_rets)
-;   movq    0(%rsp), %rdx
-;   movq    8(%rsp), %r9
-;   movq    16(%rsp), %r11
-;   movq    24(%rsp), %rdi
-;   movq    32(%rsp), %rcx
-;   movq    40(%rsp), %r8
-;   movq    48(%rsp), %r10
-;   movq    56(%rsp), %rsi
-;   movq    64(%rsp), %rax
-;   movq    72(%rsp), %rdx
-;   movq    80(%rsp), %r9
-;   movq    88(%rsp), %r11
-;   movq    96(%rsp), %rdi
-;   movq    104(%rsp), %rcx
-;   movq    112(%rsp), %r8
-;   movq    120(%rsp), %r10
+;   movq    0(%rsp), %rax
+;   movq    8(%rsp), %rdx
+;   movq    16(%rsp), %r9
+;   movq    24(%rsp), %r11
+;   movq    32(%rsp), %rdi
+;   movq    40(%rsp), %rcx
+;   movq    48(%rsp), %r8
+;   movq    56(%rsp), %r10
+;   movq    64(%rsp), %rsi
+;   movq    72(%rsp), %rax
+;   movq    80(%rsp), %rdx
+;   movq    88(%rsp), %r9
+;   movq    96(%rsp), %r11
+;   movq    104(%rsp), %rdi
+;   movq    112(%rsp), %rcx
+;   movq    120(%rsp), %r8
 ;   movq    128(%rsp), %rax
 ;   addq    %rsp, $144, %rsp
-;   virtual_sp_offset_adjust -144
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -425,26 +425,26 @@ block0:
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-; block1: ; offset 0x4
 ;   subq $0x90, %rsp
+; block1: ; offset 0xb
 ;   leaq (%rsp), %rdi
 ;   callq 0x14 ; reloc_external CallPCRel4 %tail_callee_stack_rets -4
-;   movq (%rsp), %rdx
-;   movq 8(%rsp), %r9
-;   movq 0x10(%rsp), %r11
-;   movq 0x18(%rsp), %rdi
-;   movq 0x20(%rsp), %rcx
-;   movq 0x28(%rsp), %r8
-;   movq 0x30(%rsp), %r10
-;   movq 0x38(%rsp), %rsi
-;   movq 0x40(%rsp), %rax
-;   movq 0x48(%rsp), %rdx
-;   movq 0x50(%rsp), %r9
-;   movq 0x58(%rsp), %r11
-;   movq 0x60(%rsp), %rdi
-;   movq 0x68(%rsp), %rcx
-;   movq 0x70(%rsp), %r8
-;   movq 0x78(%rsp), %r10
+;   movq (%rsp), %rax
+;   movq 8(%rsp), %rdx
+;   movq 0x10(%rsp), %r9
+;   movq 0x18(%rsp), %r11
+;   movq 0x20(%rsp), %rdi
+;   movq 0x28(%rsp), %rcx
+;   movq 0x30(%rsp), %r8
+;   movq 0x38(%rsp), %r10
+;   movq 0x40(%rsp), %rsi
+;   movq 0x48(%rsp), %rax
+;   movq 0x50(%rsp), %rdx
+;   movq 0x58(%rsp), %r9
+;   movq 0x60(%rsp), %r11
+;   movq 0x68(%rsp), %rdi
+;   movq 0x70(%rsp), %rcx
+;   movq 0x78(%rsp), %r8
 ;   movq 0x80(%rsp), %rax
 ;   addq $0x90, %rsp
 ;   movq %rbp, %rsp
@@ -665,12 +665,13 @@ block0:
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $160, %rsp
-;   movq    %rbx, 112(%rsp)
-;   movq    %r12, 120(%rsp)
-;   movq    %r13, 128(%rsp)
-;   movq    %r14, 136(%rsp)
-;   movq    %r15, 144(%rsp)
+;   subq    %rsp, $480, %rsp
+;   virtual_sp_offset_adjust 320
+;   movq    %rbx, 432(%rsp)
+;   movq    %r12, 440(%rsp)
+;   movq    %r13, 448(%rsp)
+;   movq    %r14, 456(%rsp)
+;   movq    %r15, 464(%rsp)
 ; block0:
 ;   movl    $10, %edi
 ;   movq    %rdi, rsp(96 + virtual offset)
@@ -711,8 +712,6 @@ block0:
 ;   movq    %r11, rsp(8 + virtual offset)
 ;   movl    $135, %r11d
 ;   movq    %r11, rsp(0 + virtual offset)
-;   subq    %rsp, $320, %rsp
-;   virtual_sp_offset_adjust 320
 ;   movq    rsp(48 + virtual offset), %r11
 ;   movq    %r11, 0(%rsp)
 ;   movq    %rax, 8(%rsp)
@@ -740,8 +739,8 @@ block0:
 ;   movq    %r11, 144(%rsp)
 ;   movq    rsp(0 + virtual offset), %r11
 ;   movq    %r11, 152(%rsp)
-;   lea     176(%rsp), %rax
-;   movq    %rax, 160(%rsp)
+;   lea     176(%rsp), %rsi
+;   movq    %rsi, 160(%rsp)
 ;   load_ext_name %tail_callee_stack_args_and_rets+0, %r10
 ;   movq    rsp(72 + virtual offset), %rcx
 ;   movq    rsp(80 + virtual offset), %rdx
@@ -750,31 +749,31 @@ block0:
 ;   movq    rsp(64 + virtual offset), %r8
 ;   movq    rsp(56 + virtual offset), %r9
 ;   call    *%r10
-;   movq    0(%rsp), %r10
-;   movq    8(%rsp), %rsi
-;   movq    16(%rsp), %rax
-;   movq    24(%rsp), %rdx
-;   movq    32(%rsp), %r9
-;   movq    40(%rsp), %r11
-;   movq    48(%rsp), %rdi
-;   movq    56(%rsp), %rcx
-;   movq    64(%rsp), %r8
-;   movq    72(%rsp), %r10
-;   movq    80(%rsp), %rsi
-;   movq    88(%rsp), %rax
-;   movq    96(%rsp), %rdx
-;   movq    104(%rsp), %r9
-;   movq    112(%rsp), %r11
-;   movq    120(%rsp), %rdi
-;   movq    128(%rsp), %rax
-;   addq    %rsp, $144, %rsp
-;   virtual_sp_offset_adjust -144
-;   movq    112(%rsp), %rbx
-;   movq    120(%rsp), %r12
-;   movq    128(%rsp), %r13
-;   movq    136(%rsp), %r14
-;   movq    144(%rsp), %r15
-;   addq    %rsp, $160, %rsp
+;   subq    %rsp, $176, %rsp
+;   virtual_sp_offset_adjust 176
+;   movq    176(%rsp), %r10
+;   movq    184(%rsp), %rsi
+;   movq    192(%rsp), %rax
+;   movq    200(%rsp), %rdx
+;   movq    208(%rsp), %r9
+;   movq    216(%rsp), %r11
+;   movq    224(%rsp), %rdi
+;   movq    232(%rsp), %rcx
+;   movq    240(%rsp), %r8
+;   movq    248(%rsp), %r10
+;   movq    256(%rsp), %rsi
+;   movq    264(%rsp), %rax
+;   movq    272(%rsp), %rdx
+;   movq    280(%rsp), %r9
+;   movq    288(%rsp), %r11
+;   movq    296(%rsp), %rdi
+;   movq    304(%rsp), %rax
+;   movq    432(%rsp), %rbx
+;   movq    440(%rsp), %r12
+;   movq    448(%rsp), %r13
+;   movq    456(%rsp), %r14
+;   movq    464(%rsp), %r15
+;   addq    %rsp, $480, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -783,27 +782,27 @@ block0:
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0xa0, %rsp
-;   movq %rbx, 0x70(%rsp)
-;   movq %r12, 0x78(%rsp)
-;   movq %r13, 0x80(%rsp)
-;   movq %r14, 0x88(%rsp)
-;   movq %r15, 0x90(%rsp)
-; block1: ; offset 0x2d
+;   subq $0x1e0, %rsp
+;   movq %rbx, 0x1b0(%rsp)
+;   movq %r12, 0x1b8(%rsp)
+;   movq %r13, 0x1c0(%rsp)
+;   movq %r14, 0x1c8(%rsp)
+;   movq %r15, 0x1d0(%rsp)
+; block1: ; offset 0x33
 ;   movl $0xa, %edi
-;   movq %rdi, 0x60(%rsp)
+;   movq %rdi, 0x1a0(%rsp)
 ;   movl $0xf, %esi
-;   movq %rsi, 0x58(%rsp)
+;   movq %rsi, 0x198(%rsp)
 ;   movl $0x14, %edx
-;   movq %rdx, 0x50(%rsp)
+;   movq %rdx, 0x190(%rsp)
 ;   movl $0x19, %ecx
-;   movq %rcx, 0x48(%rsp)
+;   movq %rcx, 0x188(%rsp)
 ;   movl $0x1e, %r8d
-;   movq %r8, 0x40(%rsp)
+;   movq %r8, 0x180(%rsp)
 ;   movl $0x23, %r9d
-;   movq %r9, 0x38(%rsp)
+;   movq %r9, 0x178(%rsp)
 ;   movl $0x28, %esi
-;   movq %rsi, 0x30(%rsp)
+;   movq %rsi, 0x170(%rsp)
 ;   movl $0x2d, %eax
 ;   movl $0x32, %r10d
 ;   movl $0x37, %r14d
@@ -818,18 +817,17 @@ block0:
 ;   movl $0x64, %r8d
 ;   movl $0x69, %r9d
 ;   movl $0x6e, %r11d
-;   movq %r11, 0x28(%rsp)
+;   movq %r11, 0x168(%rsp)
 ;   movl $0x73, %r11d
-;   movq %r11, 0x20(%rsp)
+;   movq %r11, 0x160(%rsp)
 ;   movl $0x78, %r11d
-;   movq %r11, 0x18(%rsp)
+;   movq %r11, 0x158(%rsp)
 ;   movl $0x7d, %r11d
-;   movq %r11, 0x10(%rsp)
+;   movq %r11, 0x150(%rsp)
 ;   movl $0x82, %r11d
-;   movq %r11, 8(%rsp)
+;   movq %r11, 0x148(%rsp)
 ;   movl $0x87, %r11d
-;   movq %r11, (%rsp)
-;   subq $0x140, %rsp
+;   movq %r11, 0x140(%rsp)
 ;   movq 0x170(%rsp), %r11
 ;   movq %r11, (%rsp)
 ;   movq %rax, 8(%rsp)
@@ -857,8 +855,8 @@ block0:
 ;   movq %r11, 0x90(%rsp)
 ;   movq 0x140(%rsp), %r11
 ;   movq %r11, 0x98(%rsp)
-;   leaq 0xb0(%rsp), %rax
-;   movq %rax, 0xa0(%rsp)
+;   leaq 0xb0(%rsp), %rsi
+;   movq %rsi, 0xa0(%rsp)
 ;   movabsq $0, %r10 ; reloc_external Abs8 %tail_callee_stack_args_and_rets 0
 ;   movq 0x188(%rsp), %rcx
 ;   movq 0x190(%rsp), %rdx
@@ -867,30 +865,30 @@ block0:
 ;   movq 0x180(%rsp), %r8
 ;   movq 0x178(%rsp), %r9
 ;   callq *%r10
-;   movq (%rsp), %r10
-;   movq 8(%rsp), %rsi
-;   movq 0x10(%rsp), %rax
-;   movq 0x18(%rsp), %rdx
-;   movq 0x20(%rsp), %r9
-;   movq 0x28(%rsp), %r11
-;   movq 0x30(%rsp), %rdi
-;   movq 0x38(%rsp), %rcx
-;   movq 0x40(%rsp), %r8
-;   movq 0x48(%rsp), %r10
-;   movq 0x50(%rsp), %rsi
-;   movq 0x58(%rsp), %rax
-;   movq 0x60(%rsp), %rdx
-;   movq 0x68(%rsp), %r9
-;   movq 0x70(%rsp), %r11
-;   movq 0x78(%rsp), %rdi
-;   movq 0x80(%rsp), %rax
-;   addq $0x90, %rsp
-;   movq 0x70(%rsp), %rbx
-;   movq 0x78(%rsp), %r12
-;   movq 0x80(%rsp), %r13
-;   movq 0x88(%rsp), %r14
-;   movq 0x90(%rsp), %r15
-;   addq $0xa0, %rsp
+;   subq $0xb0, %rsp
+;   movq 0xb0(%rsp), %r10
+;   movq 0xb8(%rsp), %rsi
+;   movq 0xc0(%rsp), %rax
+;   movq 0xc8(%rsp), %rdx
+;   movq 0xd0(%rsp), %r9
+;   movq 0xd8(%rsp), %r11
+;   movq 0xe0(%rsp), %rdi
+;   movq 0xe8(%rsp), %rcx
+;   movq 0xf0(%rsp), %r8
+;   movq 0xf8(%rsp), %r10
+;   movq 0x100(%rsp), %rsi
+;   movq 0x108(%rsp), %rax
+;   movq 0x110(%rsp), %rdx
+;   movq 0x118(%rsp), %r9
+;   movq 0x120(%rsp), %r11
+;   movq 0x128(%rsp), %rdi
+;   movq 0x130(%rsp), %rax
+;   movq 0x1b0(%rsp), %rbx
+;   movq 0x1b8(%rsp), %r12
+;   movq 0x1c0(%rsp), %r13
+;   movq 0x1c8(%rsp), %r14
+;   movq 0x1d0(%rsp), %r15
+;   addq $0x1e0, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
diff --git a/cranelift/filetests/filetests/isa/x64/uext-sext-handling.clif b/cranelift/filetests/filetests/isa/x64/uext-sext-handling.clif
index 0908bec08..367dff018 100644
--- a/cranelift/filetests/filetests/isa/x64/uext-sext-handling.clif
+++ b/cranelift/filetests/filetests/isa/x64/uext-sext-handling.clif
@@ -11,7 +11,7 @@ block0(v0: i8):
     return
 }
 
-; check: pushq   %rbp
+; check:  pushq   %rbp
 ; nextln: movq    %rsp, %rbp
 ; nextln: block0:
 ; nextln: movzbq  %dil, %rdi
@@ -30,9 +30,9 @@ block0(v0: i8):
 
 ; check: pushq   %rbp
 ; nextln: movq    %rsp, %rbp
-; nextln: block0:
 ; nextln: subq    %rsp, $$32, %rsp
 ; nextln: virtual_sp_offset_adjust 32
+; nextln: block0:
 ; nextln: movzbq  %cl, %rcx
-; nextln: load_ext_name userextname0+0, %r9
-; nextln: call    *%r9
+; nextln: load_ext_name userextname0+0, %rdx
+; nextln: call    *%rdx
diff --git a/cranelift/filetests/filetests/isa/x64/winch.clif b/cranelift/filetests/filetests/isa/x64/winch.clif
index cf47a84ed..808b73d57 100644
--- a/cranelift/filetests/filetests/isa/x64/winch.clif
+++ b/cranelift/filetests/filetests/isa/x64/winch.clif
@@ -292,29 +292,26 @@ block0(v0:i64):
 ; VCode:
 ;   pushq   %rbp
 ;   movq    %rsp, %rbp
-;   subq    %rsp, $48, %rsp
-;   movq    %rbx, 0(%rsp)
-;   movq    %r12, 8(%rsp)
-;   movq    %r13, 16(%rsp)
-;   movq    %r14, 24(%rsp)
-;   movq    %r15, 32(%rsp)
-; block0:
-;   subq    %rsp, $16, %rsp
+;   subq    %rsp, $64, %rsp
 ;   virtual_sp_offset_adjust 16
+;   movq    %rbx, 16(%rsp)
+;   movq    %r12, 24(%rsp)
+;   movq    %r13, 32(%rsp)
+;   movq    %r14, 40(%rsp)
+;   movq    %r15, 48(%rsp)
+; block0:
 ;   lea     0(%rsp), %rdi
 ;   load_ext_name %g+0, %r15
 ;   call    *%r15
 ;   movq    4(%rsp), %rax
-;   movq    0(%rsp), %rdi
-;   addq    %rsp, $16, %rsp
-;   virtual_sp_offset_adjust -16
-;   andl    %eax, %edi, %eax
-;   movq    0(%rsp), %rbx
-;   movq    8(%rsp), %r12
-;   movq    16(%rsp), %r13
-;   movq    24(%rsp), %r14
-;   movq    32(%rsp), %r15
-;   addq    %rsp, $48, %rsp
+;   movq    0(%rsp), %r11
+;   andl    %eax, %r11d, %eax
+;   movq    16(%rsp), %rbx
+;   movq    24(%rsp), %r12
+;   movq    32(%rsp), %r13
+;   movq    40(%rsp), %r14
+;   movq    48(%rsp), %r15
+;   addq    %rsp, $64, %rsp
 ;   movq    %rbp, %rsp
 ;   popq    %rbp
 ;   ret
@@ -323,27 +320,25 @@ block0(v0:i64):
 ; block0: ; offset 0x0
 ;   pushq %rbp
 ;   movq %rsp, %rbp
-;   subq $0x30, %rsp
-;   movq %rbx, (%rsp)
-;   movq %r12, 8(%rsp)
-;   movq %r13, 0x10(%rsp)
-;   movq %r14, 0x18(%rsp)
-;   movq %r15, 0x20(%rsp)
-; block1: ; offset 0x20
-;   subq $0x10, %rsp
+;   subq $0x40, %rsp
+;   movq %rbx, 0x10(%rsp)
+;   movq %r12, 0x18(%rsp)
+;   movq %r13, 0x20(%rsp)
+;   movq %r14, 0x28(%rsp)
+;   movq %r15, 0x30(%rsp)
+; block1: ; offset 0x21
 ;   leaq (%rsp), %rdi
 ;   movabsq $0, %r15 ; reloc_external Abs8 %g 0
 ;   callq *%r15
 ;   movq 4(%rsp), %rax
-;   movq (%rsp), %rdi
-;   addq $0x10, %rsp
-;   andl %edi, %eax
-;   movq (%rsp), %rbx
-;   movq 8(%rsp), %r12
-;   movq 0x10(%rsp), %r13
-;   movq 0x18(%rsp), %r14
-;   movq 0x20(%rsp), %r15
-;   addq $0x30, %rsp
+;   movq (%rsp), %r11
+;   andl %r11d, %eax
+;   movq 0x10(%rsp), %rbx
+;   movq 0x18(%rsp), %r12
+;   movq 0x20(%rsp), %r13
+;   movq 0x28(%rsp), %r14
+;   movq 0x30(%rsp), %r15
+;   addq $0x40, %rsp
 ;   movq %rbp, %rsp
 ;   popq %rbp
 ;   retq
diff --git a/crates/wasi-http/tests/all/main.rs b/crates/wasi-http/tests/all/main.rs
index ecbdd8d7e..00a17e825 100644
--- a/crates/wasi-http/tests/all/main.rs
+++ b/crates/wasi-http/tests/all/main.rs
@@ -278,7 +278,7 @@ async fn do_wasi_http_hash_all(override_send_request: bool) -> Result<()> {
                 let response = handle(request.into_parts().0).map(|resp| {
                     Ok(IncomingResponseInternal {
                         resp,
-                        worker: Arc::new(wasmtime_wasi::runtime::spawn(future::ready(()))),
+                        worker: wasmtime_wasi::runtime::spawn(future::ready(())),
                         between_bytes_timeout,
                     })
                 });
diff --git a/tests/disas/winch/x64/load/grow_load.wat b/tests/disas/winch/x64/load/grow_load.wat
index 6da66e24b..e8c3abea9 100644
--- a/tests/disas/winch/x64/load/grow_load.wat
+++ b/tests/disas/winch/x64/load/grow_load.wat
@@ -65,7 +65,7 @@
 ;;       movq    %r14, %rdi
 ;;       movl    0xc(%rsp), %esi
 ;;       movl    $0, %edx
-;;       callq   0x307
+;;       callq   0x2f7
 ;;       addq    $0xc, %rsp
 ;;       addq    $4, %rsp
 ;;       movq    0x48(%rsp), %r14
diff --git a/winch/docs/testing.md b/winch/docs/testing.md
deleted file mode 100644
index 859db45b7..000000000
--- a/winch/docs/testing.md
+++ /dev/null
@@ -1,63 +0,0 @@
-# Testing Winch
-
-Winch is tested through integration testing using the `winch-filetests` crate
-and manual exploratory testing. A CLI is available to run these tests
-conveniently. To add the `winch-tools` binary to your `PATH`, run `cargo install
---path winch` from the root of `wasmtime`. The CLI provides two commands: `test`
-and `compile`. To see the help text for each command, run `winch-tools test
---help` or `winch-tools compile --help`.
-
-## Integration Testing (`winch-tools test`)
-
-The `test` command will run a suite of tests that validates Winch output for a
-WebAssembly module is consistent with our expectations.
-
-### Running `test`
-
-Running `winch-tools test` will run all integration tests in the
-`winch-filetests` crate. All arguments following two dashes (`--`) will be
-passed directly to `cargo test -p winch-filetests`. This will allow you to
-configure the tests to run based on your requirements. All tests in the
-`winch-filetests` crate get named in the following convention:
-`winch_filetests_${filepath}`. This makes it possible to filter if you don't
-want to run the entire suite.
-
-If the output of Winch changes for a test in a run due to code updates, the test
-will fail and the difference between the two outputs will be shown. If the new
-output is expected, the tests can be re-run with an `WINCH_TEST_BLESS`
-environment variable set to `1`.
-
-### Adding a test
-
-To add new tests, create a `.wat` file in the `winch/filetests/filetests` folder
-in the following format:
-
-```wat
-;;! target = "x86_64"
-(module
-  (func (result i32)
-    (i32.const 42)
-  )
-)
-```
-
-It is encouraged to use folders to organize tests. For example, tests targeting
-the x86_64 architecture can be placed in the `winch/filetests/filetests/x64`.
-
-The first block of comments are a TOML compatible configuration passed to Winch
-during compilation with a `!` at the start of each line. The body of the file
-will be the subject of the test. A final block of comments is reserved for the
-output of the compilation, and it will be used to compare the output of the
-current run with the output of previous runs.
-
-## Manual Exploratory Tests (`winch-tools compile`)
-
-The `compile` command will run Winch for particular architecture against
-provided input file, and print the disassembled output to the console. Only
-`.wat` files are supported.
-
-### Running `compile`
-
-```bash
-winch-tools compile $wat_file --target $target_triple
-```