More tightly integrate the pipelining buffer to reduce allocations