Fix load-after-upsample.

Also eliminate some unnecessary allocations.