Remove from source inventory, not the holder of the destination.