Fix flicker between swap and unswap