Unswapping IVM register accesses.