Cache a single interface to gnttab

Unmapping pages with a different interface to the one used to map them
in the first place doesn't work.