Move wgetting the vagrant.pub file outside the chroot

Signed-off-by: Jon Ludlam <jonathan.ludlam@citrix.com>