After exiting the top level bounding box, restore the current margins, not an old copy of them

This pull request aims to fix #645.