[gui] Fix triple-modal issue on Mac for point displacement dialog

- Ensure initial dialog is a 'sheet' or gui will freeze after closing next opened modal (new color) dialog