Add automation tests for UI dialogs

Issue #504