Remember on demand dialog settings across reboots

Fixes #1812