Allow disabling notifications

E.g. while the application is in the foreground