Close dialogs along with activities - fixes #2503

Replaces #408 

Uses `hide` signal now @godiard