macosx: implement libvlc exit handler to properly terminate the app when requested

This is the only supported way to terminate an cocoa application and also ensure
that the gui settings are getting saved. Adapted from similar code in the qt
module.
macosx: fix and cleanup quit-after-playback main menu item

close #6073