QtDialog: Fix Qt 5 build on non-Windows.

The Qt5Core_QTMAIN_LIBRARIES variable is defined to the Qt5::WinMain
target on Windows, and undefined elsewhere.
Disallow INTERFACE libraries with add_custom_command(TARGET).

Don't attempt to trace their dependencies.
Merge branch 'fix-Qt5-non-Windows' into release
Merge branch 'add_custom_command-no-INTERFACE-lib' into release