New layout for qt4 subplottool + QMainWindow -> QDialog

This supersedes the subplottool part from #2112.