use 'inlinepopup' plugin in tinymce to open modals instead of popups for links/images insertion