Get folder path relative from the plugin folder.

This should also remove the need to relativize the file from the plugin
folder again but I'm not completely sure, need to test it first.