Add support for different config files in the config editor

This will improve our config editor to edit the currently used config file.

This problem was pointed out in #387 by @b2z :wink: