Catch errors thrown watching config file

Closes #3617