Fix automatic severity setting, add tests for that