Change metric checker to accept a ConfigFile representation, not just an...

... actual file. @syamp please review