Run tests against a configurable string json file